Not All End User Experience Monitoring Solutions Are Created Equal
Four APM Approaches That May Not Meet All of Your End User Experience Monitoring Needs
December 10, 2014
Lee Guthrie
Share this

End user experience monitoring goes hand in hand with Application Performance Management (APM). In its Magic Quadrant for Application Performance Management, Gartner considers End User Experience Monitoring to be the first of the five functional dimensions necessary for enterprises to achieve the objectives of APM. After all, analyzing the performance of an application requires data on how application parameters like availability, latency, execution correctness, and quality appear to the end user.

As a result, it seems that every leading APM vendor includes End User Experience Monitoring as part of its solution. Yet, as enterprise IT Ops teams implement these solutions, they can still find themselves blind to what their workforce users are actually experiencing as they use the broad range of business critical applications in the enterprise portfolio. Explaining this conflict requires a deeper look at what other vendors classify as end user experience monitoring. Four different approaches commonly found in the market solve some of the problem some of the time, but each has important limitations to be aware of.

1. Synthetic Monitoring

Synthetic monitoring is an approach in which a script is executed that emulates users interacting with applications. The scripts are run at periodic intervals to proactively identify availability or major execution issues before end user experience is impacted. This approach is useful in determining application availability and establishing general performance baselines, especially for applications that are not accessed by real users 24/7. It is also valuable for applications that rely on APIs to access third party services.

However, this solution knows nothing of an organization’s end users and their actual experience. If you were to call your helpdesk with a problem, this solution would tell them nothing about what you were actually doing or experiencing. In addition, it can be time-consuming to create and maintain the scripts on which this approach relies.

2. Network-Based Packet Capture

These solutions gather network-based response time and error metrics from the network, browser, or application that impact end user experience, such as HTTP/HTTPS or a variety of other network transactions on the wire, such as TCP. This approach requires close coordination with network engineers to identify the optimum points in the network to place the specialized networking equipment required to aggregate and filter traffic for analysis. Although packet aggregation and brokering gear can also be used for security and network management, it becomes more expensive as network speeds increase, driving exponential increases in cost for this approach.

Network-based packet capture solutions do not provide visibility into the actual screen render time within the browser or application, resulting in cases of dubious end user experience reports. Web and network requests/responses can go over the wire sub-second, but if there is heavy client-side processing or a lot of data, the screen within the browser or application can take 10 seconds or more to actually render. Apps delivered via cloud or virtualization also present challenges to this approach.

Furthermore, this approach cannot monitor the wide variety of enterprise applications that are not web-based. In addition, these solutions cannot monitor application activities that don’t generate network traffic, such as opening a cached email in MS Outlook. Finally, these solutions provide no data about what is happening on the user’s device, which often correlates to user experience.

3. JavaScript Injection

This solution injects JavaScript code into a web application in order to get timings of what is occurring within a user’s browser. This approach can include calls to 3rd party apps, so it gets closer to the actual user’s experience, but works only for web applications and hybrid mobile apps.

However, it cannot provide any correlating metrics about what’s happening on the user’s device. Monitoring applications that are hosted outside of the control of the enterprise, such as SaaS based applications, requires JavaScript injection using proxy servers or load balancers, which adds complexity, expense, and implementation challenges for fragmented network topologies.

4. Physical and Virtual Device Monitoring

In today’s converged, next generation end user computing environment, the workforce navigates seamlessly between physical, virtual, and mobile devices as they conduct business using a broad portfolio of apps. Device monitoring solutions monitor operating system metrics like resource utilization (CPU, memory, etc.) and health (application crashes, blue screens, etc.), which are certainly important factors that can impact user experience. Device monitoring has also become popular in the VDI space for physical to virtual assessments and for monitoring the virtual desktop infrastructure.

However, device monitoring does not provide any insight into how end users are actually experiencing the applications they use. When users call to complain that the application in their VDI environment is running slowly, they are not complaining about device metrics like CPU or memory. They are complaining about their ability to execute a business activity, like look up patient, process an insurance claim, or search a customer record.

Closing the APM Visibility Gap

Closing the visibility gap between your workforce’s real user experience and what application-centric APM tools provide requires a different approach. The problem is that you can’t accurately measure end user experience by starting at the data center and looking out. You have to start from the perspective of the end user.

One such approach is to focus from the perspective of the end user across all the applications and devices they use. Monitoring via an agent on the end user’s device, or via an SDK compiled into the code of a native mobile app, enables IT Ops to capture the user experience of the workforce as they use any type of app – mobile, cloud, SaaS – delivered on any type of device. As enterprises adopt the constant stream of new business-critical applications delivered via cloud or on premise, running on physical, virtual, and mobile devices, IT must keep pace, to ensure quality of service and workforce productivity.

Real World Assessment of User Experience Monitoring Approaches

The following table illustrates a side-by-side comparison of the approaches to end user experience monitoring for seven common real world scenarios encountered by the enterprise workforce. The table illustrates the benefits of instrumenting physical or virtual devices with agents, or mobile apps with an SDK. This approach uniquely provides a complete view of end user experience and device metrics correlated to that experience, for the full range of applications and devices that the enterprise workforce relies on in today’s converged cloud, mobile, and virtual environments.

Click on chart for a larger version.

ABOUT Lee Guthrie

Lee Guthrie serves as Director of Worldwide Sales Engineering at Aternity, whose award-winning Workforce Application Performance Management (APM) solution monitors any app on any physical, virtual or mobile device providing a unique, user-centric vantage point that closes the visibility gap existing with network- and server-centric application performance management tools. Guthrie works alongside Fortune 500 prospects and customers, assisting them in architecting and implementing Aternity within their diverse, complex, and rapidly evolving IT environments to deliver optimal end user experience and increased business productivity. Prior to joining Aternity, Guthrie was a Principal Solution Architect with HP (formerly Mercury Interactive). He received a Bachelor of Science in Physics from WPI and his MBA from Clark University.

Share this

The Latest

May 23, 2019

The first word in APM technology is "Application" ... yet for mobile, apps are entirely different. As the mobile app ecosystem is evolving and expanding from pure entertainment to more utilitarian uses, there's a rising need for the next generation of APM technology to stay ahead of the issues that can cause apps to fail ...

May 22, 2019

For application performance monitoring (APM), many in IT tend to focus a significant amount of their time on the tool that performs the analysis. Unfortunately for them, the battle is won or lost at the data access level. If you don’t have the right data, you can’t fix the problem correctly ...

May 21, 2019

Findings of the Digital Employee Experience survey from VMware show correlation between enabling employees with a positive digital experience (i.e., device choice/flexibility, seamless access to apps, remote work capabilities) and an organization's competitive position, revenue growth and employee sentiment ...

May 20, 2019

In today's competitive landscape, businesses must have the ability and process in place to face new challenges and find ways to successfully tackle them in a proactive manner. For years, this has been placed on the shoulders of DevOps teams within IT departments. But, as automation takes over manual intervention to increase speed and efficiency, these teams are facing what we know as IT digitization. How has this changed the way companies function over the years, and what do we have to look forward to in the coming years? ...

May 16, 2019

Although the vast majority of IT organizations have implemented a broad variety of systems and tools to modernize, simplify and streamline data center operations, many are still burdened by inefficiencies, security risks and performance gaps in their IT infrastructure as well as the excessive time it takes to manage legacy infrastructure, according to the State of IT Transformation, a report from Datrium ...