14 Metrics to Optimize End-User Experience
As any IT expert will tell you, ensuring application performance in SaaS, cloud and WAN environments brings significant challenges—challenges that seem to increase on a daily basis as the technical landscape becomes ever more complex. It’s not surprising when you consider the following trends:
- As companies switch to SaaS and cloud-based versions of Salesforce, Office and other critical business applications, IT is forced to cede control of key application delivery elements to their vendors and the public internet
- Network topologies (both public and private) have become more diverse forcing application delivery systems to navigate an increasing number of paths, protocols and configurations
- BYOD and mobility requirements mean IT can no longer specify how applications are accessed and optimized at the user level
- Applications increasingly rely on external data storage systems and third-party integrations to provide a complete solution
- While demand for bandwidth has skyrocketed, most organizations do not yet have the ability to accurately prioritize mission-critical applications within their dynamic allocation algorithms
- Companies are rapidly expanding around the globe which creates fluctuations in network and application performance from region to region
While these issues complicate an IT group’s ability to ensure application performance, it has not stopped management from demanding more. After all, sluggish application response has negative and highly visible implications. While most companies focus on the cost of application downtime (ranging from $100,000–$500,000 per incident), even slow performance in employee-facing applications will result in loss of productivity, higher costs, user frustration and turnover. An application that fails to perform as expected even 2% of the time will generate a 45 working-hour deficit in a year.
On the customer side, poorly performing websites and applications directly cause dissatisfaction, churn and revenue loss. In fact, a one-second user delay produces a 7% loss in conversions, 11% fewer page views and a 16% decrease in customer satisfaction. At the same time, companies without the ability to monitor application performance of specific SaaS and cloud-based applications risk overpaying against SLAs.
In fact, a one-second user delay produces a 7% loss in conversions, 11% fewer page views and a 16% decrease in customer satisfaction.
Bottom line, to avoid these problems companies must be able able to quantify—and optimize—the most critical measurement of all: end-user experience. End-user experience can be measured by how long it takes an individual user to locally perform a specific task within a designated application. For example, it reports the time is takes a partner in Texas to update a record in Salesforce, an employee in the UK office to access an email message or a customer in Germany download account details from a company’s AWS application. In the end, measuring the end-user experience provides a clear, aggregated understanding of how well each and every network, infrastructure, device and application element are working together to meet end user expectations of how the service should perform. Focusing on end-user experience will ensure that IT groups can easily identify, diagnose and preempt the technical issues that can cause the most damage to any organization.
Characterizing the Problem: Quantifying End-User Experience
End-user experience is essentially a quantification of the entire time it takes a user to perform a specific function, regardless of how many network hops, data calls, applets and other application elements are utilized in the process. For most organizations, this presents a significant challenge as IT teams tend to evaluate latencies in a disparate, reactive way. While they may see outage alerts for the different parts of the network they control, IT often has to wait for users to complain before they realize that there are delays and performance issues at the application level. There are a variety of reasons for this:
- Tools and metrics have traditionally been siloed and tied to individual IT assets
- Networking and application teams are not fully integrated
- APM and NPM data is rarely integrated in a single toolset
- SaaS and cloud-based application vendors typically provide performance data from their perspective and don’t offer a consumer view
Clearly, IT needs a more sophisticated solution to obtain the end-user experience data they need, by application, geography and function. While end-user experience should be viewed as an aggregated and singular value (i.e., it took an employee 20 milliseconds to update a record in Salesforce or a customer had to wait 5 seconds for their account details to be displayed), there are a variety of factors that impact the end result. Because the end-user experience is impacted by many factors, IT must be able to track all these metrics in order to optimize performance and the value of its application investments.
| Network Data | |
|---|---|
| Link Latency and Round Trip Time | Shows which path in the network (public or private)is causing performance delays |
| Jitter | Explains inconsistent user experience measurements across multiple applications |
| Throughput | Exposes network related issues |
| Available Capacity | Identifies problems stemming from improper resource allocation |
| Bandwidth Consumed | Provides a better understanding of the network resources available to the application in question |
| Application Data | |
| Response Time | Clearly identifies problems in SaaS, AWS, cloud and on premise applications |
| QoS | Provides intelligence on delivered vs expected performance |
| Capacity Used | Ensures administrators understand when they need additional resources |
| Throughput | Exposes application-level problems |
| Defects per Interval | Identifies problems stemming from missing or partial responses, content errors, missing components and more |
| Invocation Data | |
| Invocation Time | Offers insight into how long it takes perform different transactions (JSP, EJB, XML, servlets, APIs, etc.) |
| Infrastructure Health | |
| Database Response Times | Reveals latency issues for certain functions |
| CPU Utilization, Memory Pool and Health Metrics | Determines if individual infrastructure elements are degrading and impacting the end user experience |
| Error Rates | Shows which elements are degrading and contributing to End User Experience issues |
| Required Filters | |
| Geographic | Quickly reveals the impact of local network and application delivery problems |
| Office/Department | Allows IT to rapidly diagnose issues |
| Number of Users Impacted | Ensures that IT can prioritize issue resolution work and optimize investments over time |
The Factors Included in the End-User Experience Metric
It’s important to select a solution that not only quantifies end-user experience at a macro level, but also allows IT to quickly visualize and drill into the complex set of factors that impact performance in SaaS, cloud, WAN and LAN environments. Without this level of intelligence, companies can’t quickly uncover the root cause or continuously optimize performance for their business-critical applications.
Optimizing Application Performance
When the end-user experience metric is presented into a single holistic view, companies can easily determine how to optimize performance for their high-impact business solutions. While a single pane of glass approach to APM is critical in WAN environments, it still takes educated IT professionals to interpret the spikes and drill into the mix of contributing factors.
Say a particular SaaS or AWS application is performing poorly and showing a six-second response for end-user login time. Anyone can diagnose an issue when they see a complete slowdown in the “San Francisco to Boston” network hop. However, only those experts that know that the database call for this transaction should take one second and is now showing four seconds will be able to quickly pinpoint the source of the problem. Conversely, an engineer who saw a six-second response time for a status report function in a tech support application would know that although the metric itself seems high, the end user knows that crunching all that data takes time and still considers the application to be performing well against his or her expectations.
By aggregating all the data into a single view, IT experts can quickly see whether or not the root cause is tied to a specific application or could be affecting multiple systems or offices across the organization. Without end-user experience intelligence, companies won’t have the insights they need to pinpoint investment, eliminate weak providers and optimize application performance to ensure that Key Business Objectives (KBOs) are continuously being met.
The Bottom Line
Using performance monitoring to quantify the end-user experience metric will streamline much of the work being done in IT groups today. First, it allows different IT groups and stakeholders to collaborate over shared data as opposed to fighting over who is to blame. This is a key factor in reducing root cause analysis in complex environments from days to minutes. Generating true end-user experience intelligence is essential for prioritizing work and investment according to impact, whether that is quantified as number of users impacted, cost of issue, application criticality or other key business objective. Additionally, having access to internally generated performance data is important for monitoring vendor SLAs and ensuring that companies are not overpaying for poorly performing services.
In the end, IT is being asked to expertly support SaaS and cloud applications in environments that they can no longer specify or control. Only those companies that can internally measure end-user experience will be able to maximize their technology investments as outsourcing continues and application delivery methods become increasingly complex.