Put Metrics Into the User’s Context to Make Them Meaningful
by September 19, 2017

Filed under: Networking Technology, Performance Monitoring

Metrics are meaningless. Well, that’s not quite true. Raw metrics, when it comes to IT monitoring, are just numbers that don’t tell you anything. A number may seem high, but not result in any problem visible to end users. Another number may seem low, but not be as low as SLAs require. The only way to make metrics meaningful is to place them in the context of how the application is used and what the users expect.

This is especially true when dealing with cloud applications. When you apply network performance monitoring tools to cloud apps, you aren’t exactly flying blind, but the environment you’re working in is pretty nearly the opposite of transparent. You know little about the innards of the applications; you may not know anything about the server they’re running on; and connecting to the app requires using the public internet, which somehow functions despite being controlled by exactly no one.

All those factors mean you can’t look at performance statistics from a purely technical point of view. The only way to make sense of cloud app performance is to look at the performance metrics from a user perspective. That’s true even when you do have visibility into the network and applications. The goal of monitoring shouldn’t be to boost metrics for the sake of metrics; it’s to boost metrics for the sake of the business.)

Metrics Aren’t Defined by Monitoring Tools

You won’t monitor effectively if you don’t choose metrics effectively. The metrics you choose to monitor shouldn’t be dictated by the capability of the tools you have available; they should be dictated by what matters to users and to the business. Know what metrics you want to collect and how you’ll use them before you turn on your application and network performance monitoring software.

In fact, the first metric to concern yourself with isn’t a user metric at all; it’s a business metric: How much does the business care about that application? If management says it’s a low priority and performance doesn’t matter, you should spend very little time and effort monitoring it and trying to improve it, even if users complain.

After you determine which applications matter to your business, then you need to find out what the user cares about so you can conduct network performance monitoring appropriately. Identify the key pages and functions where performance really counts. These aren’t just the most frequently used pages, but also pages that support infrequent but critical business process, even if they are used only end of month, end of quarter or end of year.

Understand How Users View the Application

To get performance metrics that accurately reflect the performance that users experience, use network performance monitoring software that lets you execute synthetic transactions.

By logging in as if you’re a user and exercising the same functionality the users do, you’ll get accurate measurements. To be truly accurate, you need to run the synthetic transactions from the same places and over the same network the users do: from the main office, from branch offices, over wireless networks and VPNs. (You can do this with a lightweight tool and transaction load that won’t overload any live business processing.)

Although users see applications as separate, they’re all using the same network. Understanding the user’s context also requires understanding what other applications are running at the same time, in order to identify what’s competing for resources.

Users perceive application and network performance continuously, but only complain at specific moments in time. Blips in performance are far less important than trends, so continuous application and network performance monitoring that provides a baseline of standard performance lets you evaluate the impact and scope of any degradation to know when it’s necessary to escalate a response. If you have a good baseline and monitor continuously, you can respond to problems before the users are irritated enough to complain.

Metrics matter for end-user experience

Correlate the Metrics You Collect to the User’s Experience

Once you have metrics, root cause analysis will let you identify where the problems are coming from. Then you can tackle fixing them, whether they require work on your network or indicate a problem that the vendor needs to resolve. While the only metrics the business care about is how long it will take to get performance back to an acceptable level and how much it will cost, you’ll likely still need to report on the findings of your analysis.

You can still capture standard network metrics like latency and jitter, but you’ll need to correlate those to the metrics the user experiences firsthand, which are how fast pages load and how fast transactions complete. Let users know how increased latency, jitter and packet loss result in any poor performance they’re experiencing, and they’ll support your efforts to improve those aspects of the network. Otherwise, to them, those are just numbers.

Download AppNeta’s whitepaper on performance monitoring metrics you should review to better optimize and improve the end-user experience.