Measuring User Satisfaction
by Dan Kuebrich June 16, 2014
When monitoring an application’s performance, start at the highest level: how is the app performing for users? It’s important to measure this using both technical metrics (eg. latency, page weight) and also experiential metrics (Apdex, speed index).
We’ve recently introduced Apdex-based performance scoring to TraceView. Now, we’re taking that a step further, with Apdex for synthetic transactions in AppView and a single, unified view of Apdex scoring called the User Satisfaction Dashboard.
Apdex is a simple formula which scores the experiences of end users by the fraction of requests they experience which meet, exceed, or fail to meet expected performance standards. Given the diversity of measurement points–app server, RUM-based end-user, and synthetic end-user–where should you be measuring user experience? The short answer is everywhere you can. And with the user satisfaction dashboard, you’ll have access to all three checkpoints’ scores in the same place. Here’s how to think about them:
RUM End-user Apdex
This is the most important Apdex score for most applications–how are users experiencing full pageloads in their browsers? It’s the bottom line of web performance for most applications.
App Server Apdex
Measuring the HTTP request latency at the webserver or load balancer level misses the work done by the browser. However, in some cases, it’s a powerful diagnostic. If you are measuring the performance of your API, this is where to look. And if you’re trying to figure out if the backend is meeting its performance goals for the work it does in supporting full end-user pageloads (via TTFB and AJAX), this is where to look.
Synthetic End-user Apdex
Synthetic users may not respond in the same manner to satisfying or dissatisfying web performance as real users–they’re mechanically loyal and will never bail on a slow pageload for a competitor. But Apdex can still measure the experience that a real user would have with the same pageloads–the variance over time, and the overall fraction of pageloads meeting your expectations. Synthetic monitoring helps to understand changes in baseline performance, and now this includes your Apdex score.