Synthetic Monitoring, Using Scripting That's Built for Humans
Scripting a web browser to imitate a human (otherwise known as automated synthetic web transactions) is awesome. Seriously—it’s an amazingly flexible and useful tool. Web developers can script against an application for testing purposes. For those figuring out how to scale a web application, you can use scripts to bring massive amounts of load to an app and see where it breaks. For AppNeta, synthetic web transactions are one of many ways that we test the performance of modern web-based applications. This is what we call Application Experience Monitoring in the AppNeta Performance Manager, since we’re monitoring the actual experience an individual user might have with a given app.
But there’s an important difference between a script and a human: the automated synthetics run at machine speed—as quickly as possible—while modern web applications have asynchronous loads designed for human interaction. This means that the web developer uses techniques which load the page as fast as possible, knowing that they can load additional resources after a quarter second and users will have a better overall experience with the app. However, you’re not going to get an accurate view of end-user experience if you’re running at machine speed. And that could be misleading in a number of ways, mainly by showing errors or performance issues that wouldn’t happen when a human uses the software.
The good news is that modern scripting languages offer ways to deal with this problem. When AppNeta recently upgraded our synthetics engines, we chose Selenium. Choosing Selenium was easy. We knew it would let our customers use their existing Selenium scripts and take advantage of the great online community. As we’ve adopted Selenium and tested it in actual production environments with our customers, we’ve come up with some tweaks to make it work best for today’s IT environments.
How to Make Selenium Work in a Modern Way
Web development and synthetics have typically worked in a certain way that pre-dates the kinds of web apps that enterprises use today. Because automated synthetics monitoring tools run at machine speed, a web developer building an application will use techniques to make sure the page loads as fast as possible up front. Additional resources can be loaded after a quarter second, and this makes sure that users have a better overall application experience.
But that planned timing of resources is problematic for a web monitoring platform that’s running flat-out. When it checks for content in that secondary load right away, it’s not there yet. To a monitoring tool, it will appear that the app is broken. Essentially, you’re scripting against something that doesn’t exist.
The good news is that Selenium has a way to handle this, by building delays into the execution of the script. We figured this out when we started scripting against some common apps (including our own). We made everything happen instantaneously. The problem with this is obvious when you think about it: if you’re trying to simulate end-user experience with an application, you need to behave like a human.
The behavior of modern web and SaaS applications has evolved as web technologies have changed to support more complex applications. In particular, we’ve seen the rise of asynchronous web applications, where different parts of the page come up at different times. While this improves the end-user experience (locking the page until it is fully rendered is a bad idea), it means that you have to take this into account when designing Selenium scripts. In particular, you need to add delays to make sure the component is actually loaded before starting to script against it.
By adding delays, you’ll get a more realistic view of the web application—and user-experience metrics such as Apdex will also be more realistic.
The Icing on the Synthetic Monitoring Cake
When we were working on developing our Selenium use, we used the comment syntax to create milestones. These let users create break points in the script for monitoring. This comes in handy for single-page applications like Office 365 or G Suite. IT teams can accurately time users’ experience of these apps, however data is loaded.
We also took advantage of the Assert and Verify possibilities of Selenium. Using the Assert command is like pulling the parachute on a script. If an element doesn’t load, then it will abort. It is an early warning system for features or data that your users have to have. The Verify command does the same thing as Assert, but allows the script to complete. Plus, you can see automatically generated screenshots when the script runs into an error to see what the user saw when the page failed.
We’re pleased with the end result of our Selenium use, which is synthetics monitoring that’s tailored to the way web applications are used today. It’s able to truly suss out how end users are experiencing an application—leading IT to a new level of visibility.