Synthetic Monitoring vs RUM

This post was originally published on the Rigor Web Performance blog.

Recently one of our customers, Gilt Groupe, posted a case study on their tech blog evaluating the value and merits of utilizing synthetic monitoring (Rigor) to understand client side performance trends and problems real-time.

At Rigor, we often find ourselves educating potential customers and business users on the differences between some of the different web performance monitoring methodologies available in the market and their various use cases. In particular, we have seen an uptick in interest for Real User Monitoring (RUM).

For those that are unfamiliar with either performance monitoring methodology, here is a brief definition of how each technique works:

  • Synthetic Monitoring – vendors provide remote (often global) infrastructure that visits a website periodically and records the performance data for each run. The measured traffic is not of your actual users, it it traffic synthetically generated to collect data on page performance.
  • Real User Monitoring (RUM) – vendors provide an agent (javascript) that is injected on each page and reports on the page load data for every request that is made for each page. As the name suggests, this technique monitors an application’s actual user interactions.

Synthetic vs RUM

So which technique is more useful? The reality is that these two technologies are incredibly complimentary. In Gilt’s case study, Eric Shepherd (Gilt’s principle frontend engineer) does an excellent job defining the benefits of using both technologies:

Both RUM and synthetic monitoring give different views of our performance, and are useful for different things. RUM helps us understand long-term trends, and synthetic monitoring helps us diagnose and solve shorter-term performance problems.

Eric gives us a great start when detailing the benefits of each solution. Below I dive into specific benefits synthetic monitoring tools provide that RUM does not.

Top Benefits of Using Synthetic Monitoring:

  1. Monitor in a Controlled Environment
  2. Synthetic monitoring allows for users to monitor the performance of their websites or applications with a set of controlled variables (geography, network, device, browser, cached vs. uncached) over time. This is valuable because it allows users to block out much of the noise that is reported with RUM. As a result, users can identify latency and downtime promptly and scientifically isolate and diagnose the root cause of performance issues.
  1. Understand the Performance of 3rd Parties
  2. Unlike RUM, synthetic monitoring tools provide waterfall charts for every website visit generated by the vendor. These charts provide full page asset load times allowing users to attribute every millisecond of load time to a piece of web content. For example, users can understand the impact of switching ad providers, content delivery networks, or using a new marketing analytics plugin.
  1. Benchmarking
  2. Synthetic monitoring doesn’t require any installation or code injection on your website to start. Subsequently, users can leverage synthetic tools to monitor the competition and effectively benchmark performance against key competitors over time.

Staples Benchmark

  1. Test at Every Stage of Development
  2. Synthetic monitoring can be used to test websites and web applications in pre-production. Pre-production test results can be used to baseline performance and set alert thresholds when applications are live.
  1. 24/7 Monitoring
  2. If an issue arises during off-hours or other low-traffic periods, synthetic monitoring provides the insight you need to quickly identify, isolate, and resolve problems before they affect users and negatively impact revenue and brand equity.
  1. Baseline and Analyze Performance Trends Across Geographies
  2. With synthetic monitoring, baseline tests can be set up to mirror the way your end users access your applications. These baseline tests can monitor key transactions and a geographic locations while testing from multiple browsers and devices.

After this quick overview you can see how synthetic provides value in many ways that RUM cannot, but synthetic monitoring alone may fall short in some areas. In my next post I look at the benefits of RUM and how its strengths effectively complement the holes found when utilizing a “synthetic-only” monitoring approach.

 

 

Published by

Chapman Lever

The Most Interesting Man in the World

Leave a Reply

Your email address will not be published. Required fields are marked *