Avoiding the Grinch this Holiday Season: Benchmarking
While some things about website speed are absolutes - for instance, if it’s taking 10 seconds before your page appears to a user, you’re toast – understanding relative performance is critical. Benchmarking is the process of evaluating a website’s operational attributes by comparison to the competition or the landscape.
A best practice for benchmarking is to pay attention to four major fields: technical performance, user experience, consistency, and competitive benchmarking.
When I say technical performance, I’m talking about the traditional measures of web performance that talk to web page load time. This is the time it takes to download each and every asset of web page. Here’s a sample of what that report looks like for Google’s website:
It’s no longer sufficient to view website performance through the technical lens because there are many optimization techniques that allow us to spend 4 or more seconds downloading all the page resources and still present an excellent user experience in just over a second. While this particular sample took 4.2 seconds to load the entire page from a user experience perspective, the page was visible in just over a second - 1.17 to be exact. And the page was fully rendered in the visible portion of the screen by 2.4s:
First Paint:
Above the Fold:
Understanding and managing Web performance, both perceived and technical, is incredibly important. The averages, however, can sometimes hide important information making it very important to also look at consistency. Here are two reporting items that can really help understand variability.
Min, Max, Avg, and Standard Deviation data from the detail report:
And here’s a look at how performance varies by hour of day for last week:
Ok, so Amazon.com doesn’t have a lot of variability. It’s Amazon.
Competitive benchmarking is last on the list, but certainly not least. I worked closely this year with a customer to complete a really great benchmarking analysis of their website performance characteristics for Home Page, Product Search, Product Detail Page, Add to Cart, and Checkout vs. key competitors.
This is what that analysis looked like:
And, of course, we offer business benchmarks for quite a few industries websites. But benchmarking is just a small part of preparing yourself for the looming holiday rush. To really ensure that you keep the Grinch away this Friday, check back for next week's installment on load testing.
See also:
