In my blog last week (Just Another Manic Cyber Monday), we started discussing why you need to start preparing now for the biggest e-commerce day of the year: Cyber Monday. Now, let’s break down the how.
So, what is an appropriate strategy to avoid a performance disaster this holiday season? Certainly, a strategy needs to be specific to your individual needs, risk appetite, budget, staffing and schedule. However, the following checklist includes what the best strategies need to include, or should at least actively consider:
- Get performance into your requirements and/or acceptance criteria (qualitative goals are better than fabricated numbers)
- Test components, stories, etc. as they are developed
- Monitor and trend key performance indicators throughout development
- Test single- and multi-user scenarios on the front end and on the back end, separately first, then at the same time. LoadUIWeb offers Web application “front end” load testing. (If you also want to test the “back end,” LoadUI will specifically test your own and any third party APIs—for more this subject, check out our eBook: Test Strategist’s Guide to Load Testing API Web Apps)
- Test in production-like environments under production-like conditions, both anticipated conditions and less-likely but potentially disastrous ones
- Practice monitoring in production-like environments, under production-like conditions – AlertSite InSite offers behind-the-firewall monitoring to provide visibility into the performance of internal applications inaccessible via the public Internet – like those running within internal dev environments
- Test multiple variations of the expected usage model – and tune for consistent performance across variants
- Employ intelligent decision making related to client-side browsers & devices (i.e. if mobile displays site version x, if flash is disabled display site version y)
- When acceptable-to-good performance is achieved under anticipated peak conditions, strive for consistent and reliable performance from single user through stress conditions (far in excess of expected peak)
- Build in performance safety features (i.e. switch to streamlined version of the site when key performance indicators exceed x, configure automatic redirects to an “emergency overflow site” when key performance indicators exceed y)
- Test safety features
- Identify “danger values” for key performance indicators & configure monitoring alerts in production as an early warning system – AlertSite Performance Threshold Notifications enable you to define acceptable performance levels and receive real-time alerts when a slowdown or outage occurs
- While generating excessive load, practice tuning dynamically under production-like conditions
Sounds like a lot of work, and it is. Luckily, it all boils down to 5 very simple principles:
- Get everyone to do their part by ensuring they spend at least a few minutes every day testing, monitoring and/or tuning performance
- Ensure that everyone has the support tools and environments they need to do their part
- Have everyone give an update on performance at every status meeting
- Don’t consider any story or iteration to be “done” until performance is tested, characterized, and deemed production worthy
- Never assume — Test and monitor
Why You Need to Test and Monitor
Many groups argue that either a good testing OR a good monitoring strategy is adequate. It is certainly true that doing one of these things well is far better than doing neither – but the reality is that it’s virtually impossible to test for everything that could happen in production. Conversely, no matter how well you monitor your site, fixing an issue discovered in production has a far greater impact on your users than if it never made it there in the first place. When testing and monitoring are part of a single strategy, your test results improve your monitoring efficiency and, vice versa, your monitoring results improve your test accuracy.
The bottom line is that if you don’t have a Cyber Monday strategy in place by the first week in September, you need to start building one. If you do have one, it’s a great time to review it to ensure it’s still adequate. If you have either a performance testing strategy or a performance monitoring strategy, but not both, champion implementing one you’re missing. If you’re involved with either performance testing or monitoring respectively, find a way to get involved in both.
LoadUIWeb Pro and AlertSite by SmartBear offer comprehensive solutions for your respective testing and monitoring needs so you can ensure your site is ready for Cyber Monday and holiday seasons to come.
- The Cost of Poor Web Performance
- Any Given Monday – Google, Microsoft and Amazon All Experience Outages
- Web Performance and the Perception of Speed