You know the feeling. You’re surfing along and then BAM! you get the dreaded
503 (Service unavailable)
The server is currently unavailable (because it is overloaded or
down for maintenance).
Do you remember what you were searching for the last time that happened? Or what company you would have bought from? Probably not.
Worse, if you had just clicked on “Submit Order” and waited and waited… and then got such an error… Did you have any idea whether the order went through, or if your credit card was charged??
Chances are that merchant lost you as a customer forever. You Google’d for someone else and never looked back.
Of course, as a developer you want your service to be heavily used. Otherwise, what’s the point? But loads depend on many things: time of day, end of month processing, overnight batch runs, press releases, product launches – you name it.
So how do you make sure your API can handle even the heaviest load? How do you keep your customers from walking away?
Don’t Lose Authority From An Unreliable API
That’s just one example – from a consumer point of view – of an API service that couldn’t handle the load. It resulted in a lost customer.
When you publish your web service, you want to be the one that everyone looks to as the authority – especially if they pay to use the API.
Perhaps your API is just for “internal use.” But there’s always someone that relies on the results. Maybe the web team uses your API to search your company’s product catalog. They then display the results to your customers. Busy server, no results – no sale. Maybe your API is not so internal after all. Maybe load testing your API sounds like a good idea- it always does after the fact.
Either way, the last thing you want is for your users to be screaming about an error like
- Service unavailable
- Connection rejected
- Server timed out
- Or “Unknown error occurred”
If you’re the consumer of a service, like a financial transaction API, you want your app to handle any exception that API throws. But as the web service publisher, it’s up to you to make sure your API can handle heavy loads so it doesn’t throw those exceptions. Or if it does, it does so gracefully and in a well-defined and documented manner.
If you can’t guarantee your users predictability under heavy load, you’ll lose that authority. They’ll pay someone else for a more reliable API.
In other words, you need to test the heck out of your APIo.
There are three typical approaches to load testing an API. These, as well as various hybrids, each have drawbacks and advantages.
#1 Faking it – API Mocking
The least useful for load testing is API mocking. These are basically temporary placeholders used during development of an API. Developers use these for early unit testing. But they also provide these “dummy” versions to other teams that need to call the unfinished API from within their own code.
Mocks return hard-coded responses, the formats of which often change during development. They have little “meat” behind them, only canned responses that can be served up immediately. And they are disposable – once the actual API is completed, the code is thrown away. In other words, the effort to create the mocks is wasted after their initial use.
And because mocks aren’t yet hooked up to actual data sources, their performance isn’t at all representative of the real world.
In summary, mocks may be useful for unit testing, but they aren’t representative of the real world.
#2 Cloning it – A Full Test Environment
The mock API approach assumes the API isn’t actually complete, or that there’s no live data source available. If the API is complete and ready for testing, we can use the clone approach. This approach stands up a full-blown test environment using a snapshot of production data. Often you’ll have a copy of all the applications running, too, because your code relies on other APIs, not just database queries.
The good in this approach is that it’s very representative of the live system’s performance. If your load test brings the system to its knees, no problem! It hurts no one.
The bad parts outweigh the good though. For one thing, production data often contains sensitive customer information, so data privacy regulations can come into play. Also, if your own API makes use of pay-per-use services, this testing can get very expensive. Finally, production data is stateful, even in a cloned environment. If you need to re-run the tests, you’ll have to reload the test bed each time.
In summary, cloned environments give you near-production-quality load test results, without harming production if your test causes it to crater. However, they raise privacy concerns, require constant data reloading, and can be very expensive if calling third party APIs.
#3 Hitting production – Load Testing Live APIs
Sadly, the method so many companies use is to test on production itself. Why? Well, it’s live, so it’s the most representative of all methods. It’s also “easy” – no separate test environment to stand up and maintain. But that’s where the usefulness ends.
In production, you can’t use the “real” data. Instead you have to maintain separate “test accounts.” Using test accounts in a live environment reduces privacy concerns to a degree. These data are stateful, of course, so you have to reset them before each test run. And you still have to worry about the cost of pay-per-use APIs.
It’s important to schedule your load test “off hours.” This can be tricky, since the Internet allows your customers access 24/7. Even then, if your load test brings the server down, or corrupts production data, you’re fired (or you’ll soon wish you were).
Remember, Service unavailable signals your customers and their customers that your site – or your API service – is unreliable.
#4 API Virtualization – The Risk-free Approach
You want to perform an exhaustive load test of your API – and keep your existing customers and your job. That rules out hitting the production system.
You also need more than a test version of your API in a cloned environment. You need to test the real API in an environment where you can control the load condition for aspect of the test.
That’s where API virtualization comes in. A virtual API provides a sandbox environment where you can simulate environmental loads on network bandwidth, server and database connections, simultaneous users and more.
With API virtualization, minimal resources are required for standing up and testing the API. You test the API itself, not the end-to-end application with all its required backend systems. This eliminates the need for a clone of the production environment. That means you don’t have to reset and reload downstream data anymore.
The data-in and data-out can be as real as you wish. You can create your own requests and responses to test against, or capture and store actual requests and responses from a known source. The data can be played back during the load test, firing the requests as fast or as slow as you wish.
And those pay-per-use API’s? You can virtualize those, too, simulating a variety of responses and latency that the actual API could introduce.
The result? You can test your API in isolation, gauging the reaction against real data, under a variety of load conditions. And you can fine tune it until it passes with flying colors. No production system required or harmed.
Don’t give in to the strain
Hammering an API in production to simulate a heavy request loads isn’t wise. You can end up angering – and losing – real customers, not to mention your job. Not to mention affecting countless downstream systems.
API Virtualization allows you to load test your API in isolation from the rest of the system. By configuring requests and responses – or capturing real ones – your API can respond to a variety of requests with any number of responses – good and bad. With complete control of the test conditions, you can simulate network load, maximum connection limits, latencies and other conditions that happen in the real world.
Knowing how your API will respond under stress allows you to fix it, tune it, optimize it – before you unleash it for your customers to use.
About the Author: Les Worley is a freelance B2B copywriter with nearly 30 years of experience in commercial software development, IT and project management. He specializes in strategic B2B content for technology companies. You can reach him at Les@LesWorley.com or via his website at www.LesWorley.com