True confession time: I’m a Candy Crush Saga addict. There was a time when we all were, but it’s become less of an open conversation these days and those of us who still linger in the parking lot to play just one more game before going into the office… well, we don’t speak of this to anyone.
Except when the Candy Crush user experience crosses over into our work lives – then it becomes fodder for morning meeting discussions. That’s what happened to me recently. I was happily playing a few rounds while having my morning coffee (I like to start my day by running out of lives – you can only go up from there) when suddenly this happened:
And I’m not the only one – the forums are full of questions about this error. For me, it was annoying – but for people who don’t even know what an API is (and I suspect that’s the majority of Candy Crush players), this would be downright unnerving.
The error beneath the error
To me, the real bug here is not that the API threw an error message. It’s that the application didn’t trap it and make it something palatable to a player. After all, there are a lot of reasons why an API may fail just like there are a lot of reasons why a network can fail or a database can throw an error. The difference here is that it is usually pretty difficult to generate an API failure so it’s a condition that is often not tested.
If you’re relying on APIs for your application (and in the web and mobile world, who isn’t?), you need to consider them as part of your testing path so you can make sure you don’t display an error like the one above. You also need to think about all the ways an API can fail, especially if you are consuming third-party APIs. Let’s look at a few of those reasons:
The API request was bad
What many people forget is that APIs are not static. As development continues, the API may change in ways that are not backward-compatible (of course, we don’t recommend this for an API provider – you should always do your best to support your previous versions so applications don’t have to change when you do). If the API provider isn’t clearly communicating those changes and how to adapt to them (and many don’t), your application may be making calls to the API that are generating errors.
The API is overloaded
If you are consuming third-party APIs, you need to be especially worried about this. You don’t have visibility or control over the production environment where that API is hosted, for one thing. For another, you don’t know how many other apps are consuming that API and how much load they’re generating against it. You also don’t know how well the provider tested the API – for example, did they actually run realistic and varied load tests?
You’ve reached your rate limit
This is especially true for third-party APIs but could also be inherent to internal API contracts. This is one way for an API provider to ensure that their API isn’t being attacked but it also helps to ward off load conditions that might make an API perform poorly. When you reach your rate limit, an API will simply refuse your request, which means your application is left hanging unless it has a Plan B in place.
The production environment is underpowered
While this can be related to load conditions, it is not necessarily the same thing. If the API is hosted on a slow server, the API responses may in turn be slow, especially if there is a large payload coming back. For their own performance reasons, most applications will only wait so long for a response before they either return an error or try again.
There are other reasons for API failures, but I think you get the idea. You cannot assume the API will always succeed.
Avoiding the Error Beneath the Error
Testing your application’s response to API errors can be difficult to impossible if you have no other option than hitting the production API. Even having the API in a test environment has its limitations if you can’t control the responses it provides. Frustrated testers everywhere find themselves controlled by the API rather than the other way around:
So, how do you get control?
API providers and API Management Platform vendors are all starting to recognize the problem and are figuring out ways to provide their consumers with sandbox environments. The challenge with sandboxing is that you still need more than just a test version of the API in a test environment. You need an API you can control and manipulate in various test scenarios and under various test conditions. This is where API virtualization comes in.
What is a Virtual API?
A virtual API is essentially a sandbox API – a simulated version of your API that you can force into specific behaviors. Ideally, you would be able to do a variety of things with that virtual API:
Create requests and responses
The benefit of a virtual API is it always responds the way you want it to within the context of your test. You need to create the requests that fit your test and, more importantly, determine the responses that you want to test against. Ideally, you would be able to hook the virtual API to a data source so it can get its inputs from a known source that fits the test criteria for your application.
Mimic a variety of error codes
Knowing how your application responds to a variety of error codes is important if you want to avoid that Candy Crush moment. For example, what does your application do if the API throws a 503 error and is just generally not available? Or if you have exceeded your rate limit and the API returns a 401-Unauthorized error? It’s important that you maintain your positive user experience by trapping errors so that you can display an alternative experience in the application. Likewise, if you get errors related to the request itself (request too long, failed dependency, etc), does your application notify the right people that there is an API error that needs to be corrected in the code?
Simulate Server and Network Behavior
An API’s response time can be greatly impacted by the capacity and power of the server that’s hosting it. Simulating various numbers of threads and setting response delays for the API response lets you validate how well your application recovers when the API under-performs. Likewise, if you’re building mobile apps especially, network capacity can be a determining factor for API performance. To fully test your mobile app’s response to varying bandwidths and congestion, you need to be able to simulate those conditions for the API as well.
Building great applications is more than just having a great user interface and a solid front-end. Your application is only as great as its ability to respond to the foundation underneath it. Virtualizing your APIs allows you to gain full control over your testing so you can ensure the best experience for your users.