TestComplete PPC Ads TC10 - CS6-02

Emulated vs. Real Device Mobile App Testing

mobile-testing-emulator-device

Pre-launch testing of any mobile application is a crucial step before going to market. Testing eliminates the chances your product will have to be recalled, not to mention the public embarrassment that goes with a defective app. And by testing your app thoroughly before it goes to market, you communicate to the consumer that your brand is one they can trust, as well as one they can buy from again.

According to International Data Corporation, 182.7 billion mobile apps will be downloaded by 2015. That’s a 1600% increase from the 10.7 billion apps downloaded in 2010. What mobile application development company wouldn’t want in on that?

While there are a number of OS’s out there, and more coming, there are two predominant ways that developers can test a mobile app. You can go with device testing or use an emulator program. At first glance, you might feel the need to choose just one avenue or the other, but the truth is that using both approaches is the best route to success.

Advantages and Disadvantages for Both

One of the key advantages of device testing is that you’re holding it in your hand and can see what works and what doesn’t work. You can test sign-up, login, dhandle data, connection speed and error messages on the fly.

You also find out if the app is as easy to use on one device as it is on another. Is navigation on a touchscreen as easy as a device with a QWERTY keyboard? Is the user experience on a Samsung Galaxy S4 circumscribed by the smaller screen of an iPhone 5? If one phone has a screen that doesn’t deliver the same resolution as another, is there a significant drop in experiential quality?

The challenge with device testing is that if you want 100 tests, you’re going to have to do them one at a time. Or get another 99 testers, all looking for the same things you are. It is also worth asking, will every single person testing the device use the same standards as you do?

Of course, to test on all platforms is going to require the acquisition of a lot of handsets. Not a deal breaker, but not cheap. As such, some mobile app developers prefer to use emulators or simulators to test the operation of an app.

Most operating systems have a readily available SDK that allows for creation of apps. As such, multiple emulators can be programmed to run doing much of the same work as a device tester, but faster and at greater volume. It does not require the same amount of device hardware, either.

Additionally, an emulator can extract data in real time and refresh reports as it runs, so the information an app development team is working with can be quickly accessed.

This is not to say emulators are the alpha and omega of mobile app testing.

Unless the software is programmed down to the finest nuance of human behavior there are things a person will catch that an emulator won’t. Likewise, a computer or computer program can’t walk across the room if something looks suspect, tap you on the shoulder and ask a question. As such, most developers are going to use a combination of testing methods to get the best results.

Ensuring the success of a mobile app in this marketplace is too important to make your testing an either/or proposition. Since neither is inherently superior, as judged by the number of developers who rely on both, it would be shortsighted for anyone to place all their testing in just one.

So if someone should find out you’re in mobile app development and ask which testing method – device or emulator is better – the answer is “yes.”

See also:

 

Comments

  1. One other way to test (well, actually automatically check, but that’s another discussion) on many different devices is https://appthwack.com/.

    • Trent Peterson says:

      Thanks for the mention, Stephan!

      I agree with the overall gist of this article. Emulators/simulators have a place in testing and that place is very close to development. From a test and QA perspective, though, carrier and OEM modifications, performance differences, environmental factors, etc. require physical hardware to accurately gauge how a real-world customer will experience your app.

      Sure, actually obtaining real hardware is expensive and time-consuming, but to your (Stephan’s) point, there are other options out there – options that don’t require 100 people to run tests on 100 devices in parallel.

      I’ll leave the automation discussion for another time as well, but for the UI and UX testing described in this article I’ll definitely say real people with real devices is the way to go. Of course, to free up time for that you might need to *cough* automate some of the mundane tasks you’d otherwise be busy performing.

Speak Your Mind

*