BLOGCTA_AQT_20130726_1_Coverage_Profiling_936x110

Performance Testing As If Your Life Depended on it

Illustration by Heather Scoggins

Illustration by Heather Scoggins

Welcome to the Internet of Things (IoT), a world in which ordinary objects and machines—from personal earbuds to corporate water treatment facilities—are becoming increasingly programmable, “smart,” and absorbed within the ever-expanding definition of the internet. In our on the IoT revolution, we looked at the need for careful code review to ensure that our newly wired world communicates with itself smoothly. Let’s turn now to a related concern for the future of the IoT: Testing and monitoring software performance as if our lives depended on it.

A recent report from IDC predicts that by the end of 2020, “the installed base of the Internet of Things will be approximately 212 billion ‘things’ globally,” including 30.1 billion of those “installed ‘connected (autonomous) things’” appearing just in 2020 alone. Cisco offers a more conservative estimate of a mere 50 billion devices by the time 2020 rolls around, with an online counter on their site ticking away, adding up the ever-growing number of “people, processes, data and things connected to the Internet” based on their projections (which Morgan Stanley reevaluates at 75 billion, according to ReadWrite). No matter whose prediction you choose to run with, it’s clear that we’re going to be seeing a lot of internet-enabled things in the next few years. And every last one of these objects, machines, or automated processes is, of course, going to require stable software to function.

In such a competitive environment, where software is running literally everything­—from trackers to toasters to T-shirts—one buggy program could sink your company. Worse still, it could sink a ship.

Indeed, as we explored in the previous IoT post, perhaps the most worrisome problem with the “smartification” of objects, machines, and appliances is that entirely new kinds of safety concerns abound—especially if the objects in question are so smart that human beings willingly relinquish control to them, putting their lives in the hands of machines (and, by extension, the infrastructure supporting those machines). If you have a fleet of autonomous cars happily driving people on their morning commute, and just one of them is running glitchy software, that single bug could cause a major pileup. But the dangers of smart cars are just the beginning.

In an IoT world, problematic software could be lurking anywhere, in anything, just waiting to strike back with litigation at the hapless company that fails to properly test and continually monitor its web-connected products and services.

Just imagine the possibilities…

The Office Worker

It’s October 18, 2020, seven years in the future, and you’re walking briskly down the street on an icy fall morning in Boston, feeling perhaps a little too warm in your climate-controlled jacket. Your shoes transmit basic health tracking data as you walk, registering your steps and burnt calories in your heads-up display—a function of your augmented-reality contact lenses. You’re late for work, so you instruct your watch to share your position with your colleagues, instantly giving them a precise estimate of your arrival time as you continue to Skype with them while you walk. You shift your gaze to the left side of your field of vision, focusing on an app representing your office. It opens to reveal a number of options within it, and you focus on the coffee icon, selecting your usual setting, and decide to explore the main kitchen icon as well. Your alarm didn’t go off this morning for some reason, so your own kitchen at home had failed to make you breakfast again. Hungry and irritated, you take comfort in knowing that a mile away at the office, a coffeemaker, toaster, refrigerator, and stovetop grill in the company kitchen are now proceeding to get to work.

You see a taxi stand and decide to take one to save time, stepping into the back of the driverless car and giving it the address. As you lean back and relax in the cab, sharing your thoughts on yesterday’s presentation with your colleagues, you hear one of them start coughing over Skype, and then another. An alarm suddenly begins blaring and you cup your hands tight over your ears. The office kitchen icon in your visual display is flashing red, and you feel a wave of anxiety as you slink down in your seat. Not for the first time, you wonder if the world’s technology just has some kind of personal vendetta against you. Then again, hasn’t the manufacturer of those kitchen appliances been in the news lately for a number of similar incidents? Some problem related to the server running their different devices?

The Robotic Surgeon

Your patient is lying, sedated, on a surgical table in a hospital in New Delhi, and you’re sitting cozy in your office in New York. Gone are the days of remote-controlling surgical instruments via a secure web connection, which were headaches you’re glad to be without, as cool as that technology was. Your main gripe with those robotic arms was that you’d have to control your movements even more carefully than you would in a live, in-person operation given the sensitivity of the remote-surgery instrumentation. But now…now—you’re one of a handful of heart surgeons selected to use the world’s first fully functional ASRs, or autonomous surgical robots, in actual clinical applications. Sure, you know there are those nagging rumors that Robixia, Inc., rushed its prototypes to market despite a number of dubious trials, but you’ve personally tested the device on animals, and it seems to work with remarkable precision. Moreover, some of the best surgeons in the world consulted with the software dev team over the past year. You’re sure it’s perfectly fine, and you’re more than a little excited. After all, these robots could change everything, radically reducing human error—no, eliminating the possibility of human error! Why? Because there are no humans involved. It is 100% automated. All you have to do, once the parameters are set for a particular patient’s diagnostics and the procedure to performed, is press a button…like so. Bingo! It’s incredible. Oh. Oh, wait. Oh, no. No, this is not…right. No…it shouldn’t be cutting like… How do you—how do you stop it? How do you stop the ASR once it’s started?! You sit in your office and watch, aghast, unable to do a thing.

The Drone Commander

You’re standing on the beach with a gun in your hand, watching a fleet of UAVs hovering directly overhead. These new drones, they’re quite something. They’re tiny and they swarm just like bees, flying closely together until they’re ready to disperse their armaments on unsuspecting enemy combatants. The best part is, they’re said to be intelligent. They know how to adapt to changing situations on the ground. Something to do with innovative machine-learning protocols, a new kind of artificial intelligence software. In any case, there’s a renowned Al Qaeda leader across the bay, and you’re about to witness these puppies in action as they proceed on one of their first operational strikes. It sure as heck beats sitting in a hot, sweaty, claustrophobic box north of Las Vegas, remote-controlling large unwieldy drone aircraft from halfway around the globe. Here, you get to join the troops on the ground, right where the action is, and you can direct your swarm and choose your target using nothing but your iPhone, which, if you recall correctly from the briefing, connects to the heavily armed bees over an encrypted SSL connection routed through the CIA’s local server bus. Your iPhone beeps, and you look down at the digital display on your wrist. 10, 9, 8… The drone swarm is hanging high overhead, twinkling in the sunlight, but will soon kick into high gear and deliver a good dose of holy fire on the unsuspecting. Your proud smirk quickly turns to a frown as a message pops up on your watch: 503 Server Unavailable—Too Many Connections. You touch the screen, and the app crashes. You hear a high-pitched noise overhead and look up to see hundreds of shining metal wings accelerating toward you. Turning to run, you realize with a resigned certainty that you aren’t going to make it. Maybe the sweaty little box in Vegas wasn’t so bad after all.

The Moral of This Story

While these tales of dire warning may be just a tad melodramatic, they’re definitely not outside the realm of possibility. In fact, once we’re living in a world where we have 212 billion devices connected to the web, with new ones being deployed all the time, these kinds of incidents should be expected. But they can also be prevented.

Rigorous functional testing may not always be the highest priority when you’re producing a website or a video game on a tight deadline, as rushed schedules often take precedence in the quest for a financial ROI. But even if some companies fail to perceive what is, to us, an obvious correlation between high-quality software testing, rate of adoption, corporate reputation, and financial success, a focus on quality has to become a priority whenever you’re dealing with software that can potentially affect the safety and welfare of other human beings. And in the IoT world, the number of devices and ordinary household objects that might prove unexpectedly dangerous once infused with potentially glitchy “smarts” will probably be greater than we can even anticipate right now. Will smart clothing have hidden dangers? Smart lightbulbs? Smart space heaters? And even if you’re not a product designer or engaged in writing home-appliance firmware, if you deal with any of the sites, apps, or server-side programming that will connect or communicate with such devices, these concerns need to become yours too.

Once you’ve tested your new product thoroughly, squeezing all bugs out of your newly minted code and releasing it into the wilds of the consumer or enterprise marketplace, the importance of an ongoing testing and performance plan should become obvious. After all, in an IoT universe, maintaining your projects after they launch is, in some ways, more important than getting them to market in the first place. If you can’t keep up, if your software products can’t adapt, evolve, and learn to deal with stresses they weren’t originally tested to handle, then they will quickly fall by the wayside and lose their stake in what Cisco estimates to be a $14 trillion industry. Plus, poorly maintained programs may cause genuine problems along the harrowing lines explored in good sci-fi fashion above, necessitating new safety regulations, government interventions, and the lawsuits that all too often accompany such consequences. And no one wants that, now, do they?

That’s why, in the Internet of Things, proper functional testing will be required to ensure that your software simply works as advertised, keeping you competitive, while ongoing performance testing and monitoring will ensure that it continues to work properly, keeping your end-users happy: safe and sound in a strange new world remade in the image of the web.

That’s the thoroughly bright and smooth-sailing IoT future that we at SmartBear are invested in helping to bring into being. The question for developers and testers everywhere is: How invested are you?

See Also:

file-240016673

subscribe-3

Comments

  1. marketomlinson says:

    The Matrix = fiction. Skynet = fiction. Singularity University = non-fiction.

    :)

Speak Your Mind

*