TestComplete PPC Ads TC10 - CS6-02

Remember when Software Testing was Considered an Artform?

art-testing

Over the last few months I’ve had the opportunity to attend a number of extremely interesting and mind-expanding conferences focusing on emerging (and somewhat disruptive) technologies and companies: APIs, mobile, cloud, big data – the works. They talk repeatedly about continuous integration, Agile methodologies, user groups and continuous deployment – but testing? Nope.

Why is that?

First, I’d like to elaborate a bit on what I mean (and don’t mean) by “testing.” I don’t mean unit tests. I’m not talking about recorded and automated scripts of your website or application. I don’t mean BDD or TDD based on some cool-named framework using a custom DSL to describe scenarios. Much of this is being done by many of these companies – which is great. I’m positive that all of this increases the quality of their products.

What I mean by “testing” is testers who try to break stuff. I’m talking about testers who do those things with your software that users weren’t intended to do, who provoke the hardware hosting your application to behave like it usually doesn’t, and who continuously challenge the design and architecture of your products by thinking outside the box and applying this in a methodological and structured way. When it comes to quality, these testers will be your greatest pain and your biggest gain. They take the quality of your products to the next level by doing all that crazy stuff your users do (and don’t do), giving your team the opportunity to fix them first.

So, back to the question: Why is it that these oh-so-crucial testers and the art of testing are so absent from these companies and conferences?

I have three possible explanations in my mind.

1. Developers aren’t testers

Developers – I love you – but you aren’t testers. A tester’s mentality and talent when it comes to quality is to find defects and ultimately break stuff. Developers, on the other hand, want to make sure things work. This might sound like a small difference, but the implications are huge. The “developer” of a steering wheel will make sure it turns the car left or right when you turn the wheel left or right (at the required rate/degree/whatever). The tester, on the other hand, will jerk the wheel back and forth to bring it out of balance, will submit the wheel to extreme (but plausible) conditions at the north pole and in the Sahara Desert, and might even compare the wheel to the competitors and tell you why theirs is better.

Developers confirm, testers overhaul. That’s just how it is. Unfortunately, though, developers are usually at center stage in a development team, and often lack the insight into both the craft of testing and the time it takes to do it right. You need both on your team for your quality to be top-notch, and neither can stand in for the other.

2. Agile Testing = Automated Testing

Don’t get me wrong – Agile development can be fantastic when executed correctly, and has surely improved the lives of many a developer/tester/product-owner out there. Unfortunately, though, Agile teams (at least in their infancy) often put testing efforts into the hands of developers (see above), who often believe that you either need to be able to automate all your tests or that a BDD/TDD specification is a valid substitute for testing. Neither is correct.

Using a BDD/TDD specification as a test is just another way of checking that your software performs as required/designed/specified. And, as already argued above, exploratory testing is key to finding those out-of-bounds conditions and usage-scenarios that need to be fixed before users encounter them.

3. Cheap-skating quality

OK – you’ve convinced your Agile team they need to do exploratory testing during their sprints, and your developers have reluctantly agreed that they aren’t testers at heart. So what happens when you approach the management team with a request to hire an expert tester?! Hands in the air if you think they might answer something like:

  • “We have a deadline – we need to release – we’ll invest in development now and testers later.”
  • “Don’t our developers have 90% code coverage? Do we really need testers?”
  • “Our users will help us iron out those out-of-bounds issues and quirks. That will be ample feedback for future improvements.”
  • <Any other “explanation” that is based on the reluctance to spend money on quality>

No one raised their hands? Phew, that’s a relief! Otherwise, given the already stated arguments, this is an obvious and probably the most common mistake. Your story-telling talents will be put to test. Hopefully you can convince management to make the investment.

What to learn from this mini-rant?

To put it simply:

  • Understand that testing, just like development, is a craft of its own
  • Cherish your testers and their expertise
  • Invest in quality – your users will love you for it.

Before I leave you, I have a confession to make: I’ve been guilty of all three of these issues. I’ve been the developer thinking that his code and unit tests will be good enough to handle out-of-bounds users and behavior. I’ve been on Agile teams voicing the “holisticity” of automated tests. And I’ve released features to users with the expectation that they will help us iron out the final issues instead of hiring a skilled tester to do the same (users, testers, everyone: I’m sorry!).

So I speak from experience as I plead with you not to make these mistakes with your products. Hire some kick-ass testers that turn your product upside-down for you. Your users and customers will love and cherish you – and so will I.

See also:

Comments

  1. Excellent read.The post is really informative.testing has become planned and methodical activity.Testing is not done at the ending phase but each phase of development cycle.New technology, cloud computing, web application, smart phones, stricter regulations have given rise to a whole new world of testing.

  2. Testing should be taken more serious for sure 5these days the testing changed due to cloud migration and less hardware to install on directly. Still I’ve found holes in Zune and Outlook online that can be called security risks. It’s just waiting for one of “the clouds” gets punished like Amazon cloud has been and Microsoft also for this lack off particular testing. Try edit your live/zune tag with the Zune interface software for instance!!!

  3. Why aren’t testers at developer conferences? Because we’re too busy organising our own conferences: http://www.ministryoftesting.com

  4. Ulrika Park says:

    Hi! 
    I think “agile” can be left out in the article. The mindset towards testers differs between teams and organization no matter if they use Scrum, XP, ad hoc or waterfallish process.  
     
    Some Scrum teams are really cherishing their testers, many old school process projects, don’t. 
     
    Most projects/developers in general lacks the insight of the value of testers, no matter what kind of process they use, and need to read this article :-)  

  5. Ulrika Park says:

    I’d also add to that that teams I’ve met who are mature in their way of working with BDD / Spec by example realize after a while that they need a qualified tester in the team.

  6. Ole Lensmar says:

    Thanks all for your comments! 
     
    I specifically mentioned Agile because its practices (and practitioners) often put a special emphasis on automated testing as a prerequisite for quality. As far as I know the same push for (test)-automation is not as evident in waterfallish processes, where testing/QA often has a separate and dedicated “place” in the lifecycle. 
     
    Thanks again, 
     
    /Ole

  7. Hi, 
    This generation is totally new about the latest technology and they are always in thirsts to know more. So thanks for your fabulous sharing. It enhances our knowledge and gives some overall ideas. I would like to read some more of your posting in future… 
    Keep it up! 

Speak Your Mind

*