I have been thinking about the role and prestige of testing for the past several years. Many things I have read and things companies have done indicate that “testing” is something that is considered a bit passe and old-school. Testers are dead weight that get into the way of releases, and they are unproductive barnacles that slow development down. Testers can all be replaced by automatic testing put in place by brilliant developers. The creative developer types are the guys with the status anyway. I might be exaggerating, but there is an issue here. I think we need to be acknowledge that testers are a critical part of the software quality puzzle, and that testing is not just something developers can do with one hand tied behind their back.
There is a classic saying that “you cannot test quality into a product” – which certainly is true. But it is also true that without testing, you will not ship quality of anything. With a view of testing as something that is bolted on and tests things after they are done and completed, that you cannot test quality in is true. But is that really what testing is? I think testing is something that has to be done continuously as a product matures. Testing is indeed part of mainstream development – but even so, I am convinced that testing and development are two different things. Testing and development do need to work together (as noted in this nicely written piece) but a good tester has a fundamentally different mindset from a developer.
I had some discussions with some guys from Vector Software about this a couple of weeks ago at the embedded world, and we all agreed that a good tester has a creatively destructive mindset. A good tester is someone who can smell out the weak spots and cracks in a product. Even the best developers have a hard time seeing those weak spots, since they are naturally focused on making things work, not making them break. The best developers are creatively creative – they thrive on making things work, not on making things not work.
I have personally experienced this many times – I have had the fortune to work with many very good engineers building generally good software. Even so, almost every time when I first test a new piece of functionality, a new release, or a new software stack, I almost invariably manage to break it within a few minutes. For some reason, I have knack for stumbling on the broken stuff. I think there is a voice in the back of my mind urging me on to step into the areas where the spec is murky or where it looks like the thinking behind the design was a bit muddled. With experience, you just learn to spot these things. And that is the key skill of a good tester – the intuition and sense of smell for what will break and how.
Note that I am not talking about other problems with classic testing in rigidly structured companies, such as only testers understanding the testing infrastructure or only testers being able (or allowed) to write tests. That is obviously broken – all members of a dev team need to understand how to run and add tests, and how to maintain the automatic testing and continuous integration system. Absolutely. Testing today is part of the software development life cycle from the first lines of code written, and testing should be done continuously during development, integration, and maintenance and modification.
The fundamental skill of a good tester is in designing tests that break things – and that can then be added to regression suites and automatic tests to make sure that nothing like it comes back into the software again. Testing is a very creative exercise – and when doing things like test-driven development, you end up with a product that contains tests and code. The tests are as valuable as the code, and require the same respect. A good test is valuable as good code, and will live along with the code.
The security issues that seems to flood us every week is strong supporting evidence for the existence of testing as a skill different from development. If you think about it, finding exploitable bugs in a piece of software is the essence of good testing – looking around for the weak spots, finding the cracks, and prying them open to get inside. It is what testing should be – finding the broken stuff. And ideally, someone inside the organization that released the software that was hacked would have had the creativity to find the weakness before it was released. When we look at how such security exploits are found (see for example this presentation, goto fail, …) it is clear to me that what is needed is more testing and more people going over their own products with that mindset of trying to find ways to make it fail.
The world needs more real testers who really test the living daylights out of software – not fewer.
One thought on “In Defense of Testing and Testers”