I have a two-part series (one, two) on testing posted on my Software Evangelist blog on the Intel Developer Zone. This is a long piece where I get back to the interesting question of how you test things and the fact that testing is not just the same as development. I call the posts Mindset and Toolset
I have been thinking about the role and prestige of testing for the past several years. Many things I have read and things companies have done indicate that “testing” is something that is considered a bit passe and old-school. Testers are dead weight that get into the way of releases, and they are unproductive barnacles that slow development down. Testers can all be replaced by automatic testing put in place by brilliant developers. The creative developer types are the guys with the status anyway. I might be exaggerating, but there is an issue here. I think we need to be acknowledge that testers are a critical part of the software quality puzzle, and that testing is not just something developers can do with one hand tied behind their back.
There is a new post at my Wind River blog, about how you can use Simics to enable the automatic testing of pretty much any computer system (as long as we can put it inside a simulator). This is a natural follow-up to the earlier post about continuous integration with Simics and Simics-Simulink integrations — automated test runs is a mandatory and necessary part of all modern software development.
During my vacation, a blog post went up on the Wind River blog with an interview with Hyungmin Cho, a researcher at Stanford. Hyungmin has done some seriously heavy and cool work with Simics, using it together with a circuit-level simulator to investigate error resiliency in hardware devices, and how errors propagate from hardware into the software. As part of this process, he has setup an automated test system using Simics, and this system has done more than a million automated Simics runs. That is an insane number – I have been using Simics for twelve years now, and if I had used it every day for all these years, I would have had to start 10 runs per hour, every hour of the day. It shows the power of automation along with parallel runs on clusters of machines – once the setup is automated, you can pour on the volume.
Continuing on the thread from my previous post about the testing of products that fail to find problems that become obvious to (some) users after a very short time, I just read an article (in Swedish) about how the famed Tesla roadster cars behaved when they were confronted with Scandinavian winters.
I recently started using a new mobile phone, a Blackberry Bold 9700. I am a bit ambivalent on some of its design features, but it is certainly a very different device from the much more friendly SonyEricsson I had before. Like anybody would do, I have been playing around with it to see what it can do and what not (notable things not working: the “AppWorld” application store is not available in Sweden, YouTube videos do not play in any way that I can figure out).
And almost inevitably, as you play around with a complex modern piece of software (which is what most of the phone is, after all), you find some obvious things which are just plain broken. You wonder, “why didn’t they think of this”, and “how could this ever escape testing?” My current best example is that the built-in web browser does not render the pages from Blackberry’s own support knowledgebase.
Another Cadence guest blog entry, about the overall impact of virtual platforms on the interaction between hardware and software designers. Essentially, virtual platforms are a great tool to make software and hardware people talk to each other more, since it provides a common basis for understanding.
Virtutech and Cadence yesterday announced the integration of Virtutech Simics and Cadence ISX (Incisive Software Extensions), which is essentially a directed random test framework for software. With this tool integration, you can systematically test low-level software and the hardware-software (device driver) interface of a system, leveraging a virtual platform.