I just read the EETimes coverage of the recently concluded court case in the US, where Toyota settled for 3 million USD in damages due to experts finding that the software in a 2005 Camry L4 could indeed cause “unintended acceleration”. In the particular case that was concluded, the accident resulting from the issue caused one driver to be injured and one driver to get killed. This feels like it could be the beginning of something really good, or just as well this could go really wrong.
The facts of the case are quite interesting. Back in 2011, a team from NASA reported that they had found no clear evidence that the control software could cause unintended acceleration. Toyota was cleared by the US NHTSA. I blogged about the report on my Wind River blog, and noted that NASA reported not running all the code in simulation due to a lack of tooling. Now, the new panel of experts appears to have actually managed to simulate the system, and found ways to make it crash in the interaction between multiple tasks. The fact that, as EETimes report, a certain task crashing can cause acceleration to continue without control, is pretty indicative of issues arising in integration rather than unit testing. What have gone wrong appears to be not the basic control logic, but its implementation with multiple tasks, shared variables, and variable values not checked against corruption.
The EETimes quote Michael Barr:
Memory corruption as little as one bit flip can cause a task to die. This can happen by hardware single-event upsets — i.e., bit flip — or via one of the many software bugs, such as buffer overflows and race conditions, we identified in the code. There are tens of millions of combinations of untested task death, any of which could happen in any possible vehicle/software state. Too many to test them all. But vehicle tests we have done in 2005 and 2008 Camrys show that even just the death of Task X by itself can cause loss of throttle control by the driver
Scary, that the software is so sensitive even a single bit value change can cause it to completely go off. It does have to be a very (un)lucky bit change, however. But the buffer overruns and race conditions I can certainly believe in as likely causes of an intermittent crash. Would have been interesting to know if some particular environmental condition caused this, or if it was indeed totally random. Probably, nobody knows.
So, on balance, it would seem we have another case of software being implicated in deadly accidents. It has happened before, but I do think this is the first time for cars. Therefore, this is a landmark in the history of embedded software.
However, pulling this kind of case through the courts feels instinctively wrong, but in this case at least the experts hired by the plaintiff appear to have done a really good job. I cannot make myself like the outlook for millions or even billions of dollars being pulled out of Toyota for various damaged parties via the US court system – it just does not really help solve any kind of problem, and randomly benefits the people in the US that sued. I am not saying that those who have been hurt should not be compensated, but the US court system really tends to do this in a way that is overly costly and without reasonable limits. Maybe selling cars in the US is now turning into one of those business you just do not want to be in due to the risk of lawsuits.
On the good side, maybe better regulations can come out of this. The industry has been preparing for that for a long time, but the car industry is also dead scared of becoming like the airplane industry where stringent certification requirements make development projects take decades and make rapid changes and feature additions completely impossible. So far there have been no accidents reported for civilian airplanes that have been traced back to software, which is a good thing. But the weight and cost of the development seems very hard to bear for the car industry.
So regulation can be both good or bad.
We will see where this ends up.