Will My Car be Safe to Drive?
Emeritus Professor of Computer Science, New York University
President, Ada Core Technologies
It’s no secret that the cars we drive today, and especially those we will drive in the near future, have huge amounts of sophisticated software aboard. By some accounts the number of lines of code in a car can significantly exceed the number of lines of code in a modern commercial jetliner. And as with the jetliner, we are entrusting our safety to the reliability of this software.
I should say right away that I am not an objective observer here: I have a special personal interest. I am just about to buy a new car myself. As we grow older, we all know that one of the dangers of driving is drowsiness at the wheel. Many car accidents occur when a driver alone in the car loses attention and drives off the road. For this reason, I find myself very interested in cars like the new 2014 Mercedes E-Series which have an impressive range of semi-autonomous driving capabilities, including automatic lane following, automatic collision avoidance, and sophisticated cruise control. Some cars even have cameras watching the driver to detect drowsiness directly. And most certainly Mercedes is not the only company moving in this direction. Ford for instance is installing similar capabilities in its new F150 truck, and every manufacturer is either deploying such features, or planning to do so in the near future.
So that sounds like a good idea, and features like this could in general be life-saving, and in particular could save me from a serious accident. So that’s of definite interest. But I am an expert in safety-critical software requirements, so I have to wonder: is this complex software really going to work? Fancy software can be much worse than useless if it malfunctions, and we already have a number of confirmed reports of cars with software problems, in some cases clearly safety-related, and rumors of other possible problems.
Now I fly on commercial airplanes all the time, and I will say that, although the software aboard such planes is complex, I don’t feel nervous, because I know the way this software is constructed and verified. There is a dedicated safety culture in place for the production of software in airborne systems, not to mention the use of effective tools (it is no accident that a lot of avionics software is written in Ada, a language designed to promote safety), and finally rigorous standards, notably DO-178B, are used to control this whole process. No one is about to claim that this guarantees perfect reliability, but we do pretty well in practice. As far as we know no one has ever died in a commercial plane crash caused by a software implementation bug. But we have had a few close calls, and we are constantly trying to improve reliability. The new version of the avionics standard DO-178C allows the increased use of formal methods, where we actually construct mathematical proofs of correctness. Even that’s not a silver bullet (after all an incorrect specification accurately implemented in a bug-free program can still be problematic), but it helps.
So where are we when it comes to automotive software? Well let’s just say that I am not nearly so ready to declare that I am not nervous. The standards and procedures applied to automotive software are not yet comparably rigorous, and I am not at all convinced that the same culture of reliability exists. That is unfortunate; even a “minor” bug that occurs only under rare circumstances and that would not be detected during normal testing, will eventually occur at some point because of the number of vehicles on the road. Is current software practice in the industry capable of preventing such bugs? Although the proprietary nature of this software and its development make it difficult to give a definitive answer, what we do know (for example, the testimony regarding code in a Toyota car in a recent court case) is not reassuring. On the other hand, we see signs of the automotive companies becoming much more interested in and committed to reliable software. We can certainly hope that there are no disasters waiting to happen, but it is difficult to be completely confident.
So, what advice should we give to software engineers working on producing this software? Tools are indeed important, but no tools can guarantee success. What is really needed is a completely rigorous approach that is oriented to getting it right the first time. If you build buggy software and then depend on testing or analysis tools to find errors, you will find some of the bugs, but you won’t be able to be confident that you have found them all. Requirements have to be carefully vetted, and the specifications derived from these requirements have to be rigorously defined and checked. Every step of the process must be carefully controlled, and even if we are not following a standard that is as rigorous as DO-178B or DO-178C, knowledge of these standards, and learning from them is an excellent idea. We are beginning to see the introduction and enforcement of standards for automotive software, but we still don’t have the kind of independent auditing that is normal in the avionics world.
The automotive software engineer of tomorrow must become well-versed in the latest tools to assure reliability. In particular the use of formal techniques is becoming practical. I think we all know that in typical C code, buffer overruns and overflows are a major source of unreliability. Well there exist languages and proof techniques that can prove that programs are free of such errors. We are not talking about testing till we think we have reached a reliable state, we are talking about an indisputable mathematical proof that there are no bugs of this kind in the code. With modern fast machines, it is becoming practical to do this even on large programs. For example, the SPARK language has been used in building a component of the new air traffic control system in England (iFacts) and this program is proved to be free of run-time errors. The software engineer of tomorrow who is determined to build reliable, safe, and secure code should consider formal methods as an essential tool for software verification. It will not replace traditional methods such as testing, but it can effectively complement them to provide the necessary assurance.
I am looking forward to driving my new car, and quite likely, its advanced features may save me from an accident one day, but I can’t help feeling a little nervous about the possibility that these same advanced features may cause an accident. With luck and dedication, the situation will improve without facing some real disaster on the roads. Let’s hope that this is the case.