a difficult one, and is an often-discussed problem in computer ethics.1 Students sometimes answer this question by saying,. âIt is good enough when the ...
fe a t u re d c o l u m n s
THINKING PROFESSIONALLY Don Gotterbarn
‘Perfection’ is not ‘Good Enough’:
Beyond Software Development I HAVE BEEN THINKING about perfection recently. Striving for perfection in all that we do is considered noble and beautiful (as in the movie The Last Samurai). Others think perfection is a goal beyond our reach as: “Have no fear of perfection- you’ll never reach it.” (attributed to Dali) People who work in the design and development of software face perfection problems every day. If perfection is not achievable, then what is the best and most efficient way to design and implement a system? When can I stop making just one more fix? Did I do the best I could for those who will use and be affected by my work? Is it enough to strive for excellence in our work and make gradual improvements? The decision when to be satisfied is a difficult one, and is an often-discussed problem in computer ethics.1 Students sometimes answer this question by saying, “It is good enough when the professor will not find a problem and I will pass the course.” Consultants sometimes answer this question by saying, “It is good enough when the customer is satisfied 1 [W. R. Collins, K. Miller, B. Spielman, and P. Wherry. “How good is good enough? An ethical analysis of software construction and use.” Communications of the ACM, Vol. 37, No. 1 (January 1994), 81-91.]
8
acm Inroads
and will pay me.” Both answers are ethically unsatisfying because they encourage a minimal approach to our responsibilities. Recently, however, especially in the toxic
“It is good enough when the professor will not find a problem and I will pass the course.” Consultants sometimes answer this question by saying, “It is good enough when the customer is satisfied and will pay me.” political speech common in the USA, the absence of a complete and perfect solution is equated with a categorically unacceptable state that we need to avoid–imperfection. This binary view–something is perfect and therefore acceptable otherwise it is imperfect and unacceptable–like most binary
2010 December • Vol. 1 • No. 4
claims is questionable and generally dangerous. The demand for perfection means people throw away what they could have. Unfortunately, then the absolute demand for “perfection” is the enemy of the good. In the 1950s, polio was killing more children than any other communicable disease. A vaccine was developed and tested that had (“only”) an 80% to 90% success rate. If perfection–the complete 100% success rate–was required before the vaccine was released, thousands more would be dead or crippled and the vaccine might not have been eradicated until much later. It seems unreasonable that anyone would hamper this forward progress by insisting on perfection. In ordinary life we are aware of the absurdity of the argument that says “if you can’t have perfection, then don’t do it”. I can still hear my children arguing that they should not have to brush their teeth because they would still get cavities–tooth brushing is not the perfect solution to oral hygiene (I had precocious children). As professionals, we realize that making absolute perfection the standard for acceptance would lead to inaction. Making the achievement of perfection an absolute requirement would lead to harm. For example, we could never release any virus protection program if a perfect and complete solution to all viruses was required. Yet such ‘imperfect’ programs are good and valuable because they do block many virus attacks. The striving for perfection leads to good and beneficial results. “Excellence” is the goal, not some indefinable unattainable model of perfection. Is your work as consistent with the highest standards achievable? Does it make a positive contribution? Is it helpful? Does it avoid harm? Is it demonstrably better than the existing alternatives? This is excellence and is acceptable. Don’t let a perfection trap that says we should reject anything less than a final and completely perfect solution. There is a second perfection mistake in addition to using failure to achieve absolute perfection. It is to reject all the potential positive impacts of excellence in the design and development of software. This other mistake relates to the delivery of the system.
featured columns
Those who write safety critical software are particularly concerned with approaching software ‘perfection’. For example, the amount of work that goes into testing avionic software is significant and many contingencies are carefully studied. All choices made are based on the best computer science and development standards: have others test your system, develop comprehensive test plans, accept and respond to critical analysis of your system in an ‘egoless way’ so the delivered system is almost perfect. When necessary, formal proofs can be given for code segments. Thorough efforts like this lead to a sense of satisfaction with the delivered product. Unlike hardware, software never ‘wears out’, so if there is a problem some people immediately blame the user for the problem. Many users of computer artifacts assume that the software is (sort of) perfect for the job, so if something goes wrong, it is probably not the fault of the software. Unfortunately, frequently our sense of moral responsibility as developers is satisfied at this stage. The client is also satisfied with this (almost) perfect system. But then, the unimaginable happens; our (almost) perfect software reveals itself to be flawed. It doesn’t exactly wear out; it just wasn’t perfect when we delivered it, but we didn’t know it at the time. We delivered the software into an environment that we knew would change. We could anticipate some of those changes in the contexts that would make our software function less than perfectly. With the exception of anti-virus software, most software artifacts are delivered with neither a warning nor means to address easily the ways in which changing contexts will affect the functioning the software. Software does not wear out but the situation often changes in a way that makes the software no longer completely fit for its original intended function. There is an analogous and equally dangerous view about the need for perfection in development, namely the belief that perfection has been achieved. This belief leads us to ignore our knowledge that the context will change, and any “perfection” will be short lived. Because of the impact of changing contexts, excellence also requires effectively addressing this problem of a
moving target: the changing environment. The investigation of a 2008 crash of Spanair flight 5022 showed that the central computer did not detect three technical problems with the aircraft because it was infected with malware. These problems should have prevented the plane from taking off. There is no indication of how the malware got into the system nor is there any information about Spanair’s scanning
Excellence in development needs to address the context in which the computer artifact is delivered. processes. When a product that manages a safety critical system is delivered, what constitutes due diligence when it comes to the security of that system against malware? The software was designed to identify technical problems with the aircraft, but what precautions are taken to isolate flight computers from possible infections? One of the things we learned from research on the Challenger disaster is that as we get accustomed to minor failures the “attention to failure mechanisms will wane. In the Challenger they gradually accepted degradation of the seal with each successive flight”.2 Knowing that a significant portion of virus scans result in “virus found–low level risk”, it is reasonable to assume that running virus scans repeatedly would get boring and little attention might get paid to low level risks. It would be reasonable (more perfect) to develop policies to address this possibility during system development. Options similar to the one’s used in elevator maintenance might be implemented; if the elevator system is not maintained, if the aircraft safety software is not scanned, then the elevator will not move, and the plane will not take off. We don’t know the details of the 2
[D. Christiansen “The Inconceivable Consequence of Failure”, Today’s Engineers, August 2010].
Spanair crash, but we do know that the software was successfully designed to disable takeoff if mechanical problems were detected. The airplane software, like the elevator software, could have encouraged software scans and updates of antivirus software, and then connect that with the takeoff disabling function thus addressing an anticipatable changing environment. Excellence in development needs to address the context in which the computer artifact is delivered. In our teaching and program evaluation, we need to require students to ask broader questions than “will this get past the instructor?” In thinking about their work, they need to ask questions about where and how their software will be delivered and how that context will change. Then they need to ‘perfect’ their work accordingly. Think how this approach might affect an airplane black box design. The resilience and reliability of recording and storing data in a black box is widely acclaimed. However, when planes crash they sometimes crash at sea and the black box is never found. Ask students how to address this limitation of storing flight data aboard the aircraft. Think how these kinds of questions will affect other designs. In order to facilitate speed of transmission, predator drone signals sent to control centers in the USA are not encrypted. This may be OK until the context changes and the enemy buys some equipment at a nearby Radio Shack. Ask “What needs to be done in development and design to address the availability of new radio equipment?” Excellence in development requires practice and creative thinking. We need to make our students aware of these issues in a way that captures their attention now, and helps them remember them on the job tomorrow. Ir Don Gotterbarn Computer and Information Sciences East Tennessee State University Johnson City, Tenn. 37614, USA
DOI: 10.1145/1869746.1869748 Copyright held by author.
2010 December • Vol. 1 • No. 4
acm Inroads
9
