THE PROFESSION
Software Development: What Is the Problem?
While it doesn’t appeal to most of us, maintenance is essential to the software industry. This activity involves more nonengineering activities. First, the maintainer must learn what the computer hardware was instructed to do—that is, what the developer expected it to do. Then the maintainer must modify these instructions.
Raghavendra Rao Loka, Synopsys
Writing and maintaining software are not engineering activities. So it’s not clear why we call software development software engineering. There is no doubt that other scientific and engineering disciplines lay the foundations for software development. But that does not imply that software development is itself an engineering discipline. Maintenance keeps getting more expensive. However, most, if not all, of the literature attempts to solve this problem from the premise that software development and maintenance are engineering activities that touch only slightly on the teaching and learning aspects that are more analogous to a craft or art. Others agree, asserting that software development isn’t really engineering, and many have written on problems with current software development.
Skepticism, expertise, and good management can right many software wrongs.
M
anaging software development is getting harder. Expensive before offshoring, it has now become expensive even in offshore countries. The clamor for more H-1 visas in the US would have us believe the country has a dearth of software professionals. Is it really a shortage of resources or a misuse of them? What exactly is the problem?
WHAT SOFTWARE DEVELOPMENT IS … According to Wikipedia (http:// en.wikipedia.org/wiki/Software_ development), software development is closely related to software engineering, and writing and maintaining software are important aspects of this discipline. Ultimately, software provides the set of instructions that enables computer hardware to perform useful work. Many developers view software development—in particular, writing and maintaining software—as a scientific or engineering activity—as the annual International Conference on Software Engineering maintains (www. 112
Computer
icse-conferences.org). Writing software is neither: I view it as a craft or art, similar to the work required of teachers and writers. The literature indicates I’m not alone, as Donald E. Knuth’s book, The Art of Computer Programming (Addison-Wesley, 1996), suggests. So, let us view software development from a different perspective. The notion of “writing software” sounds rational, so I would like to explore it. Software is a set of instructions for computer hardware. Writing software resembles instructing someone in how to perform a task. Teaching isn’t an engineering task, even when we teach engineering. However, we can easily see the results of teaching computer hardware design because, unlike human beings, hardware is deterministic. With persistence, it is easier and cheaper to get computer hardware to complete tasks for us than to try having a human follow our instructions— at least without compensation. Despite producing the “correct” result, the instructions are not necessarily correct or effective—as all software developers know.
… AND IS NOT
TOO MANY REINVENTIONS Most employers select software developers based on how well they solve a set of known problems, with little emphasis placed on whether they can teach or learn well. Nor do employers pay attention to how candidates approach or solve problems. Consider the scenario in which a new hire must resolve a problem. Often, this person does not understand the real issue. But this is software, so it is easy to work around the problem. Further, while software is a set of instructions, there is no canonical set as such. Rewriting or reinventing the solution any way the new hire sees fit is easy. This isn’t a bad approach—if the reinvention is better than the old soluContinued on page 110
THE PROFESSION Continued from page 112 tion. However, in most cases, the problem only gets shifted to a different place. Now the whole process repeats with the next replacement. So the problem gets re-solved repeatedly. After a few iterations, nobody knows how the whole system works. Ultimately, the amount of maintenance work is much greater and more complex than the volume of new work. So hiring people who can understand the existing software and work with it becomes crucial. Yet seldom is this effort recognized and rewarded.
QUALITY AND ITS CONSEQUENCES To quote political humorist Molly Ivins (Nothin’ But Good Times Ahead, Vintage, 1994, p. 19), “When corruption becomes the norm, no one recognizes it as corruption.” This applies to software if we substitute mediocrity for corruption. It is almost impossible to get software products perfect, without problems or bugs and while including every feature imaginable. That would take too long, and corporations might miss their market window. So a company must strike a wise balance between how long product development should last and its confidence in its product’s quality. Given current market conditions, the time to market appears more critical than product quality. This approach brings an influx of too many products that sport numerous bells and whistles yet accomplish little. After encountering several of these products, customers lower their expectations. Because inferior quality has become the norm, customers become accustomed to mediocrity.
THE CUSTOMER’S ROLE IN QUALITY AND PRICE Blaming the software industry for lower quality is easy. But customers also play a role. Although companies might tout their innovations and technological breakthroughs, most focus on making profits. If customers pay for a product that isn’t up to par, vendors have less incentive to adhere to high quality standards. On the other hand, if customers are discerning 110
Computer
about selecting products, vendors must be more careful about the products they bring to market. For example, one large corporation paid a huge sum for a product that they had already rejected earlier. The vendor changed the product’s brand name but left the actual software intact. Predictably, end users encountered exactly the same problems that led to the product being rejected the first time. Executives furiously demanded that the vendor fix the problems.
Software management is even less an engineering activity than software development. Yet in this case the fault lies with the executives who decided to buy the product. They could have avoided the situation if they had attempted to evaluate it instead of being seduced by the brand name. They either didn’t have the proper discretion or failed to use it. After observing several similar instances, I remarked to my friends that “I wouldn’t trust our customers’ discretion if my life depended on it.”
MANAGEMENT Software management is even less an engineering activity than software development. Analysts have noted there are no genuine metrics to objectively evaluate a software manager’s performance. In my experience, a project eventually completes, but rarely as planned. Occasionally, a brilliant or dedicated software engineer comes along, accidentally recruited or dissatisfied with the project’s general progress, and completes it. In most cases, management takes credit for the fortuitous accident and reaps the rewards. Although software development is more about people working together than about processes, there is rampant preaching about the need for processes,
procedures, and methodologies. Yet these are of little use if we forget that we are working with people, some of whom have different agendas. Since software development is viewed as an engineering activity, developers place an undue emphasis on numbers. Yet numbers can only be used for extrapolation. Nor are numbers themselves very dependable. We don’t seem to be good at collecting the right numbers or making the right use of them. Defendit numerus—there is safety in numbers—is the maxim of the foolish; deperdit numerus—there is ruin in numbers—is the maxim of the wise. Unfortunately, we seem to be addicted to numbers. Most estimates I’ve seen appear to be naïve exercises in simple arithmetic with randomly chosen numbers, and developers repeat these exercises even though earlier estimates have proven way off the mark.
IS THE REAL PROBLEM SOLVED? Software developers tend to do things because they can be done, not because they solve the problem. Doing things with software is easier than doing them with hardware, which encourages developers to make changes to obtain expected results, without comprehending why they couldn’t obtain the expected results in the first place. This tends to mask the real problem, which will later rear its head in a still uglier and more difficult form. The same behavior percolates up through management because in most cases the managers previously were software developers. One thing management can do is enforce processes and methodologies—so they do just that. However, they never address the underlying problem. For example, if estimates show a project will take too long, managers tend to add more people to the team, despite evidence that doing so only delays projects.
ARE WE BECOMING EXPERTS? As many fields do, ours often relies on experts, and thus we must understand what it means to be one. This is especially crucial for software managers. Genuine experts have the abil-
ity to learn from mistakes. Yet if we are capable of learning, few people care what we know, whether we’re engineers or managers. The frequency of schedule slips calls into question managers’ expertise. Such slips would be understandable when tackling new projects or technology— yet these delays occur very often, which suggests that schedules might be inconsequential. However, I would expect experts to be forthcoming about schedules and their irrelevance. Experts also use discretion and make judgment calls in the absence of complete information. In most cases, for reasonably sized projects, gathering an exhaustive store of pertinent information is impossible. Experts are invaluable in such cases. The predictive capability of managers is, however, often worse than that of TV weather forecasters. The lack of discretion and judgment plays out disastrously in tightly scheduled environments. I’ve seen several instances where managers—in an attempt to follow instituted processes and methodologies—refused to let developers rectify a problem, only to encounter the same problem later on. They essentially let defective products out the door—a particularly unacceptable practice when the solution is already known.
IT ISN’T ALL BAD Despite these issues, in some ways the software industry is thriving, perhaps because of the abundant hardware resources available. The accelerating decrease in hardware prices, combined with plummeting human resource costs thanks to offshoring, power this trend. Taking advantage of existing abundance is wise. However, relying on abundance isn’t a good strategy. To improve software development and management, we must encourage teaching and learning among software developers. At the same time, managers, supervisors, and project leads must identify unnecessary reinventions and address them appropriately. Recognizing and rewarding learning
capabilities—and consequently reusing existing software—is also necessary. We must stop rewarding the contrived creativity that leads to too many reinventions.
D
isciplining customers is difficult, but it would help if they could do proper homework and obtain only software that meets their real needs, instead of letting hype determine their product choices. Likewise, instead of buying based on numbers, we should examine what the numbers really mean and how they were obtained. We must realize that discrete use of numbers can help management, but numbers alone cannot run management. Nor will automated managers be likely to get us out of critical situations. Managers must realize that people are much more important than processes and methodologies. We must also put people with the right type of discretion in the right places, especially in roles such as project leads and managers. This applies
to other fields but is more critical in software development because it is easier to hide problems in software than in hardware. Hence, it can take longer to realize the real problems in software development. Last but not least, we must all become experts who learn from our mistakes and at least try to avoid repeating them. We must also kick our addiction to numbers and make better judgments based on incomplete information. ■ Raghavendra Rao Loka is a principal engineer at Synopsys in Mountain View, California. Contact him at rao@acm. org. Some of the sources used in this essay are cited at www.comp.utas.edu. au/users/nholmes/prfsn.
Editor: Neville Holmes, School of Computing, University of Tasmania;
[email protected]. Links to further material are at www.comp.utas. edu.au/users/nholmes/prfsn.
Get access to individual IEEE Computer Society documents online. More than 100,000 articles and conference papers available!
$9US per article for members $19US for nonmembers www.computer.org/publications/dlib
February 2007
111
