The Business of Software
Phillip Armour
Closing the Learning Application Gap Attempting to overcome the challenges to effectively conveying knowledge. “Between the idea and the reality…falls the shadow” T.S. Eliot The Hollow Men
PETER HOEY
T
hree of the most feared words in the English language must surely be “some assembly required.” True, most objects we are called upon to “some assemble” are accompanied by that peculiar collection of text and graphics known as instructions, but it is almost a point of honor for many engineers to reflexively ignore them. Instructions obviously pander to the lowest common assembly denominator— only when all else fails would we actually read them. A friend suggested to me that, in the licensing for engineers debate, the routine nonuse of directions could be used as a matriculation criterion. Perhaps some of this reluctance comes from a sort of professional hubris, but there are other reasons why engineers routinely decline to use instructions, and they relate to the way we learn. Engineers and developers are not, by inclination or by training, understanders of problems—they are solvers of problems. Under-
standing and solving are quite different modalities of thought and differentiate a scientist-type from an engineer-type. When given a proposition at a certain level of abstraction, the scientist’s thought process routinely goes upward to
higher levels of abstraction. The scientist says, “Hmm, I wonder which abstract domain of logical reasoning explains this behavior?” The engineers’ more practical thoughts go downward to the implementation. They think: “…how do I build this?” The difference is that to the scientist the
understanding is the goal; to the engineer the understanding is a means to an end—it is a way to create something that works. This end-focused reasoning can be so strong that I have seen engineers happily provide an answer to a problem they don’t understand. Sometimes they will simply produce an engineeringly consistent and workable answer to a totally different problem. I think some of this tendency is innate; development work attracts and retains people who are good at building things. This predisposition is reinforced by the training developers are given, training that tends to focus heavily on the detailed design and coding phases of systems. There is another consequence to this combination of disposition and practice—developers learn more from doing than from thinking. Or perhaps more correctly, their understanding of things is so integrated with the activity of building them they cannot reasonably be separated. I saw this attitude clearly displayed recently, while teaching a workshop on software configuration management (SCM). During a quite abstract discussion of the “change-set” and “long transac-
COMMUNICATIONS OF THE ACM
September 2003/Vol. 46, No. 9
27
The Business of Software tion” models of SCM, one participant’s animated involvement in the discussion consisted mostly of saying things like: “Hey, I could write a script to do that” or “If we put these relations in a database, we could manage them like this...” or “...what kind of parameters would you feed to a function that did that?...” Clearly, he wasn’t planning on banging out a program on the spot. His retreat into code-design thinking was something else. It was driven by two things: one, coding and codinglike thinking were his primary and most practiced methods for understanding things; two, by thinking about how he would build something he was able to better understand what it was he might build. I would wager that if you somehow prevented him from thinking about the problem in terms of its solution, he’d have a lot of trouble understanding the problem at all. Try this as an experiment: pick a reasonably complex construction project and read the instructions carefully a couple of times. Heck, try it three or four times. Then put the instructions down for a few hours, and try to duplicate them. What kind of job would you be able to do? Now go ahead and actually build the thing and then write out the instructions on how to build it. Compare. Most engineer-types tend not to “own” the information unless they are allowed to actually build the thing. Simply reading about it, abstractly designing it, planning it, or thinking about it just doesn’t seem to work very well. However, once 28
you allow engineers to build something, not only can they then explain what it is and how to build it, they might also be able to define a better process and can, and will, liberally criticize the existing instructions, pointing out gleefully where they are incomplete or ambiguous. Traditional Learning
Traditional generic (usually classroom-based) learning is rather like reading the instructions. It usually does not allow the student to do. The attempt to convert learning into a generic activity causes the subject to be, at least to some extent, “generified.” This is inevitable and can be useful— we abstract the key lessons and thought processes from a situation and teach people skills that they can, theoretically, employ across a range of domains. But there are also problems with this approach. The primary issue is that any abstraction removes details and specific knowledge of the application in which it might be used. But usually it is the application of the lesson the engineer is seeking. In addressing this problem, Corinne Miller, Motorola’s Director of Engineering Learning and Development, noted: “We found that while general training is necessary, it is not sufficient to create business value.” We can partition learning into a spectrum of knowledge learned ranging from very, very general to precisely specific. While it is a continuous spectrum, we can use-
September 2003/Vol. 46, No. 9 COMMUNICATIONS OF THE ACM
fully divide it up into levels. Very Generic: this covers basic human skills, such as the ability to read and write; Generic: skills that are usable across a wide range of domains as in say, presentation skills. Discipline Generic: skills that generically apply across a discipline. Most programming language training falls in this area. Applied Generic: these are skills modified by a framework in operation in a business. An example might be a particular company’s implementation of the SEI CMM assessment process. Domain Context: this is generalized knowledge and application across and engineering domain. Teaching an architecture such as CCITT’s Signaling System 7 (SS7) implementation of the ISO communication standard might be an example. Domain Specific: this would involve learning how a generalized domain is used across an area of a business. This starts to get quite specific to the company. An example could be, say, Motorola’s implementation of the Bluetooth architecture in its cellular phones. Domain Application: this level addresses the application of a particular design in a particular business scenario. Any example here would have to be specific to a particular system. Domain Implementation: this is the most detailed level, and addresses the implementation of knowledge in a specific application. In most cases, this cannot be taught, it must be learned by actu-
Learning Effectiveness
A key challenge to conveying knowledge is the learning effectiveness of the learning event or program from the perspective of the recipient. It would make very little sense for a company to employ illiterate people to build a system reasoning that, to be successful, they will have to learn how to read and write. We would rarely staff a project with people who did not know the programming language of choice for the same reason.
Generally created in-house
cific
Ap
Spe
Ge Pre neri sen c: tat ion
Do : Blu main etoo th pli ca tio Do n: ma De in vic e
Generally created externally Domain 7 Context: SS mm leco r Te d + fo plie C + Ap eric: n Ge e– + lin C + ip c: isc ri D ene G
ally doing the job. These levels overlap considerably, and specific instances may fall in one or several areas. When we look at how we might conduct a learning event and acquire the knowledge at a particular level, we see significant differences between levels. The more generic learning (Very Generic) is not usually taught in a business context; companies hire people with these skills. The next generic levels are commonly taught in discipline schools (for example, in college engineering curricula) or are developed externally to the business. As the learning becomes increasingly specific, the involvement of the business in the creation and delivery of the learning must increase, since the content references knowledge that is present only in the business. At the most detailed levels, the learning can only occur on the job, by doing the job. It cannot occur elsewhere, since the knowledge exists only in the job itself.
Ski
lls
Must be created in-house
Students must somehow navigate this E gap TH G
IN RN ION P A LE CAT GA I PL AP
Domain ntation e m le p Im
Very Generic: Reading and Writing Generally taught outside of the business environment
Equally, it would not be possible for a college to create a training program to resolve a specific programmer’s problem with a memory leak in a specific application written in a particular language running on a specific microprocessor in a certain machine control system. A training course can be created to help with generic memory-leak issues, or a generalized language, or a type of application or target processor. But we cannot create a training class just for the particular problem this person is experiencing. It is up to this engineer to learn whatever needs to be learned on his or her own, doing the job. In fact, as readers of previous columns might note, I would contend this learning is the job. Between the end of the generic learning that can be reasonably externally created and the applica-
Can only be obtained on-the-job
The learning application gap.
tion of that learning to successfully perform a function that adds value to the business lies the Learning Application Gap. The width of this gap determines the effort needed to use the skills and knowledge learned. For institutions whose function is generic learning, this is not an issue. Graduates of an engineering school are expected to know about engineering, they are not expected to know the details of a specific application of engineering. The more generic the learning is made, the wider the audience for the learning activity. Since volume is a major consideration for most dedicated learning organizations, developing a program that might be applicable to only a few people is not viable. Business-based educational resources do not have this luxury.
COMMUNICATIONS OF THE ACM
September 2003/Vol. 46, No. 9
29
The Business of Software The goal of a university is to develop smart people. The goal of a corporate learning institution is to develop smart people who can directly add value to the business. There is an extra step. Certainly there are generalized programs that must be taught to wide-ranging audiences, but often the focus needs to be on quickly changing the business practice. Volume-based training goals can be harmful to this ability, since in order to create engineering learning programs applicable to a lot of people, we must
Out of This Class. There’s nothing wrong with the Theory of Wrenches, but most engineers don’t want theory, they want a wrench. I have a problem right here in my work, so please give me a wrench (preferably the right one) and show me how to use it on this problem. Few training programs address this need, and so leave the student with the task of figuring out how the theory really might work in real life. Since the distance between the theory and the practice can be
more people with the same thing. More “effective” learning from the point of view of the learner occurs much closer to the application and this invariably means fewer people. There is no standard answer to this, but there are strategies that help: • Integrate the learning activity with the project activity, targeting the learning support at the project’s learning needs. • Provide a consistent framework with customizable components—build the learning activity
At the most detailed levels, the learning can only occur on the job, by doing the job. It cannot occur elsewhere, since the knowledge exists only in the job itself. make them more generic. Such programs inevitably end up with more breadth and less depth. I call this “Teaching the Theory of Wrenches.” The Theory of Wrenches
Training programs that teach the Theory of Wrenches have the following typical outline: 1. Introduction: Your Experience with Wrenches; 2. Why Wrenches are a Good Idea; 3. The History of Wrenches; 4. The Theory of Wrenches: Attributes of Wrenches, Taxonomy of Wrenches, Empirical Measurements of Wrenches; 5. Exercise: How to Make a Meal Using Wrenches; 6. Wrench Certification; and 7. Final Participant Activity: How I Plan To Use Wrenches When I Finally Get 30
large, this leaves the student with a lot to do. Under high pressure, attempting to figure out how to use a new technique may be difficult, and the impulse to return to the previous way of doing things is very strong. I have found this to be very true of the application of methodologies and modeling techniques in software engineering. Teaching engineers generically is equivalent to teaching someone the rules of chess and then expecting them to go and play a good game. Knowing the rules of chess and having the ability to play a good game are different skills. Some Ideas for Closing the Gap
There is a paradox here: more “efficient” training (from the supply side) implies teaching
September 2003/Vol. 46, No. 9 COMMUNICATIONS OF THE ACM
from a more generic structure with more specific elements focused on key constituencies of participants. • Include self-customized elements that allow participants to bring in and integrate their specific work elements. • Develop a range of offerings of learning components that address each of the spectrum sections, accepting that the more specific will necessarily have a narrower audience but using the return on these activities to justify the expense of creating them. • Measure effectiveness in the business rather than “efficiency” through training delivery metrics. • Provide in-project resources to convert their in-project detailed knowledge into learning resources where valuable.
CRA Conference on Grand Research Challenges in Information Security and Assurance • Provide in-project expert help to ease the burden of applying the learning. Grow Mushrooms
Often, we find projects have created their own valuable learning elements simply because someone thought it was a good idea, they received an appreciative audience, and the projects needed something to help them do the job. In a study at Motorola, literally thousands of such homegrown “educational” components were found. Sometimes, these things appear like mushrooms overnight. They arise because of the intersection of a business need and a business capability. Motorola’s Corinne Miller observed: “We found that engineering environments spontaneously created a lot of their own training elements. Our job is to encourage and enable the propagation of these ‘mushrooms’ and, where useful, to transplant them to other parts of the company.” The business of developing software is, at its core, a learning activity. The most farsighted companies are working toward a convergence of their educational/ learning resources and the projects actually doing the work, where the projects do the work, and when the projects do the work. c Phillip Armour (
[email protected]) is a vice president and senior consultant at Corvus International Inc., Deer Park, IL.
Airlie House, Warrenton, VA November 16–19, 2003 The CRA-sponsored “Grand Research Challenges in Computer Science and Engineering” conference last year was the first in a series of highly nontraditional conferences intending to define important questions rather than expose current research. Because of the clear importance and pressing needs in information security and assurance, CRA’s second Grand Challenges conference will be devoted to defining technical and social challenges in information security and assurance. We are seeking scientists, educators, business people, futurists, and others who have some vision and understanding of the big challenges (and accompanying advances) that should shape the research agenda in this field over the next few decades. We hope to convene a diverse group from a variety of fields and at all career stages; indeed, we seek insight and vision wherever it may reside. Attendance is limited to 50 people and is by invitation only. Individuals invited must commit to attending for the entire three-day conference. If you are interested in attending, please submit a two-page (or less) statement of two or three examples of a “grand research challenge” problem in the IS/IA area: Call for Papers (September 17, 2003 deadline)
© 2003 ACM 0002-0782/03/0900 $5.00
COMMUNICATIONS OF THE ACM
September 2003/Vol. 46, No. 9
31
