A PROJECT PLANHING AND DEVELOPMENT PROCESS FOR SMALL TEAMS
V Marc Rettig and Gary Simons A good manager is a remover o f obstacles and a provider o f r e s o u r c e s . his phrase circulates through m a n a g e m e n t circles in various versions. It normally conjures up visions of managers resolving crises during projects, providing people and equipment as they are needed. But the work of removing obstacles and providing resources should start before a project even begins. A good m a n a g e r can remove m a n y obstacles before they appear by providing one of the most fundamental resources any project can have: a dearly defined plan and process definition. The plan tells team m e m b e r s what needs to be done and when, while the process definition tells them how it should be done and how to know when it has been done with adequate quality. This leads to another aphorism: "A good m a n a g e r makes sure that all team members know what they are supposed to do, and how to tell whether they have done a good job." W i t h a good process definition in hand, the project manager becomes a shepherd or facilitator, seeing to it that the team has what it needs and ensuring that the project is progressing as it should. This article documents a process for software development designed and successfully used by the Academic Computing Department of the Summer Institute of Linguistics (SIL) in Dallas, Texas. Standing at the start of a complex five-year development project, the department realized that seat-of-the-pants management techniques would not be successful. We decided to organize ourselves as a "structured open team," and to adopt some of the techniques promoted by the total quality movement.
Organization T h e goal of this article is to describe our process in enough detail that readers can borrow from it to define or improve their own development process. T h e process embodies a few new ideas, a few twists on old ideas, and m a n y timehonored and often-repeated ideas. In the interest of completeness we have described everything, even the old notions, recognizing that m a n y project managers have never had any m a n a g e m e n t training. T h e bulk of this article describes the development process outlined in the sidebar, addressing both the ' h a r d ' technical issues and the 'soft' social aspects of our work. Limits in space and the reader's tolerance for t e d i u m prevent us from elaborating on every point. Since the headings in the article correspond to points in the outline, it should be easy to keep track of the discussion's context. T h e process description is preceded by background on our project, to give context, and by a description of our team organization.1 This article presents a snapshot of an evolving method as it is used at SIL. This is not a formal method, and it is not guaranteed to be complete and coherent. Readers should feel free to borrow and adjust wherever they see fit without offending the methodology gods.
The Academic C o m p u t i n g Department at S I L is creating an objectoriented software development environment called C E L L A R [15], w h i c h f a c i l i t a t e s c o n s t r u c t i o n of applications that deal with large collections of structured multilingual information. This is R & D work: "research" in that we do not always know how to accomplish our goals, "development" in that we are committed to delivering q u a l i t y s o f t w a r e to o u r u s e r community. Now that C E L L A R is nearing completion, we are using it to prototype a " P e r f o r m a n c e S h e l l " - - a tool for building performance support systems [11, 14]. O t h e r departments in our organization are forming teams to build C E L L A R applications. These efforts e x p a n d e d our focus to include applications development in addition to systems development, motivating m a n y revisions in our development process. This article describes the result: a process designed to work for many kinds of projects and many kinds of p e o p l e - - s o far we are pleased with its success. Project Teams and Guidance Teams O u r current a r r a n g e m e n t of a project team, team leader, and guidance team follows the approach suggested in Peter Scholtes's Tam Handbook [16]. Every project is carried out by a project team u n d e r the watchful eye and helpful hands of a guidance team. The guidance team. Each project team is initially formed by, and then remains responsible to, a guidance team. T h e guidance team is m a d e up of at least three managers who must sign off on the results produced by the project team, and who have the clout to remove obstacles and provide resources at the corporate level. T h e y do not manage the project t e a m - - t h a t is the role of the team leader. Rather,
C O M M U N I C A T I O H S O P T H l l A C M O c t o b e r 1993/VoL36, No,10
as
Project
Organization
they oversee the work, remove obstacles, and serve as an interface between the project 'team and the result of the organization. T h a n k s to the guidance team, the project team can forget about corporate politics and concentrate on their work. T h e guidance team has the following responsibilities: • Identify project goals • Prepare a mission statement (by writing a first draft o f the project b r i e f - s e e the following) • Select and assign the project team members # Determine other resources n e e d e d by the project team, and provide them # Monitor the progress o f the project team • "Sign off" on the results p r o d u c e d by the project team • Provide accountability to the rest of the organization for the work o f the project team
The project team. This team is m a d e u p o f the people who will do the work. O u r project teams were originally inspired by Larry Constantine's notion o f "structured open teams" [4, 1 2 ] - - g r o u p s o f at least three people acting as peers and making decisions by consensus. As the team m a t u r e d and people a d o p t e d roles befitting their personal strengths, we evolved aspects o f F r e d Brooks's "surgical team" organization [2]. T h a t is, the lead designer spent less time coding and m o r e time designing, as o t h e r people learned e n o u g h to take over the coding b u r d e n and s u p p o r t the designer in building his vision. The Project-Level
Process
Establish t h e P r o j e c t
As the outline in the "PADRE Project Planning and Development Process" sidebar shows, "establishing" a project means f o r m i n g teams and writing clear goals. T h e latter takes the form o f a "project brief," a short d o c u m e n t that sets forth the charter for the Principles Where the artil:le focuses on process, these comments senre to extract the principles that underlie the recommendations in the accompanying text. Some principles appear because they motivated features of the process. Others appear because they describe important lessons we have learned along the way.
46
October 1993/Vol.36, No.10 C O M M U N I C A T I O N S
and M a n a g e m e n t
project team. O f course, it may be necessary to write separate requirements documents that lead to and s u p p o r t the project brief. Requirements specification. In o u r c u r r e n t projects, and hence in the process described in this article, we have chosen to evolve requirements rather than attempt to complete a formal requirements statement at the start. We believe that requirements for large, complex systems are best evolved t h r o u g h an iterative process. Eric R a y m o n d explains this in The
New Hacker's Dictionary: creationism n. The (false) belief that
large, innovative designs can be completely specified in advance and then painlessly magicked out of the void by the normal efforts of a team of normally talented programmers. In fact, experience has shown repeatedly that good designs arise only from evolutionary, exploratory interaction between one (or at most a handful of) exceptionally able designer(s) and an active user population--and that the first try at a big new idea is always wrong. Unfortunately, because these truths don't fit the planning models beloved of management, they are generally ignored. In o u r project, end users motivate the effort, but they have never seen anything comparable to what we are building. Until we had a working p r o t o t y p e - - t h e only specification that communicates a d e q u a t e l y - - i t was difficult for us to teach the language o f o u r innovation to others and begin incorporating their ideas into subsequent iterations o f the specification. 2 Rapid prototyping tools let us get user i n p u t early and often, "growing" o u r software in a series o f stages. Each stage produces a working product, each makes use o f the one before. T h e decision whether or not to work from careful specifications is a strategic decision that should be m a d e early in the project. A n d it should be m a d e consciously. Many projects write (or fail to write) specifications simply out o f the habit o f following the same creationist (or evolutionary) m a n a g e m e n t model for every project, regardless o f its nature. 2When a system seems too complex to specify, build a rapid prototype [31.
OF THE ACM
Project brief. So, projects in o u r organization typically start with an informal but detailed statement o f known requirements, summarized in a project brief. T h e brief has four parts:
Project title. T h e name the entire organization will use to describe the project. Purpose. A concise statement o f the specific goals o f the project. This serves as a one- or t w o - p a r a g r a p h requirements statement for the project stated in measurable terms. T h e p u r p o s e statement may refer to a separate user requirements document for a fuller description. Stages. A n u m b e r e d list o f the develo p m e n t stages the project will go t h r o u g h on its way to completion. T h e stage descriptions should include a schedule o f completion dates. A detailed discussion o f stages appears later in this article. Team. T h e people assigned to the project team, with role assignments. Form a project team. T h e first several weeks of o u r team's existence were dedicated to planning and process designing. 3 T h e project leader (Gary Simons) was assisted in this work by Dave O'Bar, a successful total quality m a n a g e r who had recently come to SIL from the aerospace industry. This pairing o f an experienced software engineer with an experienced m a n a g e r was a good combination for facing the task o f figuring out how to manage a large project. T h e y d r a f t e d a process, then p u t it t h r o u g h a series o f refinements: the team m a d e suggestions on paper, we had meetings to discuss changes, and finally, a g r e e d we could all work together in the m a n n e r described by the process. 4 In fact, everyone seemed excited to see how it would work. T h e results o f this early work included much m o r e than the kind o f description included in this article.
3Teams should create standard development processes for themselves, and describe them in detail. This is part of what it means to form a team. flnclude the entire team in creating the development process, thus building a communal sense of ownership for both the project and the process.
We worked together to produce detailed standards for coding, testing, and configuration management; we agreed on meeting conventions, created metrics gathering forms and time sheets: a n a we set u p a g r o u p archive with an electronic index. This collection o f forms, procedures, conventions, and unwritten guidefines developed over the next few months (and which are constantly evolving to accommodate new people and new situations) is the full development process, of which this article describes only an essential shell. 5 These standards relate to the adage in our introduction: they tell people how to work. and they define in specifics the meaning o f 'quality.' Roles in the project team. T h e particular roles filled by project team members vary according to the project. A systems d e v e l o p m e n t project might involve a lead designer and two p r o g r a m m e r s . We have begun to involve people from user d e p a r t ments in application teams, and are having encouraging results with assigning explicit roles to members of these "mixed teams." At a minimum. sir your process Isn't changing, if it isn't a subject of discussion and debate, It probably isn't being used. A good process Is organic, embodied in the habits and conversations of the team. Like any behavior, you can document it, you can do your best to guide its development, but attempts to enforce it by strict mandate are more likely to encourage rebellion than participation.
The PADRE Project Planning and Development Process Project-level process 1. Establish t h e project
The organization decides to do the project forms a guidance team The guidance team writes a project brief forms a project team 2. Plan t h e project
The project team breaks the project into stages establishes standards and procedures for dOing quality work gets approval for plans from guidance team (revising as necessary) 3. Do t h e project
The project team follOWS the stage-level process for each stage in the project plan The guidance team removes obstacles and provides resources as needed 4. Evaluate the project Both the guidance team and the project team lOOk for ways to improve the project's product improve the project's plan Improve the project's process
Stage-level process 1. Plan t h e stage
The project team breaks the stage into modules assigns each module to a team member gets approval for Plans from the guidance team (revising as necessary) 2. DO t h e stage
The project team follows the module-level process for each module in the stage plan revises product of stage following review from customers The team leader removes obstacles and provides resources as needed keeps track of progress on the stage progress chart 3. Evaluate t h e stage
PADRE and PDCA At each level, the process involves steps called 'Plan,' 'Do,' and 'Evaluate.' Each level also includes external approval for the resuits of the planning step, and itersUve review and revision of the results of the 'Do' step. Putting it all together gives a five-lstter acronym, PADRE. PADRE is similar to the total quality movement's PDCA (Plan-Do-Check-Act) cycle, originally applied to manufacturing situe. tions [1]. In our experience PADRE better expresses the development cycle in a teambased approach, where what is planned by a few should be approved by the rest of the team, and what Is done by a few should be reviewed by the team. Note Where this outline uses 'programmer' and 'code,' it could just as well substitute other words suitable to other kinds of modules, such as 'documentor' and 'write' or 'interface specialist' and 'prototype.'
The project team looks for ways to improve the stage's product improve the stage's plan improve the project's process
Module-level process 1. Plan the module
The programmer (or the lead designer] develops a detailed implementation design develops a detailed testing design gets approval for design from the project team (revising as necessary) 2. Do t h e module
The programmer follows the detailed design to code the module keeps track Of progress on the module progress chart revises Implementation following review by project team The team leader removes ObStacles and provides resources as needed 3. Evaluate t h e module
The team leader and the programmer look for ways to Improve the module's I~roduct improve the stage's plan improve the project's process
COMMUNIf~ATiOHSOI=THII I t e m October 1993/Vol.36, No.10
47
T a c k U n g C o m p l e x i t y W i t h an Iteratlve Strategy omplex systems are difficult to specify. You never get them r ght the first time, and you rarely get them right the second time. Requirements for such systems notoriously change throughout development. One way to deal with these troublesome facts of life is to adopt an iterative strategy for software development. We have adopted just such a strategy, bringing five weapons to bear on the problem of software complexity. • Divide a n d conquer. We plan in minute detail, breaking the project into twoweek chunks of work, which we run through a mini product life cycle. • A spiral' life cycle (see Figure A). A fundamental strategy that we seek to follOW in our project is to have every implementation module result in a working system with added functionality. Rather than performing a large testing and integration step at the end of each stage, we include testing and integration at the end of each two-week module. The implementer includes the new code into the base code and makes the new version available to the rest of the team. At each level of planning we set aside time for user or peer review. • Object tecllnology, which we believe lends itself well tO iteratlve development. In ,our experience, object-oriented analysis, design, and programming facilitate the strategies described in the following t w o bullet points. In particular, we are using Smalltalk. • Rapid p r o t o W p l n g tOOlS. Software such as HyperCard and Authorware lets us quickly mock up several alternative designs. Prototyplng and user testing helps designers', express their ideas to users, and gives users something concrete to respond 1:o. • Bite-sized I n c r e m e n t a l testing [131. Each mOdule is fully tested before the resuits are released to the team: programmers write automated test scripts which exercise their new code, then execute all the scripts for the entire system. Since this usually Involves no more than a day or t w o spent writing test procedul~es, and since test plans and test code are both subjected to peer review, we believe people are more likely to do a thorough job than they would in an extended test phase. Since every module includes a validation of the entire system, we catch many side effects of new code before they are released to the rest of the team.
an application project team consists of three people uniquely filling the following three roles:
Implementer.
A p r o g r a m m e r who writes the program code that implements the project Domain specialist. A specialist in the application domain, who need not know much at all about programming, who serves to specify requirements and review results Technical reviewer. A p r o g r a m m e r other than the i m p l e m e n t e r who serves to review technical aspects of the implementation whenever n e e d e d - - a source of design ideas, consultation, and peer review A large project team of five to seven members could have two or three people designated for each role. Even though a single person might have the skills to fill more than one of the preceding roles, this person should be identified as filling a single role on the team. 6 We assume the three core team members are resident in the same location and are available to meet together once a week. Many teams, however, will be well served by including members who are at different locations and not available to participate in regular team meetings. These members are designated as filling the role of:
Consultant. A specialist in the application d o m a i n who participates in the team from a distance, or intermittently at the request of the core team. Evaluate
F i g u r e JL A s p i r a l l i f e c y c l e
0
Unlike the potentially n u m e r o u s reviewers who will evaluate the work of the team at the e n d of each stage (described later), a consultant is a full m e m b e r of the team who is kept informed of the activity d u r i n g each stage, and has the opportunity to review the team's work as it is developing. It is up to each team to work out how they will incorporate consultants into their work. Social roles. T h e roles just described are all technical roles. Constantine [4, 8] and many others describe another set of roles necessary SExplicitly designate the role of each member of the team, and make sure that each team has a person designated for each key role.
48
October 19cP3/Vo].36, N O . | 0 C O M M U N I ( : A T I O N S O I = l r N B A ¢ I I
Pro|ect Organization qnd Management 0 ~
for effective communication in a project team, noting that the social interaction within a team is a commonly overlooked cause o f trouble for a project. Early in o u r team's formation, we explicitly assigned roles such as facilitator, critic, and scribe at the beginning of each meeting, rotating the assignments a m o n g the team members [12]. 7 With time, these roles either become habits or do not, and role assignments now tend to be less formal: "I see Larry is taking notes--will you send us all a copy?" Still, we believe attention to interpersonal communication was an important ingredient in the early success of o u r team. The team leader. Each project team has one m e m b e r designated as the team leader. This could be one of the persons already slotted for one o f the three positions previously mentioned, it could also be someone appointed solely to fill the position o f leader. T h e role o f team leader is not synonymous with guiding light or chief decision maker; a consensus a p p r o a c h to decision making should be followed within the teams [5-7, 16]. Rather, being team leader means taking on the following responsibilities:
Facilitator. Schedule and lead team meetings, or nominate another team m e m b e r to do so. Archivist. Take minutes o f team meetings and maintain an archive o f project documents, or nominate another team m e m b e r to do so [8, 12]. Manager. Maintain a chart o f progress toward project milestones to give the project team feedback on progress toward goals, to assist the project team in planning its activities, and to provide accountability to the guidance team and others outside the project team. Contact. Serve as the contact point between the project team and the guidance team. T h e day-to-day details o f a project leader's life (and that of all the team members) will vary d e p e n d i n g on individual strengths and personalities. T h e roles we have described are 7Buildeffectivecommunicationinto your process--traln team members in team dynamics and effective meeting techniques.
essential. T h e i r assignment to a particular person on the team may vary (with the exception o f the "contact" role). Plan the Project. Planning r i g h t - taking time to do a t h o r o u g h j o b before j u m p i n g into the w o r k - - ~ a k e s the rest of project m a n a g e m e n t far more effective and pleasant. T h e r e fore, the rest of this article is almost entirely about planning, saying very little about the daily life and responsibilities of a project manager. Divide the project into stages. A project is usually too big to plan in detail all at once, so planning progresses t h r o u g h a series o f top-down steps. Complex projects may involve several semiautonomous components, which may be p l a n n e d independently. I f the work on any component is expected to take a long time, it should be divided into stages. Each stage lasts four to six months (adjust to suit the situation), and each produces a working subset o f the project's results. A stage is defined in terms o f the resulting behavior and appearance o f its product, and the information structures that underlie it. T h e project team plans its work by identifying a succession of stages from most basic to most advanced (in other words, the team performs requirements analysis and general system design at this point in the process). I f a project involves a n u m b e r of tools that work with the same data, each could be relegated to a separate stage. I f implementing all the features in a tool seems too big a task for one stage, divide the functionality into subsets and assign each set to a separate stage. Stages go through phases. Each comp o n e n t goes t h r o u g h its own software life cycle. We use the following four generic phases:
Mockup.
Build a quick-and-dirty mockup o f the application to get quick feedback from potential end users. O u r team is using Macromedia's Authorware for this purpose. O t h e r p e o p l e r e c o m m e n d anything from overhead transparencies to H y p e r C a r d applications. Prototype. T h e mockup (with changes called for by reviewers) becomes a functional specification for a
team o f p r o g r a m m e r s who develop a fully functional prototype. Refinement. T h e prototype is successively refined as end users evaluate it and suggest changes. Maintenance. After the refined program has been officially released as version 1.0, it is maintained in response to user requests for service. O t h e r systems o f life cycle phases could just as well be used. For some components--documentation, for e x a m p l e - - ' p r o t o t y p e ' will mean something very different than it would for p r o g r a m code. We typically skip the m o c k u p phase for stages that p r o d u c e internal system enhancements. Your life cycle should suit the nature o f your efforts. In o u r case, since we are iteratively developing a large system with test releases every six months, the maintenance phase is just rolled into the next stage. T h a t is, bug reports from users become part o f the requirements for the next stage, and there is no specific planning or staffing for maintenance. This is likely to change when version 1 o f the software ships to the field, a fact that illustrates the notion o f process refinement. Since the process is a living, working document, we are free to change it to acc o m m o d a t e new circumstances. At this point, the team has decomposed the project by function and by time. T h e y can create the axes o f the planning chart shown in Figure 1, a framework for the next level o f detail. T h e chart accompanies the project brief, which describes the functional boundaries o f each component, and assigns beginning and e n d i n g dates to the stages. Complex projects may require more detailed descriptions of the functional decomposition. (In practice, things d o n ' t always divide tidily into components. We sometimes define stages to i m p l e m e n t a set o f enhancements or user requirements, possibly scattered across several components.)
The Stage-Level Process Plan the Stage O u r entire process is based on the observation that small projects are easier to manage than big projects. Planning proceeds top-down until the work has been divided into many
COMMUNI~mTIONS Om'lrHI a C M October 1993/Vol.36, No.10 4 9
Project Organizatinn and Management
small pieces called 'modules.' Each team m e m b e r uses the module-level process definition (described later) to manage the progress o f his or her c u r r e n t module, while the m a n a g e r guides the progress o f the whole series o f modules toward the final goal. 8 We have experienced many benefits to this approach, a m o n g them: Plans are m o r e accurate than they were before we started working this way. It is easier to estimate how much one person can accomplish in two weeks than it is to estimate how much five people can do in six months. • It is easier to track progress. A prog r a m m e r who is 5 days t h r o u g h a 10day module can easily tell if the schedule is slipping and either apply more effort o r notify the m a n a g e r so d e p e n d e n t modules can be rescheduled. O n any day you can tell at a glance where the project stands. This is in strong contrast to the uncomfortable mysteries o f tracking a monolithic 12-month project. • We can often resolve problems before they affect the whole project. U n e x p e c t e d problems are sure to arise, and questions like, "What was the cause, exactly?" and, "How long will the delay be?" are notoriously difficult to answer for long projects. But a d e p e n d e n c y chart o f small modules can quickly reveal the longterm effect o f a delay in any single m o d u l e and help managers plan how to adjust. • Developers get m o r e "bones," "biscuits," "herring," "six-packs," . . . . T h a t is, it':s nice for everyone involved to see the progress they are making. Ew:ry couple o f weeks or so, each person gets the satisfaction o f completing some useful bit o f work and releasing it to the rest o f the team. This builds m o m e n t u m a n d morale, two things that can suffer over the long haul. •
At the end o f each stage (every four to six months in o u r d e p a r t ment), the project team summarizes the results o f the j u s t completed stage, plans the details for the next stage, and publishes both in a written eDIvlde end conquer--break projects Into lots of bite-sized chunks. "To make a long journey, take lots of small steps."
SO October1992/Vol.36,No.10C O M M U N I C A T I O N S
status r e p o r t for the guidance team. 9 T h e detailed plan is the most difficult part o f this report, since although they know the major deadlines for the next stage, the team still has to divide the work into "bite-sized chunks" they can confidently assign and monitor. Producing the detailed plan involves breaking the stage into modules. These are not "modules" in the sense of subprograms, but simply "modules" in the sense o f "units o f work." A module should be no m o r e than about two weeks' work for one person. I n o u r object-oriented setting, typical modules might require i m p l e m e n t i n g a single object class, i m p l e m e n t i n g the application window for a tool, or writing a section o f documentation. In o u r case, the j o b o f planning a stage falls to the designer with the most experience in whatever component the stage involves. H e or she makes an inventory o f all the work that must be d o n e to accomplish the goal for the stage. Additional modules may come from user testing, b u g - r e p o r t forms, o r work d e f e r r e d from previous stages. T h a t is, planning takes into account scheduled work, surprises left over from earlier stages, and predictable maintenance tasks whose substance is unknowable in advance. T h e team members most qualified to do so g r a d e each module's difficulty as "hard," "medium," or "easy," as a guideline for estimating the time r e q u i r e d to complete them. As an additional and m o r e concrete measure, the designers estimate the key complexity factors that affect productivity. In o u r Smalltalk project, they estimate the n u m b e r o f classes and methods that will need to be created or modified in the m o d u l e ? ° I f it looks as if it will take longer than two weeks, we try to break it down into smaller modules, with the schedule taking into account dependencies between tasks.
T h e result o f this work is given to the project leader or lead designer, who uses project m a n a g e m e n t software to assign and schedule each module. This gives a detailed plan for the stage, consisting o f a PERT chart showing the dependencies between the modules, and a Gantt chart showing the schedule for each team member's assignments d u r i n g the stage. Attached to this, a description of each m o d u l e specifies the work a n d often includes preliminary design notes. This plan is circulated a m o n g the project team for review, modification, and final acceptance, a n d a copy is given to the guidance team for the same purpose. Only then does "real work" begin.n ]
9Produce a written status report at the end of each stage, however pressed for time you might be. This forces you to think about what just happened, and to plan the next stage carefully before you start working. PublishIng the report for your peers and superiors Is great for corporate communicstlon.
11Everyone agrees on a plan before anyone
OP THII ACM
The Module-Level
Process
Many o f the principles m e n t i o n e d in this article are i n c o r p o r a t e d into o u r process at the m i c r o m a n a g e m e n t l e v e l - - t h e process each p r o g r a m m e r follows to develop a module. Each m o d u l e goes t h r o u g h a minilife cycle with five milestones. T h e actual interpretation o f the milestones may differ with the phase and type o f module, b u t they essentially describe a plan-do-refine a p p r o a c h p u n c t u a t e d by p e e r reviews o f results. In general terms, the five milestones are as follows: Plan. This milestone is passed when some m e m b e r o f the project team has p r e p a r e d a plan for implementing and testing the module, and submitted the plan for review by o t h e r m e m b e r s o f the project team (the l°A few metrics ere better than none, and almost as good as a lot of metrics. The trick Is to measure the right things and measure them consistently. Early In the prolect we kept e lot of statistics so we could learn to estimate the number of classes and methods needed for a module, and to convert those numbers to effort estimates. Later In the project, when modules have been way off schedule we have run the same metrics In the post mortem, usually finding that programmer productivity rates have remained constant; we just grossly underestlmsted complexity.
begins. Our team has decided to make declslons by consensus, which means we work together to find mutually acceptable solutions before implementing any plans. This isn't always easy, but group consensusbuilding techniques help [5-7, 16].
Database
FUNCTION Query Tool
Reports
Stage 1 LU
I--
Figure 1. The top part of a sample project overview. The h o r i z o n t a l axis shows h o w the work has been d i v i d e d i n t o f u n c t i o n a l c o m p o n e n t s . The v e r t i c a l axis shows how the work has been d i v i d e d i n t o t e m p o r a l stages. The boxes s h o w h o w each c o m p o n e n t ' s life cycle phases map o n t o the t e m p o r a l stages.
Stage 2
Stage 3
Peer review of detailed design and preliminary test plan
Peer review of finished code and test procedures
\
I Design
200/0
[] /
\
I
[]
I
Approve/Code
,oo o /
/50/0/400/o
Design documentation Preliminary test p l a n
Meeting minutes Reviseddesign Revised test plan
bigger the team gets, the less practical it becomes to have the whole team participate in reviews). Approve. This milestone is passed when the whole project team has worked t h r o u g h the initial plan and has reached a g r e e m e n t on a revised plan that is ready to be given to the implementer. Do. This milestone is passed when the i m p l e m e n t e r has completed an initial i m p l e m e n t a t i o n - - f o r example, working code and test procedures for a p r o g r a m m i n g module, or a completed first draft for a writing module. Review and Revise. This milestone is passed when the i m p l e m e n t e r has revised the code to the satisfaction o f the project team, and the a p p r o v e d test procedures have r u n successfully. T h a t is, other members o f the team have reviewed the work o f the implementer, communicated any suggestions or concerns, and worked out remaining issues in a project meeting. T h e i m p l e m e n t e r has then revised the work to the team's satisfaction. In a particularly troublesome module, further iterations of the review-and-revise cycle might be needed. Often the work is a p p r o v e d
I
Untested code Detailed test plan Test code and data
Meeting minutes Revisedc o d e Revisedtests
as submitted, with no revision necessary. Evaluate. W h e n the module is completed, the team leader and the imp l e m e n t e r evaluate the work, the plan, and the process to look for opportunities for i m p r o v e m e n t in the future. For example, they may decide on new features or design elements, change subsequent module descriptions or assignments, or modify the way meetings are conducted. We know this must be a good process because the names o f the five milestones make an acronym that spells a real word, and a religious one at that: PADRE. In m a n a g e m e n t discussions, we sometimes talk about "PADREs all the way down," meaning that whether we are m a n a g i n g the project as a whole, managing the day-to-day routine o f an individual p r o g r a m m e r , or any of the levels in between, we try to go t h r o u g h a cycle of plan, approve, do, review and revise, and evaluate. W h a t we are describing is known in software engineering as a "spiral life cycle," as o p p o s e d to the traditional waterfall model o f software development. T h e project's develop-
Revised,tested code Revisedtest code Bug fix reports New release of software
Figure 2. Once around the spiral. This r e p r e s e n t s t h e w o r k typically s p e n t in c o m p l e t i n g a single c o d i n g m o d u l e , i n c l u d i n g perc e n t a g e s o f t o t a l e f f o r t f o r each m i l e s t o n e and t h e d o c u m e n t s created o r u p d a t e d . m e n t could be viewed as a large n u m b e r o f turns t h r o u g h a spiral, each bringing the team closer to its goal (see the sidebar entitled, "Tackling Complexity With an Iterative Strategy"). Figure 2 shows the development o f a single module in detail, providing a m a p o f a team member's work d u r i n g the three days to two weeks it takes to complete a typical module. Notice that we have DACRR here instead o f PADRE, since for a coding module it is m o r e natural to speak o f planning as designing and doing as coding. Review and revision are split out as separate milestones to show how that step is initiated by a peer review session, after which the prog r a m m e r makes any changes requested d u r i n g the review, executes the test procedures, and revises the code until it passes all the tests. ]2 Finally, evaluation is left off the time line completely, since it is usu-
¢OMMUHICATIQHSOIITHll
AC~M
October 1993/Vol,36, No.10 Sill
Original Plan
Milestone
Revised Plan
Actual Date
1. Begin F i g u r e 3. A " m o d u l e p l a n n i n g
2. Design ready for review
s h e e t " used by p r o g r a m m e r s t o plan and t r a c k t h e p r o g r e s s o f a m o d u l e o f w o r k . Steps 2 t h r o u g h 5 c o r r e s p o n d t o "PADR" in t h e P A D R E process.
3. Design revision complete 4. Code ready for review 5. Code revision complete
F i g u r e 4. ,An e x a m p l e o f a m i l e stone progress chart
Milestone % Effort
6. Integration complete
Designed 20%
Approved 5%
Coded 40%
Reviewed 15%
Revised 20%
Est. Effort
100% 100% 100% 100% 100%
100% 100% 100%
100% 100% 50%
100% 100%
100% 50%
3 days 5 days 7 days 10 days 3 days 3 days 7 days 10 days
% Done
Modules
Select prototype tool Tree editor Nested box editor Graph layout editor Graph overview Zoom and pan Paint rnult, views Data maps Total estimated effort Total % done
48 days 24
ally done by the team leader rather than the programmer. Evaluation is taking place t h r o u g h o u t the stage as modules are completed, with adjustments to plan or process whenever scheduling goals are not met. The percentages shown in Figure 2 are typical breakdowns that we f o u n d in our project. T h e r e are frequent surprises that cause these figures to change. But programmers record the time they spend on each step in the process, and these figures make interesting reading d u r i n g the 12plan to test. Plan to spend about one-third of your time coding, one-third of your time testing and revising, and one-third doing everything else. Or as one project leader put It, "Half of us are burning toast, the ether half are scraping." For some reason people consistently underestimate their ability to design end build things the wrong way. Because it Is much cheaper to kill bugs early, teams need to integrate testing Into all levels of their development process. [13]
52
October 1993/Vol.36, No.10
COMMUNICAYIONS
100 90 45 20 20
postmortem of a module that took significantly longer than we planned. Life in the team. O f course, all the team members are working on modules at the same time, circulating documents for review and calling for meetings. T h a t means each person not only has to complete assignments, but also read other people's design documents, participate in review meetings, and share in the other responsibilities of keeping the team organized and moving ahead. W h e n we started this process, some were worried that there would be so many meetings and d e m a n d s on their time that there would be little time left to focus on their assigned tasks. 1~ A n d in fact, we have had to take steps to cut down on such overhead: • Limit the n u m b e r of team meetings to no more than one per day. • As much as possible, limit meetings
OF THE A¢lll
to one hour. If a meeting needs more time, use the last five minutes to schedule a continuation meeting. • Use a facilitator (or at least keep the mind-set of a facilitator) to make sure action items are recorded and assigned, to draw everyone into discussions, and to make meetings productive a n d effective. • Ask each person to designate a "principal reviewer" for a review meeting. This person is honorb o u n d to read the design or code, a n d to r e t u r n comments to the author within 24 hours of receiving it. Other people are asked to partici-
13So far as possible, protect people from red tape, interruptions, and unnecessary overhead. This Isn't easy advice to follow, since It Involves everything from organlzaUonal change to facilities planning and ergonomics. Many books give good advice in this area, among them DeMarco and Uster's excellent Peopleware [10].
Project
Organization
and M a n a g e m e n t
A
l q ~
Our entire process is based on the observation that s m a l l
projects projects.
are
easier
Planning
to manage proceeds
than
big
top-down
until the work has been divided into many small modules. pate, but if they are busy they can bow out without suffering pangs o f guilt. T h e role o f principal reviewer rotates t h r o u g h the team. * Make p e e r review meetings optional. Now that the team is established, we hold review meetings at the discretion o f the a u t h o r and the principal reviewer, who may call a meeting if a module involves difficult questions or has far-reaching effects. New team members' work is reviewed in meetings as a way o f acquainting them with the team, the project, and the process. * Provide good administrative supp o r t to the team: someone to answer phones, manage the g r o u p archive, make photocopies, schedule meetings, fend off unnecessary demands, and in general grease the wheels of progress. * Keep the teams small. T h e p e e r review process works if no m o r e than four people are developing modules at once. L a r g e r teams will need to divide into smaller peer review groups. With these measures in place, we have been satisfied with o u r productivity. W h e n we plan, we assume that people will be able to spend four hours p e r day focusing on their assigned task. T h e other hours are spent in meetings, reading, and existential overhead. This sounds high until you realize that the four hours spent away from the assigned task are largely taken u p with making other people on the team product i v e - f a r more than half the day is spent f u r t h e r i n g the interests o f the team. O u r untested suspicion is that the variety of tasks imposed by this process, together with the momentum built by three or four people pushing to get modules completed, makes the team far m o r e productive than if all spent eight hours a day on
their own work. A n o t h e r advantage comes from the cross-training everyone receives by reviewing one another's work. People tend to settle into specialties, often receiving assignments that relate to the same subsystem. But because they constantly r e a d about one another's work, no one is completely in the d a r k when asked to work on an assignment outside one's specialty.
Do
the
module:
micromanaging.
W h e n a team m e m b e r starts a module, it is clear what should be done, when it is due, and how it fits into the larger picture. W h e n a team m e m b e r sits down to work each day, he or she usually knows exactly what to work on, and how much must be accomplished to stay on schedule. Team members are, in effect, managers over a small two-week project, charged with the responsibility o f h e r d i n g their assignments t h r o u g h the PADRE process. We have designed a form to help these micromanagers, called the "module planning sheet" (Figure 3). This sheetis a key tool for time mana g e m e n t and project tracking, at once a b y p r o d u c t of detailed planning a n d an enabling technology for tracking d e v e l o p m e n t ? 4 T h e first day at work on a module, a team m e m b e r fills in the "estimated days to complete" field and the "begin" date in the "original" column in the module planning sheet, taking figures from the original schedule for the stage. T h e other dates in the first column are calculated by multiplying the estimated days to complete the 14Make the process concrete In your forms-design them to reflect the expectations and dellverables specified In the process. Forms such as the module planning sheet help guide people through the process of completing a module, serving as a reminder and a tool for gathering Important Information. A good form Is an effective job aid.
work by standard percentages for each milestone (see Figure 2). T h e "revised plan" column is filled in after the team m e m b e r has progressed far e n o u g h into the module design effort to evaluate the original estimate for days to complete the module. It may now be clear that the work was originally over- or underestimated, and the plan needs to be adjusted accordingly. I f a p r o g r a m m e r thinks the m o d u l e will take longer than planned, he or she must negotiate the revised estimate with the team leader. T h e blank for "revised estimate" is then filled in, and the "revised plan" column is completed by beginning with the actual start date and using the standard milestone percentages to extrapolate the other dates. T h e new set o f target dates for each o f the milestones then becomes a measuring stick, for j u s t this m o d u l e in isolation, for answering the question, "Am I doing a good job?" By agreeing on the new estimate, the team leader is saying, "I think you have done a good j o b if you can finish the module in this many days." T h e last column gets filled in as the work on the m o d u l e proceeds, d o c u m e n t i n g the p r o g r a m m e r ' s actual progress toward completion. This sheet, together with a time sheet that lists hours spent on each step of the module, is given to the project leader when the module is finished. T h e project leader uses the information to u p d a t e the project status, u p d a t e the productivity history for use in the next r o u n d o f estimating, and evaluate what is going wrong with the process if the module has taken an excessive a m o u n t o f time. Tracking progress. Having defined the process o f project m a n a g e m e n t in such detail, we have a powerful tool for tracking the progress o f a project. F o r each stage, the project
¢OMMUNI¢&TION! OF THIi ACM October 1993/%1.36, No.10
S3
Pro|eel
Organization
leader creates a "milestone progress chart," like the one shown in Figure 4, using spreadsheet software. This chart lists the modules involved in completing the stage and shows the progress o f each module t h r o u g h the spiral. As team m e m b e r s fill out m o d u l e planning sheets, they r e p o r t their progress to the team leader, who updates the a p p r o p r i a t e progress chart. T h e chart is kept on the wall in the team's conference room, so everyone can see how things are progressing. 15 It was stated earlier that breaking big tasks into bite-sized modules makes it possible to m o n i t o r the task. Figure 4 shows a concrete example o f that concept, providing an accurate, easy to read, detailed picture o f the project's status. It also provides a tool for p l a n n i n g - - t h e spreadsheet can use the dates at the top o f the form to extrapolate to the probable finishing date given the n u m b e r o f days exp e n d e d so far and the percentage o f scheduled days that have actually been completed. T h e chart shown in Figure 4 is o u r simple version. We also have a m o r e complex version that keeps track by individual team m e m b e r , so we know not only what the schedule variance for the whole team is, but also each team member's contribution to that variance. Evaluate and Refine
At the end o f each stage, the team sets aside time to reflect on the previous few months. Did any modules take far m o r e o r less effort than planned? Why? W h a t can be done to avoid such miscalculations in the future? This makes an excellent framework for talking about problems and identifying improvements. I f a module takes too long to complete, people are less likely to place blame on the p r o g r a m m e r than they might be in other frameworks. In the spirit o f g r o u p ownership o f the project and the process, the poststage discussion lSKeep everyone Informed about the project's status. We post progress charts publicly, Including the ones that show the progress of individual team members. The simple act of posting this information builds morale, helps people feel included, and provides incentive to make progress.
S4
October 1993/Vol.36, No.10 ¢ O M I W U N I C A T I O N S
and Management
is more likely to focus on how the p r o g r a m m e r could have been better supported, how planning could have i m p r o v e d o u r original time estimates, or how the process could change to improve p e r f o r m a n c e in the future. T h e productivity metrics collected d u r i n g each m o d u l e provide objective data on which to base improvements. W h e n analyzing a troublesome m o d u l e the project leader has time sheets, complexity measures (class a n d m e t h o d counts, p e r h a p s with counts o f decision points), design documents, results o f user tests, and meeting minutes at his or h e r disposal. Additionally, the team m e m b e r responsible for the module has been trained to think not only about technology but also about process and communication. As the total-quality books suggest, accurate m e a s u r e m e n t a n d a desire to improve are two i m p o r t a n t ingredients for excellence. Summary
and True Confessions
We have described a process for planning a n d m a n a g i n g software development. O u r experience has m a d e us enthusiastic believers in the principles mentioned in this article a n d cautious believers in the specific e m b o d i m e n t o f those principles in the PADRE process. We say "cautious" as a r e m i n d e r to ourselves a n d our readers that this is only a snapshot o f something that constantly changes. We can no m o r e tell others they should r u n their projects exactly this way than we can predict how we will be r u n n i n g projects next year. T h e specifics will change to fit circumstances a n d personalities; the principles will not. During the first two stages, we were eager to know how this approach would affect o u r productivity and quality, so everyone readily contributed detailed data about his o r h e r work: metrics forms, time sheets, planning sheets, and so on. We were pleased with the results. We were m o r e productive here than on previous projects, we were improving o u r ability to plan and estimate, and the process was becoming a habit rather than an exercise. As the stages went by, it was all less new a n d exciting. We now collect far
O1U T H E
ACM
less data, having settled on a few key metrics. We are less formal in assigning roles d u r i n g meetings. We sometimes skip review meetings, having learned a little about recognizing the difference between troublesome a n d benevolent modules. For these reasons, and because the nature o f o u r work is changing as we move from systems d e v e l o p m e n t to applications, o u r process is evolving. For example, after using the m o d u l e planning sheets for a year or so, they fell into d i s u s e - - t h e team felt that the milestone progress chart was adequately meeting o u r need for tracking and accountability. Now the team leader makes the r o u n d s once every week to check each team member's progress, and enters the findings on the chart. T h e process d o c u m e n t e d here is the result o f a recent refinement, an effort to incorporate the lessons o f the past three years a n d to accommodate a b r o a d e r range o f activities. We wanted a process that would work for other teams in o u r o r g a n i z a t i o n - something we could easily teach to o t h e r managers a n d apply to new situations. T h e process continues to refine itself. ['4 References
1. Arthur, LJ. Improving Software Quality: An Insider's Guide to TQM. Wiley and Sons, New York, 1993. 2. Brooks, F.P. The Mythical Man-Month. Addison-Wesley, Reading, Mass., 1978. 3. Connell, J. and Shafer, L. Structured
Rapid Prototyping: An Evolutionary Approach to Software Development. 4.
5. 6. 7. 8. 9. 10.
Prentice-Hall, Englewood Cliffs, N.J., 1989. Constantine, L. Building structured open teams to work. In Proceedings of Software Development '91, Miller Freeman, San Francisco, Calif., 1991. Constantine, L. Decisions, decisions. Comput. Lang. (Mar. 1992). Constantine, L. Consensus and compromise. Comput. Lang. (Apr. 1992). Constantine, L. Negotiating consensus. Comput. Lang. (May 1992). Constantine, L. The lowly and exalted scribe. Comput. Lang. (June 1992). Constantine, L. Team harmony. Camput. Lang. (Sept. 1992). DeMarco, T. and Lister, T. People-
ware: Productive Projects and Teams. Dorset House, 1987. 11. Gery, G.J. Electronic Performance Sup-
port Systems. Weingarten Press, Bos-
ton, Mass., 1991. 12. Rettig, M. Software teams. Commun. ACM 33, 10 (Oct. 1990), 23-27. 13. Rettig, M. Testing made palatable. Commun. ACM 34, 5 (May 1991), 2529. 14. Rettig, M. Cooperative software, Commun. ACM 36, 4 (Apr. 1993), 2328. 15. Rettig, M., Simons, G. and Thomson, J. Extended objects. Commun. ACM, 36, 8 (Aug. 1993), 19-24. 16. Schohes, P.R. The Team Handbook. Joiner Group, Madison, Wis., 1988.
CR Categories and Subject Descriptors: K.6.1 [Management of Computing and Information Systems]: Project and People Management--training; K.6.3 [Management of Computing and Information Systems]: Software Development; K.7.2 [The Computing Profession]: Organizations General Terms: Management Additional Key Words and Phrases: Software teams
About the Authors: MARC RETTIG is a former member of the technical staff at the Summer Institute of Linguistics and currently a performance support architect at Andersen Consulting. Current research interests include supportive applications and information design. Author's Present Address: 69 West Washington Street, Chicago, IL 60602; emaih 76703.1037@ compuserve.corn GARY SIMONS is director of academic computing at the Summer Institute of Linguistics. Current research interests include object-oriented conceptual modeling and the Text Encoding Initiative. Author's Present Address: 7500 West Camp Wisdom Road, Dallas, TX 75236; email:
[email protected]
© A C M 0002-0782/93/1000-044 $1.50
1. E A S Y
December 6 - 12, 1993 Orlando Marriott International Drive, Orlando Florida The only Information Systems Security conference that gives you real solutions to real problems
~ ~ j ~ _ NEWt/
Technical
Vendor Track TIS Belcore
Stimu lating
CTA, Inc. SecureWare PRC Verdix
Brilliant Papers
Panel Discussions
and more. Controversial Great Debates
Distinguished Lecturer
Keynote Speaker
H.O. Lubbes Naval Research Laboratory
Bob Ayres Dir. of Center for Information Systems Security DISA
Advance Programs are now available. Contact one of the following:
Diana Akers Publicity Chair The MITRE Corporation 7525 Colshire Drive McLean, VA 22102 (703) 883-5907 akers @mitre org
Prize Winning PC Software! Artificial
Applications Conference
[
R a i n forests o c c u p y just 2% of t h e earth's surface. Yet, t h e s e rain forests are h o m e to half of t h e planet's tree, plant and wildlife species. Tragically, 96,000 acres of rain forest are b u r n e d e v e r y day. Y o u can h e l p s t o p this senseless destruction. Right n o w you can join The National A r b o r D a y F o u n d a t i o n , / i ~" t h e world's largest tree-planting e n v i r o n m e n t a l organization, and s u p p o r t Rain Forest Rescue to s t o p f u r t h e r burning. Y o u ' d b e t t e r call now.
Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the A C M copyright notice and the title of the publication and its date appear, and notice is give that copying is by permission of the Association for C o m p u t i n g Machinery. To copy otherwise, or to republish, requires a fee and/or specific permission.
9th Annual ComputerSecurity
Informative Tutorials
THE BURNINGMUST STOP. NOW!
Dr. Ronald Gove Conference Chairman Booz-Allen & Hamilton 8283 Greensboro Dr. McLean, VA 22102 (703) 902-5280 gover @jmb.ads.com
Intelligence
for IBM/PC
NEURAL NETWORKS tsiest way to quickly learn about is f a s c i n a t i n g n e w t e c h n i q u e . . . cludes a working Neural Network y o u r c l a s s c a n t r a i n $59
2. P C T H E R A P I S T I V - a n i m a t e d talking head talks thru the PC Speaker! First software to pass a limited Turing Test of conversational ability at t h e B o s t o n C o m p u t e r M u s e u m $69 B O T H $99 + I n c l u d e s 3 B I G C a t a l o g s and FREE Talking Expert System Demo! P l e a s e s p e c i f y d i s k s i z e o r we s h i p 3.5" - C h e c k , A m e r i c a n E x p r e s s , o r P . O . to: THINKING SOFTWARE, INC. 46-16 65TH PLACE , Dept 3000 WOODSIDE, N . Y . 1 1 3 7 7 PHONE (718) 803-3638 p.s. ---Professors
- make computing come alive for your students, order our fascinating T u r i n g T e s t L a b ...... o n l y $ 1 4 9 . 9 5 / 4 d i s k s 100 p a g e m a n u a l - I n f o r m a t i o n o n R e q u e s t .
Circle # 101 on Reader Service Card ¢OMMUNICATIONSOIETHIEACM October 1993/Vol.36, No.10 S 5
