Experiences Using Agile Software Development for a Marketing Simulation David Mills, Linda Sherrell, Jeff Boydstun, and Guoqing Wei Department of Computer Science University of Memphis
[email protected] abandon documentation or discipline. Some detractors misinterpret agile methodologies as cowboy approaches, which throw caution to the wind. But the Agile Manifesto is explicit in its philosophy; that is, although traditional practices alone should not drive development, they are still deemed valuable. Although 40% of software development companies still use the documentation-driven waterfall model [3], a growing number of companies and academics are adopting agile practices [4, 5] due to their flexibility and the changing marketplace. Practitioners may debate whether traditional or agile approaches are universally adoptable, but a more realistic view is different projects require different methodologies, and some projects may benefit from combining traditional and agile practices [1]. This paper discusses the authors’ experiences in using agile practices to develop a marketing simulation. The next two sections describe the project requirements and the methodology selection process, respectively.
Abstract Agile software development methodologies present an enticing alternative to the heavyweight plan-driven methods that are traditionally employed. This paper describes how the authors used a home-ground chart [1] to determine which approach to apply in the development of International Market (IM), an online tool to administer shopping simulations for marketing studies. In addition, the paper gives a brief overview of eXtreme Programming (XP), the agile methodology selected by the authors, and discusses the authors’ adherence to the 13 practices of XP as outlined by Ron Jeffries [12]. This discussion should prove beneficial to those developers considering the adoption of agile practices.
1. Introduction Agile software development methodologies present an alternative to traditional methodologies. Traditional or plan-driven methodologies are generally heavyweight, documentation-driven processes. Agile practices, on the other hand, embrace change and support strong developercustomer relationships [1]. There are a number of agile methodologies, all of which share a set of common goals. These goals were formalized in the Agile Manifesto [2], a document composed in February 2001 by a representative sample of agile enthusiasts and others from the software engineering community. The following quote from the Manifesto expresses agile objectives in comparison to those of traditional software development.
2. Project Description In May 2005, planning began for a product requested by two professors in the Department of Marketing and Supply Chain Management at the University of Memphis. The task was to develop an online tool to facilitate the creation, execution, and analysis of manipulation-based shopping studies. The project was named the International Market (IM). As described by the clients, the IM project required three major components. The first component was a tool to run a single marketing study. A study was developed by creating a fictitious store with items for purchase. Once a study was initiated, users could create accounts, log on, and shop. For a particular user, a study consisted of a set of shopping trips, initially eight. On each trip the user would proceed through a collection of categories, each of which contained a set of items. The user could view a picture of the item, as well as click for information about the various products including price, country of origin, weight, and name. Also, on selected trips, each category would contain some product manipulation such as a sale price. Users would eventually select one item to purchase from each category. The study would record various elements such as the time spent on
Individuals and interactions over processes and tools. Working software over comprehensive documentation. Customer collaboration over contract negotiation. Responding to change over following a plan. That is, while there is value in the items on the right, we value the items on the left more. In other words, while agile software development stresses stakeholder interactions, quick responses to changes, and working software, agile processes do not 0-4244-0169-0/06/$20.00 © 2006 IEEE.
285
each trip/category/product, the items purchased, and the information viewed. The second component was an administrative interface to manage users and study-related information. The marketing researchers would use the interface to collect the raw data to use in statistical analysis. The third component was an additional part of the administrative interface that allowed studies to be created. In order to define a study, researchers needed to have the ability to create the following items: categories, products, and manipulations. Once all of the items were created, researchers could activate or deactivate the study. The customer team included the two marketing professors and a Ph.D. student, who was also from the Department of Marketing and Supply Chain Management. The implementation team was comprised of four developers and a research/project manager. The developers included one undergraduate and three graduate computer science students. The research/project manager was a computer science professor with research interests in software engineering.
Dynamism describes the rate at which requirements can be expected to change. Plan-driven approaches tend to work best in stable environments, while agile ones are capable of handling more dynamic situations. Culture measures the nature of the team in terms of their preference for freedom versus order. Teams that thrive on chaos, preferring more freedom, tend to work well with agile approaches. Plan-driven approaches tend to work better in an ordered environment, where members are more comfortable with clearly defined roles. As defined in [1], size refers to both the team size and the product size. Agile methods are better suited for smaller teams and projects, while plan-driven methods are scalable for large teams and projects. Criticality expresses the degree that software failures could harm the public or individual welfare. This severity ranges from highly critical where lives are at stake to low criticality that might result in individual discomfort. Traditionally, plan-driven methodologies work better with projects of high criticality, whereas agile approaches support projects with a lower criticality. Developers plot points, which correspond to values for each of the five factors, and then connect the points to complete the home-ground chart. After completion of the graph, the final step is to assess whether an agile or plandriven approach is favored. Points lying closer to the graph’s center favor agility, whereas points near the graph’s border indicate that a plan-driven approach is more appropriate. The home-ground chart for the International Market project appears in Figure 1. To complete this chart, the project was evaluated on each of the five critical factors described above. Working clockwise from the top axis: Personnel. While all of the developers had prior experience using plan-driven approaches, only two had applied an agile methodology. To familiarize the two inexperienced developers with agile practices, a small practice project, which is known as a spike, was recommended by the project director. The undergraduate student easily adapts to new situations and has led several team projects, but the project director hesitated to designate an undergraduate as a Level 2 developer. Without an additional level, the project director decided to assign this developer half 1A and half 2. The other inexperienced developer was described as a Level 1A. One of the experienced team members was designated as a Level 2, and the other member was described as half 1A and half 2 due to the requirement that a Level 2 person must feel comfortable in a management role. Therefore the value assigned to the personnel axis is 0% Level 1B and 50% Level 2. These skill levels favor an agile method of development. Dynamism. The nature of the project suggested that requirements were likely to change frequently. While the client provided a clear overview of the project, many
3. Methodology Selection Selecting a methodology is an important and challenging task, but some tools are available to aid in the decision process. In particular, Boehm and Turner [1] present a graph with five axes radiating outward from the origin, where each axis represents a critical factor of the project. The graph is known as a home-ground chart. To complete the graph, the development team must consider five factors: personnel, dynamism, culture, team size, and criticality. Note that the home-ground chart associated with the project International Market appears in Figure 1. The Personnel axis captures the level of understanding and experience of the developers comprising the team. Table 1, which is adapted from [1], provides a brief description of each skill level. Agile approaches generally require a larger percentage of highly skilled individuals at Level 2 or 3, while plan-driven approaches require fewer highly skilled people, particularly once the project is defined. For a more complete explanation of the distinction among level designations, the interested reader should consult [1]. Table 1. Personnel Skill Levels Level 3 2 1A
Ability Adaptable to unprecedented new situations. Adaptable to precedented new situations. With training, can perform discretionary steps.
1B
With training, can perform procedural steps.
-1
Unable/Unwilling to follow shared methods.
286
Figure 1. Home-ground chart for the International Market of the details were undecided. Additionally, it appeared due to scheduling that one of the customers would frequently be unable to attend meetings. Based on these factors, the developers predicted roughly 40% of requirements to change. The dynamic nature of this project favors an agile approach. Culture. The culture of the development team was split. Two of the four developers clearly favored a chaotic environment thriving on the freedom it provided the team. While the other two developers seemed slightly more comfortable with an ordered approach, they were capable of working well in a chaotic environment. Therefore, the project manager assigned a value of 75% for the culture aspect feeling that this was not a black-and-white issue. Size. This project was small both in terms of team and product. The implementation team had four developers, which is a very small group. The product is somewhat more difficult to gauge, but based on the initial timeframe and project overview appeared to be relatively small in scope. The small size of both the team and product favor an agile approach. Criticality. The academic nature of this project means that no lives are at stake, but the customers did invest discretionary funds to support development. Since this is not a safety-critical project with any lives or large sums of money at stake, the risk of defects and/or failures favors an agile approach.
After completing the home-ground chart, it was evaluated to aid the final decision as to whether a plandriven or agile approach was more appropriate. The homeground chart for the IM project clearly favors an agile software development approach. The next step was to decide which specific methodology to apply. A few agile methodologies were considered including most notably XP and SCRUM [6], which has recently been adopted by several software development teams at Microsoft [7, 8]. The final decision by the coach and developers was to use eXtreme Programming (XP). This decision was based on four main considerations. First, XP is one of the most well-known and well-documented agile methodologies. Second, two of the four developers had prior experience with XP on multiple software engineering projects. Third, XP practices such as frequent releases and simple design seemed to fit well with customer and developer goals. Finally, there was a strong desire on the part of both the customer and developers to experiment with alternative methodologies; XP was deemed a better choice than SCRUM due to its stricter guidelines and twoweek iteration cycles as opposed to month-long sprints [6].
4. XP Summary Extreme Programming (XP) [9] is an agile methodology developed by Kent Beck, Ward Cunningham, and Ron Jeffries [10]. As such, it embraces the core values of the
287
Collective Code Ownership. Each of the developers reviewed, corrected, and improved code as needed. Code reviews and refactoring supported a sense of shared ownership, and all developers had the trust and courage, which is an XP value [9], needed to complete the tasks. Continuous Integration. This practice was adopted faithfully by the developers with the desired effect that development ran smoothly and releases were timely. Because the team worked with a versioning system (CVS), the development team was able to update/access the latest code at all times. Customer Tests. Throughout development the team attempted to validate the features and functionality against the user stories. Additionally, the customer frequently provided feedback in software demonstrations that were performed after each iteration. While these techniques were somewhat helpful in ensuring correctness, they were completed manually and did not cover the full functionality of the product. Ideally, automated testing tools would have provided frequent and consistent validation of the product, but the team was unsuccessful in finding free and easy-tounderstand tools. The main reason for this difficulty was only one member of the development team had prior
agile community set forth in the Agile Manifesto, placing special focus on people, flexibility, and working software. As mentioned in Section 3, XP is one of the most well known and well-documented agile methodologies. It is also the most disciplined of the agile approaches [1]. Some of the distinguishing features of XP include: pair programming, test-driven development, short iterations, and user stories [11]. Pair programming specifies that every line of code is developed in the presence of two developers which alternate in the role of driver (at the keyboard) and co-pilot (watching for errors). The main advantages of pair programming include better code, higher endurance, and a strong shared knowledge of the code. Test-driven development means that all code is developed as a means of passing a test. In other words, functionality is added by first writing tests to verify the desired functionality, and then writing code to make it pass. This ensures high test coverage, fast integration, and courage to make changes as tests will catch problems. Short iterations entail frequent releases or demos of working code to the customer. XP iterations are most commonly two week cycles in which the iteration is planned, the appropriate functionality is designed and implemented, and finally the working software is demonstrated to the customer. There are two major advantages to short iterations: 1) they allow customers to provide valuable feedback about the product early on, and 2) they help the developers to build the code incrementally while ensuring its functionality and validity. User stories are critical units in XP development, as they document customer requirements, thereby forming the basis for acceptance tests. Each user story, which is written by the customer on a 3” x 5” index card, describes some aspect of the final product. The customer prioritizes the user stories; then, the developer breaks each user story into tasks and estimates the build time. This planning process is a part of each iteration. Thirteen practices [12], which cover all aspects of the project, including analysis, design, implementation, testing, and modification, define XP. Table 2 presents an overview of these practices.
Table 2. The 13 XP Practices Practice Coding Standards Collective Code Ownership
Description Follow a single uniform standard. Allow every developer to add, edit, or delete any line of code.
Continuous Integration
Design, test, and integrate code frequently.
Customer Tests
Create black-box tests for user stories and run frequently.
Design Improvement
Update the code and design constantly to maintain simplicity.
Metaphor
Develop a metaphor to ensure a common understanding of the project.
Pair Programming
Develop all code in the presence of two (or more) developers together.
5. Experiences
Planning Game
Monitor the project by release and iteration planning.
The development team strived to adhere to each of the 13 XP practices throughout the project; there were both successes and failures. The following is a description of the team’s experience with each practice. Coding Standards. Although the team agreed on a standard, it was not formally recorded and was therefore difficult to maintain. Individual habits and a lack of experience with the implementation language PHP further hindered adherence. Pair programming and refactoring helped to support the standard, but the product as a whole lacked unification.
Simple Design
Keep the design simple.
Small Releases
Release working software frequently.
Sustainable Pace
288
Work a consistent and reasonable schedule.
Test-Driven Development
Add functionality by writing a test and the coding to make it pass.
Whole Team
Include customers as part of the development team.
Sustainable Pace. Although other responsibilities of team member schedules made the times vary from week to week, each developer scheduled and worked between 10 and 20 hours a week. This time commitment was agreed upon by the project director and the clients before the project was undertaken. Test-Driven Development. Although the developers initially attempted to follow this practice, they ultimately did very little test-first development. Becoming proficient in the implementation languages and delivering bi-weekly builds to the customer consumed team focus, preventing the developers from finding automated unit testing tools. The team felt that unlike customer tests, unit tests prior to code development seemed too tedious and time consuming to write and perform manually. Once again, in later stages of the project unit testing tools were found (such as PHPUnit and JSUnit), but they were not implemented for two main reasons. First, although unit tests would have been useful, it was too late to write tests to drive the development since much of the code had been developed. Secondly, the team felt that the time-cost to write retroactive tests was too high for the possible benefit at such a late stage. Whole Team. The customers and developers collaborated throughout the project, and everyone had an equal voice in posing questions and providing feedback. The developers interacted with equal roles, although one of the developers periodically took on the role of coach or team secretary keeping the team focused, facilitating meetings, and communicating with the customer. The clients guided the project progression and made decisions about the project requirements and their priorities. Although the clients could not be collocated during development sessions, they made themselves continually available for questions through e-mail and met at least biweekly with the developers. In retrospect, the team felt that the agile approach was a positive experience. Additionally, some of the team members acknowledged that many of the areas in which they struggled to adopt the XP practices might have resulted in smoother or more effective development. The two biggest problems that developers emphasized were time constraints and discomfort with some of the new practices. The discomfort could most likely have been eased with additional training and experience with XP and agile practices if time had allowed. A summary of the team’s experience with the 13 practices appears in Table 3. Each practice has an adoption rating from 0 (none) to 4 (complete).
experience using PHP and JavaScript. In later stages of development, some testing tools were found such as Simple Test and FIT, but they were not utilized due to time constraints. Furthermore, the entire team, both customers and developers, was satisfied with the success of the short cycles and frequent communication. Design Improvement. Refactoring of the design and code was regularly performed, as a common part of all coding sessions. Furthermore, the development team’s faithful adoption of collective code ownership meant developers were comfortable modifying code to improve efficiency or to follow coding standards. Metaphor. To foster a common understanding of the project, the team maintained both a dictionary and a project flowchart. However, an actual formal metaphor was never developed. Pair Programming. Pair programming was well followed in early iterations, and greatly helped the team to build a working knowledge of the implementation languages. In later iterations individual schedules strained adherence to this practice. An additional reason for moderate usage was a lack of comfort from some developers, despite the fact that such sessions tended to be more productive and fulfilling. Furthermore, the development team found pair programming the web pages for the user-interface less efficient than pair programming logical components. Whether or not this phenomenon is common for web page development is unknown; however a definite contributing factor to the developers’ difficulty was the coupling of the interface and application logic. Planning Game. The overall project deadline was updated after each release. Iteration planning was based on the user stories’ priorities as well as customer feedback. Because of misunderstandings between the clients and developers, there was some difficulty with inconsistent prioritization. However; the prioritization problems seemed to have little impact on project progress or customer satisfaction. Simple Design. Completed as needed by the development team, designs most commonly included diagrams, white board sketches, and descriptions. Designs were always created as a group; collaboration among all team members helped to foster a shared understanding of the project. While no rigorous metrics were maintained on design versus coding, the developers estimate roughly 25% of group meetings and development time was devoted to design, while 50% was consumed producing code. The remaining time was spent on various internal and organizational tasks. Small Releases. Software was released to the clients every two weeks throughout the main development phase. Each release included a demonstration of the working software with added functionality, as well as a discussion with the client to ensure satisfaction. User stories were updated when necessary.
6. Conclusions and Future Work This paper has discussed the application of agile practices in the development of the International Market, a Web-based tool to support manipulation-based shopping studies. More specifically, the authors have discussed their
289
IM’s third component, which allows researchers to create and manage marketing studies. The clients and authors expect to more strictly follow XP practices such as coding standards, customer tests, and test-driven development.
Table 3. Summary of Practice Adoption Practice
Adoption
Coding Standards Collective Code Ownership Continuous Integration
♦♦ (Moderately) Although standards were adopted, they were not strictly followed. ♦♦♦♦ (Completely) ownership of all code.
The
team
shared
7. Acknowledgments The authors would like to thank the project clients Dr. Dan Sherrell and Dr. Balaji Krishnan from the Department of Marketing and Supply Chain Management at the University of Memphis for all of their effort and patience. Additionally, the authors would like to express gratitude to Sai Kolli, a graduate student in the University of Memphis Department of Computer Science, who was another member of the development team.
♦♦♦♦ (Completely) Code was updated and integrated continuously.
Customer Tests
♦ (Minimally) A small set of tests were created and executed manually.
Design Improvement
♦♦♦♦ (Completely) The code and design were continually refactored for simplicity.
Metaphor
♦♦♦ (Generally) A dictionary and flowchart facilitated a shared understanding.
Pair Programming
♦♦ (Moderately) A substantial amount of code was developed in isolation.
8. References
Planning Game
♦♦♦♦ (Completely) Release and iteration planning were done according to XP.
[1] B. Boehm and R. Turner, Balancing Agility and Discipline: A Guide for the Perplexed, Addison-Wesley, Boston, 2005.
Simple Design
♦♦♦♦ (Completely) Design elements were created as needed in each iteration.
[2] Manifesto for Agile Software Development. Available at: http://www.agilemanifesto.org/
Small Releases
♦♦♦♦ (Completely) Working software was released to the client every 2 weeks.
[3] C. Neill and P. Laplante, “Requirements Engineering: The State of the Practice”, IEEE Software, Dec 2003, pp 40 - 45.
Sustainable Pace
♦♦♦♦ (Completely) Each member of the team worked at a reasonable pace.
Test-Driven Development
♦ (Minimally) A minimal amount of test driven development was performed.
Whole Team
♦♦♦♦ (Completely) Clients and developers worked closely throughout the process.
[4] M.Lindvall, V.Basili, B. Boehm, P. Costa, K. Dangle, F. Shull, R.Tesoriero, L. Williams, and M. Zelkowitz, “Empirical Findings in Agile Methods,” Proc.of XP/Agile Universe 2002, pp. 197-207. [5] A Sanja, Overview of Agile Management Project Perfect: White Paper. Available at: http://www.projectperfect.com.au/info_agile_programming.php
adherence to the 13 practices of eXtreme Programming (XP) as described by Ron Jeffries [12]. Even though the IM development team struggled to fully adopt some of the core practices such as pair programming, test-driven development, and customer tests, the members were successful in their efforts to follow the spirit of XP. Of particular note, the team was able to maintain a shared vision, short release cycles, and continuous integration of working software throughout the project’s life cycle. Despite certain challenges, team members grew to internalize the values of the Agile Manifesto and understand most of the fundamental benefits of agile practices. Finally, while the project provided the authors with additional experience in agile software development and was considered generally successful by the clients, there is still work to be done. The existing product will be optimized and tested to verify its functionality with additional acceptance tests at the clients’ site for validation. Furthermore, three of the authors plan to design and implement a new version of International Market. The revision will have a cleaner design and better support for
[6] K. Schwaber and M. Beedle, Agile Software Development with SCRUM, Prentice-Hall, 2001. [7] E. Breckner, “Journey of Enlightenment: The Evolution of Development at Microsoft,” Proc. of, the 27th International Conference on Software Engineering, pp. 39-42, May 2005. [8] D. Taft, “Microsoft Lauds ‘Scrum’ Method for Software Projects, eWeek, Nov 2005. Available at: http://www.eweek.com/article2/0,1895,1885883,00.asp [9] K. Beck, Extreme Programming Explained: Embrace Change (2nd Edition), Addison-Wesley, Boston, 2004. [10] D. H. Steinberg and D. W. Palmer, Extreme Software Engineering: A Hands-On Approach, Pearson Education, 2004. [11] K. Beck, Embracing Change with Extreme Programming, IEEE Software, Oct 1999. [12] R. Jeffries, “What is Extreme Programming?,” XP Magazine, Nov 2001. Available at: http://www.xprogramming.com/xpmag/whatisxp.htm.
290
