Integrating UX with Scrum in an Undergraduate

Integrating UX with Scrum in an Undergraduate Software Development Project Chase Felker, Radka Slamova, Janet Davis Dept. of Computer Science Grinnell College Grinnell, IA 50112, USA

@grinnell.edu> ABSTRACT We report our experiences using the Scrum agile software development method in an undergraduate user-centered web development project. Our chief contributions are to report on using Scrum in a summer research setting as distinct from academic-year coursework and to consider the integration of Scrum and user experience (UX) development methods in a learning environment. Our experience with combining Scrum and UX was positive: this methodology gave our project a clear structure, kept us motivated, and focused us on developing a usable final product. We discuss our adaptations of Scrum to UX development and to the summer research setting, along with challenges we faced and lessons learned, to inform students and faculty who wish to apply such methods in future projects.

Categories and Subject Descriptors D.2.9 [Software Engineering]: Management.. H5.2 [Information Interfaces]: User Interfaces – user-centered design. K.3.2 [Computers and Education]: Computer and information science education – computer science education.

General Terms Management, Human Factors.

Keywords Agile development, Scrum, user experience, user-centered design, undergraduate research.

1.

INTRODUCTION

While agile software development practices have become popular in professional environments, relatively little literature examines how learners can use agile methodologies in a non-professional setting. Moreover, published reports of combining user experience (UX) with agile development have focused on industry rather than academic settings. In this paper, we report on our experiences with combining the Scrum agile software development method with UX methods to improve the user experience of a local food co-op’s website. Our approach integrated design and UX evaluation with software implementation in the context of a student summer research project. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. SIGCSE’12, February 29–March 3, 2012, Raleigh, North Carolina, USA. Copyright 2012 ACM 978-1-4503-1098-7/12/02...$10.00.

In the next two sections, we explain our context and then describe how professional software developers have used Scrum alone and in combination with UX methods. In turn, we clarify how we modified Scrum for the undergraduate research setting and how we integrated Scrum with UX. Finally, we present benefits and challenges of our approach, arguing that Scrum can effectively be integrated with UX in our setting despite difficulties with estimating tasks. While some challenges remain, Scrum methodology gave our project a clear structure, kept us motivated, and focused us on developing a usable final product.

2.

CONTEXT

Our software development project aimed to improve the user experience of the Grinnell Local Foods Co-op website. The Co-op is a student-run organization at a small liberal arts college that helps community members buy a variety of seasonal produce from local farmers each month during the school year. The Co-op links 7 farmers with about 250 members. The website was created to facilitate local food sales through the Co-op. The essential features include three interfaces: for members to browse the available produce and place their orders, for farmers to post their inventory, and for coordinators to manage orders. The existing website was difficult to use: members had to place orders by viewing the entire inventory on a single page; they had difficulties with editing orders; and they sometimes accidentally ordered sold-out items. Our major goal was to implement a shopping basket feature and any other high priority features identified by coordinators, members, and farmers. We revised the website as part of a Mentored Advanced Project, a faculty-directed summer research project at Grinnell College. During the summer, two students worked closely with a faculty advisor for about 40 hours per week for 10 consecutive weeks. Both students spent the first two weeks of the summer learning Scrum, PHP, MySQL, HTML, CSS, and revision control tools. During the next weeks, we continued to allocate learning time as we transitioned into fixing bugs and refactoring the website. Half way through the summer, we felt confident enough in PHP to start building new, complex pages, like the shopping basket page. By the end of the 9th week, we finished major coding and began reflecting on our experience in a final paper. We built on the work of students from a Human-Computer Interaction class taught by the project advisor during the spring semester preceding our research. One team member was a student in this class; the other was not. The Human-Computer Interaction class conducted a single iteration of Hartson and Pyla ’s UX development “Wheel” process [7] to redesign the Grinnell Local Foods Co-op website. The class first conducted an in-depth contextual inquiry, during which students interviewed farmers, members, and organizers about their role in the Co-op and observed users’ interactions with the website. Data from the contextual inquiry were then summarized in a Work Activity Affinity Diagram [3,7], which helped the class develop models and requirements for

the new website. The class also created a high-fidelity prototype of a revised website and tested the with several participants.

3. 3.1

PRIOR WORK Scrum

Scrum [13] is a popular agile software development methodology that “concentrates on how the Team members should function in order to produce the system flexibly in a constantly changing environment” [1, p. 27]. In contrast to the “waterfall method” of designing and detailing implementations at the start of the project, Scrum assumes that the project will change as the software develops. Scrum Teams work in short, fixed-length cycles called Sprints, typically two weeks to one month in length. At the end of each Sprint, the software is potentially shippable and all work that the Team committed to do at the start of the Sprint is done [5]. In Scrum, there are three major roles: The Product Owner, the Team, and the ScrumMaster [5]. The Product Owner assesses development items' business value and prioritizes them in a spreadsheet called the Product Backlog. The Product Owner thus shapes the direction of the project by specifying what is developed and when. The Team is a multi-skilled group of five to ten people who build the product. The ScrumMaster manages the project on a macro-level to make sure that none of the Team members is blocked on a task and Scrum is used effectively. Scrum sets aside about 10 percent of the overall time for processes such as Sprint Planning, Review, and Retrospectives [5]. Before the start of each Sprint, the Team holds a Sprint Planning meeting where it selects the highest priority items to be completed in the next Sprint, plans how they will be implemented in detail, and estimates how long each will take. New items may not be added to the Sprint in progress. Several structures help the Team to stay on track. The Sprint Backlog categorizes tasks as “Not started, In progress, Done, Blocked.” The Burndown Chart graphs the number of hours that the Team needs to complete the Sprint. Finally, the Daily Scrum requires the Team and ScrumMaster to review the past day's tasks, coordinate on the coming day's tasks, and get help on blocked tasks. At the end of each Sprint, the Team holds a Review where it shows work that has been completed and receives feedback on it. Following the Review is the Retrospective, where the Team reflects on what practices from that week should be sustained and what should be improved [5]. Scrum's structure helps the Team be flexible, operate at a sustainable pace, and maintain a multi-skilled team [5]. Recent work has addressed Scrum and other agile software development methods in the context of undergraduate education (e.g., [6, 8, 10, 15, 16]). However, all of the cited articles consider agile methods in the context of traditional courses rather than an intensely mentored summer research experience. Only one article discusses Scrum in particular [15], and none consider the relationship between software development and UX development.

3.2

UX Development

User experience (UX) encompasses the usefulness of a product, usability of a product, and emotional impact of the product on the user. Key aspects of UX include user satisfaction, ease-of-use, effectiveness, learnability, and retainability [7,14]. Beyond the structure of the UX Wheel [7], several existing UX practices contributed to our project: Paper prototypes allow user interface designs to be tested before they are implemented in software [7,14]. Paper prototypes are easier

to create and change than software, and they can help identify major usability problems before undergoing the costs of implementation. Because paper prototypes do not include computation, one of the designers must play the role of “computer” to operate the paper prototype. Formative UX evaluation guides design by uncovering major impediments to usability [7,14]. Representative users participate remotely or in person, in a large group or individually, and at any point in the design process. Participants perform a set of tasks and think aloud to reveal their intentions and reactions so that the UX developers can understand usability problems. The Rapid Iterative Testing and Evaluation (RITE) method facilitates formative UX evaluation by allowing the designer to make changes to prototypes as soon as problems are identified. By allowing greater flexibility to address problems and test proposed solutions within a single session, RITE minimizes the number of participants needed to uncover major usability problems and is therefore much more efficient [7,11]. Brief, regular tests are recommended by usability expert Steve Krug. He recommends that developers schedule UX tests even before knowing what they will be testing to produce a short list of the most essential problems with the current product [9, pg 28].

3.3

Integrating UX and agile development

Several industry professionals have reported on their experiences integrating UX development with agile software development [2,4,12]. Most report significant challenges. Even in companies that already support UX practices, UX experts and software experts often work in separate groups, and UX groups have difficulty adjusting to an agile methodology. For example, Paypal’s UX Teams reported surprise at the fast-changing schedule of programmers and experienced uneven work loads. In order for the workload to become manageable, UX Teams adopted Scrum, with their own Product Owners and Backlogs, and learned to design a sprint ahead of the developers [4]. Theoretically, the major road block to integrating UX and agile development comes from tension between the incrementalism of agile processes and the holistic nature of some UX processes. In particular, contextual inquiry is a process of discovery that should be undertaken early in a project [3]. Rannikko suggests that while UX and Agile are not mutually exclusive, they need to make compromises [12]. Specifically, he suggests that comprehensive studies such as contextual inquiry should be completed before agile development begins. During agile development, both UX tasks and development tasks can be completed incrementally [12].

4. 4.1

OUR USE OF SCRUM + UX Scrum Practices

We significantly modified the major roles of Scrum. The Product Owner was a Co-op coordinator, the small Team consisted of two undergraduates, and the ScrumMaster was the faculty advisor. The Team was atypical in that it was smaller than normal (just two vs. 510 developers [5]), and its members were not experienced software developers. Our ScrumMaster was likely more involved with details than is typical; this was partially because our project was a learning experience and the faculty advisor had greater expertise than the Team. Our Product Owner differed significantly. As a coordinator for the Co-op she clearly had extensive experience with the site, but she took a comparatively hands off approach to the Product Owner role. As she held a full-time summer internship in another state, she was not present at any UX evaluations and her availability was quite limited. We spoke with her through Skype once per week during our Sprint Review where we told her what we did, demoed what was

done, prioritized new items with her, and received her approval for what we would do next week; she was not available to address further questions during Sprint Planning. Although we modified Scrum roles, we closely followed the structure of Scrum. We maintained a Sprint Backlog with notecards on a white board, maintained a Burndown Chart, had Daily Scrums, and followed other Scrum processes. During the week, Team members averaged about six hours per day on Sprint work. Our multi-skilled Team programmed together much of the time. At the end of each Sprint, we had a Review where we met (online) with our Product Owner and demoed our finished tasks. A Retrospective always followed the Review. During the Sprint Planning meetings, the Team decided what tasks to commit to in the next Sprint, and then extensively planned and estimated all these tasks. These planning meetings lasted about 10 percent of the Sprint length, as suggested in the Scrum Primer [5]. We decided on one-week Sprints, a cycle shorter than the typical two to four weeks [5]. Choosing Sprint length was a challenge, which we will discuss in section 6.2. Although we followed Scrum’s structure for the majority of the project, three weeks were not proper Sprints: for the first two weeks, we focused on learning Scrum and development tools, developing the Product Backlog and reviewing existing materials. We spent the final week preparing a project report. Finally, our use of Scrum was unusual because we combined it with UX. This led us to separate “definitions of done” [5] for software and UX development tasks, along with other changes discussed below.

4.2

Combining UX with Scrum

Integrating UX with Scrum felt natural for our Team, since we were new to software development. Lacking any existing software development culture, there were no conflicts when introducing Scrum. Because our small Team did both UX and software development, collaborating across separate UX and software Teams was not a problem. As noted earlier, Rannikko recommends that contextual inquiry be completed prior to starting software development [12]. Fortunately, we were able to rely on the results of the contextual inquiry completed by the Human-Computer Interaction class the previous semester. All the other UX development was completed during Sprints within the Scrum structure, consistent with Rannikko’s recommendations [12]. During our Sprint Planning meetings, we determined whether we wanted to conduct a UX test that week, per Krug’s advice to evaluate regularly [9, p. 27]. As our tests were always towards the end of the Sprint, and because we would test new features designed or implemented during that Sprint, we did not always know at the time of Sprint Planning exactly what we would be testing. A task during our Sprint would be to plan the evaluation session and write tasks for the participants to complete. We also included design and prototyping tasks in our Sprint Planning. We conducted three formative UX evaluations: results from the evaluation were intended to improve the design rather than to draw conclusions about the system. The first two tests involved paper prototyping the login process and the ordering system, and the third tested these features after they were implemented. We spent about half a Sprint for each evaluation of a paper prototype, including sketching designs, creating the prototype, conducting a pilot test to discover obvious problems with the prototype or tasks, conducting the sessions, modifying the prototypes, and debriefing. While Krug advises that regular usability testing requires little more than half a day [9], he assumes developers are testing existing software.

Using the RITE method [11] gave us the flexibility to modify the prototypes and tasks between test sessions. We always made some modifications: no two participants worked with exactly the same tasks and system. Our tests lasted 30 minutes, and we allowed an additional 30 minutes between participants to either modify the prototype or change the tasks. We also did frequent informal tests with the Product Owner where she both looked at sketches and experimented with implemented features. However, because of her remote location, she did not participate in testing paper prototypes or observe UX tests with participants acting as Co-op members.

5. 5.1

BENEFITS Benefits of Scrum

We benefited from the clear structure and motivation that Scrum supplies. At the start of the project, even small tasks were daunting due to our inexperience with PHP and other tools. Scrum helped by forcing us to plan out items for a set period of time, break them down into smaller tasks, and specify in detail what they would involve so we could estimate them accurately. Our Product Backlog provided constant temptation to work on one of the many other things that could be improved on the site, but Scrum strategies such as the Burndown Chart helped us stay on task. As work on a newly added item would not be reflected in the chart, we were motivated to only work on tasks we committed to do at the beginning of the Sprint. The Sprint Retrospectives and Reviews were a good learning opportunity; in fact all of our Recommended Strategies came from Retrospective discussions. We did not have to deviate significantly from overall Scrum’s structure to use it in a learning environment instead of a professional development setting. The project advisor easily assumed the role of ScrumMaster: her focus was to make sure we were not blocked on tasks rather than to micromanage our work. The Daily Scrum gave the advisor a reason to come in every day to check on student progress. As Hart observes, “frequent interactions help prevent small problems from snowballing into larger delays” [6]. The Team benefited from a daily deadline, since knowing that the advisor would come in at 1:30 gave motivation to accomplish something every day. This helped to give our Sprints a sustainable pace compared to previous school projects. When we deployed our code, there was no need for a “hardening Sprint” dedicated just to fixing bugs that were not caught during the Sprints. We attribute this to Scrum's emphasis on having a working system at all times.

5.2

Benefits of UX Design and Evaluation

Scrum emphasizes integrating the Product Owner into development. This is compatible with the user-centered design philosophy of involving users in all stages of development. However, the Product Owner is only one person and may not be able to represent the perspectives of all users. The UX process enabled many insights that, as individuals, the Scrum Team had overlooked. The Contextual Inquiry helped us set initial priorities. For example, we knew from the first day that everyone thought the ordering process on the old system needed to be fixed. UX evaluation of paper prototypes gave us confidence that we would implement a usable system. These tests were an effective and fast way to compare different design possibilities. UX testing was always an effective deadline. Our paper prototypes or implementations needed to be complete enough to test. We paid our participants for 30 minute sessions, so we knew that we had to put some thought into what to test during that time. Participants in the third UX test caught bugs that we had overlooked; for example our search feature would work only when

the user pressed enter instead of the form submit button because of the way we displayed “Search produce” in the text box. This test also pointed out the most urgent items from our large Product Backlog that should be addressed before the site went live.

6.

CHALLENGES

We faced several challenges related to the role of the Product Owner, Sprint planning, Sprint progress, and incorporating UX development.

6.1

A Different Role of Product Owner

Different Location and Constrained Time. One of our biggest challenges was trying to use Scrum as it is described in the literature while not having a fully committed Product Owner. Because our Product Owner could not watch our UX tests, she sometimes disagreed with our Team about users’ needs. Moreover, we had difficulty remotely sharing our work with her due to erratic Internet connections. Our Product Owner was also not available to answer questions at any time, which forced us to make important decisions, like aborting a Sprint, without her. Lack of Real Ownership. Our Product Owner did not have a real ownership of our project because she did not pay us to do the project, and it was not even her idea to do the project. Because we were paid by the school, not by the Local Foods Co-op, our Product Owner was grateful for any improvements we did on the website and felt it would be inappropriate for her to pressure us to work harder. Even when we did not finish our Sprint, she was grateful for our work and did not criticize our time management. In general, our Product Owner did not have high expectations for us and rarely offered constructive feedback or asked us to improve our work. Assuming Some Responsibilities of Product Owner. Because our Product Owner was not able to take on all typical responsibilities, our ScrumMaster and Team members took over some of the tasks of a Product Owner, including updating Product Backlog and directing Sprint Review. Moreover, we made decisions such as aborting a Sprint by group consensus, when those decisions should have been made by the Product Owner.

6.2

Sprint Planning

Sprint length. We had a hard time deciding whether our Sprints should last one week or two weeks. Two-week Sprints, which are more typical, give time to complete larger Product Backlog items and allow more room for error in estimating individual tasks. However, we would have had only three full Sprints during the summer, and thus would not experience Scrum as an iterative process. Another disadvantage was that the Team's inexperience made it difficult to estimate future tasks using skills the Team had not yet learned. We decided instead to use one-week sprints, which enabled us to reflect on our Scrum experience on a weekly basis and thus to quickly adjust our planning to how long different kinds of tasks usually took. The negative consequence of doing one-week Sprints was that we sometimes failed to present a working system at the end of the Sprint because some tasks required more than a week of work. Time estimates. Our greatest challenge during Sprint Planning Meetings was estimating how long different tasks would take. Although the Scrum Primer claimed that groups usually become accurate at estimating tasks within four weeks, we had problems coming up with realistic time estimates during the whole summer. We especially struggled with estimating time for complicated tasks, tasks that required learning a new skill, and tasks involving the use of CSS. We also constantly underestimated how long complicated tasks would take, usually because we failed to think of all things that

needed to be done for the task to be complete or because we did not allocate enough time for instances when we got stuck. Furthermore, many of our tasks were different from those of professional software developers. Unlike in typical practice, Team members had to learn new web development tools and UX methods, and also wrote a research paper. Another challenge was to set a minimum time estimate for our tasks. We decided that estimating less than 30 minutes for a task would be risky because, if we ended up getting stuck on the task, the time to complete the task would end up being many times longer than our time estimate. Thirty minutes was a strategic overestimate for most of the tasks that we hoped would balance out the time spent on unpredictably hard tasks. Planning too much vs. too little. We had difficulty finding a healthy balance between planning tasks in excessive detail and not planning them well enough. Not breaking tasks into small pieces usually resulted in leaving something important out and thus underestimating how long it would take to complete a task. Planning tasks in great detail, on the other hand, usually resulted in a 4-hour long Sprint Planning Meeting and a Sprint Backlog overflowing with 30-minute task cards. By breaking complex tasks into smaller pieces during Sprint Planning, we usually avoided leaving something out, but we also sometimes spent more time planning a task than it would take to complete it. New Functionality vs. Refactoring and Fixing Old Bugs. Because we were building on an existing website, one of our challenges was to decide how much time we wanted to spend improving the existing code and how much time we wanted to devote to creating a new functionality. Working with the previous website was difficult because code was duplicated and had inconsistent variable names and the code files were not organized in any way. During our project, we struggled to find a balance between making the code beautiful but never get any new features done versus suffering with the code as it was and focusing totally on implementing new features.

6.3

Sprint Progress

Doing Easy Tasks First - A Misleading Burndown Chart. The line of our Burndown Chart usually showed great progress during the first day of the Sprint and then leveled off during the next days of the Sprint, hardly ever reaching the desired “0 hours of work left” at the end of the Sprint. Seeing this pattern, we usually became overoptimistic during the first Daily Scrum of each Sprint and assumed that we would finish the Sprint without problems. It took us several Sprints to realize that we appeared so productive after the first day of the Sprint not because we got a lot of work done, but because we picked the easiest, overestimated, 30-minute tasks first and we left the harder, underestimated tasks for the later stages of the Sprint. Thus, our curve would inevitably stop running steeply downwards as soon as we started working on the more complicated tasks, which were often underestimated. Sprints Interrupted by Weekend. One reason why we tended to start with short and easy tasks was because we started our Sprints on Fridays and wanted to finish some tasks before leaving for the weekend. If we picked a complicated task on Friday, we would not complete it that day and have to remember it on Monday. Ideally we would have started our Sprints on Mondays, but we had to accommodate our Product Owner in scheduling the Sprint Review and Planning sessions.

6.4

Prototyping and UX Evaluation

Balancing UX and Implementation Work. We found it hard to decide which of our features needed to be paper prototyped and

evaluated given our limited time. To provide a good user experience, we would ideally evaluate all of our features using sketches and paper prototypes before implementing them, and again after implementing them. However, UX evaluation took time away from implementation because we did not have separate UX and development Teams; both members of our Team needed to contribute to creating, testing, and evaluating prototypes.

prototyping or refactoring, we made sure to include smaller coding tasks (e.g., bug fixes) in case we got stuck. Plan how to test code as part of Sprint Planning meeting. Planning tests in advance helped us set standards for our code and also helped us budget enough time for testing.

7.3

Sprint Progress

When to design and test. We followed the parallel track development model that recommends conducting UX testing a Sprint ahead of implementation (Nodder & Nielsen, as cited in [12]). Although this model helped us successfully combine design with implementation for most of the summer, we wasted some time at the end of the summer when we did not get to implement a feature that we carefully designed a Sprint ahead.

Use your definition of “done.” This strategy will help you stay on task and know when it is time to move on to the next task.

Estimating UX Tasks. Guessing how long a design task would take was difficult because design tasks were more open-ended than coding tasks, and it was less obvious when we were done.

Decide on an exact time when the Sprint ends. At first, the last day of the Sprint was hectic because we would continue working on new tasks until the start of the Sprint Review. It was tempting to work some extra time on the last day of the Sprint to finish all our tasks, but doing so made us exhausted, hungry, and stressed, which goes against Scrum’s philosophy of sustainable work pace. Deciding to end our Sprint mid-day left a buffer for planning the review meeting and ensuring our work was ready to demo.

7.

LESSONS LEARNED

Sprint Retrospectives exposed many lessons along the way, in addition to those synthesized at the end of the project. We relate lessons learned to communication with the Product Owner, Sprint Planning, Sprint progress, Retrospective, and Integrating UX.

7.1

Communication with Product Owner

Start your Sprint on Monday. We found that starting a Sprint on Friday led us to pick short tasks on Friday because we were afraid we would not finish larger tasks before the weekend. Having the weekend in the middle of our Sprint disrupted the sustainable pace of our Sprints.

7.4

Retrospectives

Do not arrange UX testing and client meeting on the same day. Scrum philosophy is to work at a “sustainable pace,” and dealing with several users and a Product Owner in one day is exhausting.

Make time for retrospectives. Unfortunately, Scrum Teams often omit the Retrospective [5]. But retrospectives are key for allowing the Team time to reflect and improve their process. Our Sprint Retrospectives were incredibly useful. They helped us realize what we needed to change in order to be more successful in the next Sprints. In fact, most of the points in this section are taken from our Sprint Retrospective notes. Retrospectives were the primary time when we would purposefully step back and reflect on what should be changed.

7.2

7.5

Only demo things that are done. This rule is part of the Scrum Primer [5], but it was challenging for our group to stick to it when our new CSS markup was almost done and we wanted to show it to our Product Owner. Not showing a new feature until it is done motivates the Team to finish the task properly.

Sprint Planning

Allocate time for learning. This practice will save time in the long run. We failed to allocate time for learning CSS at the beginning of the summer research, hoping to find quick solutions to our CSS problems. Because we did not understand how CSS works, we always spent much more time on tasks involving CSS than we predicted. By contrast, we did allocate time for learning PHP. Before diving into bug fixes and writing our own code, we spent several days reading about the basics of PHP and trying to understand existing code. Because we understood PHP quite well before we started using it, we were able to resolve PHP problems much more quickly than CSS problems. Plan each task in detail. When we underestimated tasks during Sprint Planning, it was often because we did not think though all aspects of the task. Though extensive planning can be timeconsuming, it let us estimate tasks more realistically. Allocate time for developing a multi-skilled Team. As noted earlier, Scrum’s philosophy is to encourage a development of a multi-skilled Team [5]. It was beneficial for each Team member to mentor the other, one having greater programming experience and the other skills in prototyping and user testing. By the end of the summer, both Team members were able to write website code and run UX evaluations. Maintain a balance of UX and software development. We found it useful to include both refactoring and prototyping tasks in a single Sprint because when we got stuck on a coding task, we could still work on something different. In Sprints that did not require any

Integrating Scrum and UX

Complete the Contextual Inquiry beforehand. Having the results of a recent contextual inquiry was incredibly helpful and allowed us to focus solely on design and implementation. Create your own definition of “done” for UX-related tasks. Having a definition of done is a standard Scrum practice. Because our design tasks were very different from our coding tasks, it was helpful to have a different definition of “done” for design and implementation tasks. Design a Sprint ahead of implementation. Designing just a Sprint before implementation for the most part prevented us from designing features we would never have time to implement. Limit time for design. Design tasks are open-ended; judging when a sketch or prototype is complete is difficult. As inexperienced designers, we decided to allocate a certain amount of time to spend on design. Once that time was exhausted, we deferred further decisions until the evaluation. Schedule testing before you know what you will test. Because of our short Sprints, we usually created our paper prototypes and task cards only days before the actual tests. However, participants usually want to know more than 2 days in advance that they are invited to a UX evaluation. Following the advice of Krug [9], we recruited participants and set up sessions with them before we knew what we would evaluate. This strategy originally sounded risky to us, but ended up working surprisingly well. Allocate twice as much time for UX evaluation as sessions will actually take. Doubling the testing time during Sprint Planning will

allow for adjusting the prototype between the participants and will leave enough time for setup. Allow separate time for pilot testing and debriefing. Having task cards for pilot testing before the evaluation sessions and debriefing after the sessions are over will ensure that you allocate enough time for these important tasks. The RITE method helps you get the most out of evaluation. RITE worked especially well when we were testing our paper prototypes because many of the things that confused the participants were easy to change and there was no point in keeping them the same. For example, after we found that participants overlooked our warnings, it made sense to write the warnings in a different position on the page and to test whether these changes will help. Thanks to RITE, we were able to greatly improve our designs with only 3-4 participants per evaluation. If evaluating a working system, allocate at least an hour between participants. Changing software is usually more time-consuming than changing a paper prototype. Test for as long as you are paying for. Pilot tests sometimes showed us that our tasks did not fill the 30-minute period that we would pay our participants for. We created more tasks so that the session would take at least 30 minutes, and we decided which tasks were optional if we were running out of time. Every minute with a participant can lead to discovering another problem or design idea, so concluding the session early would be a missed opportunity to discover something new and a waste of money. Debrief right after UX tests. Reflecting on our results from UX evaluations right after the last session worked well because we still remembered most of our participants’ comments and reactions. Debriefing early ensured that even Team members who could not take notes during the session could still contribute.

8.

CONCLUSION

We learned that Scrum can successfully be used to guide usercentered software development in an undergraduate summer research project. We benefited most from Scrum’s clear structure. Many of our “lessons learned” came from Sprint Retrospectives. Sprint Planning forced us to think about what each task entailed, and short, iterative cycles helped us maintain a sustainable pace. Integrating UX methods into our software development process was not that difficult, and it provided us with several benefits: it helped us to prioritize desirable features and choose among competing designs; it revealed usability problems prior to implementation; and it even uncovered some bugs. Our completed software is being used by the organization for which it was developed, and users are pleased with the results. The project was not without its challenges. We had to significantly adjust Scrum since we did not have a committed Product Owner or a long time to work on the project. We also had to replan constantly, leave many Sprints incomplete, and even abort a Sprint, largely because we failed to accurately estimate many tasks and our oneweek Sprints left little room for error. If Scrum is to be used in a learning environment, the Team needs to create ample task cards for learning and may wish to schedule longer Sprints despite having fewer Sprints. We hope that our lessons learned will inform future undergraduate summer projects that aim to develop usable software.

9.

ACKNOWLEDGMENTS

We thank the Grinnell Local Foods Co-op organizers and members for their cooperation and enthusiasm, our user study participants for their time and reflections, computer science summer research

students and faculty for their insights, Grinnell College Writing Lab staff for proof-reading our paper, and Grinnell College for its financial support.

10.

REFERENCES

[1] Abrahamsson, P., Salo, O., Ronkainen, J., Warsta, J.. Agile software development methods. Retrieved September 1, 2011 from http://www.vtt.fi/inf/pdf/publications/2002/P478.pdf [2] ACM case study. UX design and agile: A natural fit? Communications of the ACM 54, 1 (January 2011), 54-60. [3] Beyer, H. and Holtzblatt, K.. 1998. Contextual Design: Defining Customer-Centered Systems. San Diego, CA: Academic Press. [4] Budwig, M., Jeong, S., Kelkar, K. When User Experience Meets Agile: A Case Study. 2009. In Extended Abstracts on Human Factors in Computing Systems (CHI EA 2009), Boston, MA, April 4-9, 2009, 3075-3083. [5] Deemer, P. and Benefield, G. The Scrum Primer. Retrieved September 1, 2011 from http://www.rallydev.com/documents/scrumprimer.pdf [6] Hart, D. 2010. Supporting agile processes in software engineering courses. Journal of Computing Sciences in Colleges, 25, 6, 136-143. [7] Hartson, R. and Pyla, P. 2012. The UXBook: Process and guidelines for ensuring a quality user experience. San Francisco, CA: Morgan Kauffman. [8] Kessler, R. and Dykman, N. 2007. Integrating traditional and agile processes in the classroom. In Proceedings of the Annual Technical Symposium on Computer Science Education (SIGCSE ‘07), Covington, KY, March 7-10, 2007, 312-316. [9] Krug, S. 2010. Rocket Surgery Made Easy. Berkeley, CA: New Riders. [10] Lu, B., and DeClue, T. 2011. Teaching agile methodology in a software engineering capstone course. Journal of Computing Sciences in Colleges, 26, 5, 293-299. [11] Medlock, M.C., Wixon, D., Terrano, M., Romero, R., and Fulton, B. 2002. Using the RITE Method to improve products: a definition and a case study. Presented at the Usability Professionals Association, Orlando, FL, July 2002. Retrieved July 22, 2011 from http://www.microsoft.com/download/en/details.aspx?id=20940 [12] Rannikko, P. User-Centered Design in Agile Software Development. Retrieved 1 September 2011 from http://www.pirkkarannikko.com/agile-ucd.html [13] Rising, L., and Janoff, N. S. 2000. The Scrum software development process for small teams. IEEE Software (July/August 2000), 2-8. [14] Rogers, S., Sharp, H., and Preece, J. 2011. Interaction Design: Beyond Human-Computer Interaction, 3/e. John Wiley & Sons. [15] Scharff, C. and Verma, R. 2010. Scrum to support mobile application development projects in a just-in-time learning context. In Proceedings of the 2010 ICSE Workshop on Cooperative and Human Aspects of Software Engineering (CHASE ‘10), Cape Town, South Africa, May 2, 2010. [16] Sherrell, L. B. and Robertson, J. J. 2006. Pair programming and agile software development: Experiences in a college setting. Journal of Computing Sciences in Colleges, 22, 2, 145153.