globally distributed software development project at Nokia, a global telecommunications company. We discuss how the case project scaled Scrum while growing ...
2016 IEEE 11th International Conference on Global Software Engineering
Scaling Scrum in a Large Globally Distributed Organization: A Case Study Maria Paasivaara, Casper Lassenius School of Science Aalto University POB 15400, 00076 Aalto, Finland Email: {firstname.lastname}@aalto.fi ”Agile and large projects” as the top burning research question. Moreover, among the top ten items three focused on distributed agile development, which is common especially in large organizations as they are often distributed. In the workshops on large-scale agile development organized in XP2013 and XP2014 conferences, adoption of agile methods was one of the highlighted themes needing more research [6], [17]. Agile consultants helping the large companies adopting agile have suggested several frameworks for scaling agile, such as Large-scale Scrum (LeSS) [18], Scaled Agile Framework (SaFE) [19] and Disciplined Agile Delivery (DAD) [20]. A few books have been published by the inventors of these frameworks describing how to scale agile using the frameworks. Even though very interesting, these books are mainly written by consultants, who do not always provide much evidence to support their suggestions. Empirical studies on how these scaling frameworks and practices really work in practice, what kinds of challenges there might be, and how to overcome the challenges are still rare. With this paper we contribute to filling in this gap by describing a case study on adopting and scaling Scrum in a large globally distributed project at Nokia, a global telecommunications company. The studied project adopted Scrum immediately, at the project start-up phase, and used the LeSS framework to guide scaling while growing from two collocated teams to twenty teams spread across four locations. In this paper we describe the practices the project used when scaling Scrum, the successes it achieved and challenges it faced. We have briefly discussed the same case in a previous paper [21]. The rest of the paper is structured as follows: Section 2 presents related literature, Section 3 describes the research goals and methods, Section 4 presents the case project, Section 5 shows the results, and finally Section 6 discusses the findings.
Abstract—We present a case study on scaling Scrum in a large globally distributed software development project at Nokia, a global telecommunications company. We discuss how the case project scaled Scrum while growing from two collocated Scrum teams to 20 teams located in four countries and employing a total of 170 persons. Moreover, we report scaling challenges the case project faced during this 2,5 year journey. We gathered data by 19 semi-structured interviews of project personnel from two sites, interviewees comprising different roles including managers, architects, product owners, developers and testers. The project was highly successful from the business point of view, as agile enabled fast response to customer requirements. However, the project faced significant challenges in scaling Scrum despite attempts at applying the Large-scale Scrum (LeSS) framework. The organization experimented with different ways of implementing scaling practices like implementing common sprint planning meetings, Scrum-of-Scrums meetings, common demos and common retrospectives, as well as scaling the Product Owner role. We conclude the paper by reflecting on the scaling approach used in the case organization in contrast to the LeSS framework.
I. I NTRODUCTION Agile methods were originally designed for use in small projects [1], members of which are collocated, working faceto-face, preferably in team rooms. As nowadays many companies develop large systems with multiple teams distributed to several geographical locations the basic agile is a poor fit for them. However, the shown and potential benefits of agile have made these methods attractive for large companies and projects as well, even though they are more difficult to implement in large organizations [2]. Scaling involves several challenges, such as coordination between the teams, lack of architecture, lack of requirement analysis, as well as possibly all the challenges of distributed projects. Despite the problems, there seems to be an industry trend towards adopting agile methodologies in-the-large [3]–[6]. Large companies such as Nokia [7], Ericsson [4] [5], Amazon [8] and British Telecom [9] have adopted agile in their large-scale projects. While the research literature contains several experience reports [10]–[13] and some case studies [14], [15] on the adoption of agile methodologies in large organizations, there seems to be little available evidence on how to scale these methods. Freudenberg and Sharp [16] asked the industrial practitioners at the XP2010 conference to create a backlog of topics they think should be studied. The practitioners voted 2329-6313/16 $31.00 © 2016 IEEE DOI 10.1109/ICGSE.2016.34
II. R ELATED W ORK In this section we briefly discuss relevant related work. We start with an overview of agile software development, followed by an overview of existing studies on introducing agile development in large organizations. The last part of this section discusses the agile scaling frameworks, concentrating on the LeSS framework adopted by our case organization. 74
A. Agile software development
for multiple-team development by including members from other or all teams, as necessary. The main practices of LeSS Huge are listed in Table II. A basic principle is to divide the product into requirement areas, each having an Area Product Owner with 4-10 feature teams and prioritized area backlogs. Each area performs its own sprint planning part I meetings, sprint reviews and retrospectives. The Scaled Agile Framework, SAFe [19] claims to provide a recipe for adopting agile at enterprise scale. It contains levels of teams, programs, and portfolio. At the team level, it adopts Scrum with XP engineering practices. At the program level, it defines the concept of agile release train, which is the analogy to Sprints at the team level, working at a slower time frame. The program level contains additional roles, e.g. system team, product manager, system architect, and release train engineer and release management team. At the portfolio level, planning is done at the epic level that define large development initiatives. Disciplined Agile Delivery claims to be a ”people-first, learning-oriented, hybrid agile approach to IT solution delivery” [20]. DAD extends Scrum with ideas from Agile Modeling, XP, Unified Process, Lean Software development and other methods. Besides the above mentioned three quite known frameworks there are a few others that might be called frameworks such as ”Scrum at Scale” and DSDM. Altogether nine scaling approaches are listed in the Agile Scaling Knowledgebase Decision Matrix (version 5.1) [32]. Interestingly, despite the fact that several scaling frameworks have been proposed, used and improved for over ten years, there is very little, if any, scientific studies on their application in industry. While the websites of the framework developers tend to present ”case studies”, they tend to contain little more than testimonials.
Agile software development is a development philosophy and set of iterative and incremental development approaches. Traditional processes view software development as a sequential activity, and strive to minimize change during development, attempting to fully define the requirements before development starts. However, adaptability during the development life cycle and the ability to respond to conditions unforeseen at the start are nowadays seen as critical factors for the success of a software development project. Agile software development approaches aim at embracing change during development, are people-centric, and aim at constantly keeping the quality of the developing software high. [22], [23] The perhaps most popular agile method is Scrum [24]. Scrum is a method focusing on the project management viewpoint of agile development [25], prescribing timeboxing, continuous tracking of project progress, and customer centricity. Scrum divides work into iterations, called sprints, which are time boxes of 1 to 4 weeks, which aim at producing a finished increment of working software [25]. B. Adopting agile methods in large organizations Introducing agile methods in large organizations is more difficult than in small organizations [26]. The difficulty is partly related to size bringing higher organizational inertia which slows down organizational change [27]. Agile development is not founded on the use of individual tools or practices, but rather on a holistic way of thinking. Adopting agile often requires change of the entire organizational culture [28]. One significant difference between small and large scale adoptions is that large organizations have more dependencies between projects and teams. This increases the need for formal documentation and thus reduces agility [29]. In addition to inter-team coordination, development teams must interact with other organizational units, which are often non-agile in nature. For instance, human resources unit may demand individuals to have strictly specified roles in projects [1], or a change control board may inhibit the use of continuous integration or refactoring [29]. All units affected by the agile transformation need to be informed and consulted, and the agile process must be adjusted according to their needs [1], [29], [30].
III. R ESEARCH M ETHOD A. Research goal In this study we focused on how the case project organized inter-team coordination and cooperation, as they are seen as the main pain points in scaling agile to several collaborative teams. We set out to answer the following research question by conducting a single case study [33]: RQ: How did the case project scale Scrum over multiple teams and sites?
C. Scaling Frameworks The creators of the LeSS framework estimate that the first LeSS adoptions took place in 2005, while they published the first LeSS description as a book [31] in 2009. The LeSS framework describes how to scale Scrum to ”big product groups”. There are two versions of less, the ”normal” less, for up to eight Scrum teams, and LeSS Huge, which the authors claim is applicable up to a few thousand people on one product. The basic idea behind LeSS is to take the basic one-team Scrum and minimally adapt the Scrum practices to fit a larger project. It retains a single product backlog, the same definition of done for all teams, synchronized Sprints leading to a potentially shippable product after each spring, and a single product owner. Scrum ceremonies are adapted
B. Case Study This study is based on a qualitative multiple-case study approach [33]. Conducting a case study was appropriate since we wanted to get an in-depth understanding on the adoption and scaling of Scrum practices in their real industrial environment. The studied case project was selected purposefully, i.e., by choosing an information rich case [34] as we encountered an extremely interesting project from a company participating in a joint research program and suggested a case study. As a result
75
qualitative coding. The four main categories for coding were created based on our research questions: scaling practice, challenge, success/positive observation, and case background. Sub-codes emerged during the coding process. Finally, the data was extracted from the coded interviews.
TABLE I I NTERVIEWEES Role
# of Intervieweesa) Finland + Greece
Team Members Scrum Master System Architect Area Product Owner Solution Architect Area Product Owner Software Architect Agile Coach Line Managers Other Managers
7 2 2 0+2 1 1 2 3+1
Total
21
a
E. Validation After the first interview round the results were presented to the case company in a feedback session to which all interviewed persons, as well as other interested project participants were invited. A lot of questions were asked and the audience was eager to discuss. The feedback session showed that taking Scrum into use in this large project was much more controversial than the researchers had realized during the interviews: even though the majority of the project personnel found this change as a step towards better, there were still a few persons who clearly resisted the change. Thus, the discussion mainly focused on the change and whether taking Scrum into use was for better or worse. However, our results on the practices used, the successes achieved and challenges faced were regarded, e.g., by the Product Owner (PO) and an internal agile coach as valuable. The feedback discussions did not lead to any changes in the findings.
Note: Two interviewees had double roles.
of our study the case company was hoping to gain understanding on how their Scrum adoption was progressing, what the successful practices were, and what kind of challenges and improvement opportunities there might be. C. Data Collection We collected data by 19 semi-structured interviews. The researchers and case company representatives discussed the criteria for choosing interviewees. To get a good overview of the project, we first interviewed a few managers and an agile coach, who could give us a good picture of the project history, goals, growth, structure and the main practices used. The rest of the interviewees were selected from different roles, different Scrum teams (representatives from 8 teams) and with differing length of experience in this project. Table I shows the roles interviewed at both sites. The interviews of the managers and an agile coach lasted 1,5 -3 hours, while the rest of the interviews were shorter, taking 1-1,5 hours. The language of the interviews was either English (7 interviews) or Finnish (12 interviews) depending on the language skills of the interviewees. Both interviewers were native Finnish speakers. The first round of interviews, altogether 16 interviews, was performed at the project main site in Finland. The second round of interviews, altogether 3 interviews, took place at one of the distributed sites, in Greece. One researcher performed all the interviews in Finland and the other researcher interviewed in Greece. The interviewer took detailed notes. In addition, all interviews were recorded. The interviews were relatively loosely structured and conversational in order to maintain adaptability to the roles and individual experiences of the employees in different roles. The interviewees were asked to describe their own experiences of the usage of Scrum, the Scrum practices used and the experienced successes and challenges in the project related to the usage of Scrum. In the interviews, the issues were dealt with both retrospectively and focusing on the current situation during the time of the interviews.
IV. C ASE P ROJECT A. Case Organization The case company, Nokia, is one of the major players in the global telecom industry with significant R&D activities. Most of the product development projects it performs are globally distributed and involve both software and hardware development. Earlier, the company used a traditional, waterfall type, stage gate model. A few years back, the company started to adopt agile methods in software development. The case project was one of the first large projects to adopt Scrum. In this case study we discuss only software development, even though the project was part of a larger program, which also involved hardware development. The majority of the work carried out in the project, however, was software development. B. Reasons to Change Management promoted Agile and the product was considered perfect for trying out Scrum: a totally new product with unclear requirements and potentially close collaboration with the first few customers. Traditionally, in the telecom industry, new software versions of this kind of systems products were released approximately every two or three years. However, the market had recently started to change, the most advanced telecom operators were not ready to wait for new products and features as long as they had used to. The case organization faced constant pressure to demo new functionality to the customers. It was the product development strategy of our company [...] that we clearly wanted to promote agile. We saw that it will improve our productivity, cost efficiency and our ability to respond customer requests flexibly. — Manager
D. Data Analysis All interviews were transcribed by a professional transcription company. The transcribed interviews were analyzed by
We had two reasons [for adopting Scrum]: Requirements were unclear and time pressure was already very high. It was clear
76
the first one, they had added a couple of development teams also in Greece. 1) Scrum Teams: The basic element of the organization structure was 6-10 person cross-competence teams, each including developers, testers and a documenter, who could be shared by two teams. In addition to these cross-competence teams located in Finland and India, special CI team worked on the continuous integration system, testing environments and test runs (in Finland), and a few verification teams worked on network verification and performance testing (in Germany and Greece). 2) Product Ownership: In addition to the Product Owner (PO), there were 9-10 Area Product Owners (APOs) responsible for different product areas. Each APO role was filed in by two persons, a System Architect and a Solution Architect. The Solution Architect APO was a representative of the product management organization, while System Architect belonged to the R&D organization. The idea was that the product would be divided into requirement areas and each APO pair would work with a few teams specializing in their specific area. At the time of the interviews all System Architect APOs were located in Finland, as well as most of the Solution Architect APO. In Greece, a couple of Solution Architect APOs were recently transferred to this project. Thus, the APOs had to communicate through teleconferencing with the teams located at other sites. 3) Software Architecture Team: The project had a Software Architecture Team consisting of 9 high level technical specialists, whose main task was to support the cross-competence teams by planning the software architecture, helping teams to plan new features, and taking care of all kinds of challenges and answer questions, so that the teams could concentrate on their work. At the time of the interviews the Software Architecture Team was located in Finland. 4) Virtual Maintenance Team: The Virtual Maintenance Team led by two maintenance managers took care of the trouble reports coming from both external customers and from the internal platform organization. Before the customer requests came to the Virtual Maintenance Team the Technical Support Organization filtered the requests that did not need to be addressed by the R&D organization. Every cross-competence team named 1-2 members to the Virtual Maintenance Team. 5) Managers: Line managers were responsible for people and monitored daily that everything was working in the teams. They were the direct superiors of the team members. Many line managers had a double role, e.g., as a software architect or as a Scrum Master. The project leaders: the Program Manager, the PO, the Project Manager, and the R&D Manager were all located in Finland.
that we needed it quickly, even though we did not know exactly what. [...] If you wait until you have a set of requirements ready, in waterfall it takes three years. This was a possibility for me to try out Scrum. — Manager
C. The First Steps The project started as a collocated project in Finland. In the beginning it had circa 15 developers in 2 teams that were formed by combining and mixing two agile teams from two previous projects. During the first couple of months these teams started to build the product while learning how to work according to agile. Subsequently, a couple of more teams were added, and the members from the two original teams coached the new teams. The first distributed site was added five months into the project by hiring over 40 developers organized into six teams at a subcontractor in India. The first Indian feature team came to Finland for hands-on product training, as well as for basic Scrum training. The company had resources with testing expertise in Germany, as well as in Greece, thus these sites were added when more resources were needed. The Greek site had tested the previous product, so the domain was familiar. An external Scrum consultant, one of the creators of the LeSS framework, Greg Larman, was invited to help quite early in the project. He, for example, gave a one-day Scrum training in the beginning and lead the first one-day requirements workshop. Thus, following basic practices of the LeSS framework came naturally. The project adopted the Scrum process using two week Sprints. However, the project did not have active internal or external coaches that could have helped the teams in their daily work in the beginning. Later on they got an internal coach who had been a member of the first Scrum team. During the first year of the development the program got the first key customer, an operator, with whom they started to collaborate closely regarding the requirements. After a bit more than one year of development the customer received the first working version of the software for testing with limited functionality. After that monthly deliveries started to a couple of key customers. During that second year of development a few more customers took the product for test usage. The first live pilots for a limited set of end customers started after 2,5 years of the start of the development. The organization aimed that after these pilots they would have two main software releases per year and two to six maintenance releases, that would contain just fixes but no new functionality. Just before our interviews there had been approximately one release per month for the main customers. The first global release came out between our two interview rounds. D. Organization Structure
V. S CALING P RACTICES
By the time of our first interviews the project had grown during the 2,5 years since the project start-up from two teams to 20 teams (altogether over 170 persons) located in four countries: Finland (10 development teams), Germany (2 testing teams), Greece (2 testing teams) and India (6 development teams). During our second interview round, a few months after
We identified five used for coordinating between multiple teams: Area Product Owners, Common Sprint Planning, Scrum-of-Scrum meetings, Common Demo and Common Retrospective. Next, we present how the case organization used each of these, the challenges they faced, and steps they took to
77
improve the practices over time. Finally, we briefly compare these to suggestion made by the LeSS framework.
Software Architects, split the features into user stories, each user story being one line in the backlog. The idea was that each product area would have a couple of development teams, and each feature would be implemented by a specific team. However, at the time of the interviews the situation looked somewhat chaotic. Because of the tight schedule, user stories from some features were given to several teams that worked on them concurrently. The teams received work from several APO’s for the same iteration. One System Architect APO even mentioned that at most he had been working with eight teams simultaneously. The APOs could suggest which team would be preferable for implementing each new feature, but the preferences could not always be realized, since the best teams were requested by several APOs.
A. Area Product Owners The LeSS framework [31] suggest using Area Product Owners (APOs) for scaling the Product Owner role, a practice adopted by our case project. Thus, in addition to the main PO role, the project had a team of APOs, each APO role filled by two persons: a System Architect and a Solution Architect. Each APO pair was responsible for features in one specific product area such as charging and policy enforcement, and worked with the teams developing those features. The Solution Architect APOs were the interface between the customers and the R&D organization, even though they were not normally in direct contact with the customers, but with the market area representatives in the product line organization.
In the worst case, eight different teams were implementing my features, so it was quite a lot running around. [...] It was a couple of features, the other one being a big system level feature that due to tight schedule we had to divide [to several teams] so that we could do different pieces at the same time. — System Architect I can suggest which team would be the best for implementing a feature, but it does not always happen that way, as that team is often good in general, and thus many other APOs would like to use it. [...] There’s kind of internal competition for these few teams that we know are good and that know some area better than others. — System Architect
We act as the interface between the R&D and the customer for that feature. So we clarify whenever R&D has a question, [...] and if we cannot decide, we contact the customer. — Solution Architect
The System Architect APOs worked closely with teams, as well as communicated with the Solution Architect APO, who did not actively work with the teams. At the main site in Finland, the System Architect APOs were located on the same floor as the teams, but the Solution Architect APOs were located in another building, as they belonged to the product management organization. Requirements came through Solution Architect APOs, one feature always through one Solution Architect. As one feature often touches several product areas, the case organization had to adjust the suggested APO model. Thus, the System Architect APO for one feature might be one or several persons depending on how many product areas the new feature touches. A small feature might touch one area but a big feature might touch almost all areas. The first step APOs were involved in when a new feature was planned was the Feasibility Study that the organization recently had started to do for all features. Previously, the study was done only for bigger features. System Architects, supported by Software Architects took care of the Feasibility Studies. One study could take from a few hours to a few weeks. The main purpose of the study was to provide planning with information on the complexity of the feature and the amount of work required, as well as some technical details. Based on the study, product management negotiated with the customers whether the feature is to be implemented, and how to prioritize it. At the time of the interviews, the majority of the features for which the Feasibility Study was done were decided to be implemented. At the time of the interviews the project had a common backlog in the form of an Excel sheet. Each Solution Architect APO was responsible for the backlog items in his area and for suggesting how to prioritize them. The PO was responsible for prioritizing the features. One of the APO pair, normally the System Architect APO updated the backlog regarding team progress. The System Architect APOs, with the help of the
The collaboration and communication between the APOs and the teams had been challenging and was not working well. Some improvements were implemented, e.g., the System Architect APOs had started to arrange Requirements Workshops and Design Workshops on each user story before the sprint planning, as needed. The idea of the Requirements Wordkshops was that the customer requirement was explained in detail, as well as how the System Architect and the Software Architect ha planned that requirement to be implemented at a high level. Each feature was split into several user stories. Ideally, a user story would be small enough for the team to implement during one sprint. In reality, the user stories were often so big that they took approximately three sprints to implement. In the case of small user stories, a team might be able to implement two or three in one sprint, all of which would be elaborated in a single requirements workshop. In a Design Workshop, ateam would plan, with the help of the architects, how to implement the user story and its main dependencies. For small or otherwise straightforward user stories, the Requirement and Design Workshops were combined. We explain [at a] higher level what the customer really wants to do, what this feature is about and then what the team has to do. [...] We [APO pair] write the user stories, some high level user stories and then during this workshop ideally we have time to break down those user stories into sub-user stories that are more technical and more detailed. Then we clarify whatever questions the team has. — Solution Architect
Our interviewees considered both workshops as very useful. After adopting the workshops, daily communication between the APO’s and the teams improved significantly and was considered good enough by the interviewees. The team members
78
felt that due to the workshops they now had a say in the design discussions.
allowed to go there, but normally we do something more useful. — Team member
For example, in the requirements workshop you can right away get onboard, starting from the requirements, and if there is something you want to give your opinion on, it is possible. — Team member
Some of the interviewed team members saw the common sprint planning as a waste of time, while some felt it was useful to get visibility to what other teams were doing and thus saw the session as useful.
However, some team members felt that there was so much hurry that often the features and user stories were not planned well enough in advance in these workshops, which caused surprises while implementing, e.g., some things were just forgotten while planning and they might pop up like new requirements during implementation. The organization had been planning that in each area there would be a few teams specializing in that area. Partially, this had been realized, as they had tried to divide the work to the teams based on their previous knowledge. However, at the time of the interview, they did not have teams assigned to specific areas. The product was large and complicated and our interviewees reported that each feature often touched several areas.
C. Scrum-of-Scrums Meeting The Scrum-of Scrums meetings (SoS) were arranged daily after the teams had had their own Daily Scrum sessions. Normally, one representative from each team participated in the daily Scrum-of-Scrums. In the beginning, when all the teams were located at the main site, they had only one common SoS meeting. As the number of teams grew when the Indian site joined, they first tried out to include India as a teleconference, but it did not work out well. Thus, the meeting was divided into two parts. The ”Finnish SoS” was arranged as a face-to-face meeting and the ”Global SoS” as a teleconference among all four sites. The project manager was the facilitator of the SoS meetings and participated in both meetings. The project manager was the messenger between the meetings. He brought to the Global SoS information gathered from the Finnish SoS that was arranged first. Finally, the project manager sent an email to everybody summarizing the things brought up in the SoS meetings. At the time of the interviews, the SoS meetings were very short, each of them taking 5-15 minutes only. In the beginning, when the project had only a few teams, each team had answered to all the three Scrum questions regarding their own team. However, when the number of teams grew, the organization ended up asking the teams to report only the impediments they had faced, or if they had started to do something new, especially if that might affect other teams. After the SoS meetings, some teams had the team representative give the highlights to the other team members. However, unfortunately not all teams used this practice. One challenge of the SoS meetings was that many teams just stated “nothing to report”, even though that might not have been true. Teams did that just because it was easiest for them. Thus, many of our interviewees felt that these meetings were not very useful in their current state.
B. Common Sprint Planning In the beginning, when there was only one site, all teams performed sprint planning in the same space concurrently. This was was considered good, as you could hear what the other teams were doing and easily discuss with them. However, when the project grew and sites were added collocation was no longer possible. Sprint planning started by having a one-hour meeting that all sites participated in through teleconference and screen sharing. This was called ”Common Spring Planning”. The PO and the program manager led the meeting and each team sent a representative. First, some general information was shared, e.g., the PO told news about the market situation. Then, the user stories for the sprint were briefly presented and assigned to the teams. Afterwards, the teams moved to their team spaces for detailed planning. In the evening each team sent an email of their commitments to the PO and the program manager. The program manager updated the backlog, i.e., an Excel sheet based upon these. Common sprint planning, like all of the ”common meetings” were open to everybody in the project. However, most teams sent a single representative to the meetings. Some teams sent their Scrum Master to all common meetings, whereas other teams had a rotating system, in which each team member took turns in participating in the common meetings, e.g., for a week at a time:
Most of the time everybody goes there [SoS meeting] and just says they’ve got no problems. I think originally it started [...] that people gave a real report about what the team was up to. But they just took too long, because we’ve got so many teams here. So now all you need to say is whether you have any problems or not. If you have any problems, then you tell what the problem is. But otherwise, if you have no problems, you just say no problems. [...] most teams don’t have any problems when they go there, because they don’t want to say anything [...] it seems to me like people don’t want to talk so much at these things. — Team member That is a place to tell about problems, but it would be good if people would really tell about problems there. Sometimes it feels like that everybody has ”no problems” and everything goes well, but then it reveals that this and this does not work. [...] Then many fight with the same problem at the same time. — Team member
In our team, we have a turn-based system. So, one week one [team member] goes, then the next week another [team member] goes. Also for the demos at the sprint planning, we have an order which I think is quite good. I notice that in some teams for example the Scrum Master goes to every SoS, every demo. But I think it’s better the way we do it, that we rotate. Even, okay, we’re eight persons, so you participate in the sprint planning and demo once in three months or something. But I think it’s nice. — Team member It’s normally only the Scrum Master [who goes to the common sprint planning], not team members so much. Of course you are
79
Another challenge was that all sites and all project members did not participate in the SoS meetings (at least not at the same time), thus solutions to the problems brought up by the teams were not always found in the meeting. One interviewee even suggested that just sending email to the project’s global mailing list saying ”Hi, we have this problem, could somebody help?” would be more useful, as that would reach everybody. Despite the problems, some of the interviewees felt that the SoS meetings were useful:
get a feeling of how it works and perhaps you could ask a couple of questions during the demo, ”What about this case and what about the other one?”. But how can it happen with 20 teams? — Solution Architect
Some even felt that the common demos were arranged just for the managers to follow progress and did not see any benefit from those to the teams, while the managers had an opposite idea. We go there through the wiki pages, every team says in their turn that ”this user story is ready and this is not” and so on. [...] there is just a lot of user story names and codes, so you cannot really get a lot out of that. So it feels quite unnecessary. Maybe the managers think it is useful, but all that information can be read from the wiki. — Team member There has been a lot of discussion: ”Do we need this review at all?” Team representatives get bored easily. Still, every time there is a discussion, for example, ”we are waiting for input from that team” and the other team says ”we have never heard about that”. [...] I want to give the other teams a view on what the others are doing. I think this is useful. — Manager
Sometimes it’s [SoS] useful. It could happen that you go and nobody has anything to share. But sometimes people just tell what problems they have. If they notice something during the morning that some problem occurred, then actually you can get quite fast feedback. Maybe you don’t even know who did something, but you can go there and say ”Okay, yesterday this worked, this morning it doesn’t work”. Quite often somebody from the other teams says ”Oh, we made this change last evening, maybe it’s that”, and then check. Basically, these daily Scrum stand-ups, whatever they are called, I think are quite useful. — Team member It’s every morning and it works well, there are the biggest problems and normally some help can be found for the problems. [...] or at least how to start attacking the problem. [...] I think it works, it’s short and simple as it should be. — Team member
Even though these common demos were open to all, only a few team members, except those presenting, participated. Most teams had a rotating team member presenting and participating. Some just sent their Scrum Master. Despite the criticism some team members liked to see what was going on in the other teams and hear Product Owners comments.
D. Common Sprint Demo The organization had tried several ways to arrange sprint demos. In the beginning, when the project had only a few teams, common sprint demos were held in a big auditorium for all teams. In this demo each team arranged a real demo and showed real test cases and test results. When the project grew, they created a new practice: the program manager and PO visited each team, and the team gave a demo to them and any other interested people. At the time of the interviews the project had moved back to having a common sprint demo meeting of two hours in a big auditorium at the main site with connection to the distributed sites. However, instead of giving a real demo, a representative of each team gave a short slide presentation of the team achievements. Before the ”Common Demo” each team collected to a wiki page all important information regarding the review:
It’s good to go there [to the common demo] to listen what the Product Owners think [...] There you can hear what the other teams have done and you can see how disappointed the Product Owners are or... Well, you can feel that there. Anyway, it’s nice to listen. Especially what the other teams have done and what the biggest problems have been, as they normally mentioned there. — Team member
At the time of the interviews there were plans to move back to having real demos in team spaces. The audience of the team demos would include the APO, who has worked with that user story or feature, as well as other interested stakeholders. This arrangement would enable the APO to give the team detailed feedback. Some team members were already looking forward to the new system and felt that this might be a singificant improvement, even though they would loose the visibility to the other teams. So the idea is that we will give the APOs a team specific demo. I would think that could be a good system, because then they can concentrate more on one team and ask more detailed questions. [...] For the team it’s good to receive feedback from the APO, that ”is that what he wanted from the user story.” But then it does not help in getting information from the other teams. But I don’t know if that would be so necessary after all, at least I have not needed so much to know what the other teams are doing. We have so many teams anyway that you cannot possibly follow them all. — Team member
Nowadays, each team has a wiki page, telling how the sprint went, and what was achieved. And now we just explain that [in the common demo], but don’t really demo the functionality or how to test it. — Team member
They called this a ”Common Demo”, but a better term to describe this would be a common sprint review. Each team had around 10 minutes to present. This way of ”demonstrating” was hugely criticized by the interviewed team members: they felt that a presentation did not show whether the software was good or not, and that you even could cheat—tell that something worked even though problems remained. Moreover, in a large common session there was little possibility for discussion. Thus, the teams felt that they did not get enough feedback.
E. Common Retrospective The organization had tried a few ways to arrange common retrospectives, as well. In the beginning, each team had their team specific retrospective first, after which everybody gathered for a common retrospective. Each team was advised to bring three issues that are too big for their own team to
We don’t do any demos now. [...] It is needed in the sense that you get trust that it works and that you see it working and you
80
hypothesize over the reasons why the chosen scaling practices did not seem to work well enough in this organization. Finally, we discuss the limitations of this study and give suggestions for future research.
solve to the common demo. Three of all those issues brought by the teams were then chosen for discussion and solutions were drafted together. The problems were often very big, such as “communication between the APO’s and the teams is not working”, thus they could not be solved during one iteration, and the implementation of the solution was not followed up. Therefore, our interviewees felt that no real improvements followed from the common retrospectives. Huge problems remained, and common retrospectives were considered useless. Since participation was voluntary, the team members voted with their feet, and finally the meeting had only very few participants.
A. Comparison to the LeSS Framework The case organization aimed to implement the LeSS framework. Table II compares the agile scaling implementation of our case organization to the main points suggested by the LeSS Huge framework. As the number of teams clearly exceeded eight, the LeSS Huge framework seemed suitable for this comparison. In the table we can see that the main difference comes from the requirement areas that the case organization had implemented in principle, but as many features crossed several areas the implementation of the requirement areas did not fully take place in practice. As areas did not have named teams, and the number of teams per area would have been lower than the suggested 4-10 teams per area, the common meetings would not have been practical to arrange per area as suggested by the framework. Another difference was the division of the Area Product Owner roles in the case organization to two persons. Even though different from the LeSS model, this pair work seemed to suit well to this organization and thus might be something to try out in other organizations as well. The LeSS framework emphasizes the importance of coaching at three levels: at the organization, team and technical levels. Even though our case organization could use the company internal group that provided training and coaching, the coaching provided seemed to be quite limited. For example, the team level and technical coaching were almost totally missing and organization level coaching did not seem to have enough resources. This was clearly a pain point. LeSS suggests the use of Communities of Practice (CoPs) for inter-team coordination and learning, e.g., component CoPs and Scrum Master CoPs. CoPs are based on self-organization and participation is voluntary. Our case organization did not have any communities. One reason for this might be that selforganization seemed to be lacking, as the organization culture seemed to still be close to the previous waterfall mindset.
People actually maybe don’t see what’s the value of being there, because we discuss, then we decide something, and if you don’t see the results, then maybe you are not so inclined to come again. [...] then we say this needs to change, but in the end nothing happens. [...] One thing that will bring more people is that they see that actually there are decisions made there and this affects their work. [...] whoever is there, gets the right to decide, so if they see that actually this affects their daily work. — Team member
The next trial was to replace the common retrospective with “an open space”, where anyone could suggest discussion topics. Several discussions took place simultaneously. Our interviewees commented that that was fun, a lot of discussions around flaps, but again the same problem remained: the discussions did not lead to any real changes. The more time pressure experienced in the project, the less people participated in the common retros. However, some of the interviewees commented that those persons who actively wanted to change things came, which was seen as positive. All the time the number of participants went down. You could notice that the more time pressure we had [in the project], the less people came. [...] So, now no more than ten people show up in the common retros. [...] But I have been going there, as there’s always something.... [...] There has started to be people that are really exited about changing things. So it’s kind of motivating that here are people who are still interested in things. — Team member It’s important and it’s interesting for me to go there and see. [...] I think that when many people just gather, then you actually start knowing what are their problems and what they face day to day. So, I think it’s also important — Team member
At the time of the interviews the common retrospective was back again, but in a new format and the first new type of a common retrospective meeting was held. The meeting was facilitated by an internal coach, who tried to avoid the earlier mistakes. In this new format the participants first brainstormed the biggest impediments, prioritized them, chose the most important one, searched root causes for that, chose one root cause for which they brainstormed solutions, chose one solution and finally drafted an implementation for that. Later on, in the next common retrospective they would follow up on the implementation of the solution and possibly continue working on it until it was permanently solved.
B. Why did the scaling practices not work well enough? The implementation of the scaling practices: Area Product Owners, Common Sprint Planning, Scrum-of-Scrums, Common Demo and Common Retrospective were clearly not working well at the time of the interviews. Many of our interviewees, especially team members, complained that the “common” meetings did not give a good big picture or help enough in coordination between the teams. For some it was not even clear why these meetings were arranged and they considered participation as a waste of time. Some teams sent their Scrum Masters to these meetings, instead of a rotating team member, since they felt that team members had other more important tasks to do. Some team members even thought that these meetings were mainly arranged for the higher level
VI. D ISCUSSION In this Section we first compare the scaling practices identified in our case organization to the LeSS framework, then we
81
TABLE II C OMPARISON TO THE L E SS H UGE F RAMEWORK LeSS Huge
Implementation in the case organization
Over 8 feature teams Requirement areas Prioritized area backlogs Each area having 4-10 feature teams Area Product Owners Sprint Planning 1 per requirement area Scrum-of-Scrums 2-3 times per week for those needing to coordinate Sprint review per requirement area Retrospectives both per area and product level Dedicated training and coaching department Communities of Practice
15 cross-competence teams + 5 testing teams Requirement areas, but features crossed several areas One common backlog Attempt to divide teams to areas failed Area Product owner roles filled by two persons Common sprint planning part 1 (not area specific) Scrum-of-Scrums every day, in two parts Common sprint review (not area specific) Product level common retrospective (not area specific) Company internal group provided some training and coaching No communities formed
to deliver constantly new features. All this pressure caused that they did not have enough time and attention to concentrate on creating a common view of what Scrum is and how to customize it to suit to their organization. Without common trainings people had different views even on what Scrum was. They did not have a common view on why things were done the way they were done (e.g. common meetings arranged). Nor did they have enough coaching resources to help in continuous improvement inside the teams or when improving the scaling practices. 4) Constant market pressure causing time pressure: Market pressure to win new customers and pressure to deliver all that was promised to existing customers was high. Due to time pressure, user stories from a single feature were often distributed for implementation to several teams, even to different countries and APOs competed for the best teams. Everybody in the project felt the time pressure. No wonder that the team members started to skip ”unnecessary” common meetings, as they had high pressure to deliver user stories. They did not see that the lack of coordination and communication might cause additional challenges in the future, such as bugs or solving the same problems simultaneously in different teams.
managers, like the PO and the program manager so that these persons could follow what was happening in the project and that teams were reporting to them in these meetings. On the other hand, the managers had a totally different view. They saw that the main purpose of these “common” meetings was sharing information between the teams, so that the teams could notice early enough when there was a need to coordinate and communicate between the teams. According to the managers the other main goal of these meetings was to give a big picture on what was happening in different parts of the project to the teams. When looking at the project from the outside, we spotted four pain points: 1) The agile mindset was partially missing, 2) The product was difficult to divide into reasonable requirement areas, 3) The lack of a common view of the Scrum implementation, and 4) Constant market pressure causing time pressure. 1) Agile mindset partially missing: It seemed that the organization had not yet adopted the agile mindset properly. Namely, the managers, including the APOs were micromanaging the teams, e.g. splitting the user stories, and software architects were designing the features instead of the teams. APO’s, software architects and managers also took care of the main part of the inter-team coordination. Thus, teams did not have to broaden their responsibilities. According to LeSS, feature teams should carry the main responsibility of these tasks by themselves. As the teams did not have to take care of the inter-team coordination, it was no wonder that many team members did not find the different kinds of common meetings useful, but thought the purpose of these meetings was mainly informing the managers, who still took care of the coordination, like in the old waterfall model. 2) Product difficult divide into reasonable requirement areas: The idea of requirement areas, in each of which a few teams would be working with the help of one APO was not a reality in this project. The case organization found the product difficult to divide into requirement areas. Many features crossed several areas, which made it difficult to work according to LeSS Huge framework and give one feature to one requirement area only. 3) Lack of common view of the Scrum implementation: At the same time with organizational growth the case project had
VII. C ONCLUSIONS We presented a case study on the application of the LeSS framework to a large, distributed agile software project. While commercially a very successful project, it experienced significant issues related to scaling agile despite the usage of the LeSS framework. This naturally leads to the question of whether the problems were due to inherent problems in the framework itself, or due to it being poorly implemented. We think that both aspects played a role. First, the product itself was very complex, and had significant dependencies between product areas, making it a poor match with a model based on (largely) independent product areas. In practice, a single feature could cross several or even all product areas, thus one area specific team would not have been able to implement a full feature as suggested by the LeSS framework. Thus, the idea of having area specific meetings, as prescribed by LeSS Huge did not work, and the organization had to keep project-wide meetings instead. These meetings
82
became too big for in-depth discussions to take place, and as a result were not considered useful by all the team members. Thus, the LeSS framework worked less than optimally in such a complex setup. Second, the implementation of agile in the project was lacking. In particular, the project did not succeed in attaining a real ”agile mindset”, and did not adopt all important practices suggested by the LeSS framework like coaching or communities of practice. Inter-team coordination was insufficient, and the experienced time pressure led to practices not being taught properly, being skipped, or implemented poorly. From the point of view of the LeSS framework, it seems that this case points to the limits of its applicability; it is better applicable to products that can be clearly split into fairly independent product areas, whereas complex products with many technical interdependencies might require other approaches to scaling. The study has limitations. We performed 19 interviews at two of the four sites. Due to time and traveling restrictions most of the interviews took place at the main site, which was also the biggest site where the managers and product experts resided. Thus, this site was not that dependent on communication with the other sites. Interviewing additional people at other sites might have given additional insights. With respect to future research, we encourage other researchers to conduct and report case studies on the usage of different scaling frameworks. That way we can provide practitioners objective pieces of advice on what kind of scaling frameworks and practices are suited to different situations and what the related challenges and successful practices are when scaling agile to large and distributed organizations.
[10] I. Gat, “How bmc is scaling agile development,” in Agile Conference, 2006, 2006, pp. 6 pp.–320. [11] J. Goos and A. Melisse, “An ericsson example of enterprise class agility,” in Agile, 2008. AGILE ’08. Conference, aug. 2008, pp. 154 –159. [12] S. McDowell and N. Dourambeis, “British telecom experience report: Agile intervention - bt’s joining the dots events for organizational change,” in Agile Processes in Software Engineering and Extreme Programming, Proceedings, ser. LNCS, vol. 4536, 2007, pp. 17–23. [13] B. Schatz and I. Abdelshafi, “Primavera gets agile: a successful transition to agile development,” Software, IEEE, vol. 22, no. 3, pp. 36 – 42, mayjune 2005. [14] H. Hajjdiab, A. Taleb, and J. Ali, “An industrial case study for scrum adoption,” Journal of Software, vol. 7, no. 1, pp. 237–242, 2012. [15] K. Korhonen, “Evaluating the impact of an agile transformation: a longitudinal case study in a distributed context,” Software Quality Journal, vol. 21, no. 4, pp. 599–624, 2012. [16] S. Freudenberg and H. Sharp, “The top 10 burning research questions from practitioners,” Software, IEEE, vol. 27, no. 5, pp. 8–9, Sept 2010. [17] T. Dingsøyr and N. Moe, “Research challenges in large-scale agile software development,” SIGSOFT Softw. Eng. Notes, vol. 38, no. 5, pp. 38–39, Aug. 2013. [18] C. Larman and B. Vodde, Practices for Scaling Lean & Agile Development: Large, Multisite, and Offshore Product Development with LargeScale Scrum, 1st ed. Addison-Wesley Professional, 2010. [19] D. Leffingwell, Scaling Software Agility: Best Practices for Large Enterprises (The Agile Software Development Series). Addison-Wesley Professional, 2007. [20] S. Ambler and M. Lines, Disciplined Agile Delivery: A Practitioner’s Guide to Agile Software Delivery in the Enterprise. IBM Press, 2012. [21] M. Paasivaara and C. Lassenius, “Scaling scrum in a large distributed project,” in Empirical Software Engineering and Measurement (ESEM), 2011 International Symposium on, Sept 2011, pp. 363–367. [22] J. Highsmith and A. Cockburn, “Agile software development: the business of innovation,” Computer, vol. 34, no. 9, pp. 120–127, 2001. [23] A. Cockburn and J. Highsmith, “Agile software development, the people factor,” Computer, vol. 34, no. 11, pp. 131–133, Nov 2001. [24] A. Hamed and H. Abushama, “Popular agile approaches in software development: Review and analysis,” in Computing, Electrical and Electronics Engineering (ICCEEE), 2013 International Conference on, Aug 2013, pp. 160–166. [25] K. Schwaber and M. Beedle, Agile software development with scrum, ser. Series in agile software development. Prentice Hall, 2002. [26] T. Dyb˚a and T. Dingsøyr, “Empirical studies of agile software development: A systematic review,” Information and Software Technology, vol. 50, no. 9-10, pp. 833–859, 2008. [27] J. A. Livermore, “Factors that significantly impact the implementation of an agile software development methodology,” Journal of Software, vol. 3, no. 4, pp. 31–36, 2008. [28] S. C. Misra, V. Kumar, and U. Kumar, “Identifying some critical changes required in adopting agile practices in traditional software development projects,” International Journal of Quality and Reliability Management, vol. 27, no. 4, pp. 451–474, 2010. [29] M. Lindvall, D. Muthig, A. Dagnino, C. Wallin, M. Stupperich, D. Kiefer, J. May, and T. Kahkonen, “Agile software development in large organizations,” Computer, vol. 37, no. 12, pp. 26–34, 2004. [30] M. Cohn and D. Ford, “Introducing an agile process to an organization [software development],” Computer, vol. 36, no. 6, pp. 74–78, 2003. [31] C. Larman and B. Vodde, Scaling Lean & Agile Development: Thinking and Organizational Tools for Large-Scale Scrum. Addison-Wesley Professional, 2009. [32] “Agile scaling knowledgebase decision matrix,” February 2016. [Online]. Available: http://www.agilescaling.org/ask-matrix.html [33] R. K. Yin, Case Study Research: Design and Methods, 4th ed. Thousand Oaks, CA, USA: SAGE Publications, 2009. [34] M. Q. Patton, Qualitative evaluation and research methods, 2nd ed. Newbury Park, Calif.: Sage Publications, 1990.
R EFERENCES [1] B. Boehm and R. Turner, “Management challenges to implementing agile processes in traditional development organizations,” Software, IEEE, vol. 22, no. 5, pp. 30–39, 2005. [2] T. Dyb˚a and T. Dingsøyr, “What do we know about agile software development?” Software, IEEE, vol. 26, no. 5, pp. 6–9, 2009. [3] VersionOne, Inc, “7th annual ”state of agile development” survey,” 2013. [Online]. Available: http://www.versionone.com/pdf/7th-Annual-State -of-Agile-Development-Survey.pdf [4] M. Paasivaara, C. Lassenius, V. Heikkila, K. Dikert, and C. Engblom, “Integrating global sites into the lean and agile transformation at ericsson,” in Global Software Engineering (ICGSE), 2013 IEEE 8th International Conference on, Aug 2013, pp. 134–143. [5] M. Paasivaara, B. Behm, C. Lassenius, and M. Hallikainen, “Towards rapid releases in large-scale xaas development at ericsson: A case study,” in Global Software Engineering (ICGSE), 2014 IEEE 9th International Conference on, Aug 2014, pp. 16–25. [6] T. Dingsøyr and N. Moe, “Towards principles of large-scale agile development,” in Agile Methods. Large-Scale Development, Refactoring, Testing, and Estimation, ser. Lecture Notes in Business Information Processing, T. Dingsøyr, N. Moe, R. Tonelli, S. Counsell, C. Gencel, and K. Petersen, Eds. Springer International Publishing, 2014, vol. 199, pp. 1–8. [7] R. Nyman, I. Aro, and R. Wagner, Agile Processes in Software Engineering and Extreme Programming: 11th International Conference, XP 2010, Trondheim, Norway, June 1-4, 2010. Proceedings. Berlin, Heidelberg: Springer Berlin Heidelberg, 2010, ch. Automated Acceptance Testing of High Capacity Network Gateway, pp. 307–314. [8] A. Atlas, “Accidental adoption: The story of scrum at amazon.com,” in Agile Conference, 2009. AGILE ’09., aug. 2009, pp. 135 –140. [9] S. Hanly, L. Wai, L. Meadows, and R. Leaton, “Agile coaching in british telecom: making strawberry jam,” in Agile Conference, 2006, july 2006, pp. 9 pp. –202.
83
