Agile Software Development and Innovation: a ...

This paper was presented at The 6th ISPIM Innovation Symposium – Innovation in the Asian Century, in Melbourne, Australia on 8-11 December 2013. The publication is available to ISPIM members at www.ispim.org.

Agile Software Development and Innovation: a Systematic Literature Review Tomi Juhola* University of Turku, Business and Innovation Development, Turku, Finland. E-mail: [email protected]

Sami Hyrynsalmi University of Turku, Turku School of Economics, Department of Management and Entrepreneurship, Turku, Finland. E-mail: [email protected]

Ville Leppänen University of Turku, Department of Information Technology, Turku, Finland. E-mail: [email protected]

Tuomas Mäkilä University of Turku, Business and Innovation Development, Turku, Finland. E-mail: [email protected] * Corresponding author Abstract: Agile software development methods have been argued to enable innovation in software development. However, the existing research is scattered and further work would benefit from a common baseline. This study presents a systematic literature review on the broad theme of agile methods and innovations. By using an automatic search strategy for ten databases and complementing this search by a manual search strategy for the publication lists of the most active researcher in the field, 43 relevant articles were identified and included in this study. With a thematic synthesis, seven recurrent themes were identified. The results show that while the adoption of agile methods as innovations is well-understood, the agile methods effect on the innovation capabilities of a development team needs further work. Furthermore, the article shows that manual search strategy focused on the publication lists of the active authors is useful in literature studies on dispersed and little researched topics. Keywords: Agile software development; innovation management; process improvement; collaborative innovation; innovation capability; product innovation; systematic literature review.

1

This paper was presented at The 6th ISPIM Innovation Symposium – Innovation in the Asian Century, in Melbourne, Australia on 8-11 December 2013. The publication is available to ISPIM members at www.ispim.org.

1 Introduction Agile software development (ASD) methods have transformed from a practice of enlightened professionals to mainstream software development practices during the last decade. ASD methods have been marketed with an ability to react changing environment, enhanced customer collaboration and with improved innovation capabilities. Since the breakthrough of the methodology, a plethora of studies has researched ASD from various viewpoints (see e.g. Dyba & Dingsoyr 2009; Dingsøyr et al. 2012). However, the empirical evidence supporting the benefits often linked to the methods remains scarce (Dybå & Dingsøyr 2008). This paper studies innovations in the ASD context. Although this topic is widely discussed and used as a selling argument for the method (see e.g. Highsmith & Cockburn 2001; Highsmith 2009), the existing knowledge on the phenomenon remains scattered. To define a baseline for further research on innovation in ASD context, the existing knowledge needs to be mapped. We utilize a systematic literature review to collect the existing studies concerning agile and innovation. We perform a thematic synthesis for the found articles. The review questions, addressed in this study, are: 1.

What do we known on innovations born in agile environment?

2.

Do agile development methods enhance the innovation capabilities of software development teams?

3. How can agile teams’ innovation capabilities be improved? The results show that while there are lots of studies addressing innovations in agile context, number of articles based on empirical evidence is low. We identified seven main themes that occurred often in the literature. While diffusion of ASD is widely addressed, there are only a handful studies on the development of product innovation in ASD context, innovation adaption with ASD methods and enabling innovation with agile methods. The following section will briefly present ASD and systematic literature studies. Section 3 presents the research protocol used in this study and it is followed by the presentation of the results. Section 5 discusses on the findings and gives a proposal for future work while the last section summarizes the study.

2 Background Agile software development research ASD is a common term for a family of modern software development processes. These processes take a different view on the nature of software development work compared to the traditional software development process. The traditional software development processes can be described to be driven by excessive planning and relying on minimizing variance by strict change control and very detailed design. The ASD treats the unpredictability and change as an opportunity. ASD tries to optimize the ability to respond to change, as well as create means for rapid feedback that can be used to improve software, development teams and the ASD process used. The term ‘Agile software development’ has been mainly attributed to Agile manifesto (Beck et al. 2001), even though the term ‘Agile Software Process’ was already mentioned

2

in the work of Aoyama (1998). The manifesto identifies key values and principles for the ASD methodologies and thus describes the family of software development processes with a common ground. A detailed overview of several ASD methods can be found from Abrahamsson et al. (2002).

Systematic literature studies During last years, scholars have demanded for more rigorous research methods and approaches in software engineering research instead of ad hoc studies conducted. Dingsøyr et al. (2012) stated that only a fraction of researches on agile software development rely on known theoretical perspectives. However, there is increasing number of articles presenting more precise research and research methodologies. For example, inspired by the success of evidence-based medicine, Kitchenham et al. (2004) suggested evidence-based software engineering (EBSE) research, which could be beneficial for increasing the body of knowledge in software engineering. To gather evidences, the scholars have used Systematic Literature Reviews. Systematic literature review “–is a well-defined methodology to identify, analyse and interpret all available evidence related to a specific research question in a way that is unbiased and (to a degree) repeatable” (Kitchenham & Charters 2007). It consists of three main phases: planning, conducting and reporting the review (Kitchenham & Charters 2007). In the first phase, research questions are set and a review protocol is defined. In conducting phase, a search strategy is piloted and decided, quality instrument are created as well as data are extracted and synthesized from the selected original publications. In the final phase, the procedure used and the results obtained are reported.

3 Research protocol Review process The data acquiring strategy consists of six phases. In the first phase, the articles were retrieved from the databases. In the second phase, three researchers went through the created list of 5,191 articles and evaluated independently based on titles. The researchers used the inclusion question “Does this article consider innovation research in agile context?” to evaluate the articles. They were allowed to answer with either ‘Yes’, ‘No’, or ‘Maybe’. We decided to left the option of uncertainty for the reviewers due to the phenomena of uninformative abstracts and titles (see Brereton et al. 2007; Budgen et al. 2008). The articles, where all researchers voted for not to include into the study, were excluded. In the third phase, the remaining articles were examined by at least two researchers while answering to the following three questions with ‘Yes’ or ‘No’: 1) “Is this related to software development?”, 2) “Is the research focused on some aspect of innovations?”, and 3) “Is the research done in agile software development context?” according to the title and abstract of the paper. It was required that a researcher answered ‘Yes’ in all questions for the articles to be kept in the study. In the fourth phase, we used the same criteria while reading through the whole paper. After the fourth phase, we analysed the authors of the selected 35 articles. We found 53 unique authors, from which 11 most actively published account for 51 % of selected

This paper was presented at The 6th ISPIM Innovation Symposium – Innovation in the Asian Century, in Melbourne, Australia on 8-11 December 2013. The publication is available to ISPIM members at www.ispim.org.

articles. The scholars, listed in Table 1, who have two or more relevant articles, were selected for further study. We went through both their manually created (e.g. CV and publication lists on personal webpages) and automatically generated (e.g. The DBLP Computer Science Bibliography) publication lists. In the fifth phase, all articles were read through at least two authors. After reading the article, a reviewer may suggest either including or excluding the article. In the latter ones, he should give a reason–most often either “Not an innovation study” or “Not in the agile software development context”–for exclusions. In this phase, we excluded the remaining non-relevant articles. In the final phase, the data were extracted from the selected papers by two researchers: one acted as the main extractor while another verified the extracted data. The reviewers disagreed in 14.5 % (Phase II), 10.5 % (Phase III) and 7.9 % (Phase IV) of the cases. In all phases, the disagreements of researchers were discussed in the meeting of all authors. It was required to reach a consensus was before a decision can be made. The process is illustrated in Figure 1.

Figure 1 The systematic article gathering process used in this study with the number of articles included in each phases is illustrated with the figure. The rectangles depict data sources while round corned rectangles are exclusion phases of the study. The total number of hits in Proquest, marked with *, includes duplicates while the other numbers indicate unique articles. The number of articles found in the first search round is shown on the top of the arrows while the number of articles found from the second search round is shown under.

Data sources and transformations The electronic data sources are shown in Figure 1. Only studies related to innovation research in agile software development context were included in the review. We welcomed publications by academics as well as students and practitioners. Only studies published from the beginning of 2001 to the end of 2012 were included. This ruled out studies done early in the 2013, e.g. (Hannola, Friman & Niemimuukko 2013). We did not included book reviews, workshops, panel summaries, and posters–or studies published in

4

other languages than in English. We did perform the search with all languages resulting into one Portuguese and one German study. We asked English version of the study directly from the authors, but there was no such study available. We targeted the automatic searches to abstracts, titles and keywords. We used the following search terms and logical constructions: 

agile AND innovation



scrum AND innovation



xp AND innovation AND development



extreme AND programming AND innovation



crystal AND innovation AND development AND (clear OR orange OR red OR blue)



dsdm AND innovation



fdd AND innovation



feature AND driven AND development AND innovation

In the manual search, the publication lists for each scholar were searched with Google and checked by two researchers. They selected unanimously a total of 37 articles, based on the title, for a new review round. From these, 16 passed Phase III. In the supplementary search, a book focusing on agile and innovation (Oza & Abrahamsson 2009)–that was not found in automated search–was found. As the book consists of individual chapters written by distinct authors, it was divided into parts by chapters and these parts were included into the review round. Finally, 8 articles from the supplementary search were selected to be included into the study. During the new round, five authors were added to the list of the scholars that have more than two articles included into this study. This time, no additions were done. Therefore, the search was terminated. We ended in a list of 43 selected articles, shown in the appendix.

Quality assessment The included articles were quality assessed using a spreadsheet created based on the Critical Appraisal Skills Programme (CASP) UK checklists and the similar form used by Dybå & Dingsøyr (2008). The assessment was done to each of the included studies in order to identify the quality studies for the synthesis purposes and to follow proposed practices by Kitchenham & Charters (2007). The assessment started with three screening questions that determined if the study is empirical and high quality enough to continue with the assessment. These screening questions were: 1. Is the study empirical? 2. Is this a research paper? 3. Does the study address a clearly focused issue?

This paper was presented at The 6th ISPIM Innovation Symposium – Innovation in the Asian Century, in Melbourne, Australia on 8-11 December 2013. The publication is available to ISPIM members at www.ispim.org.

Table 1 The most active researchers in this field found in the automatic search. The scholars marked with * were addressed in the second round. Scholar

Number of articles included

Conboy, Kieran

7

Pikkarainen, Minna

5

Morgan, Lorraine

4

Aaen, Ivan

4

Senapathi, Mali

4

Koskela, Kaisa

3

Wang, Xiafeng

3

Judy, Ken H.

2

Krumins-Beens, Ilio

2

Näkki, Pirjo

2

Sandmeier, Patricia

2

Järvilehto, Mikko*

2

Kuvaja, Pasi*

2

Oivo, Markku*

2

Similä, Jouni*

2

Srinivasan, Ananth*

2

After the screening questions, and if the study was judged empirical, the following questions were answered: 4. Is there an adequate description of the context in which the research was carried out? 5. Did the authors use an appropriate method to answer their question? 6. Was the recruitment strategy appropriate to the aims of the research? 7. Was there a control group with which to compare treatments? 8. Was the data collected in a way that addressed the research issue? 9. Has the relationship between researcher and participants been considered adequately? 10. Was the data analysis sufficiently rigorous? 11. Is there a clear statement of findings? 12. How valuable is the research? The overall quality of the study was assessed by counting the “Yes” answers on the quality assessment sheet resulting into a quality score. Each study was assessed by two authors and the quality score of the studies was compared. If the quality score differed, then the assessment was discussed and a new common score was agreed.

Thematic synthesis Only the studies judged as empirical research by the pervious quality assessment were included into a thematic synthesis. The thematic synthesis was done following the procedures from Braun & Clarke (2006). The output of this phase was following themes:  Innovation adoption with agile methods,

6

 Diffusion of agile methods,  Innovation process,  Innovative teams,  Product innovation,  Enabling innovation in agile methods, and  Innovation driving practices in agile. The thematic analysis describes the empirical data from each theme in the results section. We added the thematically relevant non-empirical studies into the discussion.

4 Results Quality assessment results The results of the quality assessment are shown in the Table 3. The empirical studies had good results from quality assessment mean being 8 and median 7. None of the empirical studies was very close to the threshold, so the division between empirical and nonempirical studies was very clear. This can also be seen from the frequency diagram (Figure 2). 18 16 14 12 10 8 6 4 2 0 0

1

2

3

4

5

6

7

8

9

10

11

12

Figure 2 Frequencies of quality assessment scores.

Mapping to themes and synthesis of empirical research Table 2 presents the mapping of the studies to the identified themes. Based on the previous quality assessment we dropped non-empirical studies from the synthesis. The resulting studies are synthesised thematically in following the subsections.

This paper was presented at The 6th ISPIM Innovation Symposium – Innovation in the Asian Century, in Melbourne, Australia on 8-11 December 2013. The publication is available to ISPIM members at www.ispim.org.

Table 2 Included publications divided into identified themes. Empirical studies, according to CASP analysis, are bolded. Theme

Related publications

Innovation adoption with agile methods

P07, P41

Diffusion of agile methods

P06, P28, P30, P32, P33, P38, P39, P44

Innovation process

P05, P16, P17, P26, P29, P31

Innovative teams

P01, P02, P03, P15

Product Innovation Enabling innovation in agile methods

P08, P21, P22, P37, P40 P04, P19, P20, P24, P27, P34, P36, P42, P43

Innovation driving practices in agile*

P13, P18, P23, P35

Open innovation*

P09, P10, P11, P12, P25

* Omitted from the synthesis due to lack of empirical evidences

Innovation process Software development is in most cases new product development. Thus the rate and amount of innovation in the end product is an important goal. New methods have been introduced to cost effectively reach these targets, and ASD can be seen as one of these. Gassmann et al. (P17) have studied the customer integration into the innovation frontend derived from the extreme programming (XP). They see the XP approach of constant learning cycles and customer collaboration as a key to product innovation. The study was done as a qualitative case study, and the data was collected from 20 companies in central Europe. In practice, this was done through 48 semi-structured interviews complemented by desk research and analyses of reports, presentations and company journals. In conclusion Gassmann et al. state that “XP captures a vision of highly effective product innovation through integrating technical and market knowledge derived directly from the customer". They also found that XP practices were seen as drivers for innovation in new product development with interview data supporting this. To analyse this further, it seems that at least the people involved in delivering software using XP practices perceive that those practices drive innovation. In order to find innovation barriers in ASD context, Pikkarainen et al. (P29) have studied a European case company that offers a complex system platform. The study was done through nine semi-structured interviews with mainly mid-level managers. The results were analysed by reflecting to each of the innovation steps used by the company: idea collection, idea evaluation, feature analysis and implementation. Pikkarainen et al. (P29) state that the area of innovation in agile context is not really researched. They also interpret that organizational agility is unlikely to be achieved in the case company due to the traditional innovation approach. Finally they conclude that innovation in agile context should be based on regular customer-and-developer interaction, group work and continuous learning.

Innovative teams Ivan Aaen has researched the creativity and innovation in teams, and especially the methods that can be used to make teams innovative (P01, P02, P03, P04). From these, only

8

P02 can be considered empirical research, as in this Aaen describes a method designed through a series of structured experiments done on student projects. As a result of these experiments Aaen states that using team roles (challenger, responder, anchor and child) has potential to improve innovativeness through diversity the roles create. However, there is only small amount of qualitative data available to support this result and the also small amount of quantitative data is not analysed enough.

Product innovation The product innovation was originally the argument in favour of ASD from Highsmith & Cockburn (2001). Highsmith (2009) also highlights this in his agile project management book. Also Ben Werner promotes the innovation benefits the agile principles give for the organizations in his Forbes article (Werner 2013). However, we did not find any empirical data on this in general. Instead we only found one study, which is very concentrated on a niche market. In P08, Campos argues that ASD is especially effective in creation of innovative interactive installations. The phenomenon is studied following an interpretive research approach on two case studies. The cases studies were interactive installations in a cultural exhibition and in a sustainable development expo. Both can be seen as innovative products. The analysis was done based on the observations, semi-structured interviews, informal meetings and discussions. As a result the author noted that interactive and innovative installations are created normally by multi-disciplinary teams in accordance to agile principles. No other connection to agile software development was noted based on the empirical data.

Innovation adoption with agile methods Fast adoption of new ways of working and new tools seems to be a special feature in agile methods. This happens through the habit of continuous inspection and adaptation of development process. Surendra (P41) has studied how agile methods enable mindful adoption of IT innovations. The study uses an action research and it studied how the agile methods can be used to develop IT innovations, and how the methods enable the organization to mindfully adopt that innovation. The research setting was a United States based business college that needed a new IT system to manage applicants. The study is limited by the amount of people involved and the limited familiar context. Surendra stated that rigidly following a set of agile practices by itself does not facilitate more mindful approach. Instead the developers have to be more mindful themselves to enable mindful IT innovation adoption. However, the involvement of the stakeholders which is built-in to agile will lead the stakeholders to take ownership of the system. Surendra concludes that "Agile development principles and practices are ideally suited to develop IT innovations that support stakeholders in using their localized knowledge and expertise." Börjesön et al. (P07) studied how agile improvement practices wielded by the change agents can be used to enable process innovation diffusion in organization. The study is an action research conducted in Ericsson AB. The study is limited to one organization and

This paper was presented at The 6th ISPIM Innovation Symposium – Innovation in the Asian Century, in Melbourne, Australia on 8-11 December 2013. The publication is available to ISPIM members at www.ispim.org.

was conducted mainly by internal people involved in the change process. This might create a bias towards certain results. Börjesön et al. (P07) found out that the agile improvement practice called 'guerrilla tactic' helped out in implementing a process innovation. Also they stated that there should be more studies made on how change agents can adopt agile improvement practices to cross the chasm.

Diffusion of agile methods The largest body of empirical research was found from diffusion of agile methods theme. This consists on studies using diffusion of innovation theories to examine adoption of agile software development, the post-adoption usage, and the reasons for the adoption. Vijayasarathy et al. (P44) researched the factors that drive the adoption and use of agile methods. The data for this study was gathered through an anonymous online survey that was announced in selected agile related online discussion groups. The survey received 98 responses. Vijayasarathy et al. (P44) found that subjective norm and training are strongly and positively related to agile use. On the other hand, perceived benefits and perceived limitations do not affect agile usage. As an interesting point, Vijayasarathy et al. (P44) stated that “perceived benefit emerges as a significant predictor of agile use only if adopters face hindrances to their agile practices”. Bonner et al. (P06) validated the perceived advantages of agile methods. This was done through online survey that was distributed through CIOs and other top executives from several organizations. The estimated potential responder pool was approximately 8000 persons, and the response rate was 6.92 %. Final sample size was 479 responses after cleaning up only partly filled responses. The study done by Bonner et al. (P06) shows that the agile methods are simpler to use, more complex processes give less perceived benefit, agile methods fit better to the developers’ needs, fitness to needs improves perceived benefits and finally agile methods have greater perceived benefits. There are some limitations to the results, main ones being that 80% of the respondents were from single country, and there is a possible sampling bias influencing the survey Pikkarainen et al. (P28) explored the application of innovation adoption theories to study agile practices in use. This was done by conducting three case studies with software development teams that had used agile methods for some time already. Data was mainly collected through semi-structured interviews and these were coded and a cross-case comparison was done. Pikkarainen et al. (P28) found that the assimilation of agile practices is not dependent on time or the period of use. Instead the agile practices addressing specific needs of the team will reach deeper assimilation levels. Also Pikkarainen et al. (P28) showed that retrospectives are an important practice for driving assimilation to deeper levels. Wang et al. (P30) built on top of Pikkarainen et al. (P28) and added one more team to the data. This data also supports the notion that time is not a good indicator for evaluation of the assimilation stages. Wang et al. (P30) also present adapted and extended concepts of innovation assimilation stages for use in agile methods context. This offers a useful theoretical lens for other empirical studies. Senapathi et al. (P33, P38, P39) studied similar areas as Pikkarainen et al. (P28) and Wang et al. (P30) by concentrating on the post-adoptive agile usage. Senapathi et al. (P33, P38, P39) identified a set of critical factors that are important for post-adoptive

10

usage. The model used was built based on diffusion of innovation, systems development methodologies and agile methods. The model was developed and tested against one case study company (P39). P33 and P38 added a second case company to the research. Each case company had used agile methods for at least two years. Data was gathered with semi-structured interviews and complemented with observations and documentations. There was some variance in the results from the different case companies studied by Senapathi et al. (P33, P38). This was due to different context, usage and requirements. However, relative advantage of agile methods compared to traditional methods and management distrust and increased morale had been seen in both cases. Senapathi et al. (P33) state also that deeper the agile practices are assimilated into the organization, the better chance there is to understand if specific improvements improve the outcomes. In (P38) Senapathi et al. found out that there was no compatibility issues in case companies related to technical practices. Interesting result from Senapathi et al. (P38) was that all the influencing factors (relative advantage, compatibility, experience level, knowledge/expertise, agile practices, tool support, team management, team leadership, top management support, methodology champion) influenced effective post-adoptive usage.

Enabling innovation in agile methods As it seems to be clear from the research synthesized so far, the agile methods do not automatically lead to innovation. However, there are ways to enable and ensure innovation in this context also. Järvilehto et al. (P24) have studied the use of innovation competitions in software development. Their aim has been to summarize the factors related to innovation competitions as means for enhancing open innovation processes. The study was executed as a series of interviews in three case companies and one of these companies was using agile methods. Järvilehto et al. (P24) discussed that the agile principles, open innovation principles and innovation competitions work well together. The competitions and agile can be linked on various levels. Simplicity, self-organizing teams, emerging requirements, continuous improvement and sustainable development are all attributes of both agile and innovation competitions. Näkki et al. (P27) presented an empirical validation for their innovation framework in agile software development context. The framework mainly focuses on product innovation and users’ role and had been developed previously by Koskela et al. (P26). The framework was validated through a series of steps. The process started with lead-user identification with a large online survey for 600 internet users and initial idea collection with 200 survey responders. From these surveys, a group of 33 lead-users were identified and invited to participate in idea generation and concept phase. Finally, the software was developed by a student team. The limitations were not discussed, but the study would need more empirical data in order to have stronger results. The results from Näkki et al. (P27) show that agile methods provide suitable grounds for use involvement and a motivated lead user group can be involved even in a long term innovation process. Against a common agile practice of minimising up-front design, Näkki et al. (P27) state that the basic product concept must be designed before the software development starts. They also elaborated on the more technical parts of the commu-

This paper was presented at The 6th ISPIM Innovation Symposium – Innovation in the Asian Century, in Melbourne, Australia on 8-11 December 2013. The publication is available to ISPIM members at www.ispim.org.

nication: “Both asynchronous and synchronous communication with users is needed” and “Online co-creation needs continuous facilitation” (P27).

5 Discussion Innovations born in agile context The first review question we had was focused on studying innovations born in agile context. The only empirical study related to this is Campos (P08). The study examined ASD and product innovations in a niche market, namely interactive installations. Some non-empirical studies on the topic were also included in the mapping. P21 had some collected data but did not address a clear topic. However, the study points out interesting observations from single study. One example was that the management in the case company relied solely to the Scrum process for delivering the product innovations, but at the same time forgetting the values and principles of agile methods in general. Also P22 and P40 offer a narrative of a case company using agile methods to deliver innovative products. The relevance of these is small. In a summary, we can conclude that there is not much known about the innovations born in agile context. We have only empirical data from a single multi-case study in a very niche market. Furthermore, the existing work seems to focus on product innovations whereas practices of agile methods, e.g. retrospective, might be better for supporting the birth of process innovations. Nevertheless, clearly more work is needed in this theme.

Agile methods improve innovation capabilities The second review question aimed to answer if agile methods enhance the innovation capabilities of software development teams. P17 was studying this, and concluded that the perception of the people working using agile methods is that the methods lead to innovations. However, more empirical data would be needed to verify that the perception correlates to output data. As such, we did not find any empirical evidence that hints that using agile methods would improve innovativeness of a team or an organization. Two studies (P07, P41) identified that agile methods do affect the innovation adoption. Surendra (P41) stated that if the stakeholders are involved in the development of IT innovation, they would take ownership of the system. This leads to easier IT innovation adoption. Börjesön et al. (P07) studied how the agile improvement practices enable process innovation adoption in organizations. They concluded that using the agile improvement practices was helpful in implementing process innovation throughout the organization. To answer the review question, we can state that the methods are perceived to improve innovation capability and are shown to ease the innovation diffusion and adoption. There is some empirical data to support this notion; however, more research is needed to validate the existing assumptions.

Improving agile team’s innovation capabilities Final review question was: “How can agile teams’ innovation capabilities be improved?” For this question, we found most empirical studies. Börjesön et al. (P07) stated that agile

12

methods improve innovation adoption capabilities as already discussed in previous subsection, thus making improvement to innovation related practices easier. One study (P28) found that retrospectives led to needs based adoption of process improvements from the selected case studies. Näkki et al. (P27) found that agile methods provide good mechanisms for user involvement throughout development project, and through that improves the possibility of product innovations. Järvilehto et al. (P24) states that innovation competitions and agile methods work well together. Aaen (P01, P02, P03, P04) has worked on improving innovation capabilities through games and team roles. Aaen has developed a new method called ‘Essence’ through experimentation with student projects. The empirical data on the usage of the Essence is not, currently, enough to confirm that the method improves innovation capabilities. Nevertheless, the retrospective practice, close user involvement and lead user groups and innovation competitions seem to be suitable means to improve teams’ innovation capabilities.

Agile adoption and post-adoptive usage Theme related to diffusion of agile methods and the post-adoptive usage was well researched with most empirical studies. Training and subjective norm were important factors in agile adoption according to Vijayasarathy et al. (P44). The post-adoptive assimilation of agile practices was based on the team’s needs according to Pikkarainen et al. (P28) and Wang et al. (P30). This was extended by Senapathi et al. (P33, P38, P39) and they noticed that deeper understanding of benefits from agile methods needs deeper assimilation of practices. This area had empirical data, though still lacks more practical insights into how this relates to delivering innovative products or implementing process innovations to improve effectiveness.

Further work and implications for research and practice This study gives some implications both for research on agile software development methods as well as for innovation research in software context. We showed the topic has been addressed from several disciplines and forming a common baseline for further research is needed. Our study reveals that there is a small number of studies on the topic and only few empirical studies done; furthermore, these empirical studies are focused on a few themes. Still, both popular themes, innovation diffusion using agile methods and improved innovativeness using agile methods, are worth further research. Both research tracks can have academic as well as practical impact. We found only a few, if any, empirical evidence on the claimed benefits on the innovation capabilities by agile methods. Thus, specially further empirical work is needed on the topic. The area of process innovations realized in agile context through continuous improvement, retrospectives and kaizen events has not been studied at all. These practices might open an interesting research line for future studies. Open innovation is a trend in innovation management research, but there was no empirical research on open innovation in agile context. Also the areas of business model innovation, service innovation, social innovation and radical innovation were not touched upon. Each of these areas might contain very useful research topics.

This paper was presented at The 6th ISPIM Innovation Symposium – Innovation in the Asian Century, in Melbourne, Australia on 8-11 December 2013. The publication is available to ISPIM members at www.ispim.org.

Finally, we showed that using systematic manual search strategy on the publication lists of the active authors in the field might be useful in systematic literature studies. Although the number of relevant papers found in the manual search is small, they represent one fifth of the papers included into the study. Even though this study is a literature review, it sets some managerial implications. First, the studies analysed in this review show that a successful assimilation of agile methods requires guidance. Second, there were wide discussion on the importance of retrospective and other process improvement practices for the innovation capabilities of a team; however, these seem to be the first practices to be dropped when a new development method is adopted for an organization. There are some issues, naturally, liming the results of this study. First and foremost, the selection of the keywords for automatic search strategy; selection of article database as well as selection of active authors for manual search strategy are crucial for a literature review. We selected often utilized databases and used a general search terms as well as specific ones in order to include a broad range of articles. Similarly, we studied the publication lists of all authors that have at least two articles included into the final phases by hand. Second, a researcher bias in selection of articles as well as in data extraction might limit the results. To avoid this, the multi-staged selection process, exclusion and inclusion criteria were designed beforehand. Furthermore, we required that each exclusion decision has to be supported by at least two researchers. In later stages, data extractions were performed individually by one author and verified by another. Furthermore, all exclude decisions were required to be discussed in meetings of all authors and consensus was required to be reached before a decision could be made.

6 Summary We investigated existing knowledge on the relationships between agile methods and innovations in this paper with a systematic literature review. While the adoption of agile as an innovation has been researched extensively, studies on the innovation mechanisms of the methodology are low in numbers. Furthermore, the study shows that the empirical investigations in the topic are rare. Further empirical works are needed to verify the claims on agile methods’ ability to enhance innovation capabilities.

Acknowledgements Tomi Juhola is grateful for the Jenny & Antti Wihuri foundation, Nokia Foundation and Turku University Foundation for financially supporting his dissertation work on innovations in agile software development context. Furthermore, the authors wish to thank AgiES: Agile Methods for Embedded System Development project for the support provided for this research.

References Abrahamsson, P, Salo, O, Ronkainen, J & Warsta, J 2002, Agile software development methods: Review and analysis, Espoo, Finland.

14

Aoyama, M 1998, “Agile Software Process and its experience,” in Proceedings of the 1998 International Conference on Software Engineering, 1998,pp. 3–12. Beck, K, Beedle, M, Van Bennekum, A, Cockburn, A, Cunningham, W, Fowler, M, Grenning, J, Highsmith, J, Hunt, A, Jeffries, R & others 2001, “Manifesto for agile software development”. Available at http://agilemanifesto.org. Braun, V & Clarke, V 2006, “Using thematic synthesis in psychology,” Qualitative Research in Psychology, vol. 3, no. 2, pp. 77–101. Brereton, P, Kitchenham, BA, Budgen, D, Turner, M & Khalil, M 2007, “Lessons from applying the systematic literature review process within the software engineering domain,” Journal of Systems and Software, vol. 80, no. 4, pp. 571–583. Budgen, D, Turner, M, Brereton, P & Kitchenham, B 2008, “Using mapping studies in software engineering,” in Proceedings of Psychology of Programming Interest Group 2008 (PPIG’2008), Lancaster University, pp. 195–204. Dingsøyr, T, Nerur, S, Balijepally, V & Moe, NB 2012, “A decade of agile methodologies: Towards explaining agile software development,” Journal of Systems and Software, vol. 85, no. 6, pp. 1213–1221. Dyba, T & Dingsoyr, T 2009, “What Do We Know about Agile Software Development?,” Software, IEEE, vol. 26, no. 5, pp. 6–9. Dybå, T & Dingsøyr, T 2008, “Empirical studies of agile software development: A systematic review,” Information and Software Technology, vol. 50, no. 9-10, pp. 833–859. Hannola, L, Friman, J & Niemimuukko, J 2013, “Application of agile methods in the innovation process,” Int. J. of Business Innovation and Research, vol. 7, no. 1, pp. 84–98. Highsmith, J 2009, Agile Project Management: Creating Innovative Products 2nd editio., AddisonWesley Professional, Boston, MA, USA. Highsmith, J & Cockburn, A 2001, “Agile software development: The business of innovation,” Computer , vol. 34, no. 9, pp. 120–122. Kitchenham, BA & Charters, S 2007, Guidelines for performing Systematic Literature Reviews in Software Engineering. Version 2.3., Keele, Staffs, United Kingdom. Kitchenham, BA, Dybå, T & Jørgensen, M 2004, “Evidence-based Software Engineering,” in Proceedings of the 26th International Conference on Software Engineering, IEEE Computer Society, Washington, DC, USA, pp. 273–281. Oza, N & Abrahamsson, P 2009, Building Blocks of Agile Innovation, BookSurge Publishing. Werner, B 2013, “How You Can Compete With The Best Businesses You’ve Never Heard Of,” Forbes. Available at http://www.forbes.com/sites/microsoftdynamics/2013/03/06/innovateto-stay-relevant/

This paper was presented at The 6th ISPIM Innovation Symposium – Innovation in the Asian Century, in Melbourne, Australia on 8-11 December 2013. The publication is available to ISPIM members at www.ispim.org.

Appendix Table 3 Selected articles and quality assessment results are shown. QA score

Empirical?

ID

Article

P01

Aaen, I., 2008. Essence: Facilitating Agile Innovation. In P. Abrahamsson et al., eds. Agile Processes in Software Engineering and Extreme Programming. Springer Berlin Heidelberg, pp. 1–10.

2

NO

P02

Aaen, I., 2010. Roles in Innovative Software Teams: A Design Experiment. In J. Pries-Heje et al., eds. Human Benefit through the Diffusion of Information Systems Design Science Research. Springer Berlin Heidelberg, pp. 73–88. Available at: http://dx.doi.org/10.1007/978-3-642-12113-5_5.

7

YES

P03

Aaen, I., Essence: Team-based Software Innovation. Available at: https://intranet.cs.aau.dk/fileadmin/user_upload/Education/Courses/2010/SWI/ EssenceBookExcerptsDraft.pdf accessed on November, 2013.

1

NO

P04

Aaen, I., 2008. Essence: Facilitating Agile Innovation. European Journal of Information Systems, 17(5), pp.543–553.

2

NO

P05

Barton, B., 2009. All-Out Organizational Scrum as an Innovation Value Chain. In 42nd Hawaii International Conference on System Sciences, 2009. pp. 1–6.

1

NO

P06

Bonner, N.A., Teng, J.T.C. & Nerur, S., 2010. The Perceived Advantage of Agile Development Methodologies By Software Professionals: Testing an Innovation-Theoretic Model. In Proceedings of the Sixteenth Americas Conference on Information Systems. Lima, Peru, pp. 1–11.

11

YES

P07

Börjesson, A., Martinsson, F. & Timmerås, M., 2006. Agile improvement practices in software organizations. European Journal of Information Systems, 15(2), pp.169–182.

7

YES

P08

Campos, P., 2012. Promoting innovation in agile methods: two case studies in interactive installation’s development. International Journal of Agile and Extreme Software Development, 1(1), pp.38–51.

6

YES

P09

Conboy, K. & Morgan, L., 2010a. Combining Open Innovation and Agile Approaches: Implications for IS Project Managers. In Proceedings of the 18th European Conference on Information Systems.

2

NO

P10

Conboy, K. & Morgan, L., 2010b. Future Research in Agile Systems Development: Applying Open Innovation Principles Within the Agile Organisation. In T. Dingsøyr, T. Dybå, & N. B. Moe, eds. Agile Software Development. Springer Berlin Heidelberg, pp. 223–235.

2

NO

P11

Conboy, K. & Morgan, L., 2011. Beyond the customer: Opening the agile systems development process. Information and Software Technology, 53(5), pp.535–542.

2

NO

P12

Morgan, L. & Conboy, K., 2010. Exploring the Role of Value Networks for Software Innovation. In F. Antonio Basile Colugnati, L. Lopes, & S. Barretto, eds. Digital Ecosystems. Springer Berlin Heidelberg, pp. 20–30.

2

NO

P13

O’hEocha, C. & Conboy, K., 2010. The Role of the User Story Agile Practice in Innovation. In P. Abrahamsson & N. Oza, eds. Proceeding of First International Conference on Lean Enterprise Software and Systems. Springer Berlin Heidelberg, pp. 20–30.

2

NO

P14

(removed as duplicate)

16

P15

Crawford, B., de la Barra, C.L. & Letelier, P., 2008. Communication and Creative Thinking in Agile Software Development. In G. Cascini, ed. Computer-Aided Innovation (CAI). Springer US, pp. 205–216.

2

NO

P16

Garbajosa, J. & Yague, A., 2009. Productizing Agile Innovation. In N. Oza & P. Abrahamsson, eds. Building Blocks of Agile Innovation. BookSurge Publishing, pp. 155–172.

2

NO

P17

Grassmann, O., Sandmeier, P. & Wecht, C.H., 2006. Extreme customer innovation in the front-end: learning from a new software paradigm. International Journal of Technology Management, 33(1), pp.46–66.

8

YES

P18

Sandmeier, P., 2008. Managerial implications for integrating customers into industrial product innovation. In Customer Integration in Industrial Innovation Projects. Gabler, pp. 197–229.

2

NO

P19

Haniotis, J., 2011. Innovation Jams: Lessons in Agile Product Development -An Experience Report. In Agile Conference 2011. pp. 223–229.

1

NO

P20

Higman, J. et al., 2001. Innovation and Sustainability with Gold Cards: I am not a load factor - I am a free man. In Proceedings of XP Universe.

2

NO

P21

Hosbond, J. & Nielsen, P., 2008. Misfit or Misuse? Lessons from Implementation of Scrum in Radical Product Innovation. In P. Abrahamsson et al., eds. Agile Processes in Software Engineering and Extreme Programming. Springer Berlin Heidelberg, pp. 21–31.

2

NO

P22

Judy, K.H. & Krumins-Beens, I., 2007b. Using Agile Practices to Spark Innovation in a Small to Medium Sized Business. In Proceedings of the 40th Annual Hawaii International Conference on System Sciences. Los Alamitos, CA, USA: IEEE Computer Society, p. 275b.

1

NO

P23

Judy, K.H. & Krumins-Beens, I., 2007a. Ript: Innovation and Collective Product Ownership. In Agile Conference 2007. Washington, DC, pp. 316–325.

1

NO

P24

Järvilehto, M. et al., 2009. Innovation Competition. In N. Oza & P. Abrahamsson, eds. Building Blocks of Agile Innovation. BookSurge Publishing, pp. 69–99.

8

YES

P25

Similä, J. et al., 2009. Agile Innovation Process. In N. Oza & P. Abrahamsson, eds. Building Blocks of Agile Innovation. BookSurge Publishing, pp. 39–68.

2

NO

P26

Koskela, K., Näkki, P. & Pikkarainen, M., 2009. Towards a framework for lead user driven innovation in software intensive companies. In The Proceedings of The XX ISPIM Conference.

1

NO

P27

Näkki, P., Koskela, K. & Pikkarainen, M., 2011. Practical model for userdriven innovation in agile software development. In 17th International Conference on Concurrent Enterprising 2011. pp. 1–8.

7

YES

P28

Pikkarainen, M., Wang, X. & Conboy, K., 2007. Agile Practices in Use from an Innovation Assimilation Perspective: A Multiple Case Study. In Twenty Eighth International Conference on Information Systems. Available at: http://aisel.aisnet.org/icis2007/7.

10

YES

P29

Pikkarainen, M., Koskela, K. & Wang, X., 2009. Innovation Challenges in Agile Context---Findings from a Case Study. In N. Oza & P. Abrahamsson, eds. Building Blocks of Agile Innovation. BookSurge Publishing, pp. 175–188.

7

YES

P30

Wang, X., Conboy, K. & Pikkarainen, M., 2012. Assimilation of agile practices in use. Information Systems Journal, 22(6), pp.435–455.

10

YES

P31

Landgraf, K. & Jochem, R., 2012. Innovation Management Needs an Interoperable Requirements Management. In M. Sinderen et al., eds. Enterprise Interoperability. Springer Berlin Heidelberg, pp. 5–19.

2

NO

This paper was presented at The 6th ISPIM Innovation Symposium – Innovation in the Asian Century, in Melbourne, Australia on 8-11 December 2013. The publication is available to ISPIM members at www.ispim.org.

P32

Senapathi, M., 2010. Adoption of Software Engineering Process Innovations: The Case of Agile Software Development Methodologies. In A. Sillitti et al., eds. Agile Processes in Software Engineering and Extreme Programming. Springer Berlin Heidelberg, pp. 226–231.

1

NO

P33

Senapathi, M. & Srinivasan, A., 2012. Understanding post-adoptive agile usage: An exploratory cross-case analysis. Journal of Systems and Software, 85(6), pp.1255–1268.

8

YES

P34

Moe, N.B. et al., 2012. Fostering and sustaining innovation in a fast growing agile company. In Proceedings of the 13th international conference on Product-Focused Software Process Improvement. Berlin, Heidelberg: SpringerVerlag, pp. 160–174.

1

NO

P35

Oza, N. & Sill, J., 2009. Agile Principles and Innovation Enablement. In N. Oza & P. Abrahamsson, eds. Building Blocks of Agile Innovation. BookSurge Publishing, pp. 25–36.

1

NO

P36

Porschen, S., 2012. Management of the Informal by Cooperative Transfer of Experience. In F. Böhle, M. Bürgermeister, & S. Porschen, eds. Innovation Management by Promoting the Informal. Springer Berlin Heidelberg, pp. 105– 142.

1

NO

P37

Raithatha, D., 2007. Making the whole product agile: a product owners perspective. In Proceedings of the 8th international conference on Agile processes in software engineering and extreme programming. Berlin, Heidelberg: Springer-Verlag, pp. 184–187.

0

NO

P38

Senapathi, M. & Srinivasan, A., 2011. Understanding Post-Adoptive Agile Usage -- An Exploratory Cross-Case Analysis. In Agile Conference 2011. pp. 117–126.

7

YES

P39

Senapathi, M., Middleton, P. & Evans, G., 2011. Factors Affecting Effectiveness of Agile Usage --- Insights from the BBC Worldwide Case Study. In A. Sillitti et al., eds. Agile Processes in Software Engineering and Extreme Programming. Springer Berlin Heidelberg, pp. 132–145.

7

YES

P40

Smith, T., 2008. Product Innovation is Practical, Important, and Possible. In Agile Conference 2008. pp. 561–565.

2

NO

P41

Surendra, N.C., 2009. Agile Development as an Enabler of Mindful IT Innovation Adoption. Lessons from an Action Research Project. Journal of Decision Systems, 18(1), pp.99–115.

7

YES

P42

Tendon, S., 2010. Tailoring Agility: Promiscuous Pair Story Authoring and Value Calculation. In D. Šmite, N. B. Moe, & P. J. Ågerfalk, eds. Agility Across Time and Space. Springer Berlin Heidelberg, pp. 47–70.

1

NO

P43

Tiwari, G. & Alikhan, Z., 2011. From “Team” to “Wow Team”: An Agile Team’s Journey. In Agile Conference 2011. pp. 296–301.

0

NO

P44

Vijayasarathy, L. & Turk, D., 2012. Drivers of agile software development use: Dialectic interplay between benefits and hindrances. Information and Software Technology, 54(2), pp.137–148.

10

YES

18

Table 4 Data extraction form. Category Background

Question Study ID Date of data extraction Data extracted by Date of extraction review Extraction reviewed by

Basic information

Bibliographic reference Type of article Aims of the study Design of the study Research hypothesis

Research question related

How is the study related to innovations Is the study conducted in agile software context? What kinds of innovations are researched? What is the connection between the agile methodologies and the innovation aspect in the study? Is there any methods, practices, principles, or tools mentioned which increased the innovation capabilities of the team? Any evidence supporting previous statement?

Results

Findings and conclusions Validity Relevance

Comments

Additional comments (e.g. relevant references)