software companies practice inspections infrequently or not at all. For instance, some .... hermeneutic spiral, which is an iterative process whereby each iteration provides .... inspections in a large company which produced medical equipment.
2010 18th IEEE International Requirements Engineering Conference
Persuading Software Development Teams to Document Inspections: Success Factors and Challenges in Practice Marko Komssi,1,2 Marjo Kauppinen,1 Maaret Pyhäjärvi,3 Jukka Talvio,3 Tomi Männistö1 1
Aalto University Espoo, FINLAND
2
F-Secure Corporation, Helsinki, FINLAND
Keywords-Software Inspection; Requirements Specification; Action Research; Quality Control; Human Factors
INTRODUCTION
Peer reviews are recognized as one of the best practices in requirements engineering (RE) [1] and seem to be a core practice of requirements quality control [2]. Inspection is the most formal technique of a peer review [3] and has been proven to be one of the most effective methods for finding defects, as it may reduce rework costs by up to 60 percent [4]. A research study on inspecting requirements specifications has quantified a return on investment of 30 to 1 in a small research support group of a large international company [5]. Despite a large number of positive inspection reports, a large number of software organizations practice inspections infrequently or not at all [6]. According to a survey, about 70 percent of the respondents from software companies produce requirements and design documents [7]. Of those who produce requirements and design documents, fewer than half perform document reviews. Human factors seem to have a major influence on practitioners’ contributions to inspections. While practitioners generally trust published inspection results, their perception is that inspections would mostly add costs in their context [8]. In fact, regardless of its importance, some 1090-705X/10 $26.00 © 2010 IEEE DOI 10.1109/RE.2010.40
Previously employed by F-Secure Corporation
practitioners perceive any review as a disturbance of their valuable time [9]. In addition, it has been pointed out that industry practitioners experience inspections as being ineffective and difficult [6, 7]. Second, it is difficult to maintain the interest of software developers in performing inspections, even when management supports their usage [10]. A typical reason for this is that developers consider inspections to be uncreative and unproductive work [10]. In the software engineering literature, the most frequently reported factor affecting software developers’ motivation relates to work tasks [11]. Since document inspection is considered an effective method but perceived as boring work, investigating human factors related to inspections is justified. Furthermore, it is important to study human and social problems in their historical and social context over a rather long period of time [12]. One empirical study [4] investigated the long-term industrial adoption of inspection, concentrating on human factors and presenting situations that led to both failure and success, at Hewlett-Packard between the years 1976 and 1993. The aim of this paper is to provide similar types of sociological insights from the newer and rather long-term adoption of document inspection in a software company. Our study concentrates on the following research question: what are the success factors and challenges in persuading review teams to document inspections? We present our knowledge accumulated from over a hundred software document inspections in the F-Secure Corporation, both successful and failed, during a ten-year period. F-Secure is a Finnish software company that produces, for example, anti-virus products, and is the global leader in providing protection solutions via internet service providers and mobile operators. The study utilized action research approach, and the lessons learned are mainly based on qualitative, and partly retrospective, data collection and analysis. The experiences of document inspections described in this paper reflect a ten-year period in the company. There have been two separate and active periods during which the inspection practices have been invested in and inspection champions have supported and enhanced the usage of inspection practices. However, despite the initial success of these attempts, document inspections have not gained a sustainable position in the company. Accordingly, the paper is organized as follows. Section 2 presents the related work. Section 3 describes the four phases of the case study, that is, the two peaks of inspection efforts and the lows that followed. To elaborate the four phases, Section 4 introduces the success factors and challenges within seven categories. Finally, Section 5 concludes the findings.
Abstract—Peer reviews have been identified as one of the best practices in requirements engineering. The most formal peer review technique – inspection – has been found to be effective for the discovery of defects in documents. Nonetheless, many software companies practice inspections infrequently or not at all. For instance, some engineers consider inspections to be uncreative work that adds costs. This paper describes the success factors and challenges involved in persuading review teams to document inspections. The case study utilized an action research approach over a 10-year period, during which two separate attempts were made to address inspection practices. Document inspections did not gain a sustainable position in the study. Consequently, key success factors and challenges were identified in order to explain how the review teams changed their perception throughout the adoption of document inspection. Long-term engagement to document inspection requires motivation and commitment on the part of the teams, and the favorable development culture of an organization, dedicated champions, and the careful tailoring of inspection practices seem to positively influence this. In the long run, it can be challenging to keep document inspections consistent with the existing values, previous experiences, and needs of the software development teams.
I.
3
283
II.
RELATED WORK
amount of defects in the document when verifying it against the inspection rules and checklist initially provided at the kickoff meeting. Process brainstorming meetings are optional. The goals of these meetings are to brainstorm the possible root causes of the most harmful defects and to find possible process improvements. In the editing phase, the writer of the document fixes the most serious defects. The moderator verifies the work and makes a decision about exiting from the process. In addition to edit auditing, the exit stage includes moderator overheads, such as inspection data recording and informing the stakeholders.
A. Difficulty in Adopting Document Inspections According to Mathiassen and Vogelsang [13], software development methods are difficult to adopt and relatively little is known about what happens in such adoption efforts over time. In particular, they found out that perceptions and approaches change completely through different phases of software method adoption. It is valuable to study how a method is applied in practice and in which ways organizational constraints limit its benefits over time [14]. The study of Grady and Van Slack [4] indicates that both success factors and challenges are partly different in the initial phase of using inspections than in the later phase. It can be deduced that the widespread use of inspection is more a leadership issue than a technical one. The study emphasizes the fact that success is heavily dependent on visionary people in the initial stage. In later stages, for example, objectives tailored to the company’s divisions and local inspection champions are required. The inspection champion is a person responsible for inspection in an organization [15]. The failed applications of document inspections have typically not been reported. In addition to detailing several success factors, the study of Grady and Van Slack [4] also reveals that some inspection deployments failed at HewlettPackard. In some situations the moderators were not able to change the existing team dynamics and critical atmosphere. Moreover, bad previous experiences and a stagnant inspection process seemed to be the main reasons for the deterioration of the use of inspections or their bad quality in some local divisions. For example, one team stopped using inspections as they felt that document inspections took too long and did not reveal enough major defects. Unsuccessful experiences made teams unmotivated and resistant to trying again. In particular, the study emphasizes the difficulty of retraining engineers in those situations. To overcome those challenges, full-time inspection champions were assigned in the company. The champions were advised to use a consultant approach to evaluate and influence the organizations’ readiness and to perform inspections. A recent report [16] presents five factors that facilitate the adoption of inspection in NASA’s Jet Propulsion Laboratory. The factors are: 1) management support for innovation; 2) a dedicated champion; 3) support materials for engineers; 4) data showing success from a similar context, and 5) tailoring the practice to the target context without eliminating its important parts.
III. INDUSTRIAL CASE To gain a deep understanding of the long-term inspection application calls for a qualitative research approach. Action research is a method applicable to the study of human aspects over time. The unique strength of action research is its capability of connecting researchers and practitioners performing together on a particular cycle of activities, including problem diagnosis, action intervention, and reflective learning [17]. Insider action researchers are internal members of practitioner organizations and they have an opportunity to collect richer data than external researchers [18]. For example, practitioner-researchers can use internal jargon in interviews and participate freely without creating suspicion. We also utilized retrospective action research [12] as a complementary approach to the study of the early years of inspection adoption. F-Secure Corporation has over 800 employees on three continents: Europe, North America, and Asia. The company’s headquarters, including over 300 employees, is located in Finland. F-Secure develops commercial software solutions mainly for mass markets and requirements specifications are created mainly for internal use. The experience presented in this paper was accumulated in Finland between 1997 and 2008. Three of the authors of this paper worked at F-Secure, at least partly, during this period and represent three of the four inspection champions in the company’s history. Together with other practitioners, they also acted as inspection moderators. The moderators led the inspection planning work and the collection of defect data and related metrics. The authors also participated in several inspections as reviewers and document authors. As presented in Table 1, our study design is divided into four temporal phases of inspection application that differ in their means of data collection, reflecting the situation of inspections in the company. The phases, Initiation, Deterioration, Tailoring, and Abandonment, are briefly introduced next. Initiation was the first attempt to invest in the use of inspections. It was followed by a period during which the inspection practices did not stay on the level initially achieved: Deterioration. The second attempt to increase the use of inspections learned from the lessons of the earlier phases and tried another approach by paying particular attention to the Tailoring of the practices for the company. In the final phase, Abandonment, document inspections were being suspended for the time being.
B. Inspection Model Applied The well-known inspection model of Gilb and Graham [15] was initiated at F-Secure in 1997. Hereafter, we refer to their model as the traditional inspection model. The model has seven defined stages. The traditional inspection model starts with the planning and kickoff stages. While the planning stage focuses on taking care of practical details and preparing the inspection strategy, the kickoff stage concentrates on sharing information and clarifying the tasks of the reviewers. The reviewers then perform individual checking. Their goal is to find the maximum
284
TABLE I.
did not reveal any unexpected information that it would be practical to collect and analyze further. A fresh effort to invest in document inspections started in the tailoring phase. The main reason was that two new persons, who had gained knowledge of inspection in other companies, began to act as inspection champions during that time. The current inspection champion and the two new persons shared the championship and focused on tailoring the current practices. They were able to successfully place inspections back in the development process in several software projects, while software engineers had a prejudice against document inspections as a result of their disappointments during their previous use. In the abandonment phase, all three champions gradually lost their interest in this role. Under time pressure, they chose to perform those work tasks that they identified as being beneficial to the company or personally fascinating. Then, none of the employees took the role of inspection champion to advocate document inspections. Consequently, the use of document inspections was discontinued during the abandonment phase.
DATA COLLECTION ACTIVITIES IN THE FOUR INSPECTION APPLICATION PHASES
Phase Initiation
Time period 1997 – 1999
Deterioration
2000 – 2002
Tailoring
2003 – 2006
Abandonment
2006 – 2008
Data collection activities Intervention, observation, defect data collection, and interviews Intervention, observation, document analysis, and interviews Intervention, observation, document analysis, defect data collection, and interviews Informal conversations, document analysis, and interviews
The data were analyzed in several iterations in between the data collection phases. The analysis followed the hermeneutic spiral, which is an iterative process whereby each iteration provides knowledge [12]. The identification and selection of seven categories was performed at the end. The data were analyzed using both a top-down and a bottomup approach [19]. In the top-down analysis, we used the success factors and challenges discovered in the related research work at Hewlett-Packard [4]. We combined the findings from that paper into the table. The timing of the success factors and challenges in the adoption of inspection was the classifier. In the bottom-up approach, we wrote summaries of the success factors and challenges related to the four phases of the adoption of inspection in a similar table. As a last phase, we clustered the success factors and challenges into high-level categories. In this work, we utilized a recent report [16] that presents five factors that facilitate the adoption of inspection in an organization.
TABLE II. Category Dedicated Champion Tailoring
IV. SUCCESS FACTORS AND CHALLENGES The perceptions of software teams and applications changed through the different phases of the adoption of document inspection. The seven categories are used to describe the success factors and challenges in persuading review teams throughout the adoption. Accordingly, the key success factors and challenges are summarized in Table 2. In the following seven sections, we elaborate the lessons learned from each category during the four phases.
Supportive Infrastructure
Previous Experiences
A. Dedicated Champion A dedicated and experienced champion was a key success factor in the adoption of document inspection in the initiation phase. Before moving to F-Secure in 1997, the champion had become familiar with inspections on a training course and he had acquainted himself with the traditional inspection model and operated with it in a large company. In particular, the definition and utilization of the software document inspection model became part of his duties. As a result, the traditional inspection model was set up and followed rather rigorously. In the deterioration phase, the rigor of the document inspections and practitioners’ perceptions of them declined. One reason was that the other tasks and responsibilities of the champion eventually became more important than the tasks related to inspections. The champion stopped collecting the inspection data in early 2000. At that time, the inspection champion also perceived that the data, metrics, and charts
KEY SUCCESS FACTORS AND CHALLENGES WHEN PERSUADING REVIEW TEAMS Key Success Factor The champions’ enthusiasm and skills were critical Tailoring was carried out based on the reviewers’ comments Building a supportive infrastructure was a vital condition for the widespread use of inspections Reviewers recognized the contribution of inspection in the development process or a software project
Development Culture
The immature development practices drove the acceptance of document inspections
Management Support
Management support was a necessity for the adoption Advocating the benefits of document inspections, good practices, and available facilitation services was important
Internal Marketing
Key Challenge Champions did not maintain their dedication over time Tailoring was timeconsuming and often unrewarding in the given situations Equipping the moderators with the skills required was challenging throughout the adoption of inspection It was challenging to avoid ineffective and boring document inspection sessions that affected the practitioners negatively Practitioners rejected document inspections when they perceived that they manage the quality of documentation and have proper communication otherwise Management support did not ensure successful adoption A lack of marketing resulted in a low level of adoption of the newest inspection ideas
B. Tailoring Initiating all the elements of the traditional inspection model at once was difficult. Some of the important elements of the traditional inspection model were not followed or they were discontinued almost immediately. The defect data were
285
properly and they did not enforce the performance by the reviewers of individual defect detection before defect collection meetings. This happened despite the fact that most of the moderators had either been trained on an inspection course or by internal training. In the tailoring phase, we faced the challenge of providing training in the new inspection practices. When we piloted the inspection planning with a set of questions and the mind map technique, the moderators needed to follow and facilitate the course of the discussion. Asking accurate questions continuously and rapidly was challenging. We found the work more demanding than facilitating the defect collection meetings; being in charge of the new type of meeting required solid interaction skills from the moderator. As a result, we found very few people to be capable of running such meetings and willing to learn the requisite skills. Providing the moderators with the required skills was challenging throughout the adoption of inspection.
not categorized on the basis of defect types, and nor were they further analyzed to improve software development practices. Neither were the inspection rule sets improved on the basis of the experience gained from software projects. Moreover, several generic checklists were experimented with, but practitioners found them useless. In addition, process brainstorming meetings were initiated but their utilization was discontinued after a few trial runs. The tailoring work of inspection practices was rather insignificant and undisciplined in the deterioration phase. The inspection guidelines were not maintained and the inspection moderators partly followed the company guidelines and partly used their common sense. As a result, the success of the tailoring work was increasingly more dependent on the skills of the moderator. During the tailoring phase, a key lesson learned was that the tailoring of inspection practices on the basis of the comments made by review teams was important in motivating them to restart inspections. Based on the comments, for example, we realized that inspection planning and causal analysis were more creative and pleasing as faceto-face meetings than as defect logging. The perceived satisfaction of reviewers at finding a root cause for a defect seemed to be higher than that at finding a defect in a document. In addition, the involvement of engineers in the tailoring work was a useful way to elicit ideas for improvements. Consequently, for example, the early timing of inspections and the creation of context-specific inspection rules were piloted. The result of using tailored inspections was identified as one of the key factors in the success of one software project and very beneficial in four other projects. While we had witnessed successful document inspections, we perceived the tailoring of document inspections as time-consuming and challenging, given the unenthusiastic attitude towards them. It was challenging to convince software engineers to perform inspections again after they had been saturated with an inapplicable and stagnant inspection process. Above all, the suggested inspection practices did not increase the adoption rate of inspections in the way that had been assumed.
D. Previous Experiences The champion had had good experiences with document inspections in a large company which produced medical equipment. This positive previous experience was a motivator in his task of initiating inspections. Very few software engineers at F-Secure had previous experience of document inspections in the year 1997. In the deterioration phase, the attitude of the review teams towards the document inspection method went downwards as they faced more bad than good inspection sessions. After five years from the day we started to use document inspections, the practitioners had typically participated in several inspection meetings that they had perceived as being useless. It was challenging to avoid ineffective and boring document inspection sessions that caused a negative perception of them. Despite the situation, the new champion was motivated to start persuading review teams to improve their inspection practices because he had had positive experiences of inspections in his previous company. However, it was hard to persuade the developers to perform inspections properly when they had had poor experiences with previous inspections. For example, we realized that negative previous experiences were a reason for the practitioners’ dismissive attitude towards document inspections in two pilot projects. Thus, we were not able to successfully pilot some inspection ideas in several other projects too, as we had wished. On the other hand, a review team discovered and solved a critical problem in two consecutive inspections, which included causal analyses, and was identified as one of the main reasons for the project’s success. The team was motivated to use inspections again. A key lesson learned from this is that software engineers seemed to be motivated to perform an inspection when they recognized the actual contribution of inspection in the development process or the software project. In the abandonment phase, not just the reviewers but also the moderators and champions gradually lost interest. It had been hard for the moderators to facilitate, time after time, practitioners who were not motivated to contribute to
C. Supportive Infrastructure In the initiation phase, the investment made in the supportive infrastructure was a success factor in the widespread use of document inspections in software projects. In practice, the inspection process was documented, and instructions and supportive inspection materials were prepared. The inspection champion organized several training events, coached the moderators, and collected inspection metrics. However, applying all the elements of the traditional inspection model at once and interpreting the collected metrics were perceived as challenging. The investment made in the supportive infrastructure did not guarantee the rigorous use of inspections over time. In the deterioration phase, we observed the current review process in practice. Twenty-three deviations or problems were identified when the process being practiced was compared against the defined process and guidelines. In particular, the moderators seldom planned the inspections
286
inspection-related work. Inspection moderators and champions were also frustrated after experiencing conflicts and dismissive attitudes.
became part of his duties. Management support was a necessity for the adoption of document inspections. In the deterioration phase, the management favored document inspection and it was a part of the company’s development process. Inspections increased the controllability of software projects, for example by showing the management when particular development gates were passed in the projects. On the other hand, the backing of the management did not ensure that software engineers used document inspections properly or over time. In 2005, the management started major investments in agile methods and gradually paid less attention to document inspections. However, the company was able to release highquality software products without performing them as much as before. In the abandonment phase, two top managerial persons commented that while applying agile methods, the company had discontinued several software practices. Lately, the management has enforced the use of code inspections, but the use of document inspections has not restarted.
E. Development Culture In the initiation phase, there was a demand for document inspections in the software organization. The development process of the company was immature and many software developers were striving to attain more formality, discipline, and guidance in their working habits. Furthermore, the communication between the software developers was inadequate and the quality of the software documents was poor. For instance, the contents of several technical documents were often contradictory. The prevailing development culture drove the acceptance of document inspections. Inspections also complemented several other software process improvement (SPI) activities which were started at the same time. In the deterioration phase, the company had invested heavily in increasing the maturity of the company’s software development process and practices. The software development process was defined and the software projects were controlled. Over time, the software developers had begun to perceive resistance towards the rigor of the software development process. They perceived inspections as a rubber stamp at the end of a software development stage. During the tailoring phase, the software teams of the company started to utilize several agile practices. After a few pilot projects, the adoption of the agile practices was fast, as both developers and managers were in favor of them. The migration from the traditional development approach to the agile process began to cut down the use of documents and, consequently, their inspections. The documentation and teamwork practices of the company were very different in the abandonment phase than in the initiation and deterioration phases. In the abandonment phase, the software projects mainly followed a customized agile development process that increased oral communication in them. Moreover, the use of collaborative documentation tools spread rapidly. In 2008, in fact, the developers commented that they did not need document inspections because documents were collaboratively owned and anyone could edit them at any time. Our interviews also showed that a group of product managers had found it beneficial to collect, determine, and sometimes even write the business requirements as a team. A product manager commented in an interview that he had nothing against document inspections but they were not needed any more. In addition, the number of informal collaborative workshops increased. The practitioners perceived that they were managing the quality of the documentation and had proper communication through other channels.
G. Internal Marketing In the initiation phase, the champion organized several training events to show the benefits of inspections on the basis of published success stories from the other companies. The training material covered motivational issues to encourage the performance of document inspections and then consisted of practical instructions. The number of training events decreased in the abandonment phase. At the beginning of the tailoring phase, we marketed the new inspection ideas, which were quite different from the existing inspection process of the company. In particular, we gave a one-hour presentation to the forty engineers and managers of the R&D department. The presentation consisted of findings about the present situation at F-Secure and preliminary ideas to improve the situation. On the basis of the presentation, the new inspection ideas were utilized immediately in two software projects. In addition, the inspection champions advocated the use of new inspection practices to software engineers and project managers in their informal discussions. Later, when we had tailored the inspection practices further, we did not invest enough energy in marketing the synthesis of tailored practices. As a result, it was difficult, e.g., to pilot early inspection planning in several projects as a typical writer of a document asked for inspection support after she had finished writing the document. V. DISCUSSION AND CONCLUSIONS This paper presents the success factors and challenges involved in persuading software development teams to use document inspections at F-Secure. The study’s findings indicate that the dedication of inspection champions was a key success factor. The dedicated champions led the tailoring work and built the supportive infrastructure and they also marketed the use of inspections in software projects and were able to convince management to make greater investments in document inspections. The main challenge was that all the inspection champions lost their enthusiasm
F. Management Support Upper-level management hired a person to manage software quality in 1997. In particular, the management saw inspections as a method to increase their awareness of the status of the projects and the quality of documents and, consequently, the championship of the document inspections
287
for the role within three years. When a person took on the role of inspection champion, they were highly motivated, which reflected positively on the use of document inspections. Over time, however, the motivation decreased and other tasks started to dominate their time. Therefore, this paper recommends that software organizations consider rotating inspection champions, e.g., at two-year intervals. Another finding was how the prevailing development culture drove the acceptance or rejection of document inspections. The document inspections were in a key position when the case study organization was increasing rigidity in its software development process. However, once the organization had reached an adequate degree of rigidity in its process, the value of document inspections as an SPI tool decreased. When the software development teams lost this type of constructive aspect, they seemed to perceive the constant use of document inspections as increasingly boring. Furthermore, when the company was shifting towards the use of more lightweight software development process, the development culture became less favorable to traditional practices such as document inspections. For instance, the migration from traditional to agile development methods gradually covered the entire software organization. In the new type of development culture, the software engineers felt that they no longer needed document inspections. During the intensive transition to the agile methods, the deliberate tailoring of document inspections was felt to be beneficial. This paper recommends involving software development teams in the tailoring work in order to elicit improvement ideas for inspection process and techniques. On the other hand, it was also found to be difficult to reimplement the tailored version of document inspections throughout the software organization, even though some development teams had perceived the benefits of using the tailored version. In fact, Rogers [20] emphasized that most new ideas diffuse at an unsatisfactorily slow rate. In particular, he stated that an idea that is incompatible with the values and norms of a social system will not be adopted as rapidly as an idea that is compatible. The present study’s findings were synthesized with two recent studies [21, 22] in order to provide recommendations for inspection champions (or change agents) who want to implement document inspections in their software organizations. Firstly, the champions are advised to introduce lightweight inspection techniques incrementally so that the techniques clearly support the team’s everyday work. Then, the champions should advocate the collaborative tailoring work, in which a software development team selects and adapts an adequate set of techniques for their software project. Finally, the champions should allocate time in internal marketing, an area that seems to play a critical role in reselling the tailored version of inspections. By presenting a rather long-term adoption of document inspection at F-Secure, this study highlights that, as things change, the successful application of document inspections should be kept consistent with the software development teams’ existing values, previous experiences, and needs. A software organization that has starting using agile development methods needs to reconsider how it can use
document inspections in the new development culture. In future, it would also be worthwhile studying how to motivate software engineers in agile development projects to perform document inspections, and how the inspection practices can be tailored to fit in with an agile development process. REFERENCES [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19]
[20] [21]
[22]
288
Hofmann HF, Lehner F, “Requirements Engineering as a Success Factor in Software Projects,” IEEE Software 18, 4, 2001, 58–66. Katasonov A, Sakkinen M, “Requirements Quality Control: a Unifying Framework,” Requirements Engineering 11, 1, 2006, 42–57. Wiegers K, “When Two Eyes Aren't Enough,” Software Development 9, 10, 2001, 58–61. Grady RB, Van Slack T, “Key Lessons in Achieving Widespread Inspection Use,” IEEE Software 11, 4, 1994, 46–57. Doolan EP, “Experience with Fagan’s Inspection Method,” Software—Practice and Experience 22, 2, 1992, 173–182. Johnson PM, “Reengineering Inspection,” Communications of the ACM 41, 2, 1998, 49–52. Ciolkowski M, Laitenberger O, Biffl S, “Software Reviews, the State of the Practice,” IEEE Software 20, 6, 2003, 46–51. Jalote P, Haragopal M, “Overcoming the NAH Syndrome for Inspection Deployment,” Proc. 20th International Conference of Software Engineering (ICSE), 1998, 371–378. Kazman R, Bass L, “Making Architecture Reviews Work in the Real World,” IEEE Software 19, 1, 2002, 67–73. Freimut B, Briand LC, Vollei F, “Determining Inspection CostEffectiveness by Combining Project Data and Expert Opinion,” IEEE Transactions on Software Engineering 31, 12, 2005, 1074–1092. Hall T, Sharp H, Beecham S, Baddoo N, Robinson H, “What Do We Know about Developer Motivation?” IEEE Software 25, 4, 2008, 92– 94. Gummesson E, “Qualitative Methods in Management Research,” 2nd edn., Sage Publications Inc., 2000. Mathiassen L, Vogelsang L, “Managing Knowledge in Software Method Adoption,”. International Journal of Business Information Systems 1, 1–2, 2005, 102–117. Davidson EJ, “Joint Application Design (JAD) in Practice,” Journal of Systems and Software 45, 3, 1999, 215–223. Gilb T, Graham D, “Software Inspection,” Addison-Wesley, 1993. Shull F, Seaman C, “Inspecting the History of Inspections: An Example of Evidence-Based Technology Diffusion,”. IEEE Software 25, 1, 2008, 88–90. Avison D, Lau F, Myers MD, Nielsen PA, “Action Research,” Communications of the ACM 42, 1, 1999, 94–97. Coghlan D, “Insider Action Research Projects: Implications for Practising Managers,“ Management Learning 32, 1, 2001, 49–60. Kauppinen M, Vartiainen M, Kontio J, Kujala S, Sulonen R, “Implementing Requirements Engineering Processes throughout Organizations: Success Factors and Challenges,” Information and Software Technology 46, 14, 2004, 937–953. Rogers EM, “Diffusion of Innovations,” 5th edn., Free Press, 2003. Komssi M, Kauppinen M, Toro K, Soikkeli R, Uusitalo E, “Lessons Learned from Integrating Specification Templates, Collaborative Workshops, and Peer Reviews,” Proc. 16th International Working Conference on Requirements Engineering: Foundation for Software Quality (REFSQ), 2010, 158–172. Savolainen J, Kuusela J, Vilavaara A, “Transition to Agile Development – Rediscovery of Important Requirements Engineering Practices,” Proc. 18th International Conference on Requirements Engineering (RE), 2010, to appear.
