Making SPI Happen: The IDEAL Distribution of Effort Anna Börjesson and Lars Mathiassen IT University, Gothenburg, Sweden Ericsson AB, St Sigfridsgatan 89, SE-412 66 Gothenburg, Sweden phone: +46-31-3446148, fax: +46-31-3446033, email:
[email protected] phone:+1-404-651-0933, fax:+1-404-463-9292, email:
[email protected]
Abstract Software Process Improvement (SPI) has become one of the most widely used approaches to increase the capability of software organisations. Many organisations experience a successful start on their SPI initiatives only to realise that the commitment to change weakens significantly after a first phase of initial excitement. This paper explores this problem based on experiences from Ericsson. Two quite similar SPI initiatives situated within the same organisational context are compared and contrasted. Both initiatives were carefully planned and managed following the IDEAL process; they got of to a successful start; and they both developed new or improved processes. But only one of the initiatives led to improvements of engineering practices while the other had little or no effect on the software operation at Ericsson. Our research shows that the effort of the two SPI initiatives is distributed quite differently between the phases of the IDEAL model and between generic actions and actions dedicated to particular software projects. The paper explores this phenomenon both as an indicator and possible explanation of differences in implementation success. Interpretations relating to key issues in SPI are offered together with a discussion of implications for research and practice. Keywords: Software process improvement, the IDEAL model, commitment. 1
Introduction
Many software organisations have committed themselves to ambitious change initiatives since the Software Process Improvement (SPI) approach was introduced by Watts Humphrey [14] through the Software
Engineering Institute (SEI). Many outstanding SPI success stories have been reported [9], [13], [14], [22], [24]. But critical evaluations have also been published [4], [5], [10], [17] and a recent report from the SEI states that of 1380 organisations reporting initial assessments, only 25 percent proceeded to make a second assessment. Of those conducting a second assessment 13.9 percent did not change their maturity level and 3.5 percent moved to a lower maturity level (SEMA 2001). A number of explanations have been offered to improve the SPI failure rate, but we know little about the impact of distributing effort between and organising the different phases of the improvement effort as described in the IDEAL (i.e. Initiating, Diagnosing, Establishing, Acting, and Learning) Model [21]. This paper offers new insights on SPI success and failure based on experiences from Ericsson Mobile Data Design AB in Gothenburg, Sweden. SPI success should be measured in terms of improved practices (the difference in quality or productivity between old and new practices). This is, however, difficult to measure. We therefore focus on the extend to which the initiative leads to actual changes in software practices, i.e. the extent to which new or modified processes are actually used in practice. This notion is easier to measure and it can be considered an important indicator for real success in terms actual improvements. Our research design is presented in Section 2. Section 3 describes two quite similar SPI projects. They were carefully planned and managed following the IDEAL model; they got of to a successful start; and they both developed new or improved processes. But only one of the initiatives led to new software practices while the other had little or no effect on the organisation. Our research shows that the two SPI initiatives in terms of man-hours
Proceedings of the 36th Hawaii International Conference on System Sciences (HICSS’03)
0-7695-1874-5/03 $17.00 © 2002 IEEE
spent were organised differently and the effort of the two SPI initiatives were distributed quite differently between the phases of the IDEAL model. Section 4 discusses these findings as indicators and possible explanations of differences in implementation outcome. In Section 5 we relate the findings to the literature on SPI and technologyenabled organisational change and we highlight key implications for practice and research. We conclude the argument in Section 6. 2
Research Approach
The presented research is part of a collaborative practice study [19] carried out in close collaboration between Ericsson Mobile Data Design AB and the ITUniversity at Gothenburg, Sweden. Ericsson Mobile Data Design is a system development centre within Ericsson with over 20 years of experience working with packet data solutions for the international market. Ericsson Mobile Data Design is today one of the world’s leading software development centres within this area. In early 1995 the company started to grow from 150 employees to 900 in 2002. During this period SPI has become an increasingly important area for Ericsson Mobile Data Design to assure quality deliveries to the customers. Ericsson Mobile Data Design has today dedicated employees that only work with SPI initiatives in order to address the relevant problems in system development projects. Collaborative practice studies target specific professional practices and they have the double goal of improving these practices while at the same time contributing to the relevant bodies of knowledge within research. This paper presents a conventional case study [12], [26] that was carried out in the early phase of the collaborative project. The focus is on SPI practices and principles and particularly on issues related to success and failure of implementation efforts. The approach is primarily diagnostic and reflective to learn about current SPI practices at Ericsson and to lay the foundation for future interventions. More specifically, two SPI projects are presented and compared [7]. One of the authors has been working in and responsible for the presented SPI Initiatives. The potential bias and subjectivity is handled through the collaborative study where the other author, not practically involved in the cases, can supply a more objective view. The first project focuses on improving subcontract management and the other on improving requirement management. Both projects were inspired by SPI principles and they were situated within the same organisational context. Based on data from the two projects we focus on a key difference between them: one was originally considered problematic but turned out to be a success in terms of implementation while the other was considered successful but gave little or no value to the organisation. Our data about the two
projects were collected during each project based on documents from the projects and with help from the time reporting system. The data were managed by the SPI group and were summarised and stored in final reports from each project. The effort spent in different phases were collected from the time reporting system and related to a phase in the IDEAL model. The data concerning perceived success were gathered through discussions with SPI practitioners from the different SPI initiatives with a focus on the perceived degree of resistance towards the change. Low resistance made the SPI practitioners feel more satisfied with their work and believe that their work was successful. But complete lack of resistance could indicate lack of deployment. The data concerning implementation success were gathered by assessing the extent to which the changed practices were used in the software engineering projects. Discussions with project managers and SPI initiative participants formed the basis for these data. 3
Two SPI Initiatives
In the following we look closer at the two SPI initiatives and focus on the differences that made one of the SPI initiatives more successful at implementing its innovations than the other. 3.1
During 1998 and 1999 Ericsson Mobile Data Design managed their SPI effort by having a small group drive it. Ericsson Mobile data Design had in early 1998 300 employees and were growing to 500 employees towards the end of 1999. The work was organised according to a Software Engineering Process Group (SEPG) and Process Action Team (PAT) structure [11]. The SPI initiative was driven from the software engineering process group who managed different parallel process action teams. The process action teams supported several software projects. All SPI decisions were taken by a management steering committee. The results produced by the process action teams were generic to fit all main software projects (see Figure 3.1).
Proceedings of the 36th Hawaii International Conference on System Sciences (HICSS’03)
0-7695-1874-5/03 $17.00 © 2002 IEEE
The Generic Initiative
Software Engineering Project
SPI Initiative (Process Action Team)
Figure 3.1 – The Generic Initiative: Executed through Process Actions Teams The software engineering process group consisted of four SPI employees. The management steering committee consisted of approximately ten managers from four different product development units. A process action team was a virtual group established to execute a focused SPI initiative defined by the management steering committee. A typical process action team consisted of six to eight employees from all product units working approximately four hours a week during a period of two to three months. The management steering committee responsibility was to decide what process action teams to initiate according to material prepared by the software engineering process group, assure resource allocation, and help find pilot projects for process action team result. The management steering committee was also used to assure commitment to the SPI work. The software engineering process group responsibility was to drive the SPI effort at Ericsson Mobile Data Design, organise and prepare monthly management steering group committee meetings, prepare decision material for suitable process action teams, write assignment specifications according to decisions, manage process action teams, and find pilot projects that could start use the process action team results. The process action team responsibility was to execute the assignment. The first SPI initiative that we explore was organised as a generic initiative across a number of software engineering projects (see Figure 3.1). The goal was to improve the organisation’s way of working with subcontract management. The purpose was to assure product quality and delivery precision from its subcontractors, choose competitive, qualified and strategic subcontractors and products, assure maintainable products, and keep track of progress at other Ericsson design centres that worked in close co-operation with Ericsson Mobile Data Design. The decision to start this SPI initiative was taken by the management steering committee and prepared by the
software engineering process group. Ten employees were allocated to work with the SPI initiative for twelve weeks. Each employee dedicated around four hours a week to the project. There was a strong commitment to find the most competent employees in the area to take part in the SPI work. There was also a strong commitment among the process action team members to participate in the work and contribute to the results. Responsible process owners were appointed in the project specification for the process action work and it was clearly stated that the process owner was responsible for deploying the results. The SPI initiative was planned to assure progress and to match the possibility for the process action team members to participate. The plan was set to achieve results each week during the 12-week period. The plan also included time for inspections and approval. A pilot project was chosen to assure that the results would be used. The people responsible for the pilot project were part of the process action team and the working assumptions in the SPI work could always be compared to the needs of the pilot project. The SPI work was concluded in time with the expected results and presented as a well structured web site including guidelines, checklist, and a process map. The web site was delivered to the process owners who had a shared responsibility to incorporate the website into the processes and deploy the results. The SPI initiative was considered successful. The pilot project used only those parts of the SPI results that were relevant for them. Lots of the subcontract management information were to be used in the early stages of a project and the pilot project had passed these milestones. The subcontract management initiative did, however, benefit from having the pilot project tightly connected so the results became of better quality. Major parts of the subcontract management initiative were never used by the organisation. Some projects used some parts of the results. The ten SPI project participants shared offices with other engineers and the information spread by itself through informal networks. A major part of the organisation was, however, never given thorough information (except from a mail that probably disappeared in the quantities) about the new support for subcontract management. No initiatives were taken by the process owners to either incorporate the results, nor to further deploy the new web site. 3.2
In late 1999 the software engineering process group was reorganised as the company received the product responsibility for Ericsson’s GSN node (GPRS Supporting Node). The GSN node is a packet switch node used in the mobile system that handles mobile data communication. Ericsson Mobile Data Design needed a focused software technology group to assure full attention to the new
Proceedings of the 36th Hawaii International Conference on System Sciences (HICSS’03)
0-7695-1874-5/03 $17.00 © 2002 IEEE
The Dedicated Initiative
product responsibility and to effectively support the geographically distributed GSN organisation. The software engineering process group was renamed to a software technology group, the initiatives were reorganised, and the group grew to eight people in the end of 2001. One major improvement effort focused on the adaptation and implementation of Rational Unified process (RUP) [18]. At that time 400 employees were working on the GSN responsibility at Ericsson Mobile Data Design and another 400 employees were working at other sites in other countries. Other software technology groups at these sites collaborated closely with the software technology group at Ericsson Mobile Data Design. The intention was to have a dedicated SPI initiative closely connected to each software engineering project that needed new processes and support (see Figure 3.2). The technology subprojects had a steering group that consisted of approximately twelve managers with different development responsibilities (e.g. requirement handling, design, verification, and maintenance). The steering group also included the main project managers that had their own dedicated technology subproject.
Software Engineering Project SPI Initiative (Technology Subproject)
Figure 3.2 – The Dedicated Initiative: Executed through technology subprojects The SPI initiative was planned according to the needs of the main software engineering project. A SPI initiative time plan was made to assure that the new requirement management adaptation would be in place when it was needed by the project. The basic planning principle was to assure that a first dry-run of a requirement management course, including material about the new requirement management adaptation and how to use it, would be in place just-in-time to manage requirement management deployment for the project‘s requirement work. The plan also included time for rework when getting feedback from engineers or inspections. The first part of the project was concluded on time with a course including information on the new requirement management adaptation, related tools and guidelines on how to use the tools, tool mentors, guidelines for requirement management modelling, and a process map. All information was available on a general GSN RUP web site. The requirement management engineers, who all worked hard with system requirements
according to the new processes, did not consider the SPI project particularly successful. Lots of negative feedback was given related to the quality of the new requirement management adaptation. The feedback was, however, considered and stepwise improvements were made within the SPI initiative. When the main software engineering project (in this case the same as the pilot project) was concluded several iterations of improving the requirement management workflow adaptation had been accomplished. All feedback was given to point out what was working in practice and the project had changed the way of working with requirements among the system analysts. 4
Comparison of Initiatives
We proceed by highlighting similarities and differences between the two initiatives, first focusing on structural issues and subsequently on distribution of effort between the different activities in the IDEAL model. 4.1
Focusing on Structure
The two infrastructures for driving SPI initiatives have several similarities. Both initiatives were run by the same SPI group and included the same key persons. The initiatives took place in the same company culture and on the same maturity level [8]. Both initiatives had the same percentage of SPI resources (compared to the number of software developers supported) dedicated to drive the SPI work. Both the initiatives focused on improving a specific key process area on level 2 on CMM [8] and they were both managed by a steering group with the responsibility and commitment to support the SPI work and make necessary decisions. Both projects were also supported by the line organisation, i.e. line managers, in the same way through steering group participation and resource allocation. The goal of the two SPI Initiatives was to improve the way of working within the software development projects. One interesting difference between the two initiatives is, however, that the Generic Initiative, during the time of the project, was considered to be a success while the Dedicated Initiative originally was considered to be problematic. People working with the Generic Initiative were more pleased with their efforts than the people working with the Dedicated Initiative. The results of the Generic Initiative were considered of high quality, while the results of the Dedicated Initiative were considered to be just acceptable. The SPI group was more criticised in the dedicated than in the generic project, which gave the generic group a feeling of doing a better job than the dedicated group. The main difference was, however, that only one of the
Proceedings of the 36th Hawaii International Conference on System Sciences (HICSS’03)
0-7695-1874-5/03 $17.00 © 2002 IEEE
SPI projects led to actual changes that had an impact in the daily project work. The Dedicated Initiative, which worked with improving requirement management, led to a new way of working for the requirement management engineers. The Generic Initiative, which worked with improving subcontract management, led to a well structured and informative web site that was hardly ever used in software projects.
If we explore the Dedicated Initiative from a time spending point of view and map how the SPI effort in man-hours was distributed over the different phases in the IDEAL model we get the curve presented in Figure 4.2. % man hours spent
When exploring these differences in perceived success and implementation success closer, a number of interesting issues and possible explanations emerge. 4.2
Focusing on Effort
If we explore the Generic Initiative spending point of view and map how the terms of man-hours was distributed over phases in the IDEAL model we arrive presented in Figure 4.1.
from a time SPI effort in the different at the curve
% man hours spent
Time INITI
DIAGN.
ESTABL.
ACTING
LEARN.
Figure 4.1 – The effort curve for the Generic Initiative Out of the 600 man hours spent during the twelve week period of the Generic Initiative most of the time was spent on preparing the SPI initiative, getting commitment from the steering group, finding the best resources, having follow-up meetings organised with the steering group, and defining the new guidelines for how to work with subcontract management. Very little time was spent on deploying the results and getting feedback from real use. The SPI group followed the guidelines for driving a Software Engineering Process Group in a software organisation and the focus was on developing a perfect definition of a general subcontract management process. The Generic Initiative did not plan for deployment and it was never really exposed to practice. Those responsible for deployment were pointed out, but no resources were set aside for this activity. No engineers were involved to provide daily feedback of what was working or was not working. In the case there were employees wanting to give feedback there were no responsible person with capacity to receive and process the feedback.
Time INITI
ESTABL.
ACTING
LEARN.
Figure 4.2 – The effort curve for the Dedicated Initiative Out of the 2000 man-hours spent on the Dedicated Initiative during 2000 most of the time was spent on defining the results, deploying the results, getting feedback, and improving the results. Time was spent on issues like finding the right resources, getting commitment from the steering group to the SPI work, and reporting progress. But a major part of the time was spent in the acting phase in the IDEAL model. The Dedicated Initiative planned for making the change happen. The Dedicated Initiative took the process to practice and the initiative was exposed to lots of feedback. The project interacted with people who were supposed to change their way of working. Their frustrations led to criticism and the criticism led to changes. The Dedicated Initiative integrated in this way acting and learning in the IDEAL model [21] to generate feedback and to assure process quality. The most important similarities and differences between the two SPI initiatives are summarised in Figure 4.3. In the following we discuss these in relation to the body of knowledge on SPI and organisational change and we highlight important implications for both practice and research.
Proceedings of the 36th Hawaii International Conference on System Sciences (HICSS’03)
0-7695-1874-5/03 $17.00 © 2002 IEEE
DIAGN.
Generic Initiative
Dedicated Initiative
Organisational Context
Ericsson
Ericsson
Key Actors
The SPI Group
The SPI Group
Structural Approach
Detached, Generic
Integrated, Dedicated
High
High
Improved Practice
Improved Practice
Subcontract Management
Requirement Management
Perceived Success
High
Low
Implementation Success
Low
High
Management Attention SPI goal Process Focus
Figure 4.3 – Comparison of Generic and Dedicated Initiatives 5
Discussion
The two SPI initiatives are both examples of organisational change processes in which new or modified technologies play a key role. We can therefore get a deeper understanding of the complexities and dynamics involved by drawing upon established, theoretical perspectives on such change processes while at the same time relating to knowledge on SPI. In the following, we start by relating the case to the literature on SPI (Section 5.1). We then adopt a broader perspective and interpret the two initiatives as diffusion of technology innovations (Section 5.2). We conclude the discussion by pointing out implications for practice and research (Section 5.3) and possible limitations (Section 5.4) 5.1
Making SPI Happen
and deployed, plans for full deployment are developed and executed) and Leveraging (lessons learned are made, collected data is analysed, conclusions for improvements of the SPI work is made). McFeeley [21] stresses that it is necessary to commit resources to drive the SPI work and commit practitioners to participate in the SPI work to assure successful SPI implementation. McFeeley also argues that change efforts of a certain magnitude causes substantial fear and uncertainty throughout the organisation. Knowledge of change management is therefore needed by both SPI drivers and practitioners to manage successful SPI implementation. The Generic and the Dedicated SPI Initiative differed in perceived and actual success, see Figure 4.3. The Generic Initiative’s success was perceived as high and the Dedicated Initiative’s success was perceived as problematic or low. But the Generic Initiative had low implementation success and the Dedicated Initiative had high implementation success. One possible explanation can be found in McFeeley’s statements regarding fear and uncertainty. The Generic Initiative was only taken to the part of the Acting phase were solutions were created. The Generic Initiative never brought the solutions into practice and was therefore not exposed to the fear and uncertainty that practitioners feel when they are exposed to change. The Dedicated Initiative did, however, involve deployment in the Acting phase and was therefore exposed to uncertainty and criticism of the SPI work. The perceived success of the Dedicated Initiative was therefore low even though the Implementation success was high. Weinberg [25] supports this view by stressing that performance in a software project, and in general, suffers during a period of change by refering to the Satir Change Model (see Figure 5.1). Weinberg argues that people in the Chaos phase become defensive and alienated because their old survival fears are aroused. When people get exposed to change and have to leave old status quo, a pattern of resistance occur, like for example not showing up at meetings or avoiding new challenges. The Satir Change Model emphasises that it’s necessary to pass this Chaos phase to reach improved performance. Performance
The IDEAL model is a model for guiding SPI initiatives [21]. The model is based on five recommended and generic steps that can be used for SPI managers to manage and drive the SPI initiatives in an organization. The five steps are Initiating (starting point, defining the vision) setting general SPI goals, defining an infrastructure for SPI), Diagnosing (SPI action plans are initiated according to SPI vision and goals, finding potential SPI areas), Establishing (prioritization of SPI areas, defining metrics, establishing tactical action plans), Acting (solutions to the prioritised SPI areas are created, piloted
Proceedings of the 36th Hawaii International Conference on System Sciences (HICSS’03)
0-7695-1874-5/03 $17.00 © 2002 IEEE
New status quo Integration & practice
Old status quo
Chaos
Time
Figure 5.1 - The Satir Change Model
The Generic Initiative had no experience of the reactions in the Chaos phase and was therefore perceived a success. The Dedicated Initiative took the SPI work into deployment activities where practitioners might react defensively and alienated as the Chaos phase is entered. The necessary step towards implementation success and improved performance is, however, taken in this way. Another important interpretation of the differences between the two intiatives at Ericsson Mobile Data Design relates to commitment within SPI [2], [3], [15]. Already in the mid 1980’s when Watts Humphrey [14] developed the Capability Maturity Model, commitment to SPI was recognized as the key success factor for successful SPI implementation. “SPI requires investment – it takes planning, dedicated people, management time and capital investment”, “To improve someone must work on it”, and “Unplanned SPI is wishful thinking” are statements from Humphrey [14] that stress the need of dedicated people working with SPI to assure implementation success. Humphrey argues that enthusiastic, technically and politically capable and dedicated resources with management’s confidence are necessary to reach successful SPI implementation. Humphrey calls these dedicated resources change agents. If SPI isn’t anybody’s job, it isn’t surprising that it doesn’t get done, says Humphrey. The dedicated resources are needed to assure that practitioners overcome resistance, that practitioners are provided with adequate training, and that projects will be provided with the necessary consultation. Both the Generic and the Dedicated Initiatives had change agents that supported the SPI work. But the change agents in The Generic Initiative were committed to creating new processes and documenting them, while the change agents in The Dedicated Initiative were committed throughout the whole Acting phase including deployment, training, and project consulting. The change agents in the Generic Initiative were also focused on supporting different software engineering projects and their solutions had to be generic to fit this variety of needs. But it turned out that the solutions were too generic for software engineering projects to use without more dedicated work from the change agents. The created solutions in The Dedicated Initiative focused on the needs of one particular software engineering project and implementation was directly supported by the change agents without further adoption activities. A final key issue in the SPI literature that relates to the two initiatives is that of participation [20]. Aaen argues that software developers need help to be able to help themselves in the area of designing software processes [1]. He describes the situation where change agents design development processes separated from the software developers as Improvement by Design and the situation
where change agents drive and help the software developers to design their own development processes as End- user SPI. Aaen argues that End-user SPI is a more effective strategy than Improvement by Design. Our two cases support this argument. The Generic Initiative is based on Improvement by Design as the change agents in that initiative never worked in close cooperation with software developers to assure successful SPI implementation. The Dedicated Initiative is in contrast an example of End-user SPI as the change agents in that initiative worked closely with software developers in the deployment phase to make the SPI implementation successful. The Dedicated Initiative supported only one software engineering project and was therefore able to collaboratively address the specific issues that were relevant in that context. 5.2
Diffusing Software Innovations
Swan et al [23] suggest two complementary perspectives on diffusion of technology innovations in general that provide further insights into the relation between the two SPI initiatives at Ericsson Mobile Data Design. The two different perspectives are technology supplier and service provider. The technology supplier focuses on methods, tools and guidelines and the critical success factor is the repertoire of software technologies offered to the software organisation. The repertoire of technologies is described using standardised representations and it is diffused to software practitioners through vertical networks. The primary claim is that this approach exploits knowledge and experiences across projects and departments within the software organisation. The role of the change agent is primarily to develop, purchase and transfer software technologies and the dominant underlying metaphors are software process and jigsaw, i.e. to design and implement innovations as elements in an overall, preconceived plan. The major challenge is to fit pieces of technology together to produce standardised software processes across the organisation in a predictable way. The Generic Initiative is mainly based on this perspective. The initiative is focused on development and documentation of new processes, these processes are to be used based on management prescriptions, the underlying assumption is that different projects can re-use key elements of the processes, and the critical success factor relates to the designed processes. The change agents develop in this way coherent, company specific processes inspired by best practices or state-of-the-art models like the Capability Maturity Models. In contrast, the service provider suggests a different
Proceedings of the 36th Hawaii International Conference on System Sciences (HICSS’03)
0-7695-1874-5/03 $17.00 © 2002 IEEE
approach focusing on the use of software technologies. Software practices vary across projects and departments and they encompass tacit knowledge that is shared through horizontal networks of collaboration. The critical success factor is trust and the primary claim is that this approach supports learning through sharing and synthesis of knowledge across communities-of-practice [6]. The primary function of the change agent is to facilitate use of software technologies and knowledge sharing and the dominant underlying metaphors are software community and kaleidoscope, i.e. to carry out interventions into current practices as part of an emerging and ever changing scheme. The key challenge is to facilitate interaction and exploration to develop a variety of communities of practice. The Dedicated Initiative is an expression of the service provider perspective. The focus is on the practical use of processes and processes are only considered to constitute a minor part of practice. Other forms of explicit knowledge and equally important various forms of tacit knowledge play important roles in shaping development practices in each project. To improve practitioners need to share, explore, and further develop knowledge across different communities-of-practice and change agents need to create sufficient trust and collaboration to make that happen. Success is therefore not so much dependent on the relations between different processes as it depends on the relation between each process and the practices in which it is used. 5.3
Implications for Practice and Research
Our research supports the widely accepted view that commitment to SPI is a necessity for successful SPI implementation [3], [14], [15], [21]. Commitment to SPI is, however, not easy to establish and maintain. Our research shows how dedicated initiatives support this process and faclitates implementation success. Managers and SPI practitioners are adviced to organize dedicated initiatives when possible, to plan and track the distribution of effort over the IDEAL activities to facilitate implementation success, to encourage practitioner participation, and finally to acknowledge frustration and even chaos as features of practical improvement efforts. The reason is simply that this approach supports not only perceived, but also implementation success. It is, however, very important not to reject the benefits of generic initiatives. Such initiatives are, together with the general role as technology supplier, needed to efficiently diffuse tested technologies to larger communites-ofpractice. The critical question to ask is: how many resources should we spent on developing, adopting and implementing specific processes? When and to what extent should we rely on dedicated intiatives and on the role as service provider to give priority to providing
quality solutions suited to specific contexts? When should we instead rely on generic approaches and on the role as technology supplier to economise with the resources spent on each new initiative? In such cases, how can we then benefit from active participation of the engineers that are going to adopt the new processes? This dilemma relates to the distinction between level 2 (project specific) and level 3 activities (organisation wide) in the Capability Maturity Model [14] and to Aaen’s distinction between End- user SPI and Improvement by Design [1]. More research is needed to explore this dilemma further in the context of software improvement initiatives. Particular questions of interest are: How should dedicated and generic initiatives be combined? To what extent is it possible to modify generic initiatives with elements of the dedicated approach without loosing the efficiency of the generic approach? 5.4
Limitations
In case study research there are many factors that can affect the outcome. In this research we can point out a few limitations that make the conclusions more an indication than a clear answer. First we stress that changed practices is not equal to improved practice. There is always a possibility that a changed can result in worsening practice. In this study we argue, however, that without implementation success there will be no improved practice and that is why this notion can be considered an important indicator of success. Another limitation is that there might be other reasons than the distribution of effort over phases that affected the results. Could factors like competence, culture and previous knowledge of the area be reasons for SPI implementation success or failure? This is of course possible. But we just argue that the findings presented in this paper are indications of one important area that affected the SPI result. We should also ask whether a generic SPI Initiative necessarily implies not focusing on deployment. Is it possible that a generic SPI Initiatives can give good result if deployment activities are not neglected? We argue that a dedicated SPI Initiative makes it easier to focus on deployment as it supports only one software engineering project. The potential of being successful in implementation with generic SPI Initiatives needs to be further explored.
6
Conclusions
Two SPI initiatives have been compared and contrasted. The initiatives differed in perceived success and implementation success. A useful indicator for these
Proceedings of the 36th Hawaii International Conference on System Sciences (HICSS’03)
0-7695-1874-5/03 $17.00 © 2002 IEEE
differences was the % of man-hours spent on the different phases of the IDEAL model [21] (see figure 6.1). The Generic Initiative spent the main man-hours on the early phases of the IDEAL model, the perceived success was high, but the actual impact on practice was low. The Dedicated Initiative spent the main man-hours on the later phases of the IDEAL model, the SPI initiative was perceived as rather problematic, but it had high impact on practice. The Dedicated Initiative took the modified processes into a deployment phase and was consequently exposed to criticism and feedback. This learning improved the solution and also assured attention, implementation and use. The Dedicated Initiative planned for and focused hard on the deployment activities in the Acting phase. % man hours spent
Time INITI
DIAGN.
ESTABL.
ACTING
LEARN.
Figure 6.1 – Moving the effort curve in the IDEAL model Change creates fear and uncertainty, which lead to criticism towards solutions independent of the quality of these solutions. Processes that are not exposed to criticism will easily be perceived as a success whereas processes taken to practice are more likely to be considered problematic. SPI initiatives that aren’t exposed to practitioners will never enter the chaos phase where resistance, alienation and fear to the change emerge and call for corrective actions. This chaos phase is, however, a necessary step towards reaching higher performance.
Acknowledgement We want to thank the reviewers at HICSS-36 for valuable criticism and support.
References [1] Aaen I. (2002) Challenging Software Process Improvement by Design. Proceedings of ECIS 2002 The Xth European Conference on Information Systems, Gdansk, Poland, June 68.
[2] Aaen, I., Arent, J., Mathiassen, L. and Ngwenyama, O. (2001) A Conceptual MAP of Software process Improvement. Scandinavian Journal of Information Systems, Vol. 13, 123-146. [3] Abrahamsson, P. (2000) Is Management Commitment a Necessity After All in III: Software Process Improvement? Euromicro '00, Maastricht, The Netherlands, IEEE Computer Society, 246-253. [4] Bach, J. (1995) Enough About Process: What We Need are Heroes. IEEE Software, 12, 2, 96-98. [5] Bollinger, T. B. and McGowan, C. (1991) A critical look at software capability evaluations. IEEE Software, Vol. 8, No. 4, 25-41. [6] Brown, J. S. and Duguid, P. (1991) Organisational Learning and Communities-of-Practice. Toward a Unified view of Working, Learning, and Innovation. Organization Science, Vol. 2, No. 1. [7] Cunningham J.B. (1997) Case Study Principles for Different Types of Cases, Quality and Quantity, Vol. 31, 401-423. [8]Curtis, B., Paulk, M. C., Weber, C. V. and Chrissis, M. B. (1994) The Capability Maturity Model: Guidelines for Improving the Software Process. CMU/SEI, AddisonWesley. [9] Diaz, M. and Sligo, J. (1997) How Software Process Improvement Helped Motorola. IEEE Software, Vol. 14, No. 5, 75-81. [10] Fayad, M. E. and Laitinen, M. (1997) Process Assessment Considered Wasteful. Communications of the ACM, Vol. 40, No. 11, 125-128. [11] Fowler, P. and Rifkin, S. (1990) Software Engineering Process Group Guide. CMU/SEI-90-TR-24, Software Engineering Institute, Pittsburgh, Pennsylvania, USA [12] Galliers, R. D. (1991) Choosing Appropriate Information Systems Research Approaches: A Revised Taxonomy. In: Nissen, H.-E., H. K. Klein and R. A. Hirschheim (Eds.): Information Systems Research: Contemporary Approaches & Emergent Traditions, Elsevier. [13] Haley, T. J. (1996) Software Process Improvement at Raytheon. IEEE Software, Vol. 13, No. 6, 33-41. [14] Humphrey, W. S. (1989) Managing the Software Process. Reading, Massachusetts: Addison Wesley. [15] Humphrey, W. S. (1997) Managing Technical people innovation, teamwork and the software process Reading, Massachusetts: Addison Wesley. [16] Humphrey, W. S., Snyder, T. R. and Willis, R. R. (1991) Software Process Improvement at Hughes Aircraft. IEEE Software, Vol. 8, No. 4, 11-23. [17] Humphrey, W. S. and Curtis, B. (1991) Comments on "A Critical Look at Software Capability Evaluations". IEEE Software, Vol. 8, No. 4, 42-46. [18] Kruchten P. (2000) The Rational Unified Process – An Introduction. Addison-Wesley [19] Mathiassen, L. (2000) Collaborative Practice Research. In: Baskerville, R., J. Stage and J. I. DeGross: Organizational and Social Perspectives on Information Technology, Kluwer Academic Publishers. [20] Mathiassen, L., Nielsen, P. A. and Pries-Heje, J. (2001) Learning SPI in Practice. In: Mathiassen et al. (Eds.) Improving Software organisations. From principles to Practice. Upper Saddle River, N. J.: Addison-Wesley. [21] McFeeley, B. (1996) IDEAL: A User's Guide for Software Process Improvement. Pittsburgh: SEI. Handbook, CMU/SEI-96-HB-001.
Proceedings of the 36th Hawaii International Conference on System Sciences (HICSS’03)
0-7695-1874-5/03 $17.00 © 2002 IEEE
[22] Sakamoto, K., Kishida, K., and Nakakoji, N. (1996) "Cultural Adaptation of the CMM: A Case Study of a Software Engineering Process group in a Japanese Manufacturing Company." in A. Fuggetta and A. Wolf, Eds., Software Process, 137-154: Wiley. [23] Swan, J., Newell, S., Scarborough, H. and Hislop, D. (1999) Knowledge Management and Innovation: Networks and Networking. Journal of Knowledge Management, Vol. 3, 262-275.
[24] Wohlwend, H. and Rosenbaum, S. (1994) Schlumberger's Software Improvement Program. IEEE Transactions on Software Engineering, Vol. 20, No. 11, 833-839. [25] Weinberg, Gerald M. (1997) Quality Software Management volume IV – Anticipating change. Dorset House Publishing. [26] Yin, R. (1994) Case Study Research. Newburry Park, California: Ssage Publicatio
Proceedings of the 36th Hawaii International Conference on System Sciences (HICSS’03)
0-7695-1874-5/03 $17.00 © 2002 IEEE
