The use of an electronic process guide in a medium‐sized software ...

SOFTWARE PROCESS IMPROVEMENT AND PRACTICE Softw. Process Improve. Pract. 2006; 11: 21–34 Published online in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/spip.250

The Use of an Electronic Process Guide in a Medium-sized Software Development Company Nils Brede Moe*,† and Tore Dyb˚a

Research Section

SINTEF ICT, NO-7465 Trondheim, Norway

For process models to be useful, more and more software companies not only tailor their process models to the specific needs of the company, but also make them available on the company’s intranet. In this article, we try to understand what characterizes the use of an electronic process guide (EPG) among developers in a medium-sized software company. We describe the findings that emerged from a grounded theory study in which we interviewed 19 developers and project managers. We found that implementing an EPG is not a straightforward process, but that it depends on several organizational factors. Our findings suggest that the process guide should be anchored among the leaders, that users should be involved in creating and updating it and offered training and support during use, that efforts should be taken to keep the guide updated, that templates and checklists are seen as the most useful features, that integration with other tools make it more useful, and, finally, that it should be easy to tailor the process guide to different project types or activities. The insights gained from this study can be used as recommendations for other medium-sized software companies when implementing electronic process guides. Copyright  2006 John Wiley & Sons, Ltd. KEY WORDS: software process improvement; software methodology infusion; electronic process guide; case study; grounded theory

1. INTRODUCTION Effectively disseminating process knowledge to process participants is crucial in any software process improvement (SPI) effort. Process participants need effective guidance when process conformance is important, when a process changes frequently, and when new personnel join a project. ∗ Correspondence to: Nils Brede Moe, SINTEF ICT, NO-7465 Trondheim, Norway † E-mail: [email protected] Contract/grant sponsor: Research Council of Norway; contract/grant number: 156701/220

Copyright  2006 John Wiley & Sons, Ltd.

Traditionally, this has been the realm of large organizations, and the way of describing and communicating processes has focused on printed standards and handbooks. These handbooks are generally difficult to keep up-to-date and almost impossible to customize based upon a specific user’s needs. Therefore, such handbooks are often of limited use as SPI facilitators, especially in small and medium-sized companies. For process models to be useful, increasingly more software companies not only tailor their process models to the specific needs of the company, but also make them available on the company’s intranet. This way the traditional process handbook shifts from a large, bulky pile of paper to a flexible on-line structure allowing easy access to all relevant

Research Section

information by means of an electronic process guide (EPG) (Moe et al. 2002, Scott et al. 2002, Dyb˚a et al. 2004). An EPG can be seen as a structured, workfloworiented, reference document for a particular process, and exists to support participants in carrying out the intended process (Kellner et al. 1998). A process guide typically includes the following basic elements: • Activities: descriptions of how things are done, including an overview of the activities and details regarding each individual activity. • Artifacts: descriptions of products created or modified by an activity, either as a final or intermediate result of the activity or as a temporary result created by one of the steps. • Agents: descriptions of entities that can perform activities. These descriptions are in terms of characteristics such as skills, costs and availability. An agent can be an individual or a group. • Roles: details regarding the roles and agents involved in performing the activities. • Resources: details regarding the tools and techniques used to support or automate the performance of an activity. On the basis of experience with paper-based guides and industrial needs of the software engineering domain, Kellner et al. (1998) proposed a set of basic requirements and design principles for EPGs. The EPG should provide the basic information units, i.e. activities, artifacts, agents, roles and resources, as well as the major relationships between them. Most importantly, an EPG should provide all the information elements and relationships contained in a good paper-based process guide. In addition, it should capitalize on diagrams, tables and narrative to provide an effective user interface. It should also make extensive use of hyperlinks to support flexible navigation and direct access to supporting information such as examples and templates. In an EPG, it should be easy to access desired information (e.g. frequently used information) very quickly. Each page in the EPG should use a familiar structure to facilitate orientation and usage, and the user should not be overwhelmed with too many overlapping windows. However, the potential of EPG’s can only be realized when key capabilities are not only adopted, but also infused across the organization. This contrast between system availability and system use Copyright  2006 John Wiley & Sons, Ltd.

22

N. B. Moe and T. Dyb˚a

has been noted by Fichman and Kemerer (1993), who distinguished between a firm’s adoption of a technology and its assimilation of it. At the individual level, there is also a growing body of studies focusing on the determinants of technology acceptance and utilization (e.g. (Davis 1989, Dyb˚a et al. 2004, Venkatesh and Davis 2000)). New information technologies, such as an EPG, represent innovations for the potential adopters. Consequently, much of the research on individual adoption of information technologies derives its roots from the diffusion of innovation literature, in which individuals’ perceived characteristics of innovating (PCI), among other factors, are claimed to be significant influences on user acceptance (Rogers 2003). Other models that attempt to explain the relationship between user perceptions, attitudes, user intentions and eventual system use include the technology acceptance model (TAM) (Davis 1989, Venkatesh and Davis 2000, Dyb˚a et al. 2004), the theory of planned behavior (TPB) (Ajzen 1991) and the model of personal computer utilization (MPCU) (Thompson et al. 1991). In an experience report on the use of process guides as a SPI in a small Norwegian company, we found that one of the main reasons for success with institutionalizing the EPG was the fact that the users were strongly involved in developing the EPG (Moe et al. 2002). The EPG in this study had been used for over a year. This study did not, however, provide any actual usage pattern. In another study, Scott et al. (2002) presented the installation and use of an EPG within a small-to-medium-sized software development company called Allette Systems in Australia. The study showed that the EPG was used in Allette, albeit with a general pattern of declining use over the period of the evaluation. The main use of the EPG was to support process improvement, but it was not used to guide projects and the users felt it was neither convenient nor useful to access templates. The motivation for the work described in this article, which extends and updates (Moe and Dyb˚a 2004), is to understand the adoption and infusion of EPGs in companies with voluntary use. The core research question is therefore: What characterizes the use of an EPG among developers and project leaders in a medium-sized software company? Softw. Process Improve. Pract., 2006; 11: 21–34

Research Section

The rest of the article is organized as follows: In the next section, we describe the research method and the research site that we call Company X. We then describe the findings that emerged from the grounded theory study, followed by the discussion and conclusion.

The Use of an Electronic Process Guide

things to work). Grounded theories, because they are drawn from data, are likely to offer insight, enhance understanding and provide a meaningful guide to action. Although grounding concepts by data is the main feature of this method, creativity of researchers also is an essential ingredient. How we used this method is described in detail in Section 2.3.

2. RESEARCH METHOD 2.1. Study Context This study is a single case study (Yin 1994) using qualitative interviews with employees as the only data source. The data was analyzed according to principles in grounded theory as described by Strauss and Corbin (1998). Grounded theory is a research method that seeks to develop theory that is grounded by data that has been systematically gathered and analyzed. It is a form of field study that systematically applies procedural steps to develop an exploration about a particular phenomenon. As Strauss and Corbin (1998) explained: A grounded theory is one that is inductively derived from the study of phenomenon it represents. That is, it is discovered, developed, and provisionally verified through systematic data collection and analysis of data pertaining to that phenomenon. Therefore, data collection, analysis, and theory stand in reciprocal relationship with each other. One does not begin with a theory, then prove it. Rather, one begins with an area of study and what is relevant to that area is allowed to emerge (p. 23). In grounded theory, data collection, analysis, and eventually theory stand in close relationship to one another. A researcher does not begin a project with a preconceived theory in mind (unless his or her purpose is to elaborate and extend an existing theory). Rather, the researcher begins with an area of study and allows the theory to emerge from the data. Constant comparison is the heart of the process. In this study, the interviews were compared to other interviews. By comparing the interviews a theory quickly emerged, and when it began to emerge the data was compared with the existing theory. A theory derived from data is more likely to resemble the reality than a theory that is derived by putting together a series of concepts based on experience or solely through speculation (how one expects Copyright  2006 John Wiley & Sons, Ltd.

The context for this research is Company X, which is a medium-sized software company with approximately 150 employees in six organizational units. About 80% of the employees have a Masters or a Doctoral degree. A separate group within Company X, called the SPI group, is responsible for building competence in software development processes, methodology and supporting tools. This group is also responsible for coordinating the use of processes, methodologies and supporting tools across Company X projects. This responsibility includes the development, support and infusion of the company’s EPG. The EPG has been developed in close cooperation with the intended users through small workshops and meetings, where the company’s best working practice has been mapped. The EPG has been introduced to the employees through department meetings, and through training and direct support from the SPI group. The EPG is mainly supplementary to the company’s work procedures based on ISO standards and, thus, of a voluntary nature. The specific EPG examined in this research consists of custom-created process models for the company’s internal use to provide process directions for the complete software life cycle. The EPG uses fundamental concepts from the Capability Maturity Model (CMM) (Paulk et al. 1995), Microsoft Solution Framework (MSF) http://msdn.microsoft.com/versustudio/enterprise/msf/ and Rational Unified Process (RUP) (Krutchen 2000). The model is meant to be scalable to both minor and larger assignments. The EPG is web-based and available through the Intranet, reflecting the company’s best working practices. Figure 1 shows the top-level view of Company X’s EPG. The model defines a series of work phase definitions providing activity descriptions, flow charts, guidelines, procedures, standards, checklists and templates for deliverables. Softw. Process Improve. Pract., 2006; 11: 21–34

23

Research Section

N. B. Moe and T. Dyb˚a

Figure 1. The top-level view of Company X’s electronic process guide

Verification and validation requirements and criteria for approval of project deliverables are included in the definition of milestones and decision points. Figure 2 shows an example of the description for the development process. Company X has also implemented a project web. The project web offers a site for the project together with all the documentation. After the templates, checklists, etc. have been collected from the process guide, these documents are placed on the project web. The researchers have worked together with the company for 3 years, but were not involved in developing or introducing the EPG in Company X. The EPG was introduced 2 years before this study started.

2.2. Data Sources We gathered data through semistructured interviews, using the following questions as a starting point: • How often do you use/access the Process Guide? • Do you like using tools like the Process Guide? Copyright  2006 John Wiley & Sons, Ltd.

24

• Have you used tools like the Process Guide previously? If yes: purpose, frequency, how was it? • What do you see as the main purpose with the Process Guide? • Who do you think benefits mostly from using the Process Guide? • What features of the Process Guide do you use most? • What tasks do you use the Process Guide to help you with? • Do you find using the Process Guide beneficial to your work? If yes, what are the benefits? • Do you think there are disadvantages with the Process Guide? If yes, in what settings? • What do you think should be improved about the Process Guide? • Do you use any other resources to support your work? If the person interviewed did not use the Process Guide, we added the following question: Why do you not use the guide? We planned to interview 20 persons. Five department leaders were asked to point out four persons from each of their departments, two whom they Softw. Process Improve. Pract., 2006; 11: 21–34

Research Section

The Use of an Electronic Process Guide

Figure 2. The developing process

thought preferred to use the EPG, and two whom they thought did not use it or were against using it. One person could, however, not attend the Copyright  2006 John Wiley & Sons, Ltd.

interview. This selection was done to find out why people use the EPG, and why they do not use it. Because of this selection strategy, it is not possible Softw. Process Improve. Pract., 2006; 11: 21–34

25

Research Section

to say anything about the average users of the EPG. The interviewer did not know who belonged to which category. The interviews lasted 20–30 minutes, and have been recorded and transcribed in full for the analysis. Eighteen of the subjects had software development as their primary job function, while one worked with administrating the databases. Four of the developers were also project leaders. These four are mentioned as project leaders in the result chapter to distinguish between this group and the group of only developers. Five of the subjects worked alone or together with another person on a project, eight persons worked in projects consisting of three to six persons, two persons worked on projects with six or more people, and four persons did not work in projects. Two of the persons interviewed reported that they had participated in the work of creating the EPG. 2.3. Data Analysis All data from the interviews were imported into a tool for analysis of qualitative data – Nvivo. 2 Nvivo is a tool for analyzing qualitative data available from QSR International, www.qsrinternational.com. In the data analysis we first used open coding followed by axial coding, and then selective coding (Strauss and Corbin 1998). During open coding, we read all interviews and coded interesting expressions of opinions in the text by assigning the expression to a category with similar expressions. Open coding is the analytic process through which concepts are identified and their properties and dimensions are discovered in data. Events, happenings, objects and actions/interactions that were found to be conceptually similar in nature or related in meaning, were grouped under more abstract concepts termed categories. A category represents a phenomenon, that is, a problem, an issue or an event that is defined as being significant to the respondents. The product of labeling and categorizing are the basic building blocks in grounded theory construction. An example of open coding are the expressions ‘‘you have one place to find information’’ and ‘‘the documentation you need in the project, is available’’ that was coded into ‘‘information available’’. After the open coding, where we created concepts and categories, it was time to create connections between categories and their subcategories. The coded pieces of text from the open coding were Copyright  2006 John Wiley & Sons, Ltd.

26

N. B. Moe and T. Dyb˚a

again categorized with other expressions of benefits of the process guide, such as working in a more structured manner and with more control, in what is referred to as axial coding. Finally, we tagged all interviews with information (attributes in Nvivo) about use level, and work position. During the selective coding, where we integrated and refined the theory, the matrices generated by Nvivo were very important analytical tools.

3. RESULTS We generated reports for eight main groups of axial codes to get the results. These are described next: • What is used in the EPG? • What is the intention of introducing the EPG in the company? • Who needs the EPG most? • Perceived usefulness • Perceived compatibility • Perceived ease of use • Subjective norm • Why is the EPG not used? We use quotes from the interviews in the following presentation of the results. 3.1. What is Used in the EPG? 3.1.1. Use Level Three persons claimed they did not use the EPG at all. Only one person used it daily, and three persons answered that they used it monthly. Nine persons used it less than once a month, but one of them used it daily in the previous project. The last three persons used an alternative model because this was required by the customer. The alternative model only consisted of a few templates, and was therefore not used frequently. The developers working in small projects, or not in projects at all, reported the lowest use level. Table 1 shows the use frequency of the process guide. There was a big difference among the developers and the project leaders. Among the four project leaders, one used it daily while the rest used the EPG monthly. This means that either the developers did not use it, used an alternative model, or used the EPG seldom. One said: The use depends on the phase of my projects. When we were starting up the last project, we Softw. Process Improve. Pract., 2006; 11: 21–34

Research Section

The Use of an Electronic Process Guide

Table 1. The use frequency of the process guide

developers was the most important method. Two persons said:

Frequency Daily Weekly Monthly Less than once a month Never Alternative model

1 0 3 9 3 3

looked at it and collected the templates, and wrote vision and scope, conceptual design and such. Then we used it actively. Now it is mostly used by the project leader, and I don’t have need for accessing it, and I know the model. 3.1.2. Templates and Checklists It was a common opinion among the developers and project leaders that the templates and checklists were used most frequently. Three of the subjects said: I download the template, and then I write the vision. I also use the checklists. I have a few entries when I need the checklist for the Freeze process. I don’t use the checklists, only the templates. It is especially in the beginning of the project, where I need the templates. Also, persons claiming that they did not use the EPG reported using the templates. Two said: I continually collect the templates from the guide. The only thing I know is that I’m supposed to write the three documents. I collect the templates from a place on the intranet. 3.1.3. Other Functionality In addition to templates and checklists, the interviewees reported usage of coding standards, tips and tricks, good advice and user documentation for the test system. 3.1.4. Alternative Sources Several of the developers with little or no use reported that they used alternative sources/techniques when they needed support. Talking to other Copyright  2006 John Wiley & Sons, Ltd.

The coding standard is oral. People talk in the corridors, and there is a person that most people listen to, and he acts like the oracle. . . . it is important that you know people. You must know whom to talk with if you are going to have a chance of knowing how to solve the tasks. . . . if you are new here, you are not helpless, but it takes a while before you are up and running.

3.2. What is the Intention of Introducing the EPG in the Company? When asked about the intention of introducing the EPG in Company X, several interviewees said that the EPG would make it possible to deliver faster, work more efficiently and to lead the projects better. One said: It must be to deliver faster, better, and with better quality, than if you did not use it. You will produce projects that are similar to each other, and this makes it easier for new people to join the project and do a job. It was also commented that the EPG makes it easier to learn from former experience. The interviewed subjects also meant that the EPG helps the developers in carrying out their work by describing how they can perform their job. It was also stated that these descriptions help them work in a more similar way. Three persons said: Get better flow in the project, make us work more similarly, and then you don’t forget the things you need to do. You need it in big projects, where several people work together, and you need to disseminate knowledge, or should I say that we are able to understand each other. It makes the workday easier for us, you get good templates, and it is pretty obvious what you need to do in which phase. You use the checklists to make sure you don’t forget things, and then you do it the right way. Softw. Process Improve. Pract., 2006; 11: 21–34

27

Research Section

Everyone stated useful aspects of the process guide independent of use level. The perceived usefulness is discussed later. 3.3. Who Needs the EPG Most? 3.3.1. The Project Leaders and Department Managers It was a common opinion among the project leaders and project participants that the project leaders and department managers need the EPG the most. Two programmers and one project leader said: I see the need for a process model, but what I really need as a developer is the help with using less time documenting what we do. The EPG helps you document the process, and this is very helpful for the project leaders. It is very important for the project leader, but it is also important that the project leader communicate the EPG as a positive thing to the developers, and not as some garbage we have to use. This is important. But I think it is more useful for the project leaders, because they enter the EPG, use the checklists and verify that the phases are completed as planned. The project leaders are sitting closest to the project model, hence the project leaders will use it most. The developers not using the EPG or using it seldom also meant that project leaders were the main target group of the EPG. 3.3.2. Developers Some meant that the EPG is most useful for the developers: We (the developers) have most use of it, since we don’t have to find all the documents ourselves. 3.3.3. People Working Together Several of the interviewed subjects meant that developers working in a team benefit from the use of the EPG. Two persons said: People working together benefit from using it, because they need to work in the same way. You need it in large projects where several work together. Then you need to disseminate Copyright  2006 John Wiley & Sons, Ltd.

28

N. B. Moe and T. Dyb˚a

knowledge and we need to understand each other. 3.3.4. Customers It was also commented that the customer would benefit from the EPG: It gives the customer an opportunity to understand how we work, and that is very useful. I have used the EPG in meetings, just to tell them how we work together on a project. 3.4. Perceived Usefulness We wanted to explore how useful the EPG is perceived. Perceived usefulness is defined as the degree to which the subjects believe that using the EPG would enhance his or her job performance (Davis 1989). In other words: How does the EPG affect the job performance, productivity and quality of work? Does the EPG influence how easy it is to do the job, and do the advantages of using the EPG outweigh the disadvantages? All the project leaders described useful aspects of the EPG, but they did not mention any useless aspects. The developers mentioned almost the same number of both useful and useless aspects of the guide. 3.4.1. Productivity Some of the developers claimed that the EPG decreases the productivity. One said: I think it steals focus away from the tasks that are to be done. On the small projects that I work on it is better to have a good dialogue with the customer. Several developers argued that it decreases the productivity, since you have to produce a lot of documentation that is not really needed. Two said: You have to produce a lot of papers, and perhaps nobody reads them later. A lot is written in all the documents, and there is little information in connection to the project. I’m a typical science graduate, and I’m not fond of writing a lot. I like lists of keywords and checklists. I have seen a lot of documents with a lot of words and little content. Softw. Process Improve. Pract., 2006; 11: 21–34

Research Section

One stated that the EPG decreases the productivity in the beginning of a project: In my opinion, the EPG is very useful. There are more advantages than disadvantages. But it is very hard to get the entire company to use the tool, because in the beginning you need to invest more time than you get back. You need to use it in the entire project before you understand the usefulness of the EPG.

The Use of an Electronic Process Guide

Some stated that the EPG makes it easier to follow the process, but does not increase the productivity. One said: I’m not a formal person. There are a lot of really good documents in the EPG, but there are also some documents that are NOT very good. The good documents are those describing the process, but the developers need help documenting what they do. We need help to write system documentation.

3.4.2. EPG Makes Work Easier It was a common opinion that the EPG makes work easier, since it supports a lot of useful checklists, and helps finding the right documents and filling them out. Two said:

3.4.3. Quality of Work The EPG was also perceived to increase the quality of work, since it gives instructions on what to remember. One argued that:

You have a template to follow . . .. You don’t need to think about what you should do. It gives you a very good start on the document. All the things you need are written there.

I see it as a useful tool. You have a customer, and then it is important not to make a blunder. It is OK to have the templates you can access, to clarify scope, risk and such things, and that you agree with the customer what you are intended to produce.

But there was an uncertainty about the value of the documentation produced. One stated: It is useful for the developers, since we do not need to find all the documents ourselves. The question is: Who reads them? People mostly want Power Point presentations. Some stated that the EPG makes it easier to perform quality assurance in the project. One said: As I mentioned earlier, this is quality assurance in a positive direction. It is much easier than big Quality Assurance documents that never get updated, and are only taking up space in your shelf. The EPG also makes it easier to get an overview of the development process. Two said: It helps giving you a good project overview. Since all projects are performed the same way, it is easy for me to look into another project to find information I need. It is very useful to have an overview of the different documents that need maintenance in connection with freeze and delivery. Then the model justifies itself. Copyright  2006 John Wiley & Sons, Ltd.

3.5. Perceived Compatibility Perceived compatibility is defined by Rogers (Rogers 2003) as the degree to which an innovation is perceived as being consistent with the existing values, needs and past experience of potential adopters. 3.5.1. Compatibility with Projects of Different Size The majority of the programmers had the opinion that the EPG was not suited for smaller projects. This was, however, not mentioned by the project leaders. The programmers thought the use of the EPG would cost them too much effort, and that it would draw focus away from central activities in the project. Two said: In small projects, I’m squeezed on how many hours I can use. . . . . . It takes focus away from writing good code. We have the big documents including everything, but we also need the smaller documents telling us what is required as a minimum when you have a small project. It was the opinion among both programmers and developers that the EPG supported larger projects. One said: Softw. Process Improve. Pract., 2006; 11: 21–34

29

Research Section

It is built for bigger projects. With bigger projects I don’t need to have a lot of people working on it, but it needs to be a team working together for a specific period of time. 3.5.2. Compatibility with the Early Phases It was a common opinion that the EPG is compatible with the early phases (before programming starts). The most intensive use of the EPG was in this phase. Three said: You start with conceptual design and GUI prototyping, and then you go over to the implementing phase, where you only write code, then you don’t use it any more. Yes, it must be in the early phases where I use it most. I have used it for planning, requirements, vision and scope, conceptual design. I was leading another project before this one, and then I also used it most in the beginning. Now I’m in the early phases with conceptual design and planning, and I think I get very good support from the process model. 3.5.3. Compatibility with the Implementation and the Last Phases Neither developers nor project leaders thought that the EPG supports the implementation process. Three said: After we start programming, we don’t use the EPG at all. The developers need help documenting what we do. It’s not good at support system documentation. As I earlier told you, my opinion is that the implementing phase has very little support. I think the ambition for the process model is very good since the technology changes very quickly. It will not be easy to maintain templates and documents describing the technology, because it will be outdated very soon. When it came to the last phases, some meant that the EPG did not support them, while others meant it supported them well. Two said: When you get to the end of the project, it is a bit unclear, but this is also common knowledge. Copyright  2006 John Wiley & Sons, Ltd.

30

N. B. Moe and T. Dyb˚a

It is easier to complete the early phases, they are easier to define. When you get to the end of the project, it depends on the product you are making – sometimes you are totally dependent on other projects, and sometimes it is a standalone or hardware involved. There are a lot of different paths in the end of a project. It is OK to have an overview over the different documents you need to maintain when you change the product in the freeze and delivery phase. It is then this model justifies itself. 3.5.4. Compatibility with the Need for Documenting The EPG only supports the manual part of the documentation process. It was, therefore, a strong wish from several of the developers that the EPG should automate the documentation process. One said: I was a bit disappointed in the beginning. I didn’t see it as a tool, only as a pile of templates located in a fixed place. The glue is missing . . .. If you work with different models in a project you need to cut and paste to update them in the different word documents. Two persons also commented that: You need to write a lot of documents that may never be read. You need a living design document that describes what should be in the product. Of course we would have used it more actively, if it had given something back, if it had helped us besides being a guide that describes how we should work. 3.5.5. Possibility to Adapt the EPG to the Project The majority of the subjects meant that it was necessary to adjust the EPG to the activities they performed. Two said: You need to do some work with the templates. The development projects vary a lot here, if you are going to write a software component with an interface and a GUI, compared with when you are supposed to write something that includes both hardware and software. Most of the templates are made for software development. Softw. Process Improve. Pract., 2006; 11: 21–34

Research Section

If you are going to follow everything that is written there, sooner or later you will get problems. There must be an opening to adapt to the task you are performing. Therefore it is important that the EPG is more general. 3.6. Perceived Ease of Use Perceived ease of use refers to the degree to which a person believes that using a particular system would be free of effort. This follows from the definition of ease: freedom from difficulty or great effort (Davis 1989). Several of the developers stated that the EPG was difficult to learn. One said: The first time I used the EPG I used plenty of time to get into it and to understand the systematics. Two of the four project leaders commented that it was easy to use. One said: I think it is easy to use, you see a graphical representation of the whole process, you just go where you need to go and click, and then you get to the document you need. Several of the developers commented that is was difficult or cumbersome to use. Two said: If you are using a template, you need to read a lot before you can start writing. You open it and start filling out the fields on the first page. Author, title, and then you need to fill in a lot of field codes. You have to do a lot of manual operations. And then you think: Can’t they automate these operations if they want this kind of document. It is very irritating, can’t they fix this. It’s even more irritating when you are working inside the document, and you still have to do a lot of manual operations. It is too much paper and they are to general, and you use a lot of time to understand the meaning of this form sheet. 3.7. Subjective Norm Subjective norm is the degree to which software developers think that others who are important to them think they should use the EPG. This Copyright  2006 John Wiley & Sons, Ltd.

The Use of an Electronic Process Guide

suggests that perceived social pressure to perform will influence a person’s intentions (Ajzen 1991). One of the department leaders preferred not to use the EPG. One said: It is decided on the top level that we must have a process model, but my boss – who is the person who should follow up the use of the process model – does not have the same opinion. And since he thinks the guide is heavy and bureaucratic and not practical, we don’t use it much. Except for the three documents we need to produce. . . . Occasionally we have looked at the model in meetings, but we are not very focused on using the model. My boss does not think much of it. Some claimed that it was very important that someone really wants and needs the produced documentation. Two said: If they want people to use it someone has to pressure our leaders, to get them to ask for the documentation we make. As long as they don’t ask for the documents we are supposed to create, there will be a low use level. On external projects where I have been the only developer, and where the project leader does not ask for it, we don’t use it. 3.8. Why is the EPG not Used? We were a bit surprised by the low use level reported, especially by the developers. The main reason for not using it was that they claimed to know what they were going to do. Two persons said: What I mostly use it for is everything that’s got to do with maintenance, bug-fixing and change management. Then I don’t need to consult the model when I’m doing this, because I know what to do. Because what is written is how I do it. We always have the model in mind, all the persons on this project; therefore we don’t need to look into it. We have made a list of activities which is a part of the process model, and now everybody has it clear what we are fighting for. Now it is very clear what we are going to do. Softw. Process Improve. Pract., 2006; 11: 21–34

31

Research Section

A number of developers and project leaders also commented that several of the documents were outdated. Two commented: I don’t use the design document because it is outdated. The problem with the process model is that it contains a lot of documents, and not all documents are relevant for all types of projects. I believe there exists a template somewhere, but I think it is dead. It doesn’t live. It was approved before we started to code. Three persons commented that they used an alternative model, while another two commented that the EPG was meant for project leaders and not for developers. Time pressure and lack of compatibility with the project were also mentioned as reasons for not using the EPG.

4. DISCUSSION AND CONCLUSIONS A qualitative field study was performed to investigate: What characterizes the use of an EPG among developers and project leaders in a medium-sized software company? Each department leader pointed out two persons they thought preferred to use the EPG, and two persons they thought did not use it or were against using it. From the answers given in the interviews, it was not possible to determine if it really was one group against and one group in favor of using the EPG. Only half of the interviewed subjects worked on medium sized or large projects. This was a low number considering that the main activity in Company X is project work, and most projects are medium sized or large. The reason for the low number could be the selection strategy. Several of the interviewed persons had the opinion that the main target for the EPG was large projects. It is therefore possible that those working alone or in small projects will be against or negative to the EPG (this group also had the lowest use level), and this again may be the reason for the department leaders pointing them out. 4.1. Use Level We were a bit surprised by the low use level of the guide. It was a common opinion that the project leaders and managers are the user groups that Copyright  2006 John Wiley & Sons, Ltd.

32

N. B. Moe and T. Dyb˚a

benefit the most from using the guide. None of the developers in the no use group saw themselves as the primary target group for the guide, but several of the developers using the guide claimed that they were the target group for the EPG. We also found that the developers who also were project leaders used the EPG the most. This could mean that some of the potential users do not use the EPG because they do not think the guide is made for them, even though the EPG was initially made for everyone developing software. Another reason could be that the project leaders see the EPG as mandatory, and they need more support for planning and reporting in this role. The main reason for the low use level reported among the programmers was that they claimed to know how to perform their work. Several persons reporting little or no use at all said that they used several templates. They did not consider themselves as users since they did not access the EPG directly, even though they used the templates. Maybe they did not have a clear understanding of what was meant with the EPG? If a person is not actively using the guide, but working according to the process described in the guide, the guide is internalized. Such internalizing is the most important goal when introducing an EPG. The only problem that could arise, if a person seldom looks into the guide, is that he or she might miss updates and changes. It does not mean that the guide is useless, nor does it mean that it is not used if it is not explicitly used. A challenge here is, thus, how we conceptualize and operationalize usage. 4.2. Perceived Usefulness and Perceived Compatibility The EPG was perceived as both useful and not useful. The checklist and templates were perceived to be the most useful functionality of the EPG. All the project leaders described useful aspects of the EPG, but they did not mention any useless aspects. The developers mentioned almost the same number of both useful and useless aspects of the guide. From this it is possible to conclude that the developers who are also project leaders perceive the EPG as more useful than the developers who are not, and that this is the reason for a higher reported use level among the project leaders. Several of the developers perceived the EPG as less compatible with the way software development Softw. Process Improve. Pract., 2006; 11: 21–34

Research Section

was done since it was not integrated with other tools, e.g. support for documenting and updating documents. In addition, some meant that the EPG decreased the productivity since they had to write a lot of documents no one really asked for, or that even could be outdated. But it was also a common opinion that the EPG made the work easier and improved the quality since it supports a lot of useful checklists and document templates. We have done similar studies in two other companies, Firm (Moe et al. 2002) and Kongsberg Spacetec (Moe and Dingsøyr 2005), and in both studies we found the key success factors for the EPG to be integration with other tools, e.g. an automatic system for process tracking. A variety of different tools is described in Moe and Dingsøyr (2005). Several had the opinion that the EPG was not compatible with the implementation process and, therefore, that they did not need it for this phase of the project. One important observation was that no one seemed to miss support from the EPG in this phase, and therefore it is possible to conclude that the developers felt they had the support they needed. When analyzing the EPG’s compatibility with the different project types, it was found that several thought the guide was primarily for larger projects, and not for small projects or people not working on projects. The majority of the people with little or no use did not work on any project, or worked on really small projects. This could also be an explanation for why some of the developers did not use the guide. On the other hand, several stated that it was easy to adjust the templates to different types of projects. According to TAM, systems that are perceived to be easier to use and less complex have a higher likelihood of being accepted and used by potential users. In Dyb˚a et al. (2004), we found that perceived usefulness was the fundamental driver in explaining current system usage and future use intentions of an EPG, and furthermore, that perceived compatibility, perceived ease of use, and organizational support were the key determinants of perceived usefulness. 4.3. Involvement and Training One important finding that influences how the EPG is perceived and used was that one department leader did not fancy the guide, and that several felt that the leaders really do not read the documents or Copyright  2006 John Wiley & Sons, Ltd.

The Use of an Electronic Process Guide

ask for the documents produced. Some also stated that the EPG was difficult to learn and cumbersome to use. No one mentioned that they had been trained in using it, which came as a surprise since one of the main goals of the SPI group was to train and support the developers in using the process. Since we know that training and support is one of the key determinants of perceived usefulness, which again influences the system usage (Dyb˚a et al. 2004), absence of training and support could explain that some did not use the EPG or had a very low use level. In Moe et al. (2002), we found that it was important to involve the users strongly in developing the EPG to succeed with institutionalizing. This was not the case with the persons interviewed in Company X. Only two persons reported such an involvement even though the EPG was said to be developed in close cooperation with the intended users. The low degree of involvement among those interviewed could explain some of the low use level. In Dingsøyr and Moe (2004) and Dingsøyr et al. (2004), we proposed the use of process workshops as a tool for developing process guides for software companies. The process workshop makes it easy to involve people in developing an EPG. In Moe and Dingsøyr (2005), we found a higher use level among those participating in the process workshop. 4.4. Recommendations We found that implementing an EPG is not a straightforward process, but that it depends on several organizational factors. Our findings from the current study suggest the following recommendations: • Anchor the EPG among the leaders and involve the users in the EPG creation and updating process. • Use time and effort to advertize who is the target group and why it will be useful, and train them. • Make sure the EPG is always updated and provides the best working practices. • Focus on measures that could increase use among the developers (if they are part of the target groups), since it seems to be easier to get the project leaders to adopt the EPG. • Templates and checklists are seen as very useful. • Automate the documentation process, and integrate the EPG with tools to make it more useful. Softw. Process Improve. Pract., 2006; 11: 21–34

33

Research Section

• Implement functionality for easy tailoring of the process to different projects types or activities, if the whole development organization is going to use the EPG. Remember that internalizing and use of company knowledge and principles is the most important goal when introducing an EPG and not the use level of the EPG itself. 4.5. Future Research Further research is needed to conceptualize and operationalize EPG usage and to examine such usage in a wide variety of organizational settings. It will be important to study documentation from the projects where the interviewed subjects have worked to find out more about how much the EPG is used in the organization. It will be interesting to find the real use level of those that do not think they use the guide, but reported use of several templates. ACKNOWLEDGEMENTS This work was supported by the Research Council of Norway under Grant 156701/220. The authors wish to thank all the employees of Company X who took part in this investigation.

REFERENCES Ajzen I. 1991. The theory of planned behavior. Organizational Behavior and Human Decision Processes 50(2): 179–211. Davis FD. 1989. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly 13(3): 319–340. Dingsøyr T, Moe NB. 2004. The process workshop – a tool to define electronic process guides in small companies. Proceedings of the Australian Software Engineering Conference (ASWEC), Melbourne, FL, IEEE Press: 350–357. Dingsøyr T, Moe NB, Dyb˚a T, Conradi R. 2004. A workshop-oriented approach for defining electronic process guides – a case study. In Software Process Modelling, Acuna ˜ ST, Juristo N (eds). Kluwer Academic Publishers: Boston, MA, 187–205. Dyb˚a T, Moe NB, Mikkelsen EM. 2004. An empirical investigation on factors affecting software developer acceptance and utilization of electronic process guides. Copyright  2006 John Wiley & Sons, Ltd.

34

N. B. Moe and T. Dyb˚a

Proceedings of the International Software Metrics Symposium (METRICS), Chicago, IL, 220–231. Fichman RG, Kemerer CF. 1993. Adoption of software engineering process innovations – the case of object orientation. Sloan Management Review 34(2): 7–22. Kellner MI, Becker-Kornstaedt U, Riddle WE, Tomal JMV. 1998. Process guides: effective guidance for process participants. Proceedings of the Fifth International Conference on the Software Process: Computer Supported Organizational Work, Lisle, IL, 11–25. Krutchen P. 2000. The Rational Unified Process: An Introduction. Addison-Wesley: MA, USA. Moe NB, Dyb˚a T. 2004. The adoption of an electronic process guide in a company with voluntary use. Proceedings of the European Software Process Improvement Conference (EuroSPI), Trondheim, Norway, SpringerVerlag: 114–125. Moe NB, Dingsøyr T. 2005. The impact of process workshop involvement on the use of an electronic process guide: A case study. To Appear in EuroMicro, Porto, Portugal. Moe NB, Dingsøyr T, Dyb˚a T, Johansen T. 2002. Process guides as software process improvement in a small company. Proceedings of the European Software Process Improvement Conference (EuroSPI), Nurnberg, Germany, ¨ 177–188. Paulk MC, Weber CV, Curtis B, Chrissis MB. 1995. The Capability Maturity Model: Guidelines for Improving the Software Process. Addison-Wesley: Reading, MA. Rogers EM. 2003. Diffusion of Innovations. The Free Press: New York. Scott L, Carvalho L, Jeffery R, D’Ambra J, BeckerKornstaedt U. 2002. Understanding the use of an electronic process guide. Information and Software Technology 44(10): 601–616. Strauss A, Corbin J. 1998. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. Sage Publications: Thousand Oaks, CA, USA. Thompson RL, Higgins CA, Howell JM. 1991. Personal computing – toward a conceptual-model of utilization. MIS Quarterly 15(1): 125–143. Venkatesh V, Davis FD. 2000. A theoretical extension of the technology acceptance model: four longitudinal field studies. Management Science 46(2): 186–204. Yin RK. 1994. Case Study Research: Design and Methods. Sage Publications: Newbury Park, CA, USA.

Softw. Process Improve. Pract., 2006; 11: 21–34