Tool Support for Requirements Management Quality from a User Perspective Anders Larsson Sogeti Sverige AB
[email protected]
Odd Steen Department of Informatics, School of Economics and Management, Lund University
[email protected] Abstract. Requirements in software development are fundamental in order to document and describe what the system to be developed basically “is” and how it should behave. The requirements are elicited, anlyzed and prioritized in the requirement process to produce a software requirement specification (SRS). The SRS can contain many requirements and large volumes of information, which calls for good requirements management (RM) and support tools (RM tools) to keep the SRS adequate and up to date. However, neither the requirement process nor RM tools have been investigated empirically but in a few cases. This study focuses on the use of the DOORS RM tool and investigates how it is perceived as a support to achieve the IEEE criteria for a good SRS. Based on qualitative interviews with six RM tool users, the conclusion is drawn that the tool in several aspects does support the users to reach good SRS quality, but that some criteria of high importance to the users, that the requirements can be ranked, traced and verified, lacks good support in the tool. Even though the tool from the IEEE criteria perspective is a good support, it is still not very much appreciated by the users. The users find the tool inflexible and of low interoperability. To explain this a slightly speculative conclusion drawn, is that the tool builds on a wrong assumption of the requirements process as rationalistic problem solving rather than problem construction. Keywords. Requirements Engineering, Requirements Management, Requirements Quality Criteria, Requirements Management Tools.
Introduction Requirements in software development are fundamental in order to document and describe what the systems to be developed basically “is” and how it should behave. According to the IEEE standard 610.121990 (IEEE, 1990) a requirement is “(1) A condition or capability needed by a user to solve a problem or achieve an objective. (2) A condition or capability that must be met or processed by a system or system component to satisfy a contract, standard, 1
specification, or other formally imposed documents. (3) A documented representation of a condition or capability as in (1) or (2).” (p. 62) Many of the problems in software development can be attributed to vague and incomplete requirements specifications according to Sommerville (2001), which is confirmed by a Standish Group survey (Standish, 1995 referenced in Pfleeger (2001)). The top one factor in the survey explaining software development failures was incomplete requirements. Increase in complexity of software has also increased the body of requirements of a system. A well functioning requirements management process for gathering, prioritizing and analyzing, specifying and documenting, and validating requirements (Pfleeger, 2001) so they become identifiable and usable, is today a prerequisite for effective software development in many organizations. There is a pressure for shorter development cycles, different configurations of software, and faster changes of requirements, which call for better management of the requirements. In a recent study comprising four thousand European companies requirement management was considered as a current and very problematic issue in software development (Loconsole, 2004). Good requirements management can lead to reduced software development costs and higher customer satisfaction since the delivered product closer matches the agreed requirements (Kotonya and Sommerville, 1998). Also, functioning management makes reuse of requirements and application of these to other products or variations of the same product, for instance the same graphical user interface on different models, possible (ibid.) Requirements management involves managing large quantities of requirements and information connected to these (Kotonya and Sommerville, 1998). The requirements are documented in a software requirements specification (SRS) and IEEE (IEEE, 1998) has developed a standard with eight criteria an SRS should fulfill to be good and manageable (discussed further below). In addition, specialized Requirements Management tools such as DOORS, CaliberRM, and RequisitePro to support the requirements management process have seen the market (Larsson, 2006). The tools have been critizised for not fully supporting the requirements management process (Grehag, 2001), for difficulties to configure them to an organization’s existing work processes (Hamann and Oort, 2001) and lack of connection between communication and cooperation (Lang and Duggan, 2001). Nguyen and Swatman (2000) instead consider the problem of the tools to be that they are locked to a fundamentally wrong idea of requirements management as a smooth and linear process, when it is better understood as a design process of discovery and problem construction. There are few other studies of requirements management (Brinkkemper, 2004; Grehag, 2001), or studies of the use of Requirements Management tools (RM tools) and how they influence the quality of requirements (Larsson, 2006). Thus, the research questions of this study are: How do users of a commersial RM tool consider its influence on their possibility to achieve IEEE’s criteria for requirements? How do the same users assess the RM tool as a tool in the company? The purposes are to describe how users assess the RM tool in connection to requirements quality and how the tool support users and requirements processes in a company. The remainder of the paper is structured as follows. In the next section the requirements engineeering process is described and why it is necessary is discussed. It also presents the IEEE criteria for requirements specification quality which is the base for the investigation of the RM tool support and discuss the need for requirements management. In the third section
2
Requirements Management tools are discussed and the investigated RM tool is described shortly. In the fourth section the design of the investigation of RM tool use is described and in section five the empirical results from the investigation is presented. These results are discussed in section six and the paper is concluded in section seven.
The Requirement Process As stated in the introduction, requirements are vital in order to understand what kind of system to build and what purposes it will fulfill. Thus, requirements must be determined through a requirement process that focuses on the customer and problem before the software is developed. In this process the customer’s and users’ wants are elicited, analyzed, and prioritized at least according to three categories: essential requirements, desirable but not necessary requirements, and possible requirements that could be eliminated (Pfleeger, 2001). Though there is no ideal requirement process, depending on the different structure of organizations (Sommerville, 2001), on a general level the process can be depicted as in Figure 1 (Kotonya and Sommerville, 1998). The arrows in the figure represent the flow of information between the different phases of the process with possible iteration from validation to earlier phases when new requirements are fed into the process.
Requirements elicitation
User needs domain information, existing system information, regulations standards, etc.
Requirements analysis and negotiation
Requirements documentation
Requirements validation
Requirements document
Agreed requirements
System specification
Figure 1. The requirements engineering process (Kotonya and Sommerville, 1998, p. 32).
The first phase is the requirements elicitation where the requirements are gathered from the requirement stakeholders. The systems developer or requirement engineer works together with the customer to understand the current problem. To be able to do this the systems developer need knowledge about and understanding of the customer’s business processes and the would-be users’ work processes that are to be supported by the new system. It is very important that this part of the process works well otherwise the system might not be what the customer expects (ibid.) It is also during this phase that the possibility to influence whether the development project will be successful or not is the greatest (Carr, 2000). When the initial requirements have been elicited and gathered the process moves on to the second phase and the requirements are analyzed and prioritized by the requirement engineer and customer together. This will likely lead to necessary compromises since not all requirements are essential, desirable or possible to fulfill. This phase can be resource consuming in both time and key competencies and in large organizations, it can be difficult to find time to resolve all requirement conflicts (Kotonya and Sommerville, 1998).
3
The aim of the analysis and prioritization phase is to reach an agreement about which requirements that should be incorporated into the requirement body and lay a ground for design and construction phases of the development project. If this phase is handled properly and time and resources are devoted to it, it will pay back in subsequent development work, since the later that changes are made the costlier they are to incorporate into the system (Carr, 2000). In the third phase, the requirements are documented and the software requirements specification (SRS) is produced. An SRS make the requirements easier to survey and compare which place demands on the quality of the documentation, which is why IEEE has developed a standard for software requirements.
Software Requirement Specification Criteria According to the IEEE standard for software requirements (1998) a good SRS should be (p. 4): • • • • • • • •
Correct; Unambiguous; Complete; Consistent; Ranked for importance and/or stability; Verifiable; Modifiable; Traceable.
That an SRS is correct means that there are only requirements stated in it that should be met by the software and no other requirements. This characteristic can be difficult to fulfil in larger development projects (Kotonya and Sommerville, 1998). If every requirement has only one interpretation, the SRS can be considered as unambiguous. This means that every person reading the SRS would reach the exactly same interpretation, which is unlikely since different persons have different education and experience and, thus, the meaning of a requirement can be interpreted differently (IEEE, 1998). The SRS can be considered as complete if all significant requirements are included, all responses to input data in all situations are defined, all terms are defined and full references are given to figures, diagrams etc. The SRS must be internally consistent with other documentation for the software and there should be no definition conflicts between the requirements in the SRS. If every requirement’s importance or stability is indicated, the SRS is ranked for importance and/or stability, where importance expresses the necessity of the requirement (essential, conditional, optional) and stability refers to the likelihood and number of changes of the requirements that can be anticipated. Each requirement should be possible to verify by human or machine check of the software’s conformance to the requirements for the SRS to be verifiable. Ambiguous requirements are in general not verifiable (ibid.) That the SRS is modifiable links several of the other characteristics since it should be easy to update the requirements in the document while preserving structure and style. Redundancy, 4
however needed sometimes, may lead to inconsistencies if a requirement is altered in only one of the places it appears (ibid.) Finally, an SRS is traceable if each requirement’s origin is clear and it is possible to trace it backward to documents from previous stages of development or forward to subsequent documents in the requirement process. The SRS can contain a large number of requirements of different sorts and a management process is very important in order keep the SRS e.g. consistent and up to date. Hence, requirement management is a vital part of the requirements process.
Requirement Management The set of requirements in an SRS is a total of business, product, and detailed requirements (Sommerville, 2001) (see Figure 2) which should be linked from the top to the bottom, so that what is finally built is what the business analysts consider suits the market demand. The requirements are broken down and refined in the requirement process and in large corporations, different departments may be responsible for achieving the requirements, which can make it difficult to create connections between the requirements (Grynberg and Goldin, 2003). Organisations’ Requirements set Market Business requirements
Product requirements Development
Release plan
Product
System requirements
Figure 2. Requirements levels in an organization (Brinkkemper, 2004; Grynberg and Goldin, 2003).
However, it is important to have a good requirements management in place so that different stakeholders have a common point of reference in discussions and the marketing department has an easier task of producing product descriptions (Grynberg and Goldin, 2003). The body of requirements of a software system undergoes changes during the life cycle of the system because of e.g. changed work processes or new components that will be incorporated into the product. These kinds of change to the requirements is handled through requirement management which is “The systematic process of organising and storing relevant information about requirements, while ensuring requirements traceability, and managing changes to these requirements during the whole lifecycle of the information system.” (Grehag, 2001, p. 2, italics in original.) Two of the main areas of requirements management are thus change management and traceability of the requirements. Change Management A requirement may change because the requirement was entered wrongly into the SRS or the customer changes the prioritization due to technological or organizational factors, or cost of a specific requirement (Kotonya and Sommerville, 1998). Before a change to a requirement is accepted, it is analyzed in terms of influence on other requirements or system components, the cost in time and resources of making the change, and the different stakeholders’ acceptance and understanding of the change (Kotonya and 5
Sommerville, 1998). If a change is accepted, it is incorporated into the requirements body and the change request is documented. Traceability The requirements of a software system are dependent on each other through relationships between the requirements. It is thus important that it is possible to perceive how different requirements relate; otherwise, it is difficult to make sound decisions about whether a change can be accepted or not (Grehag, 2001). In addition, traceability between requirements makes it possible to trace the effects from a change to a business requirement to the detailed requirements (Hayes, 2003). The problem of traceability is the volume of work and cost of registering changes to requirements and keeping the requirement documentation up to date. To manage all types of traceability information is too expensive and policies stating what traceability information it is essential to maintain must be defined (Kotonya and Sommerville, 1998).
Requirement Management Tools Because requirement management and requirements engineering can be very time consuming and involve large quantities of information, an RM tool is often a necessary support in the requirements engineering process to maintain e.g. requirements’ traceability, relationships and priorities (Larsson, 2006). Especially in large organizations it would be impossible to handle the requirements set without an RM tool (Kotonya and Sommerville, 1998). These software tools are developed specifically for handling large quantities of requirements. At the time when this paper was written, there were several RM tools available on the market. Some of the larger commercially available products come from i.e. IBM (IBM, 2005), Telelogic (Telelogic AB, 2005) or Borland (Borland Software Corporation, 2005). Several advantages can be identified when an organization is using an RM tool for their requirement management process. One example is increased help for resource planning, i.e. if a development group finish ahead of schedule, more requirements can be implemented. Another example is the possibility for prioritising and choosing the requirement for the optimal customer satisfaction or from a technical perspective. If the RM tool is used correctly, less time can be spent on synchronisation meetings, and, in addition, the possibility arises for the organization to have an increased control over what requirements are really implemented in the product (Grynberg and Goldin, 2003).
Generic requirements for an RM tool Hoffmann has written one of the more extensive papers on requirements for RM tools, where the requirements were labelled based on their respective grade of importance (Hoffmann, et al., 2004). Also, Lang and Duggan (2001) and Hamann and Oort (2001) have contributed to generic requirements for RM tools. Since RM tools are developed for a general market with many potential different customers, they must be designed and equipped with a set of standard functions. Kotonya and Sommerville (Kotonya and Sommerville, 1998, p. 32) have described some main areas that are vital for an RM tool:
6
• • • • • • •
Database Search facility Organizing support Requirement change management Explorer or user interface Interoperability Report generator
Some sort of storage for the requirement is needed and the predominant way up to now is in relational databases. Due to many relationships between requirements, the architecture of the relational database can cause problems such as slow access to requirement information if the organization has many requirements. A user must also be able to search and organize the requirements. The RM tool must be able to maintain traceability and the changes that occur in the requirements (Kotonya and Sommerville, 1998). Several articles emphasize traceability as an important factor or requirement for RM tools (Hamann and Oort, 2001; Lang and Duggan, 2001). Traceability can be solved with links, which is not a popular solution among developers since it is time consuming. Change management is also important and must function properly (Hoffmann, et al., 2004). If all the requirements are stored in a repository, a user must have an interface to access the information. This is where an explorer comes in handy and it involves almost all the functions presented in the above bullet list, since this is how a user interacts with the system. An important aspect of the user interface is the easiness for non-technical users to use the RM tool functions (Lang and Duggan, 2001). Furthermore, specifications of requirements should be possible to create in many different ways, i.e. both with text and models. A user could be accustomed to work in Word or Excel, where there are many possibilities for adding multimedia (ibid.) RM tools, on the other hand, present a different work environment. Users must be prepared to change their working habits to use an RM tool properly (Grynberg and Goldin, 2003) but it would facilitate them if they could configure an RM tool by themselves, by creating personalised user views or make graphical changes to the requirements (Hoffmann, et al., 2004). The RM tool should also have good learnability so it is easy for new users to get started and use the tool (Hamann and Oort, 2001). Another essential aspect is interoperability. Since an RM tool needs to operate together with other systems in the organization, it must be possible to connect the RM tool with e.g. test or document tools. The last, but not least, function in the bullet list above is a report generator. A user should be able to design freely how a generated report is presented. In addition, a report should be completely autonomous from the RM tool. (Hoffmann, et al., 2004) To be able to design freely what information a report should contain is similar to the user view requirement above. Of course, other important RM tool aspects are not related directly to functions, e.g. tool support for the organization’s business process or politics. If an RM tool should provide an optimal support to the organization, it requires mature requirements engineering and requirement management processes. It is not possible to use the RM tool’s full potential if the users’ and organization’s processes themselves are not ready – to quote Grynberg and Goldin (2003, p. 6):
7
Then, the tool merely reflects the reality, which is (in some areas) not so glamorous. Reality, that is, unclear and incomplete requirements was shown, additionally and even more importantly, it reflected the decision making mechanism and culture requirements definition within the company. Some people choose to slip into denial.
Without commitment to requirement initiatives, it will of course be even more difficult (Grynberg and Goldin, 2003). Involving users on different levels in the organization is one way to tackle this problem (Zylbermann, et al., 2003).
RM tools currently on the market During the last years, the market for RM tools has increased. A comparison between RM tools would be hard, since there are nowadays a considerable number of available tools (Lang and Duggan, 2001). However, there are a few leading actors on the market, such as IBM and Telelogic. Of the generic requirement on RM tools that we mentioned earlier in this section, all the leading tool vendors can provide some functionality. Nevertheless, limitations that appear are connected to usability, design or support for group collaboration (ibid.) The investigated organization in this study used the RM tool DOORS, which is manufactured by Telelogic. The DOORS product family consists of four products (Larsson, 2006): • • • •
DOORS DOORS XT DOORS/Analyst DOORSnet
The basic product DOORS is an RM tool for a delimited geographical area, i.e. an organization with one physical location. Many of the generic requirements stated above, such as traceability, change proposal or requirement storage, are in some way met as functions in the tool. The user interface is a configurable view with requirements text, graphics and traceability. DOORS XT is designed for supporting requirements engineering on several development sites on a more global scale. The last two tools could be considered as add-ons. With the Analyst tool graphical or UML requirements modeling is possible. DOORSnet is a web interface for remote access to the requirement repository (Telelogic, 2005). DOORS is the RM tool investigated as described next.
The investigation of RM tool usage This paper is based upon a case study that was mainly conducted during the summer 2005 at a major telecommunication company (Larsson, 2006).
Interviewee background As mentioned before the overall goal of the study was to describe users’ assessment, in connection to their ability to maintain good requirements quality, of RM tools. To investigate this empirically, six RM tool users were interviewed. The interviews were semi-structured
8
with a couple of predefined questions (Yin, 2003). This meant that the interviewees had an opportunity to answer freely the different questions at hand. An RM tool can affect many people in an organization, why it was important that the interviewees also had different user roles. This contributed to a solid overall picture (see Figure 3). Requirements coordinator
Requirements management manager
Requirements engineer
Tool administrator
Project manager
Requirements administrator
RM-Tool
Figure 3. Six tool users with different user roles.
The project manager In the work with requirement, the project manager uses many different tools i.e. web-based forums, Word, Excel and tools developed in-house. The RM tool is therefore only one of many components, but it is used for the direct requirement specification regarding a product. The project manager’s work also include deciding whether a requirement should be changed or not. The requirements engineer The requirements engineer is responsible for the requirements in a specific group segment. The work involves breaking down requirements and storing them in the RM tool. The requirements engineer handles almost anything regarding requirements, such as requirement changes and code reviews, which mean that he or she interacts with the RM tool. The requirements management manager Interview number three was with the requirement management manager. His or her objective is to collect requirements from all the customers and thereafter divide and distribute them internally within the organization. The work also involve handling requirements questions from i.e. product planners, which means working with the RM tool or Excel. The requirements coordinator Working as a requirements coordinator involves a lot of decision handling in many different product projects. The goal is to strive for a similar user perspective in all the products. The occupational role results in RM tool interaction several times per week. The requirements administrator As an operator requirements administrator it is important to be able to discern technical terms and concepts. This is especially essential when requirements are grouped together. The role includes handling a large quantity of requirements in the RM tool database.
9
The tool administrator The RM tool administrator has spent a lot of time during the last four years configuring or adapting the tool to better suit the organization. More and more the work has shifted towards investigating how the organization works with the requirements and the RM tool.
The process of analysing the data The interviews were recorded for an accurate reproduction for the upcoming analysis. All the recorded material was transcribed and the transcripts were validated by the respective interviewee by respondent validation (Kvale, 1997), which is considered important for strengthening the quality of the collected data. After the interviewees had commented the transcripts, the collected interview material was first analysed on individual basis, as shown in Figure 4 below. To bring all the data together a variant of cross-case synthesis was used (Yin, 2003). The concluding result was partly made up of joint bullet lists and diagrams. Total result
Comparison
Individually collected material
Figure 4. The total result as a compilation of each analyzed interview.
Empirical findings regarding the RM tool It is not possible to present all the findings (cf. Larsson (2006) for a full account) therefore we choose to structure the results in the following sections: • • •
General reflections of requirements engineering, management and tools Users’ reflection regarding the RM tool and its effect on the organization The RM tool’s support concerning requirements criteria
Requirements management, engineering and tools During the last couple of years, the investigated organization has noticed a large increase in customer requirements. Since the organization’s requirements body today is very large, the interviewees find it impossible to work without requirements engineering, requirement management and a supporting tool. The developed products are technically advanced and therefore have numerous requirements, as many of the interviewees pointed out. There is also a positive attitude towards the use of the RM tool, as seen in these quotes: “it creates clarity when all the people are working with the same requirements set. Thereby there is no doubt which requirement is the correct one”, “by the use of RM tool it is most definitely a leap forward in the development.” Then again, as one interviewee pointed out, the RM tool is just
10
a sort of requirements repository. A working set of requirements and management processes are necessary to back it up. Requirements engineering and requirements management are seen as a foundation when working in a project. These make good planning possible, but also create a collective view of what is to be developed. The main thread from requirements to test can also be upheld with the help of requirements engineering, according to the interviewees. It is, on the other hand, important to see requirement engineering as an integrated part of the development cycle. The process must be monitored, so that the requirement work continues to be productive – to quote one of the interviewees: “there must be a watchful eye that watch what is produced and what the return of investment to the company is.”
Users reflections regarding the RM tool The opinion among the interviewees is that the RM tool is a good concept from a general perspective. What is their view if we specifically look at it from an organizational perspective? The organization’s requirements process is designed with the RM tool taken into consideration. Therefore, the RM tool will definitely support the organization’s requirements process. Some interviewees stated that the tool needed better marketing to achieve its full potential, or to quote: “some employees are or have been allergic. [It] has had a bad reputation, which results in lack of power.” A drawback of the RM tool is its lack of interoperability with other systems. This can be noticed in change management, which is supported by another tool. The lack of interoperability between the two systems results in manual work, which in turn leads to lost information. The tool provides many opportunities but the organization finds it difficult to take full advantage of all the functions. To the interviewees it is of great importance to study the overall picture when thinking of requirements management and engineering. When thinking of the overall picture one of the respondents pointed out that it is vital that the employees share the work approach towards requirements and the RM tool. This requires both good information and communication. The RM tool users need clear and uncomplicated instructions for how to use the tool. The organization has many active RM tool users today, which calls for continuous education. This is a struggle and the organization is at present not able to live up to the continuous education demand. “Requirements engineering is a great challenge when several people are involved” one of the interviewees said. The RM tool is nevertheless helping the organization to maintain the customer satisfaction despite increase in incoming customer requirements. As one of the interviewees said, “if we had worked in the old way, our customers would have been furious that we didn’t take care of their requirements. It has gone hand in hand. Our customers have more requirements, but we are also working harder to collect and maintain their requirements. The bottom line is: our customers are as satisfied now as they were before.” The user perspective regarding the RM tool involves interviewees’ general opinions of e.g. usability and the graphical user interface. The main approach of the study was not to investigate the user interaction with the RM tool. During the interviews this aspect were brought up by all the respondents, which is why we will shortly present their opinion on the matter. The RM tool’s demand on employees’ work time is not considered excessive, as one interviewee expressed: “requirement management and requirement engineering takes time
11
regardless of tool usage.” If we look at the RM tool’s graphical user interface, there were many comments from the respondents. One of the user comments: this is the big drawback of the tool we’re using. It is probably really good when you have learnt to think in the right way, but there are some fundamental differences when you are used to work with Word or Excel.
Several of the interviewees made comparison between e.g. Excel and the RM tool. The key factor of comparison is that the RM tool is considered ineffective. Users also experienced the RM tool as controlling when working with it.
The RM tool’s support concerning requirements criteria During the interviews several points of view were identified other than the specified requirements criteria. A theme that many respondents brought forth was the importance of clarity and education. Other areas were understanding, communication and requirements reviews, or to quote the requirements engineer: “a positive communication and discussion between developers and testers is important to achieve good quality requirements. To achieve a shared understanding the developer and tester must have thought through the requirements.” There was a general accord that education in requirements management, engineering and the RM tool are important. People that are working with requirements generally need a better feel for language, according to one interviewee. A continuous and effective requirements change management is an essential factor, in accordance with the interviewees. It would be especially advantageous between organizations, one interviewee added. If you look at the change management inside the organization, it has to be followed up regularly, as one interviewee said, “it is not enough to make a decision; you must also make sure that the decisions are updated in the RM tool.” Moving over to quality criteria for requirements, all the interviewees found it difficult to answer whether or not there have been any quality improvement, since the RM tool was introduced. The requirements criteria, mentioned in the introduction, are divided into two columns in the table below. Respondents where asked about their opinion of the different criteria, both regarding importance and user experience. In the following sections of text, we will shortly describe their opinions. Table 1. User experience ranked after importance for requirement quality. IEEE criteria Ranked Traceable Verifiable Complete Correct Modifiable Consistent Unambiguous
Importance 23 22 22 20 19 18 15 15
User experience 10 15 11 16 18 15 16 13
Deviation 13 7 11 4 1 3 -1 2
In Table 1 the interviewees’ evaluations have been summarized. Every response per criteria was graded between one and four. Grade 4 is equal to very important for good quality requirements and grade 1 to unimportant. An equal scale was used for the RM tool user
12
experience regarding the same requirement criteria. In total, there were six respondents, which results in a maximum value of 24. Correct Five of the respondents thought correctness was important. As the requirement management manager said: “we would get an unbelievable amount of questions from our customers if it was not correct.” The majority of the interviewees thought that the RM tool delivered good support. It was pointed out that a tool never could control if a requirement is correct. Unambiguous This criterion was considered as the least important as seen in the table above. Nevertheless, all of the criteria were in fact considered important when they were summarized. To make it easier to interpret the requirements and make them less ambiguous it would help if e.g. flowcharts or pictures could be attached to a requirement. Voices were raised that the RM tool is not flexible enough to support attachment e.g. pictures or documents, but according to the tool administrator it is capable to handle this. Complete All the interviewees thought that this criterion was important or very important. It helps in the end if the specifications are complete, the requirements administrator said. Completeness is also important for overview, according to the requirements coordinator. The requirements engineer considers it possible to create complete specifications. Nonetheless, most people would rather work in an Excel sheet, according to the requirements engineer. In Excel, it is possible to type in things that is impossible in the RM tool. It is also hard to see whether the specification is complete, since the layout is hard-to-grasp, the project manager said. Consistent As for the consistent criterion, the responses are mixed between important and less important. On the one hand, this criterion is less important because the requirement set is tested later on anyway. On the other hand, the criterion was considered important because it makes it easier to reuse the requirement set for a new product. To keep the requirement set consistent you need to be active and communicate with other groups and check if there have been any changes, according to the requirements engineer. Ranked This is the most important criterion according to the respondents. The possibilities to make e.g. comparisons, sort by difficulty grade or customer benefit, were some of the reasons. To quote the tool administrator: “a ranked requirement set facilitates the process when the functionality must be limited due to lack of time, or when the opportunity arises to add extra functionality.” Five of six respondents thought that the RM tool could not live up to their expectations. Points of view were that the ranking is a too coarse and in-flexible selection, which might lead to suboptimal ranking as can be read in the following quote: “in some situations the projects themselves rank the requirements. Maybe operator requirements are prioritized higher than market requirements, which could mean that we do not have as attractive offers that we could have had.” The requirements engineer considers the function messy and rather exports the requirements to Excel for easier handling. 13
Verifiable The criterion is considered as important for good quality requirements. One of the reasons is that it secures the quality of the requirements specifications. A problem is the lack of traceability, which makes it harder to verify the requirements. All of the respondents think that the support from the RM tool is not good enough. Modifiable The requirements process is dynamic; therefore, it is also important that the requirement set is modifiable. A problem with modifying a requirement is that other departments are affected by a change, according to the project manager. A majority of the interviewees considered this criterion important. To modify a requirement is considered rather easy. Nevertheless, a requirement can be changed without all involved people knowing about it. According to the tool administrator most problems have their origin in the organization’s accepted information model. Redundancy is created between requirements and therefore it is hard to modify them. Traceable This criterion was considered important or very important by all the respondents. To be able to see the relationship between requirements and e.g. subordinated requirements is one factor that was mentioned. Another essential factor was to identify the source of a requirement. The current traceability is not acceptable, according to the project manager. It should be easier to trace requirements from higher levels down to lower levels and back up again. If the built-in traceability function is used it creates problems with the requirement management.
Discussion In the following section we will discuss the presented empirical result in the light of the presented theory. The research questions were: How do users of a commersial RM tool consider its influence on their possibility to achieve IEEE’s criteria for requirements? And: How do the same users assess the RM tool as a tool in the company?
Requirements management in general The result show that without some sort of requirements engineering and requirement management it would be impossible to maintain and handle the present requirements situation, which is in line with current theories. A non-functional requirement engineering and management process leads to an increase in project costs (Kotonya and Sommerville, 1998). The respondents’ thoughts about requirement engineering and management also agree relatively well with the presented requirements theory. According to the interviewees, the requirements engineering should be an integrated part of the rest of the development process. This is in line with the future way of software development (Sommerville, 2005).
RM tool and effect on users and organization According to the tool administrator there is a lot of potential in the RM tool. However, the results show that the organization has not been able to fully realize it. The requirements
14
processes must be adequately mature to take full advantage of an RM tool. Maybe the investigated organization is not mature enough yet. Then again, if its users do not appreciate the RM tool support or there is a lack of commitment, it is much harder to succeed with requirements management (Grynberg and Goldin, 2003). This is also partly the case in the study since the RM tool has gained a bad reputation in the organization. In contrast, the RM tool fully supports the organization’s requirements process. To a certain level this depends on the fact that the processes are designed with the RM tool taken into consideration. A majority of the respondents are positive to the idea with a consolidated requirements set. Several interviewees point out that the organization could work without some sort of RM tool support. In addition, interoperability (Hoffmann, et al., 2004), apart from traceability (Lang and Duggan, 2001) and requirement change management (Hoffmann, et al., 2004), was considered an important quality factor. If we compare this with the results, the RM tool is not fully adequate since it lacks connection to change management and validation tools. The results has also shown that the RM tool is experienced as slow, which has a repellent effect on tool usage. One reason for this could be an inadequate database (Kotonya and Sommerville, 1998). The main problem of the study was functionality directly connected to requirement engineering and management. Nevertheless, several interviewees described the graphical user interface as insufficient. For that reason, we here choose to describe shortly their thoughts about this. A user is almost certainly used to work with Microsoft Word and Excel. These programs enable a user to work with different fonts, font sizes, colours or a variety of multimedia (Hoffmann, et al., 2004). When changing to an RM tool a user must be prepared to change their way of thinking (ibid.) The learning threshold for the RM tool is considered far too high, which also makes it harder for users to accept a new way of thinking. Of course, it is recommended that should it be fairly easy to learn to use an RM tool (Hamann and Oort, 2001). Easy report generation (Kotonya and Sommerville, 1998), in combination with userdesigned reports (Hoffmann, et al., 2004), in an RM tool is considered important. This is something that the respondents agree with. Nevertheless, as one respondent said, “the tool lacks the possibility for easily influencing the report outcome”, which is inconsistent with the current understanding.
Requirements criteria In a previous section, we described that maybe requirements need to be updated before they can be incorporated into the requirements set. This is necessary because the RM tool may place higher demands on the specification of a requirement (Grynberg and Goldin, 2003). Indirectly, this could be interpreted as the RM tool contributing to better requirements quality, since a user is forced to consider how a new requirement should be incorporated into the present requirements set. However, the study and the results do not indicate any direct improvement from the tool on the requirements quality. During the interviews, the respondents were asked to assess the importance of eight quality criteria for requirements. In addition, they were asked to grade the RM tools support for the
15
eight criteria. On the one hand, if the assessment of the quality criteria is summarized (Figure 5), the RM tool is clearly able to support the users rather well. 192 Very good
154
Very important
144 Good
114
Important
96 Less important
Moderate 48
Unimportant
Bad 0
Assessment
Importance
Figure 5. The RM tool’s capability and importance for quality requirements.
On the other hand, another view emerges when the criteria are considered in isolation. As seen in table 1 some criteria were graded as more important than others were. The top three criteria of importance were “ranked”, “verifiable” and “traceable.” The assessment of the most important criteria also deviated from the RM tool’s support the most. An effective analysis and ranking could mean that the organization gains an increase in ability to deliver what the market demands (Damian, et al., 2004). The result clearly show that the RM tool lacks this specific capability. Regarding traceability, it too was regarded as essential. Through traceability the organization have control and can immediately see what effect a proposed change would have (Hayes, 2003). We described earlier that traceability implemented with links are not popular among RM tool users, which is also confirmed in this study.
The underlying model of the RM tool This study and investigation was focused on the RM tool and users’ perception of the support for achieving the IEEE criteria. Thus, the study has not specifically addressed the requirements engineering process implicit in the tool. Though the tool in many cases does supports the process, and is perceived as a good support, the users did not quite like it and instead used, for instance, Excel when feasible. Maybe the explanation is not to be found in the tool in it self but rather in the assumption about the engineering process laid down in the tool. Hence, we choose to propose a slightly speculative explanation based on the view on the requirements engineering process. The requirements engineering process as depicted in Kotonya and Sommerville (1998) and Pfleeger (2001) is a linear and rational process with no interrupts and limited need for backtracking and iteration – in the words of Nguyen and Swatman (2000) major textbooks describe “[…] the RE process as smoothly evolutionary in a cumulative manner […]” (p. 2.) In that sense, the process is typical of the Computer Science and Software Engineering view of problem solving and software development with its underlying rationalistic epistemology.
16
With this perspective methods “[…] are seen as rule systems for finding a solution” (Floyd, 1992, p. 21) and “[…] tend to assume a machine-like behaviour […]” (ibid., p. 18). Part of the criticism that Floyd (ibid.) voices against the rationalistic foundation of Computer Science is an over-emphasis on formalization at the expense of communication and learning. This view is also that of Nguyen and Swatman (2000) in their action research study of an RE project in Australia. They found that the RE process was not smooth and incremental evolution but involved “crisis” where the “[…] requirements model was reconceptualised, restructured and simplified” (p. 2). Rather than analyzing a “world out there” of gathered requirements, the RE process involved the construction and reshaping of the problem. The RM tool DOORS builds on the standard assumption of the requirements engineering process criticized by Nguyen and Swatman, which can explain some of the critique against it from the interviewees. As the tool and the process was implemented concurrently in the organization, the tool’s incorporated philosophy of the requirements engineering process implicitly controls the requirements process and partly prescribes how the requirements are supposed to be managed. An example of this control is given by one the interviewees, when she explains that if some one creates a link to one of “her” requirement she cannot delete the requirement because of the link – the link has to be removed first, which requires a call to the person who created the link. As has been discussed above consistency is important for an SRS but in this case the modifiability is hampered and it is easier to export the requirements specifications to Word or Excel and use the RM tool as a repository instead of as a tool. Based on this perspective of the tool and the process it supports and implements, it might be misguiding to refer to it as a “tool” when it could equally well be considered as an actor or controller. Thus, the conclusions of the investigation in this study is in line with the Nguyen and Swatman’s (2000) conclusions that the smooth and rationalistic process, which is also implicit in the tool, is not a valid conceptualisation of the requirements process and the tool is therefore not flexible and non-constraining enough to support the requirements process as a design process of discovery and problem construction.
Conclusions An RM tool can in it self of course not lead to good quality requirements, but merely provide a repository for the RM tool users. The RM tool’s partly unsatisfactory functionality results in the users finding other ways to reach their goals by e.g. exporting data to Excel. This might also be explained by a wrong assumption about the requirements process. The interviewees regard requirements change management as important for good quality requirements. Change management is affected by incoming requirements from customers. It can be a time consuming task to sort out the changed requirements from all the incoming requirements, hence, a better standard for requirements exchange between stakeholders’ organizations could result in time savings. Education in specifying requirements, the organization’s requirements process and the support tool is a necessity for good quality requirements. However, due to a poor user interface, the usability and learnability of the current RM tool is not good enough. The bottom line is that the RM tool as a repository is perceived very positively by the respondents. A consolidated requirements set have shown many benefits such as easier
17
requirements reuse, communication and stability. All this in combination with proper education and involved users could create a good foundation for quality requirements. In the future, it would be interesting to study if the lack of acceptance of the RM tool can be linked to the lack of user involvement in the adoption of the tool in the organization. Another interesting future study would be to investigate how prioritization of requirements could be performed better. A third future interest could be to investigate how the support for traceability could be enhanced in RM tools.
References Borland Software Corporation (2005). CaliberRM, http://www.borland.com/us/products/caliber/index.htm, Accessed 20051119. S. Brinkkemper (2004). "Requirements Engineering Research the Industry Is (and Is Not) Waiting For. Invited Anniversary Keynote", in B. Regnell, E. Kamsties and V. Gervasi, (eds), Foundation of Software Quality, pp. 251-264, Essener Informatik Berichte, J. J. Carr (2000). "Requirements engineering and management: the key to designing quality complex systems", The TQM Magazine, vol. 12, no. 6, 400-407. D. Damian, D. Zowghi, L. Vaidyanathasamy and Y. Pal (2004). "An Industrial Case Study of Immediate Benefits of Requirements Engineering Process Improvement at the Australian Center for Unisys Software", Empirical Software Engineering, vol. 9, no. 1-2, 45-75. C. Floyd (1992). "Human Questions in Computer Science", in C. Floyd, H. Züllighoven, R. Budde and R. KeilSlawik, (eds), Software Development and Reality Construction, pp. 15-27, Springer Verlag, Berlin Å. Grehag (2001): Requirements Management in a Life-Cycle Perspective: a Position Paper. Department of Computer Science, University of Skövde, Skövde. A. Grynberg and L. Goldin (2003): Product management in telecom industry - using requirements management process. IEEE International Conference on Software: Science, Technology and Engineering, 2003. SwSTE '03. Proceedings., p. 63-70. R. J. Hamann and M. J. A. Oort (2001). "Using a requirement management tool for verification management", Systems Engineering, vol. 4, no. 3, 169-178. F. Hayes (2003). "Required Management." Computerworld, vol. 37, no. 27, 48. M. Hoffmann, N. Kuhn, M. Weber and M. Bittner (2004): Requirements for requirements management tools. 12th IEEE International Requirements Engineering Conference, 2004. Proceedings., p. 301-308. IBM (2005). Rational RequisitePro, http://www-06.ibm.com/software/awdtools/reqpro/, Accessed 20051119 IEEE Std 610.121990 (1990). IEEE Standard Glossary of Software Engineering Terminology, The Institute of Electrical and Electronics Engineers, New York IEEE Std. 830-1998 (1998). IEEE Recommended Practice for Software Requirements Specifications, The Intsitute of Electrical and Electronics Engineeers, New York G. Kotonya and I. Sommerville (1998). Requirements engineering : processes and techniques. J. Wiley, Chichester ; New York. S. Kvale (1997). Den kvalitativa forskningsintervjun, Studentlitteratur, Lund. M. Lang and J. Duggan (2001). "A Tool to Support Collaborative Software Requirements Management", Requirements Engineering, vol. 6, no. 3, 161-172. A. Larsson (2006): Kravadministrationsverktyg: ur ett funktionalitets- och kravkvalitetsperspektiv. Master thesis, Department of Informatics, Lund University, Lund. A. Loconsole (2004): Empirical studies on requirement management measures. Proceedings. 26th International Conference on Software Engineering (ICSE 2004) IEEE, Edinburgh International Conference Center, Scotland, UK, p. 42-44. L. Nguyen and P. A. Swatman (2000). “Managing the Requirements Engineering Process”, http://www.deakin.edu.au/buslaw/infosys/docs/workingpapers/archive/Working_Papers_2000/2000_15_ng uyen.pdf, Accessed 20060320
18
S. L. Pfleeger (2001). Software engineering : theory and practice, 2nd ed., Prentice Hall, Upper Saddle River, N.J. I. Sommerville (2001). Software engineering, 6. ed., Addison-Wesley, Harlow. I. Sommerville (2005). "Integrated Requirements Engineering: A Tutorial." IEEE Software, vol. 22, no. 1, 16-24. The Standish Group (1995): “The Scope of Software Development Project Failures”, The Standish Group, Dennis, MA. (referenced in Pfleeger, 2001) Telelogic AB (2005). DOORS, http://www.telelogic.se/corp/products/doors/index.cfm, Accessed 20051119. R. K. Yin (2003). Case study research : design and methods, 3rd ed., Sage, Thousand Oaks, Calif. D. Zylbermann, Y. Cohen and L. Goldin (2003): The road to requirements maturity. IEEE International Conference on Software: Science, Technology and Engineering, 2003. SwSTE '03. Proceedings, p. 71-79.
19
