Improving Requirements Engineering Processes in ...

Improving Requirements Engineering Processes in Very Small Software Enterprises Through the Use of a Collaborative Application Sergio F. Ochoa, Alcides Quispe, Andrés Vergara, José A. Pino Computer Science Department Universidad de Chile Santiago, Chile {sochoa, aquispe, avergara, jpino}@dcc.uchile.cl Abstract— Very small software enterprises (up to 25 employees) represent the most important software task force around the world. Several researches report these organizations remain a low success rate in development projects, because they implement inefficient mechanisms to support requirements engineering practices. This paper presents a collaborative software application, named Software Requirements Manager (SRM), which automates several requirements engineering practices. This tool acts as a communication and coordination space for team members. SRM has been used in a software development academic scenario during the last three years. The obtained results show that SRM can significantly improve the projects’ success rate by facilitating the collaboration among members of the development team. Keyword. CSCW, Communication and Coordination Support, Collaborative Tool, Requirements Engineering.

I.

INTRODUCTION

While much research effort has been done on software engineering, the number of projects failures remains high and the success percentage remains low [1, 3, 6]. Software systems are frequently delivered too late, with budget overrun, and do not meet the needs of the stakeholders [6, 15, 33]. Several studies indicate that poor requirements engineering practices are often the main cause of these problems [13, 17, 28, 32]. A survey over 3800 organizations in 17 countries concluded that most of the perceived software problems are related to requirements specification (>50%) and management (50%) [16]. Several researchers have identified the following issues as cause of problems in the requirements engineering process: Inadequate Supporting Tools [12, 13, 14]. Many requirements engineers do not use adequate supporting tools when engineering their requirements. Typically they use a mix of specification documents, spreadsheets and Web repositories. The use of these scattered solutions is expensive and error-prone because both (1) it involves manual processing and (2) requirements are kept in several dispersed resources and sometimes they are incoherent.

Inadequate Requirements Management [2, 12, 14]. Provided requirements are frequently managed in documents they become difficult manipulate, maintain and validate. Lack of access control does not allow limiting the access to sensitive requirements and implementing proper change control mechanisms. Lack of centralized and automated requirements management also make difficult the capture, analysis, and report of requirements-based metrics (e.g., requirements stability, maturity, and completion). These two issues generate communication and coordination problems to the development team. This paper presents a collaborative software application, named Software Requirements Manager (SRM), which was designed and evolved to support the requirements engineering process. This tool acts as a communication and coordination space for team members, and it was particularly designed to be used in Very Small Software Enterprises –VSSE (up to 25 employees [19])– . The goals for supporting this type of software companies are two: (1) these organizations represent the most important software task force around the world [23, 27, 31] and (2) most current software requirements tools have shown to be inappropriate to support small software companies [2, 12, 14]. SRM was used in 18 small software projects from 2007 to 2009. This tool has been evolved three times, trying to include the lessons learned. The improvement was focused on the communication and coordination mechanisms allowing us to increase the project success rate. Therefore our work hypothesis indicates that a mix of requirements automated processing and an appropriate set of requirements-based metrics can help reduce the requirements engineering effort and the communication and coordination problems in VSSE. If this hypothesis is true, the tool would help improve the success rate of software projects. Next section describes the challenges to be addressed when doing requirements engineering in VSSE. Section III discusses the related work. Section IV describes the SRM tool and its main components. Section V presents and discusses the

obtained results. Section VI presents the conclusions and future work. II.

REQUIREMENTS ENGINEERING IN SMALL SOFTWARE COMPANIES

Several researchers report the company size matters [2, 11, 27]. Although large and small software organizations face similar software engineering challenges, they often require different approaches to deal with their needs. These approaches consider differences in various aspects, such as the business models and goals, market niche, availability of (financial and human) resources, process and management capability, and organizational differences [27]. Requirements engineering (RE) is one of the key process areas being performed in a different way, depending on the organization size [2]. In addition, requirements engineering practices are highly related to the success and failure rates of software projects [7, 9, 13, 14, 16, 20, 24]. For that reason, it must be conducted following an appropriate strategy. The strategy have to consider the requirements engineering process in VSSE must be as lightweight as possible because: (1) the VSSEs feel there is no time to effectively elicit and analyze requirements [4], (2) RE activities are often treated as a time-consuming, bureaucratic and contractual process [24, 25]. After analyzing several projects in small software companies, Vergara arrives to a similar conclusion: “VSSE team members have few time to spend on engineering requirements; therefore the requirements engineering process must be as cheap (in terms of required effort) as possible” [29].

status. If the tool is not able to provide such capabilities, then it becomes unsuitable for the development team. Next section presents SRM, which is a RE tool conceived to be used in VSSE software development scenarios. Such tool automates several RE best practices in order to reduce the effort to conduct the RE process. Moreover, SRM helps to communicate and coordinate team members. IV.

SOFTWARE REQUIREMENTS MANAGER

SRM is a collaborative application which supports team members during a software project development. The tool acts as an information repository providing communication and coordination inside the team and also with the stakeholders. The requirements are the central element of the tool. SRM is able to manage multiple projects; each one as an independent shared space. A project (i.e. a shared space) is composed of users and software requirements, design artifacts, test cases and a list of team members with roles (Fig. 1). The tool has a Web user interface allowing developers to view or manipulate the project artifacts depending on their access grants.

After several years of experimenting in VSSE, the authors have found the automation of the RE process helps to reduce the required effort to perform this activity. Furthermore, a shared view of the project status and evolution helps to deal with communication and coordination problems inside the development team. The Software Requirements Manager (SRM) tool was designed and evolved following these two principles. III.

RELATED WORK

There is a large list of tools to support the requirements engineering process. Examples of them are CaliberRM [5], RequisitePro [26], DOORS [8], CORE [30] and RDT [22]. However, several software engineering researchers state the mainstream RE practice relies on office tools (editors and drawing tools) rather than targeted RE-tools [18, 22]. One of the reasons for not using RE-tools includes financial causes, like high price and low return on investment [21]. Other researchers state that VSSE develop software with very tight schedules; therefore they are not able to spend time on using solutions that represents an overload to the requirements engineering process [2, 14]. Clearly, the use of these complete, but bureaucratic solutions will produce delays on the project execution, which represents a risk for VSSE. VSSE require simple and flexible RE tool, because they develop under high pressure. Therefore they need to adapt the RE process according to the project evolution and current

Figure 1. SRM Work Context

SRM supports two almost independent data flows: (1) the input, which represents the feeding process, and (2) the output, that represents the feedback process. The input process typically allows performing several activities depending on the team members’ role: Project Manager: This role creates projects (i.e. shared spaces), manages the work team (i.e. project users), and provides access grants to the users. Analyst: The analyst role specifies and manages the users and software requirements, and keeps the link between these two types of requirements. Designer: This role defines and manages the design artifacts, and relates these artifacts with the defined software requirements.

Programmer: The programmer adds URL links to the design artifacts. Typically such links allow access to a prototype implementation of each artifact.

shows the interface for creating and modifying software requirements (SR).

Tester: The tester role specifies and manages the test cases, and relates these instruments with the software requirements and design artifacts. By default, SRM allows team members to view all artifacts stored in the project. Typically it is useful to keep developers communicated and coordinated. However, access grants to the project resources can be changed by project manager in order to adhere with any other setting.

the the the the

Unlike the input process, the output distinguishes (by default) just two user categories: team members and stakeholders. On the one hand, the team members are able to retrieve: (1) a large list of metrics describing the project status and evolution, and (2) the documents which are generated ondemand by the tool, based on the project information and a set of standardized templates. The templates used to generate these documents are adaptations of the templates defined by the ESA software engineering standard [10]. On the other hand, the stakeholders are also able to access some project information. By default, they can access the users’ requirements list, the accomplishment level of them, and a metric indicating the progress of the project. A.

SRM Architecture SRM has a layered architecture because the separation of concerns in the product is clearly defined. The lower layer is in charge of providing information persistency (Fig. 2). The middle layer includes the managers of the several resources included in the shared space (i.e. in a project). Every manager represents an independent subsystem, except the Design Artifact manager and the Test Cases manager. These components use services provided by the Requirements Manager component in order to relate test cases and design artifacts with the project software requirements. This link allows automatic computation of several metrics; e.g. the project progress level or the pending requirements related to a software piece (i.e. a design artifact).

Figure 3. User interface to create and update software requirements

A new SR is created by defining the several requirements attributes. These attributes are an adaptation of those proposed by the ESA Software Engineering Standard [10]. Every SR must be associated at least with one User Requirements (UR) in order to ensure that every SR contributes to reach the stakeholders expectations (i.e. the user requirements). A similar process is followed to add/update/delete design artifacts and test cases. All project artifacts are mapped to software requirements in order to keep the requirements track along the project. SRM implements several correspondence matrices, which show the matching between project artifacts and requirements. These matrices are part of the reporting process. C. Reporting Process SRM has several reports available for stakeholders. For example, the requirements and test cases documents (in RTF format) are generated on-demand based on the information stored in the tool. In addition, part of the design document is also generated using that information.

Figure 2. Basic Architecture

Finally, the upper layer allows, through Web pages, access to the services provided by the managers. CRUD (Create, Read, Update and Delete) components support the input flow described in the previous section; and the reporting components support the output flow. B. Feeding Process Create, update or delete actions on requirements, design artifacts or test cases are part of the feeding process. Fig. 3

Figure 4. Matching between UR and SR

The User Requirements vs. Software Requirements report (Fig. 4) shows the state of every UR (the icons indicate the current accomplishment status). This report also shows the SR and user profile related to each UR. It allows developers to know, e.g., the accomplishment level of the project, or those requirements related with a particular user profile.

Figure 5. SRM Dashboard

SRM has also a dashboard depicting coarse-grain information. Figure 5 presents two useful dashboard charts: (1) the level of requirements accomplishment (percentage of requirements implemented and approved) per milestone and (2) the project level of accomplishment. Each column represents a project milestone in the first chart. The color of each column indicates the accomplishment level of the requirements involved in such milestone. The second chart uses the same color code, but it represents the current status of the project requirements. D. Users/Project Management Figure 6 shows the users management module. Once the user has selected a project to work with, this module shows the user names of people who may access the project resources, and the rights to access each resource (R: read, W: write; 0: No access). Typically, the project manager is in charge of assigning the access grants to the team members.

Figure 6. Users management interface

The users’ management module also allows creating new users for a project. A new user is created by entering a username, a password, the role s/he plays in the project (i.e. Project Manager, Analyst, Designer, Programmer or Tester) and a contact email, as shown in Fig. 6.

V.

OBTAINED RESULTS

The first version of SRM was released in 2006. The tool has been used in every semester since 2007 in an advanced software engineering course of the computer science program at the University of Chile. The course is delivered in the ninth semester of the program, and it requires the students to form groups (each one with 5 to 7 members) and then, they must develop a real software project for a client. Most participants already have experience developing software for the industry; therefore each group can be considered as a novice development team. The instructors and the course dynamics were the same during the whole period in which this experience was performed. These students have several other activities that push them to interact in an asynchronous way, because it is not easy to match the team members’ time availability. In such scenario there were historically several communication and coordination problems, not only among team members but also with the project clients and users. Trying to deal with such situation, the SRM tool was evolved three times following the hypothesis stated in section 1: a mix of requirements automated processing and an appropriate set of requirements-based metrics can help reduce the requirements engineering effort and the communication and coordination problems in VSSE. Next table shows a summary of the project results, in which the teams used the several versions of SRM. The first track (2006) corresponds to the period in which the SRM was not used in the course. The second track (2007) corresponds to the use of SRM version 1.0. Such version included support to specify and export UR and SR. It also had a correspondence matrix, able to match both types of requirements. Table 1. Project evolution vs. SRM evolution

The third track (2008) corresponds to the use of SRM version 1.5, which included several indicators and alarms related to requirements problems or warnings. Finally, the fourth track (2009) corresponds to the use of SRM version 2.0, which included several improvements in the user interface and also other indicators and alarms were added. The improvement of this track was focused on reducing the effort required to use the tool and increase the communication and coordination mechanisms for the team members. Analyzing the information presented in table 1 we can see the correspondence matrix (included in the second track – 2007) seems to help avoid the loss of requirements. In addition, the inclusion of indicators and alarms (third track – 2008) seems to help improve communication among team members and reduce the number of errors in the requirements specification. The use of this SRM version also increased the rate of accomplished requirements. These numbers have improved in 2009 with the use of a more usable and proactive version of the tool. The authors think the improvement of the requirements accomplishment rate can be explained, in part because now it is easier to do requirements engineering using SRM. Figure 7 presents a summary of the software requirements accomplishment, considering the several tracks and involved projects.

VI.

CONCLUSIONS AND FUTURE WORK

Requirements are the driving force behind a software project. Most of the software lifecycle phases depend on requirements. Therefore, it is important to gather and manage them in an appropriate way. This activity is particularly critical in VSSE, because these organizations work with very tight schedules and budgets. Considering these small companies represent the most important software task force around the world, an improvement in their requirements engineering practices could represent a great positive impact for this industry. Communication and coordination issues, and also the effort to carry out the requirements engineering process have been identified as key elements that influence the low rate of success that currently shows the VSSE. This article presented the Software Requirements Manager (SRM), which is a tool intending to address these challenges. SRM has evolved during the last three years, trying to include (in a best way) these improvement directives. The obtained results have improved when each new version of the tool was used. It could be indicating the evolution direction of SRM could be the right improvement for RE practices in VSSE. The experimental results were obtained in an academic setting, which is a bit different to the professional work in the software industry. However, most of the students participating in this experience are part-time developers for VSSE. For that reason the authors think the tool could fit the needs of these organizations. The authors know that some Chilean VSSE are using the tool since SRM is already an Open Source project. However, there is no access to these results due to business privacy reasons. Therefore, the next step in this research work is to conduct a formal evaluation of SRM in the industry. Such experience will allow us to understand the contribution and limitations of the tool to support the requirement engineering process in VSSE. ACKNOWLEDGEMENTS

Figure 7. Percentage of requirements accomplishment

Clearly, the last two versions of SRM have helped to increase the requirements accomplishment rate, reduce the number of alarms and requirements problems (i.e. erroneous, ambiguous and lost requirements). It means that probably the shared information of the project helped improve the communication and coordination among team members. In addition, there is a reduction of the effort to use the tool, which also contributes to improve the results. Therefore, these numbers show some relevant evidence about the validity of the stated hypothesis. SRM is currently being used by other software organizations. Unfortunately, the authors do not have access to those projects results due to business privacy reasons.

The work of Alcides Quispe was partially supported by the NIC Chile scholarship for PhD students. REFERENCES [1] [2]

[3]

[4]

[5] [6]

A. I. Anton. “Successful Software Projects Need Requirements Planning”. IEEE Software, Vol. 20, No 3, pp. 44 – 46, May/Jun. 2003. J. Aranda, S. Easterbrook, G. Wilson. “Requirements in the Wild: How Small Companies Do It”. Proc. of the 15th IEEE Requirements Engineering Conference, pp. 39 – 48, Oct. 15–19, 2007. R. Berntsson-Svensson, A. Aurum. “Successful Software Project and Products: An Empirical Investigation”, Proceedings of the 5th ACM/IEEE International Symposium on Empirical Software Engineering, pp. 144 – 153, Sep. 21–22, 2006. Borland Software Corporation. “Effective Requirements Definition and Management”. White Paper, May 2009. URL: http://www.borland.com/resources/en/pdf/solutions/rdm_whitepaper.pdf Caliber-RM, Borland, 2009, URL: http://www.borland.com/caliber/index.html. Last visit: Sept. 2009. R. N. Charette. “Why Software Fails”. IEEE Spectrum, Vol. 49, No 9, pp. 42-49, Sept. 2005.

[7]

[8] [9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

[19]

D. Damian, J. Chisan, L. Vaidyanathasamy, Y. Pal. “An Industrial Case Study of the Impact of Requirements Engineering on Downstream Development”. Proc. of the Int. Symp. on Empirical Software Engineering (ISESE’03), IEEE CS Press, pp. 40-49, Sept. 30- Oct. 1, 2003. Doors Telelogic (IBM Company), 2009, URL: http://www.telelogic.com/doors. Last visit: Oct. 2009. C. Ebert, J. De Man. “Requirements Uncertainty: Influencing Factors and Concrete Improvements”. Proc. of the 27th Int. Conf. on Software Engineering (ICSE'05), pp. 553-560, May 15-21, 2005. European Space Agency (ESA). “ESA Software Engineering Standards”. PSS-05-0 Issue 2. ESA Board for Software Standardization and Control (BSSC). February, 1991. M. E. Fayad, M. Laitinen, R. P. Ward. “Software Engineering in the Small”. Communications of the ACM, Vol. 43, No 3, pp. 115 – 118, Mar. 2000. D. Firesmith. “Common Requirements Problems, Their Negative Consequences, and the Industry Best Practices to Help Solve Them”. Journal of Object Technology, Vol. 6. No. 1, pp 17-33, JanuaryFebruary 2007. H. F. Hofmann, F. Lehner. “Requirements Engineering as a Success Factor in Software Projects”. IEEE Software, Vol. 18, No 4, pp. 58-66, Jul/Ago 2001. N. Juristo, A. M. Moreno, A. Silva. “Is the European Industry Moving toward Solving Requirements Engineering Problems?”.IEEE Software, Vol. 19, No 6, pp. 70 – 77, Nov/Dec 2002. S. Konrad, M. Gall. “Requirements Engineering in the Development of Large-Scale Systems”. Proc. of the 16th IEEE Int. Requirements Engineering Conference, pp. 217-222, Sept. 8-12, 2008. A. Lamsweerde. “Requirements Engineering in the Year 00: a Research Perspective”. Proceedings of the 22nd International Conference on Software Engineering, pp. 5 – 19, Jun. 4 – 11, 2000. A. Lamsweerde. “Requirements Engineering: From Craft to Discipline”.Proc. of the 16th ACM SIGSOFT Int. Symp. on Foundations of Software Engineering (FSE’08), pp. 238 – 249, Nov. 9-14, 2008. M. Lang, J. Duggan, J. “A Tool to Support Collaborative Software Requirements Management”. Requirements Engineering, 6 (3), pp. 161172, 2001. C. Y. Laporte, Simon Alexandre, Alain Renault. “Developing International Standards for Very Small Enterprises”. IEEE Computer, Vol. 41, No. 3, pp. 98-101, Mar. 2008.

[20] A. Loconsole. “Empirical Studies on Requirements Management Measures”. Proc. of the 26th Int. Conf. on Software Engineering (ICSE’04), pp. 42-44, May 23-28, 2004. [21] R. Matulevičius. “Validating an Evaluation Framework for Requirements Engineering Tools”, In: Krogstie, J., Halpin, T., Siau K. (ed.): Information Modeling Methods and Methodologies (Adv. Topics of Database Research), Idea Group Publishing, pp. 148-174, 2005. [22] Matulevičius, R., Sindre, G. (2005). “Tool Support and Specification Quality: Experimental Validation of an RE-Tool Evaluation Framework”. LNCS Vol. 3770, pp. 433-443, 2005. [23] C. Neumuller, P. Grunbacher. “Automating Software Traceability in Very Small Companies: A Case Study and Lessons Learned”. Proc. of the 21st IEEE Int. Conf. on Automated Software Engineering (ASE'06), pp. 145 – 156, Sep. 18-22, 2006. [24] B. Nuseibeh, S. Easterbrook. “Requirements Engineering: A Roadmap”. Proc. of the Future of Software Engineering Conference (FOSE’00), pp. 35 – 46, Jun. 04-11, 2000. [25] D. J. Reifer. “Requirements Management: The Search for Nirvana”. IEEE Software, Vol. 17, No 3, pp. 45 – 47, May 2000. [26] RequisitePro, IBM Rational Software, 2009, URL: http://www306.ibm.com/software/rational/. Last visit: Oct. 2009. [27] I. Richardson, C. G. von Wangenheim. “Why Are Small Software Organizations Different?”. IEEE Software, Vol. 24, No 1, pp. 18 – 22, Jan./Feb. 2007. [28] J. Ropponen, K. Lyytinen. “Components of software development risk: how to address them? A project manager survey”. IEEE Transactions on Software Engineering, Vol. 26, No 2, pp. 98 – 112, Feb. 2000. [29] A. Vergara. “Generación Automática de Métricas en Proyectos de Software a Partir de la Especificación de Requisitos”. Master Thesis (in Spanish). Computer Science Department, University of Chile. June 2008. [30] ViTech Corp. CORE 6.0. URL: http://www.vitechcorp.com/. Last visit. Oct. 2009. [31] C. G. von Wangenheim, A. Anacleto, C. F. Salviano. “Helping Small Companies Assess Software Processes”. IEEE Software, Vol. 23, No 1, pp. 91 – 98, Jan./Feb. 2006. [32] K. E. Wiegers. “Software Requirements (Second Edition)”. Microsoft Press, 2003. [33] K. E. Wiegers, S. McKinsey. “Requirements Management: the Proven Way to Accelerate Development”. Serena Software Inc., White Paper, 2005. URL: http://www.serena.com/docs/repository/products/rm/wp900001-0505.pdf