Management View on Current Requirements Engineering Practices in Small and Medium Enterprises Uolevi Nikula(1, Jorma Sajaniemi (2, and Heikki Kälviäinen(1 (1 Department of Information Technology, Lappeenranta University of Technology, P.O. Box 20, FIN-53851 Lappeenranta, Finland, {Uolevi.Nikula|Heikki.Kalviainen}@lut.fi; (2
Department of Computer Science, University of Joensuu, P.O. Box 111, FIN-80101 Joensuu, Finland,
[email protected]
Abstract The state-of-the-practice in requirements engineering (RE) is still one of the major problems in software development. Even if technology transfer has been exercised for decades now many of the industry representatives still do not know how they could solve their RE related issues. This paper presents the results of an empirical survey showing that the problem is not in the practitioners’ lack of desire for improvement but in the management not knowing that many RE issues can be solved with standard practices that are well documented in literature. Raising the management awareness of RE practices would make it easier to start RE process improvement efforts in industry and thus eventually also raise the RE process maturity in companies. Keywords: Requirements engineering, software engineering, software process improvement
1. Introduction The current requirements engineering (RE) practices in companies are surely best known by the developers who do the actual requirements and software development work. However, they are seldom the people who decide about improvement efforts in companies. Instead, decisions about improvements, changes, and investments in training or new technology are authorized by the management. Therefore the attitude and knowledge of the management is of key importance when improvement efforts are planned. But how much does the management know about current RE practices, literature, and tools and how important do they rate the RE improvement efforts? A number of researchers have made state-of-the-practice surveys. For example [1], [3], and [13] do very good in-depth analysis of the companies they surveyed and clarify the fundamental problems in RE practices. However, from our point of view they were not that interested in the management view but rather in RE practices as seen in the company in general. To get a more specific indication of the knowledge on current RE practices and desire to improve them, and at the same time to be able to somehow predict the chances of successful improvement efforts, we decided to interview the people who decide how software development is conducted in companies, i.e., the management. The goal of the survey was to get some numerical data on the knowledge on current RE practices and the desire to improve them. To this end a common questionnaire form based on multiple-choice questions was used for all interviews. The questions addressed the knowledge on RE literature and requirement management (RM) tools as well as the current RE practices and desire to improve them. This way a solid understanding of the current RE practices in small and medium enterprises (SMEs) was elicited. Since only twelve SMEs participated in the survey the data is not suitable for
statistical analysis. Despite of that we find the quantitative approach useful for evaluating the current state in companies. The rest of the paper is structured as follows. Section 2 discusses the related research on this topic, Section 3 describes the research methods used, Section 4 presents the results, and Section 5 includes the discussion and conclusions of this survey.
2. Related Research The research problems in RE have been presented in regular intervals in the literature. For example, Siddiqi and Shekaran [20] and Nuseibeh and Easterbrook [16] outline the on-going research and future directions for it. Sawyer, on the other hand, presents the RE problems especially with package software [19]. The Siddiqi and Shekaran paper also discusses the gap between leading edge research and practice. The gap has also been the topic for, for example, Harel’s keynote talk [6] and panel discussion in ICRE 2000 [10]. Especially Harel and Siddiqi make a very clear point that continuous discussion between researchers and practitioners is needed if we want to reduce the gap between research and practice. The in-depth analysis of RE practices include the studies by Curtis, Krasner, and Iscoe [1]; Lubars, Potts, and Richter [13]; and El Emam and Madhavji [3]. Curtis et al. study was initiated from the software engineering viewpoint but it “focused on how requirements and decisions were made, represented, communicated, and changed, as well as how these decisions impacted subsequent development process.” As a conclusion the authors state that knowledge sharing and integration, change facilitation, and broad communications and coordination were the three key issues impacting the project success. Lubars et al. interviewed ten organizations to find out how they defined, interpreted, analyzed, and used requirements. Their results concluded that organizational solutions were preferred over technological ones and general-purpose tools were more common than special purpose tools in RE. El Emam and Madhavji studied RE processes and found seven key issues that must be addressed in a successful RE process improvement effort: package consideration, managing the level of detail of functional process models, examining the current system, user participation, managing uncertainty, benefits of CASE tools, and project management capability. A workshop setting has been used in at least two studies. These are the ones by Morris, Masera, and Wilikens [13] and Kamsties, Hörmann, and Schlich [11]. Morris et al. studied RE and industrial uptake and report that the key problems are training, inherent complexity, internal business integration, and business culture. Their main recommendations to improve the situation were support for the discipline, project flexibility, proposal acceptance, and project relevance. Kamsties et al. studied RE practices in SMEs and asked the workshop participants to rank six different topics in order of relevance and priority to their own environment and development plans at the end of the workshop. The most relevant topics were modeling, improvement of requirements document, inspections, and tools. The first priority was assigned to modeling while requirements document, inspections, and tools were considered equally important. Experience reports are not very common in literature but for example Fowler et al. [3], Regnell, Beremark, and Eklundh [17], and Tvete [23] report their experiences from RE improvement efforts. The Fowler et al. study concerned technology transfer with a “transition package” in the RM area. The authors developed a prototype web site containing RM related materials that the companies could access and use for their own process development tasks. Most of the people utilizing the site
did not have a defined RE process and seemed also in many other ways to be quite in the initial stages of their development efforts. General information on RM was the most accessed type of documents and many participants expressed a need for a primer for RM. Regnell et al. describe a process improvement effort in one company, their positive results, and the new challenges confronted after the deployment of the process. Tvete describes experiences in defining the RE process, selecting and introducing RM tool in one company and problems during the project. Any statistical results are hard to find in literature. However, Weidenhaupt et al. [24] have published some statistical information on scenarios and their usage in RE. Selecting the things to look at is often hard to decide so to help in this problem Sommerville and Sawyer have published their RE process maturity model [21]. This REAIMS model can be used to assess current RE processes and it provides a ready template for doing RE practice assessments. This model is best suited for own development efforts and not for general benchmarking between companies. As of now no papers have been published using the model as a reference model.
3. Research Methods The companies for our survey were selected from different application areas, sizes, ages, etc. with focus in SMEs. Thus the survey gives a general overview of different kinds of companies and their attitudes to RE. It also means that the companies are not fully comparable, but the different kinds of companies were selected since it gives an idea of the development path from a start-up to a large software company. The questions for the survey were selected mainly from the RE literature ([7], [9], and [21]). All these questions were compiled to a four-page questionnaire form and it was used to guide all the interviews. Most of the questions were multiple selection ones or could be answered with few words. All the interviews were conducted by the first author of this paper. The interviews consisted of an introduction to the topic lasting from 15 to 30 minutes and the question and answer part of the interview which lasted from 1 to 4 hours with an average of about 1.5 hours. All the interviews took place on company premises. The interviewees were selected so that they supervised the software developed in the companies. In practice these people participate in prioritizing the process improvement efforts and may also review the projects and how they follow the company standards. The interviews were documented on a structured questionnaire form and tape recorded. The discussions were lead by the interviewer and all the topics were explained in more detail when necessary. In most cases the form was filled in by the interviewer.
4. Results The twelve interviewed companies were located in six different cities in Finland. Only one of the companies operated domestically only while all others competed in international markets with world-class products and three companies had also international development teams. The number of employees varied from four to over one thousand. Most of the companies developed general business information systems but also telecommunications, digital media, data security, industrial applications, and software engineering tools were produced by the companies. The companies were involved in custom software projects and component-based projects as well as commercial product development. The project sizes varied a lot both as effort in man months (MM)
and duration in calendar months. The minimum effort for a project with a project plan was normally 1 or 2 MM while a typical effort was between 12 and 36 MM. The maximum efforts were generally between 20 and 50 MM. Some odd estimates included also minimum effort of 30 MM and maximum of 1200 MM. Typical project duration was between 6 and 12 months. The product sizes were estimated in code lines and the estimates ranged from ten thousand to one million lines with majority between 10 and 60 thousand lines. In small companies with less than ten people the interviewees were chief executive officers while in larger companies quality or development managers or directors participated in the interviews. Any explicit definitions of software development paradigm or lifecycle model were not common in these companies. Only one of the companies had an audited quality system while nine had accepted their processes internally and two applied informal internal processes. The Software Engineering Institute’s Capability Maturity Model (CMM) was not recognized by most of the companies. No company had had its maturity assessed according to the CMM by an outside party while three had done their own evaluations: two were in level 1 and one in level 3. A more accurate description of the companies, projects, and software can be found in [15]. The rest of the section is structured as follows. Section 4.1 discusses the specialization level of the participated companies, 4.2 describes the current RE practices, 4.3 talks about the familiarity with RE books and RM tools, 4.4 summarizes the importance of RE improvement efforts in the companies, and finally 4.5 closes the section with comments on the role of academia in RE improvement efforts. 4.1. Level of Specialization To get a picture of the specialization levels in the companies (n=12) two questions were asked about the employee roles in the companies and the tools used in them. The most common specialist role for team members was designer for user interface, database, or alike that was present in seven companies. The next most common specialist groups were the technical writers and systems analysts, both found in six companies. Three companies had testers and five did not have any such specialists but called all employees developers. Figure 1 shows the team member roles graphically. 8 6 4 2 0 Des UI/DB
Sys. anal.
Tech writer
Testers
All dev.
Figure 1. Team member roles in the companies.
The most commonly used tool in the interviewed companies was a configuration management tool with ten companies. Eight companies used some special testing tools and four had CASE tools in use. No company had RM tools in use. Figure 2 shows the tool usage in graphical format.
12 10 8 6 4 2 0 Conf. mgmt
Testing
CASE
RM
Figure 2. Tool usage in the companies.
4.2. Current RE Practices The current requirements engineering practices in the companies were determined with multiplechoice questions and three open questions. The multiple-choice questions had the same four options as in the REAIMS maturity model [21]: never applied, applied at the discretion of the project manager, normally applied, and applied as a standard practice in the company. The first question was the number of requirements. Since there is no standard definition for such a number the question was “if all the things that a developer must consider when she starts to design/code the software were written in one list, how many items would it contain?” Since none of the companies had a definite answer to this question three categories were used: tens, hundreds, and thousands. Seven companies estimated the requirements count to be in tens, four in hundreds, and one in thousands. In the open questions one company noted that they did not create a separate requirements document but used the database as such as the requirements document. The RM method was also specified in more detail by two companies: one company used email and another change management and tracking system for it. The use of other RM methods is shown in Figure 3 which also reveals that no company used commercial RM tools. 1. Word processor
Standard Normal Discret. Never
2. Spreadsheet 3. Own DB 4. Commercial tool Company count
0
3
6
9
12
Figure 3. Use of different RM methods in Pareto order.
Figures 4-5 show the results of the current RE practices in the interviewed companies. Figure 4 shows answers to six general questions about RE from which it is easy to see that most use natural language in the requirements document and RE process definition looks like the first thing to do in the companies. From Figure 5 we can see what kinds of things are included in the requirement document. In summary the requirement document contents look quite promising for the few companies that actually do it. 1. Use natural lang.
Standard
2. Use semi formal meth.
Normal
3. Use formal methods 4. RE process defined
Discret.
5. Data dictionary done
Never
6. Reqs metrics gathered Company count
0
3
6
9
12
Figure 4. General questions related to RE.
1. Interface descr.
Standard
2. Stakeholders 3. Viewpoints
Normal
4. Domain description
Discret.
5. System description 6. Prototypes
Never
7. Scenarios Company count
0
3
6
9
12
Figure 5. Requirement document contents, inclusion of 7 different topics.
Figure 6 presents the REAIMS Top Ten practices and their application in the companies. The full REAIMS maturity assessment includes 66 different questions and from the outset these questions seemed too advanced for our assessment. Thus we decided to explore the REAIMS model with only the top ten practices. 1. Standard doc structure
Standard
2. Use simple language 3. Formal insp. done 4. Easy change planned
Normal
5. Reqs have unique id 6. Reqs template used
Discret.
7. Analysis checklist used 8. Conflict resol. planned
Never
9. RM policies defined 10. Doc checklist defined Company count
0
3
6
9
12
Figure 6. REAIMS Top Ten –questions and ratings in Pareto order.
The point gains were calculated as follows. First the selected options were scored according to the REAIMS instructions: standard use was scored with 3, normal with 2, discretion with 1, and never used with 0 points [21]. The full REAIMS model divides the practices into basic, intermediate, and advanced ones and calculates the maturity using also this division. On the other hand, though, the REAIMS authors say that the top ten practices should be applied by all companies irrespective of their maturity level. Thus we decided to do a quick assessment just by summing up the top ten practice points and use it as an indication of the maturity. Figure 7 shows the total points for the companies in this survey; clearly most of them still have a lot of room for improvement.
Point Gain
30 27 24 21 18 15 12 9 6 3 0 1
2
3
4
5
6 7 8 Company
9
10 11 12
Figure 7. REAIMS Top Ten -point gains for the companies. The maximum number of points was 30; the best result was 28 points and two companies did not get any points.
4.3. Familiarity with RE Books and RM Tools Familiarity with RE books and tools was clarified with two lists from which the interviewees were requested to indicate familiar titles. The book list consisted of the references ([2], [5], [9], [12], [18], [21], and [22]) and the tool list of the 13 RM tools in the INCOSE survey [8]. The question about these books and tools was whether the name was known; no reading of a book or usage of a tool was required. Four interviewees (n=15) recognized one book and one person two books while ten interviewees did not recognize any book. The best-known book was the book by Davis [2] that was recognized by two interviewees. The books by Gause and Weinberg [5], Jackson [9], Kovitz [12], Sommerville and Sawyer [21], and Thayer and Dorfman [22] were all recognized by one person. The Robertson and Robertson book [18] was the only one that was not recognized by anybody but, on the other hand, it was published only a few months before the survey. Seven interviewees (n=15) recognized from 1 to 3 tools in the tool list and eight did not recognize any of them. The best recognized tool was RequisitePro with four interviewees, DOORS with three, CORE and QSSrequireit with two interviewees each. Caliber RM, Cradle, RDT, RTM, and SLATE were each recognized by one person. 4.4. Importance of RE Improvement Efforts The attitude to RE was clarified with questions whether RE is of strategic importance to the company and what is the importance of RE improvement activities. In this context the term “strategic area” meant that RE is considered essential for the company’s business; however, it was not considered as part of the key business areas in the sense that the RE practices would be confidential information in the company. Presented in this way, ten companies (n=12) considered RE to be strategically important. Five companies also considered their RE process improvement activities to have high importance. Six considered them to be of medium importance and one rated them as being of low importance. 4.5. Role of Academia in RE Improvement Efforts The questions on technology transfer addressed also the possible role of academia in such efforts. Eleven of the companies were interested in starting a RE development project with an academic partner. Ten companies wanted this cooperation to result in new specialists in the RE field while eight companies wanted support in their process development work.
5. Discussion and Conclusions This state-of-the-practice survey studied management knowledge on RE and its desire to improve the current state in twelve small and medium software houses. The current RE practices in the companies were mostly add-hoc ones. A lightweight maturity assessment was done with the REAIMS Top Ten –questions that – according to the REAIMS community – should be applied by all companies irrespective of their maturity level. Only three out of twelve companies got more than 7 points out of 30 revealing the extremely low level of RE practices. As a consequence, basic systematic approaches alone would provide a lot of help for most of the companies. The desire to improve was, however, clear within the companies. Half of the companies considered their RE improvement efforts to have medium importance and over 40% considered them to have high importance. Thus the desire to improve RE practices exists but no such development was under way. So why does the management not initiate process improvement in RE and RM? A major source of problems seems to be the management’s unfamiliarity with available techniques and tools. The fact that only one third of the interviewees knew any book on RE and less than half any RM tool can be expected to slow down the adoption of RE practices and RM tools in companies. The fact that only few companies have people specialized in RE does make the situation even worse. Such people could inform the management about standard practices, but the lack of RE specialists within the companies makes this source of information unavailable. As a consequence, the management wishes help from universities, demonstrated by over 90% of the companies wanting to cooperate with universities in RE improvement efforts. Thus a need for major technology transfer efforts in the requirements engineering is evident. This survey was done in SMEs and with fairly small projects, and the benefits of more formal RE practices in this scale may be questioned. However, the size of the software is hardly the right determinant for RE quality requirements. We feel that implementing the right functionality and fulfilling the right requirements is most important for successful projects. The survey revealed that the management of software companies feel the same. Thus even the smallest projects should utilize the standard RE practices – applied in a more of less lightweight way. But to achieve this, the management must know that standard solutions do exist.
Acknowledgements The authors would like to thank all the companies that participated in this survey. The East Finland Universities Graduate School in Computer Science and Engineering (ECSE) is acknowledged for financial and scientific support. We are also very grateful to Pete Sawyer for his invaluable comments to our earlier version of this paper.
References [1] Curtis, Bill, Herb Krasner and Neil Iscoe (1988), A Field Study of the Software Design Process for Large Systems, Communications of the ACM, Vol. 31, No. 11, pp. 1268-1287. [2] Davis, Alan M. (1993), Software Requirements: Objects, Functions, and States. Prentice Hall. [3] El Emam, Khaled and Nazim H. Madhavji (1995), A Field Study of Requirements Engineering Practices in Information Systems Development, in: Proceedings of the Second IEEE International Symposium on Requirements Engineering, York, England: IEEE Computer Society, pp. 68-80. [4] Fowler, Priscilla, Malcom Patrick, Anita Carleton and Barbara Merrin (1998), Transition Packages: An Experiment in Expediting the Introduction of Requirements Management, in: Proceedings of the Third IEEE International Conference on Requirements Engineering, Colorado Springs, Colorado: IEEE Computer Society, pp. 138-145.
[5] Gause, Donald C. and Gerald M. Weinberg (1989), Exploring Requirements: Quality Before Design. New York: Dorset House. [6] Harel, David (1997), Will I be Pretty, Will I be Rich? Some Thoughts on Theory vs. Practice in Systems Engineering, in: Proceedings of the Third IEEE International Symposium on Requirements Engineering, Annapolis, Maryland, USA: IEEE Computer Society, pp. 184-186. [7] IEEE, Institute for Electrical and Electronics Engineers (1998), IEEE Recommended Practice for Software Requirements Specifications, in: Software Requirements Engineering, Second Edition, eds. Richard H. Thayer and Merlin Dorfman, Los Alamitos, California: IEEE Computer Society Press, pp. 207-244. [8] INCOSE, International Council On Systems Engineering (1999), Tools Survey: Requirements Management (RM) Tools (Online), Available World Wide Web: URL: http://www.incose.org/tools/tooltax.html (Accessed 28 June 1999). [9] Jackson, Michael (1995), Software Requirements & Specifications – a lexicon of practice, principles and prejudices. Addison-Wesley. [10] Kaindl, Hermann (2000), Why Is It so Difficult to Introduce Requirements Engineering Research Results into Mainstream Requirements Engineering Practice, in: Proceedings of the 4th IEEE International Conference on Requirements Engineering, Schaumburg, Illinois, USA: IEEE Computer Society, pp. 67-68. [11] Kamsties, Erik, Klaus Hörmann and Maud Schlich (1998), Requirements Engineering in Small and Medium Enterprises, Requirements Engineering, Vol. 3, No. 2, pp. 84-90. [12] Kovitz, Benjamin L. (1999), Practical Software Requirements: A Manual of Content and Style. Greenwich: Manning Publications Company. [13] Lubars, Mitch, Colin Potts and Charles Richter (1993), A Review of the State of the Practice in Requirements Modeling, in: Proceedings of the IEEE International Symposium on Requirements Engineering, San Diego, CA: IEEE Computer Society, pp. 214. [14] Morris, Philip, Marcelo Masera and Marc Wilikens (1998), Requirements Engineering and Industrial Uptake, in: Proceedings of the Third IEEE International Conference on Requirements Engineering, Colorado Springs, Colorado: IEEE Computer Society, pp. 130-137. [15] Nikula, Uolevi, Jorma Sajaniemi and Heikki Kälviäinen (2000), A State-of-the-Practice Survey on Requirements Engineering in Small and Medium Sized Enterprises. Lappeenranta, Finland: Lappeenranta University of Technology (TBRC Research Report 1). [16] Nuseibeh, Bashar and Steve Easterbrook (2000), Requirements Engineering: A Roadmap, in: Proceedings of the 22nd International Conference on Software Engineering, New York: ACM, pp. 35-46. [17] Regnell, Björn, Per Beremark and Ola Eklundh (1998), A Market-Driven Requirements Engineering Process: Results from an Industrial Process Improvement Programme, Requirements Engineering, Vol. 3, No. 2, pp. 121-129. [18] Robertson, Suzanne and James Robertson (1999), Mastering the Requirements Process. Harlow, England: AddisonWesley. [19] Sawyer, Pete (2000), Package Software: Challenges for RE, in: Proceedings of the 12th International Conference in Advanced Information Systems Engineering, Stockholm, Sweden: Springer, pp. 137-142, (Lecture Notes in Computer Science). [20] Siddiqi, Jawed and Chandra M. Shekaran (1996), Requirements Engineering: The Emerging Wisdom, IEEE Software, Vol. 13, No. 2, pp. 15-19. [21] Sommerville, Ian and Pete Sawyer (1997), Requirements Engineering – A Good Practice Guide. Chichester, England: John Wiley & Sons. [22] Thayer, Richard H. and Merlin Dorfman (eds) (1997), Software Requirements Engineering, 2nd edition. Los Alamitos, California: IEEE Computer Society. [23] Tvete, Bjarne (1999), Introducing Efficient Requirements Management, in: 10 th International Workshop on Database & Expert Systems Applications, Florence, Italy: IEEE Computer Society, pp. 370-375. [24] Weidenhaupt, Klaus, Klaus Pohl, Matthias Jarke and Peter Haumer (1998), Scenarios in System Development: Current Practice, IEEE Software, Vol. 14, No. 2, pp. 34-45.
