Co-located and distributed natural-language ...

JOURNAL OF SOFTWARE: EVOLUTION AND PROCESS J. Softw. Evol. and Proc. 2016; 28:205–227 Published online 26 February 2016 in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/smr.1772

Co-located and distributed natural-language requirements specification: traditional versus reuse-based techniques Juan M. Carrillo de Gea1,*,†, Joaquín Nicolás1, José L. Fernández Alemán1, Ambrosio Toval1, Sofia Ouhbi2 and Ali Idri2 1

Software Engineering Research Group, Regional Campus of International Excellence “Campus Mare Nostrum”, University of Murcia, Murcia, Spain 2 Software Project Management Research Team, ENSIAS, Mohammed V University, Rabat, Morocco

ABSTRACT Requirements Engineering (RE) includes processes intended to elicit, analyse, specify and validate systems and software requirements throughout the software life cycle. Mastering the principles of RE is key to achieving the goals of better, cheaper and quicker systems and software development projects. It is also important to be prepared to work with remote teammates, as distributed and global projects are becoming more common. This paper presents an experiment with a total of 31 students from two universities in Spain and Morocco who were assigned to either a co-located or a distributed team. Both traditional and reuse-based requirements specification techniques were applied by the participants to produce requirements documents. Their outcomes were then analysed, and the approaches were compared from the point of view of their effect on a set of performance-based and perception-based variables in co-located and distributed settings. We found significant differences in only productivity (Z = 2.320, p = 0.020) and difficulty (Z = 2.124, p = 0.034) as regards the scores attained for non-reuse and reuse conditions, both in the co-located modality. Our findings show that, in general, the participants attained similar results for requirements specification when using the two strategies in both distributed and non-distributed environments. Copyright © 2016 John Wiley & Sons, Ltd. Received 3 March 2015; Revised 2 November 2015; Accepted 25 January 2016 KEY WORDS:

experiment; global software development; internationalisation; requirements reuse; requirements specification; software engineering education

1. INTRODUCTION Poor requirements are one of the most common causes of project failure in any domain [1, 2]. A typical estimation for a regular project is that around 10% of its effort will be devoted to requirements; however, research by Hofmann and Lehner [3] shows that the most successful software projects of the 15 projects analysed were those that allocated more than 28% of their resources to requirements. Furthermore, a study at the National Aeronautics and Space Administration corroborates that those projects that devoted more than 10% of their resources to Requirements Engineering (RE) resulted in lower costs and smaller deviations from their estimates [4]. RE activities continue to play an essential role in both traditional and agile developments [5], even though the traditional means used to engineer requirements have been challenged by the rapidly changing business environment and dynamic context in which most organisations currently operate [6, 7]. According to Bourque and Fairley [8] and Kotonya and Sommerville [9], the documentation or specification of requirements is *Correspondence to: Juan M. Carrillo de Gea, Department of Informatics and Systems, Faculty of Computer Science, University of Murcia, Campus Universitario de Espinardo 30100, Murcia, Spain. † E-mail: [email protected] Copyright © 2016 John Wiley & Sons, Ltd.

206

JUAN M. CARRILLO DE GEA ET AL.

one of the main activities of RE. Software professionals are expected to be familiar with it, and paying sufficient attention to this activity in the Software Engineering (SE) curricula is highly advisable. Requirements are generally specified in an industry using unstructured natural language (NL) [10], because it is easier for all stakeholders, even those who lack any kind of technical training, to understand than other non-textual notations. However, NL is inherently ambiguous and can lead to different interpretations depending on the context [11–13]. This drawback will therefore be exacerbated as regards language, social and cultural differences, and problems resulting from a lack of tacit knowledge related to requirements will arise when globally distributed stakeholders are involved in the process [14]. Nevertheless, this is currently not an impediment to systems and software development becoming more and more globalised, as is shown by the growth of global software development (GSD) in recent years [15]. It is thus necessary to reconsider existing findings regarding challenges and interventions for GSD from a competence-based perspective that emphasises team member training and education [16]. There is a lack of research focused on the study of software reuse in GSD. Although the importance of software reuse is acknowledged by many researchers, Rine and Nada [17] were the first authors to empirically show that the reusability level determines the effectiveness of improvements as regards productivity, quality and development time. They thus concluded that greater benefits are obtained when reusability is applied during the initial processes of the software development life cycle. More recently, case studies carried out by Pacheco et al. [18] and Goldin and Berry [19] have provided further evidence of this. However, to the best of our knowledge, there is no empirical evidence regarding the impact of requirements reuse on GSD. The innovative contribution of this paper resides in the implementation and comparison of two strategies for requirements specification, which were put into practice by both co-located and globally distributed students in a nearshore scenario [20] involving two universities: the Mohammed V University (Rabat, Morocco) and the University of Murcia (Murcia, Spain). The experiment was carried out using internationalisation (i18n) standards and an i18n requirements catalogue [21, 22]. The i18n context described in the paper can be easily extrapolated to many other RE scenarios because the type of artefact—NL requirements—does not differ significantly from those usually found in other domains. Moreover, the i18n domain is very appealing at present, given the growing need for software that has been tailored to users located in different countries around the world. A traditional strategy that is used to specify requirements, namely requirements specification from source documents, was first applied by the students (i.e. the i18n standards). The students then applied several techniques related to requirements reuse with the purpose of specifying requirements from a catalogue of reusable requirements [18] (i.e. the i18n catalogue). Finally, an empirical evaluation of the outcome of these activities was carried out. It is relevant to make this comparison from a practical perspective because companies need to have some certainty that investing in producing requirements catalogues is worth the effort before doing so. From the point of view of research, no previous investigations related to catalogue-based requirements reuse in GSD have been undertaken to date. Furthermore, experimentation with students is viable in situations in which the tasks to be performed do not require industrial experience [23, 24]. Students may play a very important role in SE experimentation. Before performing costly studies in industry, it is generally useful for researchers to carry out pilot studies with students in academic environments [25, 26]. This paper is structured as follows: Section 2 presents a review of the related literature. The techniques applied in the study are described in Section 3. Section 4 reports the hypotheses, participants, design, instrumentation and measures of the experiment. Our results and the main findings of the study are presented in Section 5. Finally, Section 6 includes some concluding remarks.

2. LITERATURE REVIEW Many empirical studies have stressed the fact that the RE process in GSD becomes easier once the problems of language, geographic and temporal distance and cultural differences between stakeholders have been overcome. In this context, Prikladnicki et al. [27] presented the lessons learned from case studies in two software development units from multinational organisations Copyright © 2016 John Wiley & Sons, Ltd.

J. Softw. Evol. and Proc. 2016; 28:205–227 DOI: 10.1002/smr

CO-LOCATED AND DISTRIBUTED NATURAL-LANGUAGE REQUIREMENTS SPECIFICATION

207

located in Brazil. They learned that RE is the main challenge from the software development process point of view. Requirements identification was a challenge in all the projects and involved activities such as meetings, requirements documentation as soon as defined, traceability, requirements control and management. One of the development managers interviewed stated that a good approach with which to minimise the problems involved in distributed RE was that of conducting as many meetings as necessary to understand all the requirements and documenting all meetings in detail in order to obtain the acceptance of all the stakeholders. Herbsleb et al. [28] reported an experience in nine of the Siemens Corporation’s globally distributed software development projects. These represent a range of collaboration models, from co-development to the outsourcing of components to an entire project. Many issues arose during this experience, one of which concerned requirements communication. It was noticed that if requirements are not clearly conveyed, issues may not begin to surface until integration, as the sites realised that their interpretations differed. Daneva et al. [29] explicated the participation and involvement of nearshore development centres’ architects in quality requirements tasks based on interviews with 16 practitioners working on large projects. Software architects from nearshore development centres are actively involved in quality requirements documentation and validation, are relatively passive participants in quality requirements elicitation and prioritisation and are not at all involved in quality requirements negotiation. Bhat et al. [30] presented real-life case studies at an Indian firm (Infosys Technologies) focused on information technology services in order to provide insights into the root causes of RE phase conflicts in client–vendor offshore-outsourcing relationships. They showed that most RE practices recommended for distributed software development teams are relevant for client–vendor outsourcing in GSD. The case studies reflected the difficulties involved in fulfilling the RE process when there are conflicting client–vendor goals, low client involvement, conflicting RE approaches, misalignment of client commitment to project goals, disagreements as regards tool selection, communication issues, disowning responsibility, sign-off issues or tools misaligned with expectations. A root-cause analysis of the nine case studies revealed that the following key strategic factors are essential to RE’s success in a client–vendor relationship: (i) shared goals; (ii) shared culture; (iii) shared process; (iv) shared responsibility; and (v) trust. With regard to tool support, Niazi et al. [31] carried out a systematic literature review and a survey-based empirical study to identify the challenges of the existing tools used in GSD projects. The top-ranked challenges in the systematic literature review were the ‘inappropriate use of synchronous and asynchronous communication tools’ and ‘difficulties in adopting and learning to use the existing tools’. The topranked challenges in the questionnaire-based empirical study were the ‘lack of awareness of existing tools used in GSD projects’ and the ‘lack of support for collaboration and group decision making’. Their conclusion is that GSD organisations should address the challenges of the tools commonly used in GSD projects. Experiments addressing RE education in GSD have also been reported in literature [32]. Damian et al. [33] described a framework for the teaching of RE in GSD that was used in a collaboration involving three universities in different locations, time zones and cultures: Canada, Australia and Italy. The students from the three locations played the roles of client and developer and experienced the iterative development of a requirements specification in global projects. The course emphasised the learning of requirements management activities in frequent synchronous computer-mediated client–developer relationships and created a GSD environment with significant time zone and language differences. One project outcome was a software requirements specification as a negotiated software contract between the software group and outsourcing company. The shared understanding of required software features was developed through a series of scheduled requirements elicitation, analysis, inspection, negotiation, prototype design and evaluation activities. The success of each software project relied on proper client–developer communication. In the same context, Gotel et al. [34] reported a GSD project in which students from three countries (USA, Cambodia and India) worked together with the intention of developing a software system to be deployed in operation. The focus was on scaling up, hence the need for both local and global integration, along with an emphasis on requirements and testing for quality. Copyright © 2016 John Wiley & Sons, Ltd.


208


With regard to the processes, activities and tasks to be applied to enable systems and software to be constructed from reusable assets, the Institute of Electrical and Electronics Engineers (IEEE) Standard 1517 [35] provides a common framework for the systematic practice of reuse. Nevertheless, even though the reuse topic has attracted the interest of standardisation bodies, to the best of our knowledge, no empirical study has been carried out to highlight the impact of requirements reuse on the RE process in GSD environments. Vavassori and da Silva [36] have studied the reuse of requirements specifications, concluding that it leads to a greater reuse of other artefacts, but their experience was not conducted in a GSD environment. They presented an approach for requirements reuse, supported by a tool, that was evaluated in a traditional, co-located setting using a quasiexperiment with university students. Their proposal is based on three pillars: (i) requirements writing patterns with which to structure knowledge in order to assist reuse; (ii) a patterns catalogue providing a mechanism to facilitate the selection of a pattern; and (iii) traceability to identify new requirements from a reused requirement. The experiment involved 24 students in the SE discipline on an undergraduate degree in computer science, the content of which included RE. Their results indicated that their proposal makes requirements elicitation and specification activities about 40% more efficient and effective than without the support of their approach. An evaluation was also conducted using the goal/question/metric method that indicated that in the view of six experts, the approach assists in requirements elicitation and specification activities.

3. EXPERIMENTAL INTERVENTIONS: REQUIREMENTS SPECIFICATION TECHNIQUES This section describes the experimental setting in which the study took place. This work has been carried out in the domain of software internationalisation (i18n). I18n is the process of designing software applications in such a way that they can be adapted to various languages and regions without the need for engineering changes [37]. Two strategies for requirements specification were applied in the context of i18n: specification from source documents (Section 3.1) and specification from catalogues of reusable requirements (Section 3.2). In other words, all the participants had to (i) elicit a set of requirements directly from a formal document (non-reuse task) and (ii) reuse a set of requirements related to a particular topic using a given requirements catalogue (reuse task). 3.1. Specification of requirements from source documents To the best of our knowledge, there is no specific standard for i18n as regards software. The related information is scattered about in around more than 20 software specifications and standards. Indeed, requirements may be contained in many sources—which must be identified—and exist in a variety of formats [38]. The identification of the requirements sources, including documentation, is a typical activity in the RE process [39]. The participants in the study described in this paper had to elicit and specify software requirements from i18n information included in a standard—source document— with which they were provided. For example: • ISO 9241-151, Section 7.2.9.7: If a Web user interface is automatically adapted, based on, for example, user profiles or behaviour monitoring, it should be possible for the user to explicitly switch off the automatic adaptation or switch to another user profile provided they are authorised to do so. The aforementioned text can be used to specify the following requirement: R1. The system shall allow users to switch off the automatic adaptation or switch to another user profile provided they are authorised to do so. One of the main objectives of the application of RE processes is the achievement of quality requirements documentation. Requirements are usually specified in unconstrained NL, although they may be supplemented with other elements (e.g. tables, figures and mathematical expressions). Individual requirements must be unambiguous, complete, consistent and verifiable, among other characteristics [40]. For example: Copyright © 2016 John Wiley & Sons, Ltd.



209

R1. The system shall make it possible to ensure that the users are unaware of some information. R2. The system shall provide an individualised system-adapted navigation structure which can make language, display, control and content information invisible to users at the location at which they expect it to be. R1 does not have sufficient quality, given that a statement of this type is incomplete and ambiguous. R2, however, rewrites R1 in order to avoid common flaws and problems in the specification of requirements. 3.2. Specification of requirements from reusable requirements catalogues Requirements reuse improves quality and cycle time in software development [41]. Two common practices for reuse can be considered, namely, developing for reuse and developing with reuse [42]. The first form consists of the identification of reusable knowledge units, their concise representation in an abstract manner and their storage in a knowledge base. The second form implies searching for reusable knowledge and modifying it, thus allowing it to fit into new situations and enabling reusable knowledge to be combined with the knowledge obtained for the new project. It is obvious that all the requirements that constitute a product cannot be reused in all projects, given that new products are the result of new requirements. However, even in these cases, some of the requirements of previously developed products might be the same as those needed in new products, thus making them candidates for reuse. What is more, software development companies normally focus on a limited variety of domains—including ‘vertical’ application domains, such as insurance or banking, and ‘horizontal’ application domains, such as security or personal data protection—in order to take advantage of their knowledge and previous work [43]. When reuse can be applied, a set of requirements can be reused, then adopted without the need for change or reconfiguration, although there are usually also new requirements that are specific to the product [44]. The process model with reuse presented in Figure 1 is an adaptation of the spiral process model proposed by Kotonya and Sommerville [9]. The extended process model draws requirements from a repository of reusable requirements catalogues, and RE activities are consequently adapted for reuse: • Elicitation (with reuse): In the meetings with the other stakeholders, the analysts gather the informal and specific requirements of the current problem domain. These requirements might not initially be in the repository—they are specific to the problem in hand—or they can be identified from one of the catalogues relevant to the problem domain. In the second case, our approach for reuse consists of providing the analysts with the requirements specification document

Figure 1. Reuse-based, spiral model for Requirements Engineering. Copyright © 2016 John Wiley & Sons, Ltd.


210


templates—with the requirements filled in—which are all stored in the repository. The analysts will reuse those requirements that are suitable for the specific application that is being developed by instantiating them when needed. • Analysis and negotiation (with reuse): As a result of the analysis and negotiation, it will probably be necessary to go back to the repository with the purpose of exchanging one previously selected requirement for another less costly one. The negotiation could therefore lead to the removal of some requirements that are replaced with others. • Documentation or specification (with reuse): This activity includes the particularisation of the requirements from the repository, and the new requirements not previously available in any catalogue, to the needs of the current project. • Validation: The draft requirements specification documents are validated by the stakeholders. These documents are used as a starting point to carry out new iterations of the RE process in order to further refine the requirements. In addition, this activity contributes to the for reuse process, because the repository should not be considered as a final static product but rather as an evolutionary one to be enriched with new reusable requirements. It is therefore expected that the requirements in the repository will be modified in order to improve their quality, thus benefiting new projects in which they may be reused in the future. In order to facilitate dealing with i18n in a rigorous and complete manner, we have collected the existing knowledge on i18n in a requirements catalogue [21, 22], which is named Internationalisation CATalogue (I-CAT). This catalogue was created as follows: once the sources had been identified (Section 3.1), we selected the corresponding text parts related to i18n and then used those parts as a starting point to generate the i18n requirements by correctly rephrasing them according to the style, good properties and structure expected for quality software requirements [43, 44]. The I-CAT requirements, their attributes and their traceability relationships are reusable, thus facilitating both reuse and specification tasks. We provided the participants in this study with the I-CAT catalogue and asked them to apply the techniques described below (i.e. parameterised requirements and traceability relationships) in order to specify software requirements. The concept of a parameterised requirement (e.g. [45, 46]) is particularly important in a reuse process, given that parameterised requirements promote reuse by factoring specific details of the system’s definition as parameters—parts that must be adapted to each application or system—thus allowing requirements engineers to specify requirements with a higher level of flexibility [43]. Furthermore, parameters allow us to specify variation points in the requirements specification (i.e. when it is necessary to choose between various alternatives in order to configure a specific product). For example: • R1. The tool shall allow the user to create a Personal Needs and Preferences (PNP) description through [PNP description means]. The parameter is specified between square brackets: [PNP description means].When R1 is chosen for reuse, the parameter can be instantiated using one of the following values: (1) an interactive form or (2) a configuration file. Traceability relationships that enable different links to be established between requirements can be used to monitor and evaluate the impact of changes on the requirements specification [47]. Furthermore, reusable requirements are not usually isolated in the catalogues but are rather related to each other in several ways by means of traceability relationships. The traceability model is typically a key component of any RE method. Ramesh and Jarke [48] proposed a comprehensive taxonomy of traceability, but (i) the kind of dependencies that should be traced to support reuse is not indicated [49] and (ii) the taxonomy as a whole is very rich and may be too complex in practice, when a reduced set of traces is sometimes more appropriate [43]. A simple set of traceability relationships was therefore adopted in this experiment, which is described in the following: • Parent–child: relationship describing a more general requirement via a sequence of more specific requirements. For example: R1. If the user has not defined a profile, then the tool shall assign a predefined user profile to the user. Copyright © 2016 John Wiley & Sons, Ltd.



211

R1.1. The tool shall have predefined user profiles or group profiles. R1.2. The tool shall allow the person who created the user or group to include predefined profiles for them. R1 Parent–child R1.1, R1.2. • Requires: directional dependency relationship between two requirements. R1 Requires R2 signifies that the reuse of R1 necessarily involves the reuse of R2. The reuse of R2 is compulsory. For example: R1. The tool shall allow the user to create a Personal Needs and Preferences (PNP) description through [PNP description means]. R2. Once a person has a PNP, (s)he should be able to change his/her PNP statement as needed. R1 Requires R2. • RelatedTo: A bidirectional dependency relationship between two requirements. R1 is RelatedTo R2 if R2 refines or supplements R1 in some way, signifying that when R1 is reused then R2 should also be considered for reuse. The reuse of R2 is optional. For example: R1. Once a person has a PNP, (s)he should be able to change his/her PNP statement as needed. R2. Once a person has a PNP, (s)he should be able to expand his/her PNP statement as needed. R1 RelatedTo R2. With regard to the process of specifying requirements from a reusable requirements catalogue, reusing the requirements selected in the catalogue involves the resolution of the variation points found in them: (i) instantiation of the parameters of the parameterised requirements with values

Table I. Software requirements specification structure. 1. Introduction 1.1 Purpose 1.2 Scope 1.3 Definitions, acronyms, and abbreviations 1.4 References 1.5 Overview 2. Overall description 2.1 Product perspective 2.2 Product functions 2.3 User characteristics 2.4 Constraints 2.5 Assumptions and dependencies 3. Specific requirements 3.1 External interface requirements 3.1.1 User interfaces 3.1.2 Hardware interfaces 3.1.3 Software interfaces 3.1.4 Communications interfaces 3.2 Functional requirements 3.3 Performance requirements 3.4 Design constraints 3.5 Software system attributes 3.5.1 Reliability 3.5.2 Availability 3.5.3 Security 3.5.4 Maintenance 3.5.5 Portability 3.5.6 Internationalisation 3.6 Other requirements Appendices Index

Copyright © 2016 John Wiley & Sons, Ltd.


212


appropriate to the current project; (ii) resolution of RelatedTo traces, which are optional; and (iii) resolution of Requires and Parent–child traces, which should normally be included. The general structure of the i18n catalogue is displayed in Table I. It follows the recommendations of the International Organisation for Standardisation (ISO)/International Electrotechnical Commission (IEC)/IEEE 29148 [40] for requirements documents. According to this standard, and in addition to the regular requirement types provided in its templates, additional requirements may be included in a separate section at the end of the software requirements specification. In order to facilitate the search for and reuse of i18n requirements, along with their possible integration into current instantiated requirements documents for a project already underway, a new subsection was added (i.e. 3.5.6 ‘Internationalisation’) that is specific to i18n requirements. This subsection provides a description of how the i18n requirements were organised following the main related standards, such as ISO/IEC 24751 [50] (Table II). When ordering the i18n requirements into subsections, an attempt was made to, as far as possible, follow the minimal coupling and maximal cohesion principles. We therefore attempted to reduce the interdependency degree (low coupling) and increase the existing conceptual and functional relationships (high cohesion) among requirements. 3.3. Comparison of the techniques Once the requirements specification techniques involved in the experiment have been presented, it would be useful to compare them systematically. The application of reusable requirements catalogues to the development of software products implies changes in the basic RE process model (Figure 1). The differences between the reuse-based process model and the general process model are not drastic but may still lead to some degree of process overload. For its part, the reuse ability is fully supported—and encouraged—when the catalogue-based approach is applied. Reuse can certainly take place when no specific techniques are applied, at least to some extent. Nevertheless, in this case, the benefits will be largely decimated, especially because of the lack of systematisation [51]. The presence of a requirements repository arranged into catalogues and the internal organisation of requirements within catalogues provide a starting point for the realisation of effective searches that deliver accurate results [44]. Traceability is highly desirable when building a requirements specification, although it is sometimes insufficient in industrial practice [52]. For its part, requirements reuse emphasises the usage of requirements traceability [53]. Furthermore, it is highly Table II. 18n requirements specific subsection structure. 3.5.6 Internationalisation 3.5.6.1 General aspects 3.5.6.2 Access for all (AfA) personal needs and preferences (PNP) for digital delivery 3.5.6.2.1 Creating and maintenance of the PNP statement 3.5.6.2.2 PNP structure 3.5.6.2.3 PNP components 3.5.6.3 AfA digital resource description (DRD) 3.5.6.3.1 Creation of DRD 3.5.6.3.2 Identify and process of DRD 3.5.6.3.3 DRD attribute 3.5.6.4 User interfaces design 3.5.6.4.1 Individualisation and user adaptation 3.5.6.4.2 Designing for cultural diversity and multilingual use 3.5.6.5 18n Data 3.5.6.5.1 General information 3.5.6.5.2 Data coding 3.5.6.5.3 Date value elements 3.5.6.5.4 Currency and measurement-type values 3.5.6.6 Vocabulary 3.5.6.7 Others 3.5.6.7.1 Providing help 3.5.6.7.2 Privacy and business policies 3.5.6.7.3 Resources




213

advisable to use specific tool support, given that an RE tool helps mitigate the complexity of the processes involved in RE [54]. Nevertheless, all-purpose documentation tools (i.e. productivity software, especially word processors) are still the most widely used solutions when managing requirements: 40–50% of companies carry out the RE process with general document software [55]. Without automated support, the application of any RE method concerning reuse becomes slow, cumbersome and thus impractical [43]. In addition, the use of tools that support reuse has many more benefits than informal reuse [36]. With regard to the availability of tools, few tools provide the requirements reuse process with specific support, and those that exist are usually non-commercial tools or plug-ins for major tools, whereas there are many tools that meet the needs of a traditional RE process [18, 56]. The primary role of domain knowledge is to bridge the gap between requirements and specifications during the process of refinement of requirements [57]. The previous domain knowledge required to develop a reusable requirements catalogue should be higher than the previous domain knowledge needed to generate the specification if the catalogue already exists [58], given that a knowledge acquisition effort has already been carried out to encapsulate part of the domain knowledge and make it available through the reusable asset [59]. When the specification is created in the context of a traditional RE process, we could expect the need for previous domain knowledge to be located at an intermediate point. Such domain knowledge can be acquired during the process [60]. The effort needed to apply the RE process and techniques can be described as low if a reuse-based approach is carried out and a catalogue from which the requirements can be reused is available or high if a catalogue is not available and has to be created. The effort would, however, be medium when specifying requirements from source documents. This is because in the latter case, it would by definition always be necessary to start from scratch. This is more costly than reusing requirements or specifications [17, 61], but not as hard as building a new catalogue of reusable requirements that requires an initial creation and continuous maintenance effort [62]. Specific training is a critical success factor when implementing RE processes [63] and is highly specialised as regards requirements reuse [18, 64]. Finally, RE processes are included in the most common software engineering practices [8]. For its part, the adoption of requirements reuse approaches by companies is still limited [65]. Table III provides a summary and synthesis of the commonalities and differences between the two requirements specification approaches in accordance with the descriptions provided earlier.

4. EMPIRICAL STUDY This section provides detailed information on the experiment designed and conducted during the autumn term of 2012. Firstly, the goal of the experiment is presented in Section 4.1. Secondly, the participants taking part in it are described (Section 4.2). Thirdly, the design of the empirical study is Table III. Comparison between requirements specification techniques. Specification of requirements from source documents

Specification of requirements from reusable requirements catalogues

Process Reuse Structuring and search Traceability Tool support Tool availability Need for previous domain knowledge Effort

No overload Not directly supported Not directly supported Desirable Desirable Many tools Medium

Training Widespread adoption

Specific training required Yes

Minor overload Supported Supported Required Required Few tools Lower (a catalogue is available); Higher (a catalogue is not available) Lower (a catalogue is available); Higher (a catalogue is not available) Very specific training required No

Medium (always from scratch)



214


presented (Section 4.3). Finally, the indicators defined to help us determine the outcome of this experience are shown in Section 4.4. 4.1. Aim The aim of the experiment was to compare the effects of specifying requirements from a reusable requirements catalogue with those of traditional requirements specification from source documents in both co-located and distributed environments. Our purpose was to study whether or not these strategies lead to a similar outcome. The hypothesis was that • There are no differences between co-located and distributed teams when specifying requirements from source documents and from reusable requirements catalogues in terms of: (a) Effectiveness. (b) Productivity. (c) Difficulty. (d) Speed. (e) Quality. (f) Understanding. 4.2. Participants A total of 31 computer science and engineering students in either their last years of university or their first years of postgraduate studies, who had similar training and experience in RE and with diverse ages, gender, educational backgrounds and average marks in the previous academic year, gave their verbal consent to participate in the study. They had a good command of technical English, which was necessary in order to be able to follow the instructions correctly and fit in with the international focus of the experiment. 4.3. Design The task statement, which was originally defined for this experiment, is shown in Table IV. This document provided the participants with guidelines on the context, goal, task, materials, planning and evaluation of the experiment. The experimental tasks were carried out by teams of two people. There were three participation modalities: (i) global (GLO), one student from the University of Murcia (UMU) in Murcia (Spain) was paired up with one student from the Mohammed V University (UMV) in Rabat (Morocco); (ii) co-located in Murcia (CLM), two students from the UMU were paired up; and (iii) co-located in Rabat (CLR), two students from the UMV were paired up. The working language was English, although informal communication in the native language was allowed for participants in the co-located modalities. Note that this imitates a real software development scenario, in which team members assigned to the same site are generally expected to communicate with each other in their mother tongue. All the participants had to present the final results in English. The study employed a randomised controlled design. After recruitment, 31 students were randomly divided into groups. There were 15 participants from the UMV and 16 participants from the UMU. We defined 14 teams of two students and one team of three students because there was an odd number of participants. Seven teams (15 students) were assigned to the co-located modality (CLM or CLR), while eight teams (16 students) were assigned to the global modality (GLO). In the co-located modality, four teams (eight students) were appointed to CLM groups, and three teams (seven students) were appointed to CLR groups. A brief summary of this information is provided in Table V. The period assigned to the execution of the experiment was 2 weeks. It started on 26 November 2012, and this coincided with the date on which the task statement and the rest of the documentation were handed out, and the procedure of the experiment was explained to the participants. The estimated effective time that they had to devote to completing the task was in the order of hours. However, during the whole 2-week period, the participants needed to meet for the first time and jointly agree on Copyright © 2016 John Wiley & Sons, Ltd.



215

Table IV. Task statement. Context. Company1 has experience in selling one certain kind of products in its local market and wishes to improve its results by expanding its business to new customers. To that end, the people in charge of Company1 believe that the best way to accomplish this objective is by selling their products on the Internet. In addition, as part of its strategy to expand to other markets, the company wishes to encourage and favour sales to markets other than the national market. Goal. The final goal of this task is to elicit software requirements related to software i18n, which can serve Company1 to develop a website with which to sell products on the Internet. During the execution of this task, traditional and reuse-based RE techniques will be applied. Task. The task will be performed in teams of two people. These teams have to: 1. Elicit requirements related to a specific software i18n topic using an i18n standard (ISO/IEC 24751, ISO 9241-151, W3C or CWA 14928X) as the starting point. Source documents will be analysed to gather requirements during this process. 2. Reuse requirements related to a specific software i18n topic using a predefined i18n requirements catalogue as a starting point. This reusable requirements catalogue is based on standards that contain i18n information and guidelines (e.g. ISO/IEC 24751, ISO 9241-151, W3C, CWA 14928X). Traceability relationships between requirements and parameterised requirements will be used during this process. Provided materials. The participants will be provided with the following materials: 1. Task statement (this document). 2.Specific assignment of each team. 3.I18n standards (ISO/IEC 24751, ISO 9241-151, W3C and CWA 14928X). 4. Generic i18n requirements catalogue in the form of a software requirements specification (SRS). 5.Traceability matrix including the relationships between the requirements in the catalogue. 6.Summary of the foundations of the techniques to be applied. 7.Questionnaire. Planning. The stipulated period for completing the task is 2 weeks, starting from the date on which the task statement is handed in. Evaluation. In order to proceed to the evaluation of the task, the team members have to submit the task results and the questionnaire answers within the stipulated period. All outcomes of the experiment must be written in English. Glossary. Terms are listed in alphabetical order: Elicitation: the task of identifying the various types of requirements from various sources including standards, project documentation, interviews, etc. Parameterised requirement: requirement that contains parts which have to be adapted to each system and that have to be instantiated when the requirement is reused. Requirement: a statement that identifies a necessary attribute, capability, characteristic or quality of a system for it to have value and utility for an user. Requirements catalogue: a set of generic and reusable requirements. Requirements traceability: documentation of the relationships between requirements to facilitate the overall quality of the product under development, the understanding of the product and its artefacts, and the ability to managechange.

Table V. Summary of participants and teams per participation modality.

Participants Teams

GLO

CLM

CLR

Total

16 8

8 4

7 3

31 15

GLO, global; CLM, co-located in Murcia; CLR, co-located in Rabat.

face-to-face or virtual work meetings and schedule them accordingly, depending on personal time constraints. The participants were encouraged to interact, collaborate and work together to complete the tasks. They were also stimulated to discuss and agree with their teammates on each aspect of the tasks (i.e. agree on meetings, help each other comprehend the basis of the techniques, cooperate in problem solving activities, etc.). The participants included in a co-located team (CLM or CLR) had to set up at least one face-to-face meeting with their teammates in order to ensure that they take advantage of being co-located, as compared with global teams (GLO), which did not have this opportunity— again, trying to mimic a real software development context. They were also asked to adhere strictly Copyright © 2016 John Wiley & Sons, Ltd.


216


to our instructions and recommendations because this was key to the correctness of the experimental procedure. 4.4. Outcome instruments and measures A total of six variables were studied, and each one was analysed separately for each modality of participation, namely, co-located and global. We can classify our variables into one of the following two types [66]: (i) performance-based variables, which measure how well subjects are able to use a requirements specification method (effectiveness, productivity); and (ii) perception-based variables, which measure how effective subjects believe a requirements specification method is (difficulty, speed, quality and understanding). These dimensions were determined using the method evaluation model [67], which was adapted with the purpose of evaluating the students’ skills and knowledge as regards the application of requirements specification techniques. The method evaluation model is a theoretical model with which to evaluate information systems design methods. It incorporates both aspects of method success, namely, actual performance and likelihood of acceptance in practice. Figure 2 shows the model defined to evaluate the requirements specification techniques in this experiment. The researchers attained effectiveness by objectively assessing the output of the task execution (1-Very good, 2-Good, 3-Fair and 4-Poor). This evaluation was carried out by the first author of this paper, because the existence of different raters can result in disagreements about measurement results of the same object [68, 69]. Deviations and inconsistencies (e.g. variations in the procedures used to carry out the experiment, interpreting the results and presenting them) that may be affected by the experimenter’s bias were therefore mitigated. Productivity is defined as output divided by the effort required to produce that output [70]. However, the notion of output is not straightforward for software [71]. In our analysis, the measure collected is requirements per hour [72], which reflects . The the amount of requirements specified per hour per team, that is, Productivity ¼ requirements hour remaining variables represent the participants’ subjective perceptions, which were gathered using a 5-point Likert scale. Difficulty is a measure of how hard it is to apply the techniques (1-Easy, 5-Difficult). Speed assesses the rapidness of applying the techniques (1-Quick, 5-Slow). Quality reflects the perceived quality of the requirements obtained by using the techniques (1-High, 5-Low). Finally, understanding is a measure of how comprehensible the resulting requirements are (1-Good, 5-Bad). Productivity, difficulty and speed are therefore focused on the process, and effectiveness, quality and understanding provide insights into the product (see the process-focused and productfocused sets in Figure 2). Note that the scale is inverted in some variables in order to facilitate the interpretation of results, so that low values indicate positive results throughout all the variables of the study.

Figure 2. Theoretical evaluation model. Copyright © 2016 John Wiley & Sons, Ltd.



217

Table VI. Questionnaire Questions concerning the non-reuse task Q1.1. Time devoted to the task (hours). Q1.2. It is easy to elicit the requirements of a new project from the i18n source document. Q1.3. The requirements of a new project can be quickly elicited from the i18n source document. Q1.4. The quality of the requirements obtained by eliciting from the i18n source document is high. Q1.5. The requirements obtained by eliciting from the i18n source document are understandable. Q1.6. Means of communication used (multiple options allowed): Face-to-face meetings. Asynchronous (email). Synchronous (chat, audio/video conference). Other (please specify). Q1.7. Problems encountered (if any). Questions concerning the reuse task Q2.1. Time devoted to the task (hours). Q2.2. It is easy to reuse the requirements of a new project from the i18n requirements catalogue. Q2.3. The requirements of a new project can be quickly reused from the i18n requirements catalogue. Q2.4. The quality of the requirements obtained by reusing from the i18n requirements catalogue is high. Q2.5. The requirements obtained by reusing from the i18n requirements catalogue are understandable. Q2.6. Means of communication used (multiple options allowed): Face-to-face meetings. Asynchronous (email). Synchronous (chat, audio/video conference). Other (please specify). Q2.7. Problems encountered (if any).

All the participants in the experiment were requested to answer the questionnaire presented in Table VI after finishing the experimental task. QX.1 was concerned with the time spent on the task. We used a 5-point Likert scale in QX.2–QX.5, in which the choices of answers ranged from 1 (‘Completely Agree’) to 5 (‘Completely Disagree’). QX.6 allowed us to check the means utilised by the participants during the study to communicate with their teammates. Finally, QX.7 was an optional question to gather feedback on the questionnaire or the experiment in general.

5. RESULTS The results of the experiment are discussed in the following sections. Section 5.1 provides a summary of the data using descriptive statistics, while Section 5.2 shows an analysis of data by means of inferential statistics. The relationship between the raw data and its interpretation are then discussed in Section 5.3. Finally, the threats to the validity of the study are examined in Section 5.4. 5.1. Descriptive statistics Most of the participants were male (61.3%, n = 19). The participants were aged between 21 and 25, with most of them belonging to the 22 age group. The final participation rate of the experiment was 93.55% (29 out of 31 participants), given that one CLR team made up of two students was discovered to be an outlier in the sample and was eventually discarded. A summary of the statistics is presented in Table VII in order to describe the variables of the study. Measures of central tendency (arithmetic mean) and dispersion (standard deviation) are included. The cells are coloured differently depending on their value (light and dark grey are used to highlight a better or a worse value, respectively). Figures 3–8 depict the variables obtained for co-located and global teams from a descriptive point of view. Stacked bar graphs are used to show the percentage different sub-groups contribute to each separate category. For example, in the case in Figure 5(a), the bars representing the individual categories (i.e. non-reuse task and reuse task) are all the same size— which corresponds to the value of 100%—and the relative contribution of the sub-groups (i.e. 1-Easy, 2-Somewhat easy, 3-Neutral, 4-Somewhat difficult and 5-Difficult) is different for each Copyright © 2016 John Wiley & Sons, Ltd.


218


Table VII. Descriptive statistics.

N, sample size; SD, standard deviation. Co-located: N = 13; Global: N = 16.

Figure 3. Effectiveness (1-Very good, 4-Poor). GLO, global; CLM, co-located in Murcia; CLR, co-located in Rabat.

Figure 4. Productivity (requirements per hour per team). GLO, global; CLM, co-located in Murcia; CLR, co-located in Rabat.

Figure 5. Difficulty (1-Easy, 5-Difficult). GLO, global; CLM, co-located in Murcia; CLR, co-located in Rabat. Copyright © 2016 John Wiley & Sons, Ltd.



219

Figure 6. Speed (1-Quick, 5-Slow). GLO, global; CLM, co-located in Murcia; CLR, co-located in Rabat.

Figure 7. Quality (1-High, 5-Low). GLO, global; CLM, co-located in Murcia; CLR, co-located in Rabat.

Figure 8. Understanding (1-Good, 5-Bad). GLO, global; CLM, co-located in Murcia; CLR, co-located in Rabat.

category. This signifies that in the case of the non-reuse task category, 2-Somewhat easy has the highest importance—more than 45%—followed by 1-Easy—more than 30%—and 3-Neutral—more than 20%—. 4-Somewhat difficult and 5-Difficult make no contribution, or 0%. The same applies to the interpretation of the other stacked bar graphs. Furthermore, the boxplot (Figure 4) was only used to represent the variable productivity, given its quantitative nature. The analysis of the descriptive data is presented next. From the perspective of the students’ distribution, we can state that, according to the effectiveness measure, the global teams performed slightly better than the co-located teams in both the non-reuse and reuse tasks (Figure 3). However, the productivity was clearly lower in the case of the global teams (Figure 4). With regard to the specification of requirements from source documents, the participants’ subjective perception of the difficulty, speed and understanding (see Figures 5, 6 and 8, respectively) was a little better—easier, quicker and more understandable—in the case of the co-located teams. Perceived quality (Figure 7) was, however, better—higher quality—in the case of the global teams. On the other hand, the participants’ subjective perception of the task of specifying requirements from the catalogue of reusable requirements was generally better in the case of the global teams, with the exception of Copyright © 2016 John Wiley & Sons, Ltd.


220


quality, which was better perceived by the co-located teams. Nevertheless, the differences between them are very slight, particularly in the case of understanding. In summary, depending on geographical dispersion, we found that the students’ perceptions of the two approaches were opposite, although the differences are mostly small or very small. From the point of view of the comparison between the outcome of the requirements specification strategies studied in this paper, our results show that the students obtained better results when applying the traditional approach (i.e. specification of requirements from source documents) than when applying the reuse-based approach. This occurred in the case of all the variables studied. 5.2. Inferential analysis Once the descriptive analysis had been completed, it was deemed useful to gain insights into the relevance of the differences between the non-reuse and reuse tasks and between the co-located and global teams. We realised that statistical inference techniques would allow us to generalise the results of our experiment and draw relevant conclusions. In this respect, parametric tests require certain assumptions about the data to be true, in particular, the variables originating from a normal distribution. This point was analysed by performing the Shapiro–Wilk (W-statistic) formal statistical test. Given that our variables are not normally distributed, a non-parametric test is therefore recommended. Wilcoxon Z matched-pairs signed-rank tests (Z-statistic) were conducted to compare the variables of our theoretical evaluation model (i.e. effectiveness, productivity, difficulty, speed, quality and understanding) in non-reuse and reuse conditions, both in co-located and non-co-located (global) modalities. Numeric values of the Likert scale have been used where appropriate to convert ordinal variables into scale variables in order to apply this test. The statistical tests carried out in this study are summarised in Table VIII. There were significant differences (Z = 2.320, p = 0.020) in productivity (co-located modality) between the scores for non-reuse (M = 26.54, SD = 23.57) and reuse (M = 13.14, SD = 7.91) conditions. This suggests that the strategy based on reusing requirements from a catalogue really does have an effect on the productivity of the requirements specification process. In other words, when co-located participants specify requirements by using the requirements reuse approach, the productivity decreases with regard to the specification of requirements from source documents. There were also significant differences (Z = 2.124, p = 0.034) between the scores for non-reuse (M = 1.92, SD = 0.76) and reuse (M = 2.62, SD = 1.04) conditions in the case of difficulty, again in the co-located modality. The strategy based on the specification of requirements from a catalogue of reusable requirements has an effect on the subjective perception of the difficulty of the process. Our results specifically suggest that when co-located students specify requirements by using the requirements reuse approach, the perceived difficulty increases with regard to the specification of Table VIII. Hypothesis testing: Wilcoxon Z matched-pairs signed-rank test. Variable Effectiveness Productivity Difficulty Speed Quality Understanding

Modality Co-located Global Co-located Global Co-located Global Co-located Global Co-located Global Co-located Global

Statistic Z = 1.261 Z = 1.190 Z = 2.320 Z = 0.982 Z = 2.124 Z = 1.658 Z = 1.145 Z = 0.273 Z = 0.183 Z = 0.741 Z = 0.791 Z = 0.514

p-value p= p= p= p= p= p= p= p= p= p= p= p=

0.207 0.234 0.020 0.326 0.034 0.097 0.252 0.785 0.855 0.459 0.429 0.607

Significance level: 0.05. Copyright © 2016 John Wiley & Sons, Ltd.



221

requirements from source documents. When the participants are distributed, this result is not significant but is noticeable (Z = 1.658, p = 0.097) because the p-value is close to 0.05. Our findings showed that the requirements specification strategy (treatment) did not elicit a statistically significant change in the rest of variables (effectiveness, speed, quality and understanding) and distribution modalities (co-located and global), and we cannot therefore reject the null hypothesis (see Section 4.1) at the 0.05 level of significance. 5.3. Discussion According to the descriptive analysis presented in Section 5.1, our results suggest that the participants applied the traditional approach (i.e. specification of requirements from source documents and nonreuse) slightly better than the reuse-based approach when comparing both strategies for requirements specification. However, our inferential analysis (Section 5.2) has not uncovered significant differences between the non-reuse and reuse approaches in general. Our results support the idea that the students achieve a similar degree of success as regards requirements specification using both strategies, regardless of whether the participants are co-located or distributed, in the case of effectiveness, speed, quality and understanding. In the case of productivity and difficulty, the distributed participants’ ability to specify requirements is also similar when using both strategies. On the contrary, when the participants are co-located and they specify requirements from source documents, their productivity is higher and the perceived difficulty lower than when they specify requirements from a reusable requirements catalogue. Productivity and difficulty are related variables (Figure 2), which is therefore reflected in the results of the study: the greater the difficulty, the lower the productivity. The lack of automated RE tool support can explain this situation and provide us with a plausible interpretation for this result. The tasks presented in this paper were manually applied by the participants in the experiment. While the specification of NL requirements from source documents is a straightforward process that does not require the mastering of complex techniques, the specification of requirements from a catalogue without automated RE tool support signified that the participants had to search the catalogue to find the requirements, check the associated traceability matrix, determine the related requirements and instantiate the parameters of the requirements. Cybulski [64] affirms that experience and knowledge of general software development principles are not a substitute for specific training in methods and techniques peculiar to the reuse activities. Although reusable requirements catalogues can increase productivity in software development [43], a period of time is required to learn the reuse techniques [64] and consequently the catalogue-based requirements specification task. Moreover, all of these activities were performed by hand, which could explain the perception of increased difficulty with regard to the non-reuse task, and the productivity accomplished could, in turn, have been impacted by the level of difficulty. In fact, the reuse-based techniques may be slow and cumbersome in the absence of specific automated RE tool support [43]. In spite of these considerations, it might be educationally interesting to focus on the process of resolution of the variation points (see Section 3.2) in order to ensure that the students learn how it works, even though the use of a tool is the solution that maximises productivity. We have shown that the effectiveness was slightly better in the case of the global teams than in the case of the co-located teams in both the non-reuse and reuse tasks. This can be explained by the fact that the productivity was lower in the case of the global teams (see the model presented in Figure 2). More time devoted to requirements normally implies better results [3, 4]. In this respect, and considering the specification of requirements from source documents (non-reuse), the students assigned to co-located teams had a slightly better subjective perception as regards difficulty, speed and understanding (see the light grey cells in column 1 in Table VII) and a worse subjective perception as regards quality (see the dark grey cell in column 1 in Table VII) than those assigned to global teams (see column 2 in Table VII). Indeed, co-located participants were able to perform the task in less time and with less effort (higher productivity) than distributed participants, while the distributed participants in turn obtained better results (higher effectiveness) than co-located participants. The language and cultural differences for geographically distributed students may have caused them to be more analytical and cautious, thus resulting in improved effectiveness and worsened productivity. In the case of the specification of requirements from the catalogue of Copyright © 2016 John Wiley & Sons, Ltd.


222


reusable requirements, the results concerning the perception-based variables are inverted. Global teams are confronted with many challenges [14] that do not impact on co-located teams with the same intensity. This situation could explain the fact that distributed teams prefer to rely on a catalogue, feel more comfortable using it or are of the opinion that they may benefit more from its use than colocated teams (see columns 3 and 4 in Table VII). Nevertheless, the actual differences between the subjective perceptions of the co-located and global participants in this task are very slight, and thus inconclusive. The effectiveness of the process is between Very good and Good in the case of the specification from source documents, whereas it is Good in the case of the specification from a reusable requirements catalogue. In addition, the difficulty, speed, quality and understanding scores are always below the mid value (3-Neutral) of the 5-point Likert scale, signifying that the students’ subjective perception of both requirements specification strategies is positive. The students’ productivity was also satisfactory, given that the requirements generation rate was between 26.54 and 8.82 requirements per hour per team. Because the teams were made up of two participants—except one team that was made up of three students—the average generation rate was approximately between 13.27 and 4.41 requirements per hour per person. This is consistent with the results obtained by other researchers using different proposals. Mavin and Maiden [73] reported an average generation rate of 10 requirements per hour, although considering that a minimum of two stakeholders were involved, the typical generation rate was below five requirements per hour of stakeholder involvement. Other approaches had generation rates of 8.2, 12.7 and 17.7 requirements per hour of stakeholder involvement [74]. Seyff et al. [72] reported an average generation rate of 21.8 requirements per hour of stakeholder participation. Such a high generation rate is, however, justified given that it was obtained by experienced analysts using a tool-supported contextual method that improves guidance and support. Our students might therefore have obtained an even higher generation rate if they had used an automated RE tool support and/or they had been provided with more specific training, as explained earlier in this section. 5.4. Limitations The threats to the validity [75] of this research are discussed next. Conclusion validity is the degree to which conclusions about the relationship among variables are correct. We checked the prerequisites of the statistical tests when analysing the results. Given that our data did not pass the normality tests, we carried out the statistical analysis using the non-parametric Wilcoxon Z matched-pairs signed-rank test. The main limitation of this study was the relatively small data sample because we had a total of 14 teams and 29 individuals. However, other studies that involve a similar number of participants are reported in Section 2. Moreover, the data collected and properties selected allowed us to gain a valuable insight into the relevant but still barely explored issues presented in this paper. The lack of a formal sample size calculation may have influenced the findings. The findings derived from this study must therefore be interpreted with caution, and future research with a larger sample is recommended. Internal validity is concerned with the reliability of the results. In this experiment, threats may be the result of individual causes attributable to both the participants (i.e. fatigue, familiarity with RE, commitment to providing quality feedback and training level) and the researchers (experimenters’ bias, i.e. different scoring criteria between participants). The fatigue effect was limited by ensuring that the students had ample time in which to perform the tasks. The design of the experiment—single-factor experiment design with two treatments and random allocation of participants to teams—should have eliminated the problem of the different levels of familiarity with RE. The students’ engagement in the experiment was high; they were well-motivated to accomplish the task, bearing in mind that this activity was part of a regular module on their course. The participants received specific training before the beginning of the experiment in order to normalise their knowledge of the techniques being applied. We attempted to reduce subjectivity in the assessment of the participants’ work by ensuring that the appraiser had a set of evaluation guidelines (fixed beforehand) at his disposal. Construct validity refers to the degree to which inferences can be made from the measurements. The experimental task was located within a module on a course with a restricted amount of time, Copyright © 2016 John Wiley & Sons, Ltd.



223

thus signifying that it was from necessity limited in its size and complexity. However, in spite of the aforementioned limitations, the task was planned to be as realistic as possible; both the learning interventions and the experiment were aimed at important aspects in RE, GSD and requirements reuse. Furthermore, the measurements used were as objective and aseptic as possible, even though the limitations are principally owing to a possible mismatch between the participants’ understanding of our measurement instrument (i.e. the questionnaire) and the participants’ actual performance and perception of the task. What is more, the participants were fully aware of being observed, and observational techniques always run the risk of changing the process simply by the very observation of it [76]. The participants were, nevertheless, not aware of the study hypotheses, and they were informed that they would not be evaluated on the basis of the measures collected during the experiment—or the scores obtained—but rather on their engagement. External validity. This is concerned with the generalisation of the results beyond the sample. Firstly, there was only 1 h of difference between the team members in the global teams, which means that only physical and socio-cultural distances were appreciable in this nearshore scenario. Secondly, the findings of this study are of interest to the RE field from an educational point of view because the techniques analysed are common in RE and linked to the industry’s needs [43]. Finally, although we are not able to conclude that our results are relevant in any nearshore experimental setting, we believe that our findings are scalable to scenarios in which there are two nodes working together, as long as each node has a representative in charge of internodal communication. Two members per team are sufficient to achieve this goal. This configuration is the most efficient: it requires communication and cooperation between parties, while we maximise the number of teams that take part in the experiment. Likewise, the approach presented in this paper could be used to train both co-located and distributed students in other requirements specification techniques with a few adjustments to the design, instrumentation and materials.

6. CONCLUSIONS At this point, the insights into the comparison of RE techniques can be revisited (Section 3.3) and contrasted with the results of the empirical study (Section 5). There were no significant differences between the scores achieved by the participants in the case of most of the variables in our controlled study, when they specified requirements from source documents and when they specified requirements from a catalogue of reusable requirements. However, there were significant differences between these approaches for software requirements specification in the cases of productivity and difficulty (in both cases in the co-located modality), which is coherent with the relationship between these two variables in our theoretical model (Figure 2). This suggests that the specification of requirements from a reusable catalogue really does have an effect on the productivity and difficulty of the requirements specification process when the participants are co-located. The lack of dedicated tool support when searching the catalogue, and especially as regards managing traceability [54] and variation points [56], has presumably resulted in a worse productivity for the catalogue-based requirements specification approach. Likewise, the pre-existence of a reusable requirements catalogue should have had a positive impact on the perceived difficulty [18], but the participants in this study found the specification of requirements from source documents easier. This finding contributes towards explaining that automated RE tool support is required if the expected benefits of requirements reuse are to be achieved [43], although it is not mandatory from an educational point of view. The results attained are also compatible with the notion that the skills and knowledge that are needed to effectively apply the reuse-based requirements specification techniques are highly specialised and demand sufficient specific training for their acquisition [64]. In other words, the degree of specialisation required to apply catalogue-based requirements reuse properly seems to be higher than in the case of the specification of requirements from source documents. With regard to the general benefits derived from carrying out this experience, the effectiveness was between Very good and Good in the case of the specification of requirements from source documents. In the case of the specification from a reusable requirements catalogue, it was Good. Furthermore, we have shown that the productivity was also satisfactory. The requirements’ generation rate was similar to Copyright © 2016 John Wiley & Sons, Ltd.


224


that reported by other researchers who have investigated this topic. With regard to the perception-based variables, the difficulty, speed, quality and understanding scores were below the mid value (3-Neutral) of the 5-point Likert scale in all cases (co-located and global teams and non-reuse and reuse conditions). The participants’ subjective perception of both requirements specification strategies is thus positive, which is also reported in similar studies by other researchers (e.g. [33, 77]). The topic presented in this paper is relevant for the definition of curriculum guidelines for university courses in SE. New initiatives with which to foster international cooperation between high-quality education institutions are emerging. One of them is the International Excellence Campus for Higher Education and Research of the Region of Murcia (Spain) [78]. This environment offers the opportunity to design educational activities in which international students enrolled on academic degrees at different universities can take part, and these initiatives could particularly be used to promote GSD education. The GSD education curriculum, outcomes and delivery have received considerable attention from both the IEEE and Association for Computing Machinery societies [79]. More specifically, the ability to identify and define requirements and put them into a requirements specification document stands out from the other technical competences required to work in a GSD team [80]. We have shown that different requirements specification approaches can be successfully applied by both co-located and global students. Our findings support the idea that an educational activity such as that presented in this paper can be planned and executed to complement and enrich the requirements reuse topic as part of Software Evolution in the Computer Science Curricula. We postulate that requirements reuse can help the stakeholders to interpret the real meaning of the requirements in a given context by leaving little room for misunderstandings and misconceptions in GSD. In future work, we will carry out a new experiment with computer science and engineering students in which they will be provided not only with the reuse-based requirements specification techniques but also with an RE tool specifically designed to support a reuse-based RE process. We shall analyse the effect on our theoretical evaluation model in both co-located and distributed settings. This issue is particularly relevant given that to the best of our knowledge, there is no empirical evidence regarding the impact of requirements reuse on GSD. ACKNOWLEDGEMENTS

This work is partially supported by the GEODAS-REQ project (Spanish Ministry of Economy and Competitiveness and European Fund for Regional Development, TIN2012-37493-C03-01) and the Mediterranean Office for Youth (grants 2010/045/01 and 2010/045/02). REFERENCES 1. The Standish Group. CHAOS summary 2009 the 10 laws of CHAOS. The Standish Group International, Inc., 2009. 2. Janes A, Remencius T, Sillitti A, Succi G. Managing changes in requirements: an empirical investigation. Journal of Software –Evolution and Process 2013; 25(12):1273–1283. 3. Hofmann HF, Lehner F. Requirements engineering as a success factor in software projects. IEEE Software 2001; 18(4):58–66. 4. Hooks IF, Farry KA. Customer-centered products: creating successful products through smart requirements management. AMACOM: New York, USA, 2001. 5. Sillitti A, Succi G. Requirements engineering for agile methods. Engineering and Managing Software Requirements, Aurum A, Wohlin C (eds.). Springer: Berlin Heidelberg, 2005; 309–326. 6. Cao L, Ramesh B. Agile requirements engineering practices: an empirical study. IEEE Software 2008; 25(1):60–67. 7. Ramesh B, Cao L, Baskerville R. Agile requirements engineering practices and challenges: an empirical study. Information Systems Journal 2010; 20(5):449–480. 8. Bourque P, Fairley RE (eds.). Guide to the software engineering body of knowledge (SWEBOK), Version 3.0. IEEE Comput. Soc. Press: Piscataway, NJ, USA, 2014. 9. Kotonya G, Sommerville I. Requirements Engineering: Processes and Techniques, Worldwide Series in Computer Science. John Wiley and Sons, 1998. 10. Ott D. Defects in natural language requirement specifications at Mercedes-Benz: an investigation using a combination of legacy data and expert opinion. Proc. of the 20th IEEE Int. Requir. Eng. Conf., RE ’12, IEEE Comput. Soc. Press: Los Alamitos, CA, USA, 2012; 291–296. 11. Mavin A, Wilkinson P, Harwood A, Novak M. Easy approach to requirements syntax (EARS). Proc. of the 17th IEEE Int. Requir. Eng. Conf., RE ’09, IEEE Comput. Soc. Press: Washington, DC, USA, 2009; 317–322. 12. Mavin A, Wilkinson P. Big Ears (the return of “Easy Approach to Requirements Engineering”). Proc. of the 18th IEEE Int. Requir. Eng. Conf., RE ’10, IEEE Comput. Soc. Press: Washington, DC, USA, 2010; 277–282. Copyright © 2016 John Wiley & Sons, Ltd.



225

13. Mavin A. Listen, then use EARS. IEEE Software 2012; 29(2):17–18. 14. Noll J, Beecham S, Richardson I. Global software development and collaboration: barriers and solutions. ACM Inroads 2010; 1(3):66–78. 15. Ebert C. Global Software and It: A Guide to Distributed Development, Projects, and Outsourcing. John Wiley and Sons: Hoboken, NJ, USA, 2012. 16. Holtkamp P, Lau I, Pawlowski JM. How software development competences change in global settings—an explorative study. Journal Software-Evolution and Process 2015; 27(1):50–72. 17. Rine DC, Nada N. An empirical study of a software reuse reference model. Information and Software Technology 2000; 42(1):47–65. 18. Pacheco CL, García IA, Calvo-Manzano JA, Arcilla M. A proposed model for reuse of software requirements in requirements catalog. Journal Software-Evolution and Process 2015; 27(1):1–21. 19. Goldin L, Berry DM. Reuse of requirements reduced time to market at one industrial shop: a case study. Requirements Engineering 2015; 20(1):23–44. 20. Carmel E, Abbott P. Why ‘nearshore’ means that distance matters. Communications of the ACM 2007; 50(10):40–46. 21. Toval A, Carrillo de Gea JM, Nicolas J, Fernandez Aleman JL, Toval R. Learning systems development using reusable standard-based requirements catalogs. IEEE Glob. Eng. Educ. Conf. (EDUCON), 2011; 907–912. 22. Cos JA, Toval A, Fernandez Aleman JL, Carrillo de Gea JM, Nicolas J, Toval R. Internationalization requirements for e-learning audit purposes. IEEE Glob. Eng. Educ. Conf. (EDUCON), 2012; 96–101. 23. Basili VR, Shull F, Lanubile F. Building knowledge through families of experiments. IEEE Transaction on Software Engineering 1999; 25:456–473. 24. Svahnberg M, Aurum A, Wohlin C. Using students as subjects—an empirical evaluation. Proc. of the 2nd ACM-IEEE Int. Symp. on Emp. Softw. Eng. and Meas., ESEM ’08, ACM: New York, NY, USA, 2008; 288–290. 25. Carver J, Jaccheri L, Morasca S, Shull F. Issues in using students in empirical studies in software engineering education. Proc. of the 9th IEEE Int. Symp. on Softw. Metr., METRICS ’03, IEEE Comput. Soc. Press: Los Alamitos, CA, USA, 2003; 239–249. 26. Carver J, Jaccheri L, Morasca S, Shull F. Using empirical studies during software courses. Empirical Methods and Studies in Software Engineering, LNCS, vol. 2765, Conradi R, Wang A (eds.). Springer Berlin / Heidelberg, 2003; 81–103. 27. Prikladnicki R, Nicolas Audy JL, Evaristo R. Global software development in practice lessons learned. Software Process Improvement and Practice 2003; 8(4):267–281. 28. Herbsleb JD, Paulish DJ, Bass M. Global software development at Siemens: experience from nine projects. Proc. of the 27th Int. Conf. on Softw. Eng., ICSE ’05, ACM: New York, NY, USA, 2005; 524–533. 29. Daneva M, Marczak S, Herrmann A. Engineering of quality requirements as perceived by near-shore development centers’ architects in Eastern Europe: the hole in the whole. Proc. of the 8th ACM-IEEE Int. Symp. on Emp. Softw. Eng. and Meas., ESEM ’14, ACM: New York, NY, USA, 2014; 19:1–19:10. 30. Bhat JM, Gupta M, Murthy SN. Overcoming requirements engineering challenges: lessons from offshore outsourcing. IEEE Software 2006; 23:38–44. 31. Niazi M, Mahmood S, Alshayeb M, Hroub A. Empirical investigation of the challenges of the existing tools used in global software development projects. IET Software 2015; 9:135–143. 32. Ouhbi S, Idri A, Fernández Alemán JL, Toval A. Requirements engineering education: a systematic mapping study. Requirements Engineering 2015; 20(2):119–138. 33. Damian D, Hadwin A, Al-Ani B. Instructional design and assessment strategies for teaching global software development: a framework. Proc. of the 28th Int. Conf. on Softw. Eng., ICSE ’06, ACM: New York, NY, USA, 2006; 685–690. 34. Gotel O, Kulkarni V, Scharff C, Neak L. Students as partners and students as mentors: an educational model for quality assurance in global software development.Software Engineering Approaches for Offshore and Outsourced Development, Lecture Notes in Business Information Processing, vol. 16, Berkling K, Joseph M, Meyer B, Nordio M (eds.). Springer Berlin Heidelberg, 2009; 90–106. 35. IEEE S2ESC. IEEE Std 1517-2010, IEEE Standard for Information Technology—system and software life cycle processes—reuse processes. IEEE: New York, NY, USA, 2010. 36. Vavassori FB, da Silva RC. Evaluation of a systematic approach to requirements reuse. The Journal of Universal Science 2013; 19(2):254–280. 37. He Z, Bustard DW, Liu X. Software internationalisation and localisation: practice and evolution, vol. 25, Waldron J, Power JF (eds.), PPPJ/IRE, ACM Int. Conf. Proc. Series. ACM: Maynooth, County Kildare, Ireland, 2002; 89–94. 38. Loucopoulos P, Karakostas V. Systems Requirements Engineering. McGraw-Hill, Inc.: New York, NY, USA, 1995. 39. Zowghi D, Coulin C. Requirements elicitation: a survey of techniques, approaches, and tools. Engineering and Managing Software Requirements, Aurum A, Wohlin C (eds.). Springer Berlin Heidelberg, 2005; 19–46. 40. ISO/IEC JTC 1 SC 7. IEEE S2ESC. ISO/IEC/IEEE 29148:2011, Information technology—systems and software engineering—life cycle processes—requirements engineering, 1st edn. ISO: Geneva, Switzerland, 2011. 41. Robertson S, Robertson J. Mastering the requirements process. (2nd edn), Addison-Wesley Professional, 2006. 42. McClure C. Software Reuse Techniques: Adding Reuse to the System Development Process. Prentice-Hall, Inc.: Upper Saddle River, NJ, USA, 1997. 43. Toval A, Moros B, Nicolás J, Lasheras J. Eight key issues for an effective reuse-based requirements process. Computer Systems Science and Engineering 2008; 23:1–13.



226


44. Toval A, Nicolás J, Moros B, García F. Requirements reuse for improving information systems security: a practitioner’s approach. Requirements Engineering 2002; 6:205–219. 45. Firesmith DG. Specifying reusable security requirements. The Journal of Object Technology 2004; 3(1):61–75. 46. Lam W, McDermid J, Vickers A. Ten steps towards systematic requirements reuse. Proc. of the 3rd IEEE Int. Symp. on Requir. Eng., RE ’97, IEEE Comput. Soc. Press: Washington, DC, USA, 1997; 6–15. 47. Heck P, Zaidman A. Horizontal traceability for just-in-time requirements: the case for open source feature requests. Journal of Software-Evolution and Process 2014; 26(12):1280–1296. 48. Ramesh B, Jarke M. Toward reference models for requirements traceability. IEEE Transactions on Software Engineering 2001; 27(1):58–93. 49. K Av, Paech B, Kiedaisch F, Houdek F. Systematic requirements recycling through abstraction and traceability. Proc. of the 10th Anniv. IEEE Jt. Int. Conf. on Requir. Eng., RE ’02, IEEE Comput. Soc. Press: Washington, DC, USA, 2002; 273–281. 50. ISO/IEC JTC 1 SC 36. ISO/IEC 24751:2008, Information Technology—Individualized Adaptability and Accessibility in E-Learning, Education and Training, 1st edn. ISO: Geneva, Switzerland, 2008. 51. Holmes R, Walker RJ. Systematizing pragmatic software reuse. ACM Transactions on Software Engineering and Methodology 2012; 21(4):20:1–20:44. 52. Markov GA, Hoffmann A, Creighton O. Requirements engineering process improvement: an industrial case study. Requirements Engineering: Foundation for Software Quality, Lecture Notes in Computer Science, vol. 6606, Berry D, Franch X (eds.). Springer Berlin Heidelberg, 2011; 34–47. 53. Moros B, Toval A, Rosique F, Sánchez P. Transforming and tracing reused requirements models to home automation models. Information and Software Technology 2013; 55(6):941–965. 54. Carrillo de Gea JM, Nicolás J, Fernández Alemán JL, Toval A, Ebert C, Vizcaíno A. Commonalities and differences between requirements engineering tools: a quantitative approach. Computer Science and Information Systems 2015; 12(1):257–288. 55. Murphy TE. Market guide for software requirements definition and management solutions. Technical Report G00235237, Gartner, Inc. 2014. 56. Thurimella AK, Bruegge B, Janzen D. Variability plug-ins for requirements tools: a case-based theory building approach. IEEE Systems Journal 2015; PP(99):1–12. DOI:10.1109/JSYST.2015.2418290. 57. Zave P, Jackson M. Four dark corners of requirements engineering. ACM Transactions on Software Engineering and Methodology 1997; 6(1):1–30. 58. Carrillo de Gea JM, Nicolás J, Fernández Alemán JL, Toval A, Vizcaíno A, Ebert C. Reusing requirements in global software engineering. Managing Requirements Knowledge, Maalej W, Thurimella AK (eds.). Springer Berlin / Heidelberg, 2013; 171–197. 59. Barber KS, Graser TJ. Requirements evolution and reuse using the Systems Engineering Process Activities (SEPA). Australasian Journal of Information Systems 1999; 6(2):75–97. 60. Windle DR, Abreo LR. Software Requirements Using the Unified Process: A Practical Approach. Prentice Hall PTR: Upper Saddle River, NJ, USA, 2002. 61. Favaro J. Managing requirements for business value. IEEE Software 2002; 19:15–17. 62. Meth H, Brhel M, Maedche A. The state of the art in automated requirements elicitation. Information and Software Technology 2013; 55(10):1695–1709. 63. Kauppinen M, Vartiainen M, Kontio J, Kujala S, Sulonen R. Implementing requirements engineering processes throughout organizations: success factors and challenges. Information and Software Technology 2004; 46(14):937–953. 64. Cybulski JL. Introduction to software reuse. Tech. Report TR 96/4, Dept. of Information Systems, University of Melbourne 1996. 65. Palomares C, Franch X, Quer C. Requirements reuse and patterns: a survey. Requirements Engineering: Foundation for Software Quality, Lecture Notes in Computer Science, vol. 8396, Salinesi C, van de Weerd I (eds.). Springer Int. Publ., 2014; 301–308. 66. Abrahao S, Insfran E, Carsí JA, Genero M. Evaluating requirements modeling methods based on user perceptions: a family of experiments. Information Sciences 2011; 181(16):3356–3378. 67. Moody DL. Comparative evaluation of large data model representation methods: the analyst’s perspective. Conceptual Modeling – ER 2002, Lecture Notes in Computer Science, vol. 2503, Spaccapietra S, March ST, Kambayashi Y (eds.). Springer Berlin Heidelberg, 2003; 214–231. 68. Fleiss JL. Measuring nominal scale agreement among many raters. Psychological Bulletin 1971; 76(5):378–382. 69. Fleiss JL, Levin BA, Paik MC. Statistical Methods for Rates and Proportions, 3rd edn. J. Wiley, 2003. 70. Maxwell KD, Forselius P. Benchmarking software development productivity. IEEE Software 2000; 17(1):80–88. 71. Premraj R, Shepperd M, Kitchenham B, Forselius P. An empirical analysis of software productivity over time. Proc. of the 11th IEEE Int. Symp. on Softw. Metr., METRICS ’05, IEEE Comput. Soc. Press: Washington, DC, USA, 2005; 37–46. 72. Seyff N, Graf F, Maiden N, Grünbacher P. Scenarios in the wild: experiences with a contextual requirements discovery method. Requirements Engineering: Foundation for Software Quality, Lecture Notes in Computer Science, vol. 5512, Glinz M, Heymans P (eds.). Springer Berlin Heidelberg, 2009; 147–161. 73. Mavin A, Maiden N. Determining socio-technical systems requirements: experiences with generating and walking through scenarios. Proc. of the 11th IEEE Int. Conf. on Requir. Eng., RE ’03, IEEE Comput. Soc. Press: Washington, DC, USA, 2003; 213–222.




227

74. Seyff N, Maiden N, Karlsen K, Lockerbie J, Grünbacher P, Graf F, Ncube C. Exploring how to use scenarios to discover requirements. Requirements Engineering 2009; 14(2):91–111. 75. Wohlin C, Runeson P, Höst M, Ohlsson MC, Regnell B, Wesslén A. Experimentation in Software Engineering: An Introduction. Kluwer Academic Publishers, 2000. 76. Gillespie R. Manufacturing knowledge: a history of the Hawthorne experiments. Studies in Economic History and Policy. Cambridge University Press, 1993. 77. Honig WL, Prasad T. A classroom outsourcing experience for software engineering learning. Proc. of the 12th Annu. SIGCSE Conf. on Innov. and Technol. in Comput. Sci. Educ., ITiCSE ’07, ACM: New York, NY, USA, 2007; 181–185. 78. Campus Mare Nostrum. What is CMN? Last access: October 2015. (Available from: http://www.campusmarenostrum.es/que_es_cmn_en.html) 79. ACM/IEEE Joint Task Force on Computing Curricula. Software engineering 2013 curriculum guidelines for undergraduate degree programs in software engineering. IEEE Comput. Soc. and ACM, 2013. 80. Saldaña-Ramos J, Sanz-Esteban A, García J, Amescua A. Skills and abilities for working in a global software development team: a competence model. Journal of Software-Evolution and Process 2014; 26(3):329–338.



Co-located and distributed natural-language ...

Co-located and distributed natural-language ...

Suggest Documents

Colocated General Medical Care and ... - Psychiatric Services

MIMO Radar with Colocated Antennas: Theoretical Investigation ...

Supporting Colocated Interactions Using RFID and Social Network ...

Comparisons of Ground Motions from Colocated and ... - David M. Boore

Supporting Colocated Interactions Using RFID and ... - Semantic Scholar

Some Observations on Colocated and Closely ... - David M. Boore

A Single-Fiber Bidirectional Optical Link Using Colocated Emitters and ...

Radiometer Calibration Using Colocated GPS Radio ... - IEEE Xplore

The White River Model of Colocated Collaborative Care

Colocated Finite Volume Schemes for Fluid Flows 1 Introduction

TARGET ESTIMATION IN COLOCATED MIMO RADAR VIA MATRIX ...

Cooperative Resynchronization to Improve the Reliability of Colocated ...

target estimation in colocated mimo radar via matrix ... - Rutgers

Examining ambient noise using colocated measurements of rotational ...

A capon beamforming method for clutter suppression in colocated ...

Distributed Cognition and Distributed Morality - Semantic Scholar

DISTRIBUTED METACOGNITION Distributed ...

distributed an distributed and coor ibuted and coor frequency ... - Cired

Distributed and Parallel Systems

Distributed Interaction Processing and

Distributed objects and components

Distributed

distributed

Distributed