Modeling and scenario simulation for decision support in management ...

JOURNAL OF SOFTWARE MAINTENANCE AND EVOLUTION: RESEARCH AND PRACTICE J. Softw. Maint. Evol.: Res. Pract. 2011; 23:35–50 Published online 23 April 2010 in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/smr.469

Modeling and scenario simulation for decision support in management of requirements activities in software projects Bernardo Giori Ambrósio1, ∗, † , José Luis Braga2 and Moisés A. Resende-Filho3 1 Universidade

Federal de Ouro Preto, João Monlevade, MG, Brazil Federal de Viçosa, Viçosa, MG, Brazil 3 Universidade Federal de Juiz de Fora, Juiz de Fora, MG, Brazil 2 Universidade

SUMMARY There are many tools and techniques readily available to support the work in requirements activities of software development processes. As a consequence, the high frequency of errors still occurring in requirements activities suggests that the misunderstanding of the relationships among key decisions is the probable reason for this. The present work presents a system dynamics model constructed to make it possible for users to better understand the relations among key decision variables in requirements activities. The model was parameterized with data taken from previous studies and from a software development company so as to run two sets of simulations with three scenarios each. Optimistic, baseline and pessimistic scenarios are created on the basis of different assumptions regarding risk factors related to requirements volatility and people turnover. We used our simulation results to foresee the effects of these risk factors on the quality and cost of work in requirements activities. Up-to-date results from the software engineering literature strongly support the simulation outcomes obtained in our research. Copyright q 2010 John Wiley & Sons, Ltd. Received 21 September 2009; Revised 10 February 2010; Accepted 17 March 2010 KEY WORDS:

software engineering; software process; requirements; risk evaluation

1. INTRODUCTION The decision-making process in the business world has become more complex over time as a consequence of the increasing number of decision variables considered, further complicated by the time lag between the implementation of decisions and the observation of their effects. Thus, decisions made without dynamically and carefully considering their future consequences are likely to result in huge losses for companies. Facing challenges like any other business, software companies have adopted development processes to give sound support for decisions in software development projects. Despite this, it is common to observe software development projects running into the risks of cost and schedule overruns, and ending up with poor-quality products. A reason for this is that project managers still do not understand all managerial aspects of a software development process [1, 2].

∗ Correspondence

to: Bernardo Giori Ambrósio, Universidade Federal de Ouro Preto, Rua 37, No. 115, Bairro, Loanda 35931-026, João Monlevade-MG, Brazil. † E-mail: [email protected] Contract/grant sponsor: Fapemig Contract/grant sponsor: CNPq Copyright q

2010 John Wiley & Sons, Ltd.

36

´ B. G. AMBROSIO, J. L. BRAGA AND M. A. RESENDE-FILHO

The proper analysis of decision options and their dynamic consequences on software development projects is often beyond human capacity, making it necessary the use of decision support tools and models. System dynamics provides a modeling technique and a language to analyze, understand and simulate decision problems with dynamic behavior [3]. Previous studies, such as [4–13], have used system dynamics models to gain systemic understanding of software project management processes by describing the mutual influences among variables. Previous studies such as [2, 14, 15] have pointed out that software development companies have often considered that requirements activities are not very important. For instance, Kappelman et al. [2] argue that a common characteristic of the majority of unsuccessful projects is the poor work performed during requirements activities. Confirming this point, Damian and Chisan [14] report that studies conducted by the Standish Group found that 28% of the software projects were completely aborted mainly because of problems with requirements activities. Besides, Jones [15] found that requirements activities are not properly executed by more than 75% of the software companies, as an adverse combination between time pressure and small-sized companies that cannot afford the necessary amount of skilled human resources. In addition, the misunderstanding of the dynamic interactions among variables related to requirements activities often leads managers to make reactive decisions on the basis of the immediate problem at hand. But reactive decisions are likely to mitigate the problem in the short run at the expenses of complicating it in the mid and long runs. This way, it is crucial to develop models that describe the dynamic behavior of variables and their mutual influence in requirements activities. Although models that own these features may help managers to understand the systemic impacts of their decisions, they have been the focus of few studies [11]. This article presents a system dynamics model focusing on key decision variables, aiming to support the decision-making process in requirements activities of a software development process. We ran scenario simulations by changing the value of model components related to risk factors (e.g., requirements errors and volatility, and workforce turnover) and conducted what–if analyses to evaluate managerial interventions such as the establishment of alternative schedule plans. In doing so, we aimed at testing our model’s capability to support the following five managerial questions: (1) what are the impacts of increased requirements errors likely caused by inexperienced team or a complex problem domain in which the team does not have the sufficient understanding of the topic? (2) What are the impacts of higher requirements change rates? (3) What are the impacts of augmented workforce turnover? (4) How do the requirements activities evolve under typical scenarios? (5) What are the effects of establishing alternative schedule plans? Our simulation results were coherent with the literature on Software Engineering, making us believe that our initial hypothesis is true by which system dynamics modeling can support the decision-making process in requirements activities. This article develops as follows. Section 2 characterizes and reviews the literature on requirements activities and system dynamics. Section 3 describes our system dynamics model. Section 4 describes how users can use our model to set, run and evaluate scenario simulations. Section 5 analyzes the results obtained from our own scenario simulations. Finally, Section 6 presents our main conclusions.

2. CONTEXT 2.1. Requirements in software development projects A requirement is an attribute, capability, constraint or quality property to be implemented in software as required by customers. Requirements are elicited and specified by means of requirements activities or discipline in software development processes. There are many elicitation techniques available in the literature, making it important to define criteria to decide when one technique is preferable over other. Davis et al. [16] systematically reviewed empirical studies that analyze the effectiveness of elicitation techniques to find the circumstances under which an elicitation technique is preferable over others in terms of reducing requirement errors and uncertainty. Copyright q


J. Softw. Maint. Evol.: Res. Pract. 2011; 23:35–50 DOI: 10.1002/smr

MODELING AND SCENARIO SIMULATION FOR DECISION SUPPORT

37

Requirement changes occur as a result of elicitation errors, the discovery of users’ new needs, improved understanding of the problem’s domain and changes in the environment in which the software will operate. DeMarco and Lister [17] point out five possible types of risks and argue that requirements volatility causes most cost increases and software delivery delays. In an empirical study, Ebert and Man [18], and Ebert [19] analyze data from 246 software projects of a telecommunication company, identifying as causes of requirements uncertainty: vague vision and strategy of the product, non-involvement of key stakeholders, no thorough evaluation of the business case, and insufficient specification and analysis of requirements. Wieringa [20] states that research in requirements engineering should study the existing difficulties in the analysis of knowledge domains. A mapping study that identifies all quality aspects of the requirements specification process and product that are currently being considered by researchers is presented in [21]. These authors found that the majority of academic work so far has basically focused on experiments, which may compromise the generalization of their results and shows the need for more research in real-life settings. The recurring problems in requirements activities suggest that the research in requirements engineering has not influenced the way work is done by software companies. Regarding this point, Kaindl et al. [22] investigate, on the basis of debates with senior researchers and practitioners, the existing difficulties to introduce research results into mainstream practice. 2.2. System dynamics System dynamics is a descriptive language and model used for system modeling and simulation, which is based on systemic thinking and the mathematical theory of Dynamic Systems [3]. This technique is especially useful for understanding the effects of policies and the structure of a system in terms of its dynamic behavior [3]. System dynamics was developed by Forrester [23] to model and analyze the behavior of complex systems composed of dynamically and nonlinearly related variables. Forrester [23] considers that mutual influences among variables give rise to feedback loops that ultimately determine the behavior of a system. In fact, feedback loops are critical to the system as changes in variables can trigger behaviors that may lead the entire system to collapse. A system dynamics model comprises three main components: stocks that accumulate resources of the system; flows that dynamically change the level of stocks; and converters/variables that affect the values of flows. 2.3. Literature review The book Software Project Dynamics: an Integrated Approach [4] is one of the most cited studies among those addressing the use of system dynamics for modeling aspects of software project management. In this book, Abdel-Hamid and Madnick [4] present a model that comprises four subsystems: human resource management, planning, controlling and software production (i.e., design and coding). Conceptually important to our system dynamics modeling, the authors mention the need for an upper bound constraint on the number of beginners in the team, the reduction of effort allocated to quality assurance under schedule pressure, the reduction in the productivity of work in long schedules and the increased number of errors caused by the high relative participation of beginners in the team. The article Adapting, Correcting, and Perfecting Software Estimates: a Maintenance Metaphor [5] presents a hybrid estimating model that combines Constructive Cost Model (CoCoMo) and system dynamics. Its author gives examples on how to use his model to adapt the CoCoMo estimates to the organizational reality before a project starts, to correct initial assumptions on sizing during software development, and to improve model estimates after project completion. As we consider in our model, the author mentions that the number of necessary workers is determined by the amount of effort needed and by the schedule plan. The article The Slippery Path to Productivity Improvement [6] presents a system dynamics model used to evaluate, via controlled experimentation, factors that can negatively impact team Copyright q



38


productivity. Examples of such factors are schedule pressure, communications overhead and a high proportion of beginners in the team. The article points out conceptual relationships that are important for our modeling process. For instance, it mentions the need to improve team size and/or to promote overtime to make it feasible to finish requirements activities under schedule pressure and the reduction in team productivity caused by communication overhead when team size increases excessively. The article Software-Engineering Process Simulation (SEPS) Model [7] presents a dynamic simulation model of a software project development process that allows the simulation of the dynamic interactions among software lifecycle development activities and management decisionmaking processes. The objectives of SEPS model were to assist software managers in the analysis of pre-project contingencies and support project preplanning during the development lifecycle. Some conceptual relationships of the SEPS model are also incorporated in our model, such as: beginners are less productive and make more errors than experienced workers; the need to promote overtime under schedule pressures; the need to improve team size under tight schedule plans and the higher number of errors when the team becomes stressed after excessive extra effort. An example of a software development process simulator to evaluate the effects of workforce turnover on organizations is given in A System Dynamics Software Process Simulator for Staffing Policies Decision Support [8]. This study shows a simulator to run ‘what–if’ scenarios aimed at assessing the effects of managerial staffing decisions on budget, schedule and quality of a project. In doing so, the authors analyze the effects of three policies: to replace engineers who leave the project; to overstaff in the beginning of the project and to do nothing hoping that the project is completed on time and respecting the initial budget. The authors mention the effect of team stress and exhaustion caused by excessive extra effort and overtime on error rate increase. This idea has been also incorporated in our model. The effects of alternative objectives on software project planning and resource allocation decisions are investigated in The Impact of Goals on Software Project Management: an Experimental Investigation [9]. The article develops and presents a role-playing simulation game wherein participants play the role of a software project manager. The article Coping with Staffing Delays in Software Project Management: an Experimental Investigation [10] uses system dynamics for examining how decision makers deal with staffing delays, and how their decisions affect the outcome of software projects. The authors report the results of a laboratory experiment in which graduate students manage a simulated software project in which delays in the hiring and/or assimilation of staff occur. The ideas that were taken from this article and incorporated into our model are: beginners learn over time according to a learning curve and become experienced workers, the need for imposing constraints on the time necessary to allocate new workers to the team and the amount of effort to finish the work is determined by total remaining work and perceived team productivity. The article Using Simulation to Analyze the Impact of Software Requirement Volatility on Project Performance [11] addresses the extraction and the specification of requirements, but is of limited scope because it focuses only on the impacts of requirements volatility. In the article, Pfahl and Lebsanft use a system dynamics model to show the impact of unstable software requirements on effort and project duration. Our model improves on [11] by considering more types of risks and alternative managerial decisions. A model to simulate planning a project release is presented in A System Dynamics Simulation Model for Analyzing the Stability of Software Release Plans [12]. The model allows the user to specify the tasks to be executed in software releasing, the relationships among tasks, the workload of each task, and staff productivity. The model can be used to run simulations to evaluate the effect of factors such as increasing task-dependency, workload variation and alternative developers’ productivity on project schedule. Madachy in his book Software Process Dynamics [13] shows examples and provides guidelines on how to apply system dynamics to software projects development and management. As of interest to our work, the book presents models related to team work and project management, process improvement and organizations management. Copyright q




39

3. A SYSTEM DYNAMICS MODEL FOR REQUIREMENTS Our system dynamics model takes key variables related to requirements activities focusing on the effort exerted in requirements extraction. The model comprises artifacts and activities conducted to elicit, specify, validate and manage requirements in software development projects. As the basis to construct the model, we executed the following seven activities: 1. identify the variables related to requirements activities; 2. select the key variables among those identified in the first activity; 3. gather information in the literature to identify and document the mutual influence paths among the key variables; 4. build a system dynamics model to portray the relationships among the key variables, using a free academic version of Vensim [24]; 5. run simulations with the model and refine it on the basis of the understanding we achieved through running simulations; 6. parameterize the relationships among model components with values taken from the literature, leaving some values to be defined by the decision maker so as to instantiate scenarios of his/her interest; 7. validate the model running further simulations and check if their results fitted well to the common understanding and knowledge of the academic community and Software Engineering experts. It is important to mention that we did not necessarily follow the seven activities in sequence. Instead, we followed an iterative process by which we went back to any previously executed activity when it was considered necessary. Also, we relied just on the literature to identify (third activity) and quantify (sixth activity) the relationships among our model components because we could not find available time series data on software projects. To present the model, we adopt the following convention. Every model component (see Figures 1–3) has its name in italics, with initial capital letter for stocks and flows and initial lower-case letters for converters/variables. Two short parallel line segments across an arrow denote that the effect of a component on the other occurs with time delay. A ‘+’ above an arrow denotes that linked variables change in the same direction such that when a variable increases/decreases the other increases/decreases, whereas a ‘−’ signal implies that variables vary in opposite directions. Size and complexity of work are all measured in function points [25] as widely used in the literature, not relying on estimates of Line-of-Code (LOC). Required effort is measured in man-days according to [26] and calculated as: Effort = Size/Productivity, where Size is in function points and Productivity is in function points per man-day. Finally, to comply with space constraints, we present parts of our full model [27] that deal with requirements volatility and workforce turnover as follows.

Figure 1. The effect of higher requirements volatility on the effort needed to make changes. Copyright q




40

Figure 2. The loop originated from higher requirements volatility and workf6ce turnover on the allocation of beginners in the team.

Figure 3. Model describing the causes of increase labor cost.

3.1. Modeling the effects of increased software requirements volatility Figure 1 portrays the part of our full model [27] that deals with requirements volatility and their effects on the effort needed to complete requirements specification. The Rate of Requirements Changes flow specifies the rate by which requirements changes are requested and conveys requirements from Specified Requirements stock to the Requirements Waiting Change stock. Increased Requirements Waiting Change stock raises the values of the variables work remaining in function points and man-days needed to finish specification. The reason for this is that the amount of effort needed to complete a project depends on the work remaining to be done [13]. Copyright q




41

The Managerial Decisions flow establishes the total effort available (e.g., the value of the mandays available daily variable) and, as a consequence, determines the effort available to process requirements changes (e.g., the value of man-days allocated for processing requirements changes variable). The man-days allocated for processing requirements changes variable determine the Rate of Requirements Changes Processing flow, which conveys requirements from Requirements Waiting Change stock to Re-specified Requirements stock. Figure 2 shows that an increase in the man-days needed to finish specification variable, possibly caused by higher requirements volatility, may increase the number of necessary workers variable [10]. Therefore the workforce balance variable reduces, thus creating the need for a bigger team that is translated into an increased value of Rate of Allocation flow. In its turn, the augmented Rate of Allocation flow increases the Beginners Workforce stock, leading to an increase in the total workforce variable. The augmented team size increases the relative participation of beginners in the team, contributing to high levels of specification errors [4, 7]. Note that we classify workers as beginners and experienced as portrayed in Figure 2 by the Beginners Workforce stock and Experienced Workforce stock. Beginners have no previous experience in the project, are less productive, make more errors than experienced workers [7] and learn over time [28] according to a learning curve represented by the Rate of Learning for Beginners flow in the model. Figure 2 also shows that it is possible to consider that an excessive increase in the team size or total workforce variable can complicate communications among the team members by reducing the average productivity variable [29]. Note that in our model, the variables total workforce and average productivity determine the value of man-days available daily. We recognize that the total number of workers allocated to the team can be less than needed when there are constraints on the maximum number of beginners in the team [4] and on the time necessary to allocate new workers to the team [10]. Thus, Figure 3 shows that in our model the difference between the number of necessary workers and total available workforce determines the lacking in workforce variable. The lack of workers creates the risk of schedule overruns, increasing time pressure. Under such circumstances, team workers are encouraged to work harder [6, 7]. This point is shown in Figure 3 by the positive influence of time pressure on increase in man-days available by team that ends up increasing extra effort provided by increasing working hours. We consider that excessive extra effort may stress the team, ultimately leading to more errors [7, 8]. We also consider that under time pressure the effort allocated to quality assurance is reduced [4, 9], which decreases the number of errors being detected. In Figure 3, the Man-days Spent stock measures the cost of finishing requirements specification by storing the amount of effort in man-days spent up to a given point in time. Figure 3 shows that increased team size and/or working hours given as variables total workforce and extra effort provided by increasing working hours increase the rate by which the team effort is spent (i.e., Rate of Man-days Spent flow), ultimately increasing costs. Thus, it may be necessary to allocate new workers to the team, use extra effort and/or delay schedule milestones to handle the need for increased effort created by higher requirements volatility. Either way, costs will increase. 3.2. Modeling workforce turnover Figure 2 shows how workforce turnover may increase the percentage of beginners in the team. For instance, an increase in the Rate of Turnover flow reduces the value of the Experienced Workforce stock, which will decrease the total workforce variable. Reduced total workforce will increase the value of the Rate of Allocation flow, creating a counteracting loop. Therefore, beginners should be hired and allocated to the team to balance the effects of an increased rate of turnover. But more beginners in the team could increase the amount of specification errors [4, 7]. To minimize this type of risk, team members can be encouraged to provide extra effort [6, 7], which may increase the number of specification errors. The model makes it possible to perceive that the turnover of experienced workers causes higher negative impact than the turnover of beginners. The model also allows perceiving that workforce turnover may cause an increase in costs, because of the need for hiring extra effort to fix more specification errors and to compensate the low productivity of beginners. Copyright q



42


4. RUNNING SIMULATIONS WITH THE MODEL The model’s control panel is the device by which the user sets scenarios and accesses simulation results. It features two categories of components the so-called input and output components. Input components are used to set the values for model components to build scenarios, whereas output components show simulation results. Input components are further classified into configuration and analysis components. Configuration components come with default values that can be altered by the user in order to adjust the model to the characteristics of the team, software project and organization under analysis. Analysis components are further divided into two subgroups according to the purpose of the component. The first subgroup comprises components used to characterize the risks such as: requirements volatility, workforce turnover, errors in the estimation of requirements amounts and the percentage of specification errors. The second subgroup comprises components that set the rules by which managerial decisions are implemented in requirements activities. For example, decisions regarding planning and management of team size, hiring extra effort from team workers and the adjustments in schedule plans. Some of the input components are shown in Figure 4.

Figure 4. Control panel: input components. Copyright q




43

Output components comprise the Man-days Spent stock that shows how total effort used during requirements activities evolves over time; the Requirements Errors stock that shows how the number of specification errors not yet detected by the quality assurance activities evolves over time and the Schedule Plan for Finishing Specification stock that stores the number of days necessary to complete the work in each point in time, indicating adjustments to the schedule plan to make it feasible to finish the work on time. Vensim [24] provides graphical outputs that show the value paths followed by model components during simulation. For example, the path for total workforce variable (Figure 2) shows how the team size evolves over time and the path for increase in man-days available by team variable (Figure 3) shows how extra effort contracted from workers evolves over time. The model’s control panel also comprises the graphical interface that allows users to interact with the model by changing its parameters without entering into model details. Decision makers can still use Vensim to view the model mathematical structure and to adjust influences among model components if a finer tuning is pursued.

5. MODEL SIMULATION AND EVALUATION We ran scenario simulations to analyze, predict and understand the impacts of risks and the effects of management decisions on requirements activities. In doing so, we used data mainly obtained in the literature to set our model and run a first set of simulations. After it, we re-parameterized some model components with data obtained from a software development company to run an additional set of simulations. In the two sets of simulations, we constructed three scenarios (optimistic, baseline and pessimistic) in accordance to the values in Tables I and II for Percentage of errors in requirements activities and Percentage of requirements change, considering workforce turnover or not. Also, we used values in Tables III and IV to set workload and team productivity that are modeled as 1-Requirements specified and delivered in a software release, and 2-Average productivity of professionals in the team. Note that these model components have the same values as those presented in Tables III and IV. 5.1. Simulations using data from the literature To run this first set of simulations, we used the values in Table III to characterize the software company, the team and the project under analysis. The value for component 3 was obtained from experts in a medium-sized software development company because we could not find this information in the literature. The definition of values for components 4, 5 and 6 was, respectively, based on [30–32]; [4]; and [4, 33, 34]. We defined it as 0% for component 4 because we were not focused in analyzing the risk of occurring errors in the estimation of the number of requirements. The three scenarios (optimistic, baseline, pessimistic) for this first set of simulations differ in terms of the model component values as shown in Table I. The values used for component 1 are based on information obtained from experts in a software development company, because we could not find this information in the literature. The values used for component 2 were taken from [35–38]. In accordance to [39, 40], we consider that workforce turnover occurs (component 3) only under the pessimistic scenario, where workforce turnover means that one worker leaves the team when half schedule is attained. Figure 5 shows the results for this first set of simulations. As expected, all curves have similar shapes since scenarios differ only by the different assumption regarding Percentage of errors in requirements activities, percentage of requirements change and if there is or not workforce turnover as in Table I. The graph on quality in Figure 5 shows how a tighter schedule affects the amount of specification errors under each scenario. In general terms, the tighter the schedule, the more frequently the errors will occur. The reason for this expected pattern [7, 8] is that it will be necessary to increase team size and/or to promote overtime [6, 7] to finish specification activities within schedule. As expected, we also observe that the optimistic and pessimistic scenarios showed, respectively, the Copyright q




44

Table I. Values used to set scenarios for simulations using data from the literature. 1. Percentage Optimistic: 2. Percentage Optimistic: 3. Turnover Optimistic:

of errors in requirements activities 6% Baseline: 12% of requirements change 2% Baseline: 5% No

Pessimistic: 20% Pessimistic: 10%

Baseline: No

Pessimistic: Yes

Table II. Values used to set scenarios for simulations using data from the software development company. 1. Percentage Optimistic: 2. Percentage Optimistic: 3. Turnover Optimistic:

of errors in requirements activities 5% Baseline: 15% of requirements change 10% Baseline: 20%

Pessimistic: 30%

No

Pessimistic: Yes

Pessimistic: 20%

Baseline: No

Table III. Values used to set simulations using data from the literature. 1. Requirements specified and delivered in a software release: 120 requirements are estimated at 120 function points 2. Average productivity of professionals in the team: 2 function points per man-day 3. Initial size of team work responsible for the specification: 2 4. Percentage of the requirements in a software release that are not expected before starting the specification: 0% (all requirements are expected) 5. Percentage of work team’s effort that must be allocated to quality assurance: 20% 6. Maximum increase in effort by team members when there are increased time pressure and risk of schedule overrun: 0.5 man-days (up to 50% more than the normal effort provided)

lowest and the highest percentage of specification errors over time. Under the pessimistic scenario, the higher need for rework was caused by the higher percentage of errors in requirements activities and requirements change. Extra effort is hired to avoid schedule overrun, which may stress the team and therefore increase the number of errors [7, 8]. Moreover, time pressure combined with the risk of schedule overruns may decrease the amount of effort allocated to quality assurance [4, 9], further increasing requirement errors. The opposite effect happens under the optimistic scenario. The fluctuations observed in curves of Figure 5 were expected since our model comprises dynamically and nonlinearly related components and several feedback loops. Cost curves in Figure 5 show that each scenario has its own point of minimum cost. As expected (Figure 6), all cost curves are U-shaped because tighter schedules can create the need to increase team size and/or use extra effort from the team [6, 7], ultimately increasing costs. But too much slack time in schedules makes it possible for workers to complete the work within schedule even wasting time in other activities not related to the project or under low productivity [4], also increasing costs. As expected (Figure 6), the cost curve for the pessimistic scenario is above the cost curve for the baseline scenario, which is above the cost curve for the optimistic scenario. But cost curves get closer to each other when the schedule becomes much longer than necessary (i.e., to the right of the point of minimum cost) because of the following reasoning. The need for more rework under the pessimistic scenario is obviously bigger than for any other scenario. But schedules that are much longer than necessary make the amount of work necessary to handle rework very small Copyright q



45


Table IV. Values used to set simulations using data from the software development company. Component

Low

1. Requirements specified and delivered in a software release 2. Average productivity of professionals in the team 3. Initial size of team work responsible for the specification 4. Percentage of the requirements in a software release that are not expected before starting the specification 5. Percentage of work team’s effort that must be allocated to quality assurance 6. Maximum increase in effort by team members when there are increased time pressure and risk of schedule overrun

Most likely

High

120 requirements are estimated at 120 function points 2 function points per man-day 2

2

3

15%

20%

25%

5%

10%

15%

10%

15%

20%

Figure 5. Results of scenario simulations. Copyright q



46


Figure 6. Expected patterns of cost curves.

when compared with the slack time available for the team, explaining why cost curves under different scenarios start to become increasingly closer to each other. Also as expected (Figure 6), the minimum cost for the pessimistic scenario is bigger than for the baseline scenario, which is bigger than for the optimistic scenario. Moreover, the schedule plan that leads to the minimum cost is lower for the optimistic scenario than for the baseline scenario, which is lower than for the pessimistic scenario. Finally, cost curves in Figure 5 show that the minimum costs for the optimistic, baseline and pessimistic scenarios are 60, 70 and 82.9 man-days and occur at the points of minimum cost 30, 35 and 40 days, respectively. The results are relatively close to each other across the scenarios, as expected, since the scenarios differ only by the few components shown in Table I. 5.2. Simulations using the information obtained from a software development company To run this second set of simulations, we relied on the experience of a director and project manager of a company who develop scientific and technological software systems to obtain the information in Table IV. It was considered as a medium-sized software project with 1000 to 10000 lines of code [26] and a schedule between 12 and 18 months. We interviewed the director and project manager and asked them for the values they considered as low, most likely and high for the components 3, 4, 5 and 6 in Table IV. We used the most likely values in Table IV to run this second set of simulations. They also provided the information in Table II that we used to set our scenarios. Figure 7 shows the quality and cost graphs for the results of the second set of simulations. The analysis and interpretation of the curves in Figure 7 follow exactly as we did for Figure 5; hence they will be omitted here. Curves in Figure 7 are very similar to the curves obtained in the first set of simulations in Figure 5, which shows again that the simulation results comply with the literature in Software Engineering. It is not essential to consider specific values in the analysis of results in Figures 5 and 7. Models are useful if they are capable of showing the correct direction and magnitude of changes in values [41]. Therefore, the coherence of the results obtained with the current literature for the two sets of simulations demonstrates that our model can be used to support decisions in requirements activities. 5.3. An alternative interpretation of the simulated cost curves A software development project consists of developing the first unit of software that implements the specifications required by users. A common characteristic of the software industry is that the Copyright q




47

Figure 7. Results of scenario simulations.

production cost of additional units of software is negligible compared with the cost of producing the very first unit of software. This way, we can take for granted that the cost graphs in Figures 5 and 7, in fact, show the total production cost of a software system. Note that the cost graphs in Figures 5 and 7 show the combinations between the effort in man-days and schedule in days that are capable to produce the first unit of software through requirements activities. Because an isoquant is a curve that shows all possible combinations of inputs that can produce a certain level of production [42], it is possible to interpret our model’s cost curves also as isoquants. Therefore, the minimum cost that is also the minimum effort level can be used to find the optimal schedule plan to produce the first unit of software through requirements activities. Under this perspective, it is clear that the schedule as a variable input in the production of the first unit of software attains different optimal levels across scenarios.

6. CONCLUSIONS It is beyond human capacity to consider all the elements of a software development project to foresee the consequences of changes in key factors and variables on the project development. As Copyright q



48


a consequence, the construction and the use of models emerge as a potential solution by creating learning laboratories [3] in companies that will make it possible to conduct controlled experiments so as to foresee the consequences of alternative managerial decisions. By using models, managers will be able to learn from simulations, without having the costs and facing the risk of a real project implementation. Based on this, we presented a system dynamics model developed to support decisions in the management of requirements activities in software projects. Unlike previous models in the literature, our model explicitly considers the elicitation and the specification of requirements and makes it possible to evaluate the effects on projects of alternative specification errors percentages, workforce turnover and requirements volatility. Our model is also capable of supporting decisions related to schedule plan, team size and extra effort to be provided by the team. We set our model using data from the literature and information obtained from a real software development company so as to run two sets of simulations. In each set of simulations, we considered an optimistic, a baseline and a pessimistic scenario. The results of the two sets of simulations were very similar and closely followed the direction and trends found in the literature on Software Engineering. As a consequence, we strongly believe that our model can be used to support managerial decisions related to the effects of increasing errors in the estimation of the number of requirements, the effect of different team sizes in the beginning of requirements specification, the effects of the extra work use and the effect of deviating effort to quality assurance activities. Managers may use our model to simulate what–if scenarios, focusing on aspects that determine the quality and work cost in requirements activities. Our model can serve as the basis for the implementation of a simulator to be used in training software project managers. Such a simulator would show in its control panel the characteristics of the project under analysis, allowing the user to change key components’ values to simulate alternative managerial options. Although we did not try to generalize our results, we believe (based on the quality of simulation outputs) that our model is adaptable to support decisions in projects similar to those discussed in this article. Our research agenda includes the collection of data from other software development companies to set and run additional simulations to further test and investigate the validity and the generality of our model.

ACKNOWLEDGEMENTS

This research was partially funded by Fapemig and CNPq.

REFERENCES 1. Humphrey WS. Managing the Software Process. Addison-Wesley: Reading, 1990. 2. Kappelman LA, Mckeeman R, Zhang L. Early warning signs of it project failure: The dominant dozen. Information Systems Management 2006; 23(4):31–36. 3. Sterman JD. Business Dynamics: Systems Thinking and Modeling for a Complex World. Irwin McGraw-Hill: Boston, 2000. 4. Abdel-Hamid TK, Madnick SE. Software Project Dynamics: An Integrated Approach. Prentice-Hall: Englewood Cliffs, 1991. 5. Abdel-Hamid TK. Adapting, correcting, and perfecting software estimates: A maintenance metaphor. IEEE Computer 1993; 26(3):20–29. 6. Abdel-Hamid TK. The slippery path to productivity improvement. IEEE Software 1996; 13(4):43–52. 7. Lin CY, Abdel-Hamid T, Sherif JS. Software-engineering process simulation model (SEPS). Journal of Systems and Software 1997; 38(3):263–277. 8. Collofello J, Houston D, Rus I, Chauhan A, Sycamore DM, Smith-Daniels D. A system dynamics software process simulator for staffing policies decision support. Proceedings of the Thirty-first Hawaii International Conference on System Sciences, 1998; 103–111. 9. Abdel-Hamid TK, Sengupta K, Swett C. The impact of goals on software project management: An experimental investigation. MIS Quarterly 1999; 23(4):531–555. 10. Sengupta K, Abdel-Hamid TK, Bosley M. Coping with staffing delays in software project management: An experimental investigation. IEEE Transactions on Systems, Man, and Cybernetics—Part A: Systems and Humans 1999; 29(1):77–91. Copyright q




49

11. Pfahl D, Lebsanft K. Using simulation to analyze the impact of software requirement volatility on project performance. Information and Software Technology 2000; 42(14):1001–1008. 12. Pfahl D, Al-Emran A, Ruhe G. A system dynamics simulation model for analyzing the stability of software release plans. Software Process: Improvement and Practice 2007; 12(5):475–490. 13. Madachy RJ. Software Process Dynamics. Wiley: Piscataway, 2008. 14. Damian D, Chisan J. An empirical study of the complex relationships between requirements engineering processes and other processes that lead to payoffs in productivity, quality, and risk management. IEEE Transactions on Software Engineering 2006; 32(7):433–453. 15. Jones C. Applied Software Measurement: Assuring Productivity and Quality. McGraw-Hill: New York NY, 1996. 16. Davis A, Dieste O, Hickey A, Juristo N, Moreno AM. Effectiveness of requirements elicitation techniques: Empirical results derived from a systematic review. Fourteenth IEEE International Requirements Engineering Conference (RE’06), 2006; 179–188. 17. DeMarco T, Lister T. Waltzing with Bears: Managing Risk on Software Projects. Dorset House: New York, 2003. 18. Ebert C, Man JD. Requirements uncertainty: Influencing factors and concrete improvements. Proceedings of the 27th International Conference on Software Engineering, 2005; 553–560. 19. Ebert C. Understanding the product life cycle: Four key requirements engineering techniques. IEEE Software 2006; 23(3):19–25. 20. Wieringa RJ. Methodologies of requirements engineering research and practice: Position statement. CERE ’03 Comparative Evaluation in Requirements Engineering: The First International Workshop on Comparative Evaluation in Requirements Engineering. Held in Conjunction with the 11th IEEE International Requirements Engineering Conference, 2003. 21. Condori-Fernandez N, Daneva M, Sikkel K, Wieringa RJ, Dieste O, Pastor O. Research findings on empirical evaluation of requirements specifications approaches. Twelfth Workshop on Requirements Engineering, 2009; 121–128. 22. Kaindl H, Brinkkemper S, Bubenko JA Jr, Farbey B, Greenspan SJ, Heitmeyer CL, Leite JCSP, Mead NR, Mylopoulos J, Siddiqi J. Requirements engineering and technology transfer: Obstacles, incentives and improvement agenda. Requirements Engineering 2002; 7(3):113–123. 23. Forrester JW. Industrial Dynamics. Productivity Press: Portland, 1961. 24. Vensim from Ventana Systems, Inc. Available at: http://www.vensim.com [10 February 2010]. 25. Matson JE, Barret BE, Mellichamp JM. Software development cost estimation using function points. IEEE Transactions on Software Engineering 1994; 20(4):275–287. 26. Pressman RS. Software Engineering: A Practitioner’s Approach (6th edn). McGraw-Hill: New York NY, 2004. 27. Ambrósio BG, Braga JL, Oliveira AP. Um Modelo Dinâmico para Análise dos Impactos da Rotatividade de Pessoal Durante a Fase de Requisitos. Proceedings of the 22th Simpósio Brasileiro de Engenharia de Software (SBES), 2008; 283–298. 28. Raccoon LBS. A learning curve primer for software engineers. ACM Sigsoft Software Engineering Notes 1996; 21(1):77–86. 29. Simmons DB. Communications: A software group productivity dominator. Software Engineering Journal 1991; 6(6):454–462. 30. Bergeron F, St-Arnaud JY. Estimation of information systems development efforts: A pilot study. Information and Management 1992; 22(4):239–254. 31. Molokken K, Jorgensen M. A review of surveys on software effort estimation. Proceedings of the International Symposium on Empirical Software Engineering, 2003; 223–230. 32. Moloekken-Oestvold K, Jorgensen M, Tanilkan SS, Gallis H, Lien AC, Hove SE. A survey on software estimation in the Norwegian industry. Proceedings of the 10th International Symposium on Software Metrics, 2004; 208–219. 33. Perlow LA. Time to coordinate: Toward an understanding of work-time standards and norms in a multicountry study of software engineers. Work and Occupations 2001; 28(1):91–111. 34. Tapia AH. The power of myth in the IT workplace: Creating a 24-hour workday during the dot-com bubble. Information Technology and People 2004; 17(3):303–326. 35. Jones C. Software Assessments, Benchmarks, and Best Practices. Addison-Wesley: Boston, 2000. 36. Jones C. Conflict and litigation between software clients and developers. Technical Note, 2001. Available at: http://www.spr.com/news/ConflictLitigationArticle.pdf [10 February 2010]. 37. Javed T, Manzil-E-Maqsood, Durrani QS. A study to investigate the impact of requirements instability on software defects. ACM SIGSOFT Software Engineering Notes 2004; 29(3):1–7. 38. Kulk GP, Verhoef C. Quantifying requirements volatility effects. Science of Computer Programming 2008; 72(3):136–175. 39. Simmons DB. Measuring and tracking distributed software development projects. Proceedings of the Ninth IEEE Workshop on Future Trends of Distributed Computing Systems, 2003; 63–69. 40. Ang S, Slaughter S. Turnover of information technology workers: The effects of internal labor market strategies. ACM SIGMIS Database 2004; 35(3):11–27. 41. McCarl BA. Model validation: An overview with some emphasis on risk models. Review of Marketing and Agricultural Economics 1984; 52(3):153–173. 42. Pindyck RS, Rubinfeld DL. Microeconomics (6th edn). Prentice-Hall: NJ, 2004. Copyright q



50


AUTHORS’ BIOGRAPHIES

Bernardo Giori Ambrósio received his BS in 2006 and MS in 2008, both in Computer Science from the Universidade Federal de Viçosa. He has been a professor at Universidade Federal de Ouro Preto since August 2008 where he teaches undergraduate courses on Information Systems and Computer Engineering. His research interests focus on software engineering, software project management and system dynamics. He has authored or coauthored five papers.

José Luis Braga got his DS (Doctor Scientiae) degree in Informatics from Pontificia Universidade Católica do Rio de Janeiro, Brazil, in 1990. He is a full professor at Departamento de Informtica, Universidade Federal de Viçosa-MG, Brazil, since August 1976. His main research interests include System dynamics applications in software engineering decision support, Software Quality and Software process. He advises many MS dissertations in Computing Sciences, leads research projects and has authored and co-authored many journal and conference papers.

Moisés A. Resende-Filho received his BS in Agronomy in 1993 and his MS in Agricultural Economics in 1996 from the Universidade Federal de Viosa, and his PhD in Applied Economics from the University of Minnesota in 2006. He is an assistant professor of Economics at University of Juiz de Fora. Among his research interests are the estimation of software development costs, applications of system dynamics and the development of decision support systems. He has authored or coauthored sixteen peer reviewed papers.

Copyright q

