Development Risk Assessment in Software Projects Using ...

2013 IEEE 16th International Conference on Computational Science and Engineering

Development Risk Assessment in Software Projects using Dependability Models A. Melo, E. Tavares, M. Marinho, E. Sousa, B. Nogueira, P. Maciel Center for Informatics (CIn) Federal University of Pernambuco (UFPE) Recife-Brazil Email:{amm4, eagt, mlmm, etgs, bcsn, prmm}@cin.ufpe.br This work presents an approach based on dependability models for evaluating project risks on software development projects. The proposed approach takes into account reliability block diagrams (RBD) and stochastic Petri nets (SPN), considering a hybrid modeling technique for estimating dependability metrics. In this paper, we are concerned with development risks related to developer turnover and requirement implementation, prominent risks that considerably affect a software project [7] [15] [16]. Two case studies are adopted to demonstrate the feasibility of the proposed technique.

Abstract—Risk management is a prominent activity, which aims the mitigation of undesirable issues during a project lifetime. In this context, techniques have been proposed to evaluate the impact of risks, but probability estimates are usually neglected, affecting a proper evaluation of such issues. This work presents an approach based on dependability models (e.g., stochastic Petri nets) for probabilistic evaluation of risks regarding the turnover of team members and requirement implementation in software development projects. Two case studies are adopted to demonstrate the feasibility of the proposed technique. Keywords-Risk Assessment, Dependability, Petri Nets, Reliability Block Diagrams.

II. R ELATED W ORKS

I. I NTRODUCTION

Quantitative assessment of risks in software projects is not a common practice [2], but some representative works have been developed over the years. Gupta and Sadiq [3] proposes Risk Assessment and Estimation Model (SRAEM) to assess risks in software projects considering requirement modification (e.g., addition, exclusion). Such a technique adopts function points to quantify risk occurence. Wattanapokasin and Rivepiboon [4] proposes a mathematical model for estimating risks related to cultural differences in a software development team, such as spoken language, communication style, and different time zones. The technique assumes a poisson process for estimating risk probabilities. Falahah [8] describes method for risk quantification, focusing on technical, cost, and schedule risks. The method provides probability estimates using a questionnaire as input data with weights associated for each risk factor. From the weights, estimates are obtained. In [5], the authors proposes a model based on Bayesian networks with causality constraints (BNCC) for risk analysis of software development projects. The model focuses on finding the correlation between risk factors and project outcome. In [6], Boness et al presents a technique for assessing software project risk and related losses during a requirement analysis phase. Such a technique performs risk assessment using high-level subjective metrics collected during requirements analysis using goal graphs for probability estimates. Different from previous works, this work presents an approach based on dependability models for estimating project risks regarding developer turnover and requirement implemen-

Software development with reduced time-to-market is a need of the global market, but it is still a risky endeavor. Reports indicate that only 32% of software projects succeed [11]. A good project planning positively contributes to the software project success, which should include a proper assessment of risks during project lifetime. Risk management has been adopted by organizations to avoid or mitigate the occurrence of undesirable events that might affect a project. Nevertheless, such management still faces obstacles to be inserted in the methodologies of software development companies [2]. Risks on a software project threaten its feasibility. In other words, whenever risks become real, they may considerably affect project execution, or even leading to its cancellation. From risk identification, companies may find good opportunities to start out risk management. A prominent risk category [1] is project risks, which may affect project schedule or the required resources for software development (e.g., developer turnover). Risk evaluation is usually carried out using qualitative models [2]. Nevertheless, quantitative models can help project managers to estimate the occurrence of risks (e.g., probability estimates), and, thus, managers can perform a better planning to avert the inherent risks in software projects. Dependability evaluation [10] provides several techniques based on formal models (e.g., stochastic Petri nets) to estimate the occurrence of system failures based on probability estimates. In the context of software projects, such techniques can also be adopted, providing important insights to project managers concerning the identified risks. 978-0-7695-5096-1/13 $31.00 © 2013 IEEE DOI 10.1109/CSE.2013.49

260

space explosion. Reliability Block Diagrams (RBD), Fault Trees and Reliability Graphs are representative combinatorial models, Markov Chains (e.g., CTMC) as well as Stochastic Petri Nets (SPN) are prominent state-based models.

tation. Besides, the proposed approach also considers faulttolerance techniques to mitigate the impact of risks. III. D EPENDABILITY Dependability denotes the capacity of a system in offering a service in a reliable way [10]. An important concept is system failure, which happens when the system stops providing the respective functionalities. A fault concerns the failure of a system component (or subsystem), in which can cause other faults or system failure. Dependability usually contemplates the following metrics/attributes: reliability, availability, maintainability, security, testability, confidentiality and integrity. In this work, the attributes of interest are: • Reliability: probability of a system making its predefined functions without failures for a specified period of time [10]: R(t) = P {T ≥ t}, in which T is the random variable representing the time to failure of the system (or a single component). Indeed, R(t) = 1 − F (t), such that F (t) = P {T < t} is the cumulative distribution function representing that a failure takes place before time t; • Availability: probability of a system being in a functioning condition. It considers the alternation of operational and nonoperating states [10]. Steady-state availability (A) is commonly adopted, and the following equations are also taken into account A = uptime/(uptime + downtime), or A = M T T F/(M T T F + M T T R). ∞ M T T F is the mean time to failure, M T T F = 0 R(t)dt; and  ∞ M T T R is the mean time to repair, M T T R = M (t)dt. Besides, M (t) = 1 − Fm (t), and Fm (t) is the 0 cumulative distribution function representing the probability that a repair will occur within time t. R(t) is the reliability function as previously presented. Redundancy is an important technique, which consists in adding external components to the system, such that if a component fails, the redundant component takes place. Dynamic redundancy is a representative technique that takes into account spare components to replace the main component whenever it is nonoperational. Hot and cold standby are representative approaches. Considering cold standby, a backup component is only activated when the primary component fails. On the other hand, hot standby component is activated, running simultaneously with the primary component. Whenever the latter fails, the backup immediately replaces it. Models for estimating dependability attributes are generally classified as combinatorial or state-based models/metrics [10]. Combinatorial models consider the conditions that make the system operational or faulty, regarding the structural relation between its components. However, such models have limitations to represent complex interactions between system components and elaborated maintenance policies. On the other hand, state-based models represent system dynamic behavior by its states and event occurrences [10]. These models are more suitable to model complex interactions between components, such as mechanisms based on dynamic redundancy. However, state-based models suffer from state

IV. P ROPOSED METHOD Usually, risk management considers the following activities [1]: (i) identification; (ii) analysis; (iii) planning; and (iv) monitoring. The process is iterative that begins at project planning and continues throughout the project lifetime. Risk assessment is adopted in analysis activity in order to estimate the quantitative/qualitative impact of identified risks. Using dependability concepts, risks in software development process can be probabilistically estimated, and techniques (e.g., dynamic redundancy) can be utilized to avoid or mitigate the occurrence of such issues. For instance, the following concepts and techniques may be adopted: •

•

• •

Failure: the occurrence of a risk (e.g., some developers left the project, affecting the minimum amount of required developers) to keep the software process operational; MTTF: mean time for a developer to leave the project or mean time for a developer to fail in the implementation of a requirement/functionality; MTTR: the mean time for replacing a developer or developer recovery to implement other requirements; Cold and Hot standby: pair programming;

Fig. 1 presents the proposed method for probabilistic risk assessment in software development projects, which is carried out during analysis activity (i.e., after risk identification). Our emphasis is on quantitative risk assessment, which is a subpart of risk analysis. As follows, the methodology activities are detailed:

  

 



      

Fig. 1.

Proposed method’s activities.

Working/Failure mode and metric definitions. In this activity, the working/failure modes of a software project is defined as well as the desired metrics considering each identified risk. For instance, assuming developer turnover, the working mode would consider a minimum of 4 developers to keep the software process executing, and the metrics of interest would include availability;

261

Data Analysis. Mean values, such as MTTF and MTTR, are commonly required to estimate the dependability metrics/attributes. In such a case, an individual may adopt historical data (from current or other projects) to estimate such values whenever available. Although software projects are distinct, past projects may provide prominent data for estimating many metrics, giving interesting insights to project managers regarding the identified risks. Concerning MTTF estimates, censoring may occur in the sense that data are incomplete, since not all components failed during a observation period [9] (e.g., a developer did not leave the project) . In such a situation a non-parametric method, Kaplan-Meier [9], provides an initial estimate for an empirical distribution, and, next, a fitting to theoretical distribution is performed and evaluated using statistical test (i.e., KS-test) [9], to assess the goodness-of-fit. In this work, phase-type distributions are considered [13], as they provide approximation to general probability distributions and can be adopted in the evaluation of CTMC and SPN models. Assuming complete data, moment-matching technique is directly applied [13]. Whether no historical data is available, MTTF and MTTR can be defined based on an individual expectation and expertise. Dependability Model Generation. This activity corresponds to the generation of dependability models according to the defined failure modes and estimated MTTFs/MTTRs. In this work, the models are constructed using a hybrid modeling, which adopts results from different models (Section VI). The approach considers the characteristics of the working/failure mode as well as the complexity to obtain a defined metric. Model validation. Once the system model is created, a validation is prominent to assess the metric/attribute estimated by the model. However, this activity may be optional, as the model may represent a project that has not yet started. The same situation may occur whenever MTTF/MTTR are defined by experience. On the other hand, assuming an ongoing project, an individual may collect data and carry out an comparison. Depending on available data and complexity, the relative error (err) or a statistical test may validate the model (e.g., paired test). err = (|v model − v data| ∗ 100)/|v data|, in which v model is the metric estimated by the model, and v data is the metric obtained by analysing the collected data. Scenarios Assessment. Once the baseline model is constructed, the manager may conceive several scenarios to evaluate the respective impact on dependability by modifying the structure and the values in the current model. After model evaluation, whenever the estimated metrics do not provide the awaited values, previous activities can be iteratively executed. The proposed method can be automated by tools, in the sense that, from a high-level model (e.g., UML diagram), the dependability models could be automatically generated considering the specified working or failure mode. Similarly, such a tool would consider an integrated functionality to estimate MTTF/MTTR from a database with historical data. Nevertheless, such (semi-) automated tool is out of scope from

this work. Besides, although this work is concerned with turnover and requirement implementation, the same approach could be adopted to estimate other risks, such as risks related to requirement analysis. V. M OTIVATIONAL E XAMPLE Let us consider a software project with 3 developers and turnover as a prominent risk. Besides, assume M T T F as 18 months and M T T R equal to 1 month. From these mean values, availability can be calculated for a single component/developer (Section III), and, from the components, the system availability is estimated (Section VI). A manager may require the project availability considering three working conditions: (i) all developers working; (ii) 2 out of 3; and (iii) at least one developer. Such assumptions may impact the order of functionality/requirement implementation, which may further affect project execution when a developer leaves. Using the models described in Section VI. Scenario i has the wost availability, 0.8502, which indicates that, in one month, the project is operational 85.02% (0.8502 ∗ 30, approximately 25.5 days). Availability for scenario ii is 0.9919. However, scenario iii indicates a better availability (0.9998 or 99.98%), despite a possible impact on performance. Based on the estimated values, a project manager should plan appropriate actions to mitigate the risk as well as associated costs due to unavailability (U A = 1 − A) for the adopted scenario. As an example, depending on project criticality, he/she may adopt measures to increase M T T F , such as bonuses or better salaries. Another alternative is the consideration of more developers in the software project. VI. D EPENDABILITY M ODELING This section presents the adopted models based on reliability block diagrams and stochastic Petri nets for estimating dependability metrics. As follows, building block models are presented. A. Reliability Block Diagrams Reliability block diagrams (RBD) represent the functioning of a system based on its working components [10]. A set of blocks (depicted as rectangles) denotes the components, in which arcs define the logical relationship. These diagrams have been adopted to evaluate series-parallel as well as more general structures, such as bridge, and the following paragraphs describe the arrangements adopted in this work. Series Arrangement. Fig. 2(a) depicts the RBD model, in which whenever one component fails, then the whole system also fails. Supposing a system with n components, availability/reliability (Ps ) is estimated as follows Ps =

n 

pi

(1)

i=1

in which pi refers to component i availability/reliability. Parallel Arrangement. Fig. 2(b) depicts the RBD model. A system is on failure state only when all components fail.

262

 







 

Fig. 2.

1) Simple Component: This block (Fig. 3) is characterized by the absence of redundancy, in the sense that the component may be in two states: active or inactive. Simple component has two parameters (not shown in the figure), namely, X M T T F and X M T T R, which represent the times associated to transitions X F ailure and X Repair, respectively. Places X ON and X OF F represent active and inactive states.



 

RBD arrangements.

Taking into account n components, availability/reliability is estimated using

  

 

Ps = 1 −

n 

(1 − pi )



(2)

i=1

Fig. 3.

in which pi refers to component i availability/reliability. K-out-of-n Arrangement. Fig. 2(c) depicts the RBD model, in which a minimum number of components (k) is required in order to keep the system operational. Supposing a system with n identical components, availability/reliability (Ps ) is estimated using  n   n pi (1 − p)n−i Ps = (3) i

Simple Component model.

Whenever #X ON > 0, the component is operational. A simple component model may represent a single developer concerning turnover or requirement implementation. 2) Cold Standby Model: Fig. 4 depicts the cold standby model [14], in which a non-active spare component is only activated when the primary component fails (X OF F ). In such a case, the transition Activate Spare X, representing the spare module starting operation. The activation takes a certain amount of time, which is represented by the Mean Time to Activate (M T Activate). After fixing the primary component (#X ON > 0), the spare is deactivated (firing of transition Deactivate Spare X).

i=k

in which p refers to the component availability/reliability. A RBD model can adopt several components with different arrangements. In such a case, reduction and sum of disjoint products [10] are representative approaches to estimate system availability or reliability.

  

B. Stochastic Petri Nets Petri nets (PN) [12] are a family of formalisms very wellsuited for modeling several types of systems, since concurrency, synchronization, communication mechanisms as well as time delays are naturally represented. In general, Petri net is a bipartite directed graph, in which places (represented by circles) denote local states and transitions (depicted as rectangles) represent actions. Arcs (directed edges) connect places to transitions and vice-versa. Tokens (small filled circles) may reside in places, which denote the state (i.e., marking) of a PN. An inhibitor arc is a special arc type that depicts a small white circle in one edge, instead of an arrow, and they represent the unavailability of tokens in places. This work adopts Stochastic Petri nets (SPN) [12], which is a prominent PN extension that allows the association of exponential distribution to timed transitions (represented by white rectangles), or zero delays to immediate transitions (depicted as thin black rectangles). The state space of SPN models may be translated into continuous time Markov chains (CTMC) [12], as well as simulation techniques may be adopted for estimating dependability metrics, as an alternative to the Markov chain generation. Henceforth, #p denotes the number of tokens in place p; and P {exp} estimates the probability of the inner expression exp.



Fig. 4.

Cold Standby Model.

Similar to simple component model, X Repair and S Repair X represent maintenance activities, as well as X F ailure and F ailure S X denote the failure delays. Besides, expression P {(#X ON +#S ON X) > 0} indicates the components working mode. Such a model can be adopted to represent pair programming in order to decrease the impact regarding developer turnover. 3) Preventive Maintenance/ Periodic Meeting Model: The models presented so far considered corrective maintenance. In this section, a preventive maintenance model is described, which takes into account periodic actions to keep the component in the operational state. In the context of this work, preventive maintenance assumes periodic meetings with stakeholders to assist developers to avoid failures concerning requirement implementation a similar approach can be adopted

263

For a better visualization, this work adopts number of nines [10] to present some results: −log10 (1−X) (in which X refers to availability). For instance, 0.99521 can be presented as −log10 (1 − 0.99521) = 2.319664487. Considering the activities of the proposed method, the case studies adopt historical data for estimating MTTF and MTTR. Regarding MTTF, censoring is present, and data are classified as type I, singly censored on the right [9]: the observation terminated after a fixed length of time; all developers had the same observation period; and some individuals did not left the project during the observation. In this case, Section IV explains the adopted approach. On the other hand, MTTRs are estimated based on complete data (no censoring).

for developer turnover (or requirement analysis). Fig. 5 depicts the model, in which a simple component is connected to a preventive block. Transition MTBP represents the mean time between preventive maintenances (e.g., meetings), and transition PM considers the execution of the preventive maintenance (the meeting itself). Place resource denotes the resource available for such a maintenance, for instance, a stakeholder, in which dashed lines indicate the resource is shared with other simple components. 

 



 

  

 





A. Case Study 1 This case study assumes developer turnover as a potential risk. The project contemplates 10 developers, which can be a trainee (TR), junior developer (JE) or a senior developer (SE). More specifically, a senior developer has more experience than other categories, and a junior developer is more experienced than trainees. A minimum of 7 developers is required (without discerning previous categories) to keep the project operational, and, based on collected data, the estimated availability is 0.999214286. In the coming sections, different scenarios are assessed, including the adoption of redundancy. Scenarios without developer distinction. These scenarios assume a k-out of-n approach in order to evaluate availability, assuming, a minimum number of developers (k) to not halt the software project. MTTF represents the mean time for a developer to leave the project, and MTTR considers the mean time for replacing a developer. In this section, the mean values are estimated taking into account all categories, and Table I depicts the values and the chosen probability distributions. Both combinatorial and state-based models are suitable representations, and Fig. 6 depicts the adopted RBD model (as a closed-form equation is available).





Fig. 5.

Preventive Maintenance/Periodic Meetings Model

As exponential distribution has the memoryless property, the preventive maintenance does not impact the elapsing of failure time. Nevertheless, failure times based on phase-type distributions are affected. Thus, as an prominent example, Fig. 5 depicts the MTTF assuming an Erlang with 2 phases. For each place representing a phase, an immediate transition is adopted as output transition with the guard expression (#P 0 > 0). Thus, whenever a preventive maintenance is executing (e.g., periodic meeting), the elapsing of failure time is reseted, improving the dependability metric. Similarly, #X ON > 0 assesses if the component is operational. C. Hierarchical composition As previously stated, RBD and SPN are prominent modeling techniques to assess a system dependability, but both have limitations. Thus, we adopt a hybrid modeling mechanism, which is very suitable for representing complex and larger systems [10]. Besides, a hierarchical composition is also taken into account. More specifically, smaller RBD and SPN models may estimate metrics for some parts of a system, and a highlevel model considers these metrics to evaluate the system dependability (e.g., availability). For instance, a component with cold standby can be modeled using SPN, and, next, the estimated metric is adopted in a RBD model to assess system dependability in conjunction with the other components. Such an approach is exemplied in the following section.

 

Fig. 6.

RBD model without developer distinction.

TABLE I MTTF/MTTR IN MONTHS WITHOUT DEVELOPER DISTINCTION . Metric Value Standard Deviation Distribution MTTF 23.3385 12.3405 Hypoexponential MTTR 2.0 1.0 Erlang

VII. C ASE S TUDIES

Table II presents the results and Fig. 7 depicts a comparison regarding number of nines. The baseline model is 7-out-of 10, in which availability is 0.994472645. The relative error is 0.55% comparing to the availability estimated for the actual software project. As should be expected, when the minimum number of required developers increases, availability is reduced. Particularly, from minimum of 4 developers, availability considerably

The proposed case studies takes into account two real-world software projects to demonstrate the conceived approach for assessing development risks in order to assist decision making for project managers. As follows, availability is estimated for several scenarios, and redundancy techniques as well as preventive maintenance are also considered to improve dependability metrics.

264

Fig. 7.

Availability for k-out-of -n without developer distinction.

Fig. 9.

TABLE II R ESULTS FOR K - OUT- OF - N SCENARIOS . k-out-of -10 1 2 3 4 5 6 7 8 9 10

Availability for k-out-of-n with distinct developers.

TABLE IV M INIMUM NUMBER OF DEVELOPERS FOR EACH SCENARIO .

Availability Number of 9’s 0.999999999990613 0.999999998895311 0.999999941000000 0.999998152000000 0.999961600000000 0.999449776000000 0.994472645000000 0.961284687000000 0.816056393000000 0.439458250000000

Scenario TR JE SE 1 1 1 5 2 3 1 3 3 2 2 3 4 3 2 2

11.0274710217 8.95675996780 7.22914798840 5.73329803310 4.41566877560 3.25946047020 2.25748264150 1.41211722530 0.73531530160 0.25139203500

TABLE V R ESULTS FOR DISTINCT DEVELOPERS . Scenario Availability Number of 9’s 1 2 3 4

drops. Thus, the manager may avoid scenarios with low availability. Scenarios with distinct developers. This experiment considers the impact of developer categories on dependability. Table III depicts the estimated MTTFs and the probability distributions for each category. Besides, we assume the same MTTR for all expertise types (Table I), and the team contemplates 3 trainees (TR), 2 junior developers (JE) and five senior developers (SE). MTTF

TABLE III

Expertise MTTF Standard Deviation Distribution TR 20.5676 10.5147 Hypoexponential JE 23.1059 22.1457 Hypoexponential SE 31.8521 17.2144 Hypoexponential

TABLE VI P ENALTIES IN US$.

Table IV depicts some conceived scenarios, requiring a minimum of 7 developers, but with different constraints for each category. All scenarios have 10 developers, but, for instance, scenario 1 requires at least 1 trainee, 1 junior developer, and 5 senior developers for being operational. Both combinatorial and state-based models are feasible approaches for modeling the scenarios, and Fig. 8 depicts the RBD model for scenario 1. Other scenarios adopt similar models, but with a different k value for each expertise type. 









Fig. 8.

0.572367997 0.603411921 0.761157551 0.445094307

From unavailability (U A = 1 − A), an individual may estimate possible costs related to the project halt. For instance, assuming a monthly fine (MF) of US$ 50.000 due to the halt, the respective cost (C) may be calculated using C = MF * UA * P, in which P is the period. Table VI shows the penalties for each scenario taking into account different periods. More elaborated cost functions may be created by integrating performance models, for instance, to assess the unavailability impact on project deadline. However, such an approach is a future work.

IN MONTHS FOR DISTINCT DEVELOPERS .



0.732310089 0.750777023 0.826682487 0.641155997

Scenario 1 Month 2 Months 3 Months 1 13.384,50 26.768,99 40.153,49 2 12.461,15 24.922,30 37.383,45 3 8.665,88 17.331,75 25.997,63 4 17.942,20 35.884,40 53.826,60

The software project could adopt pair programming for senior developers (SE), as they have more experience and they are usually associated with more prominent activities in the software construction. Indeed, when experienced developers abandon a software project, they create a knowledge gap that has to be controled [16]. Two approaches are assumed: cold and hot. The latter assumes an additional developer is active, and the former considers a delay to activate the spare individual. Although pair programming may improve dependability aspects, project budget is impacted, for instance, to consider additional human resources.

RBD model for Scenario 1.

Table V presents the results, and Fig. 9 depicts availability result considering number of nines. Scenario 3 has the best availability value, but availability (A) is lower than the values presented in Table II.

265

Combinatorial models are not very suitable to represent the dynamic behaviour of cold standby redundancy, but statebased models are feasible. To mitigate the state space size, each SE with cold standby is represented by a SPN model (Section VI) and the final model is a RBD. The mean time to activate is one day. From the cold standby model, availability is estimated (P {(#X ON + #S ON X) > 0}) for 1 SE, and the respective value is adopted in the RBD model (e.g., Fig. 8) to estimate the availability of a scenario. Concerning hot standby, two senior developers are in parallel arrangement. The respective availability is estimated and adopted in Fig. 8 (instead of considering the cold standby availability) Table VII presents the results, and Fig. 10 depicts them, indicating that scenario 1 is considerably improved, since it requires a minimum of 5 SEs to keep the project operational. The other scenarios are not significantly impacted, since they require less SEs, and TRs as well as JEs do not have spare developers. Nevertheless, hot standby technique provides improved availability than the cold counterpart for one scenario.

Minimum of k developers. A minimum number of developers (k) are required to not interrupt project execution. RBD is adopted due to closed-form equations.

TABLE VII R ESULTS FOR COLD / HOT STANDBY.

K - OUT- OF - N SCENARIOS FOR REQUIREMENT IMPLEMENTATION .

TABLE VIII M ETRICS IN DAYS FOR SCENARIO BASE . Metric

Value Distribution

MTTF 29.6298 Exponential MTTR 1.6923 Exponential

Table IX presents the results and Fig. 11 depicts a comparison regarding number of nines. The baseline model is 2out-4, which indicates a relative error equal to 0.06%. As the reader should note, availability is impacted when the minimum number of required developers increases. Periodic meetings with stakeholders may improve dependability, in the sense that they may assist developers in understanding the requirements. TABLE IX

k Availability Number of 9’s Scenario 1 2 3 4

Av. Cold N. of 9’s Cold 0.922313764 0.752170840 0.828217221 0.641193065

Fig. 10.

1.109655920 0.605847595 0.765020376 0.445139172

Av. Hot

N. of 9’s Hot

1 2 3 4

0.992962256 2.1525665348 0.752193670 0.6058876041 0.828242359 0.7650839333 0.641193211 0.4451393482

Fig. 11.

Availability for cold/hot standby.

0.999991479 0.999394687 0.983721279 0.800775849

5.0695094347 3.2180199989 1.7883797201 0.7006580153

Availability for k-out-of-n for requirement implementation.

Fig. 12 depicts the adopted model for periodic meetings using SPN (Section VI), as combinatorial models are not very suitable for this context. This model contemplates 4 developers sharing stakeholders, assuming the time for transition PM equal to 0.0416 day (1 hour) and for transition MTBP as 1 day. MTTF and MTTR are the same values previously adopted, but assuming an Erlang with 2 phases. The scenarios assume a minimum of k developers, and, considering k-out-of -4, the metric is defined as P {(#DE ON 1 + #DE ON 2 + #DE ON 3 + #DE ON 4) ≥ k}. Fig. 13 presents the results considering one and two stakeholders (#Stakeholder = 2). Availability is doubled in comparison to previous experiment, but an additional stakeholder does not considerably improve the results. Considering one stakeholder, Fig. 14 depicts some results varying the mean time between periodic meetings (from 1 to 5 days). Project

B. Case Study 2 In this section, we adopt the proposed method to quantify availability in a software project concerning risks related to requirement implementation. The project contemplates 4 developers, and a failure is assumed whenever 3 developers simultaneously fail to implement a requirement/functionality. From historical data, availability is evaluated as 0.999994419, considering the uptime and downtime. MTTF and MTTR are utilized to assess availability for many scenarios. The latter is considered as mean time for a developer to fail in the implementation of a requirement/functionality, and the former is the mean time for a developer to be ready for implementing another requirement. Table VIII depicts the adopted values and probability distribution. As follows, scenarios are evaluated for such a project.

266







 





 



 

 







 



 

 

   



 

 

 















 



   





 





 





  











Fig. 12.



Periodic meetings.

availability is impacted, but periodic meetings with stakeholders provide better results than the approach without the meetings.

including the adoption of dynamic redundancy and maintenance policies for improving dependability. Although this work has demonstrated the application of such techniques for turnover of team members and requirement implementation, other risk types may be evaluated as well. As future works, we are considering other risk types as well as performability models to estimate the impact of risks in the performance of development processes. R EFERENCES

Fig. 13.

Fig. 14.

[1] A Guide to the Project Management Body of Knowledge (PMBOK Guide) - Fifth Edition, 2013 [2] K. Bakker and et al. Does risk management contribute to IT project success? A meta-analysis of empirical evidence. International Journal of Project Management, 2009. [3] Daya Gupta, Mohd Sadiq. Software Risk Assessment and Estimation Model. ICCSIT, 2008. pp 963-967. [4] Watcharee Wattanapokasin, Wanchai Rivepiboon. Cross-Cultural Risk Assessment Model. ICSPS, 2009. [5] Y. Hu and et al. Software project risk analysis using Bayesian networks with causality constraints. Decision Support Systems, 2012. [6] K. Boness and et al. A lightweight technique for assessing risks in requirements analysis. Software, IET (Volume:2, Issue: 1), 2008. [7] Basit Shahzad, Abdullah S. Al-Mudimigh. Risk Identification, Mitigation and Avoidance Model for Handling software Risk. CICSyN, 2010. pp 19 1-196. [8] Falahah. Risk management assessment using SERIM method. ICEEE, 2011. [9] Charles E. Ebeling. An Introduction to Reliability and Maintainability Engineering, Waveland Press, Inc., 2005. [10] P. Maciel and et al. Dependability Modeling In: Performance and Dependability in Service Computing: Concepts, Techniques and Research Directions. Ed. Hershey: IGI Global, Pennsylvania, USA, 2011. [11] The Standish Group International, Inc. Chaos Summary for 2010. Technical Report, 2010. [12] G. Balbo. Introduction to stochastic petri nets. Lectures on Formal Methods and Performance Analysis, 2001. [13] James F. Watson III and Alan A. Desrochers. Applying Generalized Stochastic Petri Nets to Manufacturing Systems Containing Nonexponential Transition Functions. In SMC, Vol. 21, 1991. [14] B. Silva and et al. ASTRO: An integrated environment for dependability and sustainability evaluation. Sustainable Computing: Informatics and Systems, 2013, pp.1-17. [15] T. Hall and et al. The Impact of Staff Turnover on Software Projects: The Importance of Understanding What Makes Software Practitioners Tick. Proceedings of ACM SIGMIS CPR, 2008. [16] D. Izquierdo-cortazar and et al. Using Software Archaeology To Measure Knowledge Loss in Software Projects Due To Developer Turnover *, System, pp. 1-10, 2009.

Availability with Periodic meetings.

Availability for Periodic meetings with different MTBPs.

The proposed experiments demonstrate that project managers may have a prominent tool to evaluate risks and different configurations to avoid or mitigate these undesirable issues. Depending on the software project and available resources, techniques based on dynamic redundancy and policy preventive maintenance may be feasible approaches to improve dependability metrics. VIII. C ONCLUSION This work presented an approach based on dependability models for evaluating development risks on software projects. Real-world case studies demonstrated the feasibility of the proposed approach, in which several scenarios were evaluated,

267