Comparative Study of Risk Assessment Models ...

IEEE-International Conference On Advances In Engineering, Science And Management (ICAESM -2012) March 30, 31, 2012

61

Comparative Study of Risk Assessment Models Corresponding to Risk Elements

Yogini Bazaz1*, Shashank Gupta2, Om PrakashRishi3, LalitSen Sharma4 1, 2, 3

Central University of Rajasthan, India, 4University of Jammu, India 1* 2

[email protected] [email protected] 3 [email protected] 4 [email protected]

 Abstract—In the modern era of software engineering, the development of software in static and dynamic environment results in several vulnerabilities that need to be handled so that they do not step in with the clear defined project goals. Previous studies show that the wide variety of different risk analysis strategies provide a valid solution to address the lack of risk management strategies in Software risk assessment model (SRAM), Software risk assessment and estimation model (SRAEM) etc. In this paper we have discussed the comparison between different software risk assessment models corresponding to certain risk elements. These risk elements must be taken into account in order to cover some perspectives of the software industry which have not been covered up to now. Based on this analysis, we have also concluded the weaknesses and strengths of risk assessment models.

Index Terms— Software Engineering, Software Risk, Software Risk Management, Software Risk Assessment Models, Software Risk Elements

I. INTRODUCTION Risk management is a action that help a software team to understand and manage uncertainty. Risk management also partly means reducing uncertainty [1]. A risk is a potential problem – it might happen or it might not. But regardless of the outcome, it’s a really good idea to identify the risk, assess its probability of occurrence, and estimate its impact. Everyone involved in the software development projects i.e. software engineers, managers and other stakeholders must participate in risk analysis and management. But software risks have been increasing as long as the software industry has been growing. Many software development projects miss their goals

of delivering a acceptable software product due to poor time management and budget and poor Software Risk Management (SRM) [2, 3]. Many software engineers and project managers have only a limited understanding of the concepts of SRM, due to which they can’t produce an expectable outcome. When risks are analyzed, it is important to measure the level of uncertainty and the degree of loss associated with each risk [4]. To accomplish this, three dimensions of risk are considered. (i) Project Risk (ii) Technical Risk (iii) Business Risk. The project risks threaten the project plan. That is, if the project risks become real, it is likely that project schedule will slip and as a result cost will also increase. Technical Risk threatens the quality and timeliness of software to be produced. Business Risk threatens the normal growth and development of software to be built and often threaten the product or project. It has been noticed that that this simple risk categorization won’t always work. Some risks are simply unpredictable in advance. In spite of this, not all organizations or software development projects uses these procedures and even if they do, they cannot use them successfully [5]. The paper is organized as follows. In section-II we present the background and related work of risk management and various risk assessment models. In section-III we present the comparison of different software risk assessment models. In section IV we have presented the comparison of different software risk assessment models corresponding to risk factors. In section V we have done the results and analysis. Finally in section VI we have concluded the work and suggested the future work.

II. BACKGROUND AND RELATED WORK Software risk management is an organized way of identifying risk factors, analyzing them, assessing their impacts on the

ISBN: 978-81-909042-2-3 ©2012 IEEE


software development projects and mitigating them when they arise. Systematic risk management may be facilitated through the use of wide variety of risk assessment models that provides quantitative assessments of the risks involved. There are few published risk assessment models that evaluate the risk of software projects. In recent times a model named Software Risk Assessment and Estimation Model (SRAEM) [6] has been developed in which risk is estimated using software metrics. This model gives the incremental risk for every phase and also the total cumulative risk as the software progress from phase to phase. Another model named Risk Identification, Mitigation and Avoidance Model (RIMAM) [7] takes care of the most frequently occurring risk and provides the way of handling all of them. This model is expected to cover all the handling and avoidance mechanism for software risks. In the series of the risk assessment models there is another model based approach named Software Risk Assessment and Evaluation Process using Model Based Approach (SRAEP) [8] which identify the risk by using the Software Fault Tree (SFT) [8]. Risk identification, risk analysis and risk prioritization are the main subparts of model based approach. The above existing risk assessment models are quite different from each other in aspect of risk identification, risk mitigation, risk prioritization and risk avoidance. In this paper we have done the comparison of above existing risk assessment models corresponding to certain risk elements. III. SOFTWARE RISK ASSESSMENT MODELS There are few published models that evaluate the risk of software projects. Among the existing models some of them have been selected for detailed comparison in study. These risk assessment models are expected to satisfy the needs of risk management for software development projects. The models were selected because they are dedicated to manage the risks or related aspects. The models are described hereafter: A. Software Risk Assessment Model(SRAM) This model makes use of comprehensive Questionnaire [9]. Test results show that using the risk indicator obtained from the SRAM, it is possible to predict the possible outcome of software projects with good accuracy. This model considers the following nine critical risk elements: complexity of software, staff involved in the project, targeted reliability, product requirements, method of estimation, method of monitoring, development process adopted, usability of software and tools used for development. A set of questions is carefully chosen for each of these elements with three choices of answers each. The answers are arranged in increasing order of risk. This model has considered the method of prioritization as a single step of risk assessment but do not specify how prioritization would be done. B. Software Risk Assessment and Estimation Model(SRAEM) In this model, the risk is estimated using risk exposure and software metrics of risk management and this metric is based

62

on Mission Critical Requirements Stability Risk Metrics (MCRSRM) [6].This model not only evaluate the risk but it also estimate the risk. Initially the model estimates the sources of uncertainty using different paradigms such as measurement error, model error and assumption error. The detailed description of this paradigm is given in the [6]. This model also uses the concept of function points [10-12] to explain these errors. This model is different from other existing models because the other models do not consider the issues related to requirement analysis. C. Risk Identification, Mitigation and Avoidance Model for Handling Software Risk(RIMAM) This model briefly presents the strategies that are expected for the purpose of identification, management and avoidance of certain risk factors such as immature Requirements, less Reusability, delivery Deadline, over-optimistic Technology Perceives, staff Experience, staff Turnover, excessive Error Detection and preservation of Intellectuals [7]. This model can be implemented easily with minimum cost. The easy and less costly processing may ensure that the already risk effected process may be completed more promptly and hence reducing the risk of further delay. The best part of the model can be customized with respect to the environment in which it is being used. D. Software Risk Assessment and Evaluation Process using Model Based Approach This method explains a better technique of risk estimation and risk prioritization. In SRAEP, software fault tree approach (SFTA) [8] is used to identify and analyze the risk. In SFTA, a high level threat is decomposed into intermediate objectives which can be further decomposed into individual attacker actions. Attacker actions are connected by AND or OR relationship. Generally, the exchange of an OR node with an AND node is expected to increase the safety of software development project. After the Identification of risk, several countermeasures come into pictures for risks. To pick an appropriate countermeasure, a new technique known as Risk Reduction Leverage (RRL) which is a simple calculation that gives a numeric value to a countermeasure enabling different countermeasures to be compared. Mathematically it can be written as: RRL= Reduction in Risk exposure / Cost of Countermeasure After computation of risk and value of RRL and its prioritization the next step is team review and action planning. IV. ANALYSIS The risk assessment models were reviewed for their ability to manage risks in multiple software project environments. In this section, a detailed comparative study of different risk assessment models has been proposed based on certain parameters. The parameters cover important different risk management aspects such as risk identification, risk prioritization etc.

ISBN: 978-81-909042-2-3 ©2012 IEEE


Table I which is shown at the end of this research paper gives detailed comparative study of different risk assessment models. After this comparative study, certain risk elements are selected which are based on the challenges, different risk areas and features of software development projects. The risk elements were identified by performing a literature survey [6, 9, 13-17]. A list of risk elements has been selected and the most frequently occurring risks were filtered. The risk elements cover important risk management aspects such as unclear requirements, software complexity, risk identification tracing, ethical issues, developer’s fidelity etc. The comparison has been carried on literature survey such as papers, earlier comparison, technical reports etc. Table II shows the related comparison corresponding to related risk elements. In Table II which is shown at the end of this research paper there are alternatives for each risk element:  When the risk element is supported or agreed by the model. X If the risk element is not supported or agreed by the model. P If the risk element is partially supported or partially agreed by the model From the figures which appear in the Table II, it can be commented that the total number of risk elements that are supported or agreed by the risk assessment models has got 91 points from the total of points which is 248 (with percentage 36%). The other ones which are partially supported or partially agreed have got 57 points(with percentage from the total of points 23%) whereas the risk elements that are not agreed or not supported by the models have got the lowest number of points i.e. 100 (with percentage of 41%). The risk elements that have got the highest support are:  Involvement of software developers and stakeholders.  Anticipation methods or prediction techniques.  Directed or Targeted Reliability.  Usability of software and tools used for software development.  Supervising strategy.  Growth or development process adopted. In general, the impact of results shown in the Table II on the existing risk assessment models has been described in the form of associated strengths and weaknesses of respective model which is elaborated as below: A. Strengths  Except SRAM, all other models focus specifically on the Immature Requirements.  Role of stakeholders and other software developers is fully cooperative in the SRM Estimation Method.  Most of the models have predefined target reliability (i.e. error handling conditions, error tolerance conditions, hardware faults etc.).  Other than SRAM and to some extent RIMAM, the other two models focus on evaluation of risk such as

63

determine the level of risk, prioritize the risk and categorize the risk.  Back-up issues have been seriously handled by almost all the risk assessment models by installing the repositories so that can one can move backward from a certain location.  Various types of prediction methods have been used by these models in order to estimate the sources of uncertainty (measurement error in SRAEM) identify the risk (SFT approach in SRAEM). B. Weaknesses  SRAM focus on the development risk but cannot assess the marketing risk. But all other models (SRAEM, RIMAM etc.) focus on both these aspects.  All the four models concentrate on the project perspective of software development and they do not pay enough care to process and product perspectives.  They are not agile enough to manage the dynamic risks in multiple project environments.  Most of the models are focussed on the theoretical aspects of SRM and do not provide clear guidelines for practicing the SRM support.  Most of them are not flexible enough to handle the risk in web and distributed software development environments  Lack of readiness to natural disaster.  Most of the models do not utilize or employ any side effect absorber mechanism to a typical risk.

V. CONCLUSION AND FUTURE WORK In this paper, we have done a comparative study of various risk assessment models corresponding to certain risk elements in order to identify their weakness and strengths in managing risk in software development environment. It can be concluded from Table II that from the existing models, SRAM is the most suffering model and does not consider as much as risk elements as others. However the SREAP is the most effective model based approach which can adapt the actual practicing in software industry practice as compared to other specified models. From Table II the following points can be concluded:  There is no model which is able to handle risks in Globally Distributed Software Development (GDSD) Projects due to multi-locations, multi-cultures, multigroups, multi-technologies.  There is no specific model which is able to manage the risks in web distributed environment alone.

ISBN: 978-81-909042-2-3 ©2012 IEEE


For the effective risk management in above two areas (GDSD, Web Distributed), the software developer has to combine the strengths of various models and tackling the weaknesses of them in a new model is a step forward in improving risk management in these two areas.

REFERENCES [1] [2]

[3]

[4] [5] [6]

[7]

[8]

[9] [10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

Roger L. Van Scoy, ”Software Development Risk: Opportunity, not problem”, Technical report, September 1992. A. Keshlaf and K. Hashim , “ A Model and Prototype Tool to Manage Software Risks,” Proc. of First Asia- Pacific Conference on Quality Software, IEEE Computer Society, 2000, pp. 297-305. S. Islam, “Software Development Risk Management Model- A Goal Driven Approach,” Proc. of ESEC/FSE Doctoral Symposium’ 09 Amsterdam – The Netherlands, ACM, 2009, pp. 5-8. Roger S. Pressman, A Manager’s Guide to Software Engineering, McGraw-Hill 1993. Susan A Sherer, “The Three dimensions of Software Risk: technical, Organizational, and Environmental”, IEEE-2005 Daya Gupta, MohdSadiq, “Software Risk Assessment and Estimation Model”, International Conference on Computer Science and International Technology, IEEE Computer Society, Singapore, 2008.pp 963-967.s BasitShahzad, Abdullah S. Al-Mudimigh, “ Risk Identification, Mitigation and Avoidance Model for Handling software Risk” International Conference on Computational Intelligence, Communication Systems and Networks, IEEE Computer Society, 2010. pp 191-196. Mohd.Sadiq1, Mohd. Wazih Ahmad2. “ Software Risk Assessment and Evaluation Process (SRAEP) using Model Based Approach”, International Conference on Networking and Information Technology, IEEE, 2010, pp 171-177. Say-Wei Foo, Armugam Muruganatham, “Software Risk assessment Model”, ICMIT 2000, IEEE, pp-536-544. Mohd. Sadiq and Shabbir Ahmed, “Computation of Function Point of Software on the basis of Average CompleXity”, Proceedings of 2nd International Conference on Advanced Computing and Communication Technologies, ICACCT 2007, Panipat, Haryana. Pp. 591-594. Mohd. Sadiq, Shabbir Ahmed, “Relationship between lines of code and Function Point and its application in the computation of Effort and Duration of Software using Software equation”, Proceedings of International conference on Emerging Technologies and Applications in Engineering, Technology and Sciences, ICATETS 2008, Rajkot, Gujarat. Chris F. Kermerer, “Reliability of Function Points Measurement, A Field Experiment”, Communication of the ACM, pp. 85-97, Vol.36, February 1993. B. Boehm, J. Kwan, D. Port, A. Shah and R. Madachy, “Using the Win Win Spiral Model: A Case Study, “ IEEE Computer, July 1998, pp.3344. J. Kontio, M. Hoglund, J. Ryden, and P. Abrahamsson, “Managing Commitment and Risks: Challenging in Distributed Agile Development, “ in Proc. of the 26th International Conference on Software Engineering (ICSE ’04), 2004, pp.732-733. J. Presson, L. Mathiassen, B. Jesper, T. Madsen, and F. Steinson, “Managing Risks in Distributed Software Projects: An Integrative Framework, “IEEE Transaction on Software Engineering, 2009, pp. 125. J. Tian, S. Rudrarjuand, and Z. Li, “Evaluating Web Software Reliability Based on Workload and Failure Data Extracted from Server Logs,” IEEE Transaction on Software Engineering, vol. 30, November 2004, pp. 754-769. Ayad Ali Keshlaf, Steve Riddle, “Risk Management for Web and Distributed Software Development Projects”, International Conference on Internet Monitoring and Protection, IEEE, 2010, pp - 22-28.

64

Yogini Bazaz: Yogini Bazaz was born on February 01, 1987, in the district of Jammu.. She has completed her secondary examination from J&K Board of Secondary Education in 2003. She obtained her B.Tech. degree from Punjab Technical University, Jalandhar (India) in 2009. Currently she is pursuing M. Tech. in Computer Science and Engineering with the specialization in Information Security from Central University of Rajasthan (India) which is one of the most prestigious universities in India and established by the MHRD, Govt. of India. Her area of Interest include Software Engineering, Data Mining. Shashank Gupta : Shashank Gupta was born on September 25, 1987, in the state of Jammu & Kashmir (India). He has completed his secondary examination from J&K Board of Secondary Education in 2004. He obtained his B.E. degree from Padmashree Dr. D. Y. Patil Institute of Engineering and Technology, University of Pune in 2009. Currently he is pursuing M. Tech. in Computer Science and Engineering with the specialization in Information Security from Central University of Rajasthan (India) which is one of the most prestigious universities in India and established by the MHRD, Govt. of India. His area of Interest include communication and networking, cryptography, software engineering, Theory of computation, DBMS, compilers.

Om Prakash Rishi : Dr. Om Prakash Rishi was born on August30, 1971, in a small village of Rajasthan (India). Dr. Rishi completed his secondary examinations from the board of secondary education Rajasthan in 1986 and graduation from University of Rajasthan in 1991. He earned his First Class M.Tech. degree in Computer Science in 2000 from Birla Institute of Technology, Mesra, Ranchi (India), and Doctor of Philosophy (PhD) in Computer Science in March 2009. Dr. Rishi’s academic credentials were burnished by the years he spent on the faculty of Birla Institute of Technology, Mesra, Ranchi (India), Banasthali Vidhyapeeth, Banasthali Rajasthan (India) and currently working with the Central University of Rajasthan (India) which is one of the most prestigious Universities in India and established by MHRD, Govt. of India. Dr. Rishi’s area of interest is Artificial Intelligence, Intelligent Systems, Cloud Computing and Information Security. He is member of IEEE, Computer Society of India, and several other National and International professional bodies. Lalit Sen Sharma: Dr. Lalit Sen Sharma was born on April 12, 1969 in the district of Bilaspur, Himachal Pradesh, India. He has obtained Master of Science in Mathematics and MCA from Guru Nanak Dev University, Amritsar (India). He has also obtained Doctorate of Philosophy (PhD) from Guru Nanak Dev University in 2008. Currently, he is working as an Associate Professor in the department of Computer Science and Information Technology in University of Jammu, India. He has been teaching to postgraduate students in computer applications for fifteen years. He is a life member of Indian Science Congress Association, Institute of Electronics and Communication Engineer, India and National HRD network, India. His area of interest include data communication and networking, internet and web services. He has also organized workshops and acted as a member of organizing committee to organize national conferences.

ISBN: 978-81-909042-2-3 ©2012 IEEE


65

Table I Software Risk Assessment Models Comparison Result

Sr. No.

Parameters

SRAM

SRAEM

RIMAM

SRAEP

1.

Technique

Assess the risk related to risk elements such as software complexity, method of estimation etc.

Not only assess the risk but also estimate the risk using risk exposure.

Focus on most frequently occurring risks and try to mitigate all risk factors such as immature requirements, staff inexperience etc.

This approach uses model based approach and give detailed explanation on risk estimation and risk prioritization.

2.

Identification of Risk

It makes use of comprehensive questionnaire in order to identify the risk.

No information regarding the identification of risk.

Certain risk factors can partly identify the risk.

SFT approach is used to identify the risk.

3.

Prioritization of Risk

No such technique is used.

Software metric (MCRSRM) is used.

Risk factors have been prioritized according to their frequency of occurrence and the impact they possess.

Risk Reduction Leverage (RRL) technique is generally used.

4.

Sources of Uncertainty

This is generally partly assumed based on specified risk factors.

It estimates using measurement error, model error and assumption error.

It calculates the sources of uncertainty using the ordered risk factors.

It measures the sources of uncertainty using SFT approach.

5.

Function Points


Concept of function points is used.



ISBN: 978-81-909042-2-3 ©2012 IEEE


Table II Software Risk Assessment Models Results Based on Risk Elements

Models

S

S

R

S

Grand

R

R

I

R

Total:

Risk

A

A

M

A

Elements

M

E

A

E

M

M

P

X

P



P X X

 P X

  X

 

 

X P X X X  P X P X P 

Unclear Requirements Perspectives: -Project -Process -Product Faculty Involved: -Software Developers -Involved Stakeholders Circumstance of: -Multi project area -Ethical Issues SRM Estimation Method Natural Tragedy Performance Rating Software Complexity Back off Consequences SRM Development Power Anticipation Methods Practical SRM Support Risk Identification Tracing Discovering from faults/errors Directed Reliability Less Recyclable Usability of tools used Dependencies Chasing Remote SRM Judging SRM Cost Interoperability Chasing Developer’s Fidelity Supervising Strategy Excessive User Stress Faculty Inexperience Readiness to a Irregular Risk Growth Method Adopted Totals: Agree or  supported Not agreed or not X supported P Partially agreed or Partially supported Total



X

P

P

1

1

2

 P P

3 2 3

1 2 1

 

 P

4 3

X X  P P P P X  P P P

X X P  P P  X  P P 

X X    P  P  P P P

 X  X X X X   P  X

 P  P P X P X  P P 

  P X P X P     X

 P  P P P P P  P P P

4 1 3





P



3

10

10

15

11

91

15

06

07

02

57

06

15

09

18

100

X

2 2 1 1 2

1

3 1 1 1

3 3 1 2

2 4 1 2 1

ISBN: 978-81-909042-2-3 ©2012 IEEE

1

4 1 1 1 2 3 2 1 1 3 4 2

2 1 3 1 1

2 1 2 3 1 3 1

2

3 2 1 1

248

66