A risk assessment model for collaborative support in software ...

Download PDF
9 downloads 12853 Views 8MB Size Report
Comment
Risk assessment in software management involved strategies to estimate and control risk ensuring proper function of the software process without system failure ...
2015 9th Malaysian Software Engineering Conference, Dec. 2015

A Risk Assessment Model for Collaborative Support in Software Management Bokolo Anthony 1nr. Dept. of Software Engineering and Information Systems Faculty of Computer Science and Information Technology University Putra Malaysia (UPM) 43400 Serdang, Selangor, Malaysia [email protected]

N oraini Che Pa Dept. of Software Engineering and Information Systems Faculty of Computer Science and Information Technology University Putra Malaysia (UPM) 43400 Serdang, Selangor, Malaysia [email protected] Yusmadi Yah Josoh Dept. of Software Engineering and Information Systems Faculty of Computer Science and Information Technology University Putra Malaysia (UPM) 43400 Serdang, Selangor, Malaysia [email protected]

Rozi Nor Haizan Nor Dept. of Software Engineering and Information Systems Faculty of Computer Science and Information Technology University Putra Malaysia (UPM) 43400 Serdang, Selangor, Malaysia [email protected]

Abstract- Risk assessment in software management involved strategies to estimate and control risk ensuring proper function of the software process without system failure. Software management related to risk assessment has attracted less interest in academic literature. Existing risk assessment models in software process cannot reduce the time involved in carrying out risk assessment; they also don't provide collaborative support among software practitioners in assessing risk and they cannot promote interactions between software practitioners who are distributed across different geographical regions. Thus this paper addresses this limitation and the gap in the current literature and proposed a risk assessment model using knowledge codification and multi software agents. The proposed model can assist practitioners in assessing risk by reducing time, estimating the risk and provides collaborative support within practitioners for managing software across different regions. Qualitative data was collected using semi-structured interview in 2 Malaysian organisations, with 3 respondents based on purposely sampling to identify how the practitioners assessing risk in their respective organisations. Results show that risks in software process are not adequately assessed based on inadequate collaborative support among practitioners and longer time taken to assess the risk. Keywords- Risk; Risk Assessment; Collaborative Support; Software Management)

I.

INTRODUCTION

Risk is a probability of occurrence of unwanted and harmful event to the project objective. Risk is also a combination of the likelihood of an event and its effects [1]. These events can result in delay, over budget, wrong functionality or termination of the project, degradation in product functionality and quality and high maintainability and reusability cost. Risk management aims to reduce risk [2]; [3]. The process of risk management includes set of the activities 978-1-4673-8227-4/15/$31.00 ©2015 IEEE

217

including identification, analysis, planning, tracking, controlling the risk [4]. Risk Assessment (RA) is one of the main activities in risk management [2]. RA involves five phases; identification, analysis, evaluation, control and mItIgation, and documentation. In general, risk assessment is referred to as the essential process to aid practitioners in implementing new business changes and where appropriate, invest in information systems to accommodate these changes [1]. Risk assessment is the practice of identitying, analyzing and prioritizing risk [5]. Risk assessments in software management aims to measure and estimate the magnitude of risk that occurs in software process. Software management is implemented to ensure that quality software is developed with affordable cost and in a shorter time. In software management there are three dimension of risk that occurs they are; technical risk, organizational risk and environmental risk [3]. The technical risk occurs from uncertainty in the task and procedure. The organizational risk results from poor communication and organizational structure. The environmental risk results from rapidly changing environment and problems with external relationship with software developers and/or users [6]. As first technical, then organizational, and more recently, environmental risk, have hindered development, use, and maintenance of IT infrastructures in developing software, procedures have been developed to assess the various risk components [7]; [8]; [9]. Risk assessment is a critical process in software management. Software management guides software practitioners to create plans for software development. Software management ensure that all of the project activities follow a certain predefined process, the activities are usually organized in distinct phases, and the process the activities are usually organized in distinct

phases, and the process specifies what artifacts should be developed and delivered in each phase [2]. One of the major steps in software management is to identify and assess potential risks. The utilization of modern information technology (IT) collaborative support systems is fast becoming an indispensable tool for software management today. Collaborative support is a process involving teams working together towards shared goals. Collaboration is about people working together and completing shared processes. Thus collaborative support in risk assessment is a joint effort of practitioners supporting each other to accomplish a common goal. There is an eminent need for software practitioners to have necessary strategies to assess identified risks, so that they can achieve organizational objectives. Collaborative support among software team members is assist in assessing risks and aims to achieve responsiveness in software processes to offer a high software service quality. Multi-agent can be applied as a technique to carryout collaborative supporting for risk assessment [10]. The proposed model identifies measures and develops risk control strategies to assess technical, organizational and environmental risk that occurs in managing software in a collaborative approach and providing support to the practitioners. The paper is organized as follows. Tn section-IT we present the literature review. Tn section-ITT we present the methodology. Tn section TV we have presented the risk assessment model. Tn section V presents the discussion section and VT concludes this research paper. IT.

LITERATURE REVIEW

This section explains on risk, risk assessment in software management. A.

Risks in Software Management There are three types of risk that occurs in software management namely; technical, organizational and environmental risk [3]; [7]; [8]; [9]; [6]. These risks have two important characteristics namely, uncertainty and loss. Uncertainty is the risk which mayor may not happen; loss is the risk which will result in unwanted consequences or losses ifit becomes a reality [11]; [12]. Table 1 shows the different types of risk that occurs in software management.

Technical risk

B.

Risk Assessment in Software Management Risk assessment is used to prepare a plan for handling and minimizing the adverse effect of the risk. It can be done by using controlling, avoiding or transferring the risk. Risk assessment consist of the various activities including planning risk control measures, implementing risk control measures, monitoring the risk, controlling the risk, learning on risk. Risk assessment practices in software management are the art and science of planning and leading software projects. It is a subdiscipline of project management in which software projects are planned, monitored and controlled. Tn software management risk assessment is needed in order to recognize, describe, and analyze the possible risk that could affect the organization' s assets. Risk assessment in software management involves the application of appropriate model's, tools and procedures to contain risk within acceptable limits by identitying, addressing, and eliminating the identified risk before they cause adverse harm to software project. Risk assessment provides a disciplined environment for collaborate, proactive decision-making to assess continuously what can go wrong (risk); determine what risks are important to deal with; implement strategies to deal with those risk. Risk assessment in software management aims at early identification and recognition of risks and then actively changes the course of actions to control and solve the risk. Thus it's a good practice to identity measure and review its probability of occurrence, estimate its impact, and set up a contingency plan [12]. C.

TABLE I. TYPES OF RISK IN SOFTWARE MANAGEMENT

Type of Risk Organizational risk

Environmental risk

Description Organizational risk involves potential budgetary, schedule, personnel, resource, stakeholder and requirements problems and their impact on a software project due to poor communication and organizational structure. Organizational risk threatens software project plan. Thus if the project risks become real, it is likely that software project schedule will slip and as a result cost will also increase [II]. Technical risks identifY potential design, implementation, interface, verification and maintenance problems risk management aims at early identification and recognition of risks and then

218

actively changes the course of actions to asses and reduce the risk. Technical risk is mostly related to technological factors that may cause the software product to be overly expensive, delivered late or unacceptable to the end users. Therefore technical risk threatens the quality and timeliness of software to be vroduced [ll i. Environmental risk results from rapidly changing environment and problems with external relationship with software practitioners and/or users. Environmental risk hindered development, use, and maintenance of IT infrastructures in developing software [3]; [7]; [8]. Thus environmental risk threatens the normal growth and development of software to be built and often threaten the software vroduct or project [II].

Existing Risk Assessment Models/Framework Although so many models have been proposed and there are few published models that assess the risk of software management. Among the existing models/framework, some of them have been selected for detailed comparison in this study. The researcher used agents as technique in assessing risk. [12] proposed a model in risk assessment using intelligent agents during risk identification phase. The model consists of four conceptual components namely, performance element, learning element, problem element and critic. The learning element is accountable for making improvements whereas the performance element is accountable for selecting external actions. The learning element uses feedback from the

critic on how the agent is doing and determines how the performance element should be modified to do better in the future. The performance element takes in instructions and decides on actions. The performance element consists such as the requirement specification, stakeholders ' inputs, risk history and procedures which can handle each risk separately. [10] developed a framework for the design of a multiagent based decision support system for the management disruptions and assessment of risks in information technology. The researcher argued that the use of multi -agent modelling can be an alternative decision making tool for collaboration within enterprise. These agents interact and cooperate with other agents, within and a cross organizations, in order to solve problems beyond their individual knowledge or expertise, and to promote a higher performance for the entire system. The framework consists of three conceptual components namely disruption risk management, event management and production fulfilment. [13] presented a multi agent based virtual enterprise risk model to provide support for decision makers. The application of agent technology can reduce the labor intensity of the work involved in information management. The model can also provide decision support for policy makers. Tn order to require the ability to learn using agents can improve system performance. The system model comprises of four components namely risk data management, risk task management, coordination and online monitoring. [14] proposed a multi agent system for web based risk model in small and medium business (SMB). The model uses multiagent system, which is a type of agent with advanced capacities for reasoning to make predictions based on previous experiences. This agent type is used to implement an evaluator agent specialized in detect risky situations and an advisor agent aimed at providing decision support facilities. The multi-agent system provides a web system interface to facilitate the remote interaction with the human users involved in the risk assessment process. [4] developed an intelligent risk assessment and management model for software development. The goal of the model is to enable engineers, managers, and other decision makers to identity, sufficiently early, the risks associated with development, integration, and software acqUIsItIOn, deployment so that appropriate management and mitigation strategies can be developed on a timely basis. The model assists in software process by in cooperating intelligent agents that carryout risk prevention, risk assessment and correction and ensure safe system failure. The intelligent agent model requires the ability to learn, which is the ability to acquire knowledge and then re-use the knowledge to effect a change in behavior. [15] presented an agent based risk management tool for concurrent engineering projects that supports project risk management as an iterative and continuous process across large-scale collaborative engineering teams throughout the lifecyc1e of projects. Tn particular, the researchers demonstrate how a flexible and extensible framework of agents is constructed to form an intelligent risk mapping and assessment system.

[16] developed IRMAS as an agent-based project risk mapping and assessment framework in a web-based project collaborative workbench that aims to support such requirements in addition to maintaining data consistency and coordination. Using the workbench, users define the characteristics of projects, select appropriate mitigation strategies, as well as review risk plans. In addition, the workbench facilitates communications and promotes interactions between project participants who are distributed across geographical regions. [17] designed a generic conceptual model of risk evaluation in order to manage the risk through related constraints and variables under a multiagent collaborative design environment. The model used an intelligent data based reasoning methodology to deal with risk assessment by combining inductive learning methods and reasoning consistency algorithms with feasible solution strategies. The effectiveness of the model aiming for risk assessment in software projects is determined by the degree of data sharing and reuse.

TTT.

METHODOLOGY

Secondary and primary data was used in this research. Secondary data was collected from reviewing existing literatures of risk assessments relating to software process. Primary data was collected using interview. This research adopted case study to find out the problems and how risk is assessed by IT practitioners in their respective organisation. The respondents from the interview session were chosen based on purposely sampling. The analyses of the interview session were analysed using Nvivo software package. The interview transcripts were analysed to find the risk assessment process and technique use in assessing risk. Figure I show the methodology followed in this research.

1)

2)

219

Phase 1- Literature Review: this phase involves reviewing existing research on risk assessment to identity the current issues on risk assessment in software management and the techniques used to solves such Issues. Phase 2- Data Collection: this phase is mainly the interviewing of IT practitioners in their organisation on how they assess risk. For sampling strategy purposely sampling is used, in which the respondents (IT practitioners and IT experts) for the interview are selected based on their idea, knowledge and experience in risk assessment practices. The interview is considered openended because even though the questions can be scripted, the interviewer usually doesn't know what the contents of the response will be. The interview questions focus more on the participant's experiences, knowledge, skills, ideas and preferences on how risk are solved in their organisations. The targeted population for this interview is experienced IT professionals that have in depth knowledge of risk assessment and management. Additionally, it is assumed that the data derived from the

3)

4)

responses given by this sample is sufficient to achieve the goal of developing a risk assessment model. Phase 3 - Data Analysis: this phase involves interpreting the interview transcripts using Nvivo software to identify the problem faced by the practitioners and the code the risk assessment practices. This phase also confirm the data collected from the literature review to the data collected from the interview session. Thus the arrow points back from phase 3 to phase 1 as seen in figure 1. Phase 4 - Proposed Model: this phase is the development of the risk assessment model using knowledge codification multi software agents as technique to design the model to aid in collaborative support among software practitioners.

results from the interview sessions based on risk assessment model functions and risk assessment process/technique. TABLE 2. RESULTS FROM INTERVIEW TRANSCRIPTS

Risk Assessment Practices Results on risk assessment model function

Results on risk assessment process/technique

A.

Data Collection Qualitative data was collected to find out problems faces by IT practitIOners in assessing risk in their organisations and also the practices of risk assessment. Interview was used as the instrument for collecting data on risk assessment practices. Experts are selected using purposely sampling based on their experiences in software process and/or risk assessment. Experts with more than 10 years of experience in dealing with software development projects or who are seek advice from when dealing with software development projects, are chosen. There is no standard on the number of experts that is required in an interview since the decision on the number of experts needed is taken after consideration of factors such as time and expenses. Study has suggest that a suitable expert size from 3 -3000 [19]; [20]; [21 ].

Phasel

Data Collection

Findings I. 2.

Provide collaboration Supporting practitioners in carrying out risk assessment. 3. Provide knowledge and training via e -tutorial. 4. Stimulate the risk! predict the risk effect for the organisation business continuity management. 5. Able to adapt to change in technological specification 6. To provide solution based on prior cases. Risk Rating / Screening Examination of risk drivers / Assumption risk analysis / Review Solutions / Benchmarking / Cost benefit analysis / Benchmark to state mission Knowledge Mapping / Standard risk management plan / Milestone tracking

TV. PROPOSED MODEL

Figure 2 shows the proposed risk assessment model. The model utilizes multi software agents and the codification of knowledge to support software practitioners in assessing risk by reducing the time it takes practitioners to measure the risk, promote interactions and providing collaborative support among software practitioners III software management processes. Risk Identification

Risk Measurement

Phase 2

Organisational, Technical and Environmental risk

support l ___________________ _ ____ _

I Risk prioritizat~

Figure I: Research Methodology Process

B.

Data Analysis 3 respondents were interviewed based on how they assess risk in their organisation. The interview was carried out in 2 different organisations. The 3 respondents were chosen based on their experience in risk assessment. Each interview lasted for a minimum of30 minutes in the respondents ' organisation. The three respondents were head ofTT and IT administrator in their organisations. Risk assessment process to be carried out in their organisation is headed by them. The analysis was carried out using Nvivo software package. Table 2 shows the

Figure 2: Risk Assessment Model in Software Management

As seen above Figure 2 shows the process involved in the proposed risk assessment model;

220

A.

Risk Assessment Process This section describes process/module description; 1)

2)

3) 4)

5)

6)

the

risk

assessment

Risk ldentification- This phase involves the context establishment of the risk assessment process, by beginning the process with team training. This training includes instructions to be applied in risk assessment. This phase aims at timely identification of potential risk and their impact on software process success. Risk Measurement- assesses the magnitude for each identified risk. This phase measures the risk by reducing the time it takes practitioners to measure the risk, promote interactions and providing collaborative support. Risk measurement defines classes of risk by selecting an appropriate, effective risk scaling algorithm; based on the identified risk. This phase provides input to the risk pnontlzation and control phase for final risk quantification and formulation of the best assessment strategy. Typical inputs in this phase include the probability of a risk factor occurring with potential effects, the severity ofthese effects ofthe risk. Risk Prioritization- produces a ranked ordering of the risk items identified and measured. The measured risks are ranked based on a qualitative or quantative scale. Risk Control- involves dealing with the risks inherent in a project and thereby exercise better mitigation of risk in managing software process and increases its chances of success. Risk Review- this phase involves communication, consultation and resolution of risk. Risk Communication helps prepare practitioners to address each risk including the coordination of individual risk with each other and with the overall project plan. Risk consultation involves tracking the risk assessment progress toward resolving its risk and taking corrective action where appropriate, mostly through milestone tracking. The last phase in risk review is risk resolution which develops a medium in which the risks are resolved. Multi Software Agents- Tn software engineering, an agent can be defined as a software entity, which is autonomous to accomplish its design objectives, considered as a part of an overall objective, through the axiom of communication and coordination with other agents. Multi software agents can facilitate autonomous risk assessment; each agent can be responsible for specific activities and performing different roles. These agents interact and cooperate with other agents, in order to solve problems beyond software team members ' knowledge or expertise, and to promote a higher performance for the entire risk assessment system [10]. The application of software agents can decrease the labor intensity of the processes involved in risk assessment. Multi-software agents can also provide decision support for policy makers; and the agent ' s leaming ability can improve system performance. Use of multi software agents in a risk assessment can foster and ensure

7)

8)

collaborative support among software practitioners. The multi software agents have the potential to proactively assess technical, environmental and organisational risk that occurs in software management. By assessing these risks, end users' e.'\pectations o n quality software can b e met while saving the costs and time. Agent based model builds the foundations for risk assessment for collaborative support, through the facilitation of multi software agents and utilization of previous successful cases for risk assessment. Knowledge Codification- Knowledge codification is the process of conversion of knowledge into messages which can be then processed as information. The codification process entails high initial fixed costs but allows agents to carry out certain operations at very low marginal costs. Thus codification of knowledge can reduce the costs of knowledge acquisition. Tn a general sense, codification reduces the costs and improves the reliability of information storage and retrieval. Tn principle the knowledge can be stored and retrieved indefinitely [18]. Knowledge codification is incorporated as a technique in the risk assessment model, thus in developing the risk assessment model for in software management, practitioners acquire knowledge by the capturing, coditying, storing and distribution of knowledge, especially through the development of explicit policies and practices to guide other practitioners and decisionmakers. Thus the ability to codity knowledge is linked to the social practices of creating and sharing knowledge for collaborative support. Equally, practitioners can codity their knowledge of risk assessment into formal structures for managing software process [27]. Organizational, Technical and Environmental risk- this are the risk that occurs in software management. These risks are to be assessed by the practitioners when managing software. V.

DISCUSSION

Risk is considered as possibility of occurrence or event/condition that will have a positive or negative effect on a project objective. The real challenge in the present software management practice is effective risk assessment that would improve the success-to-failure ratio of software products. This would require a deep change in the attitude ofthe practitioners towards the way the software is managed. The scope has to be broadened to account for altemative ways of software development and to assess various factors that could cause risk. Thus practitioners must develop a deeper understanding of organisational objectives to be able to recognize risks before they start to adversely influence software process [26]. Software management related risk assessment has gained interest in the academic literature. One of the major steps in software risk assessment is to identity and assess the potential risks. Despite many scholars and practitioners recognizing the risk assessment models in software project management

221

insufficient attention has been paid by researchers to develop a model to assess risk and provide collaborative support [25]. Software practitioners need to ensure delivery of software to cost, schedule and performance requirement. To achieve this involves assessing technical, organisational and environmental risks in software processes from the initial assessment of strategic options through the utilization of the software product. Risk Assessment mainly involves assessing and estimating the magnitude of risk [22]. In order to minimize and control these risks successfully, risk assessment policies and strategies have been developed and implemented in organizations [1]. Efficient collaborative support aids software process to be effective. Software management encompasses the knowledge, techniques, and tools necessary to manage the development of software products [2]. Thus there is need for the proposed risk assessment model in software management. The model uses multi-software agents and the codification of knowledge from the risk repository to reduce the time involved in carrying out risk assessment; thereby providing collaborative support among software practitioners in assessing risk and also promoting interactions between software practitioners who are distributed across different geographical regions. The proposed model assesses risk in software management by identifYing the organizational, technical and environmental risk. Furthermore the model provides collaborative support to measure the identified risk in a short time and hence promotes interaction among the practitioners in assessing the risk. Tn the next phase the measured risk are prioritized based on a scale. Then the risk with more severity are controlled and later reviewed by the practitioners assessing the risk.

internal auditors and management. Future work will include the validation of the risk assessment model using Rasch measurement/analysis and the development of an architecture design to assist in the implementation of an agent based/ knowledge codified risk assessment system in software management. ACKNOWLEDGEMENT

The authors gratefully acknowledged the interview respondents for their time for providing the data needed to proceed with this research. REFERENCE [I]

[2] [3]

[4] [5]

[6]

[7]

VI. CONCLUSION AND FUTURE WORKS

This paper presents a risk assessment model to provide collaborative support among practitioners in software management process. Existing risk assessment models in software process cannot reduce the time involved in carrying out risk assessment; they also don't provide collaborative support among software practitioners in assessing risk and they cannot promote interactions between software practitioners who are distributed across different geographical regions. Secondary data was collected in this research by reviewing the literatures on risk assessment. Primary research was gotten using purposely sampling to interview 3 IT practitioners in 2 Malaysian organisations. The respondents were interviewed based on their experience and skills on risk assessment practices in their organisations. The proposed model could help software managers, quality managers, risk assessment team and software management team in identifYing and assessing technical, environmental and organisational risk that impacts and effects software process and delivery of software development objectives. The model may be valuable for the business and academic circles to follow and refer. It is hoped that this research can offer a guideline for risk assessment suitable for enterprise and researchers and can be used as a reference for

[8]

[9]

[10] [II] [12]

[13] [14]

[15]

222

C. P. Noraini, A. J. Bokolo, N.H. N. R ozi and A.A. M. Masrah, "Ri sk Assessment of IT Gove11lance: A Systematic Literature Review," Journal of Theoretical and Applied Information Technology, vol. 71, 2015,pp.184-193. C. P. Noraini, A. J. Bokolo, "A Review on Decision Making of Risk Mitigation f or S fotware lvlanagement," Journal of Theoretical and Applied Information Technology, vol 76,2015, pp. 333-341. M. A. Khan, S. Khan, M. Sadiq "Systematic Review of Software Risk Assessment and Estimation Models," International Journal of Engineering and Advanced Technology (1.IEAT), vol. I, 2012, pp. 298305. D. John, N. Isaac and K. Admire, "Intelligent Ri sk Management Tool s for Software Development," SACLA, AClvl, 2009, pp.33-40. A. S. lVlohamud, and S. Basri , "A Study on Ri sk Assessment for Small and Medium Software Development Proj ects," Intemational Joumal on New Computer Architectures and Their Applications (IJNCAA), vol I, 2011, pp. 325-335. G. Daya, and S. r,/lohd, "Software Ri sk Assessment and Estimation Model," Inte11lational Conference on Computer Science and International Technology, IEEE Computer Society, Singapore, 2008, pp. 963-967. P. Sneh, and R. Ujjawal, Software Risk Evaluation and Assessment using Hybrid Approach. National Workshop-Cum-Conference on Recent Trends in Mathematics and Computing (RTMC), International Journal of Computer Applications (IJCA), vol 1,2011, pp. 6-8. S. Mohd, and W. A. Mohd, K.I. Rahmani and S ..lung, "Software Risk Assessment and Evaluation Process (SRAEP) using Model Based Approach," Inte11lational Conference o n Netwo rking and Infonnation Technology, IEEE, 20 I 0, 171-177. N. MinaI and V. Bha sin, "Software Ri sk Assessment and Estimati on Models - A Review," International Journal of Computer, Communication and Information Technology (IJCCIT), v01.1, 2013, pp 59-65. G. Mihalis and L. Michali s, "A multi-agent based framework for supply chain risk management," Jou11lal of Purchasing and Supply Management, vol 17, 2011, pp. 23-31. S. Bharat, D. S. Kapil, and C. Subhash, "A New Model for Software Ri sk Management," Int.J.C omputer T echnology & Appli cations, vol 3, 2012, pp.953-956. P. Shikha and R. Selvarani, "An Efficient Method of Risk Assessment using Intelligent Agents," 20 12 Second Inte11lational Conference on Advanced Computing & Communication Technologies IEEE, 2012, pp. 123-126. S. Xianli, M .Huang and X. Wang, "Web and Multi-agent Based Virtual Enterprise Ri sk Management System," IEEE 20 11 Chinese Control and Decision Conference (CCDC), 2011, pp. 902-906. B. Javier, L. B. Maria, P. Juan, M . C. Juan and A. P. Maria, "A multiagent system for web-based risk management in small and medium business," Jou11lal of Expert Systems with Applications, vol 39, 20 12, pp. 6921-693 1. Y. B. Khoo, M. Zhou and B. Kayis, " An agent-based risk management tool f or c noCUlTent engineering projects," complexity Intemati onal journal, vol. 12, 2005, pp. I-II.

[16] LB. Kayis, M. Zhou, S. Savci, YB. Khoo, A. Ahmed, R. Kusumo and A. Rispler, " IR11AS - development of a risk management tool for collaborative multi-site, multi-partner new product development proj ects," Joulll al of Manufac turing Techn ology Management, vol. 18, 2007, pp. 387-414. [17] 1. Ruan and S.F Qin, "A Generic Conceptual Model for Risk Analysis in a Multi-agent Based Collaborative Design Environment," Proceedings of the 19th CIRP Design Conference - Competitive Design, Cranfield University, 30-31 March 2009, pp.193. [18] D. Pratilll and W. Acar, "Software and hum an agents in Kn owledge Codification," Mowledge Management Research and Practice, vol 3, 2010, pp. 45-60. [19] J.T. S. Moorthy, S. B Ibrahimn and M. N. l\!lahrin, T ' he Need for Usability Risk Assessment Model," SDIWC, 2013, pp. 215 -220. [20] K. K. Lilja, L. Kimmo, and 1. Palomki, "Using the Delphi method," 2011 Proceedings of PICMET'I1: Technology Management in the Energy Smart World (PICMET), 2011, pp: 361-367. [21] S. Roy, L., K. K. Mark, and C. Paul, "IdentifYing software project risks: an international Delphi study," Journal of management infofll1ation systems, voI17,2001,pp. 5-36. [22] D. Gupta and M. Sadiq, " Software Risk Assessment and Estimation Model," International Conference o n C m o puter S cience and Infol1nation Technology IEEE, 2008, pp. 963-967. [23] A Davide, R. Dulmin and V. Mininno, "Risk Assessment in ERP projects. Infofll1ation Systems," 20 12 doi:l O.1 0 16/j.is.20 11.1 0.00 1, pp: 183-199. [24] F. Say-wei and A. Muruganantham, "Software Risk Assessment Model," International Journal of Digital Infol1nation and Wireless Communication (IJDlWC), vol 4, 2005, pp. 95- 102. [25] A KarimiAzari, N . Mousavi, S. F. Mousavi and S. Hosseini, "Ri sk assessment model selection in construction ni dustry," Expert Systems with Applications, vol 38, 2011, pp. 9105-9111. [26] 1. Miler and 1. Gorski, "Software support for collaborative risk management," proc. of 8th International Conference on Advanced Computer Systems, October 17-19, 2001 Mielno, Poland, 2001, pp. 1-9. [27] W. L. Currie, " A Mowledge-Based Risk Assessment Framework for Evaluating Web-Enabled Application Outsourcing Proj ects," International Journal of Project Management, vol. 21, 2005, pp. 207217.

223