Risk framework for outsourced strategic IT system development ... - DPO

Proceedings 5th Software Measurement European Forum, Milan 2008

Risk framework for outsourced strategic IT system development from the client perspective Lili Marziana Abdullah, June M. Verner

Abstract While a series of risks connected with IT development and IT outsourcing are identified in the literature, research on risk in the context of strategic IT system development outsourcing is almost non-existent. This study investigates risks that are most likely to cause failure in a strategic IT system development outsourcing project, from the client perspective. This study examines strategic IT systems from a business strategy perspective and presents the characteristics of a strategic IT system by synthesising material from the management literature and the IT literature. Three different streams of literature, namely IT development, strategic IT, and IT outsourcing, and reported examples of strategic IT system development outsourcing project failure are examined. Based on the risks in the literature and using IT outsourcing stages as a basis, a risk framework is developed and tested in a preliminary case study. The case study investigates the development of a strategic system jointly developed by an in-house IT team and outsourcing vendors in Australia. The risk framework developed and tested here will assist clients to focus their attention on risks that can lead to failure of the outsourced strategic IT system developments.

1. Introduction IT plays an increasingly strategic role in the business performance of organisations. News of organisations venturing into development of strategic IT systems is a common occurrence [16, 37, 49]. Because of confidentiality and security issues it has been suggested that only non-strategic IT applications should be outsourced, and strategic IT applications should be retained in-house or in-sourced [23]. Furthermore, outsourcing strategic IT applications require extensive and specific knowledge of the business and its needs [29], and because of greater dependency on the vendor [23], it has been regarded as unwise to outsource such systems. However, there is a noticeable move towards strategic IT applications outsourcing [43, 55], and, for an organisation where its in-house IT department is incapable or unable to develop strategic IT systems, outsourcing to a vendor may be the only option. The increasing perception of the vendor as a source of resources, capabilities and expertise, and increasing confidence from clients in vendors’ competence in developing strategic IT systems has led organisations to pursue strategic IT applications outsourcing, and more outsourcing of such systems is expected to occur [18]. A series of risks connected with IT development, IT outsourcing and strategic IT projects from case studies, surveys, focus groups and accounts of developer experience have been described in the literature. Most of the work on risk has been for general IT systems development or for software development projects [6, 20, 30, 33, 34, 44, 50, 51]. Only a few researchers have discussed risks in the context of strategic IT [12, 17, 22, 51]. Much of the research on IT project risk has centred on the vendor side, identifying risks that can threaten in-house systems or software developments. Owing to the increasing level of IT outsourcing [25, 32], the growing trend of IT outsourcing to different countries [27, 46] and most importantly, the growing concern with failure and dissatisfaction of IT outsourcing arrangements [52], research on risk in IT outsourcing has recently increased. 1

Proceedings 5th Software Measurement European Forum, Milan 2008 However, the scope of IT outsourcing is generalised to a broad view of IT outsourcing which includes software development, software maintenance, support operations, disaster recovery and data centre operations [2-4, 15, 21, 26, 46, 54]. Overall, only Natovich [36] gives a specific account of a strategic IT development outsourcing project but his discussion of risk merely focuses on the vendor’s ability and willingness to fulfil its obligations and is without an analysis of strategic IT implications. Most of the research is from the client perspective with only Taylor [47, 48] taking the vendor’s view. It is evident that research on risk in the context of strategic IT development outsourcing projects is almost non-existent with little recognition of risks of outsourced strategic IT development projects. A client’s understanding of the risks when outsourcing a strategic IT development project can not only ensure success with the achievement of the business goals, but may also help to foster better cooperation and partnership with the vendor(s). Hence, the primary motivation for the study presented here is to build on previous research and to provide a much needed practical means for clients to identify the risks most likely to cause the failure of a strategic IT system development outsourced project. Risk in this study is defined as a condition that may lead to the failure of the outsourced strategic IT system development project to deliver the intended or promised benefits to the client. Failure here is accepted to mean any of the following [42]: x The client’s real requirements are not met by the system. x The system does not work as intended even though the requirements are met. x The system is not delivered when actually needed by the clients. x The system is difficult to use and hence, not used. A completed project may be a failure or a failure may refer to a cancelled project. We next examine strategic IT systems from a business strategy perspective and present the characteristics of a strategic IT system by synthesising management and IT literature; such definitions are currently lacking. The development of a risk framework and a preliminary case study which tests the framework are described in Section 3. The results of the case study are presented in Section 4. We conclude with suggestions for further research in Section 5.

2. Strategic IT Systems The characteristics of a strategic IT system, addressed by this study, apply mainly to forprofit organisations. Hence, for the purpose of this work an IT system is a strategic IT system if it meets all of the following criteria: 1. It can be any type of hybrid IT system that may comprise internally or externally focused systems but is either applied to processes or practices that support the business’s core product or service, or is part of the core product or service, that the organisation provides to its customers. 2. It improves the competitive position of the organisation relative to its competitors and consequently, contributes to the growth, earnings or other aspects of profitability of the organisation. 3. It enables the organisation to differentiate itself from its competitors in the industry. 4. It enables the organisation to sustain its competitive advantage for a long enough period before being replicated by competitors.

2

Proceedings 5th Software Measurement European Forum, Milan 2008 Based on Porter’s [41] view of strategy and Oliver’s [38] working definition of strategy, strategy provides the means for an organisation to improve its competitive position or ability to compete with its competitors. An organisation possesses competitive advantage over its competitors if it sustains growth and earnings ahead of the average of all other organisations competing in its industry [19, 24, 40]. In essence, the ultimate aim of strategy is to enable the organisation to achieve sustainable competitive advantage – an ongoing state of competitive advantage. Strategic IT systems can either be those that are built for internal purposes or those that transform or make a difference in the ways an organisation competes. An IT system only become “strategic” when it contributes directly to the organisation’s business strategies and affects its competitive position. Strategic IT systems can have great impact and provide significant returns to an organisation. However, developing such systems involves a certain level of innovation and entails high risk [11, 53].

3. Method IT project risk has been receiving much attention recently. Review of the research in IT development, strategic IT, and IT outsourcing, uncovered a number of authors who identify risks that may occur during IT development projects. Although the risks were discussed in different contexts, these sources are useful as they allow us to develop an initial risk framework for outsourced strategic IT developments. However, while risk from these sources refers to negative consequences, not all risks were reported as contributing to the failure of a project. Hence, accounts of outsourced strategic IT system development project failure reported in the literature were investigated. Only IT systems that conform to the proposed characteristics of strategic IT systems were examined. Risks were then evaluated against project failure and the failure factors existing in the literature, to determine which risks are most likely to contribute to outsourced project failure. Using the risks in the literature and the different IT outsourcing stages as a basis, a risk framework was developed and tested in a preliminary case study with a strategic IT system.

3.1. Risk Framework The characteristics of a strategic IT system were used as the basis for identifying cases in the literature that can be classified as strategic IT systems development. Drawing on the cases of failed strategic IT system development that involved a major outsourcing activity [7, 9, 10, 14, 28, 39, 45, 56], the failure factors that emerge are various combinations associated with the following three major factor groups: x Client. x Vendor. x Project. Examination of risks from IT development, strategic IT and the IT outsourcing literature show a similar association with these three sources. Since many of these risks also apply to outsourced strategic IT system development, the risks are synthesised and evaluated based on their possible relevance. The resulting risks are then grouped according to these three categories and later, compared against failure factors identified from cases of outsourced strategic IT system failure. Categorising risks into three groups helps to establish which risk falls under the responsibility of the client or vendor or both. There are similarities between failure factors and risks in the literature but the cases provide further insight into how the risks lead to project failure and the subtle differences arising from different contexts.

3

Proceedings 5th Software Measurement European Forum, Milan 2008 Table 1 shows the risks identified from the literature that aligns with the failure factors. The resulting list of risks from the literature is not included in this paper due to space constraints but is available from the authors.

A. 1.

2.

3.

4.

5.

6. 7. 8.

9.

10.

11.

Table 1: Risk list Risks associated with Client and Vendor Client has unrealistic expectations about what can be achieved. This includes having multiple objectives for the outsourced project and ambitious use of bleeding-edge technology. Client has insufficient financial reserves. Implementing a project requires substantial budget and this risk concerns the lack of client’s capital to fund the project and cover hidden service costs. There are changes in client’s and/or vendor’s business model that affects the project. Decline in the strategic importance of a client’s business environment may result in de-escalation of a vendor’s commitment to the project. The business environment may change dramatically forcing adjustments in business strategy and the system is no longer deemed functionally appropriate. Disagreements among client’s internal groups or departments. Serious differences in requirements, design, deliverables etc., among the project stakeholders. Client’s weak management. Staff members are not capable of managing the contract and relationship. This includes poor project management, failure to recognise project threats, lack of vendor governance and not involving the real users. User’s resistance to change. Users are not willing to cooperate during development or use the system. Adversarial relationship between client and vendor. This can lead to disputes, which may lead to a mutual lack of trust and escalating costs. Client lacks experience with outsourcing. Clients who lack experience with outsourcing may underestimate contracting costs, project setup costs and management costs. This also affects the monitoring abilities of the client. Client and/or vendor lack experience and expertise with outsourced project activities. There is a possibility of incurring unexpected costs if the client does not have enough knowledge of the activity to provide the vendor with a complete description of the task to be carried out and to clearly communicate its needs. Additionally, lack of experience and expertise can impact the ability of the client to adequately manage the contract since the client will have difficulty in assessing the quality of work done. A lack of experience and expertise by the vendor may be a cause of poor service quality and may expose the client to competitors which in turn affects the client’s profitability. Client and/or vendor lack technical skills. The client’s IT department may not have the skill levels needed to cooperate effectively with the vendor. On the other hand, the vendor may actually lack the skills to select and adapt technology appropriate to the task or to carry out the real task. Size of client and/or vendor organisation. A vendor of small size may not have the necessary resources for the project and cannot ensure an appropriate level of service. 4

[44, 54]

[6, 27]

[12, 36]

[44]

[15]

[51] [36] [4, 5, 15]

[2, 4, 5, 15]

[17, 26, 47]

[2, 4]

Proceedings 5th Software Measurement European Forum, Milan 2008 12.

13.

14.

B. 15.

16.

17.

18.

19.

20. 21.

22. 23.

24.

High turnover in either client or vendor staff. Once a development project is outsourced, client staff may see their own value to the company declining and this may lead to low morale. Biased portrayal by vendor. A vendor may exaggerate its abilities and resources, or the vendor was overoptimistic about its true capacity to adequately do the project. Vendor lacks understanding of the capabilities and readiness of client. The vendor understands little about the capabilities and readiness of the client for the implementation of a new system. This includes the organisational culture of the client that may be very different from that of the vendor. Risks associated with Project Interdependence of activities. The performance of one piece of work depends either directly or indirectly on the completion of other pieces of work done in-house, outsourced or sub-contracted. It can be difficult to address the interfaces, but if not addressed properly, the conduct of the client’s business is disrupted. Too much technological indivisibility. Current systems are very integrated or interconnected and problems can occur in terms of responsibility between different vendors or between vendor’s and client’s domain. Significant integration and customisation required. Many strategic systems include different combinations of packages and systems. This complicates the task and makes cost assessment more difficult thus increasing the probability of hidden service costs. Poorly articulated requirements. The client has difficulties in defining the scope and requirements of the project and this leads to constant negotiation and revisions. Miscommunication and misperceptions can result in people not sharing or communicating ideas. Project has unrealistic schedule and budget. This includes setting impractical deadlines or functionality expectations in a specified time period, and setting the budget before defining the project, or without considering the scope, or poorly estimating the scope. Uncertainty about the legal environment. This can lead to unexpected management costs, disputes and litigation. Unmanaged scope creep. Continual requests for changes can give rise to disputes and even litigation. Increasing scope may result in losses for the vendor with a fixed price contract and leads to de-escalation of vendor’s commitment to the project. Vendor’s noncompliance with specified methodologies. Expected methodologies are not used in practice. Lack of audit and control. The client must continuously monitor the performance of the system and vendor. The need for audit and control becomes more critical since corporate information and confidential information are accessible to vendor. Level of change required to processes, practices, workflow, structures etc. This involves the extent of changes that the system development will require to the client’s processes, practices, workflows, structures etc.

5

[27, 46]

[2, 4]

[47]

[2, 4, 5]

[15]

[4, 47]

[27, 36]

[8, 44]

[4] [27, 36]

[27] [46]

[6, 35]


3.2. Preliminary Case Study In order to test our framework and to better understand risks that are significant and require close attention by a client of an outsourced strategic IT development project, a preliminary case study was conducted. The case study also offers an opportunity to examine the risks according to the stages of an IT outsourcing process shown in Figure 1 [1, 13, 31].

Figure 1: IT outsourcing process The study conducted is an electronic bill payment development project, a strategic initiative of a group of Australasian financial institutions. Data were collected from a face-toface semi-structured interview with the chief technology officer (CTO) who has thirty-five years IT experience, and who was involved in the decision making process and oversight of the project. In order to ensure that the system studied was an appropriate strategic IT system, questions related to the characteristics of strategic IT systems were asked. Responses to the questions confirmed that the project reviewed involved the development of a strategic IT system. The project, which aimed at providing payment services for 190 of its client members, was considered a first in financial services in Australasia. It used a technology that was new at the time. The project involved fifteen internal staff members, who acted as the system integrator, and several hardware and software vendor companies. The prime vendor for the project located in Australia, was responsible for the back-end web based part of the system. The project took nine months to complete and was regarded as a success. The interviewee was more than ready to talk about a successful project at this initial stage. He highlighted several risks that were critical and that had to be mitigated to ensure successful completion and thus, required close attention by the client during the course of the project.

4. Results This study revealed a close match between the risks highlighted by the interviewee and several of the risks in Table 1. Which risks were perceived as significant to the organisation, as the system integrator, as well those relating to the client and how these risks relate to the IT outsourcing phases and what risk-mitigating practices were adopted by the organisation were discussed.

6

Proceedings 5th Software Measurement European Forum, Milan 2008 Table 2 shows the practices identified and the risks that were mitigated. Table 2: Risks and mitigating practices Pre-contract Client has unrealistic Organisation was clear on the type of system needed and expectations about what can be the scope of work to be achieved within the estimated time achieved. frame. Though the object-oriented technology that it wanted had not been deployed by the financial services organisation before, it had been used in other types of systems. Poorly articulated requirements A clear understanding of the organisation’s and clients’ needs enabled the requirements, in particular, the back end system to be specified in detail in a request for proposals. Client’s weak management The project manager appointed was an experienced project manager, and effective in managing tasks and governing the team and vendors. The CTO emphasised the importance of appointing a project manager with a business background rather than a technical background. He felt that a project manager with a technical background is more likely to be interested in experimenting with possible technical solutions and this can lead to scope creep and a lack of focus on the deliverables. These three risks can be interrelated since a vendor who x Biased portrayal by vendor x Vendor lacks experience and exaggerates, or is overoptimistic about its capacity, may expertise with outsourced turn out to be lacking in experience, expertise and the skills to fulfil the task. The organisation went through a project activities x Vendor lacks technical skills tender process to select the vendor. Final selection was based on the most cost-effective solution, and even though this was the first banking project for the vendor, the vendor had experience with big, complex projects e.g., in the defence sector. This increased the confidence of the client organisation. In addition the client was only interested in vendors who responded quickly to the proposal noting “if they had the expertise and were confident, they would respond quickly. If they were unsure about their capabilities and finally decided they might be able to do it they would take longer.”

7


Contract Client and vendor are unclear about their roles and responsibilities Poor or incomplete contracting

The roles and responsibilities of the internal staff and the vendor, together with the expected deliverables, were clearly specified in the contract. A comprehensive contract was developed and adhered to. The CTO also stressed the need to get an agreed set of requirements before starting the project and defined sets of testing data at an early stage. Failure to do so may lead to scope creep. His contract covered the definition of requirements with milestones, penalties for nonperformance and non-delivery, and post-outsourcing activities and support.

Post-contract Adversarial relationship The organisation closely monitored the vendor even between client and vendor though the vendor was not on-site. Regular face-to-face meetings were held with the vendor. Any arising issues were resolved immediately. Unmanaged scope creep Requirements had been agreed and were detailed in the contract, and delivery was monitored throughout. Changes to the back-end components of the system were not required. However, there were frequent changes made to the system interfaces but these were well managed. Lack of audit and control Because of the new technology employed and a few uncertainties with technical implementation of the requirements, the organisation conducted frequent scenario testing to test functionality as the development progressed. It was interesting to note that several of the risks in Table 2 are interrelated and some practices of the organisation at the early stages of development were crucial in mitigating risks that may occur at a later stage of the project. Though these risks are of interest to the client, the owner and the responsibility for managing the risks from the contract phase and beyond, belonged to both client and vendor. In addition, the risks concerning the contract, which were identified in the IT outsourcing literature, were not discussed in the literature that dealt with cases of outsourced strategic IT failure but were highlighted as critical factors in our discussion. The study also drew attention to the following important lessons learned for a strategic IT development outsourcing project: 1. Lock in a clear set of requirements in the contract at the start of the project right down to test data. 2. Appoint a capable project manager with a business background. 3. Conduct regular testing. However, more studies with varied project outcomes will follow to enable a richer study of risks.

8


5. Conclusions The study reported here has identified some critical risks that clients should be aware of when outsourcing a strategic IT system development project. The IT development, strategic IT and IT outsourcing literature provided an extensive set of risks but less is known about risks in cases where a strategic IT development project includes outsourcing activities. Examples of failed strategic IT development projects have provided some understanding of the applicability and distinctiveness of some of the risks in the literature with respect to outsourced strategic IT development projects. Further research is needed to gain better insight into risks that are of importance to clients of outsourced strategic IT system development initiatives and this paper represents a first move in that direction. Our preliminary case study highlighted risks that were identified as important to a client organisation in a strategic IT development project and how the practices of the organisation mitigated these risks. It is important to recognise that during the contract and post-contract phases, the owner of the risks and the responsibility for managing the risks may include both the client and vendor. However, this particular case study is an example of a project with a successful outcome and therefore, more case studies with varied project outcomes especially failed projects are needed to build a richer risk framework. This will assist clients to focus their attention on potential problems that can lead to failure of the outsourced strategic IT system developments. Hence, the significance of the risks discussed in this paper will be further explored and examined through comprehensive case studies.

6. References [1]

Alborz, S., Seddon, P. B., and Scheepers, R., "A Model for Studying IT Outsourcing Relationships", in 7th Pacific Asia Conference on Information Systems, Adelaide, South Australia, 2003, pp. 1297-1313. [2] Aubert, B. A., Patry, M., and Rivard, S., "Assessing the risk of IT outsourcing", in Proceedings of the Thirty-First Hawaii International Conference on System Sciences, 1998., 1998, pp. 685692 vol.6. [3] Aubert, B. A., Patry, M., and Rivard, S., "Managing IT Outsourcing Risk: Lessons Learned", H. a. C. Montreal, Ed., 2001, pp. 1-19. [4] Aubert, B. A., Patry, M., and Rivard, S., "A Framework for Information Technology Outsourcing Risk Management", The Database for Advances in Information Systems, 2005, vol. 36, pp. 9-28. [5] Bahli, B. and Rivard, S., "Validating measures of information technology outsourcing risk factors *", Omega, 04/01/ 2005, vol. 33, pp. 175-187. [6] Barki, H., Rivard, S., and Talbot, J., "Toward an assessment of software development risk", Journal of Management Information Systems, Fall 1993, vol. 10, p. 203. [7] Barret, L., "McDonald's: McBusted", July 2, 2003, http://www.baselinemag.com/article2/0,1540,1631804,00.asp. [8] Boehm, B. W., "Tutorial: Software Risk Management", IEEE Computer Society Press, 1989. [9] Bulkeley, W. M., "When Things Go Wrong: FoxMeyer Drug took a huge high-tech gamble; It didn't work", Wall Street Journal (Eastern Edition), Nov. 18, 1996. [10] Clark, L., "Sainsbury's writes off £260m as supply chain hits profit", Oct. 2004, http://www.computerweekly.com/Articles/2004/10/25/206226/sainsburys-writes-off-260m-assupply-chain-it-trouble-hits.htm. [11] Clemons, E. K., "Evaluation of strategic investments in information technology", Commun. ACM, 1991, vol. 34, pp. 22-36. [12] Clemons, E. K. and Weber, B. W., "Strategic Information Technology Investments: Guidelines for Decision Making", Journal of Management Information Systems, Fall 1990, vol. 7, pp. 9-28. 9

Proceedings 5th Software Measurement European Forum, Milan 2008 [13] Dibbern, J., Goles, T., Hirschheim, R., and Jayatilaka, B., "Information systems outsourcing: a survey and analysis of the literature", SIGMIS Database, 2004, vol. 35, pp. 6-102. [14] Duvall, M., "Airline Reservation System Hits Turbulence", July 25, 2007, http://www.baselinemag.com/article2/0,1397,2162308,00.asp. [15] Earl, M. J., "The risks of outsourcing IT. (information technology)", Sloan Management Review, 03/22/ 1996, vol. v37, p. p26(7). [16] Fujitsu, "Sony Marketing ordered the solution build from Fujitsu", April 09, 2007, http://www.fujitsu.com/vn/en/casestudies/sony/. [17] Gilbert, A. L. and Vitale, M. F., "Containing strategic information systems risk: control and intelligence", in Proceedings of the Twenty-First Annual Hawaii International Conference on System Sciences, 1988. Vol.III. Decision Support and Knowledge Based Systems Track, Hawaii, USA, 1988, pp. 52-60. [18] Gurbaxani, V., "The new world of information technology outsourcing", Commun. ACM, 1996, vol. 39, pp. 45-46. [19] Hamel, G. and Prahalad, C. K., "Strategy as stretch and leverage", Harvard Business Review, 03 1993, vol. 71, pp. 75-84. [20] Jiang, J. and Klein, G., "Software development risks to project effectiveness", Journal of Systems and Software, 2000, vol. 52, pp. 3-10. [21] Jurison, J., "The role of risk and return in information technology outsourcing decisions", Journal of Information Technology (Routledge, Ltd.), 12 1995, vol. 10, p. 239. [22] Kemerer, C. F. and Sosa, G. L., "Systems development risks in strategic information systems", Information and Software Technology, 1991, vol. 33, pp. 212-223. [23] Ketler, K. and Walstrom, J., "The outsourcing decision", International Journal of Information Management, 1993, vol. 13, pp. 449-459. [24] Kettinger, W. J. and Grover, V., "Do strategic systems really pay off?" Information Systems Management, 1995, vol. 12, pp. 35-44. [25] Khan, I., "Outsourcing Not Slowing Down: Study", 2007, http://www.ddj.com/architect/197005853?cid=RSSfeed_DDJ_All. [26] Klepper, R. and Jones, W. O., "Outsourcing Information Technology, Systems & Services", Prentice Hall, New Jersey, 1998. [27] Kliem, R., "Managing the Risks of Offshore IT Development Projects", EDPACS, 2004, vol. 32, pp. 12-20. [28] Koch, C., "Project Management: AT&T Wireless Self Destructs", April 2004, http://www.cio.com/article/print/3228. [29] Lacity, M., Hirschheim, R., and Willcocks, L., "Realising outsourcing expectations. (Cover story)", Information Systems Management, Fall 1994, vol. 11, pp. 7-18. [30] Lyytinen, K. and Mathiassen, L., "Attention Shaping and Software Risk--A Categorical Analysis of Four Classical Risk Management Approaches", Information Systems Research, 09 1998, vol. 9, pp. 233-255. [31] Mahnke, V., Overby, M. L., and Vang, J., "Strategic Outsourcing of IT Services: Theoretical Stocktaking and Empirical Challenges", Industry & Innovation, 2005, vol. 12, pp. 205 - 253. [32] McCue, A., "Global IT outsourcing deals rocket up to $163bn", 2005, http://management.silicon.com/itdirector/0,39024673,39127146,00.htm. [33] McFarlan, F. W., "Portfolio approach to information systems", Harvard Business Review, 09 1981, vol. 59, pp. 142-150. [34] Moynihan, T., "An inventory of personal constructs for information systems project risk researchers", Journal of Information Technology (Routledge, Ltd.), 12 1996, vol. 11, p. 359. [35] [35] Moynihan, T., "How experienced project managers assess risk", Software, IEEE, 1997, vol. 14, pp. 35-41. [36] Natovich, J., "Vendor Related Risks in IT Development: A Chronology of an Outsourced Project Failure", Technology Analysis & Strategic Management, 2003, vol. 15, pp. 409 - 419. [37] NTT Data Corporation, "Basic Outsourcing Agreement Formed with the Nishi-Nippon City Bank for System Development and Operation", October 27, 2006, http://www.nttdata.co.jp/en/media/2006/102700.html.

10

Proceedings 5th Software Measurement European Forum, Milan 2008 [38] Oliver, R. W., "What Is Strategy, Anyway?" Journal of Business Strategy, 11 2001, vol. 22, pp. 7-10. [39] Oz, E., "When professional standards are lax: the CONFIRM failure and its lessons", Commun. ACM, 1994, vol. 37, pp. 29-43. [40] Porter, M. E., "Competitive Advantage: Creating and Sustaining Superior Performance", Free Press, New York, 1985. [41] Porter, M. E., "What Is Strategy?" Harvard Business Review, 11 1996, vol. 74, pp. 61-78. [42] Procaccino, J. D. and Verner, J. M., "Software project managers and project success: An exploratory study", Journal of Systems and Software, 2006, vol. 79, pp. 1541-1551. [43] Rao, H. R., Nam, K., and Chaudhury, A., "Information systems outsourcing", Commun. ACM, 1996, vol. 39, pp. 27-28. [44] Schmidt, R., Lyytinen, K., Keil, M., and Cule, P., "Identifying Software Project Risks: An International Delphi Study", Journal of Management Information Systems, Spring 2001, vol. 17, pp. 5-36. [45] Songini, M., "Media giant BskyB sues EDS over troubled CRM system", Aug. 17, 2004, http://www.computerworld.com/managementtopics/roi/story/0,10801,95316,00.html. [46] Tafti, M. H. A., "Risks factors associated with offshore IT outsourcing", Industrial Management & Data Systems, 2005, vol. 105, pp. 549-559. [47] Taylor, H., "The move to outsourced IT projects: key risks from the provider perspective", in Proceedings of the 2005 ACM SIGMIS CPR Conference on Computer Personnel Research, 2005. [48] Taylor, H., "Critical Risks in Outsourced IT projects: The Intractable and The Unforseen", Communications of the ACM, 11 2006, vol. 49, pp. 75-79. [49] Thibodeau, P., "Starwood Taps HP for System Development, Outsourcing Deal", November 01, 2004, http://www.computerworld.com/action/article.do?command=printArticleBasic&articleId=9... [50] Wallace, L. and Keil, M., "Software project risks and their effect on outcomes", Commun. ACM, 2004, vol. 47, pp. 68-73. [51] Wallace, L., Keil, M., and Rai, A., "Understanding software project risk: a cluster analysis", Information & Management, 2004, vol. 42, pp. 115-125. [52] Weakland, T. and Tumpowsky, B., "2006 Global IT Outsourcing Study", Diamond Management & Technology Consultants, Chicago, Illinois 2006. [53] Weil, P. and Aral, S., "IT Savvy Pays Off:How Top Performers Match IT Portfolio and Organisational Practices", in MIT Sloan Research Paper: Center for Information Systems Research, MIT Sloan School of Management, 2005. [54] Willcocks, L. P. and Lacity, M. C., "IT outsourcing in insurance services: risk, creative contracting and business advantage", Information Systems Journal, 07 1999, vol. 9, pp. 163180. [55] Woollacott, E., "Outsourcing used to drive down costs", 2006, http://www.techworld.com/applications/news/index.cfm?NewsID=5588&inkc=0. [56] Yardley, D., "Successful IT Project Delivery", Pearson Education, Harlow, UK, 2002.

11