The current issue and full text archive of this journal is available at www.emeraldinsight.com/1753-8378.htm
Comprehensive framework for estimating the deployment cost of integrated business transformation projects
Business transformation projects 131
Young Hoon Kwak School of Business, The George Washington University, Washington, DC, USA
Rudy J. Watson International Business Machines (IBM), Bethesda, Maryland, USA, and
Frank T. Anbari School of Business, The George Washington University, Washington, DC, USA Abstract Purpose – This paper is a summary of a successfully defended doctoral dissertation. The purpose of this paper is to place this research in context to emerging areas of project management and service science, management and engineering and to encourage others to embark on further research related to this important topic. Design/methodology/approach – Results reported in this paper were based upon action learning from research in which a project management tool for estimating deployment cost was developed by capturing the knowledge of subject matter experts (SMEs) and subsequently tested against projects from various geographic areas. Findings – There were two primary findings. A development and analysis of the conceptual estimating framework supports the assertion that the use of the framework provides an awareness of the project that may not otherwise be observed or, at best, would be observed later in the life of the project and potentially addressed at a higher cost. A strong association has been found between the conceptual estimate produced by the comprehensive framework and the conceptual estimate produced manually through the use of SMEs. Originality/value – From academic perspective, the synthesis of technology management, business processes, and the conceptual estimating framework enhances the body of knowledge of project management. For practical applications, the method and framework employed can be utilized to build functioning conceptual estimating tools for deployment, which can lead to cost savings during the estimating process and, as this study surmises, will lead to more effective project management, control, and implementation. Keywords Project management, Organizational change, Costs Paper type Research paper
The dissertation is available from the UMI Dissertation Services (http://proquest.umi.com/login). It can be accessed through ProQuest using the dissertation and thesis database (http://proquest. umi.com/pqdweb). Search on “Cost estimating framework for deployment of integrated software application development projects” as the title of the dissertation. It can be located with a subset of the title such as, “Cost estimating framework”. Once located, there is an abstract, a 24 page pre-view, a full text pdf download, and a link for ordering copies.
International Journal of Managing Projects in Business Vol. 1 No. 1, 2008 pp. 131-139 q Emerald Group Publishing Limited 1753-8378 DOI 10.1108/17538370810846469
IJMPB 1,1
132
Summary of the doctoral dissertation Services are an increasing part of the global economy as all national economies are shifting to services and service businesses employ a large and growing share of the science and engineering workforce. There is an emerging academic discipline known as services science, management and engineering (SSME), which addresses the needs of services business. SSME is a multidisciplinary research approach that focuses on fundamental science, models, theories and applications to drive innovation, competition, and quality of life through services (Bitner and Brown, 2006). Services science research yield models, methodologies, processes, and software tools that create and deliver services more efficiently (Bitner et al., 2006). This study develops a framework for producing conceptual cost estimates which combines the knowledge of technology with that of business process and organization in an effort to improve productivity. The research is focused solely on deployment, an area that in the past has primarily been combined with software application development. Deployment, as an entity independent from software development, is a key project phase in the growing field of services (Watson and Kwak, 2004b, 2005b). An important factor in understanding a conceptual cost estimate for deployment is the time frame in which it is created. Figure 1 shows that a conceptual estimate for the deployment phase is made prior to the plan phase of the development project for a software application. An estimating framework lessens the continual need for subject matter experts (SMEs) to be involved in the conceptual cost-estimating process (Kwak and Watson, 2005). This scenario allows an organization to gain the benefit of conceptual estimating for deployment without diverting the SME resources from other development projects. The accuracy of an estimate is primarily influenced by the project stage in which it is made. An estimate made in the early part of a project life will have a lower accuracy than one made in the later stage of a project life (Emhjellen et al., 2003). The reliability of the estimate is based upon the basis of the prediction, the methods or tools used, and the effectiveness of the estimator. According to Kinsella (2002), the use of project characteristics in parametric modeling to predict project cost is considered an Concept DCP Phases
Concept
Plan DCP Plan
Availability DCP Development
End of Life DCP
Quality Deployment Manage Life End of Life
t1 *
t2*
Conceptual Estimate
Revised Conceptual Estimate
– Optional pre-concept activity
– Based upon concept activities
– Estimate of deployment resources
– Estimate must occur prior to Concept Decision Check Point (DCP)
Full Plan – Detailed Estimate – Assumes requirements are known
Figure 1. Conceptual estimating timeline
– DCP is similar to Robert Cooper's Stage-Gate (Go/No Go)
t1 = time of use of the conceptual estimating framework t2 = time for performance of work that was estimated at time t1
adequately accurate method. When historical information as input is used to develop the model, the parameters used in the model are quantifiable, and the scale of the project can be used to adjust the model. Since, many decisions are not finalized during the concept phase, conceptual estimates are primarily based on assumptions (Kwak and Watson, 2005). The potential volatility in the ultimate decisions leads to conceptual estimates, which by nature are relatively inaccurate. This lack of accuracy contributes to the reason many organizations lack interest in performing conceptual estimating for deployment. A premise of this study was that the accuracy of the resulting estimate was not the most important reason for performing the conceptual estimate. Stewart (1991) stated that the knowledge learned during the process of estimating often provides more value than the estimate itself, supported this premise. Although the conceptual estimate is primarily based upon assumptions, thinking through and documenting these assumptions provide value to the project team responsible for the deployment. By analyzing different scenarios and selecting among the most likely, the project manager begins to identify potential issues. A major assumption is that this occurs much sooner than would occur if the conceptual estimating process were not implemented, because the project manager would be more focused on development than deployment during the conceptual phase. This early awareness and the subsequent actions may provide the opportunity to prevent or resolve issues and contribute to a more effective deployment. While acknowledging the value of conceptual estimating for deployment, organizations are more likely to keep SMEs working on development activities as opposed to generating deployment estimates due to the cost associated with development delays. As a result of using the framework developed by this study, an organization can have the option of using the valuable SME resources on a one-time basis to build a framework for generating deployment estimates that can subsequently be used without further involvement of SMEs. The ability to perform conceptual estimating for deployment without tying up the SME resources for each deployment estimate may be reason enough for organizations to make the one-time investment. Project-driven organizations need to better utilize available information along with innovative tools and methodologies to adapt to a competitive environment. The conceptual estimating framework is a method for sharing knowledge. It is an attempt to capture the knowledge of subject-matter experts, codify that knowledge and disseminate it thorough use of this process.
Research steps The research method executed the following six steps of framework development: (1) construct definition; (2) selection of cost-estimating relationships (CERs); (3) determination of estimates; (4) development of the estimating framework; (5) execution of the framework; and (6) documentation of assessments and findings.
Business transformation projects 133
IJMPB 1,1
134
The first step of the research method was construct definition. A comprehensive literature review of published articles, books, research papers, and dissertations on grounding theories supporting the area of conceptual estimating was conducted. This review provided the theoretical foundation and identified the construct definitions for framework development. It also included the identification of categories for segmenting the cost-estimating relationships. The second step of the research method was the selection of CERs. A part of the framework-building process involved determining the CER variables for estimating the deployment cost of a software application development project (Kwak and Watson, 2005). The Delphi technique was used to identify the CERs used as the independent variables of the framework. Without knowledge of the responses of other project managers, each individual was asked to identify project characteristics as potential CERs. The responses were compiled and disseminated to the project managers as input for reaching an agreement on the final CERs used in developing the framework. A separate survey of the subject matter expert project managers was performed at the conclusion of the Delphi technique to gain an understanding of their rationale for excluding potential CERs. These findings were incorporated into the framework-building section of the dissertation. The third step of the research method was the determination of estimates. The framework contains an established range of estimates for each CER based upon the explicit knowledge of subject-matter experts captured through the use of the Delphi technique to gain agreement on the appropriate range of estimated values. The input was gathered from the same project managers who provided input on the selection of the CERs. Their input was based upon access to historical data from projects in their past experience. The responses were obtained from the individual project managers, compiled, and disseminated. The decision on the range of values associated with the CERs was reached by agreement. The fourth step of the research method was the development of the estimation framework. The data collected via the Delphi technique were embedded within the framework. Rules for use of the framework were established as well as guidelines for assessment of the cost-estimating independent variables. The fifth step of the research method involved executing the framework. Using project definition documents from the concept phase of various projects, the framework was exercised against existing projects deployed in Asia Pacific South, China, European Union, and Latin America to produce an output for each project. This output was compared to the original conceptual estimates. The final step of the research method was the performance and documentation of the assessment. The output was analyzed with findings and recommendations produced on both the process of framework building and the framework produced. Framework building In executing the framework building process, the identification of constructs used to categorize information was solicited from the study participants. These categories were based upon nomenclature commonly used within the software development industry. The final categories were project criteria, transformation management, project management, data conversion and migration, process management,
communications management, information technology, job design, measurement systems, test, education and training, and move to production. Figure 2 shows that the study consisted of three Delphi iterations. The first iteration was to reach agreement on the project and task characteristics. The second Delphi iteration was to reach agreement on the complexity aspects of the characteristics. The third Delphi iteration was to reach agreement on a range of estimates. The grey boxes overlaying a box representing each round represent the participants. Round one to round n represent a full iteration. In determining the CER project and task characteristics, the initial round of the first iteration solicited involvement from a sample group of 67 project managers.
Business transformation projects 135
Context of the dissertation This dissertation was submitted to the faculty of the Graduate School of Business of The George Washington University in partial fulfillment of the requirements for the degree of Doctor of Philosophy (PhD). The fields of study were logistics, technology, and project management. Each PhD program of study is individualized. This specific program consisted of 17 courses, each a semester (14 weeks) long, examinations, a dissertation proposal and a final dissertation, both orally defended. In addition to analytical methods and advanced methods, courses were selected to fulfill the requirements of the primary field of project management and the secondary field of the management of science, technology and innovation. The analytical methods courses included classes and seminars on the philosophical foundations of administrative research, public-private Round - n
Round - n cs
sti
Round - 2
ri cte
ara
Round - 1
Round - 2 Round - 1 ne mi
h yC
De
Id
Area outside of dashed marks is un-codified knowledge domain
Construct Definition
Co
ter
tif
en
y
xit
le mp
Round - n s
ate
Round - 2
m sti
fE
Round - 1
eo
ine
g an
R
m ter
De
Conceptual Estimate
Codified knowledge is reusable in deployment conceptual estimating for similar projects
Execute 4-Step Guideline Instructions
Analyze Input Build Tool
Figure 2. Conceptual estimating framework
IJMPB 1,1
136
sector institutions and relationships, research methods, and statistical modeling and analysis. The advanced methods course included addressing problems in research methodology, using models, theoretical frameworks, and various sampling and data analysis approaches. The advanced statistical modeling and analysis course focused on topics associated with the general linear model. This included testing for re-mediation of assumption violations, detection of outliers, influential observations and multi-collinearity. There was also exposure to alternative design strategies in the analysis of variance, latent growth analysis, hierarchical linear modeling, as well as testing for interactions and parallelism. The primary and secondary field seminars and courses covered the topics of international development, cross-cultural management, risk management, project estimating and cost management, knowledge management, information policy, emerging technologies, and institutional change. The purpose of the research was to develop a conceptual estimating framework for an organization to improve the decision-making process related to project portfolio management and project implementation. During the doctoral study, several papers were presented at major conferences including INFORMS Winter Simulation Conference (Watson and Kwak, 2004b), International Association of Management of Technology Annual Conference (Watson and Kwak, 2004a), and Project Estimation Conference by IBM Academy of Technology (Watson and Kwak, 2005a), as well as published in major journals (Kwak and Watson, 2005; Watson and Kwak, 2005b). Currently, additional journal and conference papers are in preparation to disseminate the framework, discuss case studies, and report on key findings to generate academic discussions. Discussion and conclusions Conceptual estimating, as a subset of parametric estimating, uses the same methodologies and approaches. However, in conceptual estimating, cost-estimating relationships and the associated values are derived using a more heuristic method rather than sole reliance on statistical data as is commonly done in parametric estimating. The advantage of this approach is that factors, such as management issues and reasons for the estimate, can have a positive impact when taken into consideration (Prince, 2002). The feature that most clearly separates parametric methods, such as the framework developed in this study, from detailed estimating is the level of detail required (Hamaker, 1995). There are assertions that it may be difficult to produce an estimate or that the results will lack the precision of a detailed estimate; however, conceptual estimating techniques play a very significant role in gaining an understanding of a project during the early stage of the project’s life (Stewart, 1991). An advantage of using the estimating framework is the promptness in obtaining an estimate and the cost savings associated with using less skilled resources for performing the cost estimate. Estimates are derived faster for a number of reasons, the first being the elimination of the need to schedule the participation of SMEs to prepare the estimate. Generally, highly skilled people are in greater demand and may not always be available exactly when desired. Since, expert knowledge is captured within the framework, the ability to use a lesser skilled individual to perform the assessments improves the opportunity for acquiring the resources and the subsequent estimates sooner. Another factor in improving the time required for obtaining the estimate
is associated with the execution of the framework. As shown in the study, the structure and method facilitates obtaining an estimate within half an hour versus methods that can take hours, days or even weeks. The most pertinent implication is that the use of the estimating framework increases awareness of the characteristics that affect costs at the earliest possible point in the life of a project. Execution of the framework forces project managers to consider factors that may be normally ignored during the conceptual phase or at least not investigated as fully as could be done. This increased awareness can be utilized to improve project execution by focusing resources on the most effective areas. For example, an understanding of the significant activity categories can be used as input into risk analysis and risk-mitigation processes. Given the detailed descriptions contained within the study, this process can be used as a template for projects other than the deployment of software applications and by different organizations. By following the systematic approach provided in this study, an organization can capture historical data, identify SMEs and perform a Delphi technique to identify CERs, complexity criteria, and estimating ranges that are applicable to their specific environment and types of projects. The method employed to build the estimating framework can be utilized by organizations to build functioning conceptual estimating tools for deployment, which may lead to large savings during the estimating process and eventually lead to effective project management, control, and implementation. References Bitner, M.J. and Brown, S.W. (2006), “The evolution and discovery of services science in business schools”, Communications of the ACM, Vol. 49 No. 7, pp. 73-8. Bitner, M.J., Brown, S.W., Boul, M. and Urban, S. (2006), “Services science journey: foundations, progress, challenges”, Proceedings of Education for the 21st Century. IBM Palisades Conference Center. New York, NY, October. Emhjellen, M., Emhjellen, K. and Osmundsen, P. (2003), “Cost estimation overruns in the North Sea”, Project Management Journal, Vol. 34 No. 1, pp. 23-9. Hamaker, J. (1995), “Parametric estimating”, in Stewart, R., Wyskida, R.M. and Johannes, J.D. (Eds), Cost Estimator’s Reference Manual, 2nd ed., Wiley InterScience, New York, NY. Kinsella, S.M. (2002), “Activity-based costing: does it warrant inclusion in a guide to the project management body of knowledge (PMBOKw Guide)?”, Project Management Journal, Vol. 33 No. 2, pp. 49-56. Kwak, Y.H. and Watson, R. (2005), “Conceptual estimating tool for technology-driven projects: exploring parametric estimating technique”, Technovation: An International Journal of Technological Innovation, Entrepreneurship, and Technology Management, Vol. 25 No. 12, pp. 1430-6. Prince, F.A. (2002), “Why NASA’s management doesn’t believe the cost estimate”, Engineering Management Journal, Vol. 14 No. 1, pp. 7-12. Stewart, R.D. (1991), Cost Estimating: New Dimensions in Engineering, 2nd ed., Wiley, New York, NY. Watson, R. and Kwak, Y.H. (2004a), “Parametric estimating in the knowledge age: capitalizing on technological advances”, paper presented at IAMOT 2004 13th International Conference on Management of Technology. Washington, DC, April 3-7.
Business transformation projects 137
IJMPB 1,1
138
Watson, R. and Kwak, Y.H. (2004b), “Development of a parametric estimating model for technology-driven deployment project”, paper presented at 2004 Winter Simulation Conference, Washington, DC, December 5-8. Watson, R. and Kwak, Y.H. (2005a), “Conceptual estimating of deployment costs for technology-driven projects”, paper presented at Project Estimation Conference, IBM Academy of Technology (Internal), Boulder, CO, January 24-26. Watson, R. and Kwak, Y.H. (2005b), “Development of a parametric estimating model for technology-driven deployment project”, ICFAI Journal of Operations Management, August. Further reading Cooper, R.G. (2006), “Winning at new products: pathways to profitable innovation”, Proceedings of the 2006 Project Management Institute Research Conference. Montreal, Canada, July 16-19. Watson, R.J. (2007), “A cost estimating framework for deployment of integrated software application development projects”, UMI dissertation services (UMI No. 3259291). About the authors Young Hoon Kwak is a Project Management Faculty at The George Washington University’s School of Business (GWSB). He earned his MS and PhD in Engineering and Project Management from the University of California at Berkeley, was a visiting Engineer at the Massachusetts Institute of Technology, and taught at the Florida International University in Miami before joining GWSB. He is currently serving as a Specialty Editor (Associate Editor) for Journal of Construction Engineering and Management, a member of the Editorial Review Board for Project Management Journal, a member of the International Editorial Board for International Journal of Project Management, and an elected member of the Construction Research Council for American Society of Civil Engineers. He is a two-time recipient of the Project Management Institute’s Research Grant and has more than 50 scholarly publications in journals, books, book chapters, and conference proceedings in the area of project management, risk management, technology management, and engineering and construction management. For more information, visit: http:// home.gwu.edu/ , kwak. Young Hoon Kwak is the corresponding author and can be contacted at:
[email protected] Rudy J. Watson is an IBM and PMI Certified Project Management Professional supporting the Executive Leadership Foundation’s Technology Transfer Project (TTP). The TTP assists Historically Black colleges and universities in integrating open source and state-of-the-art technologies into business, computer science and engineering curricula. He is an adjunct faculty member at the University of Maryland University College in the Information and Technology Systems Department. He has over 32 years of broad and diverse experience in information technology including 30 years with IBM. His experiences include marketing support, systems engineering, technical support, operations management, project management, and information management. He is a member of the Project Management Institute and the Academy of Management. He has also served on the Advisory Board of Visitors at Mary Baldwin College, Staunton, VA. He holds a PhD in Logistics, Technology and Project Management, an MS in Project Management, an MBA in Information Systems Technology, and a BBA in Information Processing, all from The George Washington University in Washington, DC. His e-mail address is:
[email protected] Frank T. Anbari, PhD in Project Management and Quality Enhancement, MBA, MS Engineering, PMP, PE, and ASQ Certified Six Sigma Black Belt, is a faculty member and past Director of the Project Management Program at The George Washington University. He taught in the graduate programs at Drexel University, Penn State University, and the University of
Texas at Dallas, and gained extensive experience serving in leadership positions in industry. He serves as member of the Editorial Boards of International Journal of Managing Projects in Business, International Journal of Project Management, and Project Management Journal. He served as Vice President-Education and Certification, College of Performance Management-Project Management Institute (2005-2007), and as Examiner (1993-1995) and Alumni Examiner (1999-2000) for the Malcolm Baldrige National Quality Award. He conducts rigorous research, presents widely, and publishes extensively on significant, timely topics in project management, quality enhancement, technology management, and six sigma method. For more information, visit his web site at: http://home.gwu.edu/ , anbarif. His e-mail address is:
[email protected]
To purchase reprints of this article please e-mail:
[email protected] Or visit our web site for further details: www.emeraldinsight.com/reprints
Business transformation projects 139
