1
Assessing Software Project Management Complexity: PMCAT tool Vyron Damasiotis, Panos Fitsilis TEI Thessaly, 41110 Larissa, Greece {bdama, fitsilis}@teilar.gr Abstract—Software projects are complex endeavors that quite often fail to satisfy their initial objectives. As such the need to systematically study and assess the complexity of software projects is quite important. This study presents a systematic framework for assessing complexity of software projects that is based on the study of project management subject areas as defined in Project Management Body of Knowledge (PMBOK). The presented framework is based on a model that combines the concepts of project, complexity model, complexity factor etc. is an attempt to systematically assess and compare the complexity of software projects. The whole concept has been implemented within Project Management Complexity Assessment Tool (PMCAT) and it is available as a software service over the web. Index Terms—Project management, software projects, project complexity, complexity measurement
I. INTRODUCTION Even though Project Management (PM) has received a lot of attention from industry and academia, a great number of projects still fail to meet their requirements in terms of time delay, cost overrun and quality restrictions [1, 2, 3]. Most of these failures have been attributed to the complexity of the projects. Many studies have been undertaken the last years in order to understand, define and determine the concept of project complexity [4, 5, 6, 7, 8, 9]. Software projects are among the most complex ones. Many studies on various types of software project have proven that their outcomes are far from the complete fulfillment of the initial requirements [10, 11, 12]. Most studies measure complexity either by measuring the software project product based on its attributes such as size, quality, reliability or the characteristics of software project process using attributes such as performance, stability, improvement [13, 14, 15]. As such the need to establish a systematic way to evaluate the software project complexity is important.
Manuscript received November 11, 2013.. V. Damasiotis is with the Accounting and Finance Department, Technological Educational Institute of Thessaly, 41110 Larissa, Greece. (email:
[email protected]). P. Fitsilis, is with the Business Administration Department, Technological Educational Institute of Thessaly, 41110 Larissa, Greece (phone: +302410684204; e-mail:
[email protected]).
The structure of this paper is as follows. Section 2 presents a short introduction to PM, the structure and the typology project complexity. In section 3, the focus is moved on software projects and an attempt is made to establish a connection between project management complexity and software project failures and to prove need for complexity assessment in these projects. In section 4 our approach towards to modeling the project complexity is presented. Further a short presentation of PMCAT tool is given, along with the description of the requirements that was set and the design approach was followed. Finally, conclusions and implications for future work are discussed. II. LITERATURE REVIEW A. Project management Several project management frameworks have been developed so far three of which gained globally acceptance from industry and academia, Project Management Body of Knowledge (PMBOK) by Project Management Institute (PMI) [16], IPMA Competence Baseline (ICB) by International Project Management Association (IPMA)[17] and Project IN Control Environments (PRINCE2) by the Office of Government Commerce (OGC) of British government [18]. ICB proposed by IPMA describes in detail the competences that are required for project management. These competencies are classified in three main categories, which are the Technical competencies, the Behavioral competences and the Contextual competencies [16]. ICB is not process based and is focused on skills, tasks and activities. PRINCE2 is used widely in UK and was introduced by OGC in 1996. It was fundamentally revised in 2009 in order to adapt to changes in the project and business environment, address weakness and adjust with other OGC methods. PRINCE2 is process driven PM framework which is very prescriptive and provides the necessary techniques and templates for project manager to apply. PMBOK is globally accepted as the main standard for PM both from companies and from organizations such as IEEE and ANSI [19, 20, 21]. PMBOK identifies nine areas in project management which are: Project Integration Management, Project Scope Management, Project Time Management Project Cost, Project Quality Management, Project Human Resource Management, Project Communication Management,
2 Project Risk Management, and Project Procurement Management. [16]. In terms of PM context, PMBOK and PRINCE2 are focusing on “hard” skills such as processes, procedures and techniques, while the ICB is focused on “soft” skills which are related to human behavior such as leadership, motivations etc. Winter et al. [22] states that project management thinking is focused on “hard” aspects which emphasizing more in planning and control rather than in “soft” skills and this is represented and in dominant PM frameworks. On the other hand the importance of “soft” skills such as leadership, social conduct and interaction between project stakeholder and active participation and accountability are revealed in researches [23, 24, 25]. PMBOK is chosen as the base PM framework in this research as although it provides project managers with a pool of procedures and techniques, does not provide any templates on them, which means that leaves the freedom in project managers to apply them according to their experience, hence, leave a port open and for PM ”soft” skills. B. Project complexity Complexity is part of our environment and appears in different domains. Many scientific fields have dealt with complex systems and have attempted to define the term complexity according to their domain. This implies that there is a different definition of complexity in computational theory, in information theory, in business, in software engineering etc. and many times there are different definitions inside the same domain [26], Schlidwein and Ison [27] states that are two major approaches of complexity. The first one describes the complexity as a property of a system, called descriptive complexity. The other approach describes complexity as perceived complexity and translates it as the subjective complexity that someone experiences through the interaction with the system. Complexity in PM attracted significant attention from researchers during the last decade [4, 8, 28, 29, 30]. However there is no consensus on what really complexity is [26, 31]). Many project managers cannot distinguish the difference between complex and complexity [32]. However a definition of project complexity should at least contain interaction, structural and dynamic elements [32]. C. Complexity typologies This lack of consensus in defining what project complexity is has resulted in a variety of approaches on classifying project management complexity. One of the first researches that deal with the concept of complexity was Baccarini [33]. He considers complexity as something “consisting of many varied and interrelated parts” and operationalized them in terms of “differentiation” the number of varied elements (e.g. tasks, components) and “interdependency” the degree of interrelatedness between these elements. Finally he describes four types of complexity: a) organizational complexity by differentiation and b) organizational complexity by interdependency c) technological complexity by differentiation
and d) technological complexity by interdependency. Extending the work of Baccarinni, Williams [34] added the dimensions of uncertainty in projects and the multi-objectivity and multiplicity of stakeholders. The definition of project complexity according to Williams is divided in structural complexity sourcing from number and interdependence of elements and uncertainty sourcing from uncertainty in goals and methods. Xia and Lee [29] focused on Information System Development Projects (ISDP) and portrayed complexity of these type of projects by defining two types of complexity: a) organizational complexity and b) technological complexity described under two dimensions the structural dimension and dynamic dimension. They proposed an ISDP complexity model in order to measure complexity in IS development projects. Geraldi and Adlbrecht [5] and Geraldi [35,36] based on structural complexity and uncertainty defined three types of complexity, the complexity of faith (CoFaith), Complexity of Fact (CoFact) and Complexity of Interaction (CoInt) Maylor et al. [7] focused on perceived managerial complexity under two dimensions structural and dynamic and identified five aspects of complexity. They defined a complexity model that is based on Mission, Organization, Delivery, Stakeholders and Team (MODeST). Vidal et al [30], studied project complexity under the organizational and technological dimensions and identified four aspects for studying project complexity: project size, project variety, project interdependence and project context. Bosch-Rekveltdt et al. [4], proposed a framework for “Grasping complexity in large engineering projects”, in which they identify three categories of project complexity the technical complexity, organizational complexity and environment complexity (TOE) and fourteen subcategories. Sedaghat-Seresht et al. [8] identified five dimensions of complexity the environmental, organizational, objective, stakeholder, task, technology and information systems complexity. They also used “DEMATEL” method to identify the relationships between project complexity dimensions. A step further is made in the studies of Xia and Lee [29], Vidal et al. [30], Bosch-Rekveltdt et al. [4], Sedaghat-Seresht et al. [37] who not only attempted to identify project complexity dimensions but as well to propose models for measuring project complexity. III. SOFTWARE PROJECT COMPLEXITY A. Software project complexity Complexity in software projects are quite similar with projects in other domains with regard to the factors that influence it, for example the tools, the processes, the restrictions to name a few [38]. Hughes et al. [39], and Kiountouzis [40] state that software projects differ since are immaterial, complicated, supple and technology dependent. Furthermore, Xia and Lee [29] state that information systems projects “are inherently complex because they deal not only
3 with technological issues but with organizational factors largely beyond the project team’s control”. Regarding the top ten factors that lead to project success or to project challenged and failure as they described in various in studies like “Chaos report” [10, 11] and “why software fails” [12], it is obvious that most of them identify project management aspects among them, both in terms of “hard” and “soft” management aspects. Indicatively, are referred as success or failure factors, factors that are related to proper planning, requirements management, scope management, risk management, procurement management, communication management, human resource management, executive management support, user involvement and technology related issues [10, 12]. Most of these failures have been attributed to the complexity of the projects. Project complexity lead to project failure because either complexity is very high [9, 41], either project complexity has been underestimated [42]. Considering the above, it is obvious that many failure factors would have been restrained, if not eliminated, if there was an indication of project complexity level, so properly management techniques would have been applied to handle this complexity. The relationship between project complexity and PM has already started to be investigated by researchers [43]. Consequently, in order to reduce the possibility of project challenge or failure, caused by complexity, we have to control it. The first step to succeed this is to know the level of expected complexity by measuring it. Whitty and Maylor [32] propose the use of complexity as a metric, to measure complexity in a system. So complexity should be deal as a variable that we should to measure, if we want to be helpful in PM. In this case we can use complexity as thermometer by developing – “complexometer” [35] and in the question “How complex is this project?” to reply “Its complexity is…..” [32]. Even though studies have proposed methods and techniques to measure project complexity [4, 29, 30, 37], the need for a more practical framework that combines a project complexity measurement model and an automated tool to support it, is evident.
COCOMO II [45]), or on analyzing the properties of software (e.g. McCabe's cyclomatic complexity [46]) Our approach in assessing project complexity is holistic and proactive, which means that we want to build a framework for assessing the project complexity affecting all areas of PM at the beginning of the project and provide by that way the project manager with the appropriate information in order to successfully manage it. To succeed that, an extensive literature review is performed and properties of projects and properties of project management process are identified in order to be determined the appropriate complexity factors and metrics. These metrics will be evaluated by experts for their contribution in total project complexity and will be prioritized. Indicative measures for each of the PMBOK subject areas are presented in Table 1 [14].
B. Complexity metrics A key element for developing an effective, efficient and trustworthy complexity measurement framework is the determination of the appropriate metrics. Metric is a property of a project that can be measured and recorded as a number, a percentage, a count or a rating scale [44]. The determination of good metrics allow the proactive rather than reactive project management and hence can reduce complexity stemming from projects process faults and dysfunctions. Metrics should satisfy a number of requirements such as to be reliable, repeatable, easy to use, consistent and independent in order to be useful. Metrics and tools in software projects are emphasizing in measuring the software product or software development process mainly and fragmentally deal project management as an entity. The most widely known measurement models are focusing on software development cost estimation (e.g.
Project Schedule
TABLE 1 COMPLEXITY FACTORS AND METRICS
Complexity factors Project Integration Management Software volatility
requirements
Project novelty
Number of changes requests Number of similar software projects within organization Number of similar software projects in the literature Legislation and regulations Competition in market
Project Environment Project Scope Management Software size
Software structure architecture
Indicative Metrics
and
Metrics such as FPA and UCP Number of deliverables Number of components and modules Number of different technologies used within project
Project Time Management
Project schedule difficulty
Project duration Number of activities Activities Synchronization Number of critical activities Number of constrains
Project Cost Management Cost estimation
Level of accuracy Cost planning
Availability of cost estimations data Sufficient time for cost estimations Quality and level of detail of tender documentation Size of project Duration of project
4 Complexity factors
Indicative Metrics
Project Quality Management Quality planning
Number of quality management procedures Importance of quality in relation to cost and schedule objectives Project Human Resource Management Team structure Number of stakeholders Geographical distribution of team members Cultural differences Personnel availability Project Communication Management Communication planning Reporting frequency Expected average percentage of labor spent in communication procedures Project Risk Management Risk Planning Number of Risks identified Project Procurement Management Procurement planning
Number of procurement orders Number of new external contractors Number of different organizations involved
Procurement execution
Further, the definition of the evaluation scale for each metric is an important component in the measurement process. It should clearly define the boundaries and the scale of each metric. An example of an evaluation scale for the metric “Expected average percentage of labor spent in communication procedures” can be seen in Table 2. TABLE 2 EXAMPLE OF METRIC’S SCALE
Complexity Metric Expected average percentage of labor spent in communication procedures
Very Low (1)
Low
High
(2)
Mode rate (3)
Up to 1%
Up to 3%
Up to 5%
Up to 10%
(4)
Very High (5)
managers to experiment, develop their own complexity models, if needed, and to apply these models to evaluate projects. The intention is to use this tool as a collaborative tool for the PM community either for complexity model development and validation or for project complexity assessment. Five basic concepts/entities were used, namely: Project, Model, Complexity Factor, Metric and Evaluation Scale. The Project entity is used to describe each project under evaluation. Each project is evaluated by the use of one or more models. These models are custom developed models, for the needs of the specific project, or are reused from a set of available to the PM community models. Each Model is composed of a number of Complexity Factors, factors that are combined with a unique way for the needs of specific project or for categories of projects. In every Model, each Complexity Factor is correlated with a specific weight that represents the contribution of this factor to the project complexity. It is not unlikely that the same factor has different weights when it is participating in different models. The calculation of the Complexity Factor’s weight is outside the scope of this paper, however it is done with the use of statistical methods and group decision techniques. Similarly, a Complexity Factor is correlated with a number of Metrics. In the simple case a Complexity Factor corresponds to a Metric, however it is not uncommon to have composite Complexity Factors that require more than one Metric to be measured. Finally, an evaluation scale is used to indicate how is Metric is measures and can be numerical, ordinal, scale, yes/no, etc. Predetermined evaluation scales satisfies the need for consistency and homogeneity in metrics evaluation. By using this structure, a project can be associated with different models allowing the execution of different scenarios in order to evaluate different project conditions and the impact to the expected project complexity. Furthermore, “advanced user” may introduce new complexity factors, metrics and evaluation scales that will allow the PM community to fully parameterize the tool according their project type and the specific project requirements. According to this model, the Project Complexity (PC) is calculated by the use of the following formula: n
PC CFVj * CFWj j 1
>10%
MODELING PROJECT MANAGEMENT COMPLEXITY AND PMCAT TOOL One of the main objectives for the design of Project Management Complexity Assessment Tool (PMCAT) was to design an overall software service that will allow project
where CFWj: is the Complexity Factor Weight, and CFVj: is the Complexity Factor Value. In case of composite Complexity Factors the CFV is calculated similarly by the following formula:
CFVj i 1 MtrVi * MtrWi n
In every case the weights of metrics that correspond to a complexity factor are summarized to 1. The same applies and for the weights of complexity factors that belong to a model.
5 The logical structure of the tool, as described above, is presented in Figure 1, while a sample screen of PMCAT tool (http://pmc.teilar.gr/), where project complexity of a sample project is calculated is given in Figure 2.
Fig. 1 PMCAT model
management processes. For this reason, the study of PMBOK subject areas, for identifying complexity sources, has been recommended. The advantage of this recommendation is that PMBOK is well-known and widely accepted framework by project managers and organizations and project managers are familiar with its processes and can easily identify and assess complexity factors sourcing from these processes. Thus we believe that we can result with a complexity framework that would be efficient, effective and reliable. Further we propose and describe the architecture of PMCAT tool that utilizes a software project management complexity framework in order to automate the complexity assessment process. PMCAT tool may be used by the PM community in a collaborative fashion or by individual users attempting to evaluate their own projects. In the future we are going to develop further this complexity framework, by systematically identifying complexity sources, and by systematically evaluating different complexity models. Further, we will extend the functionality of PMCAT tool by providing support for collaborative model development and collaborative project assessments with the use of social networks. V. ACKNOWLEDGMENTS The research presented in this paper has been co-financed by the European Union (European Social Fund) and Greek National funds through the Operational Program “Education and Lifelong Learning” of the National Strategic Reference Framework. In particular, this research work has been partially financed by the R&D project "ONSOCIAL" which take place in the context of the "ARCHIMEDES III" National Research Programme. Further, we would like to thank Costas Athanasiou that contributed significantly with the implementation of PMCAT tool. REFERENCES [1] [2]
Fig. 2 PMCAT model
IV. CONCLUSION In this paper we have presented our proposed software project management complexity framework that can be used for systematic assessment of software project’s complexity. This framework is based on the assumption that the project complexity is a composite phenomenon and in order to evaluate it, we need to examine all software project
Holmes A, (2001) Failsafe IS. Project delivery, Aldershot: Gower. Flyvbjerg B., Bruzeliusm N., Rothengatter W., (2003) Megaprojects and Risk. An Anatomy of Ambition. Cambridge University Press, Cambridge [3] Morris, P.W.G., Hough, G.H., (1987) The Anatomy of Major Projects: a Study of the Reality of Project Management. John Wiley, Chichester. [4] Bosch-Rekveldt, M.,Jongkind, Y., Mooi, H., Bakker, H., Verbraeck, A., (2011) Grasping project complexity in large engineering projects: The TOE (Technical, Organizational and Environmental) framework. International Journal of Project Management, 29, 728-739. [5] Geraldi J., and Adlbrecht G., (2006) On faith, fact, and interaction in projects, Project Management Journal, 38(1), 32–43. [6] Hass, K., (2007) Introducing the project complexity model. A New Approach to Diagnosing and Managing Projects—Part 1 of 2. PMWorld Today, IX(VII), pp. 1–8. [7] Maylor, H., Vidgen, R., Carver, S., (2008) Managerial complexity in project based operations: a grounded model and its implications for practice. Project Management Journal, 39, S15–S26 Supplement [8] Vidal, L.-A., Marle, F., (2008) Understanding project complexity: implications on project management. Kybernetes, 37(8), 1094–1110. [9] Williams, T.M., (2002) Modelling Complex Projects. John Wiley & Sons, London. [10] The Standish Group, (2009) CHAOS Summary 2009 The 10 Laws of CHAOS, The Standish Group International [Online] Available at:
, [Accessed: Dec. 10, 2010]. The Standish Group, (1995) Charting the Seas of Information Technology—Chaos. West Yarmouth, MA: The Standish Group International. Charette R., (2005) Why Software Fails, [Online] Available at: , [Accessed: Nov, 15, 2010]. Kitchenham, B., (2010) What’s up with software metrics? – A preliminary mapping study. International Journal of Systems and Software, 83, 37-51. Fitsilis P., A. Kameas and L. Anthopoulos, (2010) Classification of Software Projects’ Complexity, Information Systems Development, 2011, 149-159. Laird L., Brennan, M., (2006) Software Measurement and Estimation. A Practical Approach, John Wiley and Sons. PMI, (2008) A Guide to the Project Management Body of Knowledge: PMBOK guide, Project Management Institute, Inc. IPMA, (2008) IPMA Competence Baseline, Version 4.0. PMI Publishing OGC, (2009) Managing Successful Projects with PRINCE2 (2009 ed.), TSO (The Stationery Office). Pant, I., Baroudi, B., (2008) Project management education: The human skills imperative. International Journal of Project Management, 26, 124128. Morris, P. W. G., Jamieson, A., Shepherd, M. M., (2006b) Research updating the APM Body of Knowledge 4th edition. International Journal of Project Management, 24, 461-473 Thomas, J., Mengel, T., (2008) Preparing project managers to deal with complexity - Advanced project management education. International Journal of Project Management, 26, 304-315 Winter, M., Smith, C., Morris, P. & Cicmil, S., (2006) Directions for future research in project management: The main findings of a UK government funded research network. International Journal of Project Management, 24, 638-649. Turner, J. R., Muller, R., (2005) The Project Manager's Leadership Style as a success factor on Projects: A literature review. Project Management Journal, 36, 49-61. Cicmil, S., Marshall, D., (2005) Insights into collaboration at the project level: complexity, social interaction and procurement mechanisms. Building Research & Information, 33, 523-535. Thamhain, H. J., (2004) Linkages of project environment to performance: lessons for team leadership. International Journal of Project Management, 22, 533-544. Morel, B., Ramanujam, R., (1999) Through the looking glass of complexity: the dynamics of organizations as adaptive and evolving systems. Organization Science, 10(3), 278-293. Schlindwein, S. L. , Ison, R., (2004) Human knowing and perceived complexity: Implications for systems practice. Emergence: Complexity & Organization, 6, 27-32 Sinha, S., Thomson, A. I., Kumar, B., (2001) A complexity index for the design process. Proceedings of the International Conference on Engineering Design, ICED'01. Glasgow, Professional Engineering Publishing, Bury St Edmunds. Xia, W., Lee, G., (2004). Grasping the complexity of IS development projects. Communications of the ACM, 47(5), 69–74. Vidal, L. A., Marle, F., Bocquet, J.C., (2011) Using a Delphi process and the Analytic Hierarchy Process (AHP) to evaluate the complexity of projects, Expert Systems with Applications, 38, 5388-5405. Latva-Koivisto, A. M., (2001) Finding a complexity measure for business process models, Research report Helsinki University of Technology, Systems Analysis Laboratory. Available: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.25.2991&rep =rep1&type=pdf Whitty, S.J., Maylor, H., (2009) And then came Complex Project Management (revised). International Journal of Project Management, 27(3), 304–310. Baccarini D., (1996) The concept of project complexity – A review, International Journal of Project Management, 14(4), 201–204. Williams, T.M., (1999) The need for new paradigms for complex projects. International Journal of Project Management, 17(5), 269–273.
[35] Geraldi J., (2008) Patterns of complexity: The thermometer of complexity, Project Perspectives, IPMA 29, 4–9. [36] Geraldi J., (2008) The balance between order and chaos in multi-project firms: A conceptual model, International Journal of Project Management, 26, 348–356. [37] Sedaghat-Seresht, A., Fazli, S., Mozaffari, M. M., (2012) Using DEMATEL Method to Modeling Project Complexity Dimensions. Journal of Basic and Applied scientific Research, 2(11), 11211-11217 [38] Fitsilis P., G. Stamelos, (2007) Software Project Management. Hellenic Open University Publications. [39] Hughes B., Cotterell, M., (1999) Software Project Management, McGraw Hill. [40] Kiountouzis E., (1999) Software Project Management, Stamoulis Publishing. [41] Williams, T.M., (2005) Assessing and moving on from the dominant project management discourse in the light of project overruns. IEEE Transactions on Engineering Management, 52(4), 497–508. [42] Neleman, (2006) Shell gaat diep. FEM Business, 9 (4), 30–34 [43] Cooke-Davies, T., Cicmil, S., Crawford, L., Richardson, K., (2007) We're not in Kansas anymore, Toto: mapping the strange landscape of complexity theory, and its relationship to project management. Project Management Journal, 38(2), 50–61. [44] Kerzner, H., (2011), PROJECT MANAGEMENT METRICS, KPIs, AND DASHBOARDS A Guide to Measuring and Monitoring Project Performance. Publishing by John Wiley & Sons, Inc., Hoboken, New Jersey. [45] Boehm, B., Abts, C., Brown, A. W., Chulani, S., Clark, B. K., Horowitz, E., Madachy, R.,Reifer, D. J., and Steece, B. (2000) Software Cost Estimation with COCOMOII. Prentice Hall,Englewood Cliffs. [46] McCabe, T. J., and Butler, C. W. (1989). Design complexity measurement and testing. Communications of the ACM, 32(12), 14151425.
Vyron Damasiotis studied mathematics at Ioannina University, Greece, 1995. He received his Msc in High Speed Networks and Distributed Systems in 2003 from Oxford Brookes University, UK and since 2011 he is conducting his PhD research in the field of project complexity at Staffordshire University, UK. He worked as software developer in various research projects and as part time lecturer at Technological Educational Institute of Larissa. Since 2007 he holds the position of Lecturer of Distributed Information Systems at Technological Educational Institute of Thessaly, School of Business and Economics, Department of Accounting and Finance. His research interests include: Software Development, Project Management, Software Project Complexity, Computer Networks and Information Systems Security, etc. Prof. Panos Fitsilis studied computer engineering at Patras University, Greece. He received his PhD (1995) in Software Engineering. He worked, as software engineer and as business unit manager at large software development companies, and he was responsible for the development, deployment and operation of a number of prestigious IT systems. Since 2003 he has been Professor at Technological Education Institute of Larissa, School of Business and Economics, Project Management Department. Currently, Prof. Panos Fitsilis is Director of School of Business and Economics at TEI of Thessaly and Vice President of TEI Thessaly Research Committee He is the author of three books and author of many articles published on scientific journals and magazines. Further, he was member of the development team of Hellenic Standard on managerial capability of public organization, ELOT -1429. His research interests include: Project Management, Software Engineering, e-Government Systems, Business Information Systems, etc.
