J Med Syst (2015) 39:99 DOI 10.1007/s10916-015-0288-1
SYSTEMS-LEVEL QUALITY IMPROVEMENT
Towards an Encompassing Maturity Model for the Management of Hospital Information Systems João Vidal de Carvalho 1 & Álvaro Rocha 2 & José Vasconcelos 3
Received: 3 April 2015 / Accepted: 20 July 2015 # Springer Science+Business Media New York 2015
Abstract Maturity models are tools that favour the management of organizations, including their information systems management task, and hospital organizations are no exception. In the present paper we put forth a preliminary investigation aimed at the development of an encompassing maturity model for the management of hospital information systems. The development of this model is justified to the extent that current maturity models in the field of hospital information systems management are still in an early development stage, and especially because they are poorly detailed, do not provide tools to determine the maturity stage nor structure the characteristics of maturity stages according to different influencing factors. Keywords Stages of growth . Maturity models . Hospital information systems strategy . Management
Introduction Health care institutions and governmental organizations are starting to understand that the reasons underlying a This article is part of the Topical Collection on Systems-Level Quality Improvement * João Vidal de Carvalho
[email protected] Álvaro Rocha
[email protected] José Vasconcelos
[email protected] 1
IPP/ISCAP/CEISE, Porto, Portugal
2
Departamento de Engenharia Informática, Universidade de Coimbra, Coimbra, Portugal
3
Universidade Atlântica, Barcarena, Portugal
certain inadequacy in the management of health processes directly relates to infrastructural limitations and their inefficient management [16,60]. Hospital Information Systems (HIS) managers usually contemplate the errors that occurred in these organizations and wonder what could have been done to avoid them. They conclude that these errors are usually a natural growth and maturation symptom of organizations, and are often the result of the development that brought the organization to its current maturity stage [55]. This phenomenon of change fits the principles behind the growth stages theory and in the current context surrounding Information Systems (IS) of health organizations. Based on this presupposition which highlights the relevance of Maturity Models in the HIS fields, this research work is intended at the development of a maturity model that is especially adapted to the needs of Hospital Information Systems Management. To develop this new model, we carried out a preliminary study on IS and HIS Maturity Models and respective specificities. Based on this preliminary review of the State of the Art, concerning these two types of maturity models, we define a research methodology to propose and validate the new maturity model. Besides this section, this article is organized in six more. Accordingly, the second section proposes a preliminary systematization of the state of the art concerning maturity models in IS Management and HIS Management. The third section defines the problem underlying this investigation work. The fourth section presents the questions and objectives of the investigation. The fifth puts forth and explains the methodological approach adopted in the development of this investigation work. The sixth section describes the main contributes of this investigation and, finally, the seventh section offers some final remarks concerning this study.
99
J Med Syst (2015) 39:99
Page 2 of 9
State of the art: Preliminary review Information Systems Management (ISM) is the activity responsible for the tasks of an organization pertaining to Information Management, Information Systems and the adoption of Communication and Information Technologies (CIT) [1]. The maturity of this activity is a key factor for the success of organizations, to the extent that an IS is fundamental for their survival, competitive edge and success. In this context, there are several instruments that help the ISM achieve an enhanced maturity, namely the so called Maturity Models. Indeed, maturity models provide organization managers an important model for the identification of the maturity stage of an IS in order to plan and implement actions that will allow them to move towards an enhanced maturity stage, and thus achieve the proposed goals [53, 54]. Maturity Models can be perceived as conceptual models, comprised by discreet stages that are used to identify Banticipated, typical, logical or desired evolution paths towards maturity^ [4]. We observe that these models have been used in multiple areas to describe a wide variety of phenomena [9,12,29,30,37]. Maturity Models are sustained by the principle that people, organizations, functional areas, processes, etc., evolve, towards an enhanced maturity and following a development or growth process, which covers a number of different stages [53]. That is, Maturity Models are based on the theory of cyclic stages of growth, where the changes observed in an IS over the course of time occur in a sequential and predictable mode, covering a certain number of cumulative and hierarchically sequential stages, which can be described and linked to a specific level of maturity [6,46,53,55]. In the same sense, Caralli & Knight [10] argue that maturity models provide organizations with a tool to address their problems and
Table 1 A maturity model structure [55]
challenges in a structured way, offering both a reference point to evaluate their capabilities and a guide to improve them. Over the last four decades several maturity models have been proposed, with differences as to the number of stages, influencing factors and intervention areas (Table 1). Each one of these factors identifies the characteristics that typify the focus of each maturity stage, that is, these factors work as reference descriptors or variables to characterize each stage and provide the necessary criteria to achieve a specific maturity stage [4].
Evolution of maturity models in IS management Richard Nolan is considered the mentor of the IS maturity approach. Indeed, after studying/researching the use of IS in the biggest US organizations, Nolan proposed a maturity model that initially included 4 stages [46]. Later, with a view to improve his first proposal, Nolan included two additional stages to the initial model [47]. In this second version, Nolan suggests that organizations start slowly in the Initiation stage, followed by a rapid spread in the use of ITs during the Contagion stage. Subsequently, the need for Control emerges, and this stage is followed by the Integration of different technological solutions. Data Management allows for development without increasing IS related costs and, finally, constant growth promotes the achievement of Maturity. Although this approach to the maturity models developed by Nolan, has been recognized as significantly ground breaking, it also raised a lot of debate and controversy within the scientific community. Several researchers have published studies that, on the one hand, validated and, on the other hand, expanded the model proposed by Nolan. Indeed, resulting
Factors
Stage 1
Stage 2
Stage …
Stage N
Factor 1
Characteristic 1 Characteristic … Characteristic N Characteristic 1 Characteristic … Characteristic N Characteristic 1 Characteristic … Characteristic N Characteristic 1 Characteristic … Characteristic N Characteristic 1 Characteristic … Characteristic N
Characteristic 1 Characteristic … Characteristic N Characteristic 1 Characteristic … Characteristic N Characteristic 1 Characteristic … Characteristic N Characteristic 1 Characteristic … Characteristic N Characteristic 1 Characteristic … Characteristic N
Characteristic 1 Characteristic … Characteristic N Characteristic 1 Characteristic … Characteristic N Characteristic 1 Characteristic … Characteristic N Characteristic 1 Characteristic … Characteristic N Characteristic 1 Characteristic … Characteristic N
Characteristic 1 Characteristic … Characteristic N Characteristic 1 Characteristic … Characteristic N Characteristic 1 Characteristic … Characteristic N Characteristic 1 Characteristic … Characteristic N Characteristic 1 Characteristic … Characteristic N
Factor 2
Factor 3
Factor …
Factor N
J Med Syst (2015) 39:99
from the researches in this field several researchers have proposed new models [e.g.,: [14,17,25,28,36]]. Amongst these new models proposed after the initial approach developed by Nolan, the most widely accepted, detailed and comprehensive is the Revised Model of Galliers and Sutherland [9,53]. This model provides an improved perspective of how an organization plans, develops, uses and organizes an IS and offers suggestions towards an enhanced maturity stage. This model involves six stages of maturity (Table 2) and assumes that an organization can occupy different maturity stages in any given moment and be conditioned by influencing factors. Moreover, it presents the characteristics of the stages aligned with modern network organizations and offers a data collection tool to evaluate maturity [53]. More recently, after the model proposed by Galliers & Sutherland [17], other models have been resealed (e.g.,: [3,27,29,42], including a new Nolan model with nine stages of maturity [48], developed as an answer to the technological evolution in the IS field and its management. As to the field of IS Management, another solid example of a Maturity Model is the model developed by de Khandelwal & Ferguson [27], proposing nine stages of maturity and combining stages theory with Critical Success Factors. Notwithstanding, the model proposed by Galliers & Sutherland [17] is still perceived as the most complete and updated in IS management [55]. Additionally, these maturity models are still being used and implemented in multiple types of organizations and to different areas inside them. Mutafelija & Stromberg [41] refer that the concept of maturity has been applied to more than 150 areas inside IS. In fact, there are several examples of maturity models focused on different organization and IS areas, namely the maturity model for Intranet implementation, by Damsgaard & Scheepers [11]; the maturity model for ERP systems by Holland & Light [22]; and the CMMI maturity model for the software development process [59]. We can also mention maturity models for fields such as Software Management [2], Business Management [32], Project Management [8,26], Project Portfolio and Program Management [40], Information Management [61], IS/ ICT Management [51], e-Business [14,15,18,33], e-Learning [34], Knowledge Management [5,35], BPM – Business Process Management [56], Enterprise Architecture [13,43], etc. Maturity models for HIS management Health related organizations, and more specifically Hospital IS Management organizations, are also increasingly adopting maturity models. This use is connected to a growing provision of health care services based on electronic systems, supported by enhanced computer capacity and an increased ability to seize and share knowledge in a digital format. It is widely agreed that ISs offer significant opportunities for health care
Page 3 of 9 99
providers and health provision in general, as well as access to information required by users [55]. In this context, some maturity models emerged, namely the Quintegra Maturity Model for electronic Healthcare [60], proposed as a model that goes beyond the limits of an organization, incorporating every service linked to the medical process applied to each health care provider in each maturity stage. Another example of a maturity model in the health field is the HIMSS Maturity Model for Electronic Medical Record, which identifies different maturity stages in the Electronic Medical Record (EMR) of hospitals [19]. IDC (Health Industry Insights) has also developed a maturity model which describes the five stages of development in Hospital SIs (Table 3). This maturity model has been used all over the world by IDC, both to evaluate the maturity of IS in hospitals and to compare maturity average differences between regions and countries in different continents [23]. To these models we can add the Maturity Model for Electronic Patient Record, directed to the system that manages every patient related information, that is, a system that manages the EPR (Electronic Patient Record) [50] and the maturity model for PACS by Wetering & Batenburg [63]. National health services from several countries have also started to develop and adopt Maturity Models. That is the case of the model created by the National E-health Transition Authority of Australia [44] and baptized Interoperability Maturity Model (IMM) [24]. This model focuses on interoperability associated with technical, informational and organizational capabilities of the different players involved in health care services. Another example concerns the Maturity Model of the NHS Infrastructure Maturity Model (NIMM). This is a maturity evaluation model that helps NHS organizations carry out an objective self-assessment in terms of technological infrastructures.
Definition of the problem Health care institutions and governmental organizations are starting to understand that the reasons underlying a certain inadequacy in the management of health processes directly relates to infrastructural limitations and their inefficient management [52,60]. An analysis to the current health context clearly reveals the weight of the technological transition problem [60]. Moreover, operational information technologies have increased in complexity to answer the demands of the sector. This increase in complexity, in its turn, led to the integration of several new enterprise integration systems, processes and approaches, and the emergence of new companies providing services in this field. Consequently, many underdeveloped products and services are being consumed by HISs undergoing a process of change and demanding, more than ever, a degree of performance and effectiveness that will answer
Ad hoc unconnected; Many applications; Many gaps; Operational; Manual and Overlapping systems; computerized IS; Centralized; Operational; Uncoordinated; Concentration Mainly financial systems; in financial systems; Little Many areas unsatisfied; maintenance. Large backlog; Heavy maintenance load.
Programmers/ contractors.
Unaware.
Technical (very low level), individual expertise.
Systems
Staff
Style
Skills
Confusion
Information centers, library records, etc. in same unit; Information services.
Integration, coordination and control.
Stage IV Cooperation
IS planners; IS Manager; Data Base; Administrator; Data Administrator; Data analysts. Abrogation/Delegation.
Senior management concerned DP defensive.
Maintain comparative strategic advantage; Monitor futures; Interactive planning.
Stage VI Harmonious
Individualistic (product champion)
Decentralized systems but central control and coordination; Added value systems (more marketing oriented); More DSS-internal, less ad hoc; Some strategic systems; (using external data); Lack of external and internal data integration of communications technologies with computing. Corporate/business/IS planners (one role).
Interactive planning.
All senior management understand IS and its potentialities.
Business team.
IS Director/member of board of directors.
Inter-organizational systems (supplier, customer, government links); New IS-based products; Externalinternal data integration.
SBU coalition(s) (many but separate). Centrally coordinated coalitions (corporate and SBU views concurrently)
Environmental scanning and opportunity seeking.
Stage V Entrepreneurial
Organizational integration; IS IS Manager – member of senior executive team; Knowledgeable knows how the business works; users in some IS areas; Users know how IS works Entrepreneurial marketing skills. (for their area); Business management (for IS staff). Cooperation Opportunistic; Entrepreneurial; Entrepreneurial.
Business analysts; Information Resources Manager (Chief Information Officer). Democratic dialectic.
Still mostly centralized; Uncontrolled Decentralized approach with end-user computing; Most major some controls, but mostly business activities covered; lack of coordination; Database systems. Some DSS-ad hoc; Integrated Office technology systems.
Data processing department; Centralized DP shop; Endusers running free at Stage 1.
Systems development methodology. IS believes it knows what the business needs; Project management.
Don’t bother me (I’m too busy).
Systems analysts; DP Manager.
IS often subordinate to accounting or finance.
Top-down IS planning.
Stage III Centralized
Page 4 of 9
Super-ordinate Obfuscation goals
None.
Structure
IT audit; Find out and meet user needs (reactive)
Acquisition of hardware, software, etc.
Strategy
Stage II Foundations
Stage I BAd hocracy^
Revised Model of Maturity Stages of Galliers and Sutherland [17]
Factors
Table 2
99 J Med Syst (2015) 39:99
Secure email (provider-provider / provider-patient) Participation in regionalized patient CDR Home health case management Remote patient monitoring / telemedicine Patient appointment scheduling Computerized physician order entry Nursing documentation Emergency department management Cardiology department management Physician portal Patient portal Wireless infrastructure Inpatient electronic medical record (EMR) Ambulatory EMR Enterprise master patient index Electronic claims submission Electronic payment processing Inventory, supply requisitioning, and distribution Basic order communications E-mail Internet access Intranet Patient registration/ inpatient admission discharge and transfer Patient billing and accounts receivable HRIS/payroll General ledger / financial reporting Purchasing/accounts payable
Laboratory information RIS/radiology results reporting PACS Pharmacy Operating room scheduling and management
Stage V Digital Virtual Enterprise Stage II Advanced HIS Stage I Basic HIS
Table 3
IDC Maturity Model for HIS [23]
Stage III Advanced HIS Core Clinicals
Stage IV Digital Hospital
J Med Syst (2015) 39:99
Page 5 of 9 99
their needs. In this scenario, several questions that require a convincing answer emerge: How can we know if we are doing a good job when managing these changes and monitoring their progress on an ongoing basis? & & &
How can we manage the interactions of systems and processes in constant evolution? How can we manage the impact of low interoperability, security, reliability, efficiency and effectiveness processes? How can we evaluate the impact of current clinical and hospital software applications in the maturity development of their respective IS?
We observe that the benefits brought by modern technology to the health field, and supported by better methods and tools, cannot be obtained via undisciplined and chaotic processes [20,21]. That is why we believe that IS Management in health organizations must be carried out based on maturity models. Several maturity models have been proposed in the course of time, for personal evolution purposes, for the general evolution of organizations and for the evolution of the IS Management task in particular. The differences in these models lie specifically in the number of stages, evolution variables and focus areas [37,55]. Each of these models identifies certain characteristics that typify the target of different growth or maturity stages and are implemented in different organizations. Where health related organizations are concerned, several maturity models are also proposed. Notwithstanding the specificities of these models that distinguish them from the models of other areas, these are still in an early development stage [37,55]. In the research that was carried out, we observed that the models pertaining to the health field are poorly detailed, do not provide maturity measuring tools and do not structure the characteristics of maturity stages according to influencing factors. This reality offers an opportunity for the development of new maturity models focused on IS management in the health field that are capable of filling the previously identified gaps. Within the universe of the maturity models that we know, we believe that the model of maturity stages reviewed by Galliers & Sutherland [17] can serve as an inspiration and reference, both to define influencing factors and to develop a measuring instrument for Hospital IS maturity. Additionally, the very concept of Maturity Model is not devoid of criticism. For instance, Pfeffer & Sutton [49] argue that the purpose of maturity models is to identify a gap that can be filled with subsequent improvement measures. However, most of these models fail to describe how to effectively carry out these actions, as the closing of such gaps can be extremely difficult to illustrate. The strongest point of criticism, where
99
maturity models are involved, is their weak theoretical basis [7]. Most of these models are based on Bbest practices^ or Bsuccess factors^ connected with processes from organizations that have shown favourable results. Therefore, although these practices are compatible with the maturity model, they provide no guarantee as to the success of the organization. There is no agreement surrounding the Breal path^ that will ensure a positive result [39]. According to deBruin & Rosemann [12] the reasons for the, sometimes, ambiguous results obtained with the maturity models stem from the insufficient testing of the models in terms of validity, reliability and generalization, as well as from the lack of documents addressing the design and development process behind this type of model. For this reason, it is fundamental to describe the work underlying the development of a maturity model based on an approach sustained by DSR (Design Science Research) principles.
Investigation questions and objectives After describing the problem we elaborate the following research question: &
J Med Syst (2015) 39:99
Page 6 of 9
What is the best model and respective stages of maturity to be applied in HIS Management?
From this research question we can pose more specific questions:
&
&
The validation of the proposed conceptual model and methodology through the adoption of statistical techniques applied to the results obtained with the Delphi Method; The development of an automatic tool that will allow us to identify the stage of maturity of a given HIS and the influencing factors that must be improved in order to achieve an enhanced stage of maturity.
Methodological approach In the context of a PhD project or any other type of research work, defining a methodological approach is fundamental for the researcher, in a way that it will allow him to put into context the activities to be developed, define how the work will be carried out and explain how the results will be measured and evaluated, with a view to validate the research. A research process entails the collection, analysis and interpretation of data, with a view to understanding a certain phenomenon. This process should be systematic and, within a certain framework and established guidelines, involve the definition of objectives, the management of data and the communication of conclusions [64]. Bearing in mind the research questions and the goals established for investigation project, the adopted approach will include the following methods: Systematic Literature Review; and Delphi Method. Systematic literature review
& &
What influencing factors, associated with the stages of maturity, are perceived as being fundamental in the health care field by stakeholders? How can these factors be determined, quantified and integrated in the context of HIS maturity stages?
To answer these questions we defined the following objectives: & & & &
Through a systematic literature review, the identification of the main maturity models adopted in IS Management and characteristics of their different stages; Through a systematic literature review, the identification of the main maturity models adopted in HIS Management and characteristics of their different stages; The identification and characterization of a set of influencing factors that can be used in different stages of maturity of HISs; The proposal of a conceptual model that will allow us to put into context, classify and describe the factors that influence the maturity stages of the HIS with the application of the Delphi Method;
The starting point for the follow up of this research work will be the systematic review of the available literature in the field under study. A systematic literature review is an essential step in any research project, to the extent that it provides the researcher a solid basis to make the knowledge advance. On the other hand, a systematic literature review promotes the development of theories, the identification of fields where a multiplicity of research projects, and the identification of areas that need to the studied [62]. In this project, the systematic literature review will try to reach the following research goals: & & & & &
Identify the main maturity models adopted in IS Management and describe its different stages; Review the State of the Art in HIS Maturity Models; Identify the success factors and the limitations of this type of Maturity Models; Analyse and debate different options for the development of a conceptual maturity model in the IS field; Identify and characterize a set of factors that influence maturity, to be adopted in each maturity stage of an HIS,
J Med Syst (2015) 39:99
and which will serve as a basis for our Delphi method study. By the end of the systematic literature review, one of the most important results, besides a detailed description of the state of the art concerning HIS maturity models, will be the identification of an initial set of influencing factors connected to different stages of the maturity model. This set of factors influencing maturity will be particularly useful for a subsequent input in the first round of the Delphi Method.
Page 7 of 9 99
&
Consultants and Professionals in the HIS field.
As to the number of panel participants and according to the literature, there is no ideal size for a panel; a number of studies recommend a number between 10 and 50 participants [57]. In this project the definition of the panel will mainly depend on the contacts established and the invited entities.
Delphi method
Expected contributions
According to the herein presented methodological approach, one of the methods that will be adopted in this research project is the Delphi method. This method supports decision making based on the opinions and contributions of participants who are experienced or specialized in the field under study [31]. Moreover, this method has been widely accepted over the last decades in the IS domain, as its application in several studies demonstrates [e.g.,: [38,45,57]. In this project, the application of the Delphi Method will have the purpose of identifying and describing the main influencing factors that are to be used in different stages of the maturity models. Based on the results obtained from the participants, we expect to develop and propose a conceptual model that will allow us to put into context and describe the factors influencing the stages of maturity of a HIS. The selection of the Delphi panel is usually one of the first stages in any study of this nature. Although there are no rules underlying the definition of a panel, according to Scheele [58], three types of participants must be included in order to achieve a good variety of opinions, namely:
From the work that is to be developed we can expect the following contributions: & & & & &
A detailed review of the state of the art where HIS maturity models are concerned; The identification and characterization of a set of influencing factors that the stakeholders perceive as being mandatory in the HIS maturity context; The proposal of a model and respective stages of maturity that will be applied in the HIS Management; The incorporation of the developed model in health field related organizations with a view to facilitate the HIS Management task; The proposal of an automatic tool that will allow us to identify the stage of maturity of a given HIS and the influencing factors that need to be improved in order to achieve an enhanced stage of maturity.
Concluding remarks & & &
Stakeholders: the elements who are involved and interested in the field under study and who are directly affected; Experts: those who are experienced or specialized in a relevant field; Facilitators: those who have the power to clarify, organize and synthesize.
Additionally, and in certain circumstances, we may consider other types of participants, people who can provide alternative perspectives and points of view [58]. The predominance of each type of participant will depend on the characteristics of the study to be developed. In the context of the present research work, we intend to invite the following types of participants for our study panel: & &
CEOs and CIOs of health organizations with implemented HISs; HIS projects leaders;
In the present paper we put forth the initial stage of an investigation aimed at the development of an encompassing maturity model for hospital information systems management, justified by several existing limitations in current maturity models in the health field. A future work will involve systematic reviews of the available literature concerning maturity models for information systems management in general and hospital information systems management in particular, which will allow us to identify a set of potential influencing factors that must be considered during the initial development stage of the maturity model, to be defined with the Delphi Method. The resulting maturity model will be validated in a group of hospital organizations to be defined, and subsequently we will develop an automatic tool that will allow us to identify the stage of maturity of a given HIS and the path that must be pursued towards a growing maturity.
99
References 1. 2.
3.
4.
5.
6. 7.
8.
9.
10.
11.
12.
13. 14. 15. 16.
17.
18.
19.
J Med Syst (2015) 39:99
Page 8 of 9
Amaral, L., e Varajão, J., Planeamento de Sistemas de Informação, 4th Edition. FCA, 2007. April, A., Abran, A., and Dumke, R., Assessment of software maintenance capability: A model and its architecture. In: Proceedings of the 8th European Conference on Software Maintenance and Reengineering (CSMR2004). IEEE Computer Society Press, Los Alamitos CA, pp. 243–248, 2004. Auer, T., Beyond IS implemention: A skill-based aproach to IS use. Paper presented at the 3rd European Conference on Information Systems: Athens, Greece, 1995. Becker, J., Knackstedt, R., and Pöppelbuß, J., Developing maturity models for IT management. Bus. Inf. Syst. Eng. 1(3):213–222, 2009. Berztiss, A.T., Capability maturity for knowledge management. In: DEXA Workshop, IEEE Computer Society, 162– 166, 2002. Bhidé, A. V., The Origin and Evolution of New Businesses. Oxford University Press, New York, NY, USA, 2000. Biberoglu, E., and Haddad, H., A survey of industrial experiences with CMM and the teaching of CMM practices. J. Comput. Sci. Coll. 18(2):143–152, 2002. Brookes, N. & Clark, R., Using maturity models to improve project management practice. POMS 20th Annual Conference, May 1 to May 4, 2009: Orlando Florida USA, 2009. Burn, J. M. A., Revolutionary staged growth model of information systems planning. Proceedings of the International Conference on Information Systems: Vancouver, British Columbia, Canada, pp. 395–406, 1994. Caralli, R., & Knight, M., Maturity models 101: A primer for applying maturity models to smart grid security, resilience, and interoperability. Software engineering institute: Carnegie Mellon University, 2012. Damsgaard, J., and Scheepers, R., Managing the crises in intranet implementation: A stage model. Inf. Syst. J. 10(2):131–149, 2000. doi:10.1046/j.1365-75.2000. 00076.x. deBruin, T., & Rosemann, M., Understanding the main phases of developing a maturity assessment model. Proceedings of the 16th Australasian Conference on Information Systems: Sydney, Australia, 2005. Doc (2003), IT Architecture Capability Maturity Model, Department of Commerce, USA Government Introduction, 2003. Earl, M. J., Management Strategies for Information Technologies. Prentice Hall, Upper Saddle River, NJ, 1989. Earl, M. J., Evolving the EBusiness. Bus. Strateg. Rev. 11(2):33–38, 2000. Freixo, J. & Rocha, Á., QUALITUS: An integrated information architecture for the quality management system of hospitals. In A. Rocha et al. (eds.), New Contributions in Information Systems and Technologies, Advances in Intelligent Systems and Computing 353: Sprinder. doi:10.1007/978-3-319-16486-1_29, 2015. Galliers, R. D., and Sutherland, A. R., Information systems management and strategy formulation: The ‘stages of growth’ model revisited. J. Inf. Syst. 1(2):89–114, 1991. doi:10.1111/j.1365-2575. 1991.tb00030.x. Gardler, R., and Mehandjiev, N., Supporting Component-Based Software Evolution. In: Aksit, M., Mezini, M., and Unland, R. (Eds.), Objects, Components, Architectures, Services, and Applications for a Networked World, Series: Lecture Notes in Computer Science, 2591. Springer Verlag, Berlin, pp. 103–120, 2003. Garets, D., and Davis, M., Electronic Medical Records versus Electronic Health Records: Yes, there is a difference. HIMSS Analytics, Chicago, IL, 2006.
20.
21.
22.
23.
24.
25. 26.
27.
28.
29.
30.
31.
32. 33.
34. 35. 36. 37.
38.
39.
40.
41.
Gonçalves, J., and Rocha, Á., A decision support system for quality of life in head and neck oncology patients. Head Neck Oncol. 4(3): 1–9, 2012. doi:10.1186/1758-3284-4-3. Gonçalves, J., Silveira, A., and Rocha, Á., A platform to study the quality of life in oncology patients. Int. J. Inf. Syst. Chang. Manag. 5(3):209–220, 2011. doi:10.1504/IJISCM.2011.044501. Holland, C., and Light, B., A stage maturity model for enterprise resource planning systems. Database Adv. Inf. Syst. 32(2):34–45, 2001. Holland, M., Piai, S., and Dunbrack, L. A., Healthcare IT Maturity Model: Western European Hospitals - The Leading Countries (Tech. Rep. No. HI210231). IDC Health Insights, Framingham, MA, 2008. N H S I n f r a s t r u c t u r e M a t u r i t y M o d e l . h t t p : / / w w w. connectingforhealth.nhs.uk/systemsandservices/nimm (Retrieved in Set/2014). Huff, S. L., Munro, M. C., and Martin, B. H., Growth stages of enduser computing. Commun. ACM 31(5):542–550, 1988. Kerzner, H., Using the Project Management Maturity Model: Strategic Planning for Project Management, 2nd edition. John Wiley & Sons, New York, 2005. Khandelwal, V. e Ferguson, J., Critical success factors (CSFs) and the growth of IT in selected geographic regions. Proceedings of 32nd Hawaii International Conference on Systems Sciences (HICSS-32), USA, 1999. King, J., and Kraemer, K., Evolution and organizational information systems: An assessment of Nolan’s stage model. Commun. ACM 27(5):466–475, 1984. King, W. R., and Teo, T. S. H., Integration between business planning and information systems planning: Validating a stage hypothesis. Decis. Sci. 28(2):279–307, 1997. Kohlegger, M., Maier, R. & Thalmann, S., Understanding maturity models results of a structured content analysis proceedings of IKNOW and I-SEMANTIC 09. School of Management, Information Systems, Austria. Graz, Austria: University of Innsbruck, 2009. Landeta, J., Current validity of Delphi method in social sciences. Institute of Applied Business Economics of University of the Basque Country (UPV/EHU), Bilbao, Spain, 2005. Levin G. & Nutt, H., Achieving excellence in business development: The business development capability maturity model, 2005. Ludescher, G. & Usrey, M., Towards an ECMM (E-Commerce Maturity Model). In Proceedings of the First International Research Conference on Organizational Excellence in the Third Millennium. Estes Park: Colorado State University, 2000. Marshall, S. (2007) E-Learning maturity model. http://www.utdc. vuw.ac.nz/research/emm/ (Retrieved in Set/2014). Maybury, M.T. (2002), Knowledge Management at the MITRE Corporation. http://www.mitre.org. McKenney, J. L., and McFarlan, F. W., The information archipelago – maps and bridges. Harv. Bus. Rev. 60(5):109–119, 1982. Mettler, T., A Design Science Research Perspective on Maturity Models in Information Systems. University of St. Gallen, St. Gallen, 2009. Miaskiewicz, T. and Kozar, K., The use of the delphi method to determine the benefits of the personas method – an approach to systems design. Proceedings of the Fifth Annual Workshop on HCI Research in MIS: Milwaukee, 50–54, 2006. Montoya-Weiss, M. M., and Calantone, R. J., Determinants of new product perfomance: A review and meta-analysis. J. Prod. Innov. Manag. 11(5):397–417, 1994. Murray, A. (2006), Capability Maturity Models - Using P3M3 to Improve Performance. Vol 2; Issue 0616-01-12. Available: www. outperform.co.uk (Retrieved in Set/2014). Mutafelija, B., and Stromberg, H., Systematic process improvement using ISO 9001:2000 and CMMI. Artech House, Boston, 2003.
J Med Syst (2015) 39:99 42.
Mutsaers, E., Zee, H., and Giertz, H., The evolution of information technology. Inf. Manag. Comput. Secur. 6(3):115–126, 1998. 43. Nascio 2003, NASCIO enterprise architecture maturity model, version 1.3, National association of state chief information officers: December 2003. 44. NEHTA, NEHTA Interoperability maturity model. 2007 ed. Level 25, 56 Pitt Street, Sydney, NSW, 2000, Australia: National EHealth Transition Authority Ltd, 2007. 45. Niederman, F., Brancheau, J. C., and Wetherbe, J. C., Information systems management issues for the 1990s. MIS Q. 15(4):475–500, 1991. 46. Nolan, R., Managing de computer resource: A stage hypotesis. Commun. ACM 16(7):399–405, 1973. 47. Nolan, R., Managing the crisis in data processing. Harv. Bus. Rev. 57(2):115–126, 1979. 48. Nolan, R., and Koot, W., Nolan stages theory today: A framework for senior and IT management to manage information technology. Holland Manag. Rev. 31:1–24, 1992. 49. Pfeffer, J., and Sutton, R., Knowing Bwhat^ to do is not enough: Turning knowledge into action. Calif. Manag. Rev. 42(1):83–108, 1999. 50. Priestman, W., ICT strategy 2007–2011 for the royal liverpool and broadgreen university hospitals nhs trust. Trust Board Meeting 6th November 2007. Document Number: V1.4, 2007. 51. Renken, J., Developing an IS/ICT manage¬ment capability maturity framework. In: Re¬search conference of the South African Insti¬tute for Computer Scientists and Information Technologists (SAICSIT). Stellenbosch, 53–62, 2004. 52. Rocha, Á., and Rocha, B., Adopting nursing health record standards. Inf. Health Soc. Care 39(1):1–14, 2014. doi:10.3109/ 17538157.2013.827200. 53. Rocha, Á., and Vasconcelos, J., Os modelos de maturidade na gestão de sistemas de informação. Rev. Fac. Cienc. Tecnol. Univ. Fernando Pessoa 1:93–107, 2004.
Page 9 of 9 99 54.
Rocha, Á., Maturidade da Função Sistema de Informação: Teoria de Estádios, Modelos e Avaliação. Universidade Fernando Pessoa, Porto, Portugal, 2002. 55. Rocha, Á., Evolution of information systems and technologies maturity in healthcare. Int. J. Healthc. Inf. Syst. Inform. (IJHISI) 6(2): 28–36, 2011. doi:10.4018/jhisi.2011040103. 56. Rosemann, M. & deBruin, T., Business process management maturity - a model for progression. In Proceedings of the 13th ECIS, May: Regensburg, 2005. 57. Santos, L.D.d. (2004), Factores Determinantes do Sucesso de Adopção e Difusão de Serviços de Informação Online em Sistemas de Gestão de Ciência e Tecnologia, Tese de Doutoramento, Departamento de Sistemas de Informação, Universidade do Minho. 58. Scheele, D.S. (2002), Reality Construction as a Product of Delphi Interaction, in: The Delphi Method: Techniques and Applications H.A. Linstone and M. Turoff (eds.), http://is.njit.edu/pubs/ delphibook/ (Retrieved in Set/2014). 59. SEI Software Engineering Institute, CMMI® for development, version 1.3, improving processes for developing better products and services, (Tech. Rep. No. CMU/SEI-2010-TR-033): Carnegie Mellon University, 2010. 60. Sharma, B., Electronic Healthcare Maturity Model (eHMM). Quintegra, Chennai, India, 2008. 61. Venkatesh, V., et al., User acceptance of information technology: Toward a unified view. MIS Q. 27(3):425–478, 2003. 62. Webster, J., and Watson, R. T., Analyzing the past and prepare for the future: Writing a literature review. MIS Q. 26(2):13–23, 2002. 63. Wetering, R., and Batenburg, R., A PACS maturity model: A systematic meta-analytic review on maturation and evolvability of PACS in the hospital enterprise. Int. J. Med. Inform. 78:127–140, 2009. doi:10.1016/j.ijmedinf.2008.06.010. 64. Williams, C., Research methods. J. Bus. Econ. Res. 5(3):65–71, 2007.
