eHealth Beyond the Horizon – Get IT There S.K. Andersen et al. (Eds.) IOS Press, 2008 © 2008 Organizing Committee of MIE 2008. All rights reserved.
461
Developing a taxonomy of communication errors in heterogeneous information systems a,1
a
Samrend SABOOR , Elske AMMENWERTH a
Institute for Health Information Systems, UMIT - University for Health Sciences, Medical Informatics and Technology, Hall in Tyrol, Austria
Abstract: Background: Although established communication standards do exist in the health care domain (i.e., DICOM and HL7) the communication within heterogeneous information systems still shows a variety of errors. The complexity of these systems aggravates the identification of error reasons. A structured summary of communication errors and their reasons is essential for developing methods that support the error detection. Methods: In order to summarize communication errors, a systematic literature review in PubMed was conducted. Selected references were filtered iteratively and analyzed by applying subsuming qualitative content analysis. Results: The taxonomy currently contains 12 different problem classes that group42 problems, with in total 130 reasons. Discussion: Although, not all selected literature references are yet analyzed, we observe a saturation concerning new errors/error classes. In order to increase validity and completeness expert interviews are in planning stage. However, the first results are promising. Keywords: Classification, HIS management, Systems architecture
1. Introduction The correct transmission of information objects between involved computer-based application systems (e.g., order entry system) has become vital for processes in health care institutions [1]. Involved communication partners must agree upon conventions in order to cooperate effectively [2-4]. These conventions pertain the structure of information objects (e.g., as sets of identifier-attributes pairs), the meaning of each attribute and consequently each application system’s communication interfaces. Although there are established communication standards in the health care domain (i.e., DICOM (Digital Imaging and Communications in Medicine, [5]) and HL7 (Health Level 7, [6])) the communication within heterogeneous information systems is still error-prone [7-9]. Even the usage of just one of the standards requires additional implementation efforts [10, 11]. A main reason for this is because standard definitions allow misinterpretations. Additionally, in case of communications including different standards, difficulties in matching the particular information models can lead to 1
Corresponding Author: Samrend Saboor. University for Health Sciences, Medical Informatics and Technology (UMIT) – Eduard Wallnöfer-Zentrum 1, 6060 Hall in Tyrol, Austria.
[email protected]
462
S. Saboor and E. Ammenwerth / Developing a Taxonomy of Communication Errors
fundamental content-related errors. Therefore, an additional framework is needed that coordinates the usage of the different communication standards [12, 13]. This framework is provided by the international initiative IHE (Integrating the Healthcare Enterprise). However, integrated hospital information systems mostly are grown architectures that include proprietary workarounds. Changes in these systems mostly succeed because of experienced IT-employees. On the other side, communications which appear successful can still have mistakes in the exchanged content. In these cases, the underlying reasons are mostly difficult to identify. Further, even changes in well working infrastructures may cause negative side-effects which are hardly predictable (e.g. [8, 13]). Problems related to communication infrastructure and processes affect the quality of patient treatment and must be examined carefully – detecting communication errors in advance. However, it seems that there is no present method that aims to assess health care communication processes. Existing methods mainly focus on time measurements (e.g., MOSAIK-M [14]) or perform reachability analyses on Petri-net based models in order to detect bottlenecks and best performing variations [15]. These methods either ignore information objects or provide only simple representation per default. In the latter case, information objects are not defined on a formal base and thus cannot be utilized by automated assessment methods. Thus, a method is needed that considers the characteristics of information objects and their processing in heterogeneous information systems. Here, an important prerequisite is a concise summary of weaknesses which can occur within the communication between computer-based application systems. Aim of this paper: This paper presents a taxonomy of weaknesses that can occur within the communication between computer-based application systems.
2. Methods Based on our experiences from earlier projects in the area of process assessment (e.g., [16]), we conducted a systematic review of available literature in PubMed in order to collect communication errors and their underlying reasons. Figure 1 illustrates the inductive approach we chose: After the initial declaration of the review’s aim, we derived search phrases from this aim in the second step – starting with simple search phrases like “quality information processing” or “experience information processing”. In steps three and four the title and abstract of all references which resulted from the search phrases were reviewed. In step five, we adjusted and augmented our search phrases according to the adequateness of the resulting references. In this way, we found 4188 references. These references dealt amongst others with information management (e.g., [17]), reports on integration projects in the field of Hospital Information System (HIS), Radiology Information System (RIS) and Picture Archiving and Communications Systems (PACS) (e.g., [18, 19]). From this set we dropped all references which were older than 20 years, dealt with the implementation of very specialized software applications or dealt with organizational issues. In the sixth step, we performed qualitative content analyses on the remaining 426 references. Here, we chose a subsuming type of qualitative content analysis (according to Mayring [20]) which aims to filter the main content by abstraction and dynamic declaration of categories. In the sixth step, also further keywords were found which we used in step
463
S. Saboor and E. Ammenwerth / Developing a Taxonomy of Communication Errors
seven to adjust our set of search phrases. The process stops when saturation of new errors is reached. The found weaknesses were collected in a taxonomy which is presented in Table 1 in the Results section.
1. Declaration of aim
2 .Selection of keywords
3. Analyze title
/
4. Analyze abstract
Qualitative content analysis 6. Collect weak points & reasons 7. Acquisition of new keywords 8. End of acquisition
[Reached sa tur atio n]
5. Identification of bad keywords acquisition of new keywords
Figure 1: Process of literature review – After declaring the review’s aim, appropriate keywords and references are searched iteratively. Adequate results are analyzed for communication weak points and their reasons. The process stops after an observable saturation
3. Results The found communication errors were collected in a taxonomy – a three-levelhierarchy: It groups communication errors into separate problem classes and names reasons for the occurrence of each error. Table 1 shows an excerpt of the taxonomy. Currently, the taxonomy contains 12 problem classes. In these, 42 problems are grouped. For these problems 130, partly redundant, reasons were collected. Additionally, 49 problems with 62 reasons are already collected but not yet inserted into the taxonomy. However, a saturation for the problem classes and the actual problems is already observable. Further details regarding the taxonomy’s content as well as concepts of how to make use of it will be presented on the MIE conference 2008.
Table 1: Excerpt of the taxonomy of communication errors Problem class
Problem
Reason for problem
Input errors
Wrong data entries
Mistakes due to manual data entry, missing entry checking routines, no usage of automated data gathering routines
Identification errors
Missing or wrong identification of information object instances
Incompatible identification attributes between requester and provider, wrong or missing assignment of identification, several systems inconsistently assign identifications to same information object
Conversation errors
Mistakable conversation
Not all details are transferred via electronic communication paths
Values of important attributes are missing
Optional / textual data fields are missing or filled with wrong values, important details are filled into optional data fields, missing identification of essential data fields, important details are passed verbally
Acquisition errors
See problem class "Acquisition errors"
Input errors
See problem class "Input errors"
Incomplete content
464
Conversion errors
Consistency errors
Transmission errors
S. Saboor and E. Ammenwerth / Developing a Taxonomy of Communication Errors
Incompatible value representation of data attributes
Usage of data attributes with implicit value representation, conversion error in communication standard, different length encodings, different data dictionaries, incompatible character sets
Incompatible data models
Usage of proprietary information objects / dependency on proprietary attributes, usage of proprietary communication protocols, different/incompatible information models, usage of obsolete communication standard versions,
Missing content
See problem "Values of important attributes are missing"
Uncoordinated data entry
See problem class "Input errors", multiple independent input-interfaces, multiple independent/not synchronized data bases
Missing data cleansing
No central data repository, missing central data correction
Corrupted main repository
No quality assurance procedures for main data repository
Delayed failure correction
Defective information objects are re-sent with delays
Redundant data management
No main repository or missing connections to established repository, oneway connections just allow data synchronization in one direction, redundant data entries in one or more repositories
No version management
Storage of incomplete versions of single information object instances
Unsupported services
The called application system doesn't support the requested service, usage of outdated application systems, new version of application system has other communication interfaces
Unsupported content
The called application system doesn't support the request type of information object, receiving system stores transferred information object differently than the original (different information model)
Missing processing rules
Missing trigger for continuation of information processing
Unstable software versions
Usage of unstable/experimental application systems
Restrictive security setup
Firewall blocks legal communications
Malfunctions of network
No alternative network-components (missing uninterrupted power supply unit, switch, cable), no security redundancies
Communication disruption / Incomplete communication
Communication partners are not connected directly (no synchronous comparison), peer misses to reply instantly
Communication with wrong partner
Ambiguous identification of application systems
Conversion errors
See problem class "Conversion errors"
Missing authorization Acquisition/storage errors Missing tracking of information objects Availability errors
Dataloss/
Incompatible/Missing or wrong identification
No or wrongly configured authorization prohibits access, information objects are not stored on central servers - no distributed access See problem class "Incomplete content", shortage of storage space Multiple duplications, storages and transmissions of information object instances, paper-based information objects easily disappear See problem class "Identification errors"
Insufficient network bandwidth
No direct connection - data must be transferred offline via storage media, usage of mixed storage media - necessity of transcriptions
Concurring access
Usage of paper-based storage media, paper-based information objects not available for distributes access
Physical data corruption
System crash or improper system shut-down, sensitive storage media
S. Saboor and E. Ammenwerth / Developing a Taxonomy of Communication Errors damage errors
Acquisition errors
465
Missing backup
Missing redundant data storage/secure long-term-archive, no usage of RAID-technology, no dedicated backup-server
Error during acquisition
Missing final visual control/quality assurance, disruptions during acquisition, exporting of incomplete information objects, no structure data entry forms
4. Discussion Communication processes in health information systems are complex interactions between multiple application systems with different and possibly diverse interpretation of common communication standards. Errors in the electronic communication of clinical documents can negatively effect patient treatment and thus must be detected in advance. Here, a structured summary of such errors and their reasons is the essential prerequisite for the development of adequate detection methods – these methods could then be used, for instance, by the hospital’s information management department during the planning stage of new integration projects but also befor updating central application systems. The taxonomy presented in this paper is meant to be such a collection. Its details were gathered through a qualitative content analysis on available literature that deals with experience reports about integration projects and the implementation of common communication standards (i.e., HL7 and DICOM). Although, not all selected literature references are yet analyzed, we observe a saturation concerning new errors/error classes. Thus, the taxonomy seems to be nearly finished. However, the subsuming qualitative approach we selected for gathering the taxonomy’s details strongly depends on the selected literature references. This selection in turn inherently contains a certain degree of subjectivity resulting from subjectively selected search phrases. Amongst others, this affects the validity and completeness of our taxonomy. Thus, we tried to reduce this subjectivity by iteratively refining the search phrases according to the found literature resources. Further, we are currently planning expert interviews in order to increase the taxonomy’s validity and completeness. However, the already collected results are encouraging. As far as we can see, a concise summarization of concrete communication such as the taxonomy is still missing – current publications (e.g., [17]) mostly concentrate on single problems or discuss them on an abstract level. 5. Conclusion It is difficult to understand the reasons for communication errors and to overlook possible side-effects within integrated information systems. A structured taxonomy of possible errors and their reasons is an essential prerequisite for solving this difficulty. In this paper we presented such a taxonomy which groups communication errors into problem classes and also names reasons for each of the errors. In order to parameterize this taxonomy, we currently develop a formal method. The method’s concept is explained in [21] in more detail. 6. References 1. 2.
Coiera E. When conversation is better than computation. JAMIA 2000;7(3):277-286. Brazhnik O, Jones JF. Anatomy of data integration. J Biomed Inform 2007;40(3):252-69.
466 3. 4. 5. 6. 7. 8.
9. 10.
11. 12. 13. 14.
15.
16. 17. 18. 19. 20. 21.
S. Saboor and E. Ammenwerth / Developing a Taxonomy of Communication Errors Giuse DA, Kuhn KA. Health information systems challenges: the Heidelberg conference and the future. Int J Med Inform 2003;69(2-3):105-14. Kun L. Interoperability: the cure for what ails us. IEEE Eng Med Biol Mag 2007;26(1):87-90. ACR/NEMA. DICOM Homepage. 2007 [cited 2007 Jun]; Available from: http://medical.nema.org HL7. Health Level 7. 2007 [cited 2007 Jun]; Available from: http://hl7.org Hammond KW, Helbig ST, Benson CC, Brathwaite-Sketoe BM. Are electronic medical records trustworthy? Observations on copying, pasting and duplication. AMIA Annu Symp Proc 2003:269-73. Nissen-Meyer S, Holzknecht N, Wieser B, Francke M, Küttner B, Adelhard K, et al. Produktivitätsverbesserung durch klinikweite RIS und PACS: eine Fallstudie. Der Radiologe 2002;42:351-60. Khorasani R. Medical management: expanding radiologists' role using information technology to improve the quality of care. Semin Roentgenol 2003;38(3):282-6. Gross-Fengels W, Miedeck C, Siemens P, Appel R, Muckner K, Finsterbusch J, et al. [PACS: from project to reality. Report of experiences on full digitalisation of the radiology department of a major hospital]. Radiologe 2002;42(2):119-24. Matthews JW, Bosch WR. Explicit-VR transfer syntax limits the value multiplicity of DICOM data elements with decimal string (DS) value representation. Phys Med Biol 2006;51(5):L11-2. HIMMS/RSNA. IHE - changing the way healthcare connects. 2007 [cited 2007 Jun]; Available from: http://www.ihe.net Mildenberger P, Wein B, Bursig HP, Eichelberg M. [Current developments in DICOM and IHE]. Radiologe 2005;45(8):682-9. Hoffmann I, Bergmann J, Bott OJ, Pretschner DP. Einsatz einer rechnergestützten Modellierungs- und Simulationsumgebung für den Entwurf telemedizinischer Systeme am Beispiel von MOSAIK-M. In: Steyer G, Tolxdorff T, editors. TELEMED; 2005 2005; Berlin: Aka-Verlag; 2005. p. 309-20. Blake J, Carter M, O'Brien-Pallas L, McGillis-Hall L. A formal model for quality assurance in coloproctologic surgery. In: Greenes R, Peterson H, Protti D, editors. MEDINFO 1995, Proceedings of the 8th World Congress on Medical Informatics. Amsterdam: North Holland; 1995. p. 527-31. Saboor S, Chimiak-Opoka J, Ammenwerth E. Supporting the systematic assessment of clinical processes: the MedFlow method. Methods Inf Med 2007;46(5):586-594. Haux R, Winter A, Ammenwerth E, Brigl B. Strategic Information Management in Hospitals. New York, USA: Springer-Verlag; 2004. Mongkolwat P, Bhalodia P, Gehl JA, Channin DS. Validating DICOM content in a remote storage model. J Digit Imaging 2005;18(1):37-41. Kotter E, Langer M. Integrating HIS-RIS-PACS: the Freiburg experience. Eur Radiol 1998;8(9):170718. Mayring P. Einführung in die qualitative Sozialforschung: eine Anleitung zu qualitativem Denken. 5., überarb. Aufl. ed. Weinheim: Beltz, Psychologie-Verl.-Union; 2002. Saboor S, Ammenwerth E. Assessing communication processes within integrated health information systems. Accepted for publication in: Proceedings of the Sixth IASTED International Conference on Biomedical Engineering; 2008; Innsbruck, Austria; 2008.
