Perceptions of the Effect of Information and Communication ...

THE PRACTICE OF EMERGENCY MEDICINE/ORIGINAL RESEARCH

Perceptions of the Effect of Information and Communication Technology on the Quality of Care Delivered in Emergency Departments: A Cross-Site Qualitative Study Joanne Callen, MPH (Research), PhD; Richard Paoloni, MBBS (Hons 1), MMed (Clin Epi); Julie Li, BAppSc (HIM) Hons 1; Michael Stewart, BIntS, MIPH; Kathryn Gibson, BM B Ch (Medicine), PhD; Andrew Georgiou, MSc, PhD; Jeffrey Braithwaite, MBA, PhD; Johanna Westbrook, MHA, PhD

Study objective: We identify and describe emergency physicians’ and nurses’ perceptions of the effect of an integrated emergency department (ED) information system on the quality of care delivered in the ED. Methods: A qualitative study was conducted in 4 urban EDs, with each site using the same ED information system. Participants (n⫽97) were physicians and nurses with data collected by 69 detailed interviews, 5 focus groups (28 participants), and 26 hours of structured observations. Results: Results revealed new perspectives on how an integrated ED information system was perceived to affect incentives for use, awareness of colleagues’ activities, and workflow. A key incentive was related to the positive effect of the ED information system on clinical decisionmaking because of improved and quicker access to patient-specific and knowledge-base information compared with the previous stand-alone ED information system. Synchronous access to patient data was perceived to lead to enhanced awareness by individual physicians and nurses of what others were doing within and outside the ED, which participants claimed contributed to improved care coordination, communication, clinical documentation, and the consultation process. There was difficulty incorporating the use of the ED information system with clinicians’ work, particularly in relation to increased task complexity; duplicate documentation, and computer issues related to system usability, hardware, and individuals’ computer skills and knowledge. Conclusion: Physicians and nurses perceived that the integrated ED information system contributed to improvements in the delivery of patient care, enabling faster and better-informed decisionmaking and specialty consultations. The challenge of electronic clinical documentation and balancing data entry demands with system benefits necessitates that new methods of data capture, suited to busy clinical environments, be developed. [Ann Emerg Med. 2013;61:131-144.] Please see page 132 for the Editor’s Capsule Summary of this article. A feedback survey is available with each research article published on the Web at www.annemergmed.com. A podcast for this article is available at www.annemergmed.com. 0196-0644/$-see front matter Copyright © 2012 by the American College of Emergency Physicians. http://dx.doi.org/10.1016/j.annemergmed.2012.08.032

INTRODUCTION Background Emergency departments (EDs) in the United States and other developed countries are facing critical challenges related to high demand for their services and inadequate system capacity.1 Information and communication technology is believed to be an essential strategy for creating efficiencies and improvements in health care delivery and patient safety.2,3 A key recommendation of the Institute of Medicine’s report Hospital-Based Emergency Care: At the Breaking Point1 advocates the use of information and communication technology to improve patient flow, operational efficiency, and the quality of patient care. Emergency medicine is ideally suited to the application of information and communication Volume , .  : February 

technology to facilitate physicians’ and nurses’ collaborative work4,5 and their need to access and share information essential for the delivery of optimal patient care.6-8 Despite meaningful use incentives provided under the American Recovery and Reinvestment Act,9 a recent survey reported that less than 2% of EDs in the United States have a fully functional ED information system.10 Importance There are gaps in the research literature reporting information and communication technology use in EDs. Few studies have been published on ED information systems integrated with the hospital’s clinical information system,11,12 with most focused on specific functionalities such as the electronic whiteboard,13 computerized provider order entry Annals of Emergency Medicine 131

Effect of Information and Communication Technology on Emergency Department Quality of Care

Editor’s Capsule Summary

What is already known on this topic Health information technology has had mixed effects on the quality, safety, and efficiency of care and has been little studied in emergency departments (EDs). What question this study addressed This qualitative study sampled 98 ED caregivers’ perceptions of the benefits and burdens associated with the introduction of an integrated ED information system in 4 institutions in New South Wales, Australia. The system did not include direct electronic charting. What this study adds to our knowledge Positive impressions related to better access to more complete information and better sharing of information between physicians and nurses. Negative impressions related to increased workload and complexity and poor fit with workflow. How this is relevant to clinical practice For information technology to be successfully introduced into ED practice, it must maximize benefits and decrease burdens, which may require the development of new methods of capturing data suitable to the ED environment. systems for laboratory orders,14-18 medication management,19,20 and digital radiology.21-23 Studies are also almost always undertaken in single hospital ED sites,11-14,16-20,22-25 which reduces the generalizability of findings. Few studies have examined the ways in which ED information systems support patient care.11,13,22 Given the potential of information and communication technology to improve patient care delivery and outcomes but also to facilitate new kinds of clinical errors26-28 and introduce additional costs,29,30 it is vital to investigate its use in actual clinical settings. There is also poor understanding of why some hospital departments achieve efficiencies and positive influences on health care delivery by using information and communication technology and others do not.15,29 Studies have shown that contextual factors play a part, including people, organizational culture, and work practices.31-34 Qualitative methods are ideally suited to explore people’s experiences in natural, actual settings.35-37 Because these methods deal with words rather than numbers, analysis is different but as meticulous as that used by quantitative researchers.38 Analytic rigor is achieved by various methods, including triangulation or corroboration of results from different data sources (for example, interviews and observations), analysis by multiple researchers, and respondent 132 Annals of Emergency Medicine

Callen et al

validation to establish concordance between the views expressed by study participants and results reported by investigators.39-41 Others have shown that applying qualitative methods to evaluate clinical information systems can facilitate a detailed understanding of the complexities of use in clinical environments.42-45 To our knowledge, there have been no multisite qualitative studies of physicians’ and nurses’ views about the effect of an ED information system interfaced with a hospital’s clinical information system on patient care. Goals of This Investigation The objective of this study was to describe physicians’ and nurses’ perceptions of the effect of a commercial integrated ED information system on the quality of care delivered in 4 EDs.

MATERIALS AND METHODS Do emergency physicians and nurses perceive that an integrated ED information system affects patient care, and if so, how? Study Design This research is part of a larger study examining the use of information and communication technology to support effective work practice innovation.46 A cross-sectional qualitative research design incorporating semistructured interviews, focus groups, and observations was used to enable detailed insights.47 Institutional review board approval for the study was granted by the human research ethics committees of the study hospitals and the University of New South Wales, Australia. Setting and Selection of Participants The study took place in the EDs of 4 Australian urban public teaching hospitals* within the same city and managed under the same geographic administrative structure (Table 1). Senior emergency physicians in the study sample were those who had completed specialist training in emergency medicine. The study hospitals had no senior emergency physicians on site during night shifts and limited on-site coverage during the weekend and evenings. Senior physicians were on call rather than on site during these times. Hospitals were enrolled according to their use of the same commercial integrated ED information system (Cerner FirstNet), with centralized organization and management of the 4 hospitals’ information and communication technology. The study hospitals maintained their own governance and management of the ED information system (Cerner FirstNet), which enabled locally controlled implementations and customization of the system. Conversely, many other hospitals in the state of New South Wales, Australia, have implemented Cerner FirstNet under governance from the State Department *Australia has a universal health insurance system similar to those in the United Kingdom and Canada. Public hospitals provide the majority of inpatient care, and many privately insured patients are also treated in the public hospital system.

Volume , .  : February 

Callen et al


Table 1. Characteristics of 4 hospitals and EDs. Hospital Sites Characteristics* Hospital beds Annual inpatient discharges Annual ED attendances Annual ED discharges home Percentage of hospitalizations for which the ED is responsible ED admission rates, % Commenced implementation of the integrated EDIS

A

B

C

D

445 29,939 51,105 40,323 36

758 83,898 61,939 40,713 25

543 45,055 35,687 23,019 28

750 69,678 63,650 43,532 29

21 August 2008

34 March 2008

36 May 2007

32 October 2007

EDIS, ED information system. *Statistics from annual reports of 4 study hospitals for 2010.

Table 2. Characteristics of participants from the 4 EDs. ED Site A

B

C

D

Title

N

Female

Male

Age, Years

Range of Time in Current ED (Mean)

Range of Time in EDs Generally (Mean)

Senior physicians Junior physicians Nurses Senior physicians Junior physicians Nurses Senior physicians Junior physicians Nurses Senior physicians Junior physicians Nurses

12 6 10 14 8 7 8 7 7 8 4 6

1 2 8 5 4 6 3 6 7 6 2 3

11 4 2 9 4 1 5 1 0 2 2 3

35–57 26–35 29–54 33–48 26–33 23–45 40–50 25–53 28–58 38–50 26–31 28–43

5 mo–10 y (2.7 y) 6 mo–2 y (1.3 y) 3–18 y (10.0 y) 2 mo–17 y (5.0 y) 7 mo–7 y (1.3 y) 4 mo–20 y (6.3 y) 19 mo–10 y (6.1 y) 16 mo–9 y (2.8 y) 4 mo–20 y (6.4 y) 2–15 y (10.2 y) 9 mo–3 y (1.7 y) 1.5–9 y (4.3 y)

1–20 y (8.8 y) 6 mo–4 y (2.3 y) 9–18 y (13.3 y) 6–20 y (11.2 y) 7 mo–7 y (2.5 y) 4 mo–20 y (10.6 y) 13–23 y (15.8 y) 4 mo–9 y (5.6 y) 4–37 y (12.6 y) 11–21 y (18.0 y) 1–5 y (2.8 y) 18 mo–4 y (6.4 y)

of Health. These latter implementations, known as the “State Base Build,” have been the subject of some controversy and the focus of several external reviews.48-50 This integrated system allowed health professionals access to clinical and management information relating to patients attending any of the 4 study hospitals. The total sample of physicians and nurses was 97 (Table 2). Sixty-nine detailed interviews were conducted with 42 senior physicians, 11 junior physicians, and 16 senior nurses. These were complemented with 5 focus groups (2 with junior physicians, 14 participants in total; 3 with nurses, 14 participants in total) and 26 hours of structured observations. Participants were selected purposefully according to variation of ED information system usage across a broad spectrum of senior and junior physicians and nurses. Senior physicians and nurses were targeted for interviews, with more junior clinicians participating in focus groups. The sample was weighted toward physicians (67 physicians versus 30 nurses) because the initial general observations indicated that they used the system more frequently for more activities: physicians frequently accessed history and test results, ordered tests, and created discharge summaries, whereas nurses predominantly used the system for triage, although some performed other activities. Data were collected between October 2009 and February 2011, which ensured prolonged engagement in the Volume , .  : February 

study sites until saturation, in which no new information was emerging. Before the implementation of the integrated ED information system, there was a stand-alone ED electronic information system that was not interfaced with other clinical information systems in use at the study sites. This previous system included triage, tracking, and reporting capabilities. At this time, ordering laboratory tests and viewing results were possible from the EDs through a separate log-in to the hospitals’ clinical information system. Between May 2007 and December 2008, all study sites were upgraded to the new ED information system, which was interfaced with the hospitals’ existing clinical information system. The first phase of implementation comprised components of a commercially available Cerner HNA Millennium system. The ED information system module of Cerner HNA Millennium (FirstNet) monitors patient volumes and tracks patient movement and clinical activities within the EDs while providing access to other information in the hospitals’ clinical patient management system. Cerner HNA Millennium is a patient data repository composed of many components of an electronic medical record. It includes a provider order entry functionality that allows clinicians to place orders for various services such as pathology, medical imaging, diets, and transport requests. Clinical staff can view diagnostic test results, test Annals of Emergency Medicine 133

Effect of Information and Communication Technology on Emergency Department Quality of Care order status, and discharge summaries from previous ED and inpatient admissions and other clinical information such as ambulatory care notes across all 11 hospitals in the region. FirstNet also facilitates the creation of data relating to triage, mandatory reporting, initial patient management plans, and ED discharge summaries. These discharge summaries, created electronically by physicians for the majority of ED patients who are discharged home, contain information related to final diagnosis, test results, medications, and follow-up. At data collection, a manual paperbased hospital medical record was used concurrently with the electronic medical record to record patient observations, medications, and progress notes. In the second phase of implementation of the integrated ED information system, the information technology department of the study site planned to incorporate clinical documentation designed to capture inpatient and ambulatory medical history, treatments, and medication management. During the data collection period, some hospital sites had commenced implementation of the second phase of the electronic medical record for some ambulatory clinic attendances. Before implementation of the picture archiving and communication system, radiology orders were placed electronically and the status of the order could be viewed from the hospitals’ clinical information system. Radiographs were printed and available to ED staff (delivered to the ED by a radiographer). For computed tomography (CT), ultrasonography, and any other imaging, the ED relied on a verbal report from the radiology registrar. These images were not available for viewing by ED staff without their physically going to the radiology department. At data collection, diagnostic images were available electronically through the picture archiving and communication system, which was viewed in a Web browser opened within FirstNet. The implementation of FirstNet and the picture archiving and communication system occurred incrementally across sites, and at data collection all 4 EDs had the same system functionality available. Clinicians also have access to clinical knowledge databases that they source through the hospital’s Intranet or with hyperlinks from FirstNet. Data Collection Three researchers (J.C., J.L., and M.S.) conducted interviews and observations. The 5 focus groups were conducted by the same moderator (J.C.). Staff rosters with e-mail contact details were used to approach participants either face-to-face or by email to organize times to conduct interviews, focus groups, and observations. A standard set of lead questions was used for interviews and focus groups (Figure), with further probing to elicit how the system affected their work. Questions were developed according to a review of literature in the area and were pilot tested with the 2 physicians from the study team (R.P. and K.G.). Each interview took approximately 35 to 40 minutes and focus groups were allocated 1 hour. To orient the data collection team to the study sites, nonparticipant observations commenced with general observations of randomly approached clinical staff working in the EDs. More targeted, 134 Annals of Emergency Medicine

Callen et al

Questions asked to all interviewees and focus group attendees: How do you use the EDIS? Do you think EDIS has changed how you work? If yes, how? Do you think EDIS has changed the way you access information? If yes, how? Do you think EDIS has changed the way you document or record information? If yes, how? Do you think EDIS has changed how you communicate in the ED with others? If yes, how? Has EDIS changed the role of physicians/nurses in the ED? If yes, how? Has EDIS affected patient care? If yes, in what way and how? Has EDIS affected decisionmaking (in relation to decisionmaking about treatment/tests to order/consultations to call for)? If yes, how? What other effects has EDIS had that we have not mentioned?

Figure. Guiding questions for semistructured interviews and focus groups.

structured observations were then undertaken in 2-hour blocks on Monday to Friday between 8 AM and 5 PM. These targeted structured observations centered on the use of the ED information system around 4 processes that were the key work activities undertaken with the integrated ED information system: triage, accessing previous patient-specific information, ordering tests and viewing results, and creating discharge summaries. Observers were placed close to participants and sometimes questioned the participant to clarify what they were observing. The key purpose of the observations was to provide context for the interview and focus group data and background details on the use of the integrated ED information system in the ED setting. Field notes from observations were documented manually during the observational period and expanded after each session. Notes from the structured observations were transcribed, and all interviews and focus groups were audiorecorded and transcribed verbatim. Primary Data Analysis Transcriptions of the 3 data sets (interviews, focus groups, and observations) were organized with NVivo (version 8; QSR International Pty Ltd, Melbourne, Victoria, Australia) software. Analysis of each data set was initially undertaken separately and triangulated by using all members of the research team. Analysis was undertaken with an inductive thematic approach to derive categories that would explain the participants’ perceptions of the effect of an integrated ED information system on patient care delivery.51 Categories were initially generated by examining all data line by line. Further analysis and comparisons identified relationships between the codes to generate higher-level themes arising from the data. The process of analysis was iterative, with repeated analysis, discussion, reflection and reanalysis to derive a final set of themes. Divergent views during analysis were resolved with discussion until a consensus was reached. Member checking of results with participants was undertaken to clarify researchers’ interpretations. A computer-supported cooperative work framework was then applied to the final set of themes derived from the data. The framework focuses on how collaborative work can be supported by computer systems52,53 Volume , .  : February 

Callen et al


Table 3. Incentives—Faster and better-informed clinical decisionmaking: representative quotes from interviews and focus groups. Category Access to patient-specific clinical information

Access to clinical knowledge databases

Data “Of course your decisionmaking is influenced hugely by knowing what the patient’s background is . . . there’s just absolutely no comparison to the way I used to see patients before and now. To me, I can’t even imagine that I’d be able to practice medicine without this kind of information available.” (Site 1, interview with senior physician) “ . . . [Y]ou can narrow down your differential diagnosis what should be, could be [the problem] . . . and then you can do the tests in a more focused way. And that serves the patient better.” (Site 1, interview with senior physician) “I would definitely say that patient care has improved, greater information from their clinical notes and old results and stuff. It’s really helpful, especially when a patient can’t communicate with you. They come into triage and they can’t tell you anything about their history or why they’ve even come to the system.” (Site 3, interview with registered nurse) “There’s been a revolution with [picture archiving and communication system] and digital technology and we can barely remember how we managed in the days when you’d have to get bags of films.” (Site 4, interview with senior physician) “The Aged Care Assessment Team has linked into Cerner . . . all their case notes and other information comes in. Previously, we would have to ring over to them and they would have to get out the old notes and have a look at the old notes and then fax us a copy or whatever, whereas now it’s—from July 08 it’s been on the system so everything comes up . . . all the documentation that they’ve done is there for us so it does make it a lot better.” (Site 1, interview with nurse) “It’s become a habit that I check all the background information. So when I talk to [the patient], it cuts out a lot of unnecessary filling in and the patient realizes that you’ve done some homework on them . . . . They feel more comfortable. They fill in the blanks much more easily. They feel they’ve been listened to.” (Site 1, interview with senior physician) “It’s right there, it’s fast, it’s for everybody to access . . . . [T]here is no kerfuffling with papers and everything like that.” (Site 2, nurse focus group participant) “I think it’s fantastic not looking for textbooks, and I think it has meant that I’m much more likely these days . . . to go and actually look up something to get even more information on a topic.” (Site 3, interview with senior physician) “Probably we refer more to guidelines more frequently than we used to because they’re . . . easily accessible. We’ll look up for antibiotics, for guidelines, for anything.” (Site 3, interview with junior physician) “In the past MIMS was only available in the books, I think, and so it was quite difficult often to find a book to begin with, but nowadays every computer has access to that, so it’s made it a little bit faster.” (Site 1, junior physician focus group participant)

MIMS, MIMS Australia.

and is particularly suited to analysis of the use of information and communication technology in EDs, given the collaborative nature of the work. Computer-supported cooperative work theory emphasizes the importance of understanding the interplay between the technical system and the social and organizational context in which the system is embedded, which is supported by many researchers in health informatics.32,54,55 An underlying concept is that shared technologies, which are used as a collective tool, are used differently from an individually used technology on a personal computer, which is used as a personal tool.56 The integrated ED information system is an example of shared collaborative technology. Three core concepts of computer-supported cooperative work were used to focus the analysis: ● Incentives: Are there incentives or disincentives for physicians and nurses to optimally use the integrated ED information system to support patient care? ● Situation Awareness: How can the integrated ED information system support individuals’ awareness or understanding of the activities of others?57 ● Workflow: How does the integrated ED information system fit with clinicians’ work processes? Volume , .  : February 

RESULTS Findings are reported under the incentives, situation awareness, and workflow dimensions, with subcategories. Disincentives are reported under the heading “Workflow: Difficulty integrating the use of the technology with ED clinicians’ work practices.” Incentives: Faster and Better Informed Clinical Decisionmaking A key incentive for clinicians’ use of the integrated ED information system was that the system was perceived as having a positive influence on clinical decisionmaking compared with the previous stand-alone electronic ED system. Comments and observations represented by this concept were assigned the following categories (Table 3). Access to Patient-Specific Clinical Information. Improved access to patient-specific clinical information at the point of care was one of the most frequently cited and observed advantages of the integrated ED information system over the stand-alone ED information system. This information includes data related to the current encounter and previous hospital and outpatient encounters, such as current and previous triage and test results, Annals of Emergency Medicine 135

Effect of Information and Communication Technology on Emergency Department Quality of Care discharge summaries, and some ambulatory clinic and aged care assessment team notes from previous attendances to any of the 11 hospitals in the Area Health Services, including the 4 study EDs. Nurses commented that they were particularly pleased to be able to easily access the same information as physicians. Physicians indicated that timely knowledge of a patient’s medications, allergies, test results, and details of past presentations to any of the 11 hospitals allowed them to make more appropriate and better-informed clinical decisions, reaching accurate diagnoses at an earlier phase of the patient encounter. It was reported that with the paper-based record system, there could be up to a 4-hour lag between request and delivery of paper-based notes from the medical record department. Fast access to patients’ past clinical information was considered crucial, especially for those with extensive medical histories who are unable to communicate their clinical history. Access to radiology imaging was singled out as being especially important. The picture archiving and communication system was described as the highlight of the information and communication technology system for a number of reasons, including that images were available for view in real time and that the radiographs could not be misplaced. Urgent radiographs no longer required collection from the radiology department, and the picture archiving and communication system enabled physicians to personally manipulate the images in ways not possible on the radiograph. Access to Clinical Databases. A number of physicians mentioned the positive effect on patient care of fast and easy access to clinical knowledge databases at the point of care. They could now locate information stored electronically, such as hospital policies, therapeutic guidelines, and treatment protocols, which facilitated application of evidence-based medicine with more current information than that provided by textbooks. Some physicians reported that they were more likely to research topics of interest simply because the information was much easier to find. Nurses also reported using Internet search engines to enhance knowledge during triage. Situation Awareness: Enhanced Coordination and Communication Within and Outside the ED Awareness of what others are doing within and outside the ED helps ensure that physicians’ and nurses’ actions relating to the care of patients are coordinated. Situation awareness refers not only to awareness of the actions of others but also awareness of information elements and the significance of these in regard to the patients’ treatment and previous conditions. These results relating to awareness of information are included in the previous category: faster and better-informed decisionmaking. Comments and observations were assigned the following categories (Table 4). Improved Coordination Between Clinicians Within and Outside the ED. The integrated ED information system was perceived to improve communication between clinicians compared with the previous stand-alone ED information system. Participants highlighted that this was specifically in 136 Annals of Emergency Medicine

Callen et al

relation to the ability of emergency physicians and nurses to access the same patient information regardless of location within or outside the ED, thus linking all hospitals in the region. The ED information system particularly facilitated ward rounds and handovers. One senior physician reported the benefits of simultaneous access to information when he described doing a “virtual ward round” when on call in the ED. Whereas previously he relied on rostered physicians telephoning if there was an issue and providing him with vital information, under the new system he could view patient records from home and anticipate problems. If he was telephoned at home, he had access to the same information as the rostered physician in the ED and was able to provide much more specific, problemoriented advice. The integrated ED information system facilitated interpersonal communication, which contributed to a more efficiently run department. For example, the introduction of mandatory electronic signatures enabled other clinicians to quickly and easily ascertain who “owned” a patient, which was especially beneficial to nurses and reduced interruptions to physicians. Enhanced Speciality Consultations with Physicians Outside the ED. Synchronous access to the same patient data set by multiple physicians supported telephone consultations between an emergency physician and specialist physicians in different locations. This was thought to support the patient care process and improve patient flow, increasing the speed with which ED patients could be transferred out of the department to the appropriate specialty ward area. Improved Clinical Information Documentation. The often-noted problem of the illegibility of physicians’ handwriting was addressed by the integrated ED information system. Both physicians and nurses expressed the view that it was much easier to read electronic information: “everything is on the computer, so there won’t be any problem reading it.” One senior physician also remarked that although written notes are frequently “full of abbreviations, contractions, [and] nonsentences,” when forced onto a computer, these notes suddenly transformed into longer and more detailed prose. There was the perception that electronic records were “slightly more complete.” Some of the current treating physicians therefore perceived that they had more comprehensive information on which to base their decisions for patients they were currently treating in the ED. They also perceived that this information would assist them in treating patients presenting in the future. Workflow: Difficulty Integrating the Use of the Technology with ED Clinicians’ Work Practices Although participants perceived significant benefits from the introduction of the integrated ED information system, they also reported difficulty with assimilating the use of the technology with their work, particularly in relation to the significant data entry demands required and the time taken. Comments and observations were assigned to the following categories (Table 5). Increased Work and Task Complexity. A number of clinicians perceived, and field researchers observed, that electronic Volume , .  : February 

Callen et al


Table 4. Awareness—Enhanced coordination and communication within and outside the ED: representative quotes from interviews and focus groups. Category Improved coordination between clinicians within and outside the ED

Enhanced specialty consultations

Improved clinical information documentation

Data “No matter where you are, on whatever ward, or if you’re seeing someone else, you can still pull it up and talk about a patient that you may have seen, you know, on the other side of the hospital without having to walk back.” (Site 1, junior physician focus group participant) “Once the tests have been ordered, I can see that the lab has received that order. So that saves me having to call the lab. If there’s a delay with the order, I can actually see that the order’s in the lab. And then obviously I see the results when they come through on the screen as well.” (Site 2, interview with senior physician) “We can check hospital-hospital. We can check why this person presented at [site 2] 2 days ago. We can check what their triage was, we can check what their discharge summary was . . . we can see where the person has tracked themselves through the hospital, from hospital to hospital, how many times they present here, there, and everywhere.” (Site 1, nurse focus group participant) “We don’t have to hunt for the x-rays to take with the patient, which is great.” (Site 3, interview with registered nurse) “Radiology, they get the forms electronically. Previously, we had to fax them and a lot of times the faxes would go missing and all that sort of stuff.” (Site 2, interview with senior physician) “When you’re on call, you’re not [on site]; you can’t see the person . . . in the past you were very dependent [on] people identifying a need to ring you, and then ringing you, and then having to provide you with every piece of information. Whereas now . . . I can actually do a kind of virtual ward round. I can see the names, the ages, the genders, the presenting patients, the triage information, the case history note in 90% of people, the blood results, what investigations have been ordered . . . . I can see that all from home, and I can go through that information at home [and] then pick up the phone and have a much more specific problem-oriented conversation.” (Site 3, interview with senior physician) “I’ve made it a point of entering my information, my clinical notes as it were, into the system so that it’s available to me and anyone else for that patient for the future.” (Site 1, interview with senior physician) “[FirstNet] provides a very good overview of the department, and so it lists the patients who are here. And I—you know, once you’ve become familiar with the program you can get a very good impression as to how the department is traveling and how an individual is traveling by just looking at some of the overview screens.” (Site 2, interview with senior physician) “ . . . who recently looked after them in hospital, their telephone number and their GPs details and all of those things are immediately accessible. So yes, I think that’s great.” (Site 2, interview with senior physician) “ . . . [Y]ou’re not afraid to challenge [physicians] if you look at something and go ‘this isn’t right’; you can go ‘well I know this isn’t right because of A, B, C, and D,’ and you go to them and say, ‘Well I don’t agree with this, and this is my reasoning why,’ and just, being able to challenge their thinking and . . . .” “And perhaps too you’ve got information from the computer . . . .” “Technology does actually back you up with that, yeah.” (Site 1, conversation between nurse focus group participants about communication with physicians) “When handing over, if, say, you’re busy and you haven’t done a handover sheet for the next shift, you can sit at the computer and hand over . . . . [The system] tells you the plan, and then if you’re not sure about what’s happened since then, you can just find out.” (Site 3, interview with registered nurse) “They’re [EHRs] very useful when you’re trying to discuss patients with senior colleagues or when you’re trying to refer patients on because you can have your computer screen, you can have all the information that you need right in front of you so when you’re making your phone call, you can click easily in and out.” (Site 3, interview with junior physician) “So if I’m actually talking to someone like a consultant or so on the other line, if he’s got access to the same system . . . they can actually access the same information that we’ve got as well and look at it at the same time.” (Site 1, junior physician focus group participant) “ . . . [P]eople can now review the same images in several different locations. The radiology guys can see them at home. The neurologists, a lot of them can see them at home. So I think for image viewing, it’s improved communication.” (Site 2, interview with senior physician) “[It’s] [e]asier to pass the patients on to our colleagues because of the availability of the same information on a computer system; rather than having to talk about all the results, we can say, ‘These are important, but you can have a look at the rest of the results of that.’ It’s quite easy.” (Site 1, junior physician focus group participant) “I think we’re seeing slightly more complete documentation . . . which not only means that the person now responsible for the patient has better-quality information on which to base their decisions but also the next person has betterquality information because we all depend on the person before us in health, to some degree.” (Site 3, interview with senior physician) “I tend to find that people’s written notes are full of abbreviations, contractions, nonsentences; they’re just point form and whatever. And yet you put them on a typewriter and suddenly it’s prose, and I think that’s really interesting, and it also tends to get longer.” (Site 3, senior physician) “It’s easier to read on the computer than to read the physicians’ notes.” (Site 3, registered nurse) “ . . . [A point that a clinician (a junior physician) raised with me was that] when things have to be documented electronically, people are more inclined to document them more completely and coherently than if they are scribbling notes . . . .” (Site 3, observation)


Annals of Emergency Medicine 137


Callen et al

Table 4. Continued. Category

Data “ . . . [O]ne of [the] doctors (a junior physician) recently put a pro forma on FirstNet so when they are entering details of either adult or pediatric history or mental health, it gives the user sections to complete, so he stated during an observation (by the researcher) that it probably meant that ‘you’re more thorough in that you are actually writing more than you would normally’ . . . so he stated that because of that, his documentation was a bit more thorough than it used to be.” (Site 3, observation)

GP, general practitioner; EHR, electronic health record. *This text is the recording of a comment by a junior physician documented by the researcher during an observation.

processes entailed more steps in the completion of previously simpler tasks and generally created more work. Participants reported an overall increase in the amount of data entry required by the system for documentation and task completion. Clinicians mentioned the relative complexity of test ordering in comparison with the manual process in terms of the numerous, and sometimes perceived to be inappropriate, mandatory questions that required a response before requests could be transmitted. There was a perception by some senior physicians that the data entry role related to ordering tests had shifted from laboratory technicians to physicians on the ED floor. The inability to draw diagrams and symbols when describing clinical profiles saw figures replaced by lengthy descriptions of patient conditions and planned interventions, resulting in longer documentation of initial care plan notes. Furthermore, physicians believed they needed to click through a multitude of screens to accomplish simple tasks such as ordering radiographs or laboratory tests. Duplicate Tasks. Participants reported that documentation duplications occurred. None of the EDs had transitioned to a fully paperless system, which meant that physicians and nurses were writing paper-based notes in addition to electronic documentation. Physicians reported electronically documenting a comment or request to a nurse but also following this up with a verbal request because the order needed to be performed urgently. Rather than reducing steps or replacing one process with another, the transitional hybrid information system with electronic and paper-based components introduced extra steps into aspects of ED clinicians’ work. Another example of duplicate work occurred with the use of the electronic status board. Enlarged replicas of the electronic status board were prominently displayed in key areas throughout the EDs and available on all desktop computers. The electronic status board was viewed and updated by physicians and nurses just as the previous manual whiteboard was, with the key advantage of one’s being able to do this from multiple locations. However, we observed that nurses printed out the “status board screen” to accompany them on ward rounds. During rounds, they manually documented information on these printed sheets, which was later transcribed into the electronic status board, resulting in work duplication. Computer Issues. Physicians and nurses reported difficulties using the integrated ED information system, particularly related to problems with system usability, hardware, and individuals’ 138 Annals of Emergency Medicine

computer skills and knowledge. There were reports that the computer system was complicated to use and was not user friendly or intuitive. Hardware issues were also mentioned, with comments about too few printers resulting in time wasted. Even though pathology test orders were completed electronically, the test request form was subsequently printed to accompany the specimen. Screen size was also mentioned, with one participant commenting that he generally looked at complex CT scans on normal screens because there were only 2 high-definition screens in the ED and it wasted too much time to log off one computer, log on to another, find the patient, and call the radiograph up on the screen. There was acknowledgement by some clinicians that they lacked expertise in using computers generally. Poor typing speed was cited as an issue that led to the perception that more time was spent electronically documenting under the new system.

LIMITATIONS This study collected qualitative data to report on ED clinicians’ perceptions about the effect of an integrated ED information system on patient care, supported by observational data. Results report perceptions and these may not reflect actual performance. We did not ask participants specific questions about system design, architecture, or implementation processes, and such questions may have raised different issues. Qualitative studies are not intended to test inferences about causation or associations. The ED information system under study did not include electronic charting, which was planned for future studies but not implemented at data collection.

DISCUSSION The study revealed new perspectives on how the use of an integrated ED information system is perceived by physicians and nurses to influence patient care delivery both positively and negatively. Results showed perceptions of faster and betterinformed decisionmaking because of improved information access and enhanced coordination and communication. However, there were also perceived difficulties in integrating the use of the technology with work practices. A dominant view was that the integrated ED information system facilitates access to clinical information, both patient-specific and knowledge databases, both of which are essential for the delivery of highquality care. Because the ED information systems at the study sites were integrated with the hospitals’ clinical information Volume , .  : February 

Callen et al


Table 5. Workflow—Difficulty integrating the use of the technology with ED clinicians’ work practices: representative quotes from interviews and focus groups. Category Increased work and task complexity

Duplicate tasks

Data “I found that physicians may not be as efficient as they used to be; they see less patients because they are doing more . . . more data entry, a lot of data entry.” (Site 3, interview with senior physician) “ . . . [I]nstead of being clinicians, we’re spending, well, a lot of our time as . . . scribes or secretaries. It’s increased the amount of documentation. It’s increased the amount of time we spend on paperwork.” (Site 2, interview with senior physician) “Sometimes you feel like we’ve just become clerks. And so you can have . . . the director of the department . . . typing a discharge summary or . . . trying to order a set of tests on a computer.” (Site 2, interview with senior physician) “It may mean that doctors tend not to spend as much time talking to patients when it’s busy because you’re spending more time having to do all this data entry stuff.” (Site 1, interview with senior physician) “We use a lot of, you know, shorthand and diagrams and . . . which, to type it all out by hand is incredibly long winded, whereas if we can just draw it, it’s much quicker.” (Site 2, interview with senior physician) “[A senior physician] was just at the end of doing an electronic discharge summary. It took him 3 or 4 minutes to complete and he thought that took too long. He then had to pick up the summary from a printer which was a long way from where he was, and he went to a number of different printers to finally find the one that had printed off that discharge summary, which was up in the triage area.” (Site 2, observation) “Time difference between writing an order on a paper order form and ordering it electronically, it takes 3 or 4 times longer to do it electronically.” (Site 2, interview with senior physician) “ . . . [W]e all have a log-in. So what happens is that sometimes [when] I order things, it’s some other person’s [log-in] . . . so what we have to do is log out. And again have to start all over again.” (Site 1, interview with senior physician) “ . . . [W]ith FirstNet you’ve got to go in, find the patient, go into the form browser, select ‘triage,’ select this, and then you get there and [when] you find the print thing, you’ve got to bloody right click something and make some other option come up . . . .” “ . . . and then find the printer.” (Site 1, conversation between nurses in focus group) “To order an x-ray . . . how many buttons do you have to press to get a chest x-ray?” (Site 1, nurse focus group participant) “It’s [time spent in front of the computer] not all data input. It, most of the time you’re waiting for it to go between its . . . screens because there’s so many little bits that it links between. Or waiting to log out and log in again.” (Site 1, interview with senior physician) “When you’re looking for a diagnosis, it’s a very complicated, drawn-out process to get the exact diagnosis . . . . I don’t have 5 minutes to be looking for a diagnosis.” (Site 2, interview with senior physician) “ . . . [D]octors should be thinking about what the patients have and talking to the patients and looking after the patients, but nowadays if you go out and look at the ward there, you’ll see almost all the doctors are at the computers and hardly any of them are with patients.” (Site 3, interview with senior physician) “In the recent past, so the last 3 of 4 years particularly, where we’ve switched from paper ordering and paper records to a lot more electronic ordering, the main shift that I’ve perceived, there’s 2 things. One is that roles that were previously nonmedical, . . . often clerical or data entry clerks, are now being done by doctors, particularly and especially senior doctors. And that’s a big negative. That particularly is something that really slows us down because it really has just shifted the data entry role from [being] done in the lab by the technicians; it’s now down on the floor by the clinicians.” (Site 2, interview with senior physician) “[F]or the x-rays, you now have to tick for every x-ray you order whether or not the patients got an IV line. And why a doctor has to tell anyone that, I have no idea. It seems to be so the orderlies know whether to bring an IV pole or not. Well, I would have thought they could arrive at the patient’s bed and have a look rather than make me, as a senior specialist, have to tick a box on a computer screen so the orderly isn’t inconvenienced. Just doesn’t seem to be a terribly useful use of resources. So I think often the people who set up these systems [think] that’s a great idea but don’t actually think about who’s going to be entering that information.” (Site 2, interview with senior physician) “[W]e all have a password. So what happens is that sometimes I order things [and] it’s some other person’s log-in. So what we have to do is . . . log out. And again have to start all over again . . . . And I thought it’s only happening to me, but there are guys who [have been] working here more than 5 years and they—it’s happening to them as well. So I think it’s a common mistake and it wastes your time.” (Site 1, interview with senior physician) “So you go through the whole process of writing the history and ordering the test and then it says that you can’t do that because you’re not the person who has to sign it. So then you have to go backwards and start all over . . . again.” (Site 2, interview with senior physician) “Once the patient is in a bed, [a senior physician] commences a physical examination and asks questions. He then selects tests from the common tests list, types in a brief history/justification for the tests after selecting [the] first test, and proceeds to select more; signs off. He looks up a past discharge summary for the patient and decides to order another test. The doctor is seen typing the patient’s clinical history into the pop-up window again.” (Site 3, observation) “[W]e always double check verbally . . . . [W]e communicate verbally as well . . . . [W]e enter the things we need to do . . . but always we go and tell the nurse verbally as well . . . . [We] don’t totally depend on the system for that . . . . [I]n a way it’s a duplication.” (Site 3, interview with a junior physician) “After reviewing an x-ray film for a patient on a random computer (logged in under another account), [a junior doctor] picks up a new patient from suspension file in the office (which hold printed triage sheets) and begins looking up past information on the patient. He then logs out and logs in as self and resumes perusing old notes. The doctor is seen logging out from an existing Cerner screen and relogging in to computers often after leaving and returning to the doctor’s office [where computers are congregated].” (Site 3, observation)




Callen et al

Table 5. Continued. Category Computer issues (includes system usability, hardware, and individuals’ computer skills)

Data “ . . . [T]he discharge summary process is quite a complicated [one] . . . . [I]t’s not user friendly . . . . [V]isually also, it’s quite unappealing.” (Site 1, interview with senior physician) “I think utilization of IT for monitoring patient flow, accessing patient information, test ordering, and reviewing results . . . has the potential to make our work environment far easier and facilitate our work. It’s just unfortunate the systems we’re currently working with are unwieldy, poorly designed, and not intuitive.” (Site 2, interview with senior physician) “ . . . [T]here’s a bit of a problem here because we don’t have enough printers and some printers are not working properly . . . . [I]t wastes time.” (Site 1, interview with senior physician) “All our older doctors and nurses have a marked reluctance to become at least basically computer literate . . . . [Y]ou can take a donkey to water but you can’t make him drink it . . . so that’s a big problem. We have got some Jurassic doctors and nurses who have got big problems with computer literacy.” (Site 1, interview with senior physician) “It’s a huge issue for older clinicians, you know, who weren’t brought up in the, you know, Gen X, Gen Y. But even a lot of the Gen Xs aren’t that great on computers. Most of the Gen Ys are fine. They’ll go in there and type like crazy . . . but the real issue is that the people who we want to be recording the clinical information are actually the senior doctors. My view of the IT system is if we can make it easy for the senior doctors to input information, we’ll actually end up with betterquality information because what I want to read is, I want to read what was in the consultant’s head.” (Site 3, interview with senior physician) “[F]or the young nurses that have come through where technology’s all they’ve ever known, I think they find that really easy. I think for the older nurses, they struggle very badly with it. One of the older nurses who is acting as my NUM at the moment . . . has very little computer skills, so when the whole of everything about life is now on the computer—all of the systems, the reporting systems, the payroll systems, personnel, everything is on the system— . . . she struggles with that and she really can’t do it. But the difficulty is that she can’t do another different-level job because of the technology inference that goes with that, and I think that that’s not such a good thing.” (Site 3, interview with a nurse)

IT, information technology; NUM, nursing unit manager.

systems, compared with the previous stand-alone system, patient-specific clinical information relating to attendances at any of the 11 hospitals in the geographic region, including the 4 study hospitals, was available. The immediate availability of past patient information is acknowledged as particularly important for ED clinicians, given the need for quick decisionmaking in a complex, high-throughput, interruption-driven environment.7,8,58-60 Information gaps have been shown to be present in almost one third of visits to one large teaching hospital ED in the United States and were also reported to be associated with prolonged length of stay in the ED (length of stay increased by 1.2 hours; 95% confidence interval 1.0 to 1.9 hours).61 The potential of using information and communication technology to improve the quality of care delivered in the ED is clearly recognized.62,63 Those studies that have explored clinicians’ perceptions of ED information systems specifically,12,13,25 although single-site studies with systems not integrated with the hospital’s clinical information system, have also reported the value of the technology in facilitating information access. Our study emphasizes the importance of promoting information access as a direct incentive to facilitate clinicians’ uptake of information and communication technology. Real-time simultaneous access to a legible common patient data set with history and treatment details was perceived to assist situation awareness for emergency physicians and nurses and others outside the team. This capability supports information sharing, which is essential for coordinated care.64 It was evident that the use of electronic status boards can contribute to situation awareness because they are able to be 140 Annals of Emergency Medicine

updated and viewed from multiple sites. A shift in use was evident from our results, highlighting important differences in use between manual and electronic systems.65-68 Two key areas in which perceived improved awareness was evident from our results were related to the role of nurses in the test management process and the changing nature of the specialist physician consultation process. Nurses’ easy access to information about ordering and reporting laboratory and radiology tests could support their increased involvement in the process. This has patient safety implications, with the potential of improved test result follow-up69 and reduced duplications in test orders. Proactive alerts, available to physicians and nurses about availability of new test results, have been reported by 98% of users as the most important feature of an electronic medical record implementation.70 These alerts could also improve communication about test result notification between physicians and nurses. The finding in our study that electronic records were perceived as slightly more complete is interesting; however, completeness is only one facet in the utility of clinical documentation. Documentation completeness may not equate with ease of extraction by treating clinicians, quality of clinical documentation, or future use by researchers.71,72 In contrast to the positive effect of the ED information system, there were perceptions of difficulties incorporating the use of the computer into clinicians’ work, particularly given the pressured interruption-driven nature of the ED. Some of these issues were related to problems with the systems’ perceived lack of usability, whereas others were associated with the users’ lack of computer skills. One of the most common concerns was related to the data entry demands of the system and concerns that this was taking time Volume , .  : February 

Callen et al


away from patient care. Two evaluation studies of the effect of computerized provider order entry on clinicians’ time were undertaken.73,74 One compared time spent writing orders on paper versus using computerized provider order entry, reporting a significant increase with the use of computerized provider order entry (2.1% before computerized provider order entry to 9.0% after; P⬍.001).73 The other was a time and motion study before and after computerized provider order entry was introduced, which reported that computerized provider order entry increased time spent on the computer by attending and resident physicians but not by emergency nurses.74 Both studies reported that the computerized provider order entry did not decrease time spent with patients, with the extra time required for computerized provider order entry taken from nonpatient care activities.73,74 The perception in our study that clinicians are equating time spent with quality of care delivered may or may not be a correct assumption, and further studies need to be undertaken to evaluate the effect of clinical information systems on clinicians’ time.75 Another concern of clinicians in our study relates to their perceptions of time-wasting duplications in recording clinical information, which is due in part to the use of hybrid information systems whereby paper-based records are used concurrently with electronic information systems, particularly for recording progress notes and nursing observations. Other studies have reported this finding, which reinforces the need to move to complete electronic information systems to eliminate documentation duplications and avoid inconsistencies in the quality of clinical information when parallel systems are used.15,28,76,77 Concern was also expressed by emergency physicians in relation to test ordering. They perceived a shift in data entry previously undertaken by laboratory technicians that was now part of their role when ordering tests with the computerized provider order entry system. Some physicians saw a disparity between who does the work in terms of data entry into the system and who receives the benefit. Shifts in responsibility and role are often understandably seen by some as negative consequences of technology.78,79 Our study reinforces the necessity to address such concerns rather than ignore them. Data entry is recognized as a key barrier to clinicians’ use of health information systems,80 and difficulties entering data into electronic information systems in EDs have been reported previously.81 Others have cited the importance of considering the clinical environment, with researchers linking the slow diffusion of information and communication technology in health to organizational and clinical work practice issues rather than technical or system problems.15,32,54,55,82 Unintended consequences have also been reported particularly in relation to the use of computerized provider order entry.27,28 These errors are related in part to entering and retrieving information27 and unfavorable workflow issues.28 Health care work is complex and nonlinear,83,84 and studies have shown that emergency physicians work with many interruptions, often managing multiple tasks and communications simultaneously.58,85-87 System developers need to take into account the context of users Volume , .  : February 

in clinical settings and their specific requirements to ensure that the potential of these technologies is realized. To be of value, information and communication technology needs to enable real-time clinician connectivity, regardless of location; enhanced functionalities such as decision support; usability suited to ED clinicians’ collaborative work; and ease of data input. ED clinicians may need to change their technological frame to accommodate the use of information and communication technology as a collective rather than a personal tool.56 Previously, the use of familiar stand-alone software such as word processing programs on fixed personal computers has not supported collaborative work. However, our study shows that ED clinicians are using the integrated ED information system as a shared tool to facilitate communication and coordination of activities. This could be further enhanced with the use of mobile technology to enable clinicians to input and access data more easily. However, there is limited evidence about the effect of mobility of devices on clinicians’ work practices.88-90 It has been claimed that an ED information system can best provide optimal opportunities for improvement if it does not operate as a stand-alone system.8 Findings from our study add further weight to this claim. Results showed that ED clinicians perceived as positive the ability to access past and current patient history and test results, including picture archiving and communication system images, using a common interface with 1 log-on. Although best-of-breed systems can be interfaced to other systems, every interface involves a loss of information and ongoing technical maintenance. Having multiple stand-alone systems also increases the amount of training clinicians require to function, rather than performing all tasks in the one system. The integrated ED information system allows them to rapidly move from one source document or image to another without changing programs and with consistency in the appearance and location of the information. In summary, to our knowledge this is the first multisite qualitative study describing physicians’ and nurses’ perceptions of the effect of an integrated ED information system on patient care. Realizing the value of clinical information systems in EDs is mired in problems around electronic clinical documentation: who enters data into the system, how does this occur, who receives the benefit, and who bears the cost? The advantages of improved information access, communication, and coordination should not be compromised by the demands of data entry. Focusing attention on developing innovative methods of data capture will facilitate the efficient use of technologies in the ED. This in turn has the potential to make emergency services safer and improve quality of care. Supervising editor: Robert L. Wears, MD, PhD Author affiliations: From UNSW Medicine, Centre for Health Systems and Safety Research (Callen, Li, Stewart, Georgiou, Westbrook), and UNSW Medicine, Centre for Clinical Governance Research (Braithwaite), Australian Institute of Health Innovation, University of New South Wales, Sydney, Annals of Emergency Medicine 141

Effect of Information and Communication Technology on Emergency Department Quality of Care New South Wales, Australia; the Concord Repatriation General Hospital, Sydney, New South Wales, Australia (Paoloni); and the Liverpool Hospital, Sydney, New South Wales, Australia (Gibson). Author contributions: JC, RP, KG, AG, JB, and JW designed the study. RP, KG, JB, and JW obtained research funding. JC, RP, AG, and JW supervised conduct of the study and data collection. JC, JL, and MS drafted the article, undertook recruitment of participating hospitals and clinicians, and managed the data, including quality control. JW chaired the oversight committee. All authors analyzed the data, contributed substantially to article revision, and approved the final version of the article. JC takes responsibility for the paper as a whole. Funding and support: By Annals policy, all authors are required to disclose any and all commercial, financial, and other relationships in any way related to the subject of this article as per ICMJE conflict of interest guidelines (see www.icmje. org). The authors have stated that no such relationships exist. This study is part of an Australian Research Council Linkage Grant (LP0989144)–funded project to investigate the use of information and communication technologies to support effective work practice innovation in the health sector. Publication dates: Received for publication January 16, 2012. Revisions received July 3, 2012, and August 22, 2012. Accepted for publication August 29, 2012. Available online October 18, 2012. Address for correspondence: Joanne Callen, MPH (Research), PhD, E-mail [email protected].

REFERENCES 1. Institute of Medicine. Hospital-Based Emergency Care: At the Breaking Point. Washington, DC: National Academies Press; 2006. 2. Bates DW, Gawande AA. Improving safety with information technology. N Engl J Med. 2003;348:2526-2534. 3. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: 2001. 4. Berg M. The search for synergy: interrelating medical work and patient care information systems. Method Inform Med. 2003;42: 337-344. 5. Laxmisan A, Hakimzada F, Sayan OR, et al. The multitasking clinician: decision-making and cognitive demand during and after team handoffs in emergency care. Int J Med Inform. 2007;76: 801-811. 6. Cordell WH, Overhage JM, Waeckerle JF. Strategies for improving information management in emergency medicine to meet clinical, research, and administrative needs. Ann Emerg Med. 1998;31: 172-178. 7. Feied CF, Handler JA, Smith MS, et al. Clinical information systems: instant ubiquitous clinical data for error reduction and improved clinical outcomes. Acad Emerg Med. 2004;11:11621169. 8. Taylor TB. Information management in the emergency department. Emerg Med Clin North Am. 2004;22:241-257. 9. US Government. American Recovery and Reinvestment Act of 2009. Available at: http://www.recovery.gov/About/Pages/ The_Act.aspx. Accessed October 30, 2011.

142 Annals of Emergency Medicine

Callen et al

10. Landman AB, Bernstein SL, Hsiao AL, et al. Emergency department information system adoption in the United States. Acad Emerg Med. 2010;17:536-544. 11. Wilson GA, McDonald CJ, McCabe GP Jr. The effect of immediate access to a computerized medical record on physician test ordering: a controlled clinical trial in the emergency room. Am J Public Health. 1982;72:698-702. 12. Reddy MC, Spence PR. Collaborative information seeking: a field study of a multidisciplinary patient care team. Inform Process Manag. 2008;44:242-255. 13. Aronsky D, Jones I, Lanaghan K, et al. Supporting patient care in the emergency department with a computerized whiteboard system. J Am Med Inform Assoc. 2008;15:184-194. 14. Banet G, Jeffe D, Williams J, et al. Effects of implementing computerized practitioner order entry and nursing documentation on nursing workflow in an emergency department. J Healthc Inf Manag. 2006;20:45-54. 15. Callen J, Westbrook J, Braithwaite J. The effect of physicians’ long-term use of CPOE on their test management work practices. J Am Med Inform Assoc. 2006;13:643-652. 16. Adam T, Aronsky D, Jones I, et al. Implementation of computerized provider order entry in the emergency department: impact on ordering patterns in patients with chest pain. In: AMIA Annual Symposium Proceedings 2005. Washington, DC; October 22-26, 2005:879. 17. Guss DA, Chan TC, Killeen JP. The impact of a pneumatic tube and computerized physician order management on laboratory turnaround time. Ann Emerg Med. 2008;51:181-185. 18. Fernando S, Georgiou A, Holdgate A, et al. Challenges associated with electronic ordering in the emergency department: a study of doctors’ experiences. Emerg Med Australas. 2009;21:373-378. 19. Sard BE, Walsh KE, Doros G, et al. Retrospective evaluation of a computerized physician order entry adaptation to prevent prescribing errors in a pediatric emergency department. Pediatrics. 2008;122:782-787. 20. Terrell KM, Perkins AJ, Dexter PR, et al. Computerized decision support to reduce potentially inappropriate prescribing to older emergency department patients: a randomized, controlled trial. J Am Geriatr Soc. 2009;57:1388-1394. 21. White FA. Emergency department digital radiology: moving from photos to pixels. Acad Emerg Med. 2004;11:1213-1222. 22. Carton M, Auvert B, Guerini H, et al. Assessment of radiological referral practice and effect of computer-based guidelines on radiological requests in two emergency departments. Clin Radiol. 2002;57:123-128. 23. Sack D. Increased productivity of a digital imaging system: one hospital’s experience. Radiol Manage. 2001;23:14-18. 24. Ayatollahi H, Bath PA, Goodacre S. Factors influencing the use of IT in the emergency department: a qualitative study. Health Informatics J. 2010;16:189-200. 25. Creswick N, Callen J, Li J, et al. What impact do emergency department information systems have on nurses’ access to information? a qualitative analysis of nurses’ use and perceptions of a fully integrated clinical information system. In: Hansen DP, Schaper LK, Rowlands D, eds. 18th Annual Health Informatics Conference, HIC. Melbourne, Australia: HISA Ltd; 2010:23-27. 26. Koppel R, Metlay J, Cohen A, et al. Role of computerized physician order entry systems in facilitating medication errors. JAMA. 2005;293:1197-1203. 27. Ash JS, Berg M, Coiera E, et al. Some unintended consequences of information technology in health care: the nature of patient care information system-related errors. J Am Med Inform Assoc. 2004;11:104-112.


Callen et al


28. Campbell EM, Sittig DF, Ash JS, et al. Types of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc. 2006;13:547-556. 29. Chaudhry B, Wang J, Wu S, et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med. 2006;144:742-752. 30. Bates DW. The effects of health information technology on inpatient care. Arch Intern Med. 2009;169:105-107. 31. Callen J, Braithwaite J, Westbrook J. Cultures in hospitals and their influence on attitudes to, and satisfaction with, the use of clinical information systems. Soc Sci Med. 2007;65:635-639. 32. Ash J. Organizational factors that influence information technology diffusion in academic health sciences centers. J Am Med Inform Assoc. 1997;4:102-111. 33. Aarts J, Ash J, Berg M. Extending the understanding of computerized physician order entry: implications for professional collaboration, workflow and quality of care. Int J Med Inform. 2007;76:S4-S13. 34. Westbrook J, Braithwaite J, Georgiou A, et al. Multi-method evaluation of information and communication technologies in health in the context of wicked problems and socio-technical theory. J Am Med Inform Assoc. 2007;14:746-755. 35. Hauswald M, Joslyn S. Qualitative approaches to the study of adverse events and near misses. Acad Emerg Med. 2008;15: 1312-1314. 36. Pope C, Mays N. Qualitative methods in health research. In: Pope C, Mays N, eds. Qualitative Research in Health Care. 3rd ed. Oxford, United Kingdom: Blackwell Publishing, BMJ Books; 2006: 1-11. 37. Xiao Y. Artifacts and collaborative work in healthcare: methodological, theoretical, and technological implications of the tangible. J Biomed Inform. 2005;38:26-33. 38. Pope C, Ziebland S, Mays N. Analysing qualitative data. In: Pope C, Mays N, eds. Qualitative Research in Health Care. 3rd ed. Oxford, United Kingdom: Blackwell Publishing, BMJ Books; 2006: 63-81. 39. Giacomini MK, Cook DJ. Users’ guides to the medical literature: XXIII. Qualitative research in health care, A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA. 2000;284:357-362. 40. Giacomini MK, Cook DJ. Users’ guides to the medical literature: XXIII. Qualitative research in health care, B. What are the results and how do they help me care for my patients? Evidence-Based Medicine Working Group. JAMA. 2000;284:478-482. 41. Mays N, Pope C. Quality in qualitative health research. In: Pope C, Mays N, eds. Qualitative Research in Health Care. 3rd ed. Oxford, United Kingdom: Blackwell Publishing, BMJ Books; 2006: 82-101. 42. Stoop AP, Berg M. Integrating quantitative and qualitative methods in patient care information system evaluation: guidance for the organizational decision maker. Method Inform Med. 2003; 42:458-462. 43. Ammenwerth E, Graber S, Herrmann G, et al. Evaluation of health information systems-problems and challenges. Int J Med Inform. 2003;71:125-135. 44. Kaplan B. Evaluating informatics applications—some alternative approaches: theory, social interactionism, and call for methodological pluralism. Int J Med Inform. 2001;64:39-56. 45. Greatbatch D, Murphy E, Dingwall R. Evaluating medical information systems: ethnomethodological and interactionist approaches. Health Serv Manage Res. 2001;14:181-191. 46. Westbrook JI, Braithwaite J, Gibson K, et al. Use of information and communication technologies to support effective work practice innovation in the health sector: a multi-site study. BMC Health Serv Res. 2009;9:201. Available at: http://www.


47. 48.

49.

50.

51. 52. 53.

54. 55.

56.

57.

58.

59.

60.

61.

62.

63.

64.

biomedcentral.com/1472-6963/9/201. Accessed September 20, 2012. Pope C, Ziebland S, Mays N. Qualitative research in health care. Analysing qualitative data. BMJ. 2000;320:114-116. Australasian College for Emergency Medicine. Submission to KPMG: the establishment of eHealth NSW. 2012:3. Available at: http://www.acem.org.au/media/ACEM_Submission_Establishment_ of_eHealth_NSW.pdf. Accessed August 20, 2012. Deloitte Australia. NSW Health independent review of Cerner FirstNet. 2011:55. Available at: http://www.ecinsw.com.au/ sites/default/files/Deloitte%20Report%20FirstNet.pdf. Accessed August 20, 2012. Robotham J. Patients put at risk by software. Available at: http:// www.smh.com.au/technology/technology-news/patients-put-atrisk-by-software-20110306-1bjn9.html. Sydney Morning Herald. Sydney, Fairfax Media; March 7, 2011. Accessed August 22, 2012. Glaser BG, Strauss AL. The Discovery of Grounded Theory: Strategies for Qualitative Research. Dallas, TX: Aldine; 1967. Grudin J. Computer supported cooperative work: history and focus. Computer. 1994;27:19-26. Fitzpatrick G, Ellingsen G. A review of 25 years of CSCW research in healthcare: contributions, challenges and future agendas. Computer Supported Cooperative Work. 2012:57. Available at: http://www.springerlink.com/content/100250/?Content⫹ Status⫽Accepted&MUD⫽MP. Accessed September 17, 2012. Kaplan B. Addressing organizational issues into the evaluation of medical systems. J Am Med Inform Assoc. 1997;4:94-101. Aarts J, Doorewaard H, Berg M, et al. Understanding implementation: the case of a computerized physician order entry system in a large Dutch university medical center. J Am Med Inform Assoc. 2004;11:207-216. Orlikowski W. Learning from notes: organizational issues in groupware implementation. In: CSCW ’92 Proceedings of the 1992 ACM Conference on Computer-Supported Cooperative Work. New York: ACM Press; 1992:362–369. Pratt W, Reddy MC, McDonald DW, et al. Incorporating ideas from computer-supported cooperative work. J Biomed Inform. 2004;37: 128-137. Coiera EW, Jayasuriya RA, Hardy J, et al. Communication loads on clinical staff in the emergency department. Med J Australia. 2002;176:415-418. Chisolm DJ, Purnell TS, Cohen DM, et al. Clinician perceptions of an electronic medical record during the first year of implementaton in emergency services. Pediatr Emerg Care. 2010; 26:107-110. Brixey JJ, Robinson DJ, Johnson CW, et al. Towards a hybrid method to categorize interruptions and activities in healthcare. Int J Med Inform. 2007;76:812-820. Stiell A, Forster AJ, Stiell IG, et al. Prevalence of information gaps in the emergency department and the effect on patient outcomes. CMAJ. 2003;169:1023-1028. Baumlin KM, Genes N, Landman A, et al. Electronic collaboration: using technology to solve old problems of quality care. Acad Emerg Med. 2010;17:1312-1321. Handel DA, Wears RL, Nathanson LA, et al. Using information technology to improve the quality and safety of emergency care. Acad Emerg Med. 2011;18:e45-51. Menke JA, Broner CW, Campbell DY, et al. Computerized clinical documentation system in the pediatric intensive care unit. BMC Med Inform Decis Mak. 2001;1:3. Available at: http://www.ncbi. nlm.nih.gov/pmc/articles/PMC57982/. Accessed September 20, 2012.


Effect of Information and Communication Technology on Emergency Department Quality of Care 65. Wears RL, Perry SJ. Status boards in accident and emergency departments: support for shared cognition. Theo Issues Ergonomics Sci. 2007;8:371-380. 66. Bisantz AM, Guarrera TK, Perry SJ, et al. Emergency department status boards: a case study in information systems transition. J Cogn Eng Decis Making. 2010;4:39-68. 67. Pennathur PR, Bisantz AM, Fairbanks AJ, et al. Assessing the impact of computerization on work practices: information technology in emergency departments. In: Proceedings of the Human Factors and Ergonomics Society 51st Annual Meeting; 2007:377-381. 68. Wears RL, Perry SJ, Shapiro M, et al. A comparison of manual and electronic status boards in the emergency department: what’s gained and what’s lost? In: Proceedings of the Human Factors and Ergonomics Society 47th Annual Meeting; 2003: 1415-1419. 69. Callen J, Georgiou A, Li J, et al. The safety implications of missed test results for hospitalised patients: a systematic review. BMJ Qual Saf. 2011;20:194-199. 70. Batley NJ, Osman JO, Kazzi AA, et al. Implementation of an emergency department computer system: design features that users value. J Emerg Med. 2010;41:693-700. 71. Rosenbloom ST, Denny JC, Xu H, et al. Data from clinical notes: a perspective on the tension between structure and flexible documentation. J Am Med Inform Assoc. 2011;18:181-186. 72. Kabir A, Hanson R, Mellis CM, et al. Is asthma documentation improved by computer-facilitated data entry? J Qual Clin Pract. 1998;18:187-193. 73. Bates DW, Shu K, Narasimhan D, et al. Comparing time spent writing orders on paper and physician computer order entry. Proceedings of the AMIA Symposium 2000. Los Angeles; November 4-8, 2000:965. 74. Yen K, Shane EL, Pawar SS, et al. Time motion study in a pediatric emergency department before and after computer physician order entry. Ann Emerg Med. 2009;53:462-468. 75. Westbrook JI, Creswick N, Duffield C, et al. Changes in nurses’ work associated with computerised information systems: opportunities for international comparative studies using the revised Work Observation Method By Activity Timing (WOMBAT). In: 11th International Congress on Nursing Informatics; June 2327, 2012: Montreal, Canada. 76. Stausberg J, Koch D, Ingenerf J, et al. Comparing paper-based with electronic patient records: lessons learned during a study on diagnosis and procedure codes. J Am Med Inform Assoc. 2003; 10:470-477.

Callen et al

77. Mikkelsen G, Aasly J. Concordance of information in parallel electronic and paper based patient records. Int J Med Inform. 2001;63:123-131. 78. Georgiou A, Westbrook J, Braithwaite J, et al. When requests become orders—a formative investigation into the impact of computerised physician order entry systems on a pathology service. Int J Med Inform. 2007;76:583-591. 79. Westbrook JI, Braithwaite J. Will information and communication technology disrupt the health system and deliver on its promise? MJA. 2010;193:399-400. 80. Walsh SH. The clinician’s perspective on electronic health records and how they can affect patient care. BMJ. 2004;328: 1184-1187. 81. Yamamoto LG, Khan AN. Challenges of electronic medical record implementation in the emergency department. Pediatr Emerg Care. 2006;22:184-191; quiz 192. 82. Massaro TA. Introducing physician order entry at a major academic medical center: impact on organizational culture and behavior. Acad Med. 1993;68:20-25. 83. Berg M. Patient care information systems and health care work: a sociotechnical approach. Int J Med Inform. 1999;55:87-101. 84. Berg M. Medical work and the computer-based patient record: a sociological perspective. Method Inform Med. 1998;37:294301. 85. Coiera E, Tombs V. Communication behaviours in a hospital setting: an observational study. BMJ. 1998;316:673-676. 86. Chisholm CD, Collison EK, Nelson DR, et al. Emergency department workplace interruptions: are emergency physicians “interrupt-driven” and “multitasking”? Acad Emerg Med. 2000;7: 1239-1243. 87. Westbrook JI, Coiera E, Dunsmuir WTM, et al. The impact of interruptions on clinical task completion. Qual Saf Health Care. 2010;19:284-289. 88. Baldwin LP, Low PH, Picton C, et al. The use of mobile devices for information sharing in a technology-supported model of care in A&E. Int J Electron Healthc. 2007;3:90-106. 89. Prgomet M, Georgiou A, Westbrook JI. The impact of mobile handheld technology on hospital physicians’ work practices and patient care: a systematic review. J Am Med Inform Assoc. 2009; 16:792-801. 90. Andersen P, Lindgaard AM, Prgomet M, et al. Mobile and fixed computer use by doctors and nurses on hospital wards: multimethod study on the relationships between clinician role, clinical task, and device choice. J Med Internet Res. 2009;11: e32.

Did you know? Podcasts are available for almost every article in Annals.

Visit http://www.annemergmed.com/content/podcast to find out more.

144 Annals of Emergency Medicine
