Connecting Health and Humans K. Saranto et al. (Eds.) IOS Press, 2009 © 2009 The authors and IOS Press. All rights reserved. doi:10.3233/978-1-60750-024-7-429
429
Development and Testing of an Observational Method for Detecting Medication Administration Errors Using Information Technology Johanna I. WESTBROOK and Amanda WOODS Health Informatics Research & Evaluation Unit, Faculty of Health Sciences, The University of Sydney, Sydney, Australia Abstract. One-third of medication errors that cause patient harm arise from medication administration errors (MAEs). Research regarding rates of MAEs is limited and has predominantly relied upon voluntary incident reports or observational studies. Traditionally, observational studies have used paper-based data collection. This is time intensive and creates distraction from actual observation, limits the amount of information recorded and requires transcription of data. Incorporating the use of handheld computers for data collection in MAE studies provides an opportunity to overcome some of these limitations. Our objective was to develop and test an observational method which incorporated the use of data collection software on a personal digital assistant (PDA, a handheld computer) for use in observational studies to monitor MAEs within hospitals. Keywords. Medication errors, nurses, observation, PDA, interruptions, computers
Introduction The volume of medication errors are of significant concern to health care systems internationally. Over one third of preventable medication errors which result in harm occur during drug administration.[1] These include errors in the preparation (eg wrong drug or dose prepared) and administration (eg drug given to wrong patient, wrong dose, etc) of drugs. One study of 36 US health care organisations found that 19% of medication doses had some form of error.[2] In Australia a small number of studies conducted in the late 1980s and early 1990s found medication administration errors (MAE) rates of between 5% and 18% of drugs administered.[3-6] The preparation and administration of intravenous (IV) medications have been found to be associated with very high error rates. Observational studies conducted in teaching and district hospitals in the US, UK, Germany and France have found error rates of between 49% and 94% of IVs administered.[7-10] An Australian study of IV fluid administrations in three surgical wards in one hospital found an MAE rate of 18%.[11] With the increasing complexity of hospital patients and the associated increase in the number and types of medications per patient, the potential for MAEs is likely to increase. Interventions to reduce MAEs are needed, and electronic medication management systems have been heralded as of potential value in reducing both prescribing and medication administration errors. Yet a significant challenge in evaluating the impact of these systems on error rates is the absence of good quality baseline data regarding current error rates and a lack of standardised methods for measuring these errors. Research regarding rates of MAEs is limited and has predominantly relied upon voluntary incident reports or observational studies. Traditionally, observational studies have used paper-based data collection. This is time intensive and creates distraction from actual observation and the amount of information which can be recorded is
430
J.I. Westbrook and A. Woods / Development and Testing of an Observational Method
limited. Technological advances allow us to develop software able to capture multilayers of information in an efficient manner. Our objective was to develop and test an observational method which incorporated the use of data collection software on a personal digital assistant (PDA, a handheld computer) for use in observational studies to monitor MAEs within hospitals. In this paper we present the method and tool and also report initial results regarding compliance with medication administration procedures at a Sydney teaching hospital. This research was the first stage of a project to evaluate the impact of an electronic medication management system on medication administration error rates. 1. Material and Methods 1.1. Development of the PDA Data Collection Tool Drawing upon previous observational studies[12-17] we identified key data elements required to calculate MAE rates which would allow comparison with reported rates. We included errors related to the preparation and administration stages. Further, we recorded number of interruptions during medication administration and multi-tasking (ie doing two things in parallel) due to their potential role in error production. The data collection program was developed using Microsoft Visual Studio C# and was implemented on the PDA running the Microsoft CE platform. Using ActiveSync enabled the use of SQL and the synchronization of multiple PDA databases with one central database on a PC. Through an iterative process the PDA device was tested in the field to examine usability issues, study participant acceptance and to calculate observers’ inter-rater reliability scores. 1.2. Observational Method We developed a process for incorporating the data collection tool using a direct observational approach and tested the approach during a study at a Sydney teaching hospital. A two stage method was developed. In stage one trained nurse researchers observed an individual nurse as she/he prepared and administered medications. Information was recorded using the PDA, about procedures undertaken (eg read medication label) and medications administered (eg type, dose and route). Figure 1 shows the list of the procedures which are checked during the observation and the details recorded regarding the drug administered. Drop down menus assist observers in the selection of the drug name and dose. Importantly, the PDA tool allows observers to have active screens relating to multiple patients and nurses. This enables real-time tracking of how medications are delivered. For example, drugs for two patients may be prepared by a nurse at the same time and then administered in close succession.
J.I. Westbrook and A. Woods / Development and Testing of an Observational Method
431
Figure 1. PDA Data collection tool and screen shots illustrating the drop down menus available to record the drug name and dose
In stage two the observational data are compared with the patient’s medication chart to determine whether the drug observed to be administered was the same as the drug ordered for the patient. At this stage any errors in the administration of drugs are identified. A further set of screens guide the reviewer through this error identification process. This stage includes classifying the type of error and the severity. Nurses on two wards were invited to participate at information sessions. Following consent the observer shadowed participants for one hour blocks, during the core administration times. Observers followed a serious error protocol if they witnessed an administration which was potentially dangerous to a patient. 2. Results 2.1. Acceptance and Reliability of the Observational Method Several weeks were devoted to gaining the trust of the nurses. The use of nurses as observers was also important. Inter-rater reliability tests were conducted. This involved two data collectors simultaneously but independently observing a nurse and recording data for 45 minutes. Results were then compared. Data collection did not commence until percentage agreement was ≥85%. Inter-rater reliability of 85% or higher was achieved for injectable and non-injectable drugs during the observational study. 2.2. Medication Administration Procedural Error Rates We observed 35 nurses who undertook 1377 medication administrations. Table 1 shows the compliance with five medication procedures.
432
J.I. Westbrook and A. Woods / Development and Testing of an Observational Method Table 1. Observed Percentage Compliance with Medication Administration Procedures Procedure observed
Percentage of all medication administrations (N=1377)
Recorded administration on patient’s medication chart
97.7% (1345)
Read medication label
97.5% (1342)
Used an aseptic technique
92.1% (1268)
Checked patient identification
26.4% (363)
Medications were temporarily stored
5.8% (80)
3. Discussion The use of the PDA data collection tool allowed more variables to be collected than is feasible using a paper data collection process. A particular strength was the capacity to record details about multiple patients at the same time. The observational method required close shadowing of nurses and it is possible that this lead to changes in the nurses’ behaviours. We attempted to guard against this by introducing several weeks of practice where nurses were observed prior to formal data collection. This assisted in nurses becoming use to the presence of the researchers. Further, the direction of any potential bias in behaviour change was known, namely that nurses were unlikely to become less careful due to the observations. Thus it is likely that any error rate produced from the study will be an underestimation of the ‘true’ rate. We hypothesised that one of the areas whether behaviours would be more likely change is that related to procedures such as checking a patient’s identification, as these are well recognised steps which are easily monitored. We found high rates of compliance with three of the five procedures observed, but a very low level of compliance with checking a patient’s identification prior to medication administration. This suggests that nurses did not significantly alter their behaviours in this respect. While there are several examples of the use of PDAs to collect information about health professionals’ patterns of work[18-20] we have found no examples of the use of handheld devices to monitor medication administration errors. Our experience demonstrates a further use of such technology. Using this observational approach we have embarked upon a large-scale study to measure medication administration error rates before and after the introduction of an electronic medication management system in two hospitals. While several studies[21] have examined the effectiveness of these systems in reducing prescribing errors, little attention has been paid to their effectiveness in reducing medication administration errors. Our approach incorporating the PDA data collection tool provides a useful way to conduct such studies. 4. Conclusion Information technology can be used effectively to overcome some of the limitations of traditional paper-based methods to measuring quality and safety indicators. Our observational approach provides a standardised method for the collection of data regarding medication administration errors which may be useful in both obtaining baseline data regarding current error rates as well as in evaluating the effectiveness of interventions designed to reduce medication administration errors in hospitals.
J.I. Westbrook and A. Woods / Development and Testing of an Observational Method
433
5. Acknowledgements The research was funded by an NHMRC Project and Program Grant. JW is supported by a NHMRC Fellowship. We thank F. Skjaeveland, M Williamson, F Ray and K Nguyen for their contributions.
References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20]
[21]
Bates, D., et al., Incidence of adverse drug events and potential adverse drug events: Implications for prevention. JAMA, 1995. 274(1): 29-34. Barker, K.N., E.A. Flynn, and G.A. Pepper, Observation method of detecting medication errors. Am J Health-System Pharm, 2002. 59(23): 2314-6. Thornton, P.D. and L.J. Koller, An assessment of medication errors in a seven-day issue individualised patient drug distribution system. Aust J Hosp Pharm, 1994. 24(5): 387-390. McNally, K., M.A. Page, and V. Sunderland, Ward stock and unit-supply drug distribution: influence on nursing time, pharmacy time and medication errors. Aust J Hosp Pharm, 1997. 27(1): 22-7. Rippe, M., et. al., A survey of medication errors in a community hospital. Aust J Hosp Pharm, 1988. 18:201-204 Stewart, R.,et. al., Establishing the need for ward pharmacy:a survey of drug administration and medication errors in a public teaching hospital. Aust J Hosp Pharm, 1991. 21: 378-383. O'Hare, M.., et al., Errors in administration of intravenous drugs. BMJ, 1995. 310(6993): 1536c-1537. Cousins, D., et al., Medication errors in intravenous drug preparation and administration: a multicentre audit in the UK, Germany and France. Qual Saf Health Care, 2005. 14: 190-195. Hartley, G. and S. Dhillon, An observational study of the prescribing and administration of intravenous drugs in a general hospital. Int J Pharm Pract, 1998. 6(1): 38-45. Wirtz, V., K. Taxis, and N.D. Barber, An observational study of intravenous medication errors in the United Kingdom and in Germany. Pharm World & Sc, 2003. 25(3): 104-11. Han, P., I. Coombes, and B. Green, Factors predictive of intravenous fluid administration errors in Australian surgical care wards. Qual Saf Health Care, 2005. 14: 179-184. Dean, B. and N. Barber, Validity and reliability of observational methods for studying medication administration errors. Am J Hlth Sys Pharm, 2001. 58(1): 54-9. Flynn, E.A., R.E. Pearson, and K.N. Barker, Observational study of accuracy in compounding I.V. admixtures at five hospitals. Am J Health-System Pharm, 1997. 54(8): 904-12. Taxis, K. and N. Barber, Causes of intravenous medication errors: an ethnographic study. Quality & Safety in Health Care, 2003. 12(5): 343-7. Taxis, K. and N. Barber, Ethnographic study of incidence and severity of intravenous drug errors. BMJ, 2003. 326(7391): 684. Taxis, K. and N. Barber, Causes of intravenous medication errors: observation of nurses in a German hospital. J Pub Health, 2004. 12(2): 132-138. Taxis, K. and N. Barber, Incidence and severity of intravenous drug errors in a German hospital. Europ J Clin Pharmacology, 2004. 59(11): 815-817. Asaro, P. Sychronized time-motion study in the ED using a handheld computer application. 11th World Congress on Medical Informatics (MedInfo). 2004. Amsterdam: IOS Press. 701-705. Westbrook, J., et al., All in a day's work: An observational work measurement study to quantify how and with whom doctors on hospitals wards spend their time. Med J Aust, 2008. 188(9): 506-509. Westbrook, J., et al. Methods for measuring the impact of health information technologies on clinicians' patterns of work and communication. in 12th World Congress on Medical Informatics (Medinfo). 2007. Amsterdam: IOS Press. 1083-1087. Ammenwerth, E., et al., The effect of electronic prescribing on medication errors and adverse drug events: A systematic review. J Am Med Inform Assoc, 2008. 15(5): 585-600.
Email address for correspondence:
[email protected]
