Correspondence RESEARCH LETTER Usability Testing and Acceptance of an Electronic Medication Inquiry System for CKD Patients To the Editor: Medication errors are a concern for patients with non–dialysisdependent chronic kidney disease (CKD) because medications commonly used in CKD often require dose adjustment or potentially can cause nephrotoxicity.1 Information technology offers tools with the potential to reduce medication errors. Although cellular telephones, mobile devices, and computers have become ubiquitous in the United States, it is unknown to what extent patients from the CKD population have access to them or are proficient in their use because they frequently are older and of lower socioeconomic status and health literacy.2-4 In this study, we evaluated the usability of a medication inquiry system prototype offering patients information about the safety of drugs in CKD, designed for implementation across multiple technology platforms: (1) mobile device with text (short message service [SMS]) capability, (2) personal digital assistant (PDA), and (3) web site. The SMS application was developed using a third-party SMS Gateway service (Twilio) and accessible with any standard cellular telephone. The PDA application was developed in XCode 4.0 using the iOS 4 platform (Apple Inc) and was downloaded on an iPod Touch. The web site application was made accessible on a personal computer with a 19-inch monitor, with Internet Explorer as the web browser. Prespecified medications entered on all platforms evoke 1 of 3 responses: “not safe in chronic kidney disease,” “caution, speak with your healthcare provider,” and “safe in chronic kidney disease.” The response is supplemented on the PDA and web platforms with traffic light signal; in the SMS platform, only the text response is displayed due to character limitations. Usability testing was conducted by a single interviewer; the 3 platforms were used in random order, and task completion was recorded by a second observer. Tasks were scored as completed (an independently completed task), a noncritical error (task was completed with minor difficulties and through a divergent pathway), or a critical error (the task was not completed successfully without assistance). To maximize comfort when evaluating participants’ proficiency with the SMS application, they were allowed to use their own cellular telephone or were provided a basic flip telephone (Verizon Wireless Samsung Gusto) if they preferred for testing. Participants were asked to complete 3 scenarios for each platform. The first and second scenarios presented incremental tasks to assess where errors may occur. In the last scenario, participants were asked to independently use the application to search for information on a specified medication and report the response from the search. Completion of the last assignment was treated as a single timed task, and if the participant was unable to complete the assignment independently or provide the expected
Figure 1. Completion times by platform type.
response for the medication, the task was scored as a critical error. A user satisfaction questionnaire was conducted at the end of testing on each platform. Twenty participants with CKD (defined as estimated glomerular filtration rate ⬍60 mL/min/1.73 m2) were studied. Most participants were non-Hispanic black (n ⫽ 14 [70%]), had at least some college education (n ⫽ 13 [65%]), and had used the internet in the past (n ⫽ 18 [90%]). All participants owned a cellular telephone, and most of them had sent or received an SMS text prior to testing (n ⫽ 17 [85%]). All 3 platforms had similar rates of critical errors, with the PDA having a slightly lower rate than the SMS or web platforms, but this was not statistically significant (Table 1). Figure 1 shows median completion times for the final search assignment. Median task completion times in the SMS, PDA, and web site groups were 81 (range, 18-187), 33 (range, 14-278), and 33.5 (range, 16-170) seconds, respectively. Overall acceptance of the platforms was high, with a median score of 7 on a Likert scale (strongly agree). Half the participants preferred the web site (50%), followed by the PDA (30%) and SMS (20%) platforms. Our findings demonstrate a wide range in proficiency of use of these platforms, with a detectable rate of critical errors in performance of assignment related to inquiries regarding medication guidance in CKD. There was some suggestion of a user preference for the web-based application, with the most cumbersome use in the SMS platform. Because medication errors are a significant problem for individuals with CKD, new strategies are needed to reduce adverse events in this population. Health information technology tools designed to promote patient selfcare should be easily adaptable to multiple technological platforms and anticipate a wide degree of familiarity with such devices. Further studies are needed to assess individuals’ acceptance of such tools in the management of kidney disease and the ability of such tools to improve patient safety in CKD. Clarissa J. Diamantidis, MD, MHS,1 Marni Zuckerman, MA1 Wanda Fink, MS, RN,1 Saurabh Aggarwal, PhD2
Table 1. Cumulative Task Completion Rates by Platform Type SMS (n ⴝ 380 tasks)
PDA (n ⴝ 440 tasks)
Web (n ⴝ 380 tasks)
Score
Median per Individual
Count per 100 Person-Task
Median per Individual
Count per 100 Person-Task
Median per Individual
Count per 100 Person-Task
Completed Noncritical error Critical error
18 0 0.5
90.5 2.1 7.4
22 0 0
93.2 1.6 5.2
18 0 1
90.2 2.4 7.4
Note: N ⫽ 20 participants. Am J Kidney Dis. 2012;xx(x):xxx
1
Correspondence Divya Prakash, MS,3 Jeffrey C. Fink, MD, MS1 University of Maryland School of Medicine, Baltimore, Maryland 2 Novel Health Strategies, LLC, Columbia, Maryland 3 St. Joseph Health System, Anaheim, California Corresponding author:
[email protected] 1
Acknowledgements Support: This study was supported by a mini-grant from the National Kidney Foundation of Maryland. Financial Disclosure: J.F. has received prior research funding from Amgen Inc and honoraria from Sandoz Inc and Amgen Inc.
2
References 1. Seyffart G. Drug Dosing in Renal Failure. Dordrecht/Boston/ London: Kluwer Academic Publishing; 2011. 2. Plantinga LC, Johansen KL, Schillinger D, Powe NR. Lower socioeconomic status and disability among US adults with chronic kidney disease, 1999-2008. Prev Chronic Dis. 2012;9:E12. 3. Becker BN. Focusing on health literacy might help us cross the quality chasm. Am J Kidney Dis. 2009;53:730-732. 4. Cavanaugh KL, Wingard RL, Hakim RM, et al. Low health literacy associates with increased mortality in ESRD. J Am Soc Nephrol. 2010;21(11):1979-1985. Received August 28, 2012. Accepted in revised form September 21, 2012. © 2012 by the National Kidney Foundation, Inc. http://dx.doi.org/10.1053/j.ajkd.2012.09.016
Am J Kidney Dis. 2012;xx(x):xxx
