Evaluation of online documentation.

Evaluation of Online Documentation Colleen M. Prophet, M.A., R.N.', Mariann E. Krall2, Ginette K. Budreau, M.A., M.B.A., R.N.', Teresa D. Gibbs, M.S.N., R.N.', Kristy P. Walker, M.B.A.2, Joy M. Eyman, M.A.2, Michael J. Hafner?

Department of Nursing and Patient Care Services' Department of Hospital Information Systems2 The University of Iowa Hospitals and Clinics 200 Hawkins Drive Iowa City, Iowa 52242-1009 Telephone 319-356-2267, FAX 319-356-4545 ABSTRACT

services during more than 41,800 inpatient stays and 581,440 ambulatory care visits during fiscal year 1996-1997. Of the 7,000 UIHC staff members, the Department of Nursing and Patient Care Services includes 1,349 Registered Nurses, 48 Advanced Practice Nurses, 40 Licensed Practical Nurses, and nearly 400 patient care assistants. more than

The University of Iowa Hospitals and Clinics (UIHC) implemented an online documentation system for patient care orders in 1994-1996. Developed entirely in-house, the INFORMM NIS (Information Network for Online Retrieval & Medical Management Nursing Information System) features order-generated task lists, defaulted charting responses, computer-generated chart forms, and graphical data displays. To measure the impact of automation on user perceptions, and documentation compliance, completeness, time, and location, a team of nursing and information systems representatives captured data before and after implementation. Staff surveys show more positive user perceptions. Documentation results indicate increased compliance and completeness, and a decrease or no change in time. Online documentation occurs mainly at unit workstations.

At UIHC, INFORMM (Information Network for Online Retrieval & Medical Management) supports 2,856 functions in 176 applications on an IBM 9672 (R44 & R24) Parallel Transaction System (PTS) with a Multiple Virtual Storage/Enterprise Systems Architecture (MVS/ESA) operating system. The INFORMM network includes more than 3,000 cathode ray tubes (CRTs) and personal computers (PCs), and 300 printers. With an average response time of less than 0.2 seconds, INFORMM operates continuously, except for approximately 20 minutes weekly. Of the 2.1 million transactions daily, online documentation of patient care orders accounts for more than 240,000. Developed entirely in-house, this system features order-generated task lists, defaulted charting responses, computer-generated chart forms, and graphical data displays [14].

INTRODUCTION

Although the literature suggests that automation improves documentation, only a few field studies have been conducted to measure the benefits. Previous studies support that user attitudes can interfere, unintentionally or overtly, with system adoption [1]. Suggested effects of computerization include improved documentation accuracy, and increased, decreased, and unchanged completeness and compliance [2, 3, 4, 5, 6, 7]. In terms of charting time and time spent in nurse-patient contact, authors variously report an increase, decrease, and no change [8, 9, 10, 7, 11]. Point-of-care devices are frequently recommended; however, terminal usage may not support the investment in devices [5, 12, 13].

STUDY

Purpose and Definitions The study purpose is to determine the impact of automation on user perceptions, and documentation compliance, completeness, time, and location. User perceptions are staff attitudes, behaviors, and satisfaction. Compliance is the percentage of time an order is charted as scheduled. Completeness is the number of results charted for an order. Time includes the number of orders and narratives, and the time used to chart orders and narratives. Location is the placement and usage of CRTs on patient care units.

The University of Iowa Hospitals and Clinics (UIHC) is a 873-bed quaternary health care facility providing 1091-8280/98/$5.00 © 1998 AMIA, Inc.

255

Questions The specific study questions are: * Is there a difference in user perceptions before and after implementation? * Is there a difference in charting compliance before and after implementation? * Is there a difference in charting completeness before and after implementation? * How does automation affect charting time? * Where does online documentation occur?

attitude/behavior increased from 3.24 to 3.57 (p=.000). Respondents expressed satisfaction with both manual and online documentation. Only 1 of 6 satisfaction items ("the current charting information provides up-to-date information") achieved a significantly higher mean rating (p=.025) postimplementation. The mean rating for all 6 satisfaction items combined increased, although not significantly (3.38 to 3.42). There were no significant differences in time for shift report, after-shift charting, communicating with other staff, or locating information. However, there was a significant decrease (p=.001) in time for completing other paperwork (from 30 to 20 minutes per 8 hours) and a significant increase (p=.038) in time spent in patient rooms (from 2 to 3 hours per 8 hours).

USER PERCEPTIONS

The study team addressed user perceptions in order to examine user readiness and to measure user experience after implementation.

Methodology The literature review did not provide a single instrument to cover the four desired aspects of user perceptions: user characteristics, attitudes, behaviors, and satisfaction. Therefore, a survey tool was developed, incorporating 12 items from the Nurses' Attitudes Towards Computerization Questionnaire [15]. This 41-item hybrid tool uses a five-point Likert scale and includes the following types and numbers of items: skill level, computer and online charting experience (3), attitudes and behaviors (22), satisfaction with charting (6), time spent on other tasks or communicating (9), and time spent in patient rooms (1).

DOCUMENTATION COMPLIANCE/COMPLETENESS

Automation of documentation affects both charting compliance and completeness. Orders are either nurse- or physician-generated patient care to be performed by nursing staff. Staff chart orders manually by entering time/initials, numbers, check marks, or letter codes on a flowsheet. Staff chart orders online by accepting defaulted responses, entering numbers, or choosing from listed responses. Methodology Since the literature review did not yield instruments to capture the desired data, two tools were created for patient record audits. The compliance tool, adapted from a UIHC quality improvement tool, captures the frequency of order documentation. Additional variables are: time of day (day, evening, night), day type (admit day, discharge day, day of surgery, hospital day), order type (vital signs, intake/output, comfort, physiologic, psychosocial), and unit occupancy. Unit occupancy is determined by dividing the number of patients at midnight by the bed capacity. The completeness tool captures all results charted on certain orders for either medicalsurgical or psychiatric patients.

The tool was field-tested by volunteer users from a unit excluded from the study. Due to the limited UIHC sample (n=40), the validity and reliability of the instrument for a general population was not tested. However, testing confirmed item consistency and the likelihood that distracters drew responses as well as providing minor revisions to the survey. Users on 11 units completed the survey pre- (n=137) and 6 to 10 weeks post-implementation (n=20 1). Results The survey results were analyzed by independent sample t-tests, using Statistical Program for the Social Sciences (SPSS) for Windows. The pre- and post-implementation groups were not significantly different for user characteristics: age, years in nursing, education, work experience, and computer skills. However, as expected, the groups were significantly different (p=.000) in skill and experience with an online documentation system.

After training, 10 fourth-year nursing students practiced using both tools until reliability was achieved. Both tools were pilot-tested on 70 patient days from 4 units excluded from the study. Additionally, the team observed the raters for at least the first 10% of audits.

The mean ratings of 15 of the 22 attitude/behavior items significantly improved (p