Scoring System-Based Data Computerized ...

Measuring Resource Use in the ICU With Computerized Therapeutic Intervention Scoring System-Based Data Gilles Clermont, Derek C. Angus, Michael R. Pinsky, Judith R. Lave and Walter T. Linde-Zwirble Chest 1998;113;434-442 DOI 10.1378/chest.113.2.434 The online version of this article, along with updated information and services can be found online on the World Wide Web at: http://chestjournal.chestpubs.org/content/113/2/434

Chest is the official journal of the American College of Chest Physicians. It has been published monthly since 1935. Copyright1998by the American College of Chest Physicians, 3300 Dundee Road, Northbrook, IL 60062. All rights reserved. No part of this article or PDF may be reproduced or distributed without the prior written permission of the copyright holder. (http://chestjournal.chestpubs.org/site/misc/reprints.xhtml) ISSN:0012-3692

Downloaded from chestjournal.chestpubs.org by guest on July 21, 2011 1998 by the American College of Chest Physicians

Measuring Resource Use in the ICU With Computerized Therapeutic Intervention Scoring System-Based Data* Gilles Clermont, MD, CM, MSc; Derek C. Angus, MB, ChB, MPH, FCCP; Walter T. Linde-Zwirble, OHP; Judith R. Lave, PhD; and Michael R. Pinsky, MD, FCCP

Background and objective: In this era of health-care reform, there is increasing need to monitor and control health-care resource consumption. This requires the development of measurement tools that are practical, uniform, reproducible, and of sufficient detail to allow comparison among institutions, among select groups of patients, and among individual patients. We explored the feasibility of generating an index of resource use based on the Therapeutic Intervention Scoring System (TISS) from hospital electronic billing data. Such an index is potentially comparable across institutions, allows assessment of care at many levels, is well understood by clinicians, and captures many of the resources relevant to the ICU. Design: We developed an automated mapping of the hospital billing database into the different items of TISS and generated computerized active TISS scores on 1,372 ICU days. The computerized score was then validated by comparison to prospectively gathered active TISS scores by trained data collectors. Setting: Eight ICUs within a university teaching institution. Patients: We studied 1,229 general medical and surgical ICU patients. Interventions: None. Measurements and main results: Active TISS scores ranged from 0 to 31 points. The two scores were well correlated (R2=0.53) and highly calibrated (as assessed by regression of active TISS on mean computerized active TISS [R2=0.85]). The scores were identical on 756 days (55.6%) and differed by ^3 TISS points on an additional 387 (28.2%) days. Interreliability assessment suggested substantial agreement (kappa statistic=0.71). The discriminatory power of the com¬ puterized score to identify different levels of ICU resource use was excellent as assessed by area under the receiver operating characteristics curves at four threshold points (0.91, 0.87, 0.89, and 0.88). Performance ofthe computerized score was similar across medical, coronary, and surgical ICU patient groups. Conclusion: An automated algorithm can reproduce valid TISS scores from standard hospital billing data, allowing comparison of patients and groups of patients in order to better understand ICU

resource use.

(CHEST 1998; 113:434-42)

Key words: computers; costs and cost analysis; health resources; intensive care; TISS; severity of illness; software; workload. Abbreviations: actTISS=active TISS; APACHE=acute physiology7 and chronic health evaluation; compTISS=computerbased active TISS; NPV=negative predictive value; PPV=positive predictive value; ROC.receiver operating characteristics;

TISS=Therapeutic Intervention Scoring System

T n this era of health-care reform and cost containment, there is increasing need to both provide care efficiently and adequately document that effi¦*¦

*From the Health Delivery and Systems Evaluation Team (HeDSET), Department of Anesthesiology and Critical Care Medi¬ cine (Drs. Clermont, Angus, and Pinsky), Graduate School of Public Health (Dr. Lave), University of Pittsburgh, and Health Process Management (Mr. CTISS. 1995, compTISS.

Rights Reserved.

Linde-Zwirble), Doylestown, Pa. 1997, University of Pittsburgh, All

Manuscript received January 27, 1997; revision accepted July 15. Derek C. Angus MB, ChB, MPH, Room 606B Reprint requests: Scaife Hall, Critical Care Medicine, University of Pittsburgh, 200 Lothrop Street, Pittsburgh, PA 15213; email: [email protected].

upmc.edu 434

ciency. In practical terms, this requires that health¬ providers examine their practice patterns to

care

determine how to reduce resource use without com¬ promising clinical outcome. However, both resource use and clinical outcome are difficult and expensive to measure. This is particularly true in the realm of intensive care where up to 30% of hospital costs are

spent managing a heterogeneous patient population

with varied outcome and resource use.1 Though debate exists over how best to measure resource use, one measure that has been used and validated in many studies and has become widely accepted by clinicians as an accurate reflection of

Clinical Downloaded from chestjournal.chestpubs.org by guest on July 21, 2011 1998 by the American College of Chest Physicians

Investigations in Critical Care

ICU

is the

resource use

Therapeutic Intervention

Scoring System (TISS).24 TISS was originally de¬ signed as a measure of severity of illness,5 but its use

rapidly expanded include personnel manage¬ cost ment, assessment, health facilities planning, measurement of therapy level,3-4'68 and prediction of the need for future ICU care.9 TISS was modified to reflect changes in ICU practice since it was introduced in 1973. There are currently four versions of the score.TISS, active TISS, intermediate TISS, and TISS-28.69-11 The traditional TISS score consists of 76 separate proce¬ dures commonly used in the ICU.6 Each of these procedures is awarded between one and four points depending on its complexity. The score is typically calculated daily and can be used either as a measure of daily resource use or summed and presented as a measure of overall "work done" during an ICU admission. Zimmerman et al9 presented a shorter version of TISS, active TISS (actTISS), which includes the 32 TISS items believed to best characterize the active items of intensive care (Table 1). The purpose of actTISS, rather than to correlate with either total TISS or total ICU costs, was to focus on capturing the incremental resource availability (or use) pro¬ vided in an ICU as opposed to on a standard floor. This has the advantage of measuring aspects of care most clearly at the discretion of the clinical team and is now collected routinely at several centers in the has

to

United States in combination with the acute physi¬ ology and chronic health evaluation (APACHE) III

prognostic score.12

The third version, intermediate TISS, which has to general floor patients as well as applicability ICU patients, has been used in Europe but has not yet been adopted in the United States.10 More recently, TISS-28, a collection of 28 interventions routinely performed in the ICU, has been demon¬ strated in Europe to correlate well with the full version of TISS in a large heterogeneous ICU

population.11

The TISS score has the potential to be a useful tool for internal quality control of ICU process of care and resource utilization as well as for compari¬ son of resource use between hospitals and ICUs. However, the TISS data collection effort is expen¬ sive, labor intensive, and outside the normal efforts required by most hospitals. In contrast, virtually all hospitals in the United States maintain a record, by day and by patient, of all resources consumed in order to generate hospital bills. We explored the possibility of generating an act¬ TISS score from hospital billing data. This index of resource use would be independent of hospital extractable from already existing hospital charges, data files, and able to serve as a basis for comparisons across institutions. In our initial pilot project pre¬ sented herein, we compared computer-based act-

Table 1.The Active TISS Score* Active TISS Items

System Respiratory Cardiovascular

4 Points

3 Points

Controlled mechanical ventilation Naso/orotracheal intubation

Emergency bronchoscopy

Cardiac

arrest within 48 h Vasoactive drugs (^2) Intra-aortic balloon pump

Assisted mechanical ventilation Fresh tracheostomy 180 record extract, showing hospitals in the Unitedof States. A billing the layout and detail information, is included in Table 3. tronic

HBO and

Algorithm Design The compTISS scores were obtained by developing a map between actTISS items and corresponding charge items in the achieved by manually review¬ billing database. The mapping was ing the detailed billing records of 10 patients with long (> 10,000 line items) billing records. In addition, the billing records of 20 other patients were examined to determine the best match for rarer actTISS items (eg, balloon tamponade of bleeding varices). These 30 patients constituted the development set. For each actTISS item, the combination of transaction codes that appeared to best correspond to therapeutic interventions noted in the development set was retained inin the mapping engine (see examples of mapping procedures the Appendix). Validation and Statistical Analyses The validation set consisted of the entire cohort. Daily scores compared for exact-matching rates. We also examined

were

Table 2.Baseline Characteristics ofthe Source

Population

No.

1,372 (1,229) 59.2 ± 16.9 (62) (median) 621 Female (45.3) (%) 5.3 ± 7.6 (3) ICU length of stay, d, mean ± SD (median) ICU deaths (% of admissions) 101 (7.4) Hospital deaths (% of patients) 185 (15.1) 502 (36.6) Perioperative admissions (% of admissions) ICU admissions

Age, yr, mean

436

(patients)

± SD

Charge

Medical Admission Transaction Date Date Record No. 000188048 000188048 000188048

Study Population The study population consisted of 1,229 patients with 1,372 admissions to eight ICUs at the University of Pittsburgh Medical

Characteristic

Table 3.Abstract From a Detailed Billing File*

4/5/94 4/5/94 4/5/94

Code

Descriptor

21401002 ECG tracing 4/5/94 22000456 Serum amylase 4/5/94 22002702 Serum creatinine 4/6/94 data file contains approximately 137,000

*The complete billing different items for the 1,372 patient

days.

of the 32 near-matching rates. As can be seen from Table 1, 28 while the actTISS items carry a value of three or four points scores a value of two four points. Consequently, remaining carry within three points of each other represent equal numbers of TISS items in most cases. We therefore defined the nearmatching rate as all cases in which the compTISS score was within three points of the actTISS score. Correlation was assessed by calculating the Pearson productmoment coefficient.13 Calibration was assessed by regressing the set of actTISS scores on the corresponding set of mean comp¬ TISS scores.13 Interrater reliability was assessed by calculating the kappa statistic.14 The performance of the algorithm was assessed for the 11 highest frequency actTISS items (eg, mechan¬ ical ventilation and vasoactive drugs), accounting for 91% of all actTISS items scored prospectively in the cohort. Setting actTISS as the gold standard, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calcu¬ lated. In addition, to explore the clinical utility of the compTISS algorithm, we assessed its ability to discriminate across different

thresholds of intensity of care. Discrimination was evaluated by receiver operating characteristics (ROC).1516 A series of ROC curves were obtained at different thresholds (defined as actTISS scores of 2, 6, 10, and 15). The choice of thresholds was arbitrary. that a well-trained ICU Though there are reports suggesting nurse can provide a workload of 40 to 50 TISS points per day, to our knowledge, there are no previous reports on equivalent actTISS scores.6 To assess the performance of the

compTISS algorithm across

different patient cohorts, we repeated the assessment of correla¬ tion, matching, and discrimination separately for medical (includ¬ ing neurologic), surgical (including trauma, neurosurgical, and cardiothoracic), and coronary care patients. Data manipulation and algorithm construction were performed using software packages (Foxpro and Excel; Microsoft Corp; Redmond, Wash). Data analysis was performed using statistical software packages (Datadesk; Data Description Inc; Ithaca, NY; and Labrocl; University of Chicago; Chicago). All significance levels are expressed at the a=0.05 level.

Results

The actTISS scores ranged from 0 to 31 with a of 3.22±3.86 (median=3) while compTISS scores ranged from 0 to 21 with a mean of 3.32±3.79 (median=3). Ninety-five percent of the actTISS scores were