Database Searching Functions Laboratory Computing Incorporated ...

UTILIZATION OF CLINICAL LABORATORY INFORMATION SYSTEM DATABASES FOR MEDICAL EDUCATION AND RESEARCH James C. Boyd, M. D.* Jack H. Ladenson, Ph.D. John W. Lewis, Ph.D.

Departments of Pathology and Medicine *University of Virginia Medical Center, Charlottesville, VA 22908 Washington University School of Medicine, St. Louis, MO 63110

Summary

Materials and Methods

The clinical laboratory information system (CLIS) represents a potential medical resource

whose value outside of its service role in laboratory data management has remained largely unexplored. We wrote several programs for performing on-line CLIS database searches and for archival storage and retrieval of data from CLIS databases. These applications showed that CLIS databases can be tapped to yield a potential wealth of information for medical education and research. Our experience also pointed out some problems: (1) CLIS database query languages were lacking. This necessitated local software development to perform on-line database searches. (2) In the absence of CLIS data archival systems for longterm data storage, we programmed our own and found it cumbersome to use. Future design efforts should be directed toward ease and flexibility of use. (3) Medical records coded discharge diagnoses retrieved from hospital discharge diagnosis archival files were frequently inaccurate as judged by patient chart review.

Introduction Increasing numbers of hospitals are purchasing turnkey Clinical Laboratory Information Systems (CLIS) to help manage their burgeoning data processing loads. A recent report by the Automation in Medical Sciences Review Committee of the National Institute of General Medical Sciences documents the general state-of-the-art of laboratory information science.1 Evaluation of existing CLIS's2 has revealed that, while all systems have their critics, most are able to fulfill their intended functions in handling the day-to-day activities of laboratory data processing.

Several programs were written to access the databases of the laboratory computer systems installed at Barnes Hospital, St. Louis, MO. These programs were of two major types: (1) database searching functions and (2) archival database storage functions. Database

Searching Functions

Unusual result report. Because of earlier successful experience with delta checking3 on a

Laboratory Computing Incorporated (LCI) CLIS,

a

similar off-line delta check program was desired when the next generation CLIS was installed at Barnes Hospital by Community Health Computing (CHC) in September, 1977. CHC supplied a rudimentary FORTRAN IV off-line delta check program which we enhanced to produce a report for daily review by the Laboratory Medicine directors and residents. This report included a listing sorted by laboratory and alphabetically by patient within laboratory of the following: test values from the current day's work which were outside of predetermined limits (extreme check); test values which differed by more than a predefined percentage from the prior test values in the same patient (delta check); test results which remained unverified at the day's end; and test results which had either crediting comments (e.g., no charge) or invalid comments. Provision was made in the delta check to compare results of tests performed on different instruments (e.g., serum enzymes determined on continuous flow analyzers versus enzymes determined on centrifugal analyzers) by means of a user modifiable test equivalence table.

Antibiotic sensitivity collection program. The KIRBY-BAUER antibiotic sensitivity zone diameters for a large number of bacterial isolates were desired for a pattern recognition project testing the ability to identify bacteria solely on the basis of their antibiotic sensitivities. Such data were stored on the Barnes Hospital Microbiology CLIS (Interpretive Data Systems, Boston, MA) but there were no vendor programs to access it. Since this CLIS was programmed in Digital Equipment Corporation's MUMPS-ll language, it took only three hours to develop a program which found all of the antibiotic sensitivity zone diameters measured over a given time period and recorded

Little attention seems to have been given to the potential use of CLIS databases outside of

their day-to-day service role. For academic institutions, in particular, a computerized record of the clinical laboratory data for each hospitalized patient represents a vast and relatively untapped resource which could potentially be applied in research and education. Below, we report our experience in using CLIS databases for research and education.

583

CH1480-3/79/0o00-0583$00.75 (©) 1979 IEEE

them on magnetic tape. This program was used only for the above-mentioned research project.

cine directors and residents. The directors have used the report to keep alerted to possible analytical problems, specimen mixups, data entry errors or laboratory operational problems. By means of this report, an average of 20 specimens per day have been identified for review and repeat analysis to confirm the originally reported values. The provision to compare results obtained from different instruments has been useful for identifying and solving instrumental problems giving analytical errors.

Archival Database Storage Function Because there was no vendor supplied data archiving capability, an assembly language program was written for the LCI CLIS which stored on magnetic tape all laboratory results which had been ordered and completed on the same calendar day. This version of the program was run daily between April, 1974 and May, 1975. The program was subsequently modified so that instead of saving only the laboratory results which had been ordered and completed on the same calendar date, it saved all of the data for patients discharged from the hospital who were awaiting discharge from the computer system. Because some patients had pending laboratory test results for a time after their hospital discharge, it was possible for redundant copies of their test results to be saved on consecutive days. This latter version of the program operated from May, 1975 through August, 1977 when the LCI CLIS was replaced by the CHC CLIS. Approximately 60 reels of magnetic tape were recorded by these data archiving programs.

So far, twelve laboratory medicine residents and fellows have been exposed to the unusual result report. Its use in their training has helped to make them aware of common laboratory analytical problems, specimen handling artifacts, and analytical differences between instruments. Further, the capability to identify hospitalized patients with unusual laboratory findings from this report has allowed laboratory and medical trainees to follow up interesting findings while the patient is still hospitalized. Many index cases for departmental training conferences have been discovered by this mechanism. Prospective research using a specific abnormal laboratory result as a criterion for entering a patient into a study has been facilitated since the introduction of this program.

During the course of a research investigation of calcium metabolic disorders, we had an opportunity to search approximately 40 of these tapes for the data of 1694 selected patients. To do this, a PL/l program was written to search the tapes on an IBM 370/145 computer and to write the data of patients selected to another tape which could be read by the Microbiology laboratory's CLIS. Software was written in MUMPS-ll to read these reformatted tapes and to form a database management system for the data of the 1694 selected patients. The MUMPS-ll software was run in a background mode on the Microbiology CLIS.

Antibiotic Sensitivity Collection The MUMPS-ll program to collect antibiotic sensitivity profiles collated more than 4000 profiles over the three and one-half month period it was used. The analysis of these profiles by pattern recognition techniques showed that atypical antibiotic resistance patterns were common enough (about 17% of our isolates) to make impractical the routine identification of bacteridi isolates solely from their antibiotic sensitivity profiles.

Archival Database Storage

Discharge diagnoses which were coded by Medical Records personnel from the hospital discharge face sheet had been keypunched and saved on magnetic tape by Barnes Hospital. These tapes were computer searched for the patients in the calcium study and their diagnoses were entered into the MUMPS-ll database. Additional data obtained by patient chart review were entered into the MUMPS-ll database by interactive terminal. Data for statistical analysis were transferred by cassette tape and/or hardwired interface from the MUMPS-ll database to a Texas Instruments 980A minicomputer.

The assembly of an 800,000 character calcium database took approximately two years, used four different computers and three sources of data, required the development of over 100 separate software modules in four different programming languages, and ultimately entailed hand reviewing over 1600 patient charts to collate the information not available in computer compatible format. Searches of the archival tapes recorded by the LCI CLIS supplied only about 40% of the laboratory test results included in the database. This relatively low data recovery rate was the result of several factors: (1) approximately 30% of the patients in the database were not hospitalized at Barnes Hospital, thus their laboratory data was not captured by the Barnes Hospital CLIS, (2) of the remaining patients hospitalized at Barnes, approximately 1/3 were hospitalized prior to the implementation of data archiving by the CLIS, (3) some of the magnetic tapes containing the archival data had deteriorated over a two-

Results

Unusual Result Report Since its introduction in January, 1978, the daily unusual result report has been a highly successful tool for the Laboratory Medi-

584

year period of suboptimal storage conditions and contained multiple data blocks which were unreadable.

Discussion Modern CLIS's have many excellent features and can be adapted to meet the data processing needs of most clinical laboratories2. While hundreds of these systems are installed and operational in laboratories across the country, little has been reported regarding their value outside of a service role in routine laboratory data management. We have demonstrated by our present study that CLIS databases can be tapped to yield a potential wealth of information for medical education and research.

The CLIS databases contained no information about patient diagnoses. Our original hope had been to retrieve the coded diagnoses from the hospital's discharge diagnosis archival tapes. Unfortunately, the hospital had changed dia gnosis coding systems from SNOD05 to H-ICDA IIt in the middle of our study. We attempted to devise a code translator program to translate SNODO codes to H-ICDA II codes. Table I shows the accuracy (judged by a chart review of 100 random patients) of the resulting diagnosis codes after their extraction from computer tape and translation as necessary. Sixty-nine percent of the patients' computer-derived diagnoses were erroneous either having extra codes and/or incomplete coding or totally incorrect codes. The computerderived diagnoses were correct for only 21% of the patients and for 10% of the patients no diagnoses were found in the discharge diagnosis archival tapes.

Important goals in medical education from the standpoint of the clinical laboratory are exposure of trainees to current analytical techniques and interpretation of the data generated by these techniques; a further aim is to gain an understanding of the artifacts which can be introduced into clinical laboratory data by problems in analytical techniques, specimen collection/ handling and patient preparation. The Unusual Result Report we developed at Barnes provided a unique opportunity for Laboratory Medicine trainees to be exposed on a daily basis to those laboratory results most likely to reveal problems in analytical technique, specimen collection/handling and patient preparation; moreover, it broadened their ability to interpret laboratory data by identifying interesting laboratory values which could be investigated while the patients were still hospitalized. The trainees felt they had a more active role in the routine operation of the laboratory by their involvement in detecting and correcting erroneous laboratory results using the Unusual Result Report.

Table 1 Accuracy of discharge diagnoses extracted from archival computer tapes as judged by chart review of 100 random patients. No. of Diagnosis coding category Patients 1. 2. 3. 4. 5.

Correct Extra computer diagnosis codes Incomplete computer diagnosis coding Completely incorrect computer diagnosis codes Computer diagnosis codes not located

21 29* 54*

Medical education might benefit in additional ways from the use of CLIS databases. One potential use will be the algorithmic selection of patients' data based on specified combinations of test values. In this manner, it should be possible for the student to define a data selection scheme and have the computer search for all patient data which fulfills the input criteria. The development of query languages for CLIS's should allow this sort of function to be performed more easily. Another potential area of application will be the use of computer generated data interpretational aids. Prototypical programs to do this have been developed by Hobbie and ReecelO and Bleichll and appear to have educational merit.

6

10

* 20 patients' computer diagnosis codes fit the criteria for both categories 2 and 3. Based on the above data, we chart reviewed every patient's record to obtain the correct discharge diagnoses and the results for laboratory tests which were not found by computer search of the CLIS archival tapes. Information obtained by chart review was inserted into the MUMPS-11 database via interactive terminal sessions. An estimated three man years of effort was expended in creating the final version of the calcium database.

Our experience in using data from CLIS databases for the study of research problems was mixed. The gathering of antibiotic sensitivity profiles over a three and one-half month period was easily accomplished because all of the desired data were in the on-line CLIS database and the program to access these data was written in a high level language.

Subsequent investigation carried out on the calcium database has been fruitful7, and suggests that computer pattern recognition techniques applied to the laboratory data of patients with calcium metabolic disorders will be helpful in delineating the best diagnostic tests and their optimal interpretation .

Our construction of a clinical database of

585

In summary, we have presented our experience using CLIS databases for medical education and research, a role quite apart from the everyday routine service application of the CLIS. Valuable information is available from the CLIS database, but custom software is usually required in order to access it. Improved data archiving methods for the CLIS should make it possible to undertake extensive retrospective studies into the use and value of laboratory tests without time-consuming chart review.

patients with disorders of calcium metabolism acquainted us with the problems of searching archival data storage tapes and ended up being a much more extensive project than had been anticipated. Our original hope had been to eliminate over one-half of patient chart reviews by searching the computerized archival tapes for pertinent laboratory test results and discharge diagnoses. This hope was not realized as we ultimately had to chart review every patient's chart. Our efforts to search the computerized tapes and merge the patient data into a single database consumed approximately one man year of programming effort, 60 hours of computer time, and yielded laboratory results for only 40% of our patients. While these figures indicate a large amount of effort expended for a small return, we demonstrated, nevertheless, that it is possible to search archival tapes and retrieve useful information for research.

References

The coded discharge diagnoses retrieved from tape archives were highly inaccurate (Table 1). This inaccuracy was the result of a number of factors: (1) the primary care physician in many cases did not list all of the pertinent discharge diagnoses on the hospital chart face sheet; (2) medical records technicians in some cases miscoded the discharge diagnoses written on the face sheet; (3) our SNODO to H-ICDA II code translator often gave redundant or inaccurate codes. We suspect that the solution to the first two sources of inaccurate discharge diagnosis coding will not be an easy one and based on our experience would caution all investigators against the blind acceptance of hospital discharge diagnosis codes.

Our experience notwithstanding, it should be possible with the development of well-planned data archiving methods to collect extensive retrospective clinical laboratory databases without patient chart reviews. We would suggest that: (1) archival data be stored on the universal medium of magnetic tape. Attention should be given to writing the data on the tape in a standard format so that it can be read by other computer systems; (2) an indexing scheme for the data would be useful to help speed up the search process when only data from a selected group of patients are desired; (3) it might prove useful to store the archival data on a different computer system which would allow more elaborate indexing and data retrieval mechanisms to be set up; (4) provision be made for improving the accuracy of medical records diagnosis codes. This might be accomplished by arranging for each attending physician to review his patients' diagnoses which have been coded by medical records technicians. Incorporation of accurate discharge diagnoses codes into the CLIS archival data storage scheme would be ideal, but perhaps unnecessary, if computer searchable files of discharge diagnosis codes were readily

available.

586

1.

Kinney, T. D. and Melville, R. S. (eds), Mechanization, Automation and Increased Effectiveness of the Clinical Laboratory. (Washington, D. C.: U. S. Dept. of Health, Education and Welfare, Public Health Service, National Institutes of Health, 1976, Publication No. (NIH) 77-145).

2.

Lloyd-Johnson, J., Achieving the Optimum Information System for the Laboratory Update, 1976, Northfield, Illinois, J. LloydJohnson Associates, 1976.

3.

Ladenson, J. H., Patients as their own controls: Use of the computer to identify "laboratory error". Clin. Chem. 21: 1648-1653, 1975.

4.

Boyd, J. C., Lewis, J. W., Marr, J. J., Harper, A. M. and Kowalski, B. R., Effect of atypical antibiotic resistance on microorganism identification by pattern recognition. J. Clin. Microbiol. 8: 689-694, 1978.

5.

Thompson, E. T. and Hayden, A. C. (eds), Standard Nomenclature of Diseases and Operations, 5th edition, New York, Published for the American Medical Association, McGraw-Hill Book Company, 1961.

6.

CPHA Nosology Group (eds), H-ICDA - Hospital Adaptation of ICDA, Volumes 1 and 2, 2nd edition, Ann Arbor, MI, Commission on Professional and Hospital Activities, 1973.

7.

Ladenson, J. H., Lewis, J. W. and Boyd, J. C., Failure of total calcium corrected for protein, albumin and pH to correctly assess free calcium status. J. Clin. Endocrinol. Metab. 46: 986-992, 1978.

8.

Ladenson, J. H., Lewis, J. W., McDonald, J. M., Slatopolsky, E. and Boyd, J. C., Relationship of free and total calcium in hypercalcemic conditions. J. Clin. Endocrinol. Metab. 48: 393-397, 1979.

9.

Ladenson, J. H., Lewis, J. W. and Boyd, J. C., Initial studies of test selection and pattern recognition in the differential diagnosis of hypercalcemia, in Benson, E. S. and Rubin, M.

(eds), Logic and Economics of Clinical Laboratory Use, Elsevier/North-Holland Biomedical Press, 1978, pp 187-195. 10.

Hobbie, R. K. and Reece, R. L., Computer interpretation of laboratory test results. Federation Proceed. 34: 2152-2157, 1975.

11.

Bleich, H. I., Computer evaluation of acidbase disorders. J. Clin. Invest. 48: 1689-1696, 1969. Acknowl edgment

We acknowledge the support of National Institutes of Health National Research Service Award Fellowship 5 F32 6M05009-02 (J.C.B.)

587