32nd Annual International Conference of the IEEE EMBS Buenos Aires, Argentina, August 31 - September 4, 2010
A Guideline-Based Decision Support System for Generating Referral Recommendations from Routinely Recorded Home Telehealth Measurement Data Mas S. Mohktar, Jim Basilakis, Stephen J. Redmond, Member, IEEE, Nigel H. Lovell, Senior Member, IEEE
Abstract—The objectives of this paper are to present a guideline-based decision support system (GBDSS) design for supporting patient telehealth management of chronic disease and to test its performance in correctly making referral recommendations using routinely recorded measurement data from home telehealth recordings. The GBDSS has been developed to manage lung disease patients in a home telehealth environment. The system operates by checking the availability of home telehealth measurement data on a daily basis, interprets these data using a rule-based decision tree classification, and ultimately generates referral recommendations based on these measured data. The system has demonstrated discriminative power when applied in the analysis of retrospective telehealth data, as a surrogate for realtime referral generation. To this end a telehealth dataset comprising 16 chronic obstructive pulmonary disease (COPD) patients monitored over a 12 month period was used. It was shown that GBDSS referral recommendations could help reduce the number of cases that required a carer’s urgent attention by 72.1%, with 81.9% accuracy, 80.8% specificity and 90.4% sensitivity.
I. INTRODUCTION
T
elehealth is defined as the support of health care by using electronic information and communication technologies when the user and provider are in a separate location [1]. A home telehealth system is one that provides remote health care to patients in their home environment. The elements involved in home telehealth include patient users interacting with measurement devices, generating clinical information that is eventually stored onto a remote database which is then available to users for review. A home telehealth measurement device provides users with the facility to measure and monitor their vital signs on a regular basis, non-invasively. Examples of measurements that can be acquired include weight, lung function, temperature, heart rate, blood pressure, and oxygen saturation. Users also have access to a range of lifestyle and health questionnaires which can be used to measure general wellness and fitness. The
M. S. Mohktar, is with the Graduate School of Biomedical Engineering, University of New South Wales, Sydney, NSW 2052, Australia, on leave from the University of Malaya, Malaysia. J. Basilakis and N. H. Lovell are with the Graduate School of Biomedical Engineering, University of New South Wales, Sydney, Australia (e-mail:
[email protected]). S. J. Redmond is with the School of Electrical Engineering and Telecommunications, University of New South Wales, Sydney, Australia.
978-1-4244-4124-2/10/$25.00 ©2010 IEEE
home telehealth system in general has a central database which acts as the main repository of data [2-3]. Home telehealth users typically include patients and their carers. The patients are often elderly people and/or people with chronic diseases, and the carer could be the patient’s relative, nurses or the patient’s primary care physician (PCP). A single carer may be responsible for monitoring more than one patient and one patient may have more than one carer. In recent years, the demand for home telehealth solutions has increased in tandem with an expanding elderly population and a rising proportion of chronically ill patients who prefer to manage their condition from home [4-5]. With an increasing number of patients requiring review, the time required to interpret patient data increases. This increases the carer’s workload and in turn decreases responsiveness to an alert [5-6]. To assist the carer in the task of monitoring the subject’s telehealth measurement data, a customized decision support system (DSS), destined for integration within the home telehealth system, has been developed. A DSS is a computer-based system to assist in clinical decision-making, typically used by health practitioners. One intention for employing a DSS in a home telehealth system is to improve workload management by alerting the carer to those patients requiring immediate review and a decisive course of action [3]. Hence, the DSS enables a carer to prioritize their workload based on disease severity, rather than having to monitor all patient measurements. The capability of a DSS is enhanced when combined with evidence-based clinical practice guidelines (CPGs), which are an important tool for managing and assisting clinical decision making. CPGs contain directions, principles and recommendations on patient care decisions [7]. The integration of CPGs within a DSS is termed a guidelinebased DSS (GBDSS). In this paper we describe an example of a GBDSS. As the system is intended for use by chronic obstructive pulmonary disease (COPD) patients, it has been developed around established COPD guidelines [8-10]. A discussion on system performance in generating referral recommendations from routinely recorded home telehealth measurement data is provided.
6166
data and occasionally visited the paatient whenever the need arose, in both cases making some sh hort-form notes in a carer journal including points of interest describing each patient’s condition.
Fig. 1. The home telehealth unit represents the measurrement device located at the patient’s home. Telehealth data is stored in a dattabase. Arrows to and from the database and GBDSS show the flow of data. The carer and patient can assume two possible user roles in the DSS. S Steps 1, 2, 5, 6 are performed by the system. Step 3 is performed manuallly by the patient and Step 4 is automatically performed by the home teleheallth unit.
II. GUIDELINE-BASED DECISION SUPPO ORT SYSTEM The objective of the GBDSS described iin this study is to provide specific recommendations that aree relevant to the home telehealth user. These recommendatiions are based on the CPGs and the patient’s measurementt data; the latter recorded by the home telehealth unit. The G GBDSS has been designed to fit within an existing hhome telehealth framework, as shown in Fig. 1. The GBDSS S system starts by creating and sending a reminder notifyinng the patient to perform their measurements and resspond to their questionnaires that have been scheduled. T The data are then automatically stored in a database. Then, periodically, the system checks the data. If any data are abseent or incomplete (missing data) it re-sends a notification advvising the patient to complete their measurement tasks. Thiis system uses a rule-based approach to generate clinical rrecommendations based on the measurement data as an outpuut to the GBDSS. The generated recommendations from the rrules will then be sent to the patient and their carer. The GB BDSS framework could be applied to support the managemennt of a number of chronic diseases in home telehealth. How wever, this paper only discusses the application in managiing lung disease patients.
B. Reference standard Two types of indicators are emplo oyed to generate a health status standard, against which the GBDSS G is later compared. The primary indicator leveraged d was the medication questionnaire data entered by the patient. p Specifically, the medication questionnaire related to o the use of respiratory therapy (or more specifically puffeers/nebulisers, antibiotics and steroids). The medication questionnaire results indicating an increase in the use off respiratory medications are considered a predictor of worsen ning patient health status. It is then established whether these patients p were managed at home or had been referred to a clin nical specialist soon after this increase in their medications,, by checking the carer journal (c.f. Section III.A) entry forr that day for any record of hospital admission or an unscheduled visit by a medical specialist. Henceforth, one of two o classification labels is assigned to the patient’s health statu us for that day: 1) Home management: If an increase in respiratory medications was observed, but the patient p was not visited by a medical expert or admitted to hosp pital soon afterward. 2) Referral: If, again, an increease in medications was observed, but this time the patient was w referred to a medical expert or admitted to hospital soon afterward. a If there is no entry in the carer jo ournal on the date which the increase in medications occurred, the date is excluded from the analysis. C. Analysis process 1) Completeness of data The system starts by checking the level of completeness of r from one day back the data set within a specified date range; to one month prior to the patient being b observed as having unstable health status, according to the reference standard. If the data are complete, classification n is performed according to the decision tree in Fig. 2.
III. METHODS A. Data set To validate the accuracy of the G GBDSS referral recommendations a database containing rooutinely recorded home telehealth measurements was used.. The data were collected from 16 COPD patients, aged betw ween 54-92 years using a remote monitoring system called tthe TeleMedCare Health Monitor – TMC-HM (TeleMedcare Pty. Ltd. Sydney, Australia). The patient data were collecteed from February 2007 to January 2008. During the data ccollection period, patients were scheduled to perform their m measurements and answer questionnaires every day. T The data were automatically transmitted and stored in a daatabase located on a remote server. The carer (nurse) regulaarly reviewed the
6167
TABLE I ES REFERRAL CRITERIA THAT INDICATE Criteria Status Measurement M technique Forced expiratory Lung L function Decrease measurement m (spirometry) volume in 1 second (FEV1) Fever Increase in Thermometer T body temperature Decrease Saturation of Pulse P oximetry measurement m peripheral oxygen (SpO2) value Weight Decrease Weight W scale Breathlessness Severe Home H telehealth health questionnaires q General condition Deteriorate Sputum
Increased amount, Change color Decrement and increment were measured in percentage change from the average measurement values of the previo ous month.
TABLE II CONFUSION MATRIX Reference standard
47(TP)
Home Management 72(FP)
5(FN)
302(TN)
Referral GBDSS
Referral Home management
Total 52 TP = True positive, FP = False positive, FN = False negative, TN = True negative.
C = Number of criteria true, F = Decrease in FEV1 S = Decrease in SpO2 T = Increase in temperature Fig. 2. Algorithm classifying cases of unstable health status into referral and home treatment based on the guideline derived clinical criteria.
2) Decision tree classification The decision tree in Fig. 2 relies on counting a number of specified criteria which are calculated to be true. These criteria are extracted from published CPGs for deciding on the stability of COPD patient health [8-10]. Eight criteria were found to be suitable for home telehealth applications; that is, they can be measured using a home telehealth measurement device and questionnaires. The criteria are listed in Table I. The published literature does not provide information on the number of criteria that indicates a referral. However, patients with six respiratory symptoms will decrease their physical health quality, and as a consequence increase their COPD severity [11-12]. Thus, a threshold of six positive criteria, from Table I, is arbitrarily selected for recommending a referral. However, there are some cases in which less than six criteria may also require referral [13-15]. A second decision rule is implemented to account for this eventuality: if four or more positive criteria are observed from Table I; and one of the criteria is either a decrease in FEV1; or a decrease in SpO2; or fever, the case will be classified as a referral [8-10, 16-19]. The threshold of 3.5% for FEV1 was selected as the accuracy requirement for spirometry measurement devices [20-21]. The relative SpO2 threshold was set at 0.2% as it is the minimum bias value that could be recorded by a finger probe pulse oximeter under the condition of poor perfusion [22]. The threshold of 1.35% for body temperature was chosen in determining fever according to published sources [23].
Out of 16 patients, only 12 had recorded at least one day of complete data, resulting in complete data for a total of 1150 days, of which 452 (39.3%) had changes in health
119 307
374
status according to the criteria in Section III.B. The classification was only performed for a subset of 426 days, as 26 (5.8%) had missing carer journal notes. 52 (12.2%) days were noted as having a referral recommended by the carer and the other 374 (87.8%) days were recommended for at-home self-management. More than 75% of days had incomplete data, and four patients had incomplete data for their entire set of measurements. After applying the decision tree classification on the remaining data set of 426 days, the outcome of the analysis demonstrates the algorithm classified 90.4% of the true referral cases correctly, with a total classification accuracy of 81.9%. The system misclassified 72 cases as requiring referral when this was not the case. The system had misclassified 5 (9.6%) cases as requiring home treatment, when in fact they were referred for medical attention. The cases representing worsening health status were distributed unevenly among the 12 patients; with patient 4 recording the largest number of cases; and patients 10 and 12 only recording 2 unstable cases (Fig. 3). The case distribution was probably affected by the filtering process based on the criteria of completeness of the data [24]. However, deploying this system in a real-time setting, could improve the incentive for obtaining the complete data set required, by detecting missing data based on scheduled measurements and then proactively sending reminders to the patient to perform the measurement. The reminder will also increase compliance with the monitoring in general [25].
D. Performance metrics To evaluate the system performance, the classification system output is compared with the reference standard. Treating a correctly identified referral as a ‘true positive’, the system accuracy, sensitivity and specificity are calculated. IV. RESULTS AND DISCUSSION
Total
Fig. 3. Distribution of cases identified as having unstable health based upon the medication record calculated in percentage among 12 COPD patients (total cases = 426).
This analysis is limited to measurement data and questionnaire data only; medical history, demographic information and patient hospital records were not available at the time of analysis. This additional information is very important in providing more contextual information that
6168
would be relevant to identifying new exacerbations in COPD patients. As an example, the criteria to define weight loss is the patient having a decrease of 5% from his/her normal weight in the past month [26]. However, the database does not provide the normal weight. This problem applies to all criteria; hence, the system could only calculate the increment or decrement based on the average measurement value which is not a reflection of any agreed ‘normal’ value. Therefore in the future, to address this deficiency, an additional data processing step needs to be done to determine the specific degree of change in the data value that is indicative of the worsening health of a patient. In this analysis several assumptions are made. It is assumed that the measurement values were extracted from signals of suitable quality, and that the carer notes accurately represent the condition of the patient and the treatment and care they received. To improve the validation methodology, it would be necessary to improve the reference standard by seeking expert opinion at more regular intervals. The system should also be tested in a prospective COPD home telehealth trial to demonstrate how the system alleviates workload burdens. It would be important to reduce the false negative error rate. This is to avoid the situation that carries the most serious clinical risk by incorrectly assigning patients to home treatment when they require hospital or clinician referral. This might be achieved by using more advanced classification techniques to improve the initial algorithm.
[8] [9]
[10]
[11]
[12] [13] [14]
[15]
[16]
[17] [18]
V. CONCLUSION The GBDSS has shown the flexibility to be used with retrospective data as the initial design was meant to be deployed in a real time environment. The GBDSS has shown an acceptable level of performance when applied to retrospective telehealth data from COPD patients. The system referral recommendation could help reduce the number of cases that need a carer’s urgent attention to only 119 cases from 426 unstable cases, that is, a decrease of 72.1%.
[19]
[20]
[21]
REFERENCES [1] [2] [3]
[4] [5] [6] [7]
G. Banks and J. Togno, "Telehealth in Australia: Equitable healthcare for older people in rural and remote areas," in 5th National Rural Health Conference Adelaide, South Australia, Australia 1999. W. H. Collinge and K. Liu, "Information architecture of a telecare system," J Telemed Telecare, vol. 15, pp. 161-164, June 1, 2009 2009. N. H. Lovell, S. J. Redmond, J. Basilakis, M. S. Mohktar, J. A. Sukor, and B. G. Celler, "The Application of Decision Support Systems in Home Telecare," in Distributed Diagnosis and Home Healthcare, R. U. Acharya, et al., Eds., ed: American Scientific Publishers, 2008. A. Kinsella, "Home telecare in the United States," Journal of Telemedicine and Telecare, vol. 4, pp. 195-200, 1998. S. Koch, "Home telehealth-Current state and future trends," International Journal of Medical Informatics, vol. 75, pp. 565-576, 2006. M. A. Hebert and B. Korabek, "Stakeholder Readiness for Telehomecare: Implications for Implementation," Telemedicine Journal and e-Health, vol. 10, pp. 85-92, 2004. D. Isern and A. Moreno, "Computer-based execution of clinical guidelines: A review," International Journal of Medical Informatics, vol. 77, pp. 787-808, 2008.
[22]
[23]
[24] [25] [26]
6169
M. Barnett, Chronic Obstructive Pulmonary Disease in Primary Care: J. Wiley, Chichester, 2006. D. K. McKenzie, M. Abramson, A. J. Crockett, N. Glasgow, S. Jenkins, C. McDonald, Richard Wood-Baker, and P. A. Frith, "The COPD-X Plan: Australian and New Zealand Guidelines for the management of Chronic Obstructive Pulmonary Disease," ed: Australian Lung Foundation, 2009. R. Rodriguez-Roisin, "Global Initiative for Chronic Obstructive Lung Disease (GOLD) guidelines for chronic obstructive pulmonary disease," National Heart, Lung, and Blood Institute, National Institutes of Health, USA, and the World Health Organization2009. M. Voll-Aanerud, T. M. L. Eagan, T. Wentzel-Larsen, A. Gulsvik, and P. S. Bakke, "Respiratory symptoms, COPD severity, and health related quality of life in a general population sample," Respiratory medicine, vol. 102, pp. 399-406, 2008. M. Eisner, L. Trupin, P. Katz, E. Yelin, G. Earnest, J. Balmes, and P. Blanc, "Development and validation of a survey-based COPD severity score," Chest, vol. 127, pp. 1890-1897, 2005. D. E. Niewoehner, "Outpatient Management of Severe COPD," N Engl J Med, vol. 362, pp. 1407-1416, April 15, 2010 2010. J. L. Viejo-Ba uelos, A. Pueyo-Bastida, and A. Fueyo-Rodríguez, "Characteristics of outpatients with COPD in daily practice: The E4 Spanish project," Respiratory medicine, vol. 100, pp. 2137-2143, 2006. M. Miravitlles, C. de la Roza, J. Morera, T. Montemayor, E. Gobartt, A. Martín, and A. S. Luis, "Chronic respiratory symptoms, spirometry and knowledge of COPD among general population," Respiratory medicine, vol. 100, pp. 1973-1980, 2006. D. Lieberman, O. Shmarkov, Y. Gelfer, R. Varshavsky, and D. V. Lieberman, "Prevalence and clinical significance of fever in acute exacerbations of chronic obstructive pulmonary disease," European Journal of Clinical Microbiology & Infectious Diseases, vol. 22, pp. 75-78, 2003. G. Caramori, I. M. Adcock, and A. Papi, "Clinical definition of COPD exacerbations and classification of their severity," Southern medical journal, vol. 102, p. 277, 2009. M. S. Guryay, E. Ceylan, T. Gunay, S. Karaduman, F. Bengi, I. Parlak, M. Çiçek, and A. H. Cimrin, "Can Spirometry, Pulse oximetry and dyspnea scoring reflect respiratory failure in patients with chronic obstructive pulmonary disease exacerbation?," Medical Principles & Practice, vol. 16, pp. 378-383, 2007. R. Murphy, K. Mackway-Jones, I. Sammy, P. Driscoll, A. Gray, R. O'Driscoll, J. O'Reilly, R. Niven, A. Bentley, and G. Brear, "Emergency oxygen therapy for the breathless patient. Guidelines prepared by North West Oxygen Group," Emergency Medicine Journal, vol. 18, p. 421, 2001. M. R. Miller, J. Hankinson, V. Brusasco, F. Burgos, R. Casaburi, A. Coates, R. Crapo, P. Enright, C. P. M. Van Der Grinten, and P. Gustafsson, "Standardisation of spirometry," European Respiratory Journal, vol. 26, p. 319, 2005. M. C. McCormack, D. Shade, and R. A. Wise, "Spirometer calibration checks: is 3.5% good enough?," Chest, vol. 131, p. 1486, 2007. D. G. Clayton, R. K. Webb, A. C. Ralston, D. Duthie, and W. B. Runciman, "Pulse oximeter probes A comparison between finger, nose, ear and forehead probes under conditions of poor perfusion*," Anaesthesia, vol. 46, pp. 260-265, 2007. M. Sund-Levander, C. Forsberg, and L. K. Wahren, "Normal oral, rectal, tympanic and axillary body temperature in adult men and women: a systematic literature review," Scandinavian Journal of Caring Sciences, vol. 16, pp. 122-128, 2002. F. Gallefoss, "The effects of patient education in COPD in a 1-year follow-up randomised, controlled trial," Patient Education and Counseling, vol. 52, pp. 259-266, 2004. D. Scherr, R. Zweiker, A. Kollmann, P. Kastner, G. Schreier, and F. M. Fruhwald, "Mobile phone-based surveillance of cardiac patients at home," Journal of Telemedicine and Telecare, vol. 12, p. 255, 2006. B. R. Celli, W. MacNee, A. Agusti, A. Anzueto, B. Berg, A. S. Buist, P. M. A. Calverley, N. Chavannes, T. Dillard, and B. Fahy, "Standards for the diagnosis and treatment of patients with COPD: a summary of the ATS/ERS position paper," European Respiratory Journal, vol. 23, p. 932, 2004.
