Pilot Study of a New Mathematical Algorithm for Early

Original Article

Pilot Study of a New Mathematical Algorithm for Early Detection of Late-Onset Sepsis in Very Low-Birth-Weight Infants Ilan Gur, MD1, Arieh Riskin, MD, MHA2, Gal Markel, MD, PhD3,4 David Bader, MD, MHA2 Yaron Nave, MD1 Bernard Barzilay, MD5 Fabien G. Eyal, MD6 Arik Eisenkraft, MD7 1 Neonatal Intensive Care Unit, Bikur Holim Hospital, Shaare Zedek

Medical Center, Jerusalem, Israel 2 Department of Neonatology, Bnai Zion Medical Center, Rappaport Faculty of Medicine, Haifa, Israel 3 Department of Clinical Microbiology and Immunology, Tel Aviv University, Sackler School of Medicine, Tel Aviv, Israel 4 Cancer Research Center, Sheba Medical Center, Tel-Hashomer, Ramat-Gan, Israel 5 Neonatal Intensive Care Unit, Assaf Harofeh Medical Center, Tzrifin, Israel 6 Intensive Care Nurseries, University of South Alabama Children’s and Women’s Hospital, Mobile, Alabama 7 Department of Pediatrics, Safra Children’s Hospital, Sheba Medical Center, Tel-Hashomer, Ramat-Gan, Israel These authors contributed equally.

Address for correspondence Arieh Riskin, MD, MHA, Department of Neonatology, Bnai-Zion Medical Center, 47 Golomb Street, P.O. Box 4940, Haifa 31048, Israel (e-mail: [email protected]).

Am J Perinatol

Abstract

Keywords

► sepsis ► premature infant ► very low-birth-weight infant ► morbidity ► mortality

received January 16, 2014 accepted after revision June 5, 2014

Background Diagnosis of late onset sepsis (LOS) in very low birth weight (VLBW) preterm infants relies mainly on clinical suspicion, whereas prognosis depends on early initiation of antibiotic treatment. RALIS is a mathematical algorithm for early detection of LOS incorporating six vital signs measured every 2 hours. Objective The aim of this study is to study RALIS ability to detect LOS before clinical suspicion. Study Design A total of 118 VLBW preterm infants (gestational age < 33 weeks, birth weight < 1,500 g) were enrolled in a prospective multicentered study. Vital signs were recorded prospectively up to day 21 of life in a blinded manner, with no effect on standard care. The primary end point was comparison of the rates and timing of detection of LOS between RALIS and clinical/culture evidence of LOS. Results Of the 2,174 monitoring days, RALIS indicated sepsis in 590 days, and LOS was positively diagnosed in 229 days. Sensitivity, specificity, positive, and negative predictive values were 74.6, 80.7, 38.8, and 95.1%, respectively. RALIS provided an indication for sepsis 3 days on the average before clinical suspicion. Conclusion RALIS has a promising potential as an easy to implement noninvasive early indicator of LOS, especially for ruling out LOS in VLBW high-risk infants.

Copyright © by Thieme Medical Publishers, Inc., 333 Seventh Avenue, New York, NY 10001, USA. Tel: +1(212) 584-4662.

DOI http://dx.doi.org/ 10.1055/s-0034-1384645. ISSN 0735-1631.

Early Detection of Sepsis in Preterm Infants

Gur et al.

Evaluation and treatment of neonates for possible bacterial infections is one of the most common clinical practices in the newborn nurseries.1 Currently, perinatally acquired bacterial sepsis is considered both in infants with signs and symptoms of sepsis and in those who are at significant risk without apparent clinical manifestations. The relevant signs and symptoms are often nonspecific,2 and may end up being attributed to other causes and thus delay diagnosis and treatment. Because of the fulminant course of true sepsis in neonates, clinical suspicion of sepsis mandates prompt diagnostic tests and administration of appropriate antimicrobial and supportive therapy.1 Unfortunately, in many cases clinical suspicion is not raised early enough. Late onset sepsis (LOS) among very low-birth weight (VLBW; < 1,500 g) premature infants remains a frequent and significant cause of morbidity. LOS is associated with various neonatal conditions, longer hospitalization, and at least twofold higher mortality rates.3–5 The incidence of LOS in VLBW infants varies from 16 to 30% in different countries, and the risk is inversely correlated with birth weight and gestational age.3–7 Neonatal conditions such as respiratory distress syndrome, patent ductus arteriosus, severe intraventricular hemorrhage, necrotizing enterocolitis and bronchopulmonary dysplasia are well known risk factors for LOS.3–5 In addition, mechanical ventilation, indwelling lines and prolonged parenteral nutrition, impose additional risks for development of LOS.3,8 The common approach to suspected LOS includes administration of empirical antimicrobial therapy after obtaining appropriate cultures.2,9 Nonspecific sepsis screen tests are used to evaluate the likelihood of infection, and specific diagnostic tests are performed to confirm the presence of a specific pathogen in body fluids.1 Ideally, screening tests for neonatal sepsis would exhibit both high positive and negative predictive accuracy for the presence or absence of disease. Currently, isolation of bacteria from a central body fluid is the standard and most specific method to diagnose sepsis. However, the most useful specific diagnostic test, the blood culture, has too many false negative (FN) results to qualify as a gold-standard test for making the diagnosis.1,9 Moreover, the time required to obtain a positive culture is relatively long. The clinician is often required for a subjective judgment call, taking into consideration the constellation of signs, symptoms, screening tests, and specific diagnostic tests, before the diagnosis of sepsis can be made or excluded.1,9,10 In turn, this may delay the diagnosis and treatment of sepsis. Therefore, an objective, sensitive tool, for early detection of sepsis as an aid to the clinician is needed. The objective of this study was to test the ability of a newly developed computerized mathematical algorithm based on noninvasive acquisition of vital signs (RALIS) to detect LOS early in VLBW premature infants before clinical suspicion. The premise underlying the development of RALIS algorithm was that subtle vital sign changes may precede a clinician’s notice of the nonspecific signs of LOS. Thus, our hypothesis was that RALIS would detect LOS with high diagnostic confidence more than 2 days before clinical suspicion. American Journal of Perinatology

Methods The pilot study was conducted as a multicenter blinded comparative prospective study. Inclusion criteria included preterm infants born after less than 33 weeks of gestation with birth weight of 1,500 g or less, who were born between June 2009 and March 2011 and treated in the neonatal intensive care units (NICU) of three hospitals in Israel: Bikur Holim (Jerusalem), Bnai Zion (Haifa), and Assaf Harofe (Tzrifin, Rishon Le’Zion). The study was approved by the institutional review boards of all the three hospitals. Preterm infants with severe congenital malformations or those who did not survive more than 3 days were excluded. During the study period time, 19,963 neonates were born in the three hospitals, of which 142 qualified for the study. All of the clinical parameters detailed later were uploaded by NICU nurses into the RALIS Sepsis Detecting System. For each infant, data collection every 2 hours was initiated immediately after delivery and was continued for 21 consecutive days. The measurements were performed without disturbing standard clinical practice, while the medical teams taking care of the infant were blinded to RALIS output. Thus, the clinical management and any decision to initiate antimicrobial therapy were conducted according to the standard clinical procedures and not according to RALIS. The two primary outcomes measured were whether RALIS diagnosed LOS before clinical suspicion and comparison of timing of LOS detection between RALIS and clinical/culture evidence of LOS. Originally, the study population included 142 patients from all the three hospitals and was comprised of 48% males. The mean gestational age was 28.3 2.3 weeks and the mean birth weight was 1,079 289 g. There were no significant differences in these parameters among the different hospitals (data not shown). RALIS was implemented from birth up to day 21 of life in 142 very low-birth-weight preterm infants for a total of 2,730 monitoring days (in hospital 1: n ¼ 88, 1,626 days; in hospital 2: n ¼ 30, 548 days; and in hospital 3: n ¼ 24, 556 days). In the NICU of hospital 3, none of the newborns developed any clinical or laboratory evidence of LOS, and accordingly there was no positive sepsis alerts by RALIS. This NICU has a different policy for administration of antimicrobial treatment, including wide use of antimicrobial medications for relatively long periods in a manner that could be considered as prophylactic antibiotic coverage, which most probably contributed to these findings. Inclusion of this data could have biased our results significantly. Therefore, data from this hospital was excluded from further analysis. Thus, this study on RALIS ability to provide positive indications of infection was eventually based on the data from the NICUs of hospitals 1 and 2 only (n ¼ 118).

RALIS Technology RALIS is a computerized mathematical algorithm for continuous monitoring of infants to detect the potential onset of sepsis. End users are medical personnel, who enter the relevant data into the algorithm in parallel to the routine medical documentation. The algorithm was originally designed based on a derivation cohort of 200 subjects, 100 with


Gur et al.

Fig. 1 Exemplar readouts by RALIS. Figure shows exemplar readouts by RALIS of a nonseptic patient (a: top left) and of a patient with confirmed sepsis (b: top right). Y-axis represents the S-factor (Sepsis-factor), plotted against time (each block represent 48 hours). Light gray (originally appears in green color) indicates S-factor < 5 (no sepsis), whereas dark gray (originally appears in red color) indicates S-factor 5 (sepsis alert). Underneath is a typical daily readout of all participating infants in one of the neonatal intensive care units (c). Each patient has his own S-factor either in color or presented numerically (patients names are covered to assure their privacy).

definite diagnosis of LOS and 100 who were obviously healthy. It was then further studied and tested in clinical settings. The algorithm was configured to process measurement values for a plurality of vital signs and to generate an alarm signal indicative of the onset of sepsis. The analyzed vital signs include heart rate, respiratory rate, core body temperature, body weight, documented low oxygen saturation (< 85%), and documented bradycardia (< 100 beats per minute). Each vital sign is associated with one or more threshold values. For example, the temperature vital sign is associated with a high alarm limit of 38.3°C, and a low alarm limit of 35.5°C. The presence of low blood oxygen saturation or bradycardia events during the 2-hour interval are marked with a (þ) sign, whereas the absence of such events are

marked with a () sign. There is no significance to the number of events during an interval. All parameters are monitored 12 times a day, except for body weight, which is determined once daily. When using the algorithm, it should be given a 72-hour training period for patient-tailored calibration to determine the personal baseline. Final readout is given in a 0 to 10 relative scale, with baseline levels defined as 0, and the threshold for definition of sepsis is defined as 5 or more. The sepsis factor (S factor) is then plotted against time (►Fig. 1).

Data Recording and Interpretation Patients’ data were recorded in Excel spread sheets and included demographic data, gestational age and birth weight, American Journal of Perinatology


Gur et al.

actual number of days on RALIS, and day by day recording of RALIS S-factor readouts (either green negative signal—no sepsis—score 0–4; or red signal—sepsis alert—score of 5–10) in correlation to clinical suspicion of sepsis and diagnosis of sepsis. Sepsis was defined as either culture-proven sepsis, where body fluid culture (mostly blood) was positive, or clinical sepsis when all clinical and laboratory parameters (e.g., complete blood count [CBC] and differential, I:T neutrophil ratio, C-reactive protein [CRP]) indicated sepsis, yet cultures were negative. In both cases, the infants got full antimicrobial treatment for sepsis. The data were interpreted in two ways. The first was according to RALIS negative and positive days. On the basis of the correlation to the clinical diagnosis of sepsis, each day was defined as true negative if no sepsis or true positive (TP) if coinciding with sepsis diagnosis (either culture proven or clinical). False-positive days were defined as days when RALIS gave positive alert yet no evidence of sepsis was found. FN days were days when RALIS gave no sepsis alert while sepsis was present. By definition of those who developed the algorithm and in correlation to its performance in previous preliminary studies, RALIS positive indication of sepsis was considered up to 10 days before the actual episode. However, if RALIS gave intermittent alerts within this time frame, days with positive alert were indeed considered as TP alerts but negative days were considered as FN. To correct for possible bias in this interpretation according to daily RALIS readouts, our other analysis of the data was performed according to sepsis episodes. Here, the outcome measures included two components. The first was whether RALIS gave positive sepsis alert before or concomitant to the

clinical suspicion and diagnosis of sepsis, or whether it was delayed or missing altogether. The second was the number of days RALIS gave sepsis alert before actual sepsis was clinically found or the number of days RALIS indication was delayed.

Statistical Analysis Statistical analysis was performed by using SigmaStat, version 2.03 (SPSS, Inc., Chicago, IL) software. Sensitivity, specificity positive and negative predictive values were calculated. The statistical tests employed included chi-square test, Student ttest and Mann–Whitney rank sum tests on medians in case of nonparametric distribution. Of these, the specific tests used each time are specified in the tables. Statistical significance was set at 0.05 levels.

Results This study on RALIS ability to provide positive indications of infection was eventually based on the data from the NICUs of hospitals 1 and 2 only (n ¼ 118), as discussed in the methods section earlier. Mean gestational age of the infants included in the study was 28.1 2.2 weeks and their mean birth weight was 1,056 292 g. There were no significant differences in birth weight or gestational age between the NICUs (►Table 1). There was some difference in the length of RALIS total recording time between the two NICUs, but practically this difference was minor (►Table 1). The prevalence of infections, as expressed in the number of sepsis days, was significantly higher in NICU 1 compared with NICU 2. However, the rate of infections, as reflected from the percentage of infants diagnosed with sepsis and the number of sepsis episodes per

Table 1 Characteristics of study participants, RALIS recordings and sepsis episodes Variable

All

NICU 1

NICU 2

p Value (NICU 1 vs. NICU 2)

Number of infants (N)

118

88

30

Total number of days of RALIS monitoring

2,174

1,626

548

Gestational age (wk)

28.1 2.2 (28.0)

28.1 2.3 (28.0)

28.1 2.0 (28.4)

NSa

Birth weight (g)

1056 292 (1,090)

1076 303 (1,112)

998 253 (930)

NSb

RALIS (d)

18.4 4.4 (21.0)

18.5 4.5 (21.0)

18.3 4.0 (20.0)

< 0.05a

Total sepsis days, n (%)

307 (14.1)

267 (16.4)

40 (7.3)

< 0.001c

Culture positive sepsis episodes

44

38

6

Clinical sepsis episodes

28

22

6

Total sepsis episodes

72

60

12

Infants diagnosed with sepsis, n (%)

52 (44.1)

42 (47.7)

10 (33.3)

NSc

Sepsis episodes per infant

1.4 0.6 (1.0)

1.4 0.6 (1.0)

1.2 0.4 (1.0)

NSa

Abbreviations: NICU, neonatal intensive care unit; NS, not significant. Note: Data presented as mean SD (median) unless indicated otherwise. a Mann–Whitney rank sum test on medians. b t-test. c Chi-square test. American Journal of Perinatology

Early Detection of Sepsis in Preterm Infants infants, was not significantly lower in NICU 2 compared with NICU 1 (►Table 1).

Diagnosis of Late Onset Sepsis by RALIS In 69.3% (1,506 of 2,174) of the monitoring days, neither were there any clinical signs of LOS nor were there any indications for sepsis by RALIS (►Table 2). RALIS gave sepsis alerts in 23.4% (590/2,174) of the monitoring days, yet only in 38.8% (229/590) of these RALIS positive days, clinical evidence of LOS was also obtained (►Table 2). RALIS failed to indicate on clinically proven LOS in 25.4% (78/307) of the days (►Table 2). Sensitivity was approximately 75%, while specificity was around 81%. RALIS had a very good negative predictive value (95%), that is, if RALIS gave no indication of sepsis it was very unlikely that the infant had infection on that day. However, unfortunately RALIS had relatively poor positive predictive

Table 2 Diagnosis of LOS by RALIS Sepsis

No Sepsis

All

n ¼ 307

n ¼ 1,867

Positive RALIS, n ¼ 590

229a, TP ¼ 74.6%

361, FP ¼ 19.3%

Negative RALIS, n ¼ 1,584

78, FN ¼ 25.4%

1,506, TN ¼ 80.7%

NICU 1

n ¼ 267

n ¼ 1359


203b, TP ¼ 76.0%

253, FP ¼ 18.6%

Negative RALIS, n ¼ 1,170

64, FN ¼ 24.0%

1,106, TN ¼ 81.4%

NICU 2

n ¼ 40

n ¼ 508


26c, TP ¼ 65.0%

108, FP ¼ 21.3%

Negative RALIS, n ¼ 414

14, FN ¼ 35.0%

400, TN ¼ 78.7%

ELBW

n ¼ 222

n ¼ 815

d


168 , TP ¼ 75.7%

151, FP ¼ 18.5%

Negative RALIS, n ¼ 718

54, FN ¼ 24.3%

664, TN ¼ 81.5%

Abbreviations: ELBW, extremely low-birth weight; FP, false positive; FN, false negative; LOS, late onset sepsis; NICU, neonatal intensive care unit; NPV, negative predictive value; PPV, positive predictive value; TP, true positive; TN, true negative. Note: NPV ¼ TN/(TN þ FN); PPV ¼ TP/(TP þ FP); Sensitivity ¼ TP/(TP þ FN); Specificity ¼ TN/(TN þ FP). a 129 culture proven sepsis þ 100 clinical sepsis. χ2 ¼ 404.32 with 1 degree of freedom (p < 0.001); Sensitivity ¼ 74.59%; Specificity ¼ 80.66%; NPV ¼ 95.08%; PPV ¼ 38.81%. b 120 culture proven sepsis þ 83 clinical sepsis. χ2 ¼ 361.68 with 1 degree of freedom (p < 0.001); Sensitivity ¼ 76.03%; Specificity ¼ 81.38%; NPV ¼ 94.53%; PPV ¼ 44.52%. c 9 culture proven sepsis þ 17 clinical sepsis. χ2 ¼ 36.07 with 1 degree of freedom (p < 0.001); Sensitivity ¼ 65.00%; Specificity ¼ 78.74%; NPV ¼ 96.62%; PPV ¼ 19.40%. d 102 culture proven sepsis þ 66 clinical sepsis. χ2 ¼ 264.86 with 1 degree of freedom (p < 0.001); Sensitivity ¼ 75.68%; Specificity ¼ 81.47%; NPV ¼ 92.48%; PPV ¼ 52.66%.

Gur et al.

value (PPV) (less than 40%) (►Table 2). This poor PPV was even more pronounced in the NICU of hospital 2, and this is also probably related to the significantly lower prevalence of LOS days in this NICU compared with NICU 1 where PPV was slightly better (►Table 2). Otherwise, there were no significant differences in the results between the NICUs of hospitals 1 and 2 (►Table 2). When secondary analysis of only the extremely low-birth weight (ELBW) (birth weight 1,000 g) was done, the performance of RALIS improved with higher PPV ( 53%), most probably because the prevalence of LOS in this population is higher (►Table 2). Receiver operating curve (ROC) demonstrating the diagnostic performance of RALIS at the preset S-factor 5 is presented in ►Fig. 2. Area under the curve was calculated to be 0.82. However, analysis of RALIS ability to predict infection based on day-to-day analysis might not be accurate enough to evaluate the ability of RALIS algorithm to predict LOS episode before clinical suspicion could be raised. Thus, to evaluate RALIS signal ability to predict the actual time of an episode of LOS, we further analyzed our data based on true LOS episodes versus false-positive system alerts and RALIS delays or failures in detecting LOS (►Table 3). In this analysis, we also analyzed how many days before LOS did RALIS give its signal of sepsis alert (►Table 3). RALIS gave a TP alert in 80.6% (58/72) of the LOS episodes, and it did so 3 days on the average (mean, 2.9 2.0; median, 3 days) before the clinical suspicion that led to the diagnosis was raised. RALIS algorithm sensitivity was approximately 80% based on this analysis (►Table 3). However, the rate of false-positive RALIS alerts’ episodes was also high (twice as much as the TP alerts [116 vs. 58], resulting again in a low PPV of 33% [►Table 3]).

Fig. 2 Receiver operating curve demonstrating the diagnostic performance of RALIS at the preset S-factor 5. Area under the curve is 0.82. ROC, receiver operating curve. American Journal of Perinatology


Gur et al.

Table 3 RALIS signal ability to predict the time of an episode of LOS Variable

All

NICU 1

NICU 2

Culture positive sepsis episodes

44

38

6

TP RALIS alert

35

31

4

Days of RALIS alert before clinical diagnosisa

3.3 1.9 (3.0)

3.2 2.0 (3.0)

4.0 1.6 (4.0)

FN RALIS episodes

9

7

p Value (NICU 1 vs. NICU 2)

NSb

2 c

Days of RALIS indication delay compared with clinical diagnosisa

2.4 2.4 (1.5)

2.8 2.6 (2.0)

1.0 0.0 (1.0)

RALIS Sensitivity to detect culture positive sepsis episodes

79.54%

81.58%

66.67%

Clinical sepsis episodes

28

22

6

TP RALIS alert

23

19

4


2.3 1.9 (3.0)

2.4 1.9 (3.0)

2.0 1.8 (2.0)

FN RALIS episodes

5

3

NSb

NSd

2 c


2.2 2.2 (1.0)

3.0 2.6 (2.0)

1.0 0.0 (1.0)

RALIS Sensitivity to detect culture negative clinical sepsis episodes

82.14%

86.36%

66.67%

Total sepsis episodes

72

60

12

TP RALIS alert

58

50

8


2.9 2.0 (3.0)

2.9 2.0 (3.0)

3.0 1.9 (3.5)

FN RALIS episodes

14

10

4


2.3 2.2 (1.0)

2.9 2.5 (2.0)

1.0 0.0 (1.0)

RALIS Sensitivity to detect sepsis episodes

80.56%

83.33%

66.67%

FP RALIS alert episodes

116

80

36

PPV of RALIS for detecting sepsis episodes

33.33%

38.46%

18.18%

NSd

NSb

NSb

Abbreviations: FN, false negative; FP, false positive; LOS, late onset sepsis; NS, not significant; PPV, positive predictive value; TP, true positive. a Data presented as mean SD (median). b Mann–Whitney rank sum test on medians. c In one episode RALIS never gave even a delayed indication of sepsis. d t-test.

Discussion LOS in VLBW preterm infants is of great medical concern due to the frequent complications, and most notably the high mortality rates.3–5 The high risks associated with untreated infection and the lack of reliable clinical or laboratory predictive methods result in a low threshold for initiation of empiric antibiotic therapy.1,11 Unfortunately, the clinical manifestations of LOS are often nonspecific and require a very high index of suspicion to note. Nevertheless, the decision to initiate antimicrobial therapy is still chiefly guided by integration of clinical skills of the medical team. The timing of initiation of antimicrobial therapy is of paramount importance, as clinical deterioration takes a fulminant course in many of the patients, sometimes even within hours after American Journal of Perinatology

initiation of therapy.10 Thus, there is a constant search for new methods that will help the clinicians to accurately detect sepsis as early as possible. The systemic inflammatory response to sepsis is highly complex. Indeed, over the last two decades many correlates with septic condition were identified, for example, Procalcitonin, CRP, G-CSF, IL-1β, TNF-α, IL-6, MIP11, CXCL8, MCP-1, MCP-2, and IL-10.12–20 However, a simple measurement of most of these single markers does not enable reliable early detection of sepsis, except for Procalcitonin.21–23 Only integration of multiple parameters and markers has the capability to predict development of sepsis.24,25 Recent systemic algorithm-based networks with the capability to analyze numerous biological parameters have been set for the analysis of systemic inflammation in humans.26 However, despite some

Early Detection of Sepsis in Preterm Infants potentially good results with these tools, most of these tests are taken only when the clinician raises the clinical suspicion of sepsis. Thus, the considerable dependency on the clinical suspicion still hasn’t been resolved. In addition, the continuous monitoring of a set of serum biomarkers does not provide a simple, cost-effective solution. This study endeavors to address the important clinical challenge of trying to reliably and accurately identify LOS early in the course of illness. RALIS presents a tool for ongoing screening of premature infants for LOS, which has a relatively low burden for use by both clinicians and patients, because it is noninvasive and uses clinical data that are collected for routine care. The premise underlying the development of RALIS algorithm was that subtle vital sign changes may precede a clinician’s notice of the nonspecific signs of LOS. This idea lied in the basis of studies on heart rate variability in VLBW preterm infants. Reduced variability and transient decelerations in heart rate may be present in the hours to days before diagnosis of late onset proven or clinical neonatal sepsis. These abnormal heart characteristics in response to systemic infection and inflammation have been characterized mathematically, and the resulting HeRo score was the fold increase in risk of sepsis during the next 24 hours.27 The index can be computed in real time and displayed continuously at the bedside. In a large, multicenter randomized trial of the impact of monitoring heart rate characteristics in VLBW infants Moorman et al28 have shown that HeRo score continuously presented as a continuous readout at the cot side for clinicians to use had the potential to serve as early indicator of developing sepsis. They have shown that monitoring this HeRo score in high-risk premature infants might result in improved outcomes through early warning of subacute potentially catastrophic illnesses characterized by systemic inflammation, such as sepsis. Mortality rate in infants, whose heart rate characteristics monitoring was displayed, was reduced from 10.2 to 8.1%. The mortality rate reduction was entirely caused by reduced mortality in ELBW infants.28,29 Here, we present the results of a pilot study in VLBW preterm infants. The primary objectives were to define the parameter settings and test the ability of the algorithm to give an early alert of sepsis. Although RALIS exhibited reasonable sensitivity ( 75%) and specificity ( 81%), the diagnostic performance of this tool in this first prospective study was unfortunately underwhelming. The PPV of the test was quite low (< 40%), except in the subgroup of ELBW infants where it was slightly better ( 53%). If used in clinical practice in its current form, RALIS would likely substantially increase the number of sepsis evaluations and courses of empiric antibiotic treatment premature infants receive while in the NICU, which could not be considered benign interventions. Even the sensitivity of the test that is only 75% implies that 25% of the cases of LOS were not identified by RALIS in this study. The negative predictive value was very high (95%). This may suggest that negative indication of sepsis by RALIS essentially rules out LOS and the need for empirical antimicrobial therapy. However, this may also be a reflection of the overall low prevalence of LOS in the population studied.

Gur et al.

Although RALIS overdiagnosed LOS in approximately 2:1 ratio, it is important to emphasize that among the newborns with clinically proven LOS, RALIS correctly indicated sepsis approximately 3 days before standard clinical practice, thus still meeting some of the primary end points of this study. This can potentially affect the medical management and overall prognosis significantly. We believe that RALIS has the potential to serve as an early indicator of sepsis, in many cases before the clinician will even raise suspicions and receive final results of blood cultures, which in turn, would enable early initiation of empirical antibiotic treatment. However, as discussed earlier, the current results suggest that the price for administration of antimicrobial therapy 3 days earlier is treating twice as more newborns. It may be argued that the potentially devastating effects of late therapeutic intervention in cases of LOS seem to overcome the potential complications or costs of the unneeded antimicrobial therapy, especially if discontinued immediately as cultures turn negative (usually within 48 hours), however, this is very debatable in face of the increasing antibiotic resistance of microorganisms in hospitals in general, and NICUs specifically. Thus, the prognostic and economic benefit of employing RALIS as an early indicator of LOS and subsequent administration of antimicrobial therapy must be tested in additional prospective multicentered trials, probably after trying to improve RALIS diagnostic performance. Potential ways of improving RALIS diagnostic performance could be, for example, increasing the frequency of vital signs entered into RALIS. Another approach could be changing the S-factor. Retrospective analysis of the data (not presented here) showed that clinically valid indication for sepsis requires the persistence of S-factor 5 for at least 6 consecutive hours. This could comprise the cutoff definition in the future prospective interventional trials. Alternatively, the S-factor could be set at 6 or 7 and ROC curves analyzed for the different cutoff values. This, unfortunately, could not have been done in this study because of RALIS current algorithm definitions set at S-factor 5 that were shut down to avoid any intervention by the researchers that might cause bias in analyzing the results. It is possible that future studies could combine RALIS with one or more of the biological markers mentioned earlier, in a way that would improve its PPV, and decrease the very high rates of FN alerts. Importantly, RALIS monitors vital signs, which are monitored routinely, such as heart and respiratory rates, saturation level, core temperature, and body weight. Therefore, RALIS offers additional advantages, as it does not require special tests, as in previously described markers of sepsis, it does not require introduction of specialized apparatuses and it enables an almost “continuous” noninvasive patient-tailored monitoring. One potential limitation of this study is that the nurse was recording the vital signs for inclusion every 2 hours. There is certainly a subjective component to recording vital signs, as the vital sign recorded for a given time interval was only one data point from a multihour window, and the vital signs recorded might have been influenced by the nurse’s perception of that patient’s clinical status. Future improvements in RALIS should include continuous connection of the American Journal of Perinatology


Gur et al.

NICU’s cardiorespiratory monitors to RALIS. Extraction of data points would then be done automatically based on computerized averaging of continuous monitoring data. Essentially, such an algorithm could then be easily integrated in any NICU. It should be emphasized, however, that RALIS is not a replacement for the clinician’s judgment, and it should have a role as an aid for medical decision making. The group of infants with culture negative clinical sepsis is an important group, especially when tools for early diagnosis of LOS, such as RALIS, are studied. However, this is many times the hardest group to define and to research. This may affect the generalization of the results to other settings and units. That is why we kept record of the two groups of culture positive–proven sepsis and culture negative clinical sepsis groups in the results section (►Tables 2 and 3) to allow the reader to judge the performance of RALIS separately in each one of them if he desires to do so. Other limitations of this study, other from those related to RALIS algorithm in its current form and the recording system as discussed earlier, include its relatively small sample size and the fact that we have ended up with only two centers that were studied, one center excluded from the analysis. Furthermore, larger prospective multicenter studies that will use improved RALIS recording system and mathematical algorithm, possibly combined with some biomarker could be promising. It should be noted that RALIS requires a 72 hours’ training period, and is therefore relevant mostly for LOS and not for early onset sepsis. In addition, RALIS provides accurate indications only in the first 21 days of life. It is still unclear why the algorithm does not apply well after day 21, even after allowing a second training period. A possible explanation might be the substantial physiological changes taking place in the preterm infants, which would require additional investigation and adaptation of the mathematical algorithm. In conclusion, we provide here first evidence for the promising potential of RALIS as an easy to implement noninvasive early indicator of LOS in VLBW infants.

Conflict of Interest The authors declare no conflicts of interest.

5 Fanaroff AA, Korones SB, Wright LL, et al; The National Institute of

6

7

8

9

10

11

12

13

14

15

16 17

18

19

20

References

21

1 Gerdes JS. Diagnosis and management of bacterial infections in the

neonate. Pediatr Clin North Am 2004;51(4):939–959, viii–ix 2 Sivanandan S, Soraisham AS, Swarnam K. Choice and duration of

antimicrobial therapy for neonatal sepsis and meningitis. Int J Pediatr 2011;2011:712150 3 Makhoul IR, Sujov P, Smolkin T, Lusky A, Reichman B. Epidemiological, clinical, and microbiological characteristics of late-onset sepsis among very low birth weight infants in Israel: a national survey. Pediatrics 2002;109(1):34–39 4 Stoll BJ, Hansen N, Fanaroff AA, et al. Late-onset sepsis in very low birth weight neonates: the experience of the NICHD Neonatal Research Network. Pediatrics 2002;110(2 Pt 1):285–291

American Journal of Perinatology

22

23

24

Child Health and Human Development Neonatal Research Network. Incidence, presenting features, risk factors and significance of late onset septicemia in very low birth weight infants. Pediatr Infect Dis J 1998;17(7):593–598 Morioka I, Morikawa S, Miwa A, et al. Culture-proven neonatal sepsis in Japanese neonatal care units in 2006-2008. Neonatology 2012;102(1):75–80 Hornik CP, Fort P, Clark RH, et al. Early and late onset sepsis in verylow-birth-weight infants from a large group of neonatal intensive care units. Early Hum Dev 2012;88(Suppl 2):S69–S74 Gaynes RP, Edwards JR, Jarvis WR, Culver DH, Tolson JS, Martone WJ; National Nosocomial Infections Surveillance System. Nosocomial infections among neonates in high-risk nurseries in the United States. Pediatrics 1996;98(3 Pt 1):357–361 Polin RA; Committee on Fetus and Newborn. Management of neonates with suspected or proven early-onset bacterial sepsis. Pediatrics 2012;129(5):1006–1015 Lukaszewski RA, Yates AM, Jackson MC, et al. Presymptomatic prediction of sepsis in intensive care unit patients. Clin Vaccine Immunol 2008;15(7):1089–1094 Horisberger T, Harbarth S, Nadal D, Baenziger O, Fischer JE. G-CSF and IL-8 for early diagnosis of sepsis in neonates and critically ill children - safety and cost effectiveness of a new laboratory prediction model: study protocol of a randomized controlled trial [ISRCTN91123847]. Crit Care 2004;8(6):R443–R450 Claeys R, Vinken S, Spapen H, et al. Plasma procalcitonin and Creactive protein in acute septic shock: clinical and biological correlates. Crit Care Med 2002;30(4):757–762 Berner R, Niemeyer CM, Leititis JU, et al. Plasma levels and gene expression of granulocyte colony-stimulating factor, tumor necrosis factor-alpha, interleukin (IL)-1beta, IL-6, IL-8, and soluble intercellular adhesion molecule-1 in neonatal early onset sepsis. Pediatr Res 1998;44(4):469–477 Damas P, Reuter A, Gysen P, Demonty J, Lamy M, Franchimont P. Tumor necrosis factor and interleukin-1 serum levels during severe sepsis in humans. Crit Care Med 1989;17(10):975–978 Damas P, Ledoux D, Nys M, et al. Cytokine serum level during severe sepsis in human IL-6 as a marker of severity. Ann Surg 1992; 215(4):356–362 Fujishima S, Sasaki J, Shinozawa Y, et al. Serum MIP-1 alpha and IL8 in septic patients. Intensive Care Med 1996;22(11):1169–1175 Bossink AW, Paemen L, Jansen PM, Hack CE, Thijs LG, Van Damme J. Plasma levels of the chemokines monocyte chemotactic proteins-1 and -2 are elevated in human sepsis. Blood 1995;86(10):3841–3847 Sherry RM, Cue JI, Goddard JK, Parramore JB, DiPiro JT. Interleukin10 is associated with the development of sepsis in trauma patients. J Trauma 1996;40(4):613–616, discussion 616–617 Hotoura E, Giapros V, Kostoula A, Spyrou P, Andronikou S. Preinflammatory mediators and lymphocyte subpopulations in preterm neonates with sepsis. Inflammation 2012;35(3):1094–1101 Raynor LL, Saucerman JJ, Akinola MO, Lake DE, Moorman JR, Fairchild KD. Cytokine screening identifies NICU patients with Gram-negative bacteremia. Pediatr Res 2012;71(3):261–266 Brunkhorst FM, Wegscheider K, Forycki ZF, Brunkhorst R. Procalcitonin for early diagnosis and differentiation of SIRS, sepsis, severe sepsis, and septic shock. Intensive Care Med 2000;26 (Suppl 2):S148–S152 Müller B, Becker KL, Schächinger H, et al. Calcitonin precursors are reliable markers of sepsis in a medical intensive care unit. Crit Care Med 2000;28(4):977–983 Neely AN, Fowler LA, Kagan RJ, Warden GD. Procalcitonin in pediatric burn patients: an early indicator of sepsis? J Burn Care Rehabil 2004;25(1):76–80 Kabir K, Keller H, Grass G, et al. Cytokines and chemokines in serum and urine as early predictors to identify septic patients on intensive care unit. Int J Mol Med 2003;12(4):565–570

Early Detection of Sepsis in Preterm Infants 25 Toh CH, Ticknor LO, Downey C, Giles AR, Paton RC, Wenstone R. Early

identification of sepsis and mortality risks through simple, rapid clot-waveform analysis. Implications of lipoprotein-complexed C reactive protein formation. Intensive Care Med 2003;29(1):55–61 26 Calvano SE, Xiao W, Richards DR, et al; Inflamm and Host Response to Injury Large Scale Collab. Res. Program. A network-based analysis of systemic inflammation in humans. Nature 2005; 437(7061):1032–1037

Gur et al.

27 Fairchild KD, Aschner JL. HeRO monitoring to reduce mortality in

NICU patients. J Res Rep Neonatol 2012;2:65–76 28 Moorman JR, Carlo WA, Kattwinkel J, et al. Mortality reduction

by heart rate characteristic monitoring in very low birth weight neonates: a randomized trial. J Pediatr 2011;159(6): 900–906, e1 29 Groves AM, Edwards AD. Heart rate characteristic monitoringHeRO or villain? J Pediatr 2011;159(6):885–886

American Journal of Perinatology