REVIEW URRENT C OPINION
Recent developments in electronic alerts for acute kidney injury Kerry L. Horne a and Nicholas M. Selby a,b
Purpose of review Efforts to improve outcomes from acute kidney injury (AKI) have focussed on timely diagnosis and effective delivery of basic patient care. Electronic alerts (e-alerts) for AKI have attracted interest as a tool to facilitate this. Initial feasibility has already been demonstrated; this review will discuss recent advances in alert methodology, implementation beyond single centres and reported effect on outcomes. Recent findings On-going descriptions of e-alerts highlight increasing variation in both detection algorithms and alert processes. In England, this is being addressed by national rollout of a standardized detection algorithm; recent data have shown this to have good diagnostic performance. In critical care, fully automated detection systems incorporating both serum creatinine and urine output criteria have been developed. A recent randomized trial of e-alerts has also been reported, in which isolated use of a text message e-alert did not affect either clinician behaviour or patient outcome. Summary As e-alerts gain popularity, consideration must be given to both the method of AKI detection and the method by which results are communicated to end-users; these aspects influence the degree of these systems’ effectiveness. This approach should be coupled to further work to study the effect on patient outcomes of those interventions that have been demonstrated to influence clinician behaviour. Keywords acute kidney injury, electronic-alerts, electronic reporting, mortality, standards of care
INTRODUCTION Although the implications of acute kidney injury (AKI) have become widely appreciated [1,2], a lack of specific treatment has meant that efforts have focussed on better delivery of basic patient care [3]. This approach responds to the range of reports across different healthcare systems that detail deficiencies in AKI treatment, with delays or even failures to recognize AKI cited as common shortcomings [4–8]. Interest in electronic detection and alerting systems for AKI has therefore increased with the aim of improving timely diagnosis and intervention. In the United Kingdom, there has been support for this approach at a national level as a part of National Health Service (NHS) England’s AKI programme [9]. Within this rapidly changing field, this review will describe recent advances in development and implementation of electronic alerts (e-alerts) for AKI with particular focus on publications within the last year.
ACUTE KIDNEY INJURY DETECTION VERSUS ALERTING Although the term ‘e-alert’ has been used widely in the setting of AKI, it is worthwhile clarifying its two essential components. Firstly, the detection element is a rule-based or mathematical process requiring an algorithm to compare serum creatinine test results to an individual’s baseline value, ideally in line with current internationally accepted definitions; the same principles can be applied to urine output data. Secondly, the alerting process refers to the method a
Department of Renal Medicine, Royal Derby Hospital Derby and Division of Medical Sciences and Graduate Entry Medicine, School of Medicine, University of Nottingham, UK b
Correspondence to Nicholas M. Selby, Department of Renal Medicine, Royal Derby Hospital, Uttoxeter Road, Derby, DE22 3NE, UK. Tel: +44 01332 340131; fax: +44 01332 789352; e-mail:
[email protected] Curr Opin Crit Care 2015, 21:479–484 DOI:10.1097/MCC.0000000000000245
1070-5295 Copyright ß 2015 Wolters Kluwer Health, Inc. All rights reserved.
www.co-criticalcare.com
Renal system
KEY POINTS Hospital-wide AKI detection systems are feasible and have been successfully implemented in a variety of settings. There is variability in AKI detection and alerting methodology between centres. The first randomized controlled trial investigating the impact of a text message alert for AKI has shown no difference in clinician behaviour or patient outcome. Further work is needed to investigate patient outcome after implementing e-alerts that have been proven to influence clinician behaviour.
by which these changes are communicated to relevant practitioners. The alerting process may take many different forms and can vary significantly in complexity, its visibility to end-users, timing, and the degree of interruption that the alert may incur when integrated with clinical information technology processes (e.g. prescribing); these factors may be highly relevant when considering potential effectiveness in triggering a clinical response.
ACUTE KIDNEY INJURY DETECTION USING SERUM CREATININE CRITERIA ALONE The widespread acceptance of diagnostic criteria for AKI has allowed the development of automated electronic systems to detect acute changes in serum creatinine concentration. Most systems, and all of those aimed at hospital-wide AKI detection, are based on detecting changes in serum creatinine alone. Increasingly, these systems are being introduced into routine clinical care, with the first development in this area summarized elsewhere [10]. In brief, initial systems were limited to AKI alerting in patients receiving potentially nephrotoxic medications [11,12], which was followed by AKI alerts in critical care [13] and then hospital-wide, the latter initially using simple methodology [14] before we reported a more sophisticated alert employing current diagnostic criteria [15]. Subsequently, there have been further reports of AKI detection systems based on the same principles but adding to an increasing number of different methodologies. Flynn and Dawnay [16] used a commercially available pathology laboratory information management system (version used Clinisys Winpath) in which a simple method to identify creatinine increase of 50% as compared with the most recently measured result within 90 days was used. Audit of 88 alerts generated over a 12-day 480
www.co-criticalcare.com
period showed good agreement between positive results and clinical adjudication of AKI, but data were not included to demonstrate the false negative rate. This is relevant as diagnostic sensitivity with so-called ‘delta checks’ has previously been shown to be reduced [14]. The algorithm methodology used was basic and did not reflect current AKI diagnostic criteria, but the subsequent alerting process was innovative consisting of several components. A comment was added to all flagged creatinine results, whereas larger increases in creatinine (>100 mmol/l) generated an additional test result to increase visibility within electronic medical records (EMR). Significant creatinine changes were telephoned to requesting clinical areas, and a list of AKI alerts was sent to a hospital outreach team twice daily. Wallace et al. [17] successfully implemented a system based on our previously published methodology, confirming our results and demonstrating scalability of technique [15]. A two-step detection process was used to apply Acute Kidney Injury Network (AKIN) criteria on a hospital-wide basis. An initial automated screening step within commercially available software identified all creatinine results with an increase of more than 26 mmol or more than 50% compared with the previous result. Flagged results were then manually checked by a clinical biochemist who selected the individual’s true baseline value and applied AKIN criteria. Alerts were displayed alongside other biochemical results and AKI stages 2 and 3 were telephoned to the requesting location. Diagnostic accuracy was good based on comparison against human adjudication (false positive rate 0%, false negative rate 2.8%), and there was a gradated increase in mortality for increasing AKI severity. Porter et al. [18] reported a fully automated AKI detection system that generated AKI results in realtime, using in-house bespoke software. The detection methodology was more complex, and depending on the time since baseline (> or 1 week), a mixture of AKIN and Risk, Injury, Failure, Loss of kidney function and End stage kidney disease (RIFLE) criteria were used. For assessment of creatinine results in the first 48 h of admission, the lowest serum creatinine concentration from the previous 12 months was selected as baseline. Those without a creatinine had a ‘theoretical’ creatinine calculated assuming an estimated glomerular filtration rate of 75 ml/min/ 1.73 m2. The alert consisted of a statement accompanying the serum creatinine result. This methodology reflects the authors’ conscious decision to maximize sensitivity of the algorithm, and although specificity figures were not quoted, it was interesting to note the relatively low mortality rate for AKI stage 1 compared with other reports. One interpretation is that this may represent a higher rate of false positive Volume 21 Number 6 December 2015
Electronic alerts for acute kidney injury Horne and Selby
results, which would be consistent with the findings of Siew et al. [19] who showed this when lowest creatinine within 12 months was used as baseline. Looking at the totality of published AKI alert systems, an increasing number are based on current diagnostic criteria, but there remains significant variation in how these criteria are applied, particularly around the selection of baseline creatinine. This variation is highlighted further when studies reported in abstract form are also considered, as seen at a recent UK national conference dedicated to AKI alerts [20]. Tackling this variability is clearly desirable, has been prioritized by expert recommendation, and in itself has driven change [21]. In England, the NHS has approached this with a mandate to all acute hospitals to adopt a standardized method of AKI detection within pathology laboratories [22]. The chosen detection algorithm is fully automated and runs in commercially available laboratory information management system, but takes a sophisticated approach in determining reference creatinine. For changes that are definitely acute, the lowest creatinine within last 48 h (for 27 mmol/l rise) or within last 7 days (for 50% rise or greater) is used; if no recent serum creatinine is available, the median value from the preceding 8–365 days is taken. A full description of the background to NHS England’s approach to this is available elsewhere [23], but implementation is increasingly happening across the United Kingdom. Of note, the mandate applies only to the process of detection; methods of alerting can be configured locally. It is reassuring that two recent studies have independently demonstrated good performance of the NHS England algorithm. In a cohort of 49 718 patients, its sensitivity in detecting International Classification of Diseases (ICD)-10-coded AKI episodes was good at 91.7%; performance was better than algorithms using a single definition of baseline creatinine [24]. Secondly, in a well characterized cohort of 4464 patients, sensitivity in detecting physician-adjudicated AKI was similar at 90.5%, with some reported differences potentially because of use of RIFLE criteria as the standard (the algorithm utilizes KDIGO) [25 ]. Although the algorithm also triggered in 14% of patients who were adjudicated as chronic kidney disease progression, the authors tested a variety of alternative methodologies and this balance between sensitivity and specificity could not be improved upon (although either could be improved at the expense of the other). In addition to these data, further coordinated measurement of the algorithm’s performance in practice will be provided by a standardized national approach, allowing an iterative and evidence-based approach to any future refinements. &
DETECTION AND ALERTING METHODOLOGY – URINE OUTPUT IN ADDITION TO SERUM CREATININE CHANGES Although there is some discordance in the literature, using urine output criteria to diagnose AKI adds prognostic information [26,27]. Although the majority of AKI detection methods are based solely on serum creatinine changes, some have sought to incorporate urine output, although only within a critical care setting [26,27]. One immediate effect of doing so is that the number of patients classified as AKI will increase [27]. Colpaert et al. [13] described an alerting system based on both urine output and serum creatinine components of the RIFLE criteria. Manual data entry was required for baseline creatinine values as well as 2-hourly urine output measurements. In a subsequent study by the same group, 90% of alerts generated were based on oliguria, which may result in a different intervention threshold compared with using serum creatinine alone [28]. With the introduction of alerts, a significant increase in the proportion of patients receiving therapeutic intervention for AKI within 60 min was seen; this was mainly because of quicker administration of fluids or vasopressors. This translated into more patients with RIFLE-R returning to baseline (i.e. no AKI) after therapy was administered, but there was no benefit on other patient outcomes. More recently, Ahmed et al. [29 ] have developed an algorithm within a customized, integrative relational research database using MatLab programming. Biochemical and urine output data from electronic charting were scrutinized every 15 min to detect AKI as per AKIN criteria. Baseline creatinine was defined as the median of results in 180 days prior to ICU admission, or when previous results were lacking was reverse calculated using an estimated glomerular filtration rate of 60 ml/min/ 1.73 m2. The study’s primary purpose was to assess performance of a combined model of oliguria and serum creatinine changes against blinded clinician adjudication of AKI. Sensitivity and specificity were good at 88 and 96% respectively, and although data presented within figures appeared to suggest that urine output and serum creatinine criteria performed differently, sensitivity and specificity for individual criteria alone were not presented. Therefore, further data are required to fully assess the effect of incorporating urine output criteria within electronic AKI detection systems; their integration has to be weighed against the additional complexity of programming and the practical implications of accurate and reliable
1070-5295 Copyright ß 2015 Wolters Kluwer Health, Inc. All rights reserved.
&
www.co-criticalcare.com
481
Renal system
recording of urine output. In time, however, incorporation of measures of oliguria may become more feasible, even on a hospital-wide basis, with increasing adoption of electronic charting.
EFFECT OF INTRODUCTION OF ELECTRONIC-ALERT SYSTEMS ON OUTCOMES The ultimate aim of e-alert systems is to impact positively on patient outcome, and the first randomized controlled trial to investigate this has been recently reported [30 ]. In a single centre, 2393 adult patients who had AKI detected by an electronic algorithm were randomized to an alert (intervention) group or usual care. In the intervention group, a responsible clinician and pharmacist received a single standardized text message on a hospital cell phone informing them of the diagnosis of AKI as well as a link to the study website. The primary outcome was a composite of death, dialysis requirement, and creatinine change up to 30 days. No significant differences were reported in the primary outcome between the alert and usual care groups in both general and critical care settings. On subgroup analysis, alerts in surgical patients resulted in a higher number of nephrology consultations and increased use of dialysis, but this did not translate into improvement in outcome. Strengths of this trial were objective primary outcomes, a large sample size and balanced demographics between the alert and control groups. However, there are several factors to consider while interpreting the lack of observed benefit, both in terms of study design and the delivery of the alert. As this was a single-centre study, care providers may have treated patients in both groups, thereby resulting in unintended changes in care of the controls. Of more importance is the design and implementation of the alert itself. The alert was generated only once per patient and was not delivered within the EMR or at the time and place of patient contact. Furthermore, it did not include recommendations for AKI care and its introduction was not supported by any other interventions such as education or AKI awareness programmes. Finally, the detection algorithm employed has been shown to be relatively insensitive in identifying cases of AKI (sensitivity only 74.2% versus ICD-10 coding) [24]. That the specific design of the alert may be the reason for its ineffectiveness is supported by the fact that there was no demonstrable difference in delivery of basic care between intervention and control groups. This is in contrast to several other studies showing that alerts delivered in a different &
482
www.co-criticalcare.com
way are effective in changing physician behaviour, both for AKI and other conditions [11,12,28,31]. In particular, within a system to identify AKI in patients taking nephrotoxic medications, the nature of the alert (passive versus interruptive) has been shown to be important in determining effectiveness [12]. Our group has also recently reported data to support the hypothesis that the nature of the alert is important [32]. Within a single-centre study, a hospital-wide AKI detection system was linked to an AKI care bundle within the EMR. The care bundle consisted of six elements: assess history and examine patient; perform urinalysis; diagnose cause of AKI; order appropriate investigations; initiate basic treatment (fluid management and medication review); and refer appropriate patients to nephrology. The care bundle was developed so that explanatory notes for each of these elements were easily visible. In a time-series design, usage of the care bundle was compared before and after the introduction of an interruptive alert, which was triggered by attempts to order investigations or prescribe medications in those patients in whom AKI had been detected. Introduction of the alert was supported by education and a poster campaign in clinical areas. Over an 11-month period, 2500 episodes of AKI occurred in 2297 patients, with 1209 and 1291 episodes occurring before and after the interruptive alert respectively. A significant increase in care bundle completion within 24 h was observed in the postimplementation period, with completion seen in only 2.2% of AKI episodes prior to the interruptive alert versus 21.6% of episodes afterwards (P < 0.001); there were also higher rates of later completion. Importantly, completion of the care bundle within 24 h was associated with improved patient outcomes. AKI progression to higher stages was lower and in an adjusted survival analysis incorporating age, sex, type of admission, ethnicity, and Charlson score; in-hospital and 30-day mortality were also lower (Fig. 1). In summary, although there are several examples to show that AKI alerts are effective at changing physician behaviour, further study of their effect on patient outcomes is required. Wilson et al. are to be applauded for conducting the first randomized study in this area, but future studies must consider the design of the alerting process to avoid testing ineffective interventions (i.e. one that does not change the process of care), as well as the improvement methodology to support its introduction. Cluster randomization or stepped wedge designs may overcome the difficulties of intervention and control groups existing side by side in a single institution. Volume 21 Number 6 December 2015
Electronic alerts for acute kidney injury Horne and Selby
0.641 (0.46, 0.891) At discharge
0.707 (0.527, 0.950) At 30 days 0.704 (0.526, 0.941) At 60 days
0
0.2
0.4
0.6
0.8
1
1.2
Odds ratio of death for patients with AKI care bundle completed within 24 hours
FIGURE 1. Odds ratio (95% confidence interval) of mortality in acute kidney injury (AKI) episodes in which AKI care bundle was completed within 24 h (reference group AKI episodes in which care bundle was completed later or not at all).
CONCLUSION Electronic detection and alerting for AKI is a rapidly evolving area. Successful implementation is increasingly described in a variety of different settings, but having tools to better detect AKI is not adequate in isolation. Detection must be combined with effective alerting processes that result in improvements in care delivery to patients with AKI; effectiveness is likely to be greater if supported by improved awareness, knowledge levels, and care pathways. These factors are also important to wider issues around AKI recognition in that blood samples or urine output data must be collected in the right patients and at the right time. It should be within this framework that future work to evaluate impact of e-alerts for AKI on patient outcomes should be evaluated. Acknowledgements We would like to thank Dr Nitin Kolhe for his assistance in manuscript/figure preparation. Financial support and sponsorship None. Conflicts of interest N.M.S. has previously received honoraria from Baxter, Pfizer, and Abbvie and is currently receiving grant funding from Kidney Research UK and the Health Foundation.
K.L.H. has no conflicts of interest.
REFERENCES AND RECOMMENDED READING Papers of particular interest, published within the annual period of review, have been highlighted as: & of special interest && of outstanding interest 1. Bagshaw SM, Uchino S, Bellomo R, et al. Septic acute kidney injury in critically ill patients: clinical characteristics and outcomes. Clin J Am Soc Nephrol 2007; 2:431–439. 2. Uchino S, Bellomo R, Goldsmith D, et al. An assessment of the RIFLE criteria for acute renal failure in hospitalized patients. Crit Care Med 2006; 34:1913– 1917. 3. Balasubramanian G, Al-Aly Z, Moiz A, et al. Early nephrologist involvement in hospital-acquired acute kidney injury: a pilot study. Am J Kidney Dis 2011; 57:228–234. 4. National Confidential Enquiry into Patient Outcome and Death (NCEPOD). Acute kidney injury: adding insult to injury. London: National Confidential Enquiry into Patient Outcomes and Death; 2009. 5. James MT, Wald R, Bell CM, et al. Weekend hospital admission, acute kidney injury, and mortality. J Am Soc Nephrol 2010; 21:845–851. 6. Mehta RL, McDonald B, Gabbai F, et al. Nephrology consultation in acute renal failure: does timing matter? Am J Med 2002; 113:456–461. 7. Stevens PE, Tamimi NA, Al-Hasani MK, et al. Non-specialist management of acute renal failure. QJM 2001; 94:533–540. 8. Cox ZL, McCoy AB, Matheny ME, et al. Adverse drug events during AKI and its recovery. Clin J Am Soc Nephrol 2013; 8:1070–1078. 9. NHS England and UK Renal Registry. Think kidneys. https://www.thinkkidneys.nhs.uk. [Accessed 16 September 2015] 10. Selby NM. Electronic alerts for acute kidney injury. Curr Opin Nephrol Hypertens 2013; 22:637–642. 11. Rind DM, Safran C, Phillips RS, et al. Effect of computer-based alerts on the treatment and outcomes of hospitalized patients. Arch Intern Med 1994; 154:1511–1517. 12. McCoy AB, Waitman LR, Gadd CS, et al. A computerized provider order entry intervention for medication safety during acute kidney injury: a quality improvement report. Am J Kidney Dis 2010; 56:832–841. 13. Colpaert K, Hoste E, Van Hoecke S, et al. Implementation of a real-time electronic alert based on the RIFLE criteria for acute kidney injury in ICU patients. Acta Clin Belg Suppl 2007; 2:322–325.
1070-5295 Copyright ß 2015 Wolters Kluwer Health, Inc. All rights reserved.
www.co-criticalcare.com
483
Renal system 14. Thomas M, Sitch A, Dowswell G. The initial development and assessment of an automatic alert warning of acute kidney injury. Nephrol Dial Transplant 2011; 26:2161–2168. 15. Selby NM, Crowley L, Fluck RJ, et al. Use of electronic results reporting to diagnose and monitor AKI in hospitalized patients. Clin J Am Soc Nephrol 2012; 7:533–540. 16. Flynn N, Dawnay A. A simple electronic alert for acute kidney injury. Ann Clin Biochem 2015; 52:206–212. 17. Wallace K, Mallard AS, Stratton JD, et al. Use of an electronic alert to identify patients with acute kidney injury. Clin Med 2014; 14:22–26. 18. Porter CJ, Juurlink I, Bisset LH, et al. A real-time electronic alert to improve detection of acute kidney injury in a large teaching hospital. Nephrol Dial Transplant 2014; 29:1888–1893. 19. Siew ED, Matheny ME, Ikizler TA, et al. Commonly used surrogates for baseline renal function affect the classification and prognosis of acute kidney injury. Kidney Int 2010; 77:536–542. 20. NHS England and UK Renal Registry. Acute Kidney Injury (AKI) Scientific Day 2014. http://www.england.nhs.uk/wp-content/uploads/2014/07/aki-sci-dayposter.pdf. [Accessed 16 September 2015] 21. Feehally J, Gilmore I, Barasi S, et al. RCPE UK consensus conference statement: management of acute kidney injury: the role of fluids, e-alerts and biomarkers. J R Coll Physicians Edinb 2013; 43:37–38. 22. NHS England. Patient safety alert on standardising the early identification of acute kidney injury; 2014. http://www.england.nhs.uk/2014/06/09/psa-aki/ 2014/. [Accessed 16 September 2015] 23. Selby NM, Hill R, Fluck RJ. Standardizing the early identification of AKI: The NHS England National Patient Safety Alert. Nephron 2015. [Epub ahead of print] 24. Sawhney S, Fluck N, Marks A, et al. Acute kidney injury: how does automated detection perform? Nephrol Dial Transplant 2015. [Epub ahead of print]
484
www.co-criticalcare.com
25. Sawhney S, Marks A, Ali T, et al. Maximising acute kidney injury alerts: a crosssectional comparison with the clinical diagnosis. PLoS One 2015; 10:e0131909. A study showing good diagnostic performance of the NHS England nationally mandated AKI detection algorithm when compared with physician adjudicated diagnosis. 26. Joannidis M, Metnitz B, Bauer P, et al. Acute kidney injury in critically ill patients classified by AKIN versus RIFLE using the SAPS 3 database. Intensive Care Med 2009; 35:1692–1702. 27. Macedo E, Malhotra R, Bouchard J, et al. Oliguria is an early predictor of higher mortality in critically ill patients. Kidney Int 2011; 80:760–767. 28. Colpaert K, Hoste EA, Steurbaut K, et al. Impact of real-time electronic alerting of acute kidney injury on therapeutic intervention and progression of RIFLE class. Crit Care Med 2012; 40:1164–1170. 29. Ahmed A, Vairavan S, Akhoundi A, et al. Development and validation of & electronic surveillance tool for acute kidney injury: a retrospective analysis. J Crit Care 2015. [Epub ahead of print] The first fully automated e-alert incorporating urine output measurement, which was implemented in a critical care setting. 30. Wilson FP, Shashaty M, Testani J, et al. Automated, electronic alerts for acute & kidney injury: a single-blind, parallel-group, randomised controlled trial. Lancet 2015; 385:1966–1974. The first randomized controlled trial to investigate the impact of e-alerts on patient outcome. 31. Lecumberri R, Panizo E, Gomez-Guiu A, et al. Economic impact of an electronic alert system to prevent venous thromboembolism in hospitalised patients. J Thromb Haemost 2011; 9:1108–1115. 32. Kolhe NV, Staples D, Reilly T, et al. Impact of compliance with a care bundle on acute kidney injury outcomes: a prospective observational study. PLoS One 2015; 10:e0132279. &
Volume 21 Number 6 December 2015