Getting NICE guidelines into practice: can e-learning help?

The current issue and full text archive of this journal is available at www.emeraldinsight.com/1477-7274.htm

CGIJ 15,1

Getting NICE guidelines into practice: can e-learning help? Kieran Walsh BMJ Learning, London, UK

6

John Sandars University of Leeds, Leeds, UK

Received 8 July 2009 Revised 16 October 2009 Accepted 20 October 2009

Susheel S. Kapoor Department of Pharmacology & Therapeutics, Byramjee Jeejeebhoy Medical College & Sassoon General Hospitals, Pune, India, and

Kamran Siddiqi Leeds Institute of Health Sciences, Leeds, UK Abstract Purpose – The aim of this paper is to assess the impact of e-learning resources based on NICE guidelines in improving knowledge and changing practice among health professionals. Design/methodology/approach – NICE in collaboration with BMJ Learning developed a series of e-learning modules based on NICE recommendations relating to osteoarthritis, irritable bowel syndrome, urinary tract infection in children, and antibiotic prophylaxis against infective endocarditis. The impact of these modules was evaluated by looking at the knowledge and skills of the learners before and after they did the modules and also asking the learners about resultant practice change. Findings – A total of 5,116 users completed the modules. Completing them enabled users to increase their knowledge and skills score from the pre-test to the post-test by a statistically significant amount ( p , 0.001): from a mean of 65 per cent to 85 per cent. Qualitative feedback to the modules was overwhelmingly positive. To test long-term effectiveness, users were e-mailed six weeks after they had completed the modules to assess practice change. The response rate to the survey was 22.2 per cent. In total 88.6 per cent of those who had cared for patients with these problems since completing the module said that it had helped them put NICE guidelines into practice. Research limitations/implications – E-learning modules have high uptake, are popular and effective at helping health professionals learn about NICE guidelines and help them to put these guidelines into practice. Originality/value – The study is valuable as it shows how interactive and multimedia resources help health professionals learn about guidelines. No previous studies have been identified. Keywords E-learning, Auditing guidelines, Clinical governance, Electronic media, Health and medicine Paper type Research paper

Clinical Governance: An International Journal Vol. 15 No. 1, 2010 pp. 6-11 q Emerald Group Publishing Limited 1477-7274 DOI 10.1108/14777271011017329

Purpose Guidelines offer an important opportunity to improve the quality of care by ensuring that there is systematic and consistent use of current best evidence to inform clinical decision-making. Ensuring that health professionals follow evidence-based guidelines is a cornerstone of clinical governance. However, the implementation of guidelines remains a challenge and a variety of approaches have been used in an attempt to

inform and improve clinical care. (Bero et al., 1998) Educational approaches are an important aspect of a multi-modal strategy to implement guidelines (Grol and Grimshaw, 2003). Peer group discussion, especially related to clinical cases, is effective but there can be problems with group dynamics and attendance at these face-to-face meetings can be difficult for busy healthcare professionals. (Watkins et al., 2004) The use of e-learning has the potential to overcome these difficulties, allowing individual healthcare professionals to access educational resources at a time and a place that is convenient. E-learning has already been shown to improve the knowledge and problem-solving skills for a variety of different kinds of health professionals. (Hugenholtz et al., 2008; Walsh et al., 2007) E-learning can also take place in a clinical setting – and this may be more effective. We found only a few previous studies of the use of online educational approaches to implement guidelines. Simply providing online access to guidelines on their own does not appear to be an effective approach (Butzlaff et al., 2003). A more complex blended educational approach is effective, combining online learning resources with face-to-face meetings, but there are issues related to the time commitment to complete the educational intervention and the required resources (Fordis et al., 2005). Only one previous study was identified that used a simple interactive online resource that can be completed by individual healthcare professionals (Robson, 2009). A problem based online approach was used by 45 participants (43 general practitioners) but this did not show conclusive changes in knowledge or change in practice across a range of clinical areas. The National Institute for Health and Clinical Excellence (NICE) is “the independent organisation responsible for providing national guidance on the promotion of good health and the prevention and treatment of ill health” (www.nice.org.uk). Since 2005, NICE has produced several e-learning tools designed to support health professionals to increase their knowledge of the latest NICE guidance and to apply it in clinical and public health practice. These tools have been produced in collaboration with organisations specialising in e-learning in the medical profession – including BMJ Learning. Topics have included a wide range of clinical and public health guidelines. The tools are directed at health care professionals working primarily within the NHS. The overall aim of these tools is to support health professionals in the exchange and application of knowledge summarised in NICE guidelines. After using these tools professionals were expected to be: . more aware of the clinical and cost-effective strategies to manage clinical conditions and to improve heath as summarised in the relevant NICE guidance; . able to challenge misconceptions about implementation of the guidance; . able to apply this newly acquired knowledge in their practice and address any potential barriers; . able to reflect and compare their own practice against the audit criteria recommended with each guidance; and . able to improve the quality of the healthcare that they provide. In this paper, we describe how these modules have been effective in getting evidence into practice, based on the evaluation of the e-learning modules produced in 2008.

Getting NICE guidelines into practice 7

CGIJ 15,1

8

Methodology NICE and BMJ Learning built a series of comprehensive interactive case histories on NICE recommendations relating to osteoarthritis, irritable bowel syndrome, urinary tract infection in children, and antibiotic prophylaxis against infective endocarditis. These consisted of: (1) An initial test (the “pre-test”) to assess users’ current knowledge. (2) A series of comprehensive interactive case presentations. At each stage of the cases the user was asked how they would manage the patient. (3) A final test (the “post-test”). This is a repeat of the pre-test to find out what users learnt from the module. (4) An audit tool to encourage health professionals to monitor their performance after completing the module. This allows health professionals to measure improvements in performance – comparing care before and after the learning intervention. The impact of these modules was evaluated by looking at the knowledge and problem solving skills of the learners before and after they did the modules. We used an SPSS statistical package to analyse the data – we used Wilcoxon’s test to see whether learners increased their score from the pretest to the postest by a statistically significant amount. We also asked users to give free text feedback online describing what they thought of the module. All the feedback was coded into themes by two independent raters and was discussed until consensus was achieved. Finally, we sent an e-mail survey to users six weeks after they completed the modules to see if the module had helped them to improve their practice or confirm that their current practice was correct. Findings A total of 5,116 users completed the modules. The pre-test and post-test scores for the modules are shown in Table I. The characteristics of the users who did the modules are shown in Table II. Feedback We gathered immediate qualitative feedback on whether users found the modules useful. We received 1,611 individual pieces of feedback on the modules: . 504 users found the modules to be helpful or useful; . 483 users said the modules were a good review of the condition under study; . 122 users felt the modules were excellent and/or informative; . 130 users felt the module would lead to a change in practice; Mean Pre-test score Post-test score

Table I.

Note: Mean improvement in score: 20.3 per cent ( p-value , 0.001; Wilcoxon’s test)

65.4 85.7

.

.

. . . . . .

. .

128 users found the modules to be a good way to be updated on the current NICE guidelines; 91 users criticised various aspects of the modules – the criticisms varied widely with some feeling the modules were too long and others thinking them too basic and others saying that they were biased towards hospital medicine; 38 users said the videos were a valuable addition to the module; five users felt the videos were slow or of poor quality; 49 users commented on particular clinical points within the modules; six users liked the interactive nature of the presentation; two users found the modules to be time saving; two users felt the modules could be used as a good teaching tool as well as a learning tool; two reviews were not comprehensible; and 49 reviews could not be classified.

Getting NICE guidelines into practice 9

E-mail questionnaire – did you put the guidelines into practice? We e-mailed users six weeks after they had completed the modules to see if they had put the guidelines into practice. The response rate to the survey was 22.2 per cent. The respondents were self-selecting. Overall, 61 per cent of respondents had cared for relevant patients since completing the module. A total of 88.6 per cent of those who had cared for patients with these problems since completing the module said that the module had helped them put NICE guidelines into practice and 86.5 per cent of those who had not cared for relevant patients thought that the module would help them to put NICE guidelines into practice in the future. When asked what they might do differently as a result of the modules many free text comments were received. Common themes to emerge from these comments were: . increased confidence in diagnosing and treating the condition; . more likely to give lifestyle advice in treatment; . better able to explain the condition to patients; . more confidence about when to refer; . more likely to investigate correctly; and . less likely to request unnecessary tests. Characteristics of the users GPs GP trainees Senior hospital doctors Foundation programme doctors Nurses Other healthcare professionals Emergency medicine practitioners Medical students

Number 2,656 359 1,019 374 303 354 15 36

Table II.

CGIJ 15,1

10

Research implications and limitations This study shows that e-learning based on NICE guidance has high uptake, is popular and effective in helping health professionals learn about NICE guidelines and put these into practice. We believe that this is an effective way of influencing health professional’s clinical behaviour. There are a number of strengths to this study. It looks at usage of the modules in real life contexts in both primary and secondary care – mainly among fully qualified doctors and postgraduate trainees (research into undergraduate medical education is frequent but studies on the effectiveness of continuous professional development are rarer). We are also able to report who exactly did the modules and are able to use the functionality of online learning to demonstrate increased knowledge and problem-solving skills as a result of the learning (something which is not possible in a lecture-based environment). One possible limitation is the response rates to the surveys where users reported changes in practice – we dont know if non-responders changed their practice. There may be a degree of responder bias whereby those who changed their practice were more likely to respond. The responders constituted only a small percentage of those who completed the modules – so this study does not give a definitive quantitaive answer but is more likely to indicate qualitative findings or trends. However, the figures among the responders were very positive. Another possible limitation is the fact that we tested for improvements in knowledge and problem-solving skills in the short-term and do not know for sure whether health professionals retained the knowledge and skills in the long-term. However, the fact that the responders to the survey said that they had changed their practice is a strong suggestion that they did retain knowledge and skills. There are different forms of e-learning and this paper only looks at e-learning using the specific template of pre-test, interactive cases and post-test. The results of this study should not necessarily be extrapolated to other forms of e-learning. The study does not compare one form of e-learning with another and so we cannot say at present which learning styles might suit guidelines in particular or if the professional type, age or gender of the learners influenced outcomes.

Originality and value Ensuring that health professionals practice evidence-based medicine is a cornerstone of clinical governance. Yet health professionals often complain that guidelines are too long or difficult to read or are difficult to put into practice. This study, which to our knowledge is one of the first of its kind, is valuable as it shows how interactive and multimedia online resources help health professionals learn about guidelines and help them to put those guidelines into practice. The study has most impact and value in that it demonstrated the acceptability of the learning delivery methods. The impact on clinical practice is not quantitative. The study is a signpost to future developments and to the use of e-learning to encourage practice change. The study also has e-learning policy implications – guideline-producers should consider the benefits of producing e-learning resources along with their guidelines in order to help put those guidelines into practice.

References Bero, L.A., Grilli, R., Grimshaw, J.M., Harvey, E., Oxman, A.D. and Thomson, M.A. (1998), “Closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings. The Cochrane Effective Practice and Organization of Care Review Group”, British Medical Journal, Vol. 317, pp. 465-8. Butzlaff, M., Vollmar, H.C., Floer, B., Koneczny, N., Isfort, J. and Lange, S. (2003), “Learning with computerized guidelines in general practice? A randomized controlled trial”, Family Practice, Vol. 21, pp. 183-8. Fordis, M., King, J.E., Ballantyne, C.M., Jones, P.H., Schneider, K.H. and Spann, S.J. (2005), “Comparison of the instructional efficacy of internet-based CME with live interactive CME workshops: a randomized controlled trial”, Journal of American Medical Association, Vol. 294, pp. 1043-51. Grol, R. and Grimshaw, J. (2003), “From best evidence to best practice: effective implementation of change in patients’ care”, Lancet, Vol. 362, pp. 1225-30. Hugenholtz, N.I., de Croon, E.M., Smits, P.B., van Dijk, F.J. and Nieuwenhuijsen, K. (2008), “Effectiveness of e-learning in continuing medical education for occupational physicians”, Occupational Medicine (London), Vol. 58 No. 5, pp. 370-2. Robson, J. (2009), “Web-based learning strategies in combination with published guidelines to change practice of primary care professionals”, British Journal of General Practice, Vol. 59, pp. 104-9. Walsh, K., Rafiq, I. and Hall, R. (2007), “Online educational tools developed by Heart improve the knowledge and skills of hospital doctors in cardiology”, Postgraduate Medical Journal, Vol. 83 No. 981, pp. 502-3. Watkins, C., Timm, A., Gooberman-Hill, R., Harvey, I., Haines, A. and Donovan, J. (2004), “Factors affecting feasibility and acceptability of a practice-based educational intervention to support evidence-based prescribing: a qualitative study”, Family Practice, Vol. 21, pp. 663-71. About the authors Kieran Walsh is editor at BMJ Learning – he has a great deal of experience in writing, editing and evaluating learning resources. Kieran Walsh is the corresponding author and can be contacted at: [email protected] John Sandars has a major interest in the use of technology to enhance teaching and learning. He has researched the use of Public Health Networks and performed several reviews to inform the evidence base of the use of various e-learning approaches. Susheel Kapoor is currently working towards his postgraduate degree in Clinical Pharmacology and Therapeutics. He is currently working as a co-investigator on a few clinical trials related to respiratory medicine Kamran Siddiqi has an interest in evaluating the effectiveness of various interventions designed to get evidence into clinical practice. He has recently served as the Education Adviser to NICE.

To purchase reprints of this article please e-mail: [email protected] Or visit our web site for further details: www.emeraldinsight.com/reprints

Getting NICE guidelines into practice 11

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.