Developing the use of simulation
Using simulation to support doctors in difficulty Kevin Stirling, George Hogg and Jean Ker, Clincial Skills Centre, University of Dundee, Tayside, UK Fiona Anderson, NHS Education for Scotland, Postgraduate Deanery, Dundee, Tayside, UK Jennifer Hanslip, Medicine for the Elderly, NHS Tayside, UK Derek Byrne, Urology Department, NHS Tayside, UK
SUMMARY Background: A small percentage of medical trainees will have performance concerns identified within their clinical practice. These trainees require specific interventions to address these concerns. The Postgraduate Ward Simulation Exercise (PgWSE) was developed as a joint collaboration between the University of Dundee and NHS Education for Scotland (NES). Context: The PgWSE takes around 90 minutes to complete. It allows assessors to directly observe the performance of trainees within a simulated ward environment. Trainees are assessed individually on their ability to prioritise competing demands, work as part of a
team and make safe informed clinical decisions. Innovation: The PgWSE is the first specific national intervention to support the assessment and remediation of trainee doctors in difficulty. Nine PgWSE scenarios have been developed to reflect clinical practice: three in general surgery, four in general medicine and two in medicine for the elderly. To enhance realism, simulated patients (SPs) are recruited and trained for each exercise. Standardised assessment tools were developed and validated for assessors to make individual and consensus judgements regarding a trainee’s performance. Self-
assessment of performance is used to disengage trainees from the simulated activity and to prepare them for constructive feedback. Implications: The PgWSE has been shown to be a valuable tool for the management of trainee doctors in difficulty in Scotland. Although the PgWSE is time and resource intensive, it has been shown to have a positive impact on the remediation of trainees and in the provision of objective, detailed and comprehensive feedback to the referring deanery. Within NES there are four geographical centres led by a postgraduate dean with responsibility for supervision of postgraduate medical education.
Trainees require specific interventions to address concerns [about their clinical practice]
Blackwell Publishing Ltd 2012. THE CLINICAL TEACHER 2012; 9: 285–289 285
The PgWSE allows the direct observation of a trainee’s performance within a safe environment
INTRODUCTION
T
he majority of postgraduate medical trainees will perform well and progress through postgraduate training as expected. However a small number will experience difficulties that will impact upon their progress.1 In Scotland, these trainees are supported by a structure introduced by NHS Education for Scotland (NES) in 2008.1 As part of this structure, a national advisory group has overarching responsibility for ensuring that trainees and their trainers have access to information and support from a variety of sources. This group facilitates access to a range of resources for remediation to supplement those available at the local deanery level.2 The Postgraduate Ward Simulation Exercise (PgWSE) is one such resource. The University of Dundee developed ward simulation exercises as part of its undergraduate medical and nursing curricula in 1999.3 In 2003, this innovative use of simulation was trialled as a potential resource for the remediation and support of postgraduate medical trainees from the east of Scotland. The PgWSE became a key resource in the management and support of these trainees, and continued to be developed over the next 5 years. In 2008, the PgWSE became a national resource accessible to the four Scottish deaneries responsible for coordinating and delivering postgraduate training.
CONTEXT After graduation, during foundation years 1 and 2, junior trainee doctors build upon the knowledge, skills and competencies acquired in undergraduate training. The learning objectives for these formative years are set out in the foundation curriculum as approved by the General Medical Council. Trainees must demonstrate competency in these core
skills prior to entering speciality training.4 When a trainee’s performance has been assessed to be below the standard expected for their stage of training, based on guidance in the foundation curriculum or through documented feedback from the multidisciplinary team, the trainee’s educational supervisor and foundation programme director will endeavour to manage them at a local ward or department level. This can include some of the interventions listed in Box 1. If this course of action is not successful, or if further evidence of performance is required, the deanery responsible for the trainee can, through the national advisory group, request a PgWSE. The PgWSE is a unique resource allowing the direct observation of a trainee’s performance within a safe and controlled environment that does not compromise patient safety.5 The PgWSE assesses the trainee’s ability to prioritise competing demands, make safe informed decisions, prescribe safely and manage the care of three patients. To increase the authenticity, volunteer simulated patients (SPs) are trained for each exercise, instead of using simulators
286 Blackwell Publishing Ltd 2012. THE CLINICAL TEACHER 2012; 9: 285–289
Box 1. Local management of trainees can include the following • A personal development plan that includes specific, measurable, achievable, realistic and time-bound (SMART) objectives. • Targeted training in procedural or clinical skills. • Patient case review or review of discharge summaries. • Support from senior colleagues and co-workers. • Ward or department educational meetings. • Use of online learning resources or local courses.
such as SimMan. Currently SPs do not provide feedback to trainees; however, the recent evaluation of a closed feedback tool will be trialled in future PgWSEs. Nine exercises have been developed to reflect actual clinical practice in the following areas: three in general surgery, four in general medicine and two in medicine for the elderly. Each exercise consists of six patient presentations that include a new admission, a patient with a communication issue and a patient who becomes acutely unwell. Exercises are mapped against the foundation curriculum outcomes to ensure
that they meet the requirements of clinical practice at foundation trainee level.6
INNOVATION The PgWSE is the first specific national intervention to support the remediation and assessment of trainee doctors in difficulty. As a national resource, the PgWSE is open to referral from across Scotland. Trainees undertaking a PgWSE may be unfamiliar with the format of the simulated ward. Therefore, prior to undertaking a PgWSE, trainees can access a dedicated internet site hosted by the University of Dundee that has preparatory information, which can be viewed and downloaded prior to undertaking their assessment (Box 2). On the day of assessment trainees are orientated to the simulated ward: this includes how to use part task trainers, for example venepuncture arms, and how to call for senior support if required. The timeline of the PgWSE is as follows. Trainee undertakes the PgWSE (30 minutes) When a trainee undertakes the PgWSE they are expected to take responsibility for the management of the simulated ward. At the start of the exercise, trainees receive a handover from a medical colleague who is available to provide advice during the PgWSE. To enhance the realism of the simulated environment a qualified nurse is present in the ward as a team member. The trainee’s performance is assessed by three trained consultants. Consultants recruited as assessors must be familiar with the performance and assessment of foundation trainee doctors within clinical practice. Prior to undertaking their first assessment, consultants must also at-
Box 2. Preparation for the Postgraduate Ward Simulation Exercise (PgWSE) Each trainee is sent a link to the PgWSE website that includes the following information: • A sample 10-minute video of a PgWSE. • A video presentation of all the equipment, resources and documentation used in the simulated ward. • An information booklet, which can be printed off for reference.
tend a preparatory training day for assessors. The process of assessing performance during a PgWSE has two key components. • Each assessor conducts their own assessment of the trainee’s performance. • The assessment panel then makes a consensus judgement regarding the trainee’s performance, including making a judgement of pass or fail. Bespoke assessment tools were developed for the PgWSE that have been shown to have good reliability (Cronbach’a a = 0.817). Assessors directly observe the trainee’s performance via a live link to a dedicated viewing room. During the PgWSE the assessors complete the individual assessment of the performance tool (Appendix 1), commenting on the trainee’s behaviours, strengths and any areas that require improvement in open-text boxes. These comments then inform scores applied across 10 domains. Each domain reflects the requirements of Good Medical Practice,7 and uses a five-point Likert scale. The process of documentation and categorisation (scoring) then informs the pass or fail judgment. Self-assessment of PgWSE by trainee (30 minutes) The self-assessment of performance is used to disengage trainees from the simulated
activity. Trainees view a video recording of their PgWSE, and conduct a self-assessment using a standardised tool (Appendix 2), which mirrors the domains used by the assessment panel. The process of self-assessment is designed to encourage trainees to reflect and develop insight into their actual performance,8 in preparation for constructive feedback. The documented evidence from the self-assessment is shared with the assessment panel prior to meeting the trainee. This process allows the panel to determine the level of insight a trainee has into their performance, and allows them to tailor feedback to each individual.
Trainees can access a dedicated internet site that has preparatory information
Trainee receives feedback on their performance (30 minutes) On completing their self-assessment the trainee meets with the assessment panel. This constructive feedback focuses solely on the trainee’s performance during the PgWSE, and is structured using stages 1–6 of Pendleton’s guidelines, as follows.9 1. Check that the learner wants and is ready for feedback. 2. Let the learner give comments and background to the material that is being assessed. 3. The learner states what was done well. 4. The observer(s) states what was done well. 5. The learner states what could be improved. 6. The observer(s) states how it could be improved. This structured feedback process is informed by the consensus judgement of the assessment panel and the trainee’s selfassessment. The feedback session is designed to highlight strengths and deficiencies in performance, and promote insight within the trainee. This feedback can also help engage the trainee in a process of further remediation and inform any future action plan.
Blackwell Publishing Ltd 2012. THE CLINICAL TEACHER 2012; 9: 285–289 287
Selfassessment is designed to encourage trainees to reflect and develop insight
Box 3. Resource and training requirements for the Postgraduate Ward Simulation Exercise (PgWSE) • A qualified suitably experienced nurse is recruited for each PgWSE. • Three or four simulated patients (SPs) are required for each PgWSE. SPs must attend a preparatory training session prior to participating in the PgWSE. • A senior registrar with relevant clinical experience is recruited to be the trainee’s senior colleague. The registrar must attend a training session prior to the exercise. • Three trained consultants are recruited to be assessors for each PgWSE. Assessor activity is monitored to ensure the consistency and accuracy of the assessment process.
IMPLICATIONS The PgWSE has been shown to be a valuable addition to the management of doctors in difficulty in Scotland. Since 2008, the national advisory group has received 38 trainee referrals from the four Scottish deaneries: nine in 2008, 13 in 2009 and 16 in 2010. Currently the PgWSE is scheduled to run every 4 weeks and provides the opportunity for two trainees to be assessed during each session. Twenty-five consultants are currently active assessors. Assessors are recruited through the national advisory group and the Supporting Clinicians On Training in Scotland (SCOTS) faculty group. Assessors have to undertake two PgWSE assessments per annum and attend an annual update. For each PgWSE, the assessment panel is intentionally composed from a range of medical specialities and, where achievable, localities, so as to avoid speciality or geographical bias. The PgWSE is time and resource intensive (Box 3).
The value of the PgWSE cannot be underestimated, and is reflected in the increasing uptake of this resource. The PgWSE has provided objective, detailed and comprehensive assessment and feedback to the referring deanery, and has been used to inform decisions relating to further remediation or the continuation of training. There has been continued engagement and support for the PgWSE from the four deaneries in Scotland, ensuring that it is maintained as a longterm resource to support trainee doctors in difficulty. The PgWSE is an innovative use of simulation. Remediation workshops focusing on the development of situational awareness, clinical decision making and prioritisation, safer prescribing, safe communication and team communication are currently being developed as a result of the findings of this educational intervention. The PgWSE management group are currently working with a statistician in conducting data analysis to identify the reliability of the assessment process and the internal consistency of the individual and consensus judgement
made by the members of the assessment panel. REFERENCES 1. NHS Education for Scotland. Postgraduate Medical Education in Scotland: Management of Trainee Doctors in Difficulty. Operational Framework. NHS Education for Scotland; Dundee 2008. 2. Anderson F, Cachia P, Monie R, Connacher A. Supporting trainees in difficulty: a new approach for Scotland. Scottish Medical Journal 2011;56:72–75. 3. Ker JS, Hesketh EA, Anderson F, Johnston DA. Can a ward simulation exercise achieve the realism that reflects the complexity of everyday practice junior doctors encounter? Med Teach 2006;28:330–334. 4. The UK Foundation Programme Office. The foundation programme: about the programme. Available at http://www.foundationprogramme. nhs.uk/pages/home/about-thefoundation-programme. Accessed on 6 July 2011. 5. Ziv A, Ben-David S, Ziv M. Simulation Based Medical Education: an opportunity to learn from errors. Med Teach 2005;27:193–199. 6. The UK Foundation Programme. The Foundation Programme Curriculum: March 2010. 2010. Available at http://www.foundationprogramme. nhs.uk/pages/home/training-andassessment Accessed on 6 July 2011. 7. General Medical Council. Good Medical Practice. Available at http:// www.gmc-uk.org/guidance/ good_medical_practice.asp. Accessed on 6 July 2011. 8. Hixon J, Swann B. When Does Introspection Bear Fruit? SelfReflection, Self-Insight, and Interpersonal Choices. Journal of Personality and Social Psychology 1993;64:35–43. 9. Pendleton D, Schofield T, Tate P. The Consultation: An approach to learning and teaching. Oxford: Oxford University Press, 1984.
Corresponding author’s contact details: Kevin Stirling, Lecturer in Simulation, Clinical Skills Centre, University of Dundee, Ninewells Hospital and Medical School, Dundee, DD1 9SY, UK. E-mail:
[email protected]
Funding: None. Conflict of interest: None. Ethical approval: Ethical approval was not necessary because this study did not involve research on human subjects. doi: 10.1111/j.1743-498X.2012.00541.x
288 Blackwell Publishing Ltd 2012. THE CLINICAL TEACHER 2012; 9: 285–289
The PgWSE has provided objective, detailed and comprehensive assessment and feedback
Appendix 1. The Postgraduate Ward Simulation Exercise (PgWSE) assessment tool
Appendix 2. The Postgraduate Ward Simulation Exercise (PgWSE) self-assessment tool
Blackwell Publishing Ltd 2012. THE CLINICAL TEACHER 2012; 9: 285–289 289
