+
Developing and implementing Entrustable Professional Activities (EPAs): The significance of context
AMEE 2016 Barcelona 1ST World Summit on Competency Based Education
Josephine Boland, Mark Knox, Dara Byrne
J. Boland et al 2016
+ Disclosure No conflicts of interest
Presenter:
In this presentation we offer our personal perspectives.
Dr Josephine Boland
J. Boland et al 2016
+
Acknowledgements
n
Medical Council
n
National Doctors Training and Planning (NDTP) Development Funding (Health Service Executive)
n
College of Anaesthetists of Ireland
n
Faculty of Radiologists of Ireland, Royal College of Surgeons
Funding Support Collaboration
J. Boland et al 2016
+ Josephine Boland (CAI) Mark Knox (FoR, RCPI) Dara Byrne (NIP)
College of Anaesthetists of Ireland
Faculty of Radiologists of Ireland, Royal College of Surgeona
National Internship Programme
J. Boland et al 2016
+
n
Context
n
Entrustable Professional Activities
n
EPA process n
Proficiency levels and milestones
n
Assessment Basis for entrustment
n
Outline
n
Language n ‘assessment’ n
‘trainee response’ vs ‘reflection’
n
Consultation and validation
n
Implementation
J. Boland et al 2016
+
Postgraduate Specialist Training (Irl)
Context Internship
Medical Council
Medical School Ottawa Conference 2016 J Boland et al
+
Regulatory Framework
n
Postgraduate specialist training bodies (13) responsible for: n n
n
Standard of delivery Assessment
n
Certification
n
n
Setting standards Curriculum design
Medical Council. n
Set and monitor the standards of bodies and programmes Assess delivery of training to accreditation standards
n
Issue recommendations to improve the standard of training.
n
Ottawa Conference 2016 J Boland et al
+Medical Councial 8 Domains of Good Professional Practice
Competency Frameworks for Postgraduate Training
Medical Council
Ottawa Conference 2016 J Boland et al
+ Elaboration by Medical Council:
Medical Ottawa Conference 2016Council J Boland et al
Drivers and Policy Context
Ottawa Conference 2016 J Boland et al
Drivers and Policy Context
Need for Evidence of Competence Ottawa Conference 2016 J Boland et al
+Entrustable
Professional Activities
A Framework for Implementing
J. Boland et al 2016
+
Entrustable Professional Activity A core unit of professional practice that can be fully entrusted to a trainee, as soon as he or she has demonstrated the necessary competence to execute the activity unsupervised. ten Cate, O. et al (2015)
. Independently executable Multiple Domains
CAI Annual Congress 2016 J Boland.
Assessable competences
+
EPAs across the continuum
AAMC (2014)
Internship
EPAs Medical Council 2016 J Boland.
+
EPAs across the continuum
AAMC (2014)
Internship
Ireland EPAs Medical Council 2016 J Boland.
+
EPAs: Components
Description
CAI Annual Congress 2016 J Boland.
Competencies
Proficiency level
Milestones
Assessment
+
EPAs: Focus for today
Description
CAI Annual Congress 2016 J Boland.
Competencies
Proficiency level
Milestones
Assessment
+ The EPAs Bespoke Templates
J. Boland et al 2016
+
EPAs for End of Internship:
J. Boland et al 2016
+
EPAs for Anaesthesia Re-framing of Training ‘Modules’
J. Boland et al 2016
+
EPAs for Radiology
Core
SubSpecialty
J. Boland et al 2016
+
Bespoke Templates: 1. Internship
J. Boland et al 2016
+
2. Specialty Training
Version 17 !!!!!
J. Boland et al 2016
+ Establishing Milestones Proficiency Levels
J. Boland et al 2016
+
Milestones
Specialty Training Anaesthesia and Radiology
+
Milestones
Specialty Training Anaesthesia
Anaesthesia: Decoupling milestones from year of training
+
Milestones
Specialty Training Radiology
Radiology: Milestones and year of training more closely aligned
+
Proficiency levels Internship
Internshsip: Over one year
+
Specialty Training: 5-6 years
J. Boland et al 2016
+
Internship: 1 year Translation
J. Boland et al 2016
+ Entrustment Supervision Assessment
J. Boland et al 2016
+
Significance of Context n
What does ‘informal backstage supervision’ mean? n
n n
n
Internship n ‘I was on site and available, just in case’ Anaesthesia ? Radiology ?
Who is providing the supervision? n
n
n
Internship n Direct observation by consultant is not common Radiology: n Close direct supervision by consultant is routine Anaesthesia n Consultant could be covering more than one theatre
J. Boland et al 2016
+
Significance of Context n
What does ‘informal backstage supervision’ mean? n
n n
n
Internship n ‘I was on site and available, just in case’ Anaesthesia ? Radiology ?
Who is providing the supervision? n
n
n
Internship n Direct observation by consultant is not common Radiology: n Close direct supervision by consultant is routine Anaesthesia n Consultant could be covering more than one theatre
J. Boland et al 2016
+
Assessment Focus on Feedback
Purpose vs Stakes
‘Work based assessment’ Summative/High stakes Performance assessment
vs Formative/Low stakes
Observation and review tools
Feedback reports
J. Boland et al 2016
A continnum
+
‘Obseration and Review’: Internship
Direct Observation of Procedural Skill (DOPS)
Case Presentation (CP)
Reflective Journal
Case Based Discussion (CBD)
Team Review (TR)
+
‘Feedback Reports’ in Anaesthesia
Fieldnotes
Direct Observation of Procedural Skill (DOPS)
Case Based Discussion
+
‘Observed Practice’ in Radiology
Image interpretation and reporting exercise (IIREx)
Case Based Discussion
Direct Observation of Procedural Skill (DOPS)
Multi-source feedback
+
Choices:
n
A ‘toolkit’ ? n
n
Generic tools? n same DOPS for all EPAs?
n
EPA –specific tools? n with competencies generated from a blueprinting process
from which assessors select tools for an EPA?
Prescribed tools? n
n
for EPAs and for specific competencies
J. Boland et al 2016
+
Assessment strategy A range of tools (e.g. Internship)
J. Boland et al 2016
+
Requirements specified for each EPA
J. Boland et al 2016
+
Criteria for Entrustment Defined no. and range of evidence Guide volume of practice Specially Training: (e.g. Anaesthesia) :
J. Boland et al 2016
+
Specialist Traininig Blueprinting competenies and tools
EPA specific tools
J. Boland et al 2016
+ EPA Data Management System
EPA specific tools
CAI Annual Congress 2016 J Boland.
+
Implementation
J. Boland et al 2016
Implementation Process The Workplace Tools
J. Boland et al 2016
Implementation Process The Workplace Tools
January 2017 ‘Just start!’
J. Boland et al 2016
+
Common strategy: Focus on global judgment
+
Common strategy: Focus on global judgment
+
Common strategy: Focus on global judgment
‘Trainee Response’ ‘Reflection’
+
Supporting effective feedback
J. Boland et al 2016
+
Ongoing Consultation and Validation n
Internship n n n n n n
Medical Council National Intern Networks Medical Schools
Benchmarking
‘EPAs for Entering Residency’, Association of American Medical Colleges (AMMC)
Postgraduate Forum Trainee doctors Interns
Specialist Training (e.g. Anaesthesia) Competency Framework Working Group Committee of Anaesthesia Trainees Consultants Tutors International collaboration: Canada, UK and Netherlands
CAI Annual Congress 2016 J Boland.
+
Ongoing Consultation and Validation n
Internship n n n n n n
n
Medical Council National Intern Networks Medical Schools
Benchmarking
‘EPAs for Entering Residency’, Association of American Medical Colleges (AMMC)
Postgraduate Forum Trainee doctors Interns
Specialist Training (e.g. Anaesthesia) n n n n n
CAI Competency Framework Working Group Committee of Anaesthesia Trainees Consultants Tutors International collaboration: Canada, UK and Netherlands
CAI Annual Congress 2016 J Boland.
+
Early Days: n
Feasibility of the process
n
Usability of the forms
n
Clarity of the guidelines
n
Meaning of ‘entrustability’
n
Technological requirements (e.g. Mobile App)
n
Supporting changes in culture and practice
n
More later……..
J. Boland et al 2016
+
Conclusions Significance of context - National - Programme - Specialty
n
National context – scale
n
EPA Methodology n
bespoke
n
grounded
n
Moving from ‘years’ to ‘milestones’
n
Concept of ‘entrustability’
n
Language of assessment
n
Stakeholder involvement
J. Boland et al 2016
+
References (2/2) n
Association of American Medical College (2014) Core Entrustable Activities for Entering Residency: Curriculum Developers’ Guide. Washington DC: AAMC
n
Boland J (2016) “Entrustable Professional Activities: Placing trust at the centre of education, training and assessment in anaesthesia” CAI Annual Congress, UCD, Dublin, May 2016
n
Boland, J, O’ Connor, P. Offiah, G. and Byrne, D. (2015) ‘Draft Framework of Outcomes for Intern Training in Ireland’ Dublin: Medical Council: https://www.medicalcouncil.ie/Education/Career-Stage-Intern/Quality-Assurance/Consultation-on-InternYear-/4_-Framework-of-Outcomes.pdf
n
Boland, J, P. Offiah, G. and Byrne, D. (2016) ‘Framework of Outcomes for Intern Training in Ireland: Entrustable Professional Activities: Post consultation process’ Dublin: Medical Council.
n
College of Anaesthetists of Ireland (2015) Draft EPA Template, developed under Creative Commons License)
n
Grofton, W. Dudek, N, Wood, T. Balaa, F, and Hamstra, S (2012) The Ottawa Surgical Competency Operating Room Evaluation (O-SCORE): A tool to assess surgical competence Academic Medicine 87 (10): 1401-1407
n
Harrison C, and Val Wass (2016). "The challenge of changing to an assessment for learning culture." Medical Education 50 (7) pp704-706.
n
Medical Council (2014) Your Training Counts Report: Results of the National trainee Experience Survey Dublin: Medical Council
J. Boland et al 2016
+
References (2/2) n
Medical Council (2015) Doctors’ Education, Training and Lifelong Learning in 21st Century Ireland Dublin: Medical Council
n
Rashid, P. (2015). Entrustable professional activities: Time to be trusted? ANZ Journal of Surgery, 85(5), 298-299. doi: 10.1111/ans.12946
n
Ross S, Poth CN, Donoff M, Humphries P, Steiner I, Schipper S, Janke F, Nichols D. (2011) "Competency-Based Achievement System Using formative feedback to teach and assess family medicine residents’ skills. "Canadian Family Physician 57 (9) e323-e330.
n
Subha R and Krackov SK (2013) "Twelve tips for giving feedback effectively in the clinical environment." Medical teacher 34(10) pp 787-791.
n
ten Cate O, Chen HC, Hoff RG, Peters H, Bok H & van Der Schaaf M. (2015). Curriculum development for the workplace using Entrustable Professional Activities (EPAs): AMEE Guide No. 99. Medical Teacher, 1-20. doi: 10.3109/0142159X.2015.1060308
n
ten Cate, O. (2014) The patient handover as an entrustable professional activity: adding meaning in teaching and practice BMJ Qual Saf 2012; 21:i9–i12. doi:10.1136/bmjqs-2012-001213
n
Voyer, S, and Hatala, R. (2015) "Debriefing and feedback: two sides of the same coin?." Simulation in Healthcare 10 (2) pp 67-68.
J. Boland et al 2016
+
Developing and implementing Entrustable Professional Activities (EPAs): The significance of context Contact details:
AMEE 2016 Barcelona 1ST World Summit on Competency Based Education
Dr. Josephine Boland:
[email protected] Dr. Mark Knox:
[email protected] Dr. Dara Byrne:
[email protected]
J. Boland et al 2016