Developing and implementing Entrustable ...

+

Developing and implementing Entrustable Professional Activities (EPAs): The significance of context

AMEE 2016 Barcelona 1ST World Summit on Competency Based Education

Josephine Boland, Mark Knox, Dara Byrne

J. Boland et al 2016

+ Disclosure No conflicts of interest

Presenter:

In this presentation we offer our personal perspectives.

Dr Josephine Boland


+

Acknowledgements

n 

Medical Council

n 

National Doctors Training and Planning (NDTP) Development Funding (Health Service Executive)

n 

College of Anaesthetists of Ireland

n 

Faculty of Radiologists of Ireland, Royal College of Surgeons

Funding Support Collaboration


+ Josephine Boland (CAI) Mark Knox (FoR, RCPI) Dara Byrne (NIP)

College of Anaesthetists of Ireland

Faculty of Radiologists of Ireland, Royal College of Surgeona

National Internship Programme


+

n 

Context

n 

Entrustable Professional Activities

n 

EPA process n 

Proficiency levels and milestones

n 

Assessment Basis for entrustment

n 

Outline

n 

Language n  ‘assessment’ n 

‘trainee response’ vs ‘reflection’

n 

Consultation and validation

n 

Implementation


+

Postgraduate Specialist Training (Irl)

Context Internship

Medical Council

Medical School Ottawa Conference 2016 J Boland et al

+

Regulatory Framework

n 

Postgraduate specialist training bodies (13) responsible for: n  n 

n 

Standard of delivery Assessment

n 

Certification

n 

n 

Setting standards Curriculum design

Medical Council. n 

Set and monitor the standards of bodies and programmes Assess delivery of training to accreditation standards

n 

Issue recommendations to improve the standard of training.

n 

Ottawa Conference 2016 J Boland et al

+Medical Councial 8 Domains of Good Professional Practice

Competency Frameworks for Postgraduate Training

Medical Council


+ Elaboration by Medical Council:

Medical Ottawa Conference 2016Council J Boland et al

Drivers and Policy Context


Drivers and Policy Context

Need for Evidence of Competence Ottawa Conference 2016 J Boland et al

+Entrustable

Professional Activities

A Framework for Implementing


+

Entrustable Professional Activity A core unit of professional practice that can be fully entrusted to a trainee, as soon as he or she has demonstrated the necessary competence to execute the activity unsupervised. ten Cate, O. et al (2015)

. Independently executable Multiple Domains

CAI Annual Congress 2016 J Boland.

Assessable competences

+

EPAs across the continuum

AAMC (2014)

Internship

EPAs Medical Council 2016 J Boland.

+

EPAs across the continuum

AAMC (2014)

Internship

Ireland EPAs Medical Council 2016 J Boland.

+

EPAs: Components

Description


Competencies

Proficiency level

Milestones

Assessment

+

EPAs: Focus for today

Description


Competencies

Proficiency level

Milestones

Assessment

+ The EPAs Bespoke Templates


+

EPAs for End of Internship:


+

EPAs for Anaesthesia Re-framing of Training ‘Modules’


+

EPAs for Radiology

Core

SubSpecialty


+

Bespoke Templates: 1. Internship


+

2. Specialty Training

Version 17 !!!!!


+ Establishing Milestones Proficiency Levels


+

Milestones

Specialty Training Anaesthesia and Radiology

+

Milestones

Specialty Training Anaesthesia

Anaesthesia: Decoupling milestones from year of training

+

Milestones

Specialty Training Radiology

Radiology: Milestones and year of training more closely aligned

+

Proficiency levels Internship

Internshsip: Over one year

+

Specialty Training: 5-6 years


+

Internship: 1 year Translation


+ Entrustment Supervision Assessment


+

Significance of Context n 

What does ‘informal backstage supervision’ mean? n 

n  n 

n 

Internship n  ‘I was on site and available, just in case’ Anaesthesia ? Radiology ?

Who is providing the supervision? n 

n 

n 

Internship n  Direct observation by consultant is not common Radiology: n  Close direct supervision by consultant is routine Anaesthesia n  Consultant could be covering more than one theatre


+

Significance of Context n 

What does ‘informal backstage supervision’ mean? n 

n  n 

n 

Internship n  ‘I was on site and available, just in case’ Anaesthesia ? Radiology ?

Who is providing the supervision? n 

n 

n 

Internship n  Direct observation by consultant is not common Radiology: n  Close direct supervision by consultant is routine Anaesthesia n  Consultant could be covering more than one theatre


+

Assessment Focus on Feedback

Purpose vs Stakes

‘Work based assessment’ Summative/High stakes Performance assessment

vs Formative/Low stakes

Observation and review tools

Feedback reports


A continnum

+

‘Obseration and Review’: Internship

Direct Observation of Procedural Skill (DOPS)

Case Presentation (CP)

Reflective Journal

Case Based Discussion (CBD)

Team Review (TR)

+

‘Feedback Reports’ in Anaesthesia

Fieldnotes


Case Based Discussion

+

‘Observed Practice’ in Radiology

Image interpretation and reporting exercise (IIREx)

Case Based Discussion


Multi-source feedback

+

Choices:

n 

A ‘toolkit’ ? n 

n 

Generic tools? n  same DOPS for all EPAs?

n 

EPA –specific tools? n  with competencies generated from a blueprinting process

from which assessors select tools for an EPA?

Prescribed tools? n 

n 

for EPAs and for specific competencies


+

Assessment strategy A range of tools (e.g. Internship)


+

Requirements specified for each EPA


+

Criteria for Entrustment Defined no. and range of evidence Guide volume of practice Specially Training: (e.g. Anaesthesia) :


+

Specialist Traininig Blueprinting competenies and tools

EPA specific tools


+ EPA Data Management System

EPA specific tools


+

Implementation


Implementation Process The Workplace Tools


Implementation Process The Workplace Tools

January 2017 ‘Just start!’


+

Common strategy: Focus on global judgment

+


+


‘Trainee Response’ ‘Reflection’

+

Supporting effective feedback


+

Ongoing Consultation and Validation n 

Internship n  n  n  n  n  n 

Medical Council National Intern Networks Medical Schools

Benchmarking

‘EPAs for Entering Residency’, Association of American Medical Colleges (AMMC)

Postgraduate Forum Trainee doctors Interns

Specialist Training (e.g. Anaesthesia) Competency Framework Working Group Committee of Anaesthesia Trainees Consultants Tutors International collaboration: Canada, UK and Netherlands


+

Ongoing Consultation and Validation n 

Internship n  n  n  n  n  n 

n 

Medical Council National Intern Networks Medical Schools

Benchmarking

‘EPAs for Entering Residency’, Association of American Medical Colleges (AMMC)

Postgraduate Forum Trainee doctors Interns

Specialist Training (e.g. Anaesthesia) n  n  n  n  n 

CAI Competency Framework Working Group Committee of Anaesthesia Trainees Consultants Tutors International collaboration: Canada, UK and Netherlands


+

Early Days: n 

Feasibility of the process

n 

Usability of the forms

n 

Clarity of the guidelines

n 

Meaning of ‘entrustability’

n 

Technological requirements (e.g. Mobile App)

n 

Supporting changes in culture and practice

n 

More later……..


+

Conclusions Significance of context - National - Programme - Specialty

n 

National context – scale

n 

EPA Methodology n 

bespoke

n 

grounded

n 

Moving from ‘years’ to ‘milestones’

n 

Concept of ‘entrustability’

n 

Language of assessment

n 

Stakeholder involvement


+

References (2/2) n 

Association of American Medical College (2014) Core Entrustable Activities for Entering Residency: Curriculum Developers’ Guide. Washington DC: AAMC

n 

Boland J (2016) “Entrustable Professional Activities: Placing trust at the centre of education, training and assessment in anaesthesia” CAI Annual Congress, UCD, Dublin, May 2016

n 

Boland, J, O’ Connor, P. Offiah, G. and Byrne, D. (2015) ‘Draft Framework of Outcomes for Intern Training in Ireland’ Dublin: Medical Council: https://www.medicalcouncil.ie/Education/Career-Stage-Intern/Quality-Assurance/Consultation-on-InternYear-/4_-Framework-of-Outcomes.pdf

n 

Boland, J, P. Offiah, G. and Byrne, D. (2016) ‘Framework of Outcomes for Intern Training in Ireland: Entrustable Professional Activities: Post consultation process’ Dublin: Medical Council.

n 

College of Anaesthetists of Ireland (2015) Draft EPA Template, developed under Creative Commons License)

n 

Grofton, W. Dudek, N, Wood, T. Balaa, F, and Hamstra, S (2012) The Ottawa Surgical Competency Operating Room Evaluation (O-SCORE): A tool to assess surgical competence Academic Medicine 87 (10): 1401-1407

n 

Harrison C, and Val Wass (2016). "The challenge of changing to an assessment for learning culture." Medical Education 50 (7) pp704-706.

n 

Medical Council (2014) Your Training Counts Report: Results of the National trainee Experience Survey Dublin: Medical Council


+

References (2/2) n 

Medical Council (2015) Doctors’ Education, Training and Lifelong Learning in 21st Century Ireland Dublin: Medical Council

n 

Rashid, P. (2015). Entrustable professional activities: Time to be trusted? ANZ Journal of Surgery, 85(5), 298-299. doi: 10.1111/ans.12946

n 

Ross S, Poth CN, Donoff M, Humphries P, Steiner I, Schipper S, Janke F, Nichols D. (2011) "Competency-Based Achievement System Using formative feedback to teach and assess family medicine residents’ skills. "Canadian Family Physician 57 (9) e323-e330.

n 

Subha R and Krackov SK (2013) "Twelve tips for giving feedback effectively in the clinical environment." Medical teacher 34(10) pp 787-791.

n 

ten Cate O, Chen HC, Hoff RG, Peters H, Bok H & van Der Schaaf M. (2015). Curriculum development for the workplace using Entrustable Professional Activities (EPAs): AMEE Guide No. 99. Medical Teacher, 1-20. doi: 10.3109/0142159X.2015.1060308

n 

ten Cate, O. (2014) The patient handover as an entrustable professional activity: adding meaning in teaching and practice BMJ Qual Saf 2012; 21:i9–i12. doi:10.1136/bmjqs-2012-001213

n 

Voyer, S, and Hatala, R. (2015) "Debriefing and feedback: two sides of the same coin?." Simulation in Healthcare 10 (2) pp 67-68.


+

Developing and implementing Entrustable Professional Activities (EPAs): The significance of context Contact details:

AMEE 2016 Barcelona 1ST World Summit on Competency Based Education

Dr. Josephine Boland: [email protected] Dr. Mark Knox: [email protected] Dr. Dara Byrne: [email protected]
