Developing and implementing Entrustable ...

6 downloads 0 Views 29MB Size Report
Josephine Boland, Mark Knox, Dara Byrne. Developing ... College of Anaesthetists of Ireland. ▫ Faculty of Radiologists of Ireland, Royal. College of Surgeons.
+

Developing and implementing Entrustable Professional Activities (EPAs): The significance of context    

AMEE 2016 Barcelona 1ST World Summit on Competency Based Education

Josephine Boland, Mark Knox, Dara Byrne

J. Boland et al 2016

+ Disclosure No conflicts of interest

Presenter:

In this presentation we offer our personal perspectives.

Dr Josephine Boland

J. Boland et al 2016

+

Acknowledgements

n 

Medical Council

n 

National Doctors Training and Planning (NDTP) Development Funding (Health Service Executive)

n 

College of Anaesthetists of Ireland

n 

Faculty of Radiologists of Ireland, Royal College of Surgeons

Funding Support Collaboration

J. Boland et al 2016

+ Josephine Boland (CAI) Mark Knox (FoR, RCPI) Dara Byrne (NIP)

College of Anaesthetists of Ireland

Faculty of Radiologists of Ireland, Royal College of Surgeona

National Internship Programme

J. Boland et al 2016

+

n 

Context

n 

Entrustable Professional Activities

n 

EPA process n 

Proficiency levels and milestones

n 

Assessment Basis for entrustment

n 

Outline

n 

Language n  ‘assessment’ n 

‘trainee response’ vs ‘reflection’

n 

Consultation and validation

n 

Implementation

J. Boland et al 2016

+

Postgraduate Specialist Training (Irl)

Context Internship

Medical Council

Medical School Ottawa Conference 2016 J Boland et al

+

Regulatory Framework

n 

Postgraduate specialist training bodies (13) responsible for: n  n 

n 

Standard of delivery Assessment

n 

Certification

n 

n 

Setting standards Curriculum design

Medical Council. n 

Set and monitor the standards of bodies and programmes Assess delivery of training to  accreditation standards

n 

Issue recommendations to improve the standard of training. 

n 

Ottawa Conference 2016 J Boland et al

+Medical Councial 8 Domains of Good Professional Practice

Competency Frameworks for Postgraduate Training

Medical Council

Ottawa Conference 2016 J Boland et al

+ Elaboration by Medical Council:

Medical Ottawa Conference 2016Council J Boland et al

Drivers and Policy Context

Ottawa Conference 2016 J Boland et al

Drivers and Policy Context

Need for Evidence of Competence Ottawa Conference 2016 J Boland et al

+Entrustable

Professional Activities

A Framework for Implementing

J. Boland et al 2016

+

Entrustable Professional Activity A core unit of professional practice that can be fully entrusted to a trainee, as soon as he or she has demonstrated the necessary competence to execute the activity unsupervised. ten Cate, O. et al (2015)

. Independently executable Multiple Domains

CAI Annual Congress 2016 J Boland.

Assessable competences

+

EPAs across the continuum

AAMC (2014)

Internship

EPAs Medical Council 2016 J Boland.

+

EPAs across the continuum

AAMC (2014)

Internship

Ireland EPAs Medical Council 2016 J Boland.

+

EPAs: Components

Description

CAI Annual Congress 2016 J Boland.

Competencies

Proficiency level

Milestones

Assessment

+

EPAs: Focus for today

Description

CAI Annual Congress 2016 J Boland.

Competencies

Proficiency level

Milestones

Assessment

+ The EPAs Bespoke Templates

J. Boland et al 2016

+

EPAs for End of Internship:

J. Boland et al 2016

+

EPAs for Anaesthesia Re-framing of Training ‘Modules’

J. Boland et al 2016

+

EPAs for Radiology

Core

SubSpecialty

J. Boland et al 2016

+

Bespoke Templates: 1. Internship

J. Boland et al 2016

+

2. Specialty Training

Version 17 !!!!!

J. Boland et al 2016

+ Establishing Milestones Proficiency Levels

J. Boland et al 2016

+

Milestones

Specialty Training Anaesthesia and Radiology

+

Milestones

Specialty Training Anaesthesia

Anaesthesia: Decoupling milestones from year of training

+

Milestones

Specialty Training Radiology

Radiology: Milestones and year of training more closely aligned

+

Proficiency levels Internship

Internshsip: Over one year

+

Specialty Training: 5-6 years

J. Boland et al 2016

+

Internship: 1 year Translation

J. Boland et al 2016

+ Entrustment Supervision Assessment

J. Boland et al 2016

+

Significance of Context n 

What does ‘informal backstage supervision’ mean? n 

n  n 

n 

Internship n  ‘I was on site and available, just in case’ Anaesthesia ? Radiology ?

Who is providing the supervision? n 

n 

n 

Internship n  Direct observation by consultant is not common Radiology: n  Close direct supervision by consultant is routine Anaesthesia n  Consultant could be covering more than one theatre

J. Boland et al 2016

+

Significance of Context n 

What does ‘informal backstage supervision’ mean? n 

n  n 

n 

Internship n  ‘I was on site and available, just in case’ Anaesthesia ? Radiology ?

Who is providing the supervision? n 

n 

n 

Internship n  Direct observation by consultant is not common Radiology: n  Close direct supervision by consultant is routine Anaesthesia n  Consultant could be covering more than one theatre

J. Boland et al 2016

+

Assessment Focus on Feedback

Purpose vs Stakes

‘Work based assessment’ Summative/High stakes Performance assessment

vs Formative/Low stakes

Observation and review tools

Feedback reports

J. Boland et al 2016

A continnum

+

‘Obseration and Review’: Internship

Direct Observation of Procedural Skill (DOPS)

Case Presentation (CP)

Reflective Journal

Case Based Discussion (CBD)

Team Review (TR)

+

‘Feedback Reports’ in Anaesthesia

Fieldnotes

Direct Observation of Procedural Skill (DOPS)

Case Based Discussion

+

‘Observed Practice’ in Radiology

Image interpretation and reporting exercise (IIREx)

Case Based Discussion

Direct Observation of Procedural Skill (DOPS)

Multi-source feedback

+

Choices:

n 

A ‘toolkit’ ? n 

n 

Generic tools? n  same DOPS for all EPAs?

n 

EPA –specific tools? n  with competencies generated from a blueprinting process

from which assessors select tools for an EPA?

Prescribed tools? n 

n 

for EPAs and for specific competencies

J. Boland et al 2016

+

Assessment strategy A range of tools (e.g. Internship)

J. Boland et al 2016

+

Requirements specified for each EPA

J. Boland et al 2016

+

Criteria for Entrustment Defined no. and range of evidence Guide volume of practice Specially Training: (e.g. Anaesthesia) :

J. Boland et al 2016

+

Specialist Traininig Blueprinting competenies and tools

EPA specific tools

J. Boland et al 2016

+ EPA Data Management System

EPA specific tools

CAI Annual Congress 2016 J Boland.

+

Implementation

J. Boland et al 2016

Implementation Process The Workplace Tools

J. Boland et al 2016

Implementation Process The Workplace Tools

January 2017 ‘Just start!’

J. Boland et al 2016

+

Common strategy: Focus on global judgment

+

Common strategy: Focus on global judgment

+

Common strategy: Focus on global judgment

‘Trainee Response’ ‘Reflection’

+

Supporting effective feedback

J. Boland et al 2016

+

Ongoing Consultation and Validation n 

Internship n  n  n  n  n  n 

Medical Council National Intern Networks Medical Schools

Benchmarking

‘EPAs for Entering Residency’, Association of American Medical Colleges (AMMC)

Postgraduate Forum Trainee doctors Interns

Specialist Training (e.g. Anaesthesia) Competency Framework Working Group Committee of Anaesthesia Trainees Consultants Tutors International collaboration: Canada, UK and Netherlands

CAI Annual Congress 2016 J Boland.

+

Ongoing Consultation and Validation n 

Internship n  n  n  n  n  n 

n 

Medical Council National Intern Networks Medical Schools

Benchmarking

‘EPAs for Entering Residency’, Association of American Medical Colleges (AMMC)

Postgraduate Forum Trainee doctors Interns

Specialist Training (e.g. Anaesthesia) n  n  n  n  n 

CAI Competency Framework Working Group Committee of Anaesthesia Trainees Consultants Tutors International collaboration: Canada, UK and Netherlands

CAI Annual Congress 2016 J Boland.

+

Early Days: n 

Feasibility of the process

n 

Usability of the forms

n 

Clarity of the guidelines

n 

Meaning of ‘entrustability’

n 

Technological requirements (e.g. Mobile App)

n 

Supporting changes in culture and practice

n 

More later……..

J. Boland et al 2016

+

Conclusions Significance of context - National - Programme - Specialty

n 

National context – scale

n 

EPA Methodology n 

bespoke

n 

grounded

n 

Moving from ‘years’ to ‘milestones’

n 

Concept of ‘entrustability’

n 

Language of assessment

n 

Stakeholder involvement

J. Boland et al 2016

+

References (2/2) n 

Association of American Medical College (2014) Core Entrustable Activities for Entering Residency: Curriculum Developers’ Guide. Washington DC: AAMC

n 

Boland J (2016) “Entrustable Professional Activities: Placing trust at the centre of education, training and assessment in anaesthesia” CAI Annual Congress, UCD, Dublin, May 2016

n 

Boland, J, O’ Connor, P. Offiah, G. and Byrne, D. (2015) ‘Draft Framework of Outcomes for Intern Training in Ireland’ Dublin: Medical Council: https://www.medicalcouncil.ie/Education/Career-Stage-Intern/Quality-Assurance/Consultation-on-InternYear-/4_-Framework-of-Outcomes.pdf

n 

Boland, J, P. Offiah, G. and Byrne, D. (2016) ‘Framework of Outcomes for Intern Training in Ireland: Entrustable Professional Activities: Post consultation process’ Dublin: Medical Council.

n 

College of Anaesthetists of Ireland (2015) Draft EPA Template, developed under Creative Commons License)

n 

Grofton, W. Dudek, N, Wood, T. Balaa, F, and Hamstra, S (2012) The Ottawa Surgical Competency Operating Room Evaluation (O-SCORE): A tool to assess surgical competence Academic Medicine 87 (10): 1401-1407

n 

Harrison C, and Val Wass (2016). "The challenge of changing to an assessment for learning culture." Medical Education 50 (7) pp704-706.

n 

Medical Council (2014) Your Training Counts Report: Results of the National trainee Experience Survey Dublin: Medical Council

J. Boland et al 2016

+

References (2/2) n 

Medical Council (2015) Doctors’ Education, Training and Lifelong Learning in 21st Century Ireland Dublin: Medical Council

n 

Rashid, P. (2015). Entrustable professional activities: Time to be trusted? ANZ Journal of Surgery, 85(5), 298-299. doi: 10.1111/ans.12946

n 

Ross S, Poth CN, Donoff M, Humphries P, Steiner I, Schipper S, Janke F, Nichols D. (2011) "Competency-Based Achievement System Using formative feedback to teach and assess family medicine residents’ skills. "Canadian Family Physician 57 (9) e323-e330.

n 

Subha R and Krackov SK (2013) "Twelve tips for giving feedback effectively in the clinical environment." Medical teacher 34(10) pp 787-791.

n 

ten Cate O, Chen HC, Hoff RG, Peters H, Bok H & van Der Schaaf M. (2015). Curriculum development for the workplace using Entrustable Professional Activities (EPAs): AMEE Guide No. 99. Medical Teacher, 1-20. doi: 10.3109/0142159X.2015.1060308

n 

ten Cate, O. (2014) The patient handover as an entrustable professional activity: adding meaning in teaching and practice BMJ Qual Saf 2012; 21:i9–i12. doi:10.1136/bmjqs-2012-001213

n 

Voyer, S, and Hatala, R. (2015) "Debriefing and feedback: two sides of the same coin?." Simulation in Healthcare 10 (2) pp 67-68.

J. Boland et al 2016

+

Developing and implementing Entrustable Professional Activities (EPAs): The significance of context   Contact details:  

AMEE 2016 Barcelona 1ST World Summit on Competency Based Education

Dr. Josephine Boland: [email protected] Dr. Mark Knox: [email protected] Dr. Dara Byrne: [email protected]

J. Boland et al 2016