Transfusion Medicine Reviews 30 (2016) 30–36
Contents lists available at ScienceDirect
Transfusion Medicine Reviews journal homepage: www.tmreviews.com
Design and Implementation of a Competency-Based Transfusion Medicine Training Program in Canada Michelle P. Zeller a,b,⁎, Jonathan Sherbino a, Lucinda Whitman c, Robert Skeate d,e, Donald M. Arnold a,b a
Department of Medicine, McMaster University, Hamilton, ON, Canada Canadian Blood Services, Ancaster, ON, Canada Department of Medicine, Memorial University, St John's, NL, Canada d Canadian Blood Services, Toronto, ON, Canada e Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, ON, Canada b c
a r t i c l e
i n f o
Available online 10 November 2015 Keywords: Transfusion medicine Medical education Competency-based
a b s t r a c t Transfusion medicine training in Canada is currently undergoing a transformation from a time- and processbased curriculum to a competency-based medical education framework. Transfusion medicine is the first accredited postgraduate medical education training program in Canada to adopt a purely competency-based curriculum. It is serving as an example for a number of other postgraduate medical training programs undergoing a similar transition. The purpose of this review is to highlight the elements of competency-based medical education, describe its application to transfusion medicine training, and report on the development and implementation of the new transfusion medicine curriculum in Canada. © 2015 Elsevier Inc. All rights reserved.
Contents Competency-Based Medical Education . . . . . . . . . . . . . . . . . . . . . . . . Evolution of Medical Education Frameworks . . . . . . . . . . . . . . . . . . . . . . Assessment in CBME . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Does CBME Work? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Applying CBME to Transfusion Medicine . . . . . . . . . . . . . . . . . . . . . . . . Case Study: The Canadian Experience . . . . . . . . . . . . . . . . . . . . . . . . . History of Transfusion Medicine Education in Canada . . . . . . . . . . . . . . . Process of Curricular Development for a CBME Transfusion Medicine Training Program Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conflict of Interest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Competency-based medical education (CBME) is an approach to education that is currently being applied to medical training programs across North America. Competency-based medical education was first described in the 1970s and since that time has evolved significantly in its definition, structure, and delivery [1-3]. The need for a revision in the method of delivering medical education and health professional training was borne out of increasing demand for public accountability, ⁎ Corresponding author: Michelle Zeller, MD, FRCPC, MHPE, HSC 3 V 51B McMaster University, 1280 Main Street W, Hamilton, Ontario, L8S 4K1, Canada. E-mail addresses:
[email protected] (M.P. Zeller),
[email protected] (J. Sherbino),
[email protected] (L. Whitman),
[email protected] (R. Skeate),
[email protected] (D.M. Arnold). http://dx.doi.org/10.1016/j.tmrv.2015.11.001 0887-7963/© 2015 Elsevier Inc. All rights reserved.
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
31 31 31 32 32 33 33 33 35 35 35
heightened focus on patient safety, and limitations of conventional educational approaches [1,3]. Transfusion medicine training is well suited to a competency-based training program and has been at the forefront of this change. As a multifaceted discipline bridging clinical and laboratory medicine, expertise in transfusion medicine requires a broad scope of knowledge and experience. Content material is experiential and requires longitudinal exposures and preceptorships; and key objectives, including the development of managerial and operational leadership skills, require hands-on experiences in the work place. In addition, trainees embarking on a transfusion medicine training program must have completed 1 or more medical training program to be enrolled; thus, they are
M.P. Zeller et al. / Transfusion Medicine Reviews 30 (2016) 30–36
generally self-sufficient, motivated by specific career goals, and familiar with the process of evaluation. Transfusion medicine was the first accredited training program in Canada to adopt a purely competencybased curriculum and currently serves as a model for other postgraduate training programs. The purpose of this narrative review and case study is to provide an overview of CBME, describe the application of CBME to transfusion medicine, and describe recent experience with competency-based transfusion medicine training. Competency-Based Medical Education In health professions education, a competency is defined as an observable ability of a health professional that integrates knowledge, skills, values, and attitudes (Text Box 1) [4]. Competency-based education in general is defined as an outcome-based approach to the design, implementation, assessment, and evaluation of curricula organized by frameworks of competencies [4]. For medical education, this means that curricula are designed to ensure that all competencies required to function as an unsupervised physician are taught, experienced, and assessed rather than focusing on the process (eg, types of patient encounters) and exposure time (eg, no. of patient encounters, length of rotations). Evolution of Medical Education Frameworks Medical education has evolved from apprenticeship models, to traditional assessment models, to a competency-based framework. Apprenticeship models were variable in content, structure, and length and involved one-on-one mentorship. The apprenticeship model evolved into a more merit-based structure, with fixed training lengths, standardized educational content, and mandatory periods of supervised practice [1,5]. After publication of the Flexner report, a study surveying the quality of medical education in the United States and Canada in 1910, prerequisites for medical training were recommended, state regulation of medical licensure was imposed, and increased structure over clinical instruction by medical schools was adopted [2]. During this time, medical instruction evolved to include bedside instruction (led by Sir William Osler) and the incorporation of humanities and patientcentered learning [6,7]. These ideals defined the structure, content, and delivery of medical education for most of the 20th century [8]. In the mid-20th century, nonmedical educational institutions began to shift focus from a purely instructional or didactic process to an outcome-based education, laying the foundation for what would eventually be known as CBME [1]. Outcome-based education is a studentcentered approach to curriculum planning that requires specification of learning outcomes and cohesion (eg, mapping) between curricular content, teaching methods, assessment, learning context, and learning outcomes [9]. Table 1 compares elements of traditional medical training to competency-based medical training. Traditional medical training emphasizes time- and process-based educational programs, whereby trainees passively become competent practitioners [11]. In this approach, the teacher is primarily responsible for the student's knowledge acquisition. A CBME model is more focused on the quality of learning rather than the process and emphasizes individual learner needs. In
Text Box 1“Competency Based Education (CBE) is an outcomes-based approach to the design, implementation, assessment, and evaluation of a medical education program using an organizing framework of competencies.” In CBME, the unit of progression is mastery of specific knowledge, skills, and attitudes and is learner-centered.Frank et al [3]
31
Table 1 Comparison of elements of time- and process-based vs competency-based educational model Elements
Educational goal Responsible for content Assessment tool Assessment timing Evaluation standard Program completion
Educational model Time and process based
Competency based
Knowledge acquisition Teacher Single subjective measure Summativea Norm referencedc Fixed time
Knowledge application Teacher and learner Multiple objective measures Formativeb Criterion referencedd Variable time
Adapted from Carraccio et al [2] and Weinberger et al [10]. a Summative refers to feedback given at the end of a rotation or at the end of a training program. A certification examination provides summative feedback. b Formative refers to feedback that is provided within the context of the learning experience, for example, mid-rotation or during a clinic, as opposed to at the completion of a rotation, or of a program. c Norm referenced refers to comparing learners to each other to determine grading criteria. d Criteria referenced refers to use of a set criteria or standard in determining grading.
addition, both the teacher and learner play active roles in content delivery, with a focus on knowledge application [2]. Over the past 2 decades, national medical training associations and regulators in the United States, Canada, United Kingdom, and Scotland have adopted CBME into medical training programs [4,12,14]. In 1996, before the shift to CBME, the Royal College of Physicians and Surgeons of Canada (RCPSC) adopted an outcome-based framework of competencies called CanMEDS, which describes the core knowledge, skills, and abilities of specialist physicians (Table 2) [15]. The CanMEDS framework of competencies went beyond the Medical Expert Role to include other roles of the physician such as collaborator, manager, and others (Table 2). CanMEDS is currently adopted in multiple jurisdictions with influence in multiple professions [15,16]. Competency-based approach to training was developed more recently to address the best way to teach, learn, and assess the CanMEDS competencies. Reasons for the shift to a competency-based approach included the following: (1) increased demand for public accountability and transparency of medical training curricula to ensure that medical graduates are safe to practice; (2) increased scrutiny of deliverables from increasingly costly graduate medical education programs; (3) emphasis on learners' ability over their knowledge; (4) emphasis on learners' developmental progression through competencies; and (5) the need to provide clear goals of training [4,17]. Assessment in CBME Miller's Pyramid (Fig 1) is a common framework used to illustrate the hierarchy of learning from the most basic level of “knows” through to the top of pyramid, “does” [18]. One of the biggest challenges with CBME is assessment and evaluation. The goals of assessment are to facilitate developmental progression of competence [19]; foster reflective practice, lifelong learning, and critical approach to guided selfassessment [20]; provide motivation and frameworks of knowledge, skills, and professionalism to drive learning [21]; maintain high professional standards and flag students for gaps in knowledge [21]; and guide decisions about student advancement [19,21]. Assessment tools have been developed to measure the “know,” “knows how,” and “shows” with variable degrees of reliability and validity. These tools are highly dependent upon repetitive sampling, content authenticity, context, and standardization [22,23]. Measuring the highest level of learning, “does,” is the greatest challenge for assessment because actions take place in the clinical environment and requires objective means of judging practice in real time [22,24]. These types of work-based assessments are a focus of CBME [25]. Competencies are observable and measurable. A program of assessment is a collection of assessment tools from multiple sources [26] that
32
M.P. Zeller et al. / Transfusion Medicine Reviews 30 (2016) 30–36
Table 2 The RCPSC CanMEDS 2005 and ACGME core competencies frameworks describing the roles of the physician CanMEDS role [12]
Description
Medical expert Physicians integrate all of the CanMEDS roles, applying medical knowledge, clinical skills, and professional attitudes in their provision of patient-centered care. Medical expert is the central physician role in the CanMEDS framework. Communicator Physicians effectively facilitate the physician-patient relationship and the dynamic exchanges that occur before, during, and after the medical encounter. Collaborator Physicians effectively work within a health care team to achieve optimal patient care. Manager Physicians are integral participants in health care organizations, organizing sustainable practices, making decisions about allocating resources, and contributing to the effectiveness of the health care system. Health Physicians responsibly use their expertise and influence to advocate advance the health and well-being of individual patients, communities, and populations. Scholar Physicians demonstrate a lifelong commitment to reflective learning, as well as the creation, dissemination, application, and translation of medical knowledge. Professional Physicians are committed to the health and well-being of individuals and society through ethical practice, profession-led regulation, and high personal standards of behavior.
Table 3 Description of assessment tools used in CBME curricula Assessment tool
Description
Direct performance measure based on a single educational encounter Objective structured Simulated real patient encounters involving clinical examination interviews, physical examinations, counseling, and multiple-choice questions done in bell-ringer/round robin format [27] Mini-CEX Short 15-20 min observed history/physical examination exercise with feedback [28] Key feature assessment Short case vignette with flexible answer types including multiple choice, short answer, selecting options from a list, etc [29] Direct performance measure based on long-term observation Multisource feedback Collection of short questionnaires completed by peers, patients, faculty, and allied health care workers [30] Portfolio Contain evidence on how trainee is developing and progressing through competencies. Varied format and content records feedback received, progress made, reflections contemplated [31]
Does CBME Work? ACGME Framework Competency [10,13]
Description
Medical knowledge
Physicians demonstrate knowledge of established and evolving biomedical, clinical, epidemiologic, and social-behavioral sciences and apply this knowledge to patient care. Physicians are able to provide patient care that is compassionate, appropriate, and effective for the treatment of health problems and the promotion of health. Physicians demonstrate a commitment to carrying out professional responsibilities and adhere to ethical principles. Physicians demonstrate interpersonal and communication skills that result in effective exchange of information and collaboration with patients, their families, and health professionals. Physicians are able to investigate and evaluate their care of patients, to appraise and assimilate scientific evidence, and to continuously improve patient care based on constant self-evaluation and lifelong learning. Physicians demonstrate awareness of and responsiveness to the larger context and system of health care and ability to call effectively on other resources to provide optimal health care.
Patient care
Learning and improvement Systems-based practice
Interpersonal and communications skills
Professionalism
can provide formative and summative feedback. Formative feedback is provided regularly and frequently throughout the training process to encourage self-directedness and self-assessment, whereas summative feedback is provided at the end of training to maintain professional standards [19,26]. Examples of assessment tools along with their descriptions can be found in Table 3.
Fig 1. Adaptation of Miller's Pyramid [18].
A number of medical education institutions including medical schools at Brown University and University of Dundee have adopted CBME into their undergraduate training programs [32,33]. Competency-based medical education is also being incorporated into postgraduate training programs around the world, including in orthopedic surgery [34] at the University of Toronto in Canada, psychiatry in Australia, and a number of specialties in the United States [1]. Dentistry [35] and nursing [36] programs are also using competency-based education curricula. Evidence supporting use of CBME is not yet robust. Studies exploring milestone development and application [37], trainee and program director-perceptions of milestone development [38], and milestone performance validity and reliability [39] are emerging. However, studies directly comparing CBME with traditional approaches to education using objective outcomes are lacking. Criticisms of CBME include concerns that an emphasis on outcomes will result in an artificial deconstruction of practice into competencies [1,40] because discrete competencies cannot capture all aspects of complex medical care [41]. Implementation of a non–time-based curriculum is unwieldy and difficult to administer [1], and learners may lose sight of the acquisition of skills due to overemphasis of productivity and outcome completion [40-43]. A recent editorial by Norman et al [44] raises conceptual, psychometric and logistical concerns with competencybased education. Logistical concerns, for example, include implications for length of training, scheduling (call, assessments, etc), and resources (costs, staffing, etc) [44]. Recognized advantages to a competency-based approach to education include transparent guided learning, regular feedback based on authentic assessment, standardized credentialing, accreditation, and certification [44,45]. Applying CBME to Transfusion Medicine The discipline of transfusion medicine requires expertise in laboratory and clinical medicine with a focus on managerial and leadership skills, policy making, and knowledge of blood product manufacturing. Trainees entering this field must have completed previous training programs in internal medicine and hematology, pediatric hematology, general pathology, or anesthesiology. Clear goals and flexible timelines provide a balance between learning autonomy and required content. In 1989, the Transfusion Medicine Academic Award Group, sponsored by the National Heart, Lung and Blood Institutes in the United States, published a curriculum consisting of goals and objectives for
M.P. Zeller et al. / Transfusion Medicine Reviews 30 (2016) 30–36
undergraduate and postgraduate transfusion medicine education. The group outlined 63 learning objectives, categorized into 12 topics headings [46]. There was a subsequent revision made in 1995 with a focus on undergraduate medical education, but the impact on medical school curricula was minimal [47]. In 2007, the Transfusion Medicine Academic Award Group developed a curriculum for residents and fellows building upon the Accreditation Council for Graduate Medical Education (ACGME) core competencies [48]. This curriculum highlighted 35 topic areas, each with recommended learning activities (didactic lectures, observation, journal clubs, clinical case conference, research projects, site visits to blood collection centers, etc) [48]. Laboratory medicine residency training programs, with some allocated transfusion medicine content, have also been integrating the ACGME core competencies into their curriculum and assessment of learning, providing insight into challenges encountered and measures taken to improve the learning process [49-51]. Implementing a competency-based approach in transfusion medicine addresses the increasingly important role of public accountability in ensuring competency of transfusion medicine graduates across all essential domains. Providing clear goals helps to direct and focus the learner and provides standardized content across trainees and programs. Moving away from a time-based approach allows flexibility in accommodating individual learner's needs. A challenge in implementing a purely CBME approach includes limited evidencebased assessment tools to demonstrate successful acquisition of competency. Recently, a validated examination was developed to assess physician transfusion medicine knowledge using 23 questions on subjects identified by transfusion medicine content experts as being highly important [52]. Programs of assessment are being developed to marry theory-based assessment instruments with work-based tasks to provide frequent, criterion-based, authentic assessment of learners [53]. Simulation laboratories are being implemented in a variety of educational settings [54] and might serve a future purpose in transfusion medicine, for example, to simulate transfusion reactions on highfidelity models or to create safe laboratory environments for learning about immunohematology. Future research will be needed to develop and validate additional methods for assessment of competencies specific to transfusion medicine training. Building a CBME curriculum in transfusion medicine can offer the possibility of specialized training to those seeking to improve upon or gain knowledge and skills in the area of transfusion medicine. With a clear road map laying out required knowledge and skills and how to achieve competency in each, individuals looking to expand or update their knowledge can use the curriculum as a self-directed learning tool for continuing medical education purposes. In addition, practice eligibility requirements in transfusion medicine in Canada have been established based on the summative portfolio. Case Study: The Canadian Experience History of Transfusion Medicine Education in Canada Transfusion medicine is defined by the RCPSC as “that domain of laboratory and clinical medicine concerned with all aspects of the collection, testing, preparation, storage, transportation, pretransfusion testing, indications for, infusion and safety of human blood components and products, nonhuman alternatives and alternative products manufactured by recombinant DNA technology. These activities are undertaken in such a way that the rights of blood donors, patients and families are respected” (Competency Training Requirements for the Area of Focused Competence in Transfusion Medicine Document, 2012, V1.3). In the final report of the Commission of Inquiry on the blood system in Canada, Justice Krever made recommendations to transform the Canadian blood system, including improvements to transfusion medicine education [55]. Canadian Blood Services (CBS) was established in 1998
33
with the mandate to rebuild a safe, secure, accessible, and affordable national blood system. To improve education, advisory groups brought together transfusion medicine experts from across the country with representatives from other stakeholder groups to design an educational program for physicians at hospital transfusion services and CBS blood centers. A more formalized approach to education of transfusion medicine was recommended for physicians based upon predefined learning objectives. The decision was made to apply to the RCPSC for recognition as a subspecialty to ensure standardized content and training consistency. Transfusion medicine was approved as a subspecialty committee by the RCPSC in 2001. This approval recognized transfusion medicine as a subspecialty and standardized training across the country. Five training programs were accredited at McMaster University, University of Toronto, University of British Columbia, University of Ottawa, and Dalhousie University. Trainees were required to complete the objectives and time-defined rotations at the hospital and at the blood center. Canadian Blood Services has been involved with all Canadian transfusion medicine residency programs, providing funding for 2 entry level positions per year, and is responsible for a considerable amount of the transfusion medicine program. Trainees gain longitudinal exposure at the local blood center and participating in medical office-related issues. Rotations also take place at CBS head office, main production/ manufacturing sites, and donor testing sites. In addition to providing funding, CBS has appointed a national program director to oversee all aspects of the training within the centers. Responsibilities include a weekly National Education Program which includes a monthly journal club, monthly scientific presentation from CBS scientists, and topic teaching by experts in the field. Historically, these educational activities have served to provide transfusion medicine trainees with exposure to nonmedical expert CanMEDS roles (such as manager, advocate, etc) and facilitate participation in lectures from world experts. In 2010, transfusion medicine training in Canada became an “Area of Focused Competence,” also called a diploma program with competencybased training and assessment. The diploma program of the RCPSC is designed to provide focused teaching in a subspecialty discipline over 1 or 2 years. Assessment is done using a summative portfolio rather than a certification examination. Transfusion medicine was the first program to receive approval as an Area of Focused Competence in April 2011. The transformation from a time- and process-based model to a CBME diploma program required curriculum redesign and continues to undergo revision. From 1995 to 2013, there have been 34 trainees enrolled in transfusion medicine training across Canada (Fig 2). Need for ongoing training of transfusion medicine specialists continues to grow as transfusion medicine plays an integral role in almost every medical and surgical specialty yet remains a gap in most practitioners' knowledge [56-59]. Expanding roles for transfusion medicine specialists now includes education of other transfusion medicine personnel such as nurses, medical students, and laboratory technologists; conducting research ranging from basic science to clinical investigations; establishment of evidence-based guidelines for transfusion; and setting patterns of practice and utilization of blood products, cell therapy, and tissue banking. In Canada, there are only 1 to 3 transfusion medicine trainees each year; however, in jurisdictions training larger numbers of individuals, developing standardized curricula and assessment criteria are even more important to ensure training consistency. Process of Curricular Development for a CBME Transfusion Medicine Training Program Eight competencies were defined (Text Box 2) by the RCPSC transfusion medicine subspecialty group (national leaders and experts in transfusion medicine) with a total of 24 milestones or stages of progression toward achieving competence [14,60]. A competency-based framework was applied to organize structure for the competencies and milestones.
34
M.P. Zeller et al. / Transfusion Medicine Reviews 30 (2016) 30–36
Transfusion Medicine Trainees
Table 4 The learning outcomes, methods of assessment, and required documentation to fulfill the core competency of “Management of the medical laboratory and blood center, including quality, safety and regulatory aspects” as it appears in the transfusion medicine portfolio
Number of Trainees
5
Key portfolio outcomes/ milestone
4
2
2013
2011
2012
2010
2009
2008
2007
2006
2005
2003
2004
2002
2000
2001
1998
1999
1997
1995
0
1996
1
Entrance Year Canadian
International
Fig 2. Number of transfusion medicine trainees in Canada, showing the number of trainees entering the program each year. Length of training is not reflected in this figure.
The RCPSC convened a 2-day face-to-face meeting for all 5 transfusion medicine program directors in Canada with a facilitator with expertise in curriculum development. The objective was to develop a portfolio for transfusion medicine training, consisting of key competencies broken into learning outcomes with methods of instruction and assessment that would reflect the decided upon goals and core competencies.
Text Box 2Goals and Core Competencies of Transfusion Medicine Diploma Program Upon completion of training, a diplomate is expected to function as a competent specialist in transfusion medicine, capable of an enhanced practice in this area of focused competence, within the scope of Internal medicine, hematology, pediatrics, hematological pathology, anesthesiology, or general pathology. The AFC trainee must acquire a working knowledge of the theoretical basis of the discipline, including its foundations in the sciences and research. 1. The diagnostic and therapeutic aspects of immunohematology, apheresis, histocompatibility, and related molecular biology and biotechnology 2. Management of the medical laboratory and blood center, including quality, safety, and regulatory aspects 3. Ensuring the appropriate use of blood 4. Ensuring the adequacy of supply for the blood system 5. Supervising the provision of a safe and effective blood supply 6. Supervising the banking and provision of cell therapy products and human tissues for transplantation purposes 7. Engaging governments, other physicians, and other health professionals in transfusion medicine 8. Advancing the discipline through basic scientific and clinically applied research
Documents to be submitted
• Write or update a standard operating, technical, or nursing procedure relevant to blood transfusion • Satisfactory completion • Using a lab audit Evaluate a laboratory's of (1) internal laboratory template or checklist, compliance with accreditation standards audit report relating to an submit 1 lab audit; only 1 external standard, signed section of a lab is and develop a strategy required; lab audit can be to ensure a laboratory is off by a supervisor completed at the hospital compliant with or blood center accreditation standards • Satisfactory completion • Using a template or and regulations of 1 report detailing a gap checklist, submit a gap analysis; write a report in analysis and strategy to response to the gap ensure compliance with analysis providing a an external laboratory strategy to ensure standard, signed off by a compliance; only 1 supervisor section of a lab is required; the gap analysis can be completed at the hospital or blood center and must be signed off by a supervisor • Satisfactory completion • Write a report Manage the of a report describing the describing validation of a administrative and procedure or instrument; validation of a new technical aspects of signed off by a supervisor laboratory procedure or effective laboratory instrument function Develop a transfusionrelated standard operating procedure (SOP)
3
Standards of assessment • Satisfactory completion of 1 transfusion-related SOP, signed off by a supervisor
Table 4 provides an example of the developmental path, educational activities, and assessment tools used to teach a core competency. A variety of assessment tools are used in the current transfusion medicine diploma program in Canada, including case reports, structured oral examination, structured written take home examination, structured blood center examination, multisource feedback, written reflection, and intraining evaluation reports. The templates for the case write-ups, feedback forms, and evaluations are currently evolving with iterative feedback from trainees and supervisors. The examinations are not certification level examinations and are designed to ensure that standard content is covered by all trainees regardless of individual locations/experiences. All forms of assessment are being collected into a portfolio (Table 3). Regular, timely, face-to-face, and documented feedback on trainee progress is carried out at the level of rotation supervisor and program director. Standard criterion is being designed to ensure consistency across programs. For example, agreed upon answer keys to the examination questions are used to determine successful completion of hospital-based and CBS-based examinations. Because there are small numbers of programs and trainees, there is ongoing communication and sharing of resources. Portfolio adjudication involves levels of review as well: trainees complete a task, create a report attesting to task completion, and submit the document for review by a supervisor; the supervisor signs off once satisfied; the document is further reviewed with the rotation supervisor and program director; the completed document is placed into the portfolio. At request for certification, credential agents at the RCPSC ensure that all documentation required to meet minimum standard is complete. The portfolio is then sent for review by 2 members of the specialty committee. Once the portfolio meets criterion standard, the candidate is deemed competent and achieves certification. Programs in Canada are in the process of piloting an electronic-based portfolio, ePortfolio. Once
M.P. Zeller et al. / Transfusion Medicine Reviews 30 (2016) 30–36
there is consensus that a trainee has achieved each competency, the trainee will achieve diploma certification. We have encountered challenges during the transition to a CBME approach and have worked toward addressing each as it arises through ongoing discussion between trainees, past and present, rotation supervisors, program directors, and the Royal College. For example, there has been discussion around the use of rotation evaluations and whether they are still necessary. We decided that regular evaluations provide the trainee with important feedback, serve to keep trainees on target with regard to milestone achievement, and identify areas of weakness. We have changed the format of these evaluations to maintain consistency in the language and objectives of a CBME training program. As final portfolios are being submitted, there is increasing need to develop standardized criteria for measuring achievement of milestones as judged by each level of assessor (rotation supervisor, program director, and external reviewers). Using the same degree of scrutiny as a rotation supervisor might use on a number of case reports would translate into hundreds of hours of review for the external reviewers on the entire portfolio. Developing rubrics to guide objective and progressive assessment is being considered. A need for ongoing faculty development surrounding curriculum delivery and use of assessment tools was recognized early on, and efforts were made to address this through education sessions and providing access to portfolio objectives [61]. Our construct for transfusion medicine training has incorporated elements from both CBME and traditional approaches. It was felt that a blended approach to education might best capture the advantages of both CBME and traditional approaches to education, while minimizing some of the challenges. Other areas of education have found that using multifaceted instructional methods can have a positive outcome in learners [62]. Studies of changing practice behavior and knowledge translation also support use of a blended approach [63-65]. One of the greatest challenges in adopting this new approach is how to determine whether it worked. What outcomes determine program success and what can we measure and compare to contribute to a lacking, yet necessary, body of literature reporting on the impact of CBME? With the limited number of trainees in transfusion medicine, maintaining strong ties with new graduates will provide valuable insights by identifying knowledge and skill gaps, inform ongoing curriculum design/revision, and promote development of valid assessment tools.
Conclusion Transfusion medicine is well suited to CBME given the need for individualized and flexible time frames, specialized scope of practice, and measurable outcomes. Although the transition to CBME is complete, set rotations of fixed times still remain. This provides rotation supervisors with a framework for scheduling instructors' time and ensures that trainees will have opportunity to complete all education activities. Program evaluation is ongoing with constant communication between faculty, trainees, and the RCPSC. In summary, CBME is becoming an important part of health professions education. Transfusion medicine is an example of early success in this transition and serves as a model for other training programs.
Conflict of Interest The authors have no conflicts of interest to disclose.
References [1] Glover Takahashi S, Waddell A, Kennedy M, Hodges B. Innovations, integration and implementation issues in competency-based education in postgraduate medical education. Members FMEC PG Consort; 2011. [2] Carraccio C, Wolfsthal SD, Englander R, Ferentz K, Martin C. Shifting paradigms: from Flexner to competencies. Acad Med 2002;77:361–7.
35
[3] Frank JR, Mungroo R, Ahmad Y, Wang M, De Rossi S, Horsley T. Toward a definition of competency-based education in medicine: a systematic review of published definitions. Med Teach 2010;32:631–7. http://dx.doi.org/10.3109/0142159X.2010.500898. [4] Frank JR, Snell LS, Ten Cate O, Holmboe ES, Carraccio C, Swing SR, et al. Competencybased medical education: theory to practice. Med Teach 2010;32:638–45. http://dx. doi.org/10.3109/0142159X.2010.501190. [5] Cameron J. William Stewart Halsted: our surgical heritage. Ann Surg 1997;225: 445–58. [6] Bryan CS, Longo LD. Perspective: teaching and mentoring the history of medicine: an Oslerian perspective. Acad Med 2013;88:97–101. http://dx.doi.org/10.1097/ACM. 0b013e31827653f5. [7] Norman G. Medical education: past, present and future. Perspect Med Educ 2012;1: 6–14. http://dx.doi.org/10.1007/s40037-012-0002-7. [8] Cooke M, Irby D, Sullivan W, Ludmerer K. American medical education 100 years after the Flexner report. N Engl J Med 2006;355:1339–44. [9] Harden RM. Outcome-based education—the ostrich, the peacock and the beaver. Med Teach 2007;29:666–71. http://dx.doi.org/10.1080/01421590701729948. [10] Weinberger SE, Pereira AG, Iobst WF, Mechaber AJ, Bronze MS. Competency-based education and training in internal medicine. Ann Intern Med 2013;153:751–6. [11] Hodges BD. A tea-steeping or i-Doc model for medical education? Acad Med 2010; 85:S34–44. http://dx.doi.org/10.1097/ACM.0b013e3181f12f32. [12] Frank JR, Jabbour M, Fréchette D, Marks M, Valk N, Bourgeois G, editors. Report of the CanMEDS Phase IV Working Groups. Ottawa: The Royal College of Physicians and Surgeons of Canada; 2005. p. 1–11. [13] Joyce B. Facilitator's Manual: Practical Implementation of the Competencies—ACGME April 2006. http://216.92.22.76/discus/messages/21/MODULE_2_Facilitator_Manual_module2-486.pdf. [14] Iobst WF, Sherbino J, Cate OT, Richardson DL, Dath D, Swing SR, et al. Competencybased medical education in postgraduate medical education. Med Teach 2010;32: 651–6. http://dx.doi.org/10.3109/0142159X.2010.500709. [15] Frank JR, Danoff D. The CanMEDS initiative: implementing an outcomes-based framework of physician competencies. Med Teach 2007;29:642–7. http://dx.doi. org/10.1080/01421590701746983. [16] Verma S, Broers T, Paterson M, Schroder C, Medves JM, Morrison C. Core competencies: the next generation. Comparison of a common framework for multiple professions. J Allied Health 2009;38:47–53. [17] Albanese MA, Mejicano G, Anderson WM, Gruppen L. Building a competency-based curriculum: the agony and the ecstasy. Adv Health Sci Educ Theory Pract 2010;15: 439–54. http://dx.doi.org/10.1007/s10459-008-9118-2. [18] Miller G. The assessment of clinical skills/competence/performance. Acad Med 1990; 65:s65–7. [19] Holmboe ES, Sherbino J, Long DM, Swing SR, Frank JR. The role of assessment in competency-based medical education. Med Teach 2010;32:676–82. http://dx.doi. org/10.3109/0142159X.2010.500704. [20] Dannefer EF, Henson LC. The portfolio approach to competency-based assessment at the Cleveland Clinic Lerner College of Medicine. Acad Med 2007;82:493–502. http:// dx.doi.org/10.1097/ACM.0b013e31803ead30. [21] Epstein RM. Assessment in medical education. N Engl J Med 2007;356:387–96. http://dx.doi.org/10.1056/NEJMra054784. [22] Van der Vleuten CPM, Schuwirth LWT, Scheele F, Driessen EW, Hodges B. The assessment of professional competence: building blocks for theory development. Best Pract Res Clin Obstet Gynaecol 2010;24:703–19. http://dx.doi.org/10.1016/j. bpobgyn.2010.04.001. [23] Van der Vleuten CPMM, Schuwirth LWT. Assessing professional competence: from methods to programme. Med Educ 2005;39:309–17. http://dx.doi.org/10.1111/j. 1365-2929.2005.02094.x. [24] Van der Vleuten CPM, Schuwirth LWT, Driessen EW, Dijkstra J, Tigelaar D, Baartman LKJ, et al. A model for programmatic assessment fit for purpose. Med Teach 2012;34: 205–14. http://dx.doi.org/10.3109/0142159X.2012.652239. [25] Gingerich A, Kogan J, Yeates P, Govaerts M, Holmboe E. Seeing the “black box” differently: assessor cognition from three research perspectives. Med Educ 2014;48: 1055–68. http://dx.doi.org/10.1111/medu.12546. [26] Norcini J, Anderson B, Bollela V, Burch V, Costa MJ, Duvivier R, et al. Criteria for good assessment: consensus statement and recommendations from the Ottawa 2010 Conference. Med Teach 2011;33:206–14. http://dx.doi.org/10.3109/0142159X. 2011.551559. [27] Harden RM, Gleeson FA. Assessment of clinical competence using an objective structured clinical examination (OSCE). Med Educ 1979;13:41–54. [28] Norcini JJ, Blank LL, Duffy FD, Fortna GS. Academia and Clinic The Mini-CEX: a method for assessing clinical skills. Ann Intern Med 2013;138:476–83. [29] Page G, Bordage G, Allen T. Developing key features problem and exam to assess clinical decision skills-making. Acad Med 1995;70:194–202. [30] Davies H, Archer J, Children S. Multi source feedback: development and practical aspects. Clin Teach 2005;2:77–81. [31] Van Tartwijk J, Driessen EW. Portfolios for assessment and learning: AMEE Guide no. 45. Med Teach 2009;31:790–801. http://dx.doi.org/10.1080/01421590903139201. [32] Smith SR, Dollase RH, Boss JA. Assessing students' performances in a competencybased curriculum. Acad Med 2003;78:97–107. [33] Davis MH, Harden RM. Planning and implementing an undergraduate medical curriculum: the lessons learned. Med Teach 2003;25:596–608. http://dx.doi.org/10. 1080/0142159032000144383. [34] Ferguson P, Kraemer W, Nousiainen M, Safir O, Sonnadara R, Alman B, et al. Threeyear experience with an innovative, modular competency-based curriculum in orthopaedic training. J Od Bone Jt Surg 2013;95:1–6. [35] Yip HK, Smales RJ. Review of competency-based education in dentistry. Br Dent J 2000;189:324–6.
36
M.P. Zeller et al. / Transfusion Medicine Reviews 30 (2016) 30–36
[36] Levin PF, Swider SM, Breakwell S, Cowell JM, Reising V. Embracing a competencybased specialty curriculum for community-based nursing roles. Public Health Nurs 2013;30:557–65. http://dx.doi.org/10.1111/phn.12042. [37] Schultz K, Griffiths J, Lacasse M. The application of entrustable professional activities to inform competency decisions in a family medicine residency program. Acad Med 2015;90:888–97. http://dx.doi.org/10.1097/ACM.0000000000000671. [38] Lindeman BM, Sacks BC, Lipsett PA. Graduating students' and surgery program directors' views of the Association of American Medical Colleges core entrustable professional activities for entering residency: where are the gaps? J Surg Educ 2015. http://dx.doi.org/10.1016/j.jsurg.2015.07.005. [39] Beeson MS, Holmboe ES, Korte RC, Nasca TJ, Brigham T, Russ CM, et al. Initial validity analysis of the emergency medicine milestones. Acad Emerg Med 2015;22:838–44. http://dx.doi.org/10.1111/acem.12697. [40] Talbot M. Monkey see, monkey do: a critique of the competency model in graduate medical education. Med Educ 2004;38:587–92. http://dx.doi.org/10.1046/j.13652923.2004.01794.x. [41] Whitehead C, Kuper A. Competency-based training for physicians: are we doing no harm? CMAJ 2014;187:E128–9. [42] Reeves S, Fox A, Hodges BD. The competency movement in the health professions: ensuring consistent standards or reproducing conventional domains of practice? Adv Health Sci Educ Theory Pract 2009;14:451–3. http://dx.doi.org/10.1007/ s10459-009-9166-2. [43] Glass J. Competency based training is a framework for incompetence. Br Med J 2014; 348:1–2. [44] Norman G, Norcini J, Bordage G. Competency-based education: milestones or millstones? J Grad Med Educ 2014;6:1–6. http://dx.doi.org/10.4300/JGME-D-1300445.1. [45] Carraccio C, Iobst WF, Philibert I. Milestones: not millstones but stepping stones. J Grad Med Educ 2014;6:589–90. [46] Simon TL. Comprehensive curricular goals for teaching transfusion medicine. Transfusion 1989;29:438–46. [47] Karp JK, Weston CM, King KE. Transfusion medicine in American undergraduate medical education. Transfusion 2011;51:2470–9. http://dx.doi.org/10.1111/j.15372995.2011.03154.x. [48] Wu Y, Tormey C, Stack G. Resident and fellow training in transfusion medicine. Clin Lab Med 2007;27:293–342. http://dx.doi.org/10.1016/j.cll.2007.03.006 [abstract vii]. [49] Alexander CB. Assessing resident competency in laboratory medicine. Clin Lab Med 2007;27:269–81. http://dx.doi.org/10.1016/j.cll.2007.03.004 [abstract vi]. [50] Alexander CB. Pathology graduate medical education (overview from 2006-2010). Hum Pathol 2011;42:763–9. http://dx.doi.org/10.1016/j.humpath.2010.11.008. [51] Naritoku WY, Alexander CB, Bennett BD, Black-Schaffer WS, Brissette MD, Grimes MM, et al. The pathology milestones and the next accreditation system.
[52]
[53] [54]
[55] [56]
[57]
[58]
[59]
[60]
[61]
[62]
[63]
[64] [65]
Arch Pathol Lab Med 2014;138:307–15. http://dx.doi.org/10.5858/arpa.20130260-SA. Haspel RL, Lin Y, Fisher P, Ali A, Parks E. Development of a validated exam to assess physician transfusion medicine knowledge. Transfusion 2014;54:1225–30. http:// dx.doi.org/10.1111/trf.12425. Chan T, Sherbino J. The McMaster Modular Assessment Program (McMAP). Acad Med 2015;90:1. http://dx.doi.org/10.1097/ACM.0000000000000707. Cook DA, Hamstra SJ, Brydges R, Zendejas B, Szostek JH, Wang AT, et al. Comparative effectiveness of instructional design features in simulation-based education: systematic review and meta-analysis. Med Teach 2013;35:e867–98. http://dx.doi.org/ 10.3109/0142159X.2012.714886. Krever H. Final report: Commission of Inquiry on the Blood System in Canada [Krever report]. Ottawa: Government of Canada; 1997. O’Brien KL, Champeaux AL, Sundell ZE, Short MW, Roth BJ. Transfusion medicine knowledge in postgraduate year 1 residents. Transfusion 2010;50:1649–53. http:// dx.doi.org/10.1111/j.1537-2995.2010.02628.x. Graham J, Grant-Casey J, Alston R, Baker P, Pendry K. Assessing transfusion competency in junior doctors: a retrospective cohort study. Transfusion 2014;54:128–36. http://dx.doi.org/10.1111/trf.12274. Arinsburg SA, Skerrett DL, Friedman MT, Cushing MM. A survey to assess transfusion medicine education needs for clinicians. Transfus Med 2012;22:44–9. http://dx.doi. org/10.1111/j.1365-3148.2011.01118.x [quiz 49–51]. Saidenberg E, Pugh D. The use of an objective structured clinical examination to assess internal medicine residents' transfusion knowledge. Transfusion 2014;54: 1537–41. http://dx.doi.org/10.1111/trf.12508. Englander R, Carraccio C. From theory to practice: making entrustable professional activities come to life in the context of milestones. Acad Med 2014;89:1321–3. http://dx.doi.org/10.1097/ACM.0000000000000324. Dath D, Iobst W. The importance of faculty development in the transition to competency-based medical education. Med Teach 2010;32:683–6. http://dx.doi. org/10.3109/0142159X.2010.500710. Steinert Y, Naismith L, Mann K. Faculty development initiatives designed to promote leadership in medical education. A BEME systematic review: BEME Guide No. 19. Med Teach 2012;34:483–503. Davis DA, Thomson MA, Oxman AD, Haynes RB. Changing physician performance. A systematic review of the effect of continuing medical education strategies. JAMA 1995;274:700–5. http://dx.doi.org/10.1001/jama.274.9.700. Mostofian F, Ruban C, Simunovic N. Changing physician behavior: what works? Am J Manag Care 2015;21:75–84. Tamblyn R, Battista R. Changing clinical practice: which interventions work? J Contin Educ Health Prof 1993;13:273–88. http://dx.doi.org/10.1002/chp. 4750130403.
