The healthcare industry has seen a recent proliferation in the adoption of team training, such as crew resource management (CRM). Empirical studies of the ...
Proceedings of the International Symposium of Human Factors and Ergonomics in Healthcare http://hcs.sagepub.com/
Is Crew Resource Management Training Effective in Healthcare Settings? Paul O'Connor, Angela O'Dea and Ivan Keogh Proceedings of the International Symposium of Human Factors and Ergonomics in Healthcare 2013 2: 105 DOI: 10.1177/2327857913021020 The online version of this article can be found at: http://hcs.sagepub.com/content/2/1/105
Published by: http://www.sagepublications.com
On behalf of:
Human Factors and Ergonomics Society
Additional services and information for Proceedings of the International Symposium of Human Factors and Ergonomics in Healthcare can be found at: Email Alerts: http://hcs.sagepub.com/cgi/alerts Subscriptions: http://hcs.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav Citations: http://hcs.sagepub.com/content/2/1/105.refs.html
>> Version of Record - Jun 1, 2013 What is This?
Downloaded from hcs.sagepub.com at Library Services HSE West on July 31, 2013
2013 International Symposium on Human Factors and Ergonomics in Health Care: Advancing the Cause
105
Is Crew Resource Management Training Effective in Healthcare Settings? Paul O’Connor1, Angela O’Dea1, and Ivan Keogh2 1
Department of General Practice, National University of Ireland, Galway, Ireland.
2
Academic Department of Otolaryngology, National University of Ireland, Galway, Ireland.
The healthcare industry has seen a recent proliferation in the adoption of team training, such as crew resource management (CRM). Empirical studies of the effectiveness of CRM-type training in healthcare were subjected to meta-analysis. Fourteen CRM evaluation studies were found to fulfill the a priori criteria for inclusion in the meta-analysis. The metrics of CRM training effectiveness analyzed were: reactions, attitudes, knowledge, behaviors, and organization. CRM trained participants responded positively to CRM, the training had large effects on the participants’ knowledge and behaviors, and a medium effect on their attitudes. The training was also found to have a medium effect on the number of errors/adverse events and a small effect on the time to complete clinical tasks. Although there are methodological limitations that mean the results from this study should be treated with a little caution, the findings are nevertheless encouraging.
general conclusion regarding the effectiveness. They found
Introduction Since Crew Resource Management (CRM) training was first adopted in civilian aviation in the early 1980s it has become Copyright 2013 Human Factors and Ergonomics Society. All rights reserved. DOI 10.1177/2327857913021020
the most widely used team training strategy in the world. CRM training is designed to target knowledge, skills, and abilities as well as mental attitudes and motives related to cognitive processes and interpersonal relationships (Flin et al.,
evidence of an improvement in attitudes and teamworking after training and some evidence of improved technical performance, efficiency and reduced errors, in the short and possibly medium term. The purpose of this paper is to use meta-analyses techniques to evaluate the effectiveness of CRM type training in healthcare settings.
2008). Method There have been a number of literature reviews and a metaanalysis of the effectiveness of CRM across a range of highrisk industries. The conclusion from these cross-industry reviews is that reactions to the training are generally positive, and CRM training has positive effects on the teamwork attitudes, knowledge, and behaviors of participants. However, there is a lack of evidence regarding the effect on safety and/or performance (O’Connor et al. 2008; Salas et al 2006; Salas et
An electronic literature search of PUBMED, Embase and PsychInfo was carried out using appropriate keywords. A total 1,283 studies were identified. For a study to be included in the meta-analysis, the training had to utilize a CRM training model and an evaluation had to be reported from at least one of the levels of Kirkpatrick’s (1976) training evaluation hierarchy
(reactions,
attitudes,
knowledge,
behavior,
organization and meet the same set of inclusion criteria as
al 2001).
used by O’Connor et al. (2008). In total 14 studies were There have also been three literature reviews specifically focused on CRM training in healthcare (Buljac-Samardzic et al, 2010; McCulloch, Rathbone & Catchpole, 2011; Zeltser & Nash, 2010). However, only McCulloch et al. (2009) drew any
appropriate for the meta-analysis (one study was under review, but given the small number studies, and the fact that it met the inclusion criteria it was included in the meta-analysis). The MIX 2.0 software was used to carry out the meta-analyses.
Downloaded from hcs.sagepub.com at Library Services HSE West on July 31, 2013
2013 International Symposium on Human Factors and Ergonomics in Health Care: Advancing the Cause
106
Results
Discussion
Six studies included measurement of reactions to CRM
This study provides evidence to support the utility of CRM
training that met the criteria for reactions level data in the
training in healthcare settings. This type of evaluation data is
meta-analysis. Overall, participants were positive about the
necessary in order to provide an argument to support the use
various training courses they attended. On average participants
of limited resources (money, time, and people) required to
rated training four out of five possible points (mean= 4.19).
implement a successful CRM training program.
Three evaluations of knowledge change were included in the
Overall, participants responded positively to CRM training,
meta-analysis. The three different comparison of knowledge
the training had large effects on the participants’ knowledge
2
and behaviors, and a medium effect on their attitudes. The
resulted in a large mean effect size (d= 1.52). However, the I 2
test indicated high levels of heterogeneity (I = 78.8%).
training was also found to have a medium effect on the number of errors/adverse events and a small effect on the time
Six evaluations of attitude change were included in the meta-
to complete clinical tasks. However, these results should be
analysis. The results show a medium mean effect size of
considered within the context of three methodological
training on attitudes (d= 0.51). However, the level of
limitations.
heterogeneity, or consistency, across the studies was found to be large, as shown by the high I2 values (I2= 75.1%).
1.
The small number of studies included in the metaanalyses represents a constraint to the present effort.
Eight evaluations were included in the meta-analysis that
The vast majority of the studies examined could not be
analyzed the effects of CRM training on behavior. A large
included in the meta-analysis due to a lack of, or
mean effect size resulted (d= 1.18), although the level of
insufficient, data reported to allow effect sizes to be
2
calculated.
heterogeneity was also large (I = 82.5%). 2.
The evaluations included in each meta-analysis were
Three organizational level effects of training were examined
not necessarily independent of each other. To illustrate,
using meta-analysis techniques: number of errors/adverse
in some cases it was possible to extract inter-group and
events (three evaluations), time to complete clinical tasks (two
repeated measures design comparisons of the impact of
evaluations), length of stay in hospital (two evaluations). A
the training on behaviors from the same study.
medium effect size of the training was found for number of errors/adverse events (d= 0.54) a small effect size was found
3.
There were high levels of heterogeneity (or lack of consistency), of effects across the studies.
for time to complete clinical tasks (d= 0.22) and no effect size of length of stay of patient (d= 0.13). There was a high
Conclusion
heterogeneity for the number of errors/adverse events (I2= 75.1%), but the Q statistics for time to complete (Q= 0.76) and
It is important that researchers and reviewers be more rigorous
length of stay (Q= 1.72) were not significant.
about the quality of the reporting of CRM training evaluation studies. Despite the methodological limitations, the results of the meta-analysis are encouraging in terms of the utility of CRM-type training in medicine. The findings provide
Downloaded from hcs.sagepub.com at Library Services HSE West on July 31, 2013
2013 International Symposium on Human Factors and Ergonomics in Health Care: Advancing the Cause
evidence to support the inclusion of CRM-type training in the
.
training curriculum of all healthcare professionals.
Bibliography
Buljac-Samardzic, Wijngaarden,
M., J.D.,
Dekker-van &
van
Doorn, Wijk,
C.M., K.P.
van
(2010).
Interventions to improve team effectiveness: a systematic review. Health Policy, 94,183-195. Flin R, O’Connor P, & Crichton M. (2008). Safety at the Sharp End: Training Non-technical Skills. Aldershot, England Ashgate Publishing Ltd. Kirkpatrick DL. (1976). Evaluation of training. In: Craig RL, Bittel LR, editors. Training and Development Handbook. New York: McGraw Hill. McCulloch,
P.,
Rathbone,
J.,
Catchpole,
K.
(2011).
Interventions to improve teamwork and communications among healthcare staff. British Journal of Surgery, 98, 469-479. O’Connor, P., Campbell, J., Newon, J., Melton, J., Salas, E., &Wilson, K.A. (2008). Crew Resource Management Training Effectiveness: A Meta-Analysis and Some Critical Needs. International Journal of Aviation Psychology,18, 353-368. O’Connor, P., Flin, R., & Fletcher, G. (2002). Methods used to evaluate the effectiveness of CRM training: A literature review. Journal of Human Factors and Aerospace Safety;2, 217-234. Salas, E., Burke, C.S., Bowers, C.A., & Wilson K.A. (2001). Team training in the skies: Does crew resource management (CRM) training work? Human Factors, 43, 641-674. Salas, E., Wilson, K.A., Burke, C.S., & Wightman, D.C. (2006). Does crew resource management training work? An update, an extension, and some critical needs. Human Factors, 48, 392-412. Zeltser, M.V. & Nash DB. (2010). Approaching the evidence basis for aviation-derived teamwork training in medicine. American Journal of Medical Quality, 25, 13-23.
Downloaded from hcs.sagepub.com at Library Services HSE West on July 31, 2013
107