Adv in Health Sci Educ (2011) 16:59–67 DOI 10.1007/s10459-010-9241-8
An exploration of the relationship between emotional intelligence (EI) and the Multiple Mini-Interview (MMI) Wendy Yen • Richard Hovey • Kathryn Hodwitz • Su Zhang
Received: 10 December 2009 / Accepted: 31 May 2010 / Published online: 11 June 2010 Ó Springer Science+Business Media B.V. 2010
Abstract The present study explored the relationship between the Multiple MiniInterview (MMI) admissions process and the Bar-On EQ-i emotional intelligence (EI) instrument in order to investigate the potential for the EQ-i to serve as a proxy measure to the MMI. Participants were 196 health science candidates who completed both the MMI and the EQ-i as part of their admissions procedure at the Michener Institute for Applied Health Sciences. Three types of analyses were conducted to examine the relationship between the two tools: reliability analyses, correlational analyses, and a t-test. The tools were found to be moderately reliable. No significant relationships were found between the MMI and the EQ-i at the total or subscale level. The ability of the EQ-i to discriminate between accepted and not-accepted students was also not supported. These findings do not support the use of the EQ-i as a potential pre-screening tool for the MMI, but rather highlight the need to exercise caution when using emotional intelligence instruments for high-stakes admissions purposes. Keywords Admissions Emotional intelligence Health sciences Non-cognitive skills Multiple Mini-Interview
Technical expertise has long been regarded as the foundation of successful medical practice. Non-cognitive skills such as communication and empathy, however, have been increasingly recognized as essential aspects of quality health care. The role of these relational qualities, as demonstrated through humanistic or tacit skills, in professional practice has become a significant focus of medical research and training in the past decade. Part of this movement can be attributed to the patient safety literature which found that a significant number of casualties W. Yen R. Hovey K. Hodwitz (&) The Michener Institute for Applied Health Sciences, 222 St. Patrick Street, Toronto, ON M5T 1V4, Canada e-mail:
[email protected] S. Zhang Education Quality and Accountability Office, Suite 1200, 2 Carlton Street, Toronto, ON M5B 2M9, Canada
123
60
W. Yen et al.
resulting from medical errors stem from poor communication and collaboration within the health care team and not from a lack of discipline-specific skills (Baker et al. 2004). The benefits of interprofessional collaboration and the requisite humanistic and tacit skills needed to enhance patient safety have been well cited in the literature (Barrett et al. 2001; Canadian Health Services Research Foundation 2006; Zwarenstein et al. 2005). To address the need for well-rounded health care professionals, a method of selecting candidates for health science programs who display not only high academic caliber but also strong interpersonal skills was developed for use in Canadian health science educational institutions. The Multiple MiniInterview (MMI; see ‘‘Instruments’’) was designed by McMaster University to assess the noncognitive qualities of prospective students, such as their ability to think critically, communicate effectively, and demonstrate ethical decision making (Eva et al. 2004). In 2008, the MMI was incorporated into the admissions process at The Michener Institute for Applied Health Sciences, a post-secondary institution for the study of allied health programs. Preliminary feedback from Faculty at Michener indicate that the process selects students who are better communicators, better able to work in team, and more suited for health science programs. However, given that only a portion of applicants are invited to participate in the MMI and that many invited candidates must travel from outside of the GTA, the ability to pre-screen candidates for relational qualities is of pertinent interest. The utility of a pre-screening tool that could differentiate candidates would also have significant economic advantages given that the MMI is a time- and resource-intensive endeavour. Some research suggests that the non-cognitive skills essential for contemporary health care professionals overlap with the broader notion of emotional intelligence (EI). Emotional intelligence is considered an individual’s ability to understand oneself, relate to others, manage one’s emotions, and adapt to various environmental demands and stressors (Goleman 1995). Lewis et al. (2005) note that the characteristics that define an emotionally-intelligent individual, according to Goleman’s (1998) model, are consistent with the qualities desired in medical students, as put forth by the General Medical Council (2003) and the Association of American Medical Colleges (1998). Similarly, Carrothers et al. (2000) found a high degree of overlap between the characteristics desired in incoming medical students, as reported by a university admissions committee, and the qualities of emotional intelligence. Research has also cited the positive effects of well-developed emotional intelligence in health care environments. Nursing literature, for example, has found that the characteristics of emotional intelligence such as empathy, self-awareness, interpersonal abilities, and optimism, are related to both increased team effectiveness (Kooker et al. 2007; McCallin and Bamford 2007; Quoidbach and Hansenne 2009) and positive patient outcomes (Cadman and Brewer 2001). The potential role of emotional intelligence in health science education has been met with considerable interest in the past decade (Carrothers et al. 2000; Epstein and Hundert 2002; Freshwater and Stickley 2004; Lewis et al. 2005; Grewal and Davidson 2008) and the utility of EI measures for selection purposes is of particular concern for both academic (Cadman and Brewer 2001) and organizational settings (Day and Carroll 2008; Grubb and McDaniel 2007). Since the construct of EI and the qualities desired in health science students appear to overlap, it is plausible that an EI instrument could serve as a prescreening tool for the MMI. Among the many tools that have been developed, the BarOn Emotional Social Inventory (BarOn EQ-I; see ‘‘Instruments’’) is the most commonly used instrument used to measure emotional intelligence. While no studies to date have explored the relationship between the MMI and the BarOn EQ-i, an examination of the literature
123
EI and MMI
61
describing the development of the two tools and the construct each purportedly measures, suggests there may be an association between the two. The present study explores the relationship between these two tools to investigate the potential for the EQ-i to serve as a proxy measure to the MMI. This research will also contribute to the validation of the MMI by providing evidence towards its construct validity. An exploration of the relationship between the EQ-i and the MMI will aid our understanding of what is measured through these instruments and provide insight into how the use of these tools may complement one another. Of interest is the comparative ability of both tools to select students who possess the relational qualities needed to succeed in Michener’s newly developed interprofessional education model focused towards enhanced patient safety and patient-centered care.
Instruments The Multiple-Mini Interview The Multiple Mini-Interview (MMI) is a method of selecting candidates for health science programs who demonstrate strong non-cognitive skills. As opposed to traditional interviews, the MMI is made up of a series of interview stations wherein candidates are rated by a number of independent interviewers. The stations are designed in such a way that they do not require or assess specific learned knowledge, but rather evaluate a candidate’s ability to logically work through a problem and express one’s ideas clearly (Eva et al. 2004). Each task station involves: reading a scenario and discussing one’s viewpoint with an interviewer; reacting to a situation played out by an actor; or participating in a collaborative activity with another candidate. Each candidate receives a score based on communication skills, the strength of the argument displayed, and his or her suitability for a health care profession. Raters give a score based on their overall impression of the candidate, as there are no specific ‘‘right’’ answers. A blueprinting process has been established so that MMI administrators outside of McMaster can map other domains or specific curricular fundamentals onto the task stations. Michener utilized this process to develop stations that assess the following core competencies for students: 1. 2. 3. 4. 5. 6. 7. 8.
Communicates Effectively Takes Responsibility for One’s Own Actions Demonstrates Ethical Decision Making Works Effectively in a Team (Inter-professional Collaboration) Solves Problems Using A Variety of Strategies Utilizes Reflective Practice for Personal and Professional Development Manages the Use of Time and Resources to Complete Tasks and Attain Goals Resolves Conflict
The MMI has been found to have an overall test generalizability in the rage of 0.65– 0.81 (Eva et al. 2004), and to significantly predict OSCE (Objective Structured Clinical Examination) performance, clerkship performance (Reiter et al. 2007), and licensing examination scores for undergraduate and postgraduate students (Eva et al. 2009). It has been successfully implemented at a number of other educational institutions and received positively (Brownell et al. 2007; Harris and Owen 2007; Humphrey et al. 2008; Razack et al. 2009).
123
62
W. Yen et al.
The Bar-On EQ-i Developed by Reuven Bar-On, the EQ-i was developed to measure emotional intelligence, defined as ‘‘an array of emotional, personal, and social abilities and skills that influence one’s ability to succeed in coping with environmental demands and pressures’’ (Bar-On 1998, p. 2). The EQ-i is a self-report measure comprised of 125 items, each framed in a 5-point response format. Respondents receive a total EQ (Emotional Quotient) score as well as scores for each of the 5 composite scales and 15 comprising subscales, as outlined in Table 1. Raw EQ-i scores are computer-generated and converted into standard scores with a mean of 100 and a standard deviation of 15, based on a normative population. The psychometric properties of the EQ-i have been well established. The internal consistency coefficient of the tool is .97 (Bar-On 1997), and it has a test–retest reliability of greater than .70 at 6 months (Bar-On 2004). The construct validity of the EQ-i has been supported, as it has been found to correlate with psychological health, social interaction, leadership, school performance, and occupational performance, and has an average predictive validity coefficient of .59 (Bar-On 2006). No significant differences in total EQ scores have been found between males and females or among ethnic groups (Bar-On 1997).
Methods Participants Although every program at Michener requires the ability to interact effectively with patients and other health care professionals, due to constraints, participation for this study was restricted to candidates applying for admission into one of four Michener graduate programs (Ultrasound, MRI, Genetics Technology, and Diagnostic Cytology). A total of 387 candidates were contacted via e-mail and asked to participate in a research study that aimed to examine the relationship between two methods of assessing candidates for Table 1 EQ-i Composite scales and sub-scales
Composite scale Intrapersonal
Subscales Emotional self-awareness Assertiveness Self-regard Self-actualization Independence
Interpersonal
Empathy Social responsibility Interpersonal relationships
Adaptability
Reality testing Flexibility Problem solving
Stress management
Stress tolerance Impulse control
General mood
Optimism Happiness
123
EI and MMI
63
admission (MMI and EQ-i). Participants were asked to complete the EQ-i online before attending their MMI session at Michener. Participation in this study was voluntary. No incentives were provided, and candidates were informed that their participation was not related to their MMI score or their overall admissions standing. Data Data was collected in the spring of 2009 for both instruments. A total of 606 candidates attended the MMI. Of the candidates who applied to one of the four graduate programs under study, 247 had complete data on the MMI and 235 on the EQ-i. Out of these candidates, 196 had data on both MMI and EQ-i. Applicants’ demographic information and Grade Point Averages (GPA) were also collected. The age range of candidates was 21– 55 years. Analyses Three types of analyses were conducted to elucidate the psychometric properties of the instruments. First, Cronbach’s alpha was computed to estimate the internal consistency of the items for each instrument. Secondly, extensive correlation analyses were conducted between these instruments. A key focus of this project was to examine the relationships, if any, between MMI and EQ-I, therefore a total of 17 correlations were conducted: one at the total score level and 16 at the subscale level. To correct for potential Type I error, Bonferroni corrections were taken into account and a significance level of p = 0.003 was utilized. An EQ-i content expert proposed a content mapping between the 15 EQ-i subscales and Michener’s eight core competencies, which were measured through a combination of the eight MMI interview stations. For example, the competency of ‘‘Communicates Effectively’’ was assessed through the MMI interview stations 5, 7 and 8, and by the following EQ-i subscales: Empathy, Social Responsibility, Interpersonal Relationship, and Assertiveness. Given such identification and mapping, the correlation coefficients were obtained between the sum of the station scores that measure a competency and the sum of the subscale scores that measure the same competency. However, it should be noted that the mapping between the competencies and the interview stations/EQ-i subscales was fairly subjective and may need further investigation for future studies. Finally, in order to investigate the ability of the EQ-i instrument to discriminate those applicants accepted by Michener from those who were not, analysis of covariance (ANCOVA) was proposed for EQ-i. At Michener, the decision to accept an applicant was made by assigning equal weighting to the Grade Point Average (GPA) and the MMI total score. Of interest, is whether or not the EQ-i could differentiate those applicants who were accepted by Michener from those who were not. Since people may be more mature emotionally and socially with life experience, it was speculated that age might be correlated with EQ-i total scores. Therefore, age was explored for its potential as a covariate for EQ-i analyses. If it turned out to be significantly correlated with EQ-i total scores, its inclusion into the model could increase statistical power. If the correlation was not significant, a simple t-test could address this last research question of interest. Exploratory factor analyses (EFA) were also proposed to investigate the underlying constructs measured by the MMI and EQ-i, however the criteria was not met so EFA was not conducted.
123
64
W. Yen et al.
Results The MMI had a moderate reliability estimate of .75, and EQ-i had a lower estimate of .65. The MMI total score and the EQ-i total score were not found to be significantly correlated (p = .14), nor were relationships found at the subscale level (p [ .003). The evidence from this correlational analysis suggests that a relationship does not exist between the MMI and the EQ-i. The age (life experience) variable did not turn out to be a defensible covariate for the ANCOVA analysis, since its correlation with EQ-i total scores was not significant. Therefore, a t-test, instead of ANCOVA, was conducted to compare the EQ-i total scores of the accepted applicants with those of non-accepted applicants. The results show that there was no difference in the scores, t(194) = 1.409, p = .16, which questions the discriminating power of the EQ-i test as a pre-assessment tool for the MMI. A post hoc power analysis revealed a power of 0.24.
Conclusion The results of this study indicate that there is no significant relationship between the MMI and the EQ-i at the total or subscale level, and that the EQ-i does not differentiate students selected through the MMI. The potential for using the EQ-i as a pre-screening tool for the MMI is therefore not supported through this research. It should be noted, however, that certain factors may have influenced these results. Firstly, although the use of Bonferroni corrections for multiple correlations decreases the chance of Type I error, it also increases the chance of Type II error. Certain correlations between the two tools may have therefore been overlooked. Similarly, the low statistical power for the t-test conducted might have increased the probability of Type II error, explaining the lack of difference found between accepted and non-accepted candidates. Future research in this area might want to ensure higher statistical power by utilizing a larger sample size and controlling for confounding variables such as selection bias. In addition to exploring the possibility of using the EQ-i as a pre-screening admissions tool, this research was also conducted to provide evidence towards the construct validity of the MMI. The findings for this psychometric analysis, however, are not conclusive based on the above results. The analyses conducted in this study are only a portion of the evidence that can be used to assess validity. For a comprehensive validation of the instruments considered, three other types of analyses are recommended for future studies as articulated by Messick’s validity framework (1989) and the framework espoused by the American Educational Research Association (AERA), the American Psychological Association (APA), and the National Council on Measurement in Education (NCME 1999). First, content experts should be consulted to judge whether the EQ-i measures the construct (i.e., the eight core competencies) that Michener has identified as important for incoming students. Second, the applicants who used these instruments can be interviewed regarding their response processes while undergoing both assessments. This analysis will provide an additional source of data towards the construct validation for the tool by providing insight into performance strategies test-takers engage into determine the fit with the construct of interest. Finally, the predictive validity of the tools can be assessed using the final grades of the accepted applicants. Such a study could help to identify which of these instruments is a stronger predictor of students’ success at Michener.
123
EI and MMI
65
While the present study was conducted as a preliminary exploratory analysis of the relationship between the MMI and EQ-i, it is important to note that there were several limiting factors to the study at the outset. Perhaps the most obvious is the fact that the EQ-i is a self-administered computerized assessment while the MMI is an in-person interview in which the respondent is rated by different interviewers. Self-report measures of EI are problematic in that they tend to assess perceived rather than actual EI, and are susceptible to self-preservation and social desirability bias (Lewis et al. 2005). In this regard, evidence suggests that assessments of emotional intelligence can be faked and that respondents may not answer honestly in high stakes situations (Grubb and McDaniel 2007; Day and Carroll 2008). While the Bar-On model does have a measure to control for fakability, the above findings imply that more research is needed in the area before utilizing such tools for highstakes admissions. We must also bear in mind that while the MMI has been validated for use across various health disciplines, it has not been validated for use with students in the applied health sciences. Caution must be exercised when conducting comparative analyses on a tool that has yet to be validated for its intended use.
Discussion Future studies focused on validating the MMI will need to supplement the statistical analyses of the tool in order to further our understanding of how humanistic and tacit skills can be measured. Of importance is the accumulation of various sources of validity evidence to complement the statistical analyses used to support the intended use of the test and test score interpretation. Addressing such questions of construct validity is essential given the focus on humanistic and tacit skills necessary for enhanced patient safety in the workplace. It will be important for future research to consider the extent to which the MMI predicts success in applied health programs, and more importantly, the extent to which these humanistic and tacit traits are transferable to the workplace when interacting with and attending to actual patients. Are these skills teachable, and if so, which elements are teachable and what should be selected for through admissions processes for our students? How are these skills transferred to professional practice? Such questions speak to the need to also address concerns around the ecological validity of the MMI process, or the extent to which the results of the assessment can be applied to real life situations with patients. More research is also needed to determine whether trait-based or ability-based measures of emotional intelligence are more predictive of non-cognitive success (if emotional intelligence is, in fact, found to be a predictor of non-cognitive success). There is a clear distinction between two different models of emotional intelligence, ability-based models (Mayer and Salovey 1997) and mixed-models or trait-based models (e.g. Bar-On 1997). The ability based model presents emotional intelligence as a cognitive ability and is measured with instruments designed to assess one’s knowledge of emotions and one’s ability to recognize and work through problems involving emotions (Grubb and McDaniel 2007); the mixed model includes an array of capabilities, competencies and skills that influence one’s ability to cope with environmental demands and pressures and are considered to be non-cognitive. Brannick et al. (2009) explored the use of different EI instruments in medical education and found that trait and ability based models seem to actually measure different things. Based on the lack of current understanding around what is assessed through various EI measures, and which would be most appropriate for educational institutions, it would appear that such tools are risky instruments to use for admissions purposes at this point in time. In addition to the findings of the current study,
123
66
W. Yen et al.
this assertion is further supported by recent studies which also failed to find a relationship between EI measures and admissions tests (Carr 2009; Kulasegaram et al. 2009). Despite the lack of support for using EI measures for high-stakes admissions purposes, the current study contributes to the field of admissions research by stimulating discussion and debate around the use of EI tools for selection purposes. Given the increased interest in assessing emotional-social intelligence in organizational and educational settings, and the potential cost-savings of using such tools, it is important to be aware of the limitations and potential hazards of using tools of this nature in higher educational institutions. While the economical advantages are easy to grasp, the literature on the use of emotional intelligence tools for high-stakes purposes, such as human resources selection processes or admissions into educational institutions, is simply not supported at this point in time. Even if a link is found in a future longitudinal study between EQ-i and student performance, it will still not be clear whether or not the tool can be used to make valid admissions decisions. Perhaps the most appropriate use of emotional intelligence tools in an educational setting is one of student learning and enhancement, as opposed to selection. Dulewicz and Higgs (2004) have supported the developmental quality of emotional intelligence in organizational settings, and suggest that EI can be improved with training. In this regard, future research may find an appropriate place for the use of EI measures in education; however, at this point in time it appears that the MMI is the only validated measure of noncognitive skills available for health science admissions. Acknowledgments We would firstly like to thank Paul Gamble (The Michener Institute for Applied Health Sciences), CEO of the Michener Institute for Applied Health Sciences, for overseeing this study. We would also like to extend gratitude to all of the Michener candidates who participated in this study. Finally, we would like to acknowledge the MMI content expert, Lisa Slack (The Michener Institute for Applied Health Sciences), and EI content expert, Brett Richards (Connective Intelligence), for their assistance in mapping the MMI and EQ-i subscales and providing construct expertise.
References American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association. Association of American Medical Colleges. (1998). Learning objectives for medical student education. Medical school objectives project. USA: Association of American Medical Colleges. Baker, G. R., Norton, P. G., Flintoft, V., Blais, R., Brown, A., & Cox, J. (2004). The Canadian adverse events study: The incidence of adverse events among hospital patients in Canada. Canadian Medical Association Journal, 170(11), 1678–1686. Bar-On, R. (1997). The Emotional Quotient Inventory (EQ-i): Technical manual. Toronto, Canada: MultiHealth Systems, Inc. Bar-On, R. (1998). BarOn emotional quotient inventory: Facilitator’s resource manual. Toronto: MultiHealth Systems. Bar-On, R. (2004). The Bar-On Emotional Quotient Inventory (EQ-i): Rationale, description and summary of psychometric properties. In G. Geher (Ed.), Measuring emotional intelligence: Common ground and controversy (pp. 111–142). Hauppauge, NY: Nova Science Publishers. Bar-On, R. (2006). The Bar-On model of emotional-social intelligence (ESI). Psicothema, 18(supl), 13–25. Barrett, J., Gifford, C., Morey, J., et al. (2001). Enhancing patient safety through teamwork training. Journal of Healthcare Risk Management, 21(4), 57–65. Brannick, M. T., Wahi, M. M., Arce, M., Johnson, H.-A., Nazian, S., & Goldin, S. B. (2009). Comparison of trait and ability measures of emotional intelligence in medical students. Medical Education, 43, 1062– 1068.
123
EI and MMI
67
Brownell, K., Lockyer, J., Collin, T., & Lemay, J.-F. (2007). Introduction of the multiple mini interview into the admissions process at the University of Calgary: Acceptability and feasibility. Medical Teacher, 29(4), 394–396. Cadman, C., & Brewer, J. (2001). Emotional intelligence: A vital prerequisite for recruitment in nursing. Journal of Nursing Management, 9(6), 321–324. Canadian Health Services Research Foundation (CHSRF). (2006). Teamwork in healthcare: Promoting effective teamwork in healthcare in Canada. Ottawa, ON: CHSRF. Retrieved from www.chsrf.ca/research_themes/pdf/teamwork-synthesis-report_e.pdf. Carr, S. E. (2009). Emotional intelligence in medical students: Does it correlate with selection measures. Medical Education, 43, 1069–1077. Carrothers, R. M., Gregory, S. W., & Gallagher, T. J. (2000). Measuring emotional intelligence of medical school applicants. Academic Medicine, 75(5), 456–463. Day, A. L., & Carroll, S. A. (2008). Faking emotional intelligence (EI): Comparing response distortion on ability and trait-based EI measures. Journal of Organizational Behavior, 29(6), 761–784. Dulewicz, V., & Higgs, M. (2004). Can emotional intelligence be developed? International Journal of Human Resource Management, 15(1), 95–111. Epstein, R. M., & Hundert, E. M. (2002). Defining and assessing professional competence. Journal of the American Medical Association, 287(2), 226–235. Eva, K., Reiter, H., Trinh, K., Wasi, P., Rosenfeld, J., & Norman, G. (2009). Predictive validity of the multiple mini-interview for selecting medical trainees. Medical Education, 43, 767–775. Eva, K. W., Rosenfeld, J., Reiter, H. I., & Norman, G. R. (2004). An admissions OSCE: The multiple miniinterview. Medical Education, 38, 314–326. Freshwater, D., & Stickley, T. J. (2004). The heart of the art: Emotional intelligence in nurse education. Nursing Inquiry, 11, 91–98. General Medical Council. (2003). Tomorrow’s Doctors: Recommendations on undergraduate medical education (2nd ed.). London: General Medical Council. Goleman, D. (1995). Emotional intelligence. New York: Bantam. Goleman, D. (1998). Working with emotional intelligence. London: Bloomsbury. Grewal, D., & Davidson, H. A. (2008). Emotional intelligence and graduate medical education. Journal of the American Medical Association, 300(10), 1200–1202. Grubb, W. L., & McDaniel, M. A. (2007). The fakability of bar-on’s emotional quotient inventory short form: Catch me if you can. Human Performance, 20(1), 43–59. Harris, S., & Owen, C. (2007). Discerning quality: Using the multiple mini-interview in student selection for the Australian national university medical school. Medical Education, 41(3), 234–241. Humphrey, S., Dowson, S., Wall, D., Diwakar, V., & Goodyear, H. (2008). Multiple mini-interviews: Opinions of candidates and interviewers. Medical Education, 42(2), 207–213. Kooker, B. M., Schoultz, J., & Codier, E. (2007). Identifying emotional intelligence in professional nursing practice. Journal of Professional Nursing, 23(1), 30–36. Kulasegaram, K., Reiter, H. I., Wiesner, W., Hackett, R. D., & Norman, G. R. (2009). Non-association between Neo-5 personality tests and multiple mini-interview. Advances in Health Science Education [Epub ahead of print]. Lewis, N. J., Rees, C. E., Hudson, J. N., & Bleakley, A. (2005). Emotional intelligence medical education: Measuring the unmeasurable? Advances in Health Sciences Education, 10(4), 339–355. Mayer, J. D., & Salovey, P. (1997). What is emotional intelligence? In: P. Salovey & D. Sluyter (Eds.), Emotional development and emotional intelligence, pp. 3–31. New York: Basic Books. McCallin, A., & Bamford, A. (2007). Interdisciplinary teamwork: Is the influence of emotional intelligence fully appreciated? Journal of Nursing Management, 15(4), 386–391. Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103). New York: Macmillan. Quoidbach, J., & Hansenne, M. (2009). The impact of trait emotional intelligence on nursing team performance and cohesiveness. Journal of Professional Nursing, 25(1), 23–29. Razack, S., Faremo, S., Drolet, F., Snell, L., Wiseman, J., & Pickering, J. (2009). Multiple mini- interviews versus traditional interviews: Stakeholder acceptability comparison. Medical Education, 43, 993–1000. Reiter, H. I., Eva, K. W., Rosenfeld, J., & Norman, G. R. (2007). Multiple mini-interviews predict clerkship and licensing examination performance. Medical Education, 41, 378–384. Zwarenstein, M., Reeves, S., & Perrier, L. (2005). Effectiveness of pre-licensure interdisciplinary education and post-licensure collaborative interventions. Journal of Interprofessional Care, 19(S1), 148–165.
123
