Developing Leaders Using Insight From An Ability ...

Developing Leaders Using Insight From An Ability Assessment Of Emotional Intelligence

Paul L Nesbit* Elizabeth King Macquarie Graduate School of Management Macquarie University Sydney, NSW, Australia

*All correspondence to first author Email: [email protected] Phone +61298509908

Emotional Intelligence in the Workplace Conference 29-31 May 2013, Berkhamstead, UK.

ABSTRACT One popular assessment tool for emotional intelligence is the Mayer Salovey Caruso Emotional Intelligence Test (MSCEIT), which measures a person’s ability in EI rather than using subjective self-assessment. While a test that measures ability overcomes the problem of subjective biases inherent in self-report measures it presents its own challenges. In particular, the use of an ability assessment may provide feedback that one’s EI is not well developed and so be confronting to the recipient. Development requires a person to be receptive to the results of feedback in order to incorporate understanding into one’s self-awareness and subsequent development action plan. Thus, leaders in development programs using the MSCEIT may not engage with the feedback or may challenge the validity of the test rather than use the insight for development efforts. This paper discusses the use of the MSCEIT within group based leadership development programs and outlines how a focus on selfreflection is useful to help deal with the issue of negative emotional reactions to feedback. As well MSCEIT data collected over a period of five years data in a variety of group leadership programs is presented. These data are complemented by qualitative analysis of participants’ reports reflecting on the results of the MSCEIT and its implications for their leadership behavior. Together these quantitative and qualitative data support the contention that MSCEIT reports often produce negative emotional reactions that need to be managed in Leadership Development programs. Practical and research implications are also briefly discussed. INTRODUCTION Emotional intelligence has become a hot topic in the area of management. A ‘Google Scholar’ search of articles with emotional intelligence in their title shows over 4,000 papers listed in the last 10 years. A more restricted search of peer-reviewed articles in business related journals found 482 articles in the same 10-year period. Of these 321 (67%) articles were published in the last 5 years. The increasing attention to emotional intelligence reflects the growing importance of emotions in the world of work. Previously, work was seen as primarily a place where reason and logic dominated. However, in recent years, scholars have begun to recognize the importance or emotions at work. For example, decision science research has shown that emotions play a significant role in the way problems are perceived and this in turn influences the nature and type of solutions considered (Zhou & George, 2003). Furthermore, emotions influence the thinking process itself with

slightly negative emotional states more likely to engender critical and detailed analysis of problems whereas more positive emotional states stimulating creative and less critical thinking states (Sevdalis, Petrides, & Harvey, 2007). From these early emotion research studies, research has increasingly demonstrated the central role of emotions play in all human behavior (Salovey & Grewal, 2005). In addition to academic attention, the popularization of EI was further stimulated by psychology journalist Daniel Goleman’s bestseller “Emotional Intelligence” as well as constant attention to the construct in popular and business related magazines (Goleman, 2006). Without the circumspection of claims associated with academic writing, popular perceptions of EI have come to see it to “be the best predictor of success in life, redefining what it means to be smart (Gibbs, 1995). Given this increasing interest in EI generally, its application to leadership development is understandable. Leading others requires skills in managing one’s emotions and the emotions of others, especially in dynamic work environments where constant change and increasing demands for efficiency and strategic innovation places considerable stress on the human resources of organisations. Since leadership development implies a change, an evolution, growth or advancement from one’s current level of performance to a capacity for more complex and sophisticated performance, an important step in development is the identification of the current capacity level (Nesbit, 2012). In measuring current emotional intelligence a variety of assessment tools have been developed to aid this purpose. While tests are ultimately linked to their underlying model of what is meant by EI, one highly regarded assessment is the MayerSalovey-Caruso Emotional Intelligence Test (MSCEIT). Unlike most assessments of EI, which rely on self-reported responses, the MSCEIT assesses a person’s ability in using emotions (Mayer, Salovey, Caruso, & Sitarenios, 2003). However, using the MSCEIT in leadership development is problematic. Given that the MSCEIT can indicate low ability for one’s EI many people find the assessment confronting (Kluger & deNisi, 1996). While leadership development requires a manager to be receptive to the results of feedback in order to incorporate insights about performance understanding into one’s self-awareness and subsequent development action plan, negative feedback also give rise to efforts to protect one’s self-concept (Nesbit, 2012; Trope, Gervey, & Bolger, 2003) . Situations that offer individuals self-relevant feedback often create motivational conflict

between providing information that guides self-improvement and threatens self-esteem (Trope, et al., 2003). Thus information from the MSCEIT assessment may be interpreted as criticism of one’s self-concept (Kluger & deNisi, 1996) and so rejected without consideration of their accuracy or value for development and learning. EI results and other sensitive feedback may be more easily managed when it is given within a coaching relationship, where trust between coach and leader can mitigate negative emotions (Coutu & Kauffman, 2009). However, negative feedback from the MSCEIT may present a major problem when feedback is part of a group level leadership development program. Leaders selected for development programs are typically highly motivated and competitive individuals who are likely to make comparisons of performance and ability with others (Nesbit, 2012). In such situations leaders’ may protect their self-concept by discounting or denying the validity of the MSCEIT data. Thus for group level programs for leadership development, the use of the MSCEIT may be too problematic to entertain. One solution is to utilize only self-report measures of EI but these data are not verifiable and can be impacted by the desire to present oneself in a favourable light (Podsakoff & Organ, 1986). For developmental purposes results that may be impacted by social desirability problems may limit the true picture of a leader’s ability in EI and give rise to unwarranted complacency in further developmental action. Another approach is to structure group level development programs in a way that help leaders marshal their own resources to control their emotional reaction (Nesbit, 2012). The remainder of this paper discusses the use of MSCEIT in a group based leadership development program that seeks to provide an environment where participants are encouraged to see psychological assessments such as the MSCEIT as “data” on which to selfreflect. In the next section nature of the MSCEIT is highlighted to elucidate the contextual background for a discussion of the leadership development program. This is followed by presentation of MSCEIT results collected over a number of years in group based leadership development programs. These data highlight the extent of the potential problem of negative feedback associated with the use of the MSCEIT. Qualitative analysis of reflective reports of a subset of the participants is also presented to show the reactions of participants to negative results for the MSCEIT as well as demonstrate that the reflective course design influences their subsequent reactions and development insights. The paper concludes by highlighting a number of implications for practice and future research.

MSCEIT Mayer’s and colleagues (Mayer & Salovey, 1993; Mayer, Salovey, & Caruso, 2008; Mayer, et al., 2003) work on emotional intelligence provides the foundation for the development of the MSCEIT. They present emotional intelligence as referring to one’s ability and skill in using emotions to support one’s goals and aspirations. Their model of EI is based around four branches or domains. Similar to most models of EI the perception of emotions and their management forms a foundation of the test and constitute two of the EI branches. Thus, the MSCEIT measures the ability of a person to perceive emotions in one-self and in others and also how well people manage their own emotions in their lives as well as the ability to manage other people’s emotions. Additionally, the third branch of EI assessed by the MSCEIT measures how well a person recognizes and uses emotions in cognitive tasks such as decision-making and communication. The fourth branch of EI relates to how much understanding a person has of the way that emotions develop and transition from one emotion to another. MSCEIT tests are all completed online in advance of the leadership development program. Participants are given access information concerning the nature of the MSCEIT test and also advised of the need to complete the assessment in a single sitting. Participants are also advised not to do the assessment when tired or in a high or low emotional state. There are 122 questions in the test, which is conducted, online. The test questions are presented in the form of problems that the person must resolve and the skill in resolving these problems is said to reflect their underlying ability. For example to assess perceiving emotions in others and in test goers are asked to identify the degree that certain emotions are displayed in pictures of people. To assess emotions in oneself, images such as rocks and scenes are shown and test-goers are asked to indicate their emotional reactions to these pictures. Emotional understanding is assessed by questions that asked about the likely emotions felt in certain situations or the likely emotional progression in a scenario. Emotional use is assessed by asking questions about which emotions would be associated with dealing with problems inherent in scenario presented. In this way, the questions present problems that participants must respond to and their responses reflect their underlying emotions. There are eight different sets of sub-tasks in the test, two for each branch. The similarity of answers given by participants to those provided by either a panel of emotion experts – academics who research

in the field of emotions – or to a cohort of 5000 test goers, indicate the correctness of responses and the degree of skill in emotional intelligence. Results provided to the test administrator include test scores for total EI, for each of the four branches of EI, as well as the eight types of tasks that make up the four branches. The test is scaled so that it has a mean score of 100 with a standard deviation of 15. There is also the option of graphical reports (see later discussion). Feedback is presented in the form of “Resource Reports” that are automatically generated after the online test and provided to the test adminstrator. These reports give graphical feedback of performance and discuss results and provide suggestions for improvement. The reports do not include the actual test scores but provide graphical indication of the skill level (expert, skilled, competent, in need of development) for total EI as well as each of the four branches of EI and the eight sub-tasks (Discussion of graphical data is given below).

THE LEADERSHIP DEVELOPMENT PROGRAM The leadership program discussed here has been developed for MBA students within a higher education environment but has been adapted for use in executive education as well. For this paper only the MBA program will be used in discussing design elements. The leadership program forms a component of the MBA curriculum at a leading business school in Australia. Student managers are generally well established in professional careers with a number of years of work related experience. Most students study their MBA part-time. The program is carried out in a block format where students attend for 5 days spread over two weekends. The course is aimed at increasing the self-directed learning capacity of student managers. Specifically, it seeks to increase participant’s self-awareness of their leadership strengths and deficits through an understanding of their performance gaps, which ultimately directs and motivates development efforts in the form of self-generated action plans. Throughout the course self-reflection is encourage – considerable time is devoted to give students an understanding of the nature of self-reflection. They engage in a range of activities, such as working in teams on projects, peer coaching, and undertake a variety of psychological tests. Reflecting on one’s performance at work and during experiential activities plays an important role in arriving at insights about their leadership performance. A

perspective actively encouraged in the course is to see performance and experiences and results of psychological tests as “data” that is turned into insights through the reflection process. In this sense, the potentially emotional laden nature of feedback about one’s performance is explicitly channeled in a way to reduce threats to their self-esteem and selfconcept. Thus one important aspect for the acceptance of MSCEIT results is the degree the program can establish a normative environment for openness and receptiveness to feedback for the purpose of development. While the MSCEIT is carried out online prior to the course commencement its feedback is not be given until this normative environment has had time to be establishment. One way this environment is established is to use the Myers-Briggs Type Indicator (MBTI) to engage participants in reflection of how psychological assessments can be used to increase self-awareness. Unlike the MSCEIT, the MBTI results do not generally incite emotional rejection as they present insights in relatively neutral ways. However, in using the MBTI for self-reflection and in group interactions it aids the process of developing openness among participants as well as establish how results are “data” for further reflection on one’s leadership behaviours. Also prior to giving feedback considerable effort is made to establish the validity the MSCEIT as a measure on EI as well as its relevance for leadership. As would be expected this involves presentation on the content of the four-branch model of EI to show the relevance of each branch to the use of emotions, how the test assesses EI, and answering questions about the MSCEIT. The MSCEIT produces numeric scores for the overall emotional intelligence and for the various sub-sections of the test, however, the test manufacturer does not recommend giving the actual score sheet back. Thus only the graphical representations of results are given in the leadership program. As noted previously competitive individuals generally engage in comparing results. While participants do compare graphical statements, it is considered less problematic than comparing more fine grain numeric results where participants may fixate on thinking that a few points higher or lower is more significant than it is. Ultimately the intention is to get participants to think about the relevance of the results as a stimulus for personal reflection on their ability to use emotions in the workplace.

There is generally a very receptive environment to the MSCEIT feedback. Participants are very interested in receiving their results and are highly focused in reading of the MSCEIT resource reports. After reading, interestingly participants typically are not quick to raise questions about the test in the class situation. However, when questions do arise about the validity of the test, usually in the form of “how do pictures of rocks and water tell me about my emotional intelligence”, there is often a contagion of questions that follow. This situation of critique of the MSCEIT is handled best by careful explanation of the rationale for the assessment tasks. Additionally, the situation is used to remind participants of how we tend to seek protection of self-concept when receiving negative results and that the results are “data” that needs to be reflected on to validate its relevance for development purposes. MSCEIT RESULTS MSCEIT data have been collected for 475 participants over a period of five years from early 2008 to end of 2012. These data were collected from the delivery of 12 MBA leadership development programs (average class size = 27 participants; average age = 34.4 years, s.d. = 7.4 years) and 7 Executive leadership development programs (average size = 21; average age = 40 years, s.d. = 8.4 years) during this period. Data on the makeup of the participants is presented in table 1. Figures in brackets refer to percentage of gender within the course type i.e., MBA or Exec Education.

MBA

Exec Ed

Total

Male

Female

Male

Female

Male

Female

171

157

96

51

267

208

(52%)

(48%)

(65%)

(35%)

(56%)

(44%)

Table 1: Gender for MBA and Executive Education participants.

MBA Exec Ed Total

EI

EI

EI

EI

EI

Total

Perceiving

Using

Understand

Managing

All

M

F

All

M

F

All

M

F

All

M

F

All

M

F

92.4

93

92

96

97

95

97

99

96

95

97

93

93

94

91

(13)

(14)

(12)

(17)

(18)

(14)

(15)

(15)

(14)

(12)

(12)

(12)

(12)

(12)

(12)

96

97

95

97

98

95

100

100

97

99

99

98

96

97

93

(13)

(15)

(11)

(19)

(21)

(14)

(15)

(17)

(12)

(10)

(11)

(9)

(10)

(11)

(9)

94

94

93

96

97

95

100

99

96

96

98

95

94

95

93

(13)

(14)

(11)

(17)

(20)

(13)

(15)

(16)

(11)

(12)

(12)

(11)

(12)

(12)

(9)

Table 2: MSCEIT results Total EI and for four branches (Standard deviation given in brackets). As can be seen from table 2 average scores for the MSCEIT in both MBA and executive education courses is below the expected mean of 100 and have a standard deviation slightly below the expected 15. The actual distribution of scores (not shown) is highly skewed with 74% of scores being 100 or less. The graphical rating is that is provided is linked to scores so that scores below 69 will be given a rating of “need to improve”, 70-90 rated as “consider developing”, 91-109 rated as “competent”, 110-130 rated as “skilled”, and above 131 is considered “expert”. Across all the leadership programs, only 7.5% of test goers received an overall rating of skilled in EI with 51% being rated as competent and 41% receiving a rating of below competent. Clearly, these empirical data highlight the potential for MSCEIT results to be a source of negative emotional reactions to feedback for many participants. QUALITATIVE DATA: REACTIONS TO FEEDBACK At the conclusion of the MBA course students are required to prepare a report that shows their reflections about insights of their leadership strengths and deficits. Analyses of these reports, see representative quotes below, indicate that the MSCEIT does indeed generate substantial discomfort.

“I was actually quite surprised at the results of my MSCEIT Emotional Intelligence assessment… I have found this the hardest part of the course to self reflect on. There are areas where you know you are strong and others that you know you are deficient so when you get the feedback you are not surprised. This one completely surprised me and to be honest I was actually dismissive of the results.” (Female MBA student 1). “The MSCEIT report was, I admit, confronting for me. I really wanted a score that showed me as skilled.” (Male MBA student 1). “Another surprising result I received was from the MSCEIT. Although I received a competent rating for ‘using emotions’ and ‘understanding emotions’, I received a developing score for ‘managing emotions’ and ‘perceiving emotions’ which gave me an overall result of developing. At first I was quiet confused and upset by this rating. I never expected to be rated as an expert in emotional intelligence, however I thought I was at least competent overall.” (Female MBA student 2). These comments are typical of the type of responses that appear in the reflective reports by MBA students. However, analysis of reports also indicates that reflective processes were continued and participants moved beyond negative reactions to gain some insight for their development. This is suggestive that the reflective environment within the leadership programs can help deal with the negative emotional reactions that arise from the use of the MSCEIT. Comments from the same participant reports listed above show this development of insight. “Later that day I wrote in my reflection journal about the positive nature of that conversation and how it would be the start of him setting high performance expectations. Then I had an ah-ha moment where I realised this is where I could focus enhancing my emotional empathy. It is not that I don’t know how to do it. It is that I did not care to in this particular circumstance.” (Female MBA student 1). “I thought the comment in the MSCEIT report asking whether I analyse the situations I am in interesting. I know that I seem to others as very analytical but, I do not practice asking someone why they reacted the way they did. I do not take that analysis through to the next step and get a fuller picture. If I am not valuing the emotional

information then it would not be likely for me seek that information. And, it is becoming abundantly clear that if don’t build on my understanding of emotions and emotional transitions I am not going to improve my accuracy about emotions.” (Male MBA student 1)

“In terms of ‘Managing Emotions’ my score suggested that I might know what works for me, but could struggle in determining what works for others in general. It stated that I ‘may not stay open to emotions, and may therefore be missing some critical information about a situation’. I think this is definitely something I need to be more aware of at work. I don’t think that this is something I can necessarily change overnight because at the end of the day I still need to ensure work is getting done in a timely manner; however it’s a good start in just being alert to the fact now as I can see how a lot of the time I am acting rationally rather than with feeling and this is also when I am most likely to fall into ‘Micro Manager Mode’ which I have been accused of doing in the past.” (Female MBA student 2) IMPLICATIONS AND FUTURE RESEARCH This paper has argued for a focus on self-reflection to deal with the potential issue of negative emotional reactions to feedback associated with the use of the MSCEIT. Reflection is often assumed as a developed skill of participants in leadership development programs however the extent that such skills do exist is unclear. What is clear is that the need for selfenhancement and protection of one’s self-concept may interfere with the learning process in these leadership programs. This is particularly the case when using the MSCEIT as it confronts the leader with an assessment of his or her EI ability. However, one could argue that in development programs generally, potentially negative feedback is always present due to the nature of exposure to development gaps that are endemic to such programs. Thus a practical implication for leadership development therefore is the need for more systematic teaching of reflection within such programs. In addition the attention to the need for management of emotional reactions to feedback within leadership development programs also provides stimulus for research. This paper has highlighted the role of establishment of a normative environment of reflection to aid in managing these reactions. More systematic research on the nature of class dynamics could

assist in enhancing the development of learning environments. Of course the very fact that so many participants in leadership programs rate so low on the MSCEIT is also cause for concern as well as research attention. REFERENCES Coutu, D., & Kauffman, C. (2009). The realities of executive coaching. Harvard Business Review, 6-7. Gibbs, N. (1995). Emotional Intelligence: The EQ Factor. Time Magazine. Goleman, D. (2006). Emotional intelligence: ; why it can matter more than IQ: Bantam. Kluger, A., & deNisi, A. (1996). Effects of feedback intervention on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119(2), 254-284. Mayer, J. D., & Salovey, P. (1993). The intelligence of emotional intelligence. Intelligence, 17(4), 433-442. Mayer, J. D., Salovey, P., & Caruso, D. R. (2008). Emotional intelligence: New ability or eclectic traits? American Psychologist, 63(6), 503. Mayer, J. D., Salovey, P., Caruso, D. R., & Sitarenios, G. (2003). Measuring emotional intelligence with the MSCEIT V2. 0. Emotion, 3(1), 97. Nesbit, P. L. (2012). The Role of Self-Reflection, Emotional Management of Feedback, and Self-Regulation Processes in Self-Directed Leadership Development. Human Resource Development Review, 11(2), 203-226. Podsakoff, P. M., & Organ, D. W. (1986). Self-reports in organizational research: Problems and prospects. Journal of Management, 12(4), 531-544. Salovey, P., & Grewal, D. (2005). The science of emotional intelligence. Current directions in psychological science, 14(6), 281-285. Sevdalis, N., Petrides, K., & Harvey, N. (2007). Trait emotional intelligence and decisionrelated emotions. Personality and individual Differences, 42(7), 1347-1358. Trope, Y., Gervey, B., & Bolger, N. (2003). The role of perceived control in overcoming defensive self-evaluations. Journal of Experimental Social Psychology, 39, 407419. Zhou, J., & George, J. M. (2003). Awakening employee creativity: The role of leader emotional intelligence. The Leadership Quarterly, 14(4), 545-568.