Evaluating Management Development Interventions: A Discursive Approach
Lisa Anderson University of Liverpool Management School Chatham Building, Chatham Street. Liverpool L69 7ZH
[email protected] 0151 795 3017 Theoretical and Methodological Issues in HRD Refereed paper
Keywords: evaluation; management development; discursive shift; language. Introduction HRD practitioners are under increasing pressure to prove the impact of their work in order to justify their continuing presence and their budgets. In doing so, there is an underlying imperative to be clear about exactly what they are trying to achieve and to offer some form of measurement of their performance. This is particularly difficult given the nature of much of the soft skills development and attitudinal and emotional change initiatives which management development, in particular, encompasses. Both HRD practice and academic communities still heavily rely on a theoretical model of evaluation which celebrates its fiftieth anniversary this year and was spawned in an era when it was when it was quite acceptable for Kirkpatrick (1959) to make the observation that, ‘Most training men agree that it is important to evaluate training programmes’. There are few other areas of management in which we call upon such theoretical anachronisms to guide practice. Here I argue that Kirkpatrick’s model reflects a mechanistic and scientific approach to the practice of management which is outmoded and increasingly rare in modern organisations and that this model of the evaluation of management development, appealing though it is in its face validity, has little relevance and applicability. The premise of this paper is that it may be timely to consider evaluation of management development as a ‘messy’ and ‘fuzzy’ activity in much the same way as several observers have conceptualised management itself (Mintzberg, 1994, Grint, 1997; Shotter and Cunliffe, 2003). Evaluation models and practice are often underpinned by the idea of assessing the value and worth of management development activities, justifying investment and measuring the return on investment (Stewart, 1999). However, the way in which prevailing evaluation models encourage
us to view worth and value is influenced by positivist research traditions which lead to a focus on quantifiable outcomes and which disregard the complex relational and dialogical activity in which most managers engage. I offer an alternative perspective on the evaluation of management development which acknowledges this social nature of managers’ jobs and the centrality of language to effective management practice. I draw on social constructionist approaches to conceptualising management work and learning (Shotter 1993; Shotter and Cunliffe, 2003; Cunliffe 2003) and represent management as a dialogical and relational activity in which action is mediated by language. I also present findings from a small study at a multinational IT services company which sought to measure learning using individual discursive shift within a group of managers who attended a management development event.
Kirkpatrick’s model The literature on evaluation of HRD is dominated by Kirkpatrick and Kirkpatrickesque approaches (for example, Warr, 1969; Whitelaw, 1972; Hamblin, 1974; Blanchard and Thacker, 1999) and there is no distinct and current literature on the evaluation of management development. Donald Kirkpatrick’s (1959) ‘Chain Reaction’ model of evaluation is the approach which most trainers and developers attempt to employ in a range of situations. It is based on the principle that there are five distinct stages of learning and change which can be measured to gauge the impact of the learning programme in a ‘chain reaction’ sequence: Training leads to Reactions which lead to Learning which lead to Changes in job behaviour which leads to Changes in the organisation The model is based on the principle that there are five distinct stages of learning and change which can be measured to gauge the impact of the learning programme. In practice, the chain often breaks down when attempts are made to link a programme’s learning outcomes to the effects on job behaviours and evaluation is then limited to an assessment of reactions through an end-of-course questionnaire and some testing of learning outcomes which can be built into the programme methodology. This may be useful in its own right, particularly as a means of
validating the programme itself, yet it fails to get to the address the question of how managers might change their practice as a result of attending a programme. In this chain reaction mode, evaluation starts with a formative look at the processes used on the programme, moving onto the measurement of reactions – how trainees felt about the programme and how they responded to various aspects of it. These reactions lead to learning, measured against learning outcomes set for the programme; in some cases, evaluators may also try to assess whether there are any unintended learning outcomes. This Level 3 learning should result in individuals changing their behaviour in the workplace (in line with the learning outcomes) which will ultimately lead to changes in organisational performance. The transfer of learning from the training environment is central to Kirkpatrick’s model and based on this, the idea that learning acquired during an intervention can subsequently be deployed in such a way as to have a positive impact on organisational performance (Blanchard and Thacker, 1999) Harrison (2000) combines two models to produce her CIRO framework which suggests what the focus of evaluation should be. This model acknowledges the ‘reactions’ and ‘outcomes’ elements of the Kirkpatrick model and suggests that ‘context’ and ‘inputs’ should also be held up to scrutiny. It provides a more rounded view of where the focus of evaluation should lie but, as with other models, it presents a picture of evaluation as a clear process with discrete steps and measures. The final step in the chain reaction evaluation process is the most problematic of all. This is particularly the case with management development because the way in which organisational impact can be measured is much less quantifiable than for example, trying to assess the bottom-line effect of a compulsory accident-prevention programme in the workplace. The problem lies in trying to disentangle and then reconnect individual learning and organisational performance in a sterile ‘cause and effect’ way rather than acknowledging that a wide range of factors impact on how organisations perform and trying to understand which of these can be affected by management development initiatives. It is as though evaluators have to track a piece of ‘learning’ moving systematically through the organisational system. It is for these reasons that some organisations either do not try to evaluate MD beyond the reactions stage and accept its contribution to organisation performance as an ‘act of faith’ Holton (1996) critiques these ‘chain reaction’ models by identifying them as merely taxonomies or classification schemes and indicating that relationships between each level have not been demonstrated by research. Holton proposes an alternative model which focuses on factors which influence learning such as motivation and the environment and secondary intervening variables such as ‘intervention readiness’ and personality characteristics. Lewis and Thornhill (1994; pp26-27), advocating a culturally related approach to evaluation (after Brinkerhoff, 1988) offer six reasons for the ineffectiveness of evaluation: 1. The confounding variables effect: difficult to disentangle the effect of training from other stimuli 2. The non-quantifiable effect; how do you quantify the effects of, for example, a teambuilding event?
3. The costs outweighing the benefits effect: follow-up evaluation make cost more than the original problem (Buckley and Caple, 1990) 4. The act of faith effect; training is a good thing per se. 5. The trainer sensitivity effect: evaluations may point out trainer weaknesses 6. The organizational political effect: evaluations which show up poor training may indicate incorrect decisions by senior managers. This also provides a rational for not evaluating at all.
Evaluation as a feature of a philosophy of management development The originators of the various chain reaction models implicitly base their work on the idea that management itself is a planned and ordered activity - as classical management theorists such as Fayol (1949) and Taylor (1949) would have us believe. They are also premised on a range of models of individual learning on a particular approach to Management Development and on a particular characterisation of the role of the trainer/developer in the MD process. The notion that evaluation approaches are based on an overall philosophy to management development provides a useful starting point for attempting to theorise and practice evaluation in a new way. For example, evaluation is often seen as an ‘add-on’ to management development practice as if the imperative were for the evaluator to work in a detached and distanced way from a learning programme. This audit approach does not allow for the learning which comes from evaluation to be fed back into the programme in real time. Also, in most cases, the trainer/developer is the evaluator so any attempt to take on the role of auditor is essentially flawed. The table below illustrates a number of conceptions of management development encompassing ideas of how learning may occur, what the relationship between trainers/developers and learners should be, how the organisational frameworks for learning might support learning processes and where the investment focus for evaluation could lie.
How individuals learn Changing behaviour the result instruction
Relationship between trainer/developer/evaluator and learner Trainer in control as as purveyor of knowledge of
Experiential: Consultant cyclical and thoughtful.
Model of learning and development Systematic – structured and planned with clear steps
Dynamic: ongoing reassessment of needs perhaps linked to a competency
Investment focus Clear learning objectives and behavioural outcomes for individuals (‘bottom up’ approach) Business focus; bottom line measures (‘top down’ approach)
Socially – by Partner imitating others and /or participating in communities of practice
framework Integrated into work and management practice.
Holistic focus on individuals and the business
Conceptions of Management Development (Anderson, 2009)
The links between conceptions may seem axiomatic; for example, behavioural approaches to learning seem to link well with other ideas at the same level such as the systematic training cycle. However, this is not always the case and an MD approach could feasibly be closely linked to the needs of the business, based on the experiential learning cycle (Kolb, 1984) and implemented with the purpose of providing information for learning needs analysis (as per the systematic training cycle model). There is ostensibly nothing wrong with a range of purposes and approaches. It would seem sensible, however, to be clear about what these are before designing and delivering learning events. Based on this analysis, a starting point is to have a clear idea of the philosophical basis for management and leadership development activities and for their evaluation. In large organisations, it is possible that individuals hold divergent views which can be surfaced through dialogue. However, small organisations and even one-person consultancies should not assume that they are immune from this tendency to operate without a clear set of principles and concepts which serve as a guide for MD practice. Conceptualising management as practice I would argue that management cannot be reduced to clear set of knowledge indicators, arranged in an orderly framework. This desire to crystallise the essence of the management task into a simple algorithm has been driven by concerns of government, academics and some managers themselves to professionalise management. The MBA is seen as the pinnacle of achievement in the amassing and accreditation of management knowledge. However, there is plenty of evidence to suggest that MBA graduates are far from being the ‘finished article’ despite their accumulated knowledge. Mintzberg (2004) is particularly scathing of MBA providers, students and the systems which surrounds them. He argues that management is neither a science nor a profession but a ‘practice’. He criticises the MBA for its content and its approach to teaching and learning, calling it management by analysis to the exclusion of soft skills development and any discussion of the ethics of being a manager. He dismisses the claim that business simulations replicate the real-world environment as ‘patent nonsense’ and suggests that the case study method reduces management to decision making and analysis, ignoring the tacit dimension of managing. This leads to a ‘secondhandedness’ or ‘thirdhandedness’ in learning which does nothing to prepare students for the realities of managing a business and translates into a ‘banking’ model of education (Freire, 1972) This approach, and that of many leadership and management courses, reflects positivist approaches to researching management as a clearly definable and orderly
activity. Chain reaction evaluations follow this lead and the literature is permeated by a performative, ROI-focussed discourse (see, for example, Huczynski and Lewis, 1980 and Blanchard, Thacker and Way, 2000) I suggest here that there is a need to ensure that management development activities focus not only on what managers need to know and the performance outcomes expected of them, but also on how they need to be. This idea of being a manager draws attention to the central activities of building relationships and promoting dialogue in the manager’s role. This approach to management development is based on the premise that management is more about being rather than knowing: acting rather than simply reading and reasoning. Shotter (1993) talks about a ‘contextualised form of knowing which only comes into being in the course of acting within the social situation within which it is known’ so ‘being ‘ and ‘acting’ are closely linked. Reed (1989) differentiates this theoretical perspective on management from others by terming it ‘management as social practice’ as opposed to the more recognisable technical, political and critical perspectives on management. This takes us back to the set of philosophical choices discussed earlier and emphasises the need for developers and evaluators to be clear about the context in which the evaluation takes place. This notion of context goes further than Harrison’s (2000) suggestion that evaluators pay attention to the organisational, analytic and diagnostic contexts. There should be a recognition of how management happens within an organisation and the powerful role that management development and its evaluation can play in bringing about significant change. Traditional evaluation approaches tend to define the success of a management development intervention in terms of prescribed learning outcomes. This micro-managing of the learning and development process leads to an examination of the minutiae of managers’ behaviour rather than attempting to assess them in a more holistic way. So success in an intervention such as a Coaching Skills workshop might normally be defined in a minute outcome-focused way in terms of participants being able to ask clarifying questions and to explain the difference between formal and informal coaching. From a social constructionist perspective, the same workshop would be evaluated by examining how managers might talk with coachees in a different way 1 and also talk about their coaching experiences with a new insight. This paper explores whether the impact of a learning event on this new way of being in relation to others – the essence of the management task- can be evaluated by analysing the discursive shift in the pre and post learning event situations. Practical authorship Integrated models of HRD have been suggested before (Brinkerhoff,1988; Lewis and Thornhill, 1994; Bates, 2004) yet none of these approaches has used the conceptualisation of management as a relational and dialogical activity as their basis. Central to this idea of social practice is the premise that language mediates action in the workplace and that managers are ‘practical authors’ (Shotter and Cunliffe, 2003) 1
This ‘different way’ would have to be defined by the organisation based on the question, how do we want our managers to be (in relation to others)? I suggest that empathy, good listening habits and a high concern for the individual and the business in the coaching situation could be indicators of success.
Shotter, (1993) acknowledges a move from the modern to the postmodern world and with it, the need to rely less on science and more on human behaviour and experience and on the language which we use to construct our world. Managers work in specific social contexts. Knowledge and knowing are embedded in these contexts rather than existing as theories to be applied at the appropriate juncture. Normative prescriptions pay no attention to the fact that managers cannot always speak and act as they please and that they operate in a ‘jointly constructed moral setting’ (Shotter, 1993). The ‘fuzziness’ of the management environment and the way managers work and learn with others is best explained by a ‘relationallyresponsive’ version of social constructionism (Shotter and Cunliffe, 2003). Understanding, describing and solving problems are all achieved through dialogue: Shotter and Cunliffe (2003) take this notion of dialogue one step further in their adoption of the phrase, social poetics to describe the co-creation of meaning in which managers are engaged. Good managers are aware of the power of language and their role in jointly constructing social reality with their colleagues. The language they use offers a sense of the situations in which they find themselves. Furthermore, through language we and they can gain insights into their ethical and moral stance on issues. Being aware of how dialogues are constructed and examining ‘words-intheir-speaking’ (Shotter and Cunliffe, 2003) can help managers to discover new possibilities and perhaps examine that which was previously taken for granted. There are links between practical authorship and Lave and Wenger’s (1991:47) socially situated view of learning: ‘Conventional explanations view learning as a process by which a learner internalizes knowledge, whether ‘discovered’, ‘transmitted’ from others or ‘experienced in interaction’ with others. This focus on internalization does not just leave the nature of the learner, of the world, and of their relations unexplored…It establishes a sharp dichotomy between inside and outside, suggests that knowledge is largely cerebral and takes the individual as an unproblematic process of absorbing the given as a matter of transmission and assimilation’. Wenger (1998) develops his view of socially situated learning to include the notion of the existence of communities of practice in which individuals in organisations participate on a daily basis. These involve groups of people with a shared interest, coming together to work and solve problems, developing shared mindsets, language and tacit knowledge. In this context, conventional approaches to analysing MD entail an analysis of how much knowledge is ‘transmitted’ from trainer to learner rather than trying to capture changes in the relational aspects of a management role. I propose here that this could be assessed by analysing discursive shift.
Analysing ‘Discursive shift’ In linguistically located studies, discourse analysis is the study of spoken or written language through the examination of syntax, intonation, grammar, register, semantics, linguistic structure, arguments and the social context of language (sociolinguistics). It is used in a wide range of disciplines including applied linguistics,
social psychology, politics, history and sociology. ‘Text’ may be either written or spoken discourse, the latter normally transcribed from a conversation, speech or other forms of spoken language. Text may be analysed using fairly well established procedures and a system of graphic conventions to annotate the text. From a management research perspective, the analysis of texts may entail a process of coding and quantifying the number of times a word or phrase has been used by respondents in, for example, interviews (see Easterby-Smith et al, 1997, p.107). However, a grounded approach in which themes and patterns emerge from the data is more common. Once transcripts of the texts have been made, an analysis can be made of relatively short passages of speech, examining syntax and semantics in close detail or the analysis may entail the examination of large amounts of material, using a system of coding and classification (again, see Easterby-Smith et al, 1997, pp110-112, for an example) Austin’s (1962) work on ‘speech acts’ or ‘performatives’ is underpinned by an assumption that by making an ‘utterance’, language users perform a social act. One of Austin’s examples is that in speaking an utterance, you may perform the social act of making a promise and, as a result, convince your audience of your commitment (Slembrouck, 2002). As management researchers, we are much more interested in the social functions of language and in this study, as an indication of how management is enacted and how managers construct their identity. Fairclough and Hardy (1997), drawing on Foucauldian themes, criticise mainstream linguistics for failing to ‘develop an adequately social view of language’ and propose a method for analysing discourse which encompasses an examination of ‘text’, ‘discourse practice’ and ‘sociocultural practice’. Discourse analysis for Fairclough and Hardy is ‘the process of identifying all the genres and discourses that are drawn upon in producing or consuming a text and the particular way they are combined together’. They focus on ‘ideational, interpersonal and textual functions of language’ by an examination of nominalization, pronouns, vocabularies, metaphors, mood (declarative, interrogative or imperative) and modality features. Discursive shift is defined by Rossiter (1994: 4) as ‘A change in the language and practices which construct identity’. This helps to clarify what I am trying to capture in analysing discursive shift as an evaluation technique and in rejecting Fairclough and Hardy’s (1997) approach to discourse analysis. Case Study This small study focuses on a management development intervention at a multinational IT services company. It sought to measure learning using individual discursive shift of a group of managers who attended a High Performing Teams event, a 2 day course which all employees at management level in the company were expected to attend. The event focused on the characteristics of successful teams and team leaders and was run in a conventional facilitative fashion, drawing on participants’ experience as team leaders and managers. The purpose of the study was to examine if and how managers talked about themselves and their role differently as a result of attending the learning event. Eight managers were interviewed over the telephone both before and after the event. They were asked to talk about their role as a team leader, what they enjoyed about
teamwork and the contribution of teams to business success. Questions were broadly similar pre and post event but post event interviews included questions about new ideas that they had gained from the programme. Transcripts of interviews were analysed in two ways; first through the use of a scalable and automatic corpus linguistics technique namely WMatrix software which allows a macroscopic analysis to inform the microscopic analysis as to which textual features should be investigated further (Rayson, 2003, 2008) and second, through an analysis of transcripts using a grounded theory approach. Turnbull (2002) evaluates the shift in beliefs, attitudes, behaviours and emotions expressed as a result of managers attending a corporate change programme by thematically analysing participants’ accounts. In that study, Turnbull explains that efforts were made to use participants’ own words as much as possible but that this was not always possible and that it was necessary ‘to act as interpreter and to make linguistic choices to express … participants’ … emotions’ (Turnbull, 2002:27). In the first stage of the study described here, a decision was made to analyse words and phrases used by managers by dint of the use of corpus linguistics software which indicated which areas of text might warrant a subsequent, deeper analysis. WMatrix software enables the user to count words, phrases and concepts used in a particular text. Significant key words and concepts are compared against standard profiles from corpora of spoken English. Results are shown statistically as the deviation from the standard frequency. (Rayson, 2003). In this study, we also compared managers’ pre and post interview data against each other to look for significant differences and we also analysed data generated by a question about new ideas which participants had gained from the programme. We used the frequency list of words not as data in their final form but looked at the use of those words in context. We engaged eight managers in conversation (in the form of telephone interviews) about their roles both before and after a learning intervention to assess whether their ‘talk’ had changed and by implication, had been influenced by their experience being a participant. The basis of this approach is that learning interventions may be catalysts for change and that the analysis of discursive shift in managers can illustrate how they might talk in a different way as a result of that learning experience. This ‘talk’ includes conversation and dialogue with others or what we might term ‘talking out loud’ it could also include inner conversations. Findings from the WMatrix analysis. The findings from the WMatrix key word and concept analysis of pre and post course data indicated that changes in pre and post-course talk occurred in the following broad areas: 1. 2. 3. 4.
How managers talked about their teams How they described their approach to working with teams The way in which they talked about customers Their understandings of leadership
Using this as an indicator of where it might be appropriate to dig deeper into the data, I then examined the transcriptions to find examples of these themes in the words spoken by respondents. Examples are given below. The nature of teams and teamwork Pre course: “…I lead different teams it is quite an unusual role at times…but if they are having problems with part of the solution or they need help actually getting the solution to work and guidance over what their part of their solution is in the overall picture that is where … my team leadership [comes in]” “We don’t sit together; we are scattered all over the country; sometimes all over the globe but very very virtual … it is not a case of I am their line manager I would say it is more informing them and telling them what we are trying to achieve and then helping them achieve it for the team” “Well a collection of individuals I don’t believe can hold or achieve as much…than a team whereby the objectives are set for everyone and the end point is known…when people belong to a team then they feel they belong to something” Post course: “…there are project managers as well as the technical links depending on what they are doing depends who they report to so I look after the technical side but also I work very closely with the project and programme management to make sure that they are finding the exercise is going to work and that there is a broad understanding of what we are trying to actually achieve technically and from a service perspective” “I think being flexible, working together, feeling ownership. I think they have lacked ownership recently. People have become a bit divorced from it. Now I am seeing people pick up ownership again and wanting to get things working. Feeling like they make a difference …it is good to see people getting on and begin to fix things” “You know you are not isolated you have got the communication you have got the dialogue, the challenge of working with other people. Whereas if you didn’t work within a team it would be pretty isolating really. You pick up ideas from other people, you learn from other people year from year it is very rewarding”
Managers’ own approach to team leadership Pre-course: “… we get together on a monthly basis just for a face to face …[there’s] not a great deal of socialising going on” “I am pushing my own agenda and pro-actively trying to add value to the business developing new offerings. Often they are being demanded of me and so the expectations I place on myself are often of my own doing so I am not sure that I can let myself off the hook quite so easily”
“We have regular chats on the phone almost daily but often my diary is so hacked that I am rushing on the calls and I am aware that some would like to linger on the call just for the sake of socialising but I rush off the calls because my agenda is so full but I sometimes pull myself up and take myself and say hang on these people should be the most important thing on my agenda”
Post-course “Having other people to talk to, to bounce ideas off. To learn from their experiences and also they can learn off me. There is a lot of coaching and mentoring goes on it goes on both ways”. “Being able to listen to your team; being able to persuade them if that is what you feel needs to be done; being able to listen to what they are saying and … to change what your thinking is”. “I think being flexible, working together, feeling ownership. I think they have lacked ownership recently. People have become a bit divorced from it. Now I am seeing people pick up ownership again and wanting to get things working. Feeling like they make a difference …it is good to see people getting on and begin to fix things”
Attitudes to customers The Wmatrix analysis showed a quantitative increase in references to customers in the post-course data compared to pre-course: “… we have got a massively changing organisation and you can’t stick to the rules you know rules are there as guidelines…. … we all have to remember that we are here to support our customers and it is very easy for people to forget that. You are your internal world sometimes that you are working in and relating it how is the decision that I am making going to help us with” “…Well good teams will always deliver on time or before time. But more than that they will always question what they are doing most of the time. In other words they are always looking, each individual member, team leaders, managers they are looking to make what they do much better in terms of service to their customers and so that could be internal or external customers” . “… Because of the varied systems and my role these days is pulling them altogether and making sure that the interactions between the systems do what they need to and trying to take the big picture view of the overall design which is where the design leadership … role lie … kicks in. To make sure all the individual bits need do what they need to do to deliver what the customer needs” …”between what can be a very detailed technical team and a non technical customer. So what I am looking into is also interpreting the design team need in a language the customer can speak. Again it is that facilitating, nurturing, coaching kind of role”
The nature of leadership Pre-course: “I want them all to contribute and feel valued; what I won’t do is put anybody down… [But]I don’t tolerate fools… everybody has got their own opinion but if they are constantly being negative or just don’t want to do the job then you know I will move them on but I would rather have people who are far more willing to do the job or willing to learn or at least showing effort” “Hopefully I am much more into carrot that stick motivation although I think … I am aware I would like to reward my staff more however I sort of sit here hoping that they will deliver something brilliant to me but there has not been that much of [that]; our stuff that hasn’t has sort of really hit targets in that respect” I have got one where he is working with another developer delivering through to me I really don’t get very involved in that other than kick them into touch if things aren’t going nicely or you know let my disappointment be known. I don’t think I kick him very hard”.
Post-course: “I am flexible with them and they are flexible back and I try and give people opportunities and I have seen a lot of the people that have worked for me move onwards and upwards and outwards and I think that is good … I have been supportive of that” . “ (my current boss is) … quite exceptional I think as a leader of a team and the thing I take from him watching him is that he really does accept people for who they are and he tries to help them do their best and when they messed up he just helps them get out of the mess… he never looks for people’s weaknesses he is always trying to help them to do their best” “[They need to} see the big picture so they need to have a kind of strategic view so that they can see how their efforts fit into the wider context and when I say their efforts I mean the team’s effort … They need to be self aware to understand themselves, their own strengths and weaknesses and they need to be able to see to the heart of people to understand first of all what people’s competencies are or their [strengths] … but then also what a person’s motivations are … If they can do all of those things they can see the context in which they are working; the goals that they should be aiming for; their own strengths and weaknesses and the strengths and weakness of those people they are leading. They should be able to bring it all together to lead and deliver” “It is about treating them with the respect they deserve … the knowledge they have got and what we are getting out of them”
Discussion
Even though this was a relatively small study, it is possible to see changes in language use as an indicator of changes in behaviour and attitude as a result of attending a learning programme. It is reasonable to conclude that managers who attended this course changed their perspective on the nature of teamwork itself, particularly how shared understandings, dialogue and ownership contribute to effective practice. There is also evidence to suggest that their leadership practice may have changed as the discursive shift between pre and post-course data illustrates a softer, more people-centred approach to team leadership. The references to the team’s impact on the customer increased after managers had attended the programme; the word was used more and an examination of transcripts showed that there was a greater awareness of the role of effective teamwork in delivering on and exceeding customer expectations. The way in which respondents talked about leadership also changed from them referring to ‘not tolerating fools’ and ‘kicking into touch’ to a more humanist approach of accepting people for who they are and helping team members to understand strengths and weaknesses and treating them with respect. It is important to note that these discursive shifts were observed across the group as a whole and that no attempt was made too gauge individual changes in the eight respondents. Traditional evaluations tend to concentrate on the minutiae of learning outcomes of individuals and then try to piece this back together to form a coherent collective picture. This approach is based on a holistic view of the cohort of learners, conceptualising the learning event as an intervention which has the capacity to help them shape their identity as team leaders, evidenced by discursive shift. Somers (1994) suggests that: ‘It is through narrativity, that we come to know, understand, and make sense of the social world and it is through narratives that we constitute our social identities’ (1994:606) In this subtle reinvention of groups of team leaders, there also exists the possibility for organizational change, based on a new way of speaking about roles and relationships and the organization itself. The idea that language can be examined to uncover organizational identity claims is not new (Ran and Duimering, 2007)
Conclusions I have sought to provide a brief illustration of how an examination of language use can provide indicators of behavioural and attitudinal change in managers. This proposed method of evaluation goes some way to addressing the criticisms of chain reaction models which play down the relevance of the social and contextual nature of management and the centrality of language in a manager’s role. There is also an argument for making managers more aware of their language use as a means of improving managerial practice: Shotter and Cunliffe (2003) believe that managers who understand the social constructionist nature of their meaning-making actions are likely to be more aware of ethical and moral considerations. Kirkpatrick and others, recommending evaluation in an essentially retrospective manner – unpicking the various threads of learning to assess how they came to be offer too simple a proposition but one which nevertheless has an alluring veneer of
science . Evaluation has also been typified as an exercise in proving or disproving that something worked rather than providing the basis for further learning and of a dynamic approach to programme development. It is difficult to completely separate evaluation, learning programmes and job behaviour in such a sterile manner and this paper has hopefully provided some insights as to how we might change this organizational practice by suggesting how it could be re-theorized.
References Anderson, L. (2009 forthcoming) ‘Evaluation’ in J. Gold and R.Thorpe (eds) Handbook of Management Development. Farnham: Gower Austin, J. L., 1962. How to Do Things with Words. Oxford: Oxford University Press. Bates, R. (2004) ‘A critical analysis of evaluation practice: the Kirkpatrick model and the principle of beneficence’ Evaluation and Program Planning. Vol 27; pp 341-347 Blanchard, P.N. and Thacker, J.W. (1999) Effective Training Systems, Strategies and Practice. New Jersey: Prentice Hall Blanchard, P.N, Thacker, J.W. and Way, S.A.(2000) ‘Training evaluation: perspectives and evidence from Canada’ international Journal of Training and Development. 4:4; pp295-304 Brinkerhoff, R.O. (1988) ‘An integrated model for HRD’ Training and Development Journal. February: pp 66-68 Buckley, R. and Caple, J. (1990) The Theory and Practice of Training London: Kogan Page. Cunliffe, A. (2002) ‘Reflexive Dialogical Practice in Management Learning’ Management Learning. Vol. 33(1): 35-61 Fairclough, N. and Hardy, G. (1997) ‘Management Learning as Discourse’ in J.Burgoyne and M. Reynolds (eds) (1997) Management Learning. London, Sage. Fayol, H. (1949), General and Industrial Management, Pitman, London Freire, P. (1972) The Pedagogy of the Oppressed. Harmondsworth: Penguin. Hamblin, A.C. (1974) Evaluation and Control of Training. Maidenhead: McGraw Hill Harrison, R. (2000) Learning and Development London: CIPD Holton, E.F. (1996) ‘The Flawed Four-Level Evaluation Model’ Human Resource Development Quarterly. Vol 7, no 1: pp 5-21 Huczynski, A.A. and Lewis, J.W. (1980) ‘An empirical study into the learning transfer process in management training’ Journal of Management Studies. May, pp227-240 Grint, K (1997) Fuzzy Management. Contemporary Ideas and Practices at Work Oxford; Oxford University Press. Kirkpatrick, D. (1959) ‘Techniques for evaluating training programmes: Parts 1 to 4’ Journal of the American Society for Training and Development. Kolb, D. (1984) Experiential Learning. Experience as the Source of Learning and Development. London: Prentice Hall.
Lave, J. and Wenger, E. (1991) Situated learning. Legitimate peripheral participation. Cambridge; Cambridge University Press. Lewis, P and Thornhill. A (1994) ‘The evaluation of training: an organizational culture approach’ Journal of European Industrial Training. Vol 18, no 8. pp 25-32 Mintzberg, H. (1994) ‘The fall and rise of strategic planning’. Harvard Business Review Jan/Feb94, Vol. 72 Issue 1, p107-114, Mintzberg, H. (2004) Managers not MBA’s. Harlow: Pearson Mumford, A. (1989) Management Development. Strategies for Action. London: IPM Ran, B and Duimering, R. (2007)’Imaging the organization: language use in organizational identity claims’ Journal of Business and Technical Communication. Vol 21, pp 155-187. Rayson, P. 2003 WMatrix: A statistical method and software tool for linguistic analysis through corpus comparison. Ph.D. thesis, Lancaster University. Rayson, P. (2008) Wmatrix: a web-based corpus processing environment, Computing Department, Lancaster University. http://ucrel.lancs.ac.uk/wmatrix/ Reed, M. (1989) The Sociology of Management’. Hemel Hempstead: Harvester Wheatsheaf. Reid and Barrington (1999) Training Interventions. London: CIPD Rossiter, A, (1994) ‘Chips, Coke and Rock-‘n’-Roll; children’s mediation of an invitation to a first dance party’ Feminist Review. No 46: pp1-20 Stewart, J.(1999) Employee Development Practice. London: Financial Times, Pitman Shotter, J. (1993) Conversational Realities: Constructing Life Through Language. London, Sage. Shotter, J. and Cunliffe, A (2003) ‘Managers as Practical Authors: Everyday Conversations for Action’. In D.Holman and R.Thorpe (eds) (2003) Management and Language. London: Sage. Somers, M.R. (1994) ‘The narrative constitution of identity: A relational and network approach’ Theory and Society. Vol 23: 605-649 Taylor,FW (1947), Scientific Management, New York: Harper & Row. Turnbull, S (2002) ‘The planned and unintended emotions generated by a corporate change programme’ Advances in Developing Human Resources. Vol 4, no 1. Pp22-38 Warr, P. (1969) ‘Evaluating Management Training’ Journal of the Institute of Personnel Management. February Wenger, E. (1998) Communities of practice: learning, meaning, and identity. Cambridge: Cambridge University Press, Whitelaw, M.(1972) ‘The Evaluation of Management Training – a Review’ London:IPM
