Exploring the complexities of validity claims in Action ...

McInnes, P., Beech, N. and Hibbert, P. (2007) Exploring the Complexities of Validity Claims in Action Research. Management Research News, 30:5 381-390.

Exploring the complexities of validity claims in Action Research Peter McInnes

Department of Management, University of Strathclyde Business School, UK. Nic Beech

School of Management, University of St. Andrews, UK. Paul Hibbert

Department of Management, University of Strathclyde Business School, UK. Abstract Purpose – To explore the problematics of validity that are inherent to the conduct of an action research project because of the disparate language games of both practitioners and academics. Design/methodology/approach – We offer in this paper an exploration of the tensions between different understandings of a research setting at different stages of the research process. Findings – In each phase of the research there are a number of tensions between different epistemological assumptions about the ‘reality’ of the research setting. Validity is not, therefore, about capturing a singular objective picture of the organisation, but rather it is produced through the negotiation of a temporary intersection of language games. Research

limitations/implications

–

The

paper

provides

a

framework

for

understanding the role of the researcher in the research process and the issues underlying validity claims made from different epistemological positions. Practical implications – The paper provides insights in to the mechanisms through which practitioners and academics come to understand each other and the limitations of this knowledge. Originality/value – The article raises awareness of the different normative assumptions at play within a variety of action research contexts. Keywords – action research, language games, validity, interpretivism. Paper type – Research paper

1

McInnes, P., Beech, N. and Hibbert, P. (2007) Exploring the Complexities of Validity Claims in Action Research. Management Research News, 30:5 381-390.

Introduction Action Research is perhaps one of the most popular yet most contentious labels in the field of organisational research. Developing out of Kurt Lewin’s attempt to forge a new approach to social research in which theory generation and change to social systems went hand in hand (Lewin, 1948), Action Research has recently been described as “an umbrella term [describing] a host of activities intended to foster change.” (MacIntosh & Wilson, 2003). Within the span of this umbrella are a variety of styles of research (Bartunek 1993; Greenwood and Levin, 2000; Quoss, Cooney and Longhurst, 2000) whose adherents often differ over such important issues as the degree to which the researcher should identify themselves as such, whether the research agenda of the researcher should be made explicit and indeed how far one should intervene in the organisations (Eden & Huxham, 1996).

However,

unifying

these

approaches

is

an

underlying

philosophy

that

conceptualises Action Research as providing a step-by-step framework for diagnosing, implementing and evaluating a change process. As can be seen in the description above, the egalitarian notion of research practitioners and clients working together in a climate of participation and empowerment is one that is deeply embedded in action research. Indeed one of the most frequently quoted definitions states that: "Action research aims to contribute both to the practical concerns of people in an immediate problematic situation and to the goals of social science by joint collaboration within a mutually acceptable ethical framework" (Rapoport, 1970: 499). As such, action research is a process whereby some of those in the organisation under study participate actively with the researcher throughout the research process from the initial design to the final presentation of results and discussion of their implications (Whyte, 1991). In this respect action research often claims not to privilege the researcher as “expert” over the other participants in the research process (Harrison and Leitch, 2000). For us, however, this liberal agenda is one that obscures the complexities of engaging with both the understandings of different communities of practice within the research setting and the epistemic assumptions of those in the academic community. Indeed, we argue that the heterogeneity of knowledge is more than local misunderstandings of what is “really” happening in the research setting. Rather, the ‘validity’ of knowledge claims derive from the language games in which those involved in the research, including academics and practitioners, participate. In order to make our argument we will, in the next section offer an exploration of the concept of language games and the problematic that such 2

McInnes, P., Beech, N. and Hibbert, P. (2007) Exploring the Complexities of Validity Claims in Action Research. Management Research News, 30:5 381-390. interactions raise for the notion of ‘validity’. This is followed by an exploration of the dynamics surrounding different language games at different stages of the research process. Finally, the concluding discussion explores the notion of validity that underpins different language games both within organisations and the Action Research community itself. It draws particular attention to the issue of leakage of epistemic assumptions between language games.

Language Games and the Notion of Validity We wish to argue that an appropriate way of conceptualising validity as it applies to Action Research can be arrived at via the philosophy of Wittgenstein (1958). In particular his concept of language games can be helpful in clarifying the use of the term ‘validity’. Language games are not playful and do not resemble the normal use of the term ‘game’. Rather, they are conceived of as rule-following processes which are bounded fields of human activity. Within these bounded fields the roles that an actor might enact are prescribed, and their associated activities are both prescribed and proscribed. Of prime importance is the epistemological stance that sees the meaning of terms as deriving from their use in a particular language game. Hence, for example, the word ‘time’ has a particular, and quite specific, meaning in the language game of Physics that may not apply in other language games. Within each language game there are rules about how the idea of time can and cannot be used, and indeed who it can be used by. That is, if a term is used inappropriately the result is either meaningless for those participating in the language game, or the person who made the inappropriate use will be regarded as an outsider who has a different way of making sense of the world. The same term: ‘time’; has rather different meanings in different language games. So, for example, in a language game of Fairy Tales, ‘time’ operates in a different way to its use in Physics. In the language games of competitive sports, such as racing, ‘time’ has another set of uses and meanings. The point of these illustrations is to note the inherent ambiguity in the term ‘time’ and to point out the fallacy in seeking to remove its ambiguity by discovering a singular ‘reference in the world’ (Wittgenstein, 1961) which would fix its meaning. Rather the meaning of a term is constituted in its use by participants in different language games, and these meanings are constructed relative to the other components and rules-for-use within each language game. In this sense, language games are regarded as ways of framing human experience and sense-making (Goffman, 1974). 3

McInnes, P., Beech, N. and Hibbert, P. (2007) Exploring the Complexities of Validity Claims in Action Research. Management Research News, 30:5 381-390.

In some interpretations, language games are seen as ‘sealed units’ between which translation does not occur. Wittgenstein himself thought that philosophical errors were often caused by mistaken transfer between language games. For example, he regarded many of the mistakes in Metaphysics as occurring because people expected metaphysical questions and answers to operate in the same way as those in Physics. Hence, to questions such as ‘what is the purpose of life?’ people expect good answers to resemble factual responses that are found in Physics. This, for Wittgenstein, is a mistake. The question-factual-response rule is part of the physics language game, but not part of the Metaphysics language game. In Metaphysics, one should expect wonderment, faith and a non-fact-based way of understanding. Hence, serious errors occur when people in the Physics language game try to answer metaphysical questions, or when people in the Metaphysical language game seek factual answers to their questions. The argument we have applied to the term ‘time’ equally applies to the terms employed by different parties within the setting of Action Research. Here the language games employed by researchers and practitioners may be quite distinct, and even those used by different groups of practitioners within the research setting itself may vary. The insight that these language games constitute distinct ways of knowing raises some interesting questions around the degree to which people within these language games can understand each other and how any process of translation between them occurs. More generally an important question arises as to the implications of any slippage that occurs between the different epistemological assumptions underpinning the language games of different communities. In order to address these questions we now introduce a structured discussion of the issues around the different language games within a research project.

Four phases in action research In order to explore the different language games involved in action research we have divided the process of conducting research into four phases. We do not suggest that these phases are universal or cleanly bounded, but in the discussions leading to the development of this article we found that they were a useful way of relating and comparing the insights that all of the authors had experienced in their research projects. The first of these phases we have described as ‘a foot in the door’, in which the researcher gains access to the research situation. The second phase is entitled ‘playing the part and 4

McInnes, P., Beech, N. and Hibbert, P. (2007) Exploring the Complexities of Validity Claims in Action Research. Management Research News, 30:5 381-390. being apart’. This describes the phase in which the researcher becomes fully embedded in the research situation, but perhaps struggles to avoid assimilation into one of the realities created by a particular language game. The third phase we have characterized as ‘telling tales’, refers to the process of disseminating the product of research analysis to research participants. Finally, we have labelled phase four ‘the end of the affair’. In this phase the researcher seeks to withdraw from the situation. Each of these phases is described in relation to the experience of one of the authors in a particular research situation. The study that we draw upon here involved research with a national retailer which had recently acquired its main competitor. The research sponsors wanted assistance in monitoring and understanding the dynamics between the two organisations as the head office function of the two organisations was merged into one location. For the purposes of this paper we employed an analytical strategy that saw the individual involved in the research prepare narratives, of significant events that occurred during the study, from field notes. These narratives were then submitted for independent reflection and subsequent questioning by the other authors. Through this iterative process we were able to explore the dynamics of the situation and gain fresh insight into the epistemic assumptions behind different ‘validity’ claims. In order to better structure our discussion, each of the phases we identified is considered in two stages. In the first, we will introduce a vignette describing the situation experienced as part of the research outlined above. Following this we draw out the different language games at work, explore whether translation between language games was achieved and highlight the validity claims made in the process.

A foot in the door Vignette - Gaining research access was greatly helped by the fact that the research sponsors were former colleagues on the University’s MBA class. This did, however, raise some questions concerning memberships and roles of both parties in the research. The opening exchanges of the first meeting were filled with almost conspiratorial chat about the MBA and the people who had taught on the course. The discussion then turned to the organisation’s situation. Here the discussion resonated with terminology from the MBA course to describe the topics of change and strategy, but was also peppered with locations and processes used by the company. While some 5

McInnes, P., Beech, N. and Hibbert, P. (2007) Exploring the Complexities of Validity Claims in Action Research. Management Research News, 30:5 381-390. of these terms were familiar to the researcher, a number required further explanation and the discussion became peppered by interruptions and misunderstandings. Discussion then turned to the research, what was being investigated and what it could deliver. This section of the discussion proved problematic as the subject of study, identity, and the research method, Action Research, were unfamilar to the research sponsors. Consequently the research sponsors struggled to get to grips with what was being proposed and what the output of the study might be. Latterly, the research sponsors agreed to allow research access, but admitted to being interested ‘in how it’ll turn out’ rather than engaged in the process of Action Research.

Discussion: In this episode the tensions between three different language games can be seen to come together. The first language game is of management education. In this game, as Knights (1992) suggests, organisations are constituted in terms of such concepts as markets, core competences and competitive pressures. It is noticeable that this first language game overlapped with the second one, namely the language game associated with the organisation itself. However, while discussing the behaviour of people in the management education language game might involve describing the organisation in terms of culture and change initiatives, this is quite different and distinct from the way in which an interpretivist researcher in identity might conceptualise the situation. This third language game, that of research, not only involves consideration of such issues as research questions and research design, it also entails a reconceptualisation of terms employed in a functionalist perspective. For example, whilst organisational culture might be conceptualised by research participants as a singular entity, it might be reinterpreted by an identity researcher as a fractured and multiple set of practices, norms and symbols that enable and constrain the social actors within different social groupings. The vignette suggests that the intersection of these language games were more or less problematic for the people within the meeting. However, the rapport between the participants, generated from their shared history, could be seen to motivate them towards sustained efforts to translate between different language games. For the gap between the managerial education and the organisational language games, this involved the mapping of the meanings of one game onto the other and the extension of categories to include nuances in local meanings. Therefore the question of the validity of the interpretation of situation

went

unquestioned

as

the

two 6

language

games

shared

a

common

McInnes, P., Beech, N. and Hibbert, P. (2007) Exploring the Complexities of Validity Claims in Action Research. Management Research News, 30:5 381-390. epistemological base. On the other hand, the gap between the language games of managerial education and identity research presented a more complex challenge in which the terms used to describe the organisational setting and the assumption of how one would ‘know’ the setting were raised. Here the translation work involved forging a new joint understanding between the parties.

Playing the part and being apart Vignette – By the second set of interviews the researcher had become friendly with a number of the staff in the head office. The changes had affected many of them, but the junior accountant was perhaps impacted more than others. The changes in the organisation had resulted in her being asked to handle an even greater volume of transaction processing. At the same time she was attempting to assert her newly acquired status as a fully-qualified accountant by moving away from this type of work and towards more analytical tasks. However, the more senior members of the Finance function simply did not recognise her change in professional status and in interviews maintained that there was a clear division between ‘Management Accounting’, as a function that did analysis, and ‘Financial Accounting’ as an area

that

processed

transactions.

Equally,

the

junior

accountant’s

colleagues and peers within Financial Accounting continued to draw upon her expertise in the systems and procedures of the department to resolve issues in transaction processing. Hence this individual seemed both frustrated by her work situation and stressed by the volume of work she was being asked to handle. This resonated with the researcher’s own situation some ten years previous, raising a personal dilemma in terms of whether to intervene, with whom and even whether the problem was a reflection of the junior accountant’s situation, or the researcher’s past experience.

Discussion: It could be argued that the junior accountant, her more senior colleagues and indeed the researcher were part of a language game of professional accounting that attaches notions of career advancement to the movement towards more analytical information. This common language game intersected with more local meanings around the division of work in which the language game of the dominant group establishing an

7

McInnes, P., Beech, N. and Hibbert, P. (2007) Exploring the Complexities of Validity Claims in Action Research. Management Research News, 30:5 381-390. artificial boundary between two areas of accounting activity that held the repetitive and problematic area of transaction processing at arms length. However, while the above is a faithful representation of the research situation as perceived by the researcher, this is itself a product of a particular life narrative and an associated set of theoretical concerns with the use of power. While ‘valid’ in the eyes of the researcher, asserting this as ‘truth’ to the senior accountants may well have precipitated the withdrawal of research access. This seeming incommensurability between different language games can be contrasted with the situation between the language games of professional accounting and those of local organisational practice. Here translation happened in a relatively unproblematic way that meant that the validity of distinctions between ‘financial’ and ‘management’ accounting went unchallenged.

Telling tales Vignette – Output from the research process consisted of a report to management and a number of academic pieces of writing including the researcher’s thesis. The report that was presented to the research sponsors involved a diagram portraying the different identity groups in the organisation, their interactions (or lack of interaction) and points of conflict. The sponsors spent a good deal of time exploring the diagram and explaining where it was, for them, wrong. However, after the meeting the sponsors came to discuss the organisation with the researcher in these terms and they took actions to address the issues identified on the diagram. Indeed, some eight weeks after the meeting one of the research sponsors exclaimed ‘do you know, that picture just summed it up. The amazing thing is we couldn’t see it.’ A similar pattern was repeated with the subsequent review of academic work with initial scepticism subsiding and actions being taken within the organisation.

Discussion: This vignette can be analysed as involving the functionalist language game of the research sponsors coming into direct, and initially oppositional, contact with the language game of the researcher. In this instance the failure to gain acceptance of the diagram was not perhaps because of incommensurability, as in the last vignette, but rather because it made claims upon an epistemic realm, the organisation, that was presumed to belong to the research sponsors. However, this raises the question of how 8

McInnes, P., Beech, N. and Hibbert, P. (2007) Exploring the Complexities of Validity Claims in Action Research. Management Research News, 30:5 381-390. the diagram came to be accepted and applied by these individuals. In this respect we would suggest that the diagram presented an enigma to the research sponsors that called upon them to account for their environment in a radically different way. That is, the mystery of how the other could conceptualise of the situation in these terms generated alternative way of seeing that provided fresh impetuous to managerial action. There is, though, an important distinction between the way in which the researcher and the research sponsors perceived the process of emerging acceptance of the report’s validity. For the researcher the diagram remained a partial interpretation of the research setting through which the research sponsors had come to co-construct their work environment. The validity claims drew not from representational accuracy, but rather from the systematic analysis of interview and observation data. On the other hand, the research sponsors came to accept the diagram as a complete and ‘true’ representation of the organisation whose ‘validity’ helped them respond to the change dynamics in appropriate ways. It is notable that even though the researcher was anxious not to position themself as an expert (Harrison and Leitch, 2000), the realist epistemology underpinning the sponsors’ perspective positioned the researcher as someone who could derive ‘truth’ from the organisational setting.

The end of the affair Vignette – Having worked with the organisation for a period of six months, the research engagement was officially at a conclusion. However, as described above, although the feedback provided to the organisation was initially dismissed by the research sponsors, the researcher was later contacted by them for guidance on how to tackle emerging issues and to request further interventions. While happy to help initially, these conversations became increasingly problematic as the reported actions of different parties within the organisation seemed to indicate that their stance on contentious issues was shifting. the researcher frequently had to encourage the sponsors not to presuppose the categories and relationships that had featured in the previous research. As a result the discussions became increasingly general in nature and unsatisfactory for both parties.

9

McInnes, P., Beech, N. and Hibbert, P. (2007) Exploring the Complexities of Validity Claims in Action Research. Management Research News, 30:5 381-390.

Discussion: The situation is almost completely reversed in the vignette when compared to the narrative in ‘telling tales’. Here it could be argued that the intersection of different language games becomes problematic as the organisational context supporting them continues to change. The presumption of validity that the research sponsors have come to invest in the researcher contrasts with the researcher’s diminishing confidence in the validity of interpretations they can offer. Where the preceding vignettes sought to establish common ground through which a translation between language games could be attempted, the task in this narrative shifted towards preventing leakage between Interpretivist research and the Realist language games of the practitioners. In this respect we can conceptualise the activity as helping the research sponsors to (re)establish themselves as language makers in their own right.

Concluding Discussion As we have seen in the vignette examples, issues of the ‘validity’ of ‘knowledge’ resonate within and between the language games of academics and practitioners. Our analysis has established a number of mechanisms through which the process of translation operates. It has also identified not only problems of a lack of translation between games within the research setting, but also problems of leakage between language games. In this concluding discussion we will develop the idea of leakage as we explore the question of the relationship between ‘validity’ made from Realist and Interpretivist epistemological positions. The language games in and around Action Research are not perfectly ‘sealed’. Rather, they contain traces of voices (Bakhtin, 1981) from other games. These voices create dialogue within the language game. For example, within the Realist epistemology underpinning the position of practitioners the term ‘validity’ typically relates to a rule that prescribes achieving certainty by clear and definite reference to the (objective) world. A term that has clear and strong reference is valid, whereas one that is ambiguous and cannot be directly tied to an element in the world is invalid. Hence, in asking whether or not Action Research is ‘valid’ the question may be framed by a leakage from Realist epistemology. Equally, as was shown in ‘playing a part and being apart’ Realist assumptions founded upon the life experience of the researcher can leak into the process of interpreting the research setting. Where the categories of ‘Financial’ and ‘Management’ accounting were considered by the researcher to be arbitrary, one of any number of 10

McInnes, P., Beech, N. and Hibbert, P. (2007) Exploring the Complexities of Validity Claims in Action Research. Management Research News, 30:5 381-390. possible ways of organising, the researcher’s ‘validity’ claims that these divisions were an objective condition responsible for constraining the junior accountant went unchallenged. The concept of leakage between different language games equally applies to the epistemological assumptions underpinning different types of Action Research itself. These might be seen as separate language games that construct validity differently and hence invoke quite different relations with practitioners. For example, some Action Research is Realist (RAR) and hence seeks to establish causation, proof and determinacy. Other forms are Interpretivist (IAR) (or Social Constructionist) and hence seek multiple meanings and interpretations, and plurivocal constructions of how actors make sense of their social setting. That which functions as ‘valid’ in RAR will not do so in IAR, and vice versa. Hence, ‘validity’ is an essentially ambiguous term in its use in Action Research that will not, and should not, have a singular meaning. We should not be asking ‘what is the reference, or way of fixing meaning, of ‘validity’?’ Rather, we should ask ‘how is ‘validity’ used in this language game?’ and ‘what are the intersecting language games in operation?’ For example, as illustrate in the first part of figure 1, translation might be relatively straightforward between RAR and a functionalist practitioner language game, because of the shared epistemic assumptions. On the other hand, translation between the same practitioner language game and IAR might be problematic. The complexity is likely to be increased where researchers are interacting with more than one practitioner language game. For instance, the research sponsors involved in the ‘telling tales’ vignette may well have had to present their views to others within the organisation. As shown in figure one, even if the research sponsors were split upon the ‘validity’ of results, the re-presentation of results to others may well evoke quite different reactions for different audiences. In such setting, it will be necessary for researchers to be able to read in advance how their research is likely to be read by others, and make a decision on how they want to present the various parties and their research. TAKE IN FIGURE 1 An additional complexity is that when Action Research is written up it is likely to be read through other language games, many of which will be hostile to it. In this sense it is worth considering the epistemological assumptions underpinning the Action Research community itself. These are often subsumed behind the drive to produce pragmatic knowledge that 11

McInnes, P., Beech, N. and Hibbert, P. (2007) Exploring the Complexities of Validity Claims in Action Research. Management Research News, 30:5 381-390. develops new ways of thinking and acting (Reason & Bradbury, 2001); but this leaves epistemic assumptions unexamined and generates significant potential for leakage of realist assumptions into the work in the publication process. As a result when submitted to journals in countries where Realism constitutes the dominant pattern of research, Interpretivist Action Research may well attract criticism about its validity that is inappropriate. In this respect it is hoped that this article goes some way towards clarifying both the distinction and the conditions for conversation between the two.

12

McInnes, P., Beech, N. and Hibbert, P. (2007) Exploring the Complexities of Validity Claims in Action Research. Management Research News, 30:5 381-390.

References Bakhtin, M. (1981), The Dialogic Imagination: Four Essays by M.M. Bakhtin, Holquist, M. (ed.), University of Texas Press, Austin, US. Bartunek, J.M. (1993), “Scholarly dialogues and participatory action research”, Human

Relations, Vol 46, pp. 1221-1233. Eden, C. and Huxham, C. (1996), “Action Research for the study of organizations”, In Clegg, S.R., Hardy, C. & Nord, W.R. (eds.), Handbook of Organizational Research, Sage, London, pp 526-542. Goffman, E. (1974), Frame Analysis: An essay on the organization of experience, Harper & Row, New York. Greenwood, D.J. and Levin, M. (2000), “Reconstructing the relationships between universities and society through action research”, In Denzin, N. & Lincoln, Y. (Eds.),

Handbook of Qualitative Research: Sage, London, pp 85-106. Harrison, Richard T. and Leitch, C. M. (2000), “Learning and Organization in the Knowledge Based Information Economy: initial findings for a participatory action research case study”, British Journal of Management, Vol 11, pp. 103-119. Knights, D. (1992), “Changing Spaces: The disruptive impact of a new epistemological location for the study of management”, Academy of Management Review, Vol 17 no3, pp. 513-537. Lewin, K. (1948), “Action Research and Minority Problems”, in Lewin, G.W. (ed.) Resolving

Social Conflicts, Harper, New York. MacIntosh, R. and Wilson, F., (2003), Publishing Action Research, 19th EGOS Colloquium, Sub-Theme 25: Challenges Faced by Action Researchers in Bridging the Gap Between Micro-Sociological Processes and Desired Macro Changes, Copenhagen, July 2003 Quoss, B., Cooney, M. and Longhurst, T. (2000), “Academics and advocates: using participatory action research to influence welfare policy”, Journal of Consumer Affairs, Vol 34, pp. 47-61. Rapoport, R.N. (1970), “Three dilemmas of action research”, Human Relations, Vol 23, pp 499-513. Reason, P. & Bradbury, H. (2001), Handbook of Action Research: Participative Inquiry and

Practice, Sage, London. Whyte, W.F. (1991), “Comparing P.A.R. and Action Science”, in Whyte, W.F. (ed.)

Participatory Action Research, Sage, Beverly Hills, CA. Wittgenstein, L. (1958), Philosophical Investigations (2nd ed.) Basil Blackwell, Oxford. 13

McInnes, P., Beech, N. and Hibbert, P. (2007) Exploring the Complexities of Validity Claims in Action Research. Management Research News, 30:5 381-390. Wittgenstein, L. (1961), Tractatus Logico-Philosophicus (Trans. Pears, D.F. & McGuinness, B.F.) Routledge and Kegan Paul, London.

14