USING PARETO-BASED SCIENCE TO ENHANCE KNOWLEDGE FOR PRACTICAL RELEVANCE Hind Benbya Department of Information Systems & Decision Science, GSCM, Montpellier Business School, France Phone 04-67-10-28-19; Fax 04-67-45-13-56;
[email protected]
& Bill McKelvey UCLAAnderson School of Management, 110 Westwood Plaza, Los Angeles, CA 90095-1481 Phone 310-825-7796; Fax 310-206-2002;
[email protected]
May 13, 2007 Copyright. All rights reserved. Not to be quoted, paraphrased, copied, or distributed in any fashion.
ABSTRACT There is a growing list of scholars who think there is something fundamentally wrong with organizations studies/science. This appears due to a break in the knowledge food-chain created by the discipline-centric nature of academic research in B-schools vs. the cure-based approach in medical research. Medical research is aimed at cures. B-school research, for the most part, is not. It is as simple as this. Discipline-push vs. cure pull basis of research legitimacy is discussed next. Then focus turns to the idea that whereas studies of ‘averages’ is what gains statistical legitimacy and academic recognition in business schools whereas practitioners worry more about positive and negative extremes: they get promoted when there are positive ones and fired when outcomes are dramatically negative. What counts as useful knowledge emerges by comparing organization studies with earthquake science. The current difference is dramatic – the most fundamental difference is that disciplines each apply discipline-centric theories to different levels of organizations – psychological studies of workers at the bottom; economic studies of industries at the top. Complexity science suggests that causal dynamics are scalable; that is, like causal dynamics in cauliflowers, the same dynamic works at multiple levels. Finally a number of changes in organizational research methods are suggested. Pareto-based science is a better alternative to many of the failures of modernism than postmodernism.
2
Scholars have for decades expressed their increasing concern about the chasm between theory and practice (Peters & Robinson, 1984), criticizing the ‘gulf between basic research and practice’ (Whyte, 1983: 502), and pointing to a crisis that results from the fact that ‘the findings in our scholarly management journals are only remotely related to the real world of practicing managers’ (Susman & Evered, 1978: 582). More recently business schools have been criticized for producing ‘academic’ research that fails to lead to, or generate useful practices, and for failing to impart practical and useful skills to their students (Pfeffer & Fong, 2002; Bennis & O’Toole, 2005; Ghoshal, 2005). These debates have recently appeared in the literature as: choices between scientific rigor vs. practical relevance; emphasis of rigor over relevance; or trying to achieve a certain balance between the two. Some recommend that researchers should concentrate more on real world problems as a way out of the problem. Other scholars believe that increasing relevance of management research is only possible at the expense of scientific rigor. Still others argue that these views are complementary rather than opposites (Eden & Huxhan, 1996). The CALL for the Crete Conference and Van de Ven and Johnson (2006) both consider ‘co-production’ as a possible solution. McKelvey (2006), in his critique, points out various weaknesses in this approach – action research has attempted this in the past with little success; practitioners may not want academics gaining privy to trade secrets; practitioners want results they can use tomorrow whereas good research takes time, and so on. Our approach zeros in on ‘what counts as valid knowledge’ and ‘methodological issues’. We don’t claim to offer a complete solution, but we believe our solution pertains to a significant portion of the academic-practitioner divide. Our belief is that the integration of academic vs. practitioner knowledge perspectives is far from an elusive ideal. In fact, it is pretty close at hand, but does require a change in what academics study and how they do it – different phenomena studied, with different methods, and perhaps even a change in incentives. We argue that the cause of the rigor-relevance gap is not the clumsiness of individual researchers but rather in differences about what offers real value to practitioners as opposed to academics: 1. Why would academic scholars spend time and energy producing actionable knowledge that can in turn be used in practice when they are in fact rewarded for doing the opposite – for producing discipline-centric research findings publishable in discipline-oriented journals? 2. Why would practicing managers consider applying knowledge produced by academics that is not relevant to the problems they face in the world they live in?
We highlight some fundamental differences between the two perspectives so that we can then be more obvious in how we integrate them. First, what counts as valid knowledge is different: • The Academics’ Perspective Academic research relies mainly on quantitative methods that rest primarily on Gaussian statistics, assuming normal distributions, eradication of outliers, and ‘robustness’ methods for translating weird distributions into ‘normal’ ones (Greene, 2002; McKelvey & Andriani, 2005). There is a strong incentive for researchers to continue this ontological assumption as their promotion, tenure, and evaluations are based on publications in refereed academic journals but not in practitioner journals. • The Practitioners’ Perspective Practitioners live in a world of extremes not averages – Toyota, eBay, Google, Southwest, Wal-Mart, and GE are good; Alitalia, Enron, Anderson, WorldCom, Lucent, and the FBI are bad. All of the case studies used in M.B.A. classrooms are stories about good and bad examples – extremes, never averages. There is nothing in an ‘average’ that tells a company how to have a competitive advantage. Yet, this is the ‘knowledge’ they get from all academics (McKelvey, 2006).
Managers live in a world of hypercompetition where companies spend billions each year on consultants in an effort to acquire and use knowledge – concrete recommendations and synthesis – at a time when companies and their leaders are seeking every possible competitive edge and where business leaders are rewarded with both money and success to make effective decisions. One might expect these decisions to be based on evidence that follows the same scientific rigor used for technical or medical issues. Yet, they are not based on facts and rigorous analysis but rather on conventional wisdom, history, ideology, or assumptions (Pfeffer & Sutton, 2006). Second, new methods are required to produce good science about extreme events. We are not proposing academics abandon academic rigor underlying their ‘truth claims’ about what
3
practitioners should do. Instead we take lessons from complexity and earthquake sciences, small-sample research methods, coherence theory from hermeutics, and agent-based modelling. Our approach is novel, bold, and likely controversial. Yet is it based on solid principles from other sciences. It is time to get beyond the normal science stranglehold of neoclassical economics so as to do good science about the world practitioners actually work in. We begin by pointing to the broken knowledge food chain. Next we consider what counts as knowledge useful to practitioners. Then we detail some of the differences between Gaussian- and Paretobased ‘cure’ research. This leads into our discussion of a Pareto-based search for managerial and organizational cures.
1
THE GROWING REALIZATION OF KNOWLEDGE & B-SCHOOL FAILURE
The 1955 Carnegie Commission Report set business schools off on the trend toward discipline-based quantitative research and away from aging and/or retired executives coming in to talk about their personal experiences and tell ‘war’ stories. At the time this was a very well-taken move. But this was some 50 years ago. Now we have discipline dominance and different perspectives prevail and a considerable literature has developed about the problem and what to do about it. Based on recent articles by Van de Ven and Johnson (2006) and McKelvey (2006) we boil it down to three perspectives. and now 1.1 THE BROKEN FOOD CHAIN VIEW OF KNOWLEDGE TRANSFER Van de Ven and Johnson see the positioning of management research as equivalent to the positioning of engineering relative to the physical sciences and medicine relative to the biological sciences (p. 18). Knowledge production and consumption are not unlike the biological food chain. At the left end we have, say, mosquitoes; at the right end we have T Rexes. Reading from the left we see the production of ever larger and more complex creatures; reading from the right we see the consumption of ever-smaller kinds of animals/plants. As Van de Ven and Johnson describe the knowledge food chain, ‘…knowledge is created and tested by academic researchers, taught to students by instructors, adopted and diffused by consultants, and practiced by practitioners’ (p. 9). In earthquake country the engineering food chain looks like this: Physics Earthquake Science Engineering City Building Code Departments Builders Buyers
Medicine looks like this: th
st
Biology Medical Research Medical Schools PhDs/MDs 4 - to 1 -level Hospitals1 GPs Patients
We would describe the B-school food chain as follows: Disciplines Management Research PhD/MBA Students Consultants Practitioners
Food chains can be read from either direction: Thus in life science, the discovery of DNA eventually leads to new molecules in drugs that cure patients. Oppositely, the increasing prevalence of Alzheimer’s disease in patients leads to biologists doing stem cell research. Arguably, B-school research is increasingly held hostage to the epistemology of basic disciplines – a problem. On the other hand, we have the following quote: The only way we can make our field more useful is to start doing – and rewarding – work that can be read and applied by business people. (Davenport, quoted in Lytras, 2005)
The broken food chain view summarizes as follows (from McKelvey, 2006): 1. 2. 3. 4.
1
Academic research isn’t put into a form that can be applied in practice; Little attention is paid to the transfer problem; Researchers don’t take responsibility for knowledge transfer; Authoritarian and coercive styles of imparting knowledge, defensiveness by teachers and researchers, and self-
Hospitals and physicians are ranked according to the rareness of illnesses they are supposed to recognize quickly. Normal GPs are at the bottom. In a quaternary hospital (e.g., Johns Hopkins, UCLA), in contrast, physicians are supposed to see the most rare illnesses often enough that they become skilled in diagnosing them without making mistakes.
4
5. 6. 7. 8. 9.
interested recommendations by consultants inhibit the flow; Academic research interpretation fails because researchers don’t collaborate with practitioners; We know little about what makes research useful; We don’t appreciate just how much knowledge changes as it goes through the transfer process; We don’t understand (Aristotle’s) art of persuasion; Researchers don’t take time to, appreciate or understand the context of the practitioner.
1.2
THE TWO SEPARATE SCHOOLS VIEW A second school Van de Ven and Johnson review holds that academics and practitioners live in two different knowledge worlds. Expecting knowledge to flow left to right is like expecting round pegs to fit square holes – no wonder we have a knowledge-failure problem. Since we already know very well how academics do research and produce knowledge, this school starts at the ‘right’ end. How do practitioners, and how should academics, learn about practitioner problems? Beginning with Kondrat’s (1992) work, This school takes a no-flow-needed stance. Each kind of knowledge exists, but this school writes off the left end as unfathomable, though not necessarily irrelevant. Left unanswered is: If practitioner knowledge is independent, validly produced, and useful to practitioners, why should B-schools bother with academic knowledge? Do it just to look good to promotion committees at universities? Keep journal editors happy? Do it even though it appears to have little, if any, practical relevance? Now one can see why the debate is heating up (March, 2000; Weick, 2001; Grey, 2001; Kilduff & Kelemen, 2001). The ‘no-flow-needed’ view is summarized by McKelvey (2006) as follows:
1. 2. 3. 4. 5.
What knowledge does a practitioner actually use and how do they obtain it? How do they construct and action? What do competent practitioners know and what do they know about knowing? The ‘knowledge-transfer’ school privileges academic knowledge and devalues practitioner-created knowledge; Practitioner knowledge is a distinct form of knowing in its own right; In Aristotelian terms, phronesis (practical knowledge) is just as important as episteme (basic knowledge) and techne (applied technical knowledge). 6. Practical knowledge is tacit and embodied in action; only immersion in the job produces relevant techne; even scientists rely on task-immersed knowledge construction; 7. Scientists study generalizable problems that are as much as possible context free; practitioners use knowledge that is site specific and date stamped – it is customized, derived from experience, and aimed at specific situations; 8. Practical knowledge is a distinct form of knowledge having epistemological status equal to that of academic knowledge; 9. The epistemological rules of good scientific knowledge are fundamentally different from what is necessary for valid practical knowledge; practice-aimed inquiry cannot stand outside practice, as scientific epistemology dictates; 10. Practitioner-relevant knowledge can also be produced with detachment; it can achieve ‘objectivity’ by relying on multiple observers to rise above idiosyncratic viewpoints; 11. Practitioners construct new theories for new contexts; 12. Valid practitioner knowledge has to be actionable.
2 2.1
WHAT COUNTS AS USEFUL SCIENCE?
DISCIPLINE- VS. CURE-ORIENTED SCIENCE 2.1.1 DISCIPLINE-DIRECTED RESEARCH – FOCUSE ON THEORY PROBLEMS; CAUSES Going left to right – the discipline effect. Disciplines create the wrong basis for management research – they focus on would-be universalist, but discipline-specific theories and terms, disciplinecentric methods, and the like. Discipline perspectives are seemingly not useful to practitioners; nor are discipline-based truth claims. Academics writing papers for disciplines have different success criteria than practitioners. The foregoing statements reflect much of what is implied by both knowledge-failure schools so we won’t expand further here. Much of B-school prestige is now defined in terms of achieving close ties with underlying disciplines. Top-ranked B-schools generally hire discipline-centric PhDs. In Starbuck’s just out essay on publication
5
quality in A journals (2005), ASQ is the only ‘business’ journal included in his analysis. But, is it really a B-school journal any more? For some of my sociologist colleagues, after John Freeman became Editor of ASQ, and tilted it more toward sociology, it has been seen ever since as the ‘third’ place to publish organizational sociology papers, after ASR and AJS. Christine Oliver broadened the Editorial Board, but now there is another sociologist as Editor. And, given that most of the Board are still discipline-centric scholars, on a paper-by-paper basis they still apply their discipline-centric standards in what, by name, is a B-school-type journal. We conclude: The institutional structure imposing upon B-schools exerts an irresistible pull toward discipline-centric research. 2.1.2 CURE-DIRECTED RESEARCH – FOCUSE ON CURES FOR MANAGEIAL PROBLEMS Going right to left – the firm effect. Practitioner’s need immediate help – they can’t wait for the scientist’s lengthy conception-to-publication time cycle. They need site and time specific insights. Tomorrow is what counts. They are not especially helped by longitudinal studies based on questions and data defined one or more decades ago. There are three additional ‘negatives’ in going from competitively advantageous site-specific findings to findings independent of time and place and then back again to application in a specific firm at a specific time. 1.
2.
3.
2.2
While Southwest Airlines may have ‘moderate complexity’ that makes it un-copyable (Porter, 1996; Rivkin, 2000, 2001), most firms, having found some kind of distinctive competitive advantage via good research, would be stupid to share it with other firms. In addition, if we were to actually accumulate particularistic research – via engaged scholarship – from a collection of individual firms into some kind of ‘average’, we would still have the same problem we already have, which is going from the average to the dated context of a specific firm. Any truth claim based on site-specific research having value to a specific firm could have little value to another – Toyota is GM’s competitor; Toyota is not Toyota’s competitor. They live in different niches.
CURE-PULL VS. DISCIPLINE PUSH: PUTTING CURES BEFORE THEORY The primary problem stemming from discipline-centric thinking is the dominance of discipline-centric causal analysis. Yes, underlying causal analysis is critically important. But, for the most part that is all we get. We see ‘discipline-pushed’ causal analysis in organization studies. In contrast, we see ‘cure-pull’ in analyses pertaining to human illnesses and building failures from earthquakes. While it is true that all sorts of biological research goes ahead without any specific ‘cure-pull’, in the long run a considerable proportion of such research is cure-driven. Same is true for earthquake studies; there is general interest in various kinds of underlying geological and building construction analysis, but in earthquake country much of it is also very specifically aimed at building structures able to withstand severe quakes. 2.2.1 EXAMPLES OF HUMAN AND ORGANIZATIONAL ‘ILLNESSES’ People in B-schools and the Academy of Management don’t really think in terms of diseases needing cures. In the Table 1 we compare Academy of Management Themes over the past 20 years with diseases needing cures – which are the focus of medical research world-wide. In the right-hand column we suggest comparable managerial problems needing cures. >>>Insert Table 1 about here>Insert Table 2 about here>Insert Figure 1 about here>Insert Table 3 about here>Insert Figure 2 about here
Insert Table 4 about here
