01 Abma (bc/d)
3/7/01
2:48 pm
Page 155
Evaluation Copyright © 2001 SAGE Publications (London, Thousand Oaks and New Delhi) [1356–3890 (200104)7:2; 155–163; 018656] Vol 7(2): 155–163
Opening Thoughts TINEKE A. ABMA Erasmus University, Holland Today many disciplines including organization and management, public administration, social psychology, educational research and political science are becoming interested in dialogue. Evaluators are only just beginning to recognize how valuable dialogue can be for their work. Dialogue may have the potential to improve the democratic quality of decision making and judgement in postmodern societies and enhance (mutual) understanding between people. In this special issue we aim to disentangle some of the complexities, meanings and aims of dialogue and dialogical evaluation. A number of evaluation scholars and practitioners will discuss different notions of the concept and its relevance for evaluation practice. They will also deal with some practical considerations: what are the essential characteristics of evaluation dialogue; who should participate and what should the role and responsibility of evaluators be. We will also identify areas where we think caution should be exercised. Examples of dialogical evaluation projects in various policy settings and countries – including Sweden, Holland and the USA – together with comments by other scholars are meant to stimulate an exchange of ideas. K E Y WO R D S :
dialogue; functions in evaluation practice; rising interest in dialogue
Dialogue and Evaluation Dialogue is an old concept. The ancient Greeks were probably among the first to develop the notion of the so-called ‘Socratic dialogues’, and the Greek roots of the word ‘dialogue’ may give us an idea of what dialogue means. Dia stands for ‘through’ and logos refers to ‘meaning’. In line with the etymology of the word one can understand dialogue to be the process of building and sharing meaning. A search through the literature on dialogue reveals, however, that there is no common definition of the word. Various theories and conceptualizations exist side by side such as those found in the work of European philosophers Buber (1932), Gadamer (1960), Habermas (1991), the Russian literary theorist Bakhtin (1953/1980), and more recently the quantum physicist Bohm (1996). Some consider dialogue to be a mode of being (with others), while others present it as a method, technique or procedure. Bakhtin (1953/1980), for example, emphasizes that dialogue is not a matter of information exchange, but a way of living. In his interactive, performance-centred dialogic theory of language he states that we are all engaged in various dialogues, conversations and discourses as human beings. 155
01 Abma (bc/d)
3/7/01
2:48 pm
Page 156
Evaluation 7(2) During the course of our lives we pick up words, language and discourses in order to make sense of situations and to communicate our ideas. For the sake of clarity, dialogue is more than just talk. Dialogue is a very particular way of organizing communication between people. In order to indicate what this particular kind of communication looks like theorists often distinguish it from ordinary conversation. There are various typologies in the literature. Buber (1932), for example, distinguishes three kinds of dialogue. The first, genuine type, that expresses itself in speaking as well as writing, is a kind of dialogue in which every party turns itself completely towards the other as a ‘thou’ versus ‘it’. The other is approached as subject not as object.1 A genuine dialogue results in lively interaction and symmetrical relationships. The second, technical type of dialogue emerges from the need for a business-like presentation. Thirdly, Buber writes of the monologue that presents itself as a dialogue. A monologue can, for example, be found in debates where the opponent is harmed and not respected as a human being or during conversations that are mainly driven by the determination to impress and to raise the value of the ego. A genuine dialogue is rare. When it happens, even if it has nothing to do with spiritual matters, it can be seen as an expression of the human spirit. A technical dialogue belongs to modern times and can sometimes lead to surprising exchanges, especially among ordinary people. A monologue, according to Buber, does not belong to the dialogical life, it belongs to the ‘under-world of dialogue spoken without a face’ (p. 126). Buber’s typology provides us with one way of stressing the fact that dialogue has features that distinguish it from other kinds of conversations. Another way to distinguish different notions and genres of dialogue is to look at the relationship between unity and diversity. Burbules (1993), for example, makes a distinction between genres working towards convergence and genres respecting difference and divergence. In the case of convergence, the desired outcome of a dialogue will most likely be defined in terms of consensus, common understanding and the reduction of diversity to facilitate decision making and problem solution. Mutual understanding and respect across differences are desired outcomes when divergence is considered to be an essential feature of dialogue. In this issue we do not intend to discuss and compare general theories of dialogue. We are, however, very aware that there are many different notions and conceptualizations. Evaluators who are interested in dialogue – the reasons behind this rising interest will be discussed in a minute – rely on existing theories and translate these to their own working context. They often have varying and sometimes conflicting ideas about what dialogue entails and what constitutes a ‘good’ dialogue. In this issue we take an interest in these differences as well as in the connections between various conceptualizations. We look at their implications for the practice of evaluation in terms of essential characteristics, participants, issues of concern, evaluator role and responsibilities. So, although readers will find extended examples of dialogical approaches to evaluation, we also think it is important to look at, reflect upon and inquire into underlying assumptions and ideas. We hope these inquiries will help us enhance our understanding of dialogue and we invite our readers to join us on this journey. 156
01 Abma (bc/d)
3/7/01
2:48 pm
Page 157
Abma: Opening Thoughts
Dialogue in Evaluation Dialogue has captured the imagination and interest of several evaluation scholars and practitioners (Abma, 1998; Guba and Lincoln, 1989; Greene, 1997; House and Howe, 1999; Karlsson, 1996, 1998; Mathison, 1996; Preskill and Torres, 1999; Rebien, 1996; Ryan and Destefano, 2000; Schwandt, 1997; Vanderplaat, 1995; Widdershoven and Sohl, 1999). They all focus on relationships, interactions and process. They rely on and elaborate work done in the area of stakeholder and participatory evaluation. We can find various arguments in their work stressing the relevance of dialogue for evaluation. Some of the arguments are recorded here: • Dialogue to improve the quality of judgement: in a postmodern world there are multiple legitimate perspectives on human experience and multiple defensible standards for making value judgements. If evaluators are not willing to accept all interpretative accounts and judgements as equally sound, they have to find ‘something’. This ‘something’ cannot be a set of external principles or standards or even methods, for these are technical solutions to what is essentially a political and moral problem. Rather, this ‘something’ must directly engage the moral–political complexities, uncertainties and pluralities of our non-foundational, postmodern world (Greene, 2000). Dialogue – especially democratic dialogue – offers considerable promise in this vein. Howe (1994) has stated this very clearly: If there are no privileged perspectives, no centers of truth, no absolute authorities in matters of taste and judgement, then all truths are working truths and relative truths. The full participation of those involved in decisions about what is going on and what should be done is the only way to define non-oppressive, culturally pertinent truths and working practical judgements.
Thus, dialogue is important for evaluation since it acknowledges that there is no external standard for what is good or bad without concluding that this implies that anything goes. • Dialogue as a force and forum of democratization: dialogue is the way to improve the democratic quality of decision making by enabling all perspectives to become part of the evaluation process. In the public sector, this means that everyone, including weak parties, should have the opportunity to participate in public debate. Citizenship is considered as participation and democracy and as a deliberative process. The underlying idea is that evaluation is not a neutral and value-free activity. All evaluations promote certain values whether objectivity and distance or inclusion. Several authors, among them House and Howe (1999) as well as Greene, and Ryan and DeStefano (both in this issue), have argued that one of the most important values evaluators should promote in their work is democracy. • Dialogue to revitalize the notion of praxis: we can find this argument in the work of Schwandt (1997, 2000). He starts from the assumption that one of the most fundamental problems of contemporary life is to be found in the deformation or distortion of the notion of praxis. This condition results from the inappropriate extension of method-driven knowledge to the conditions of 157
01 Abma (bc/d)
3/7/01
2:48 pm
Page 158
Evaluation 7(2) moral–political life. Put somewhat differently, this distortion is the result of attempting to transform what are essentially moral–political practices into technical practices. The modernist project of evaluation contributes to this deformation via its attempt to tame the unruly social world, to bring order into our way of thinking about what does and does not work for improving social life. To resist the ‘colonization’ of social and political life by scientific method and to resolve a sense of praxis – both in public and civil society – we must come to understand that praxis involves judgement, in the sense of phronesis. It requires moral wisdom, engagement and practical application to oneself. By recovering a view of dialogue as an event of understanding in which we participate and through which we are transformed, evaluators might be able to revitalize the notion of praxis. • Dialogue as a forum for reconsideration and reflection: one of the dominant ideals in our Western information society concerns fast decision making and knowing what you want, think and value. Karlsson (2000) has argued that this is a dangerous situation and could easily lead to badly informed decisions in situations that need to be considered more carefully. He argues that dialogue could provide an important forum for reconsideration and reflection. There is indeed a strong tendency in our society to think that all problems can be judged and solved in a technical and administrative way using general standards. Problems are viewed from this type of perspective and are seen as puzzles that, although complex, can nevertheless be solved. This might be the case if problems were ‘tame’. But in the case of ‘wild’ problems like abortion, euthanasia, social justice and the quality of education, this is simply not realistic. These problems involve political, moral and ethical questions that must be reconsidered in a more reflexive, humble and thoughtful way. When dealing with and taking responsibility for these problems, we must reconsider and admit doubt not as a sign of weakness, but as something that indicates ripeness of judgement and this can be done through dialogue. • Dialogue as a contribution to civil society: a recent development that makes dialogue relevant for evaluation is experiments that have been done with interactive policy making and related concepts like the ‘co-production’ of policies and ‘open planning processes’. These experiments are an extension of a growing awareness among policy makers and civil servants that governments are no longer central actors, but, like other stakeholders, are dependent on several actors in order to realize their goals and interests. The rediscovery of civic society can also be understood in this light. It is an acknowledgement that the government and market cannot solve the problems of society alone. When governments are working with other actors to develop policies, it would be counterproductive to evaluate them in a traditional way by deriving evaluation criteria from policy goals (Abma, 1997). Dialogical evaluation offers an alternative that resonates with the values underlying interactive policy making. Dialogue contributes to a civil society because ‘it assumes stakeholders and the public put aside narrow self interest and address issues among themselves through respectful, reciprocal conversation’ (Ryan, 2000). This is possible because, through dialogue, the notion of what constitutes ‘higher authority’ or 158
01 Abma (bc/d)
3/7/01
2:48 pm
Page 159
Abma: Opening Thoughts ‘legitimate knowledge’ is broadened beyond scientific rationality to legitimize, for example, the voices of marginalized stakeholder groups. More broadly speaking this recapturing of the concept of dialogue can be located within a body of anti-individualist critique. Philosophers and (social) scientists have disputed the Cartesian notion of the self as an autonomous entity (Braidotti, 1994; Flax, 1993; Gergen, 2000) and literary theorists argue that texts are ‘written’ by readers who interpret the surplus meaning in texts. Both forms of critique imply a rehabilitation of relationships and interactions and have paved the way to a rising interest in dialogue. Besides these intellectual efforts, several authors have expressed their dissatisfaction with the human and cultural effects of the individualism that informs the bulk of the social sciences and Western culture in general. With the decline of meta-narratives (Marxism, emancipation) people experience more freedom and no longer automatically accept traditional laws and rules. Besides benefits, we face several drawbacks. Narcissism and competition between individuals give rise to feelings of isolation, disconnection and fragmentation (Lasch, 1979). An individualistic and consuming attitude may also threaten the development of intimate relationships, marriages and communities if people calculate their behaviour in terms of personal gains and losses (Bauman, 1993; Bellah et al., 1985; Giddens, 1992). Others have expressed concerns about the negative effects of the Puritan work ethic on employee motivation and the quality of the work environment (Ellinor and Gerard, 1998). Bohm (1996) has argued that our planet and the environment are badly served by an individualistic way of life. Complex problems require a collective thinking process and an acknowledgement of a shared responsibility for future generations. Another concern is related to liberal democracy and public decision making. Individuals taught to think as isolated participants in the market are unlikely to be able to think as common participants in a polity (Barber, 1984). Finally, doubts have been raised about an individualistic mentality in a world where tools for massive destruction are widespread and problems of international co-operation ever expanding (Perrow, 1972).
The Contributions to this Issue The contributors to this issue have experienced dialogue in the context of evaluation and have written excellent pieces about it. Their work reflects differences in terms of theoretical tradition and working context. In order to stimulate a dialogue within the context of this issue, this introduction is followed by a dialogical text on dialogue. The text is based on essays and a panel meeting on the topic at the European Evaluation Society (EES) meeting held in Lausanne in October 2000. The dialogical text will enable readers to meet most of the contributors to this issue and the ideas that will be worked out later in this issue. The dialogical text aims to engage readers in the conversation and provide them with impressions and insights into the differences, questions and complexities that surround the concept and its practical implementation as an approach to evaluation. Issues discussed by the panel included the aim of dialogical evaluation, the constituents 159
01 Abma (bc/d)
3/7/01
2:48 pm
Page 160
Evaluation 7(2) of ‘good’ dialogue and the role of the evaluator in the dialogue process. The issues that dialogue can address as well as the degree of caution that should be observed when evaluators conduct evaluations in a more dialogical way were also addressed. The audience raised questions relating to (preventing) subtle ways of excluding certain social segments and the problems related to power imbalances between stakeholders. In her article Greene expresses a strong sense of what is meant by the term ‘dialogue’. She sees inclusion, engagement and respect as essential characteristics and considers dialogical evaluation to be ‘a force and forum for democratization’. In her eyes dialogue serves a moral–political function instead of an instrumental function; it can lead to better relationships between participants. Although in dialogue people produce information, Greene strongly opposes the idea that dialogical evaluation is only relevant in terms of generating better knowledge. As she noted in the EES panel ‘In the dialogical process knowledge is created, but as far as I am concerned relationships come first’. Three pairs of articles follow Greene’s. Each pair consists of an initial piece followed by a rejoinder. The first article of the pair consists of an extended example of a dialogical evaluation project. Since a dialogue is only a dialogue when there is reciprocity, we asked three authors to read these initial articles and to respond to them in a way they felt would keep the conversation going. Starting from the idea that dialogue implies a different way of relating to others (including writing), I encouraged them to break from academic conventions in their replies. I hoped their responses would shape, extend, amplify or even redirect the initial contributions. It appears that the responses have the character of short essays. Karlsson has written the first article in the first set. He presents an example of what he calls ‘critical dialogue’. Essential characteristics are a constant probing to stimulate participants to critically examine their views and perspectives. In order to do so, the evaluator acts as a ‘critical inquirer’. As an inquirer, the evaluator may use several artistic methods to foster a dialogue between stakeholders, such as metaphors and performance theatre. Karlsson explains and illustrates the value and meaning of his critical, Socratic approach with an evaluation project of the development of day-care centres in Swedish counties. Schwandt responds with an essay in which he introduces the concept of ‘practice’ to characterize the nature of Karlsson’s evaluation project and the action at the day-care centres. Practitioners, including evaluators, are not acting instrumentally, but have to negotiate the challenges they encounter in social life. They will do so in a structured and meaningful way via face-to-face and indirect conversations. Both practices are in that sense dialogical. Schwandt also draws our attention to different understandings of a so-called Socratic dialogue, the tension between learning and judging and the conflicting interpretations of the term ‘critical’. The second pair of articles begins with work presented by Ryan and DeStefano which was inspired and grew out of House and Howe’s (1999) conception of deliberative justice and democratic evaluation in which dialogue and deliberation are central principles. Ryan and DeStefano explain what this entails and provide two case studies as an example. One is derived from an evaluation project of the 160
01 Abma (bc/d)
3/7/01
2:48 pm
Page 161
Abma: Opening Thoughts Teaching Residents programme in a large American urban school district and focuses on the role of the evaluator as mediator. The other concerns an internal evaluation project that dealt with teaching at a large Midwestern university. The authors carefully examine what dialogue looks like in practice, and conclude that conversations may change over time in terms of multiple dialogical genres. Among other things they have found that it may be hard to create and sustain participation over a long period of time and to bring (disenfranchised) groups to the table. In one of the cases, for example, they had actively to intervene to keep the dialogue going by moving the dialogue from virtual space to a face-to-face interaction. While Ryan and DeStefano attempt to stay close to the lived experiences of stakeholders, Edelenbos and Van Eeten stress the importance of an instrumental focus and the meta-analysis of various stakeholders’ positions in the dialogue in order to facilitate the decision-making process. In addition to the roles of counsellor, mediator and critical inquirer envisioned by Ryan and DeStefano, these authors argue that the evaluator should act as a ‘variation processor’. Based on experiences with policy dialogues in the context of infrastructural projects in the Netherlands, they signal problems related to representation and a disconnection between a dialogue informed by a participative notion of democracy and the formal, more instrumentally driven decision-making process. The authors argue that this disconnection might be resolved through the application of sophisticated techniques. Variation, which is created in the dialogue, can only be retained when dialogical evaluation also supports selection and condensation processes. As the first author of the third set of articles I built on the work of Bakhtin and several social constructionists, to outline an evaluation approach that stimulates reflexive dialogues on inherited myths. Dialogue is understood as a platform for the exchange of stories between different stakeholders – persons with a name and face – who are also engaged in the process. The evaluator acts as a ‘process facilitator’ who stimulates participants to reflect on the way they handle different perspectives and values. This kind of approach is illustrated by an evaluation of an injury-prevention programme in two Dutch performing-art schools. Given asymmetrical relationships it was not possible to bring people together physically. Letting participants from homogeneous groups of students, teachers and paramedical experts respond to each other via stories proved to be a good alternative. Widdershoven recognizes the approach, but strongly opposes my idea that consensus is not a realistic and desirable criterion for ‘good’ dialogue. He offers a more nuanced conception of consensus: one that is always fragile and in danger of breaking down. Another point he makes concerns the danger of being seen as an advocate of ‘silent voices’, even if one does not promote their specific interests. This can make the evaluator unacceptable to the established parties and block the dialogical process. In the last section Schwandt offers a typology as an overall guide to understand and connect the different contributions to this issue, which he sees as examples of substantive versus procedural approaches to dialogue in evaluation. His advice on how we might improve our understanding of dialogue as procedure, and encouragement to think differently about dialogue in a substantive way, may be possible ways to carry on the practice and conversation about dialogue in our field. 161
01 Abma (bc/d)
3/7/01
2:48 pm
Page 162
Evaluation 7(2) We end the issue with a message to our readers. We hope our contributions have enhanced your insight and interest in the concept of dialogue. We trust that this will not be the last word on the matter but that it will provide a point of departure for investigating the topic further.
Note 1. In the relationship with ‘it’ the world remains distanced from the self, or ‘I’. The world has a thing-like character. It is experienced inside the ‘I’, but there is no relationship between the ‘I’ and the world. The world does not take part in the experience; it is passive and does not change in the experience. The relation with ‘thou’ (nature, people or spiritual powers) is a gift and challenge. It requires absolute devotion. Of the ‘thou’ one can say nothing in particular; ‘thou’ does not present itself as a set of parts, but as a whole entity. The ‘it’ can be ordered; ‘thou’ cannot be captured by any system. Every human being needs both kinds of relations.
References Abma, T. A. (1997) ‘Playing with/in Plurality, Revitalizing Realities and Relationships in Rotterdam’, Evaluation 3(1): 25–48. Abma, T. A. (1998) ‘Writing for Dialogue, Text in Evaluation Context’, Evaluation 4(4): 434–54. Bakhtin, M. (1953/1980) The Dialogic Imagination. Austin, TX: University of Texas Press. Barber, B. (1984) Strong Democracy, Participatory Politics for a New Age. Berkeley, CA: University of California Press. Bauman, Z. (1993) Postmodern Ethics. Oxford: Blackwell. Bellah, R. et al. (1985) Habits of the Heart, Individualism and Commitment in American Life. New York: Harper & Row. Bohm, D. (1996) On Dialogue. London: Routledge. Braidotti, R. (1994) Nomadic Subjects, Embodiment and Sexual Difference in Contemporary Feminist Theory. New York: Columbia University Press. Buber, M. (1932) ‘Dialogisch Leven (Zweisprache)’, in J. Binger (collector and translator) Martin Buber: Zijn leven en zijn werk [Martin Buber: His Life and Work]. ’s Gravenland: De Driehoek. Burbules, N. (1993) Dialogue in Teaching. New York: Teachers College Press. Ellinor, L. and G. Gerard (1998) Dialogue: Rediscovering the Transformative Power of Conversation. New York: John Wiley & Sons. Flax, J. (1993) Disputed Subjects. Essays on Psychoanalysis, Politics and Philosophy. New York: Routledge. Gadamer, H.-G. (1960) Wahrheit und Methode. Tübingen: Mohr. Gergen, K. (2000) The Saturated Self. Dilemmas of Identity in Contemporary Life, 2nd edn. New York: Perseus. Giddens, A. (1992) The Transformation of Intimacy: Sexuality, Love and Eroticism in Modern States. Cambridge: Polity Press. Greene, J. C. (1997) ‘Participatory Evolution’, in L. Mabry (ed.) Evaluation and the PostModern Dilemma. Advances in Program Evaluation 3, pp. 171–90. Greenwich: JAI Press. Greene, J. C. (2000) Paper presented for the panel on ‘Dialogue and/in Evaluation’ at the Fourth Conference of the European Evaluation Society: Taking Evaluation to the People, Lausanne, 12–14 October.
162
01 Abma (bc/d)
3/7/01
2:48 pm
Page 163
Abma: Opening Thoughts Guba, E. G. and Y. S. Lincoln (1989) Fourth Generation Evaluation. Beverly Hills, CA: Sage. Habermas, J. (1991) Erläuterungen zur Diskursethik. Frankfurt am Main: Suhrkamp. House, E. R. and K. Howe (1999) Values in Evaluation and Social Research. Thousand Oaks, CA: Sage. Howe, D. (1994) ‘Modernity, Postmodernity and Social Work’, British Journal of Social Work 24: 513–32. Karlsson, O. (1996) ‘A Critical Dialogue in Evaluation: How Can the Interaction Between Evaluation and Politics be Tackled’, Evaluation 2(4): 405–16. Karlsson, O. (1998) ‘Socratic Dialogue in the Swedish Political Context’, New Directions for Evaluation, 77: 21–38 (Scandinavian Perspectives on the Evaluator’s Role Informing Social Policy, ed. T. A. Schwandt). Karlsson, O. (2000) Paper presented for the panel on ‘Dialogue and/in Evaluation’ at the Fourth Conference of the European Evaluation Society: Taking Evaluation to the People, Lausanne, 12–14 October. Lasch, C. (1979) The Culture of Narcissism. New York: Norton. McNamee, S. and K. J. Gergen (1999) Relational Responsibility, Resources for Sustainable Dialogue. Thousand Oaks, CA: Sage. Mathison, S. (1996) ‘Evaluation as a Democratising Force in Schools’, International Journal of Social Education 11(1): 40–8. Perrow, C. (1972) Complex Organizations: A Critical Essay. Glenview, 111. Preskill, H. and R. Torres (1999) Evaluative Inquiry for Learning in Organisations. Thousand Oaks, CA: Sage. Rebien, C. (1996) ‘Participatory Evaluation of Development Assistance: Dealing with Power and Facilitative Learning’, Evaluation 2: 151–72. Ryan, K. (2000) Paper presented for the panel on ‘Dialogue and/in Evaluation’ at the Fourth Conference of the European Evaluation Society: Taking Evaluation to the People Lausanne, 12–14 October. Ryan, K. and L. DeStefano (eds) (2000) Evaluation as a Democratic Process: Promoting Deliberation, Dialogue, and Inclusion. New Directions in Evaluation, 85. San Francisco, CA: Jossey-Bass. Schwandt, T. S. (1997) ‘Evaluation as Practical Hermeneutics’, Evaluation 3(1): 69–83. Schwandt, T. S. (2000) ‘Dialogue, “Local” Evaluation, and Education’, Paper presented at the Annual Conference of the American Evaluation Association, Waikiki, Hawaii, November. Vanderplaat, M. (1995) ‘Beyond Technique: Issues in Evaluating for Empowerment’, Evaluation 1: 81–96. Van Eeten, M. (1998) Dialogues of the Deaf: Defining New Agendas for Environmental Deadlocks. Delft: Eburon. Widdershoven, G. and C. Sohl (1999) ‘Interpretation, Communication and Action: Four Stories about Supported Employment’, in T. A. Abma (ed.) Telling Tales, on Narrative and Evaluation. Advances in Program Evaluation 6, pp. 109–30. Stamford, CT: JAI Press. TINEKE A. ABMA is working at the Department of Health Care Policy and Management of the Erasmus University, Rotterdam, Holland. She has published on responsive evaluation approaches and takes a special interest in social constructionist and postmodern theories. Her practice is mainly located in the health care sector. Please address correspondence to Tineke Abma, Courzandseweg 57, 3089 PE Rotterdam, Holland. [email:
[email protected]]
163
