Science and Public Policy, 37(5), June 2010, pages 369–379 DOI: 10.3152/030234210X501216; http://www.ingentaconnect.com/content/beech/spp
Doing research, creating impact: using ‘PROTEE’ to learn from a genetically modified tree field trial Helena Valve, Ruth McNally and Ari Pappinen
If a research project is to achieve its intended impacts, it must learn to become sensitive to the public context it implicitly assumes and enacts. This article presents a case study in which we used the perspectives provided by the ‘PROTEE’ methodology to create an opportunity to explore, and to reflect on, the reality in which research on genetically modified trees was expected to make a difference. Identifying potential barriers to persuasive communication brought out additional capacities and limitations of the research strategy. Paradoxically the very same strategic choices which had allowed the research project to claim its policy-relevance ended up undermining its public role. Since PROTEE can help making such contradictions explicit, we claim that it has much to offer to research management.
S
CIENTIFIC PROBLEM formulations contain and put forward imaginations of the public world and its needs (Wynne, 2005). First, in order to be eligible for public funding, research projects must explain how they will benefit society by having positive impacts. In so doing, much is assumed about the reality in which the research outputs are to make a difference. Furthermore, once funded, research projects enact policy phenomena and their interrelated publics (Mol, 2002). They create analytical settings in which humans and nonhumans can become visible and in which they can gain capacities (Gomart and Hajer, 2003). At the same time they necessarily disregard some dimensions and relationships as (policy) irrelevant.
Helena Valve (corresponding author) is at the Environmental Policy Centre, Finnish Environment Institute, PO Box 140, 00251 Helsinki, Finland; Email:
[email protected]; Tel.+358 (0)400148858. Ruth McNally is at ESRC Centre for Economic and Social Aspects of Genomics, Institute for Advanced Studies, Lancaster University, Lancaster LA1 4YD, UK; Email:
[email protected]; Tel: +44 (0)77 389 42898. Ari Pappinen is at North Karelia University of Applied Sciences, Tikkarinne 9, 80200 Joensuu, Finland; and University of Eastern Finland, Faculty of Science and Forestry, PO Box 111, 80101 Joensuu, Finland; Email:
[email protected]; Tel: +358 (0)50 438 2527.
Science and Public Policy June 2010
However, scientific research projects do not always achieve their intended world-shaping outcomes. Even projects with an applied orientation often fail to deliver anything of enduring commercial or practical use. Although research implicitly proposes scripts for human and non-humans by ‘defining a framework of action together with the actors and the space in which they are supposed to act’ (Akrich, 2000: 208), there is no guarantee that the actors will move and interact in the limits defined by the scripts. It is hence clear that if a research project is to have a viable and responsible public role, it must learn to become sensitive to the public context it implicitly assumes and (re)produces. Moreover, such imaginations are not static, but can evolve with the research. This dynamism creates the potential to reflexively apply what is learnt to re-orientate research projects while still in progress. • But how do research projects actually imagine and enact their public context? • How does the context come to affect what the projects promise and what they do? • Why does doing what the projects say they will do not guarantee the expected outcomes?
0302-3427/10/050369-11 US$12.00 © Beech Tree Publishing 2010
369
Doing research, creating impact
Dr Helena Valve works as a Senior Researcher at the Finnish Environment Institute. Her research focuses on the dynamics between knowledge practices, policy-making and politics in the field of environmental policy and governance. Her recent publications analyze the creation of collective awareness and controllability of GMOs, asking how practices of knowledge production and regulation condition the enactment of public concerns (see Journal of Environmental Law, 2008, 20, 339–362, with Jussi Kauppila, and New Genetics and Society, 2008, 27(4), 339–352). Currently she leads a project that studies the role of environmental quality standards in water management. Dr Ruth McNally is a Senior Research Fellow at the ESRC Centre for Economic and Social Aspects of Genomics, Lancaster University, UK. Her research interests are research methods and transformations in knowledge production in the biosciences. Her recent publications have been on forensic DNA profiling (see The Truth Machine, 2009, Univ. Chicago Press, with Mike Lynch, Simon Cole and Kathleen Jordan). Her current research is on digital data-sharing in data-intensive research. Dr Ari Pappinen is a Principal Lecturer at North Karelia University of Applied Sciences having a docentship at University of Eastern Finland. His resent research topics have been on analyses of GM trees in a field trial (see Planta, 2009, 230(5), 973–983, Trees, 2008, 22(4), 413–421 with Pasonen et al.) and molecular analysis of new fungal species (see Mycological Research, 2008, 112(12), 1475–1488, with Linnakoski et al.). He has also published some analyses on deforestation aspects in tropics (e.g. Environment, Development and Sustainability, 2008, 10(4), 503–518).
In this article we provide some answers to these questions. The article is based on a case study of a research project whose aim was to provide policy-relevant scientific outcomes. Together with the project we analysed how its promises and practices conditioned the constitution of its public role. To do this we applied the analytical perspectives provided by the socalled PROTEE methodology which provides an opportunity to explore and reflect upon the reality in which a project’s outcomes are expected to be meaningful and to make a difference. During years 2004–2006 we used the PROTEE approach with a research project carried out at the Department of Applied Biology, University of Helsinki, and Faculty of Forest Sciences, University of Joensuu. The project was entitled ‘Interbiontic processes between genetically modified trees, forest pests and fungi: development of a risk assessment procedure’ (later called the SUNARE project) (Von Weissenberg et al., 2001). Its aim was to develop a novel method for assessing the environmental impact of birch trees modified with a foreign chitinase gene. During the period of our study, the research project was still in progress, but coming to an end. In the next section we provide a succinct description of the PROTEE approach. After that we turn to the case study in which we evaluated the SUNARE project relying on PROTEE principles and indicators. Step by step we learned more about the project and the strategic choices it embodied. These lessons and their implications are discussed. In the final
370
section we draw conclusions on the contradictions of policy-relevant science and its evaluation. PROTEE
PROTEE was developed by a consortium of industrial and academic partners under the leadership of Bruno Latour (PROTEE, 2000; McNally and Woolgar, 1999; Laredo, 1999).1 It is intended to be a novel approach for managing radical innovation projects. These are projects that seek to address particular societal problems and needs by developing highly innovative solutions. PROTEE calls these innovative solutions ‘hopeful monsters’ because they are strange new entities of unknown potential.2 One of the justifications for PROTEE was that conventional project-management tools require quantified information which is simply unavailable for innovations that are truly radical (Hommels et al., 2007; Höyssä and Hyysalo, 2009; Jolivet et al., 2003). The inherent uncertainty in radical innovation projects leads not only to wide margins of error in forecasts and estimates, but also to an optimistic bias in the errors that are made (Freeman and Soete, 2000). The upshot is that not only do the majority of radical innovation projects fail to deliver what was promised and anticipated, but also there is little on the official documentary record to account for the disappointments. Typically, the project proposal was promising and plausible, which was why it was funded in the first place, and the subsequent interim reports and evaluations did nothing to dispel these expectations. In other words, conventional management and appraisal approaches seem to cultivate a systematic and uninformative, optimistic bias in project forecasting and reporting practices. PROTEE seeks to break with conventional project-management approaches based on calculation by introducing an alternative paradigm of collective ‘redescription’. This approach is informed by the social studies of science and technology. In particular, it conceptualises technological innovations as ‘scripts’ of future worlds (Akrich, 2000). PROTEE’s goal is not to increase the success rate of such projects, but to add value by increasing what can be learnt from the doing of them. It does this by supporting them to become more articulate and experimental in exploring possible interactions between themselves as innovators and potential users. Underpinning PROTEE is a critical reading of the written documentation that bears witness to conventionally-managed innovation projects. PROTEE views these typically optimistic, or ‘sweet’, project descriptions as symptomatic of three ‘pathologies’ which it calls ‘inevitability’, ‘indisputability’ and ‘unanimity’ (see Table 1). Under conventional project-selection and -management approaches, these pathologies are cultivated in both the performance and writing of innovation projects because such regimes reward ‘sweet-talking’ projects.
Science and Public Policy June 2010
Doing research, creating impact Table 1. ‘Pathologies’ and corresponding PROTEE indicator classes
Pathologies in the conventional management of innovation projects
Corresponding indicator class in PROTEE
1. Inevitability
1. Realism
The project forecasts and then retells an unchanging ‘ballistic’ story of unhindered technical development from drawing board to final product, with limited conceptualisation of social dimensions and relationships, or acknowledgement of uncertainties and their qualities and likelihoods
The realism indicators are attentive to the richness of the project’s descriptions of itself, and the realisability of its goals. Dialogues using this class of indicators explore how the project came to be the way it is, what it hopes to become, and what its progress is dependent upon. Crucially, the dialogues seek to develop an enriched awareness of innovation as a socio-technical process involving humans and non-humans. These dialogues bring into focus key choices that have already been made, and lead the project team to acknowledge and explore the uncertainties, unknowns and contingencies in the project’s imagined future
Project descriptions are focused almost exclusively on scientific and technical elements to the exclusion or neglect of other factors and interrelationships
2. Indisputability
2. Strategy
No coherent picture of opposition. Opposition, competition and indifference (human and non-human) are absent or, if present, are presented as irrational, irritating, ignorant or irrelevant Lack of consideration of which aspects of the project could be modified or even abandoned for the sake of negotiation, compromise and taking the project forward
The strategy indicators encourage productive engagement with obstacles, competitors and opponents, through using them strategically to renegotiate the project. These dialogues encourage the project team to consider the project from the perspectives of those actors who are opposed or indifferent, and to consider how to use such perspectives to learn and adapt. However, this is not intended to be a route to the endless opening up of the project. Rather it is an exploration of its limits of flexibility, ultimately leading to informed closure, for example, through awareness of how choices made in order to deepen engagement with one constituency could lead to the loss of another
3. Unanimity
3. Falsifiability
Tests, trials, demonstrations, pilot studies, opinion polls, and other assessments are biased or irrelevant
The falsifiability indicators are intended to encourage the project to undertake trials which will maximise learning. As the name of this class of indicators suggests, this includes the design of trials which have the potential to solicit challenging and unfavourable evaluations and responses, from humans and non-humans
They lack novelty and are repetitive and so make only a small, if any, additional contribution to what was already known. They lack specificity so it is unclear what is being tested and what has been learnt. They lack relevance with respect to assessing the project’s key claims. They are defined by, and confined to, a small circle of those already enrolled. They confirm what was already claimed and are a waste of resources
Good trials, according to this class of Indicators, should have the potential not only to surprise, but also to jeopardise the claims made by the project team
Potentially viable project variants are ruled out without testing, while those that are unviable are artificially sustained through lack of rigorous testing
PROTEE, by contrast, seeks to diagnose and eradicate these pathologies from both the doing and the recording of innovation projects through a dialogic process conceptualised as ‘socio-technical therapy’ (McNally and Sondermann, 1999). It is envisaged that the PROTEE approach would be operationalised through a relationship, called a ‘learning pact’, between the ‘Innovators’ and the ‘Evaluators’, where the Innovators are spokespersons for the project team, and the Evaluators represent the project’s sponsors. These two parties would enter into a succession of dialogues structured by ‘indicators’ which correspond to the ‘pathologies’ (see Table 1).3 These indicators are the core of PROTEE. They attempt to replace the conventional ‘sweet’ accounts of innovation with accounts that are more realistic, but also more risky. Through a dialogic process, the parties to the learning pact use the indicators to diagnose project’s past, make predictions about its possible futures, and design and evaluate trials which put their predictions to the test. The outcome of this process is what is meant by a
Science and Public Policy June 2010
PROTEE ‘redescription’. A summary of each redescription should be recorded so that comparisons can be made over time.4 The experiment
In the PROTEE experiment analysed in this article, the Evaluator (HV) was an expert only on paper. The experiment was her first acquaintance with PROTEE, as it was also for the Innovator (AP). Therefore supervision was provided by RM, one of PROTEE’s developers. In the evaluation of the SUNARE project we did not try to follow the steps identified by PROTEE literally, or to apply all of its indicators. This was largely because of the nature of the task in hand. SUNARE was a rather short project which was already coming to an end. In fact, it took some time to find an angle from which PROTEE could be fruitfully used. However, in the end the main emphasis was put on the development of a communication
371
Doing research, creating impact
strategy for the project and by so doing encourage learning from, and engagement with, the public world of the project. Our PROTEE experiment included four types of encounters. The first type brought the Evaluator and supervisor together, face-to-face. In all, we had three such meetings: one in the very beginning, one halfway (Innovator present too), and one in the end. Second, the Innovator and the Evaluator met for PROTEE dialogues on five separate sessions, one of which included the supervisor.5 Third, the Evaluator interviewed all together 13 members of the SUNARE research group. This proved useful since the tasks and responsibilities in the project were dispersed to several people, although the Innovator and his closest assistant drew the strings together. The interviews, together with the documents and papers produced, helped the Evaluator to acquaint herself with the history of the SUNARE project. The fourth type of interaction took place via email. The Evaluator consulted the supervisor before and after her meetings with the Innovator. The face-to-face meetings were recorded and most of them transcribed too. This material, together with the interviews, documents, and our email correspondence provide the empirical data analysed in this article. Results The consensus story
Before any redescription is possible, it is necessary to establish a base-line description of the project at ‘time zero’ before intervention with PROTEE. This is what follows in this section. It is a description of SUNARE in summer 2004 based upon how it described itself in the proposal, reports and papers. In 1999 an interdisciplinary research group submitted a research proposal to the Academy of Finland to study: ‘Interbiontic processes between genetically modified trees, forest pests and fungi: development of a risk-assessment procedure’. The proposal argued that: research in this field is important not only to identify and evaluate potential hazards, but also for restoring popular confidence in biotechnological applications. (Von Weissenberg et al., 2001) Against this background, the aims of the project were: 1. To develop new methods for use in the risk assessment of genetically modified (GM) trees; 2. To develop predictive theories to address acute political questions concerning the use of GM organisms (GMOs) in agricultural and/or natural ecosystems.
372
It is the second of these two aims that we focus on here. The biotechnological manipulation of GM plants often involves modifying ‘interbiontic’ processes, that is, some aspects of the ways in which plants interact with other organisms. The SUNARE project used trees, called ‘chitinase birches’, genetically modified with a gene from sugar beet. The genetic modification was intended to increase the expression of an enzyme called chitinase which breaks down a molecule called ‘chitin’, an important component in the cell walls of fungi and insects. This was expected to affect interactions between the GM trees and pathogens (fungi, insects). However, both parasitic (harmful) and symbiotic (beneficial) interactions between forest trees and fungi involve similar molecular mechanisms. Therefore there was a risk that the chitinase birches might have detrimental effects on symbiotic organisms, such as mycorrhizal fungi. The proposal was successful and from 1 August 2001 until 31 July 2004 the project was carried out. The interbiontic processes and potential risks of the chitinase birches were studied in a field on the outskirts of Helsinki planted with about 225 seedlings of GM birch trees and 120 control plants. Earlier tests carried out in greenhouses had found that some of the GM birch tree lines were more resistant to a fungal disease (leaf spot disease) than control plants (Pappinen et al., 2002). The plan was to grow the trees in the field for four years, but the trial had to be terminated one year early because the land was needed for a building site. The project findings were that interbiontic processes could be studied in a field trial and that the novel risk-assessment method tested was able to detect differences in disease-resistance between the GM saplings and the controls. It was also found that the GM birch lines that had been resistant in the greenhouses lost, as expected, their statistical resistance in the field. However, new resistances to birch rust disease emerged in the field (Pasonen et al., 2004). The SUNARE project found that the expression of the foreign gene had little impact on non-target organisms such as soil fauna, indicating that the chitinase birches were, in that sense, practically riskless. However, the genetic modification affected the
Interbiontic processes could be studied in a field trial and the novel riskassessment method tested was able to detect differences in disease-resistance between the genetically modified saplings and the controls
Science and Public Policy June 2010
Doing research, creating impact
stress status of the chitinase birches, making them less ‘fit’ than the control plants (Pasonen et al., 2008).6 Redescription 1: Why this project?
As the story told in the official paper trail shows, the project’s aspiration was to make a difference to public debate on GMOs by providing scientific riskassessment. Clearly, from the very beginning, the project was an activity situated in relation to something else. By linking its goals with the specific means (a field trial) of achieving them, the project team had formed a strategy within an assumed, but understated, reality. One of the Evaluator’s goals was to make these implicit underpinnings more explicit. The consensus story was used as the starting point for the first series of PROTEE dialogues which the Evaluator structured mainly by using questions based on Class 1 indicators which are designed to steer the project from ‘inevitability’ to ‘realism’. These dialogues asked ‘how’ and ‘why’ questions, such as: ‘Why this project?’; ‘How has it developed into the one it is today?’ The dialogues revealed how one of the factors that had shaped the project proposal was to provide a means to follow up and capitalise on work already done. A key manifestation of this work was the chitinase birches themselves. Additionally, the project proposal also assembled existing expertise in relation to a specific type of theoretical understanding of ecological interactions and the interplay between forest trees and their pathogens. This understanding became an important asset for the development of the project’s research methodology. A critical decision had been the selection of risk-assessment as the unifying goal and defining perspective. Through this decision, the project team created a niche for itself within the university department, thus avoiding having to compete for funds with other projects involving colleagues. The scientific and policy rationale for choosing risk-assessment as the organising principle for the project proposal was easy to justify: chitinase birches are GMOs, and as such have, in the EU, an institutionalised identity as being potentially risky. It also happened that the period during which the team was developing the proposal coincided with an upsurge in the public interest in GMOs in Finland, a debate which seemed lacking in knowledge about the risks of GM plants. One symptom of this debate was the formation of a public interest group in opposition to gene technology in Finland. With representatives of this organisation strongly opposed to the release of GMOs, and scientists doing research in the domain of green biotechnology tending to be in favour of the technology, the debate had become highly polarised.7 The Innovator, who had been following the debate, had become frustrated by the apparent deadlock. Then one day it occurred to him
Science and Public Policy June 2010
that the risks expressed by the anti-GM organisation could be tested scientifically. Doing so became one of the goals of the project; the vision was to help to mediate the conflict by providing a common ground between the two poles of the Finnish GMO debate in the form of scientific facts. • How then should one study the interbiontic processes and risks of chitinase birches? • What kinds of methods and data would be useful and fruitful? While for the Evaluator these questions seemed difficult and challenging, for the project team the answers had been — at the time the decisions were made — almost self-evident. Plant breeding, whether based on genetic modification or not, is all about experimentation. In the case of GM plants, individual cells containing the transferred genes are isolated from cell cultures and submitted to series of tests. The Innovator already knew how the chitinase birches grew in the laboratory and in greenhouses, and what qualities they exhibited there. Therefore, planting them outdoors seemed the only rational next step to take. What the team needed to do next with their trees was to assess their qualities in a field trial. Proposing a field trial as the next step was also in accordance with the prevailing EU regulatory regime for assessing the human and environmental risks of GM plants. According to the ‘step-by-step’ principle, testing of GM plants should start in laboratories, proceed through greenhouses, and finally progress to field trials.8 The field trial, which later became the backbone of the project, was the first documented field trial of GM birches in Finland. Including it in the proposal gave the project novelty and placed it at the forefront of scientific progress. In fact, their estimation of the kudos that the field trial added to the proposal was such that the project team used it as ‘bait’ with which to hook the funders. Taking a deliberate and conscious risk, they set up the field trial prior to any commitment to fund the project. Had the proposal been unsuccessful, the team would have been forced to dismantle the field trial and the resources invested in it would have been wasted. As it turned out, the risk proved worth taking. However, while this strategy found favour with the funders, not everybody was happy about the field trial. Although the project’s aim was to mediate disagreement, the establishment of the field trial itself provoked a new confrontation. Representatives of the anti-GM organisation set up a demonstration just outside the fence with a banner reading ‘biohazard’. Thus the high perimeter fence containing the experimental site assumed a dual function, not just isolating but also protecting the chitinase birches. It became clear that the activists opposed to GM trees were, and still are, also opposed to field trials. Ironically, however, as the Innovator noted, without the opponents the project might not
373
Doing research, creating impact
have been funded, for it was their opposition, together with the enthusiastic advocates, that politicised gene technology and thus gave the project its apparent policy-relevance. The first series of dialogues, summarised above, redescribed how the project came into being. The redescription introduced an array of actors, relationships and contingencies which had not been explicit in the official paper trail. The dialogues revealed how some research paths had been closed because of the success of competing groups, withdrawal of invited collaborators, and failure of previous proposals. In contrast to what was recorded about the rationale for the project in the official paper trail, stories were told in which the project proposal provided the answer through which a heterogeneous network of people, ideas, competences, GM birches, available facilities and funding opportunities could be assembled. Thus the project was not only a response to what was viewed as societal demand, but also a means to synchronise available resources and opportunities. The project had managed successfully to use contingencies and relations to its advantage. It had also been able to adjust. Thus, the first redescription of the project was that of a success story. Redescription 2: Realism and the field trial
The dialogues that produced the above redescription took place just as the project was coming to its end. By this stage, the project had carried out a set of scientific experiments, analysed a rich array of data, and written manuscripts, papers and reports. In terms of its first aim, the project was a success.9 However, it had not achieved its second aim of having a mediating influence in the polarised debate over the risks of GM plants in Finland. Exploring how the project could make a difference in this arena became the focus of a series of dialogues with the Innovator who was the member of the project team who had initially envisaged this outcome. The dialogues began with agreement that the project would not make a difference to the public debate unless its findings were communicated effectively. This focus on communication gave the PROTEE dialogues a practical, operational goal. It was assumed that successful communication could not be treated as a separate downstream step or phase, but would be an outcome whose conditions of possibility had been constructed throughout the project, including the methods used to produce the results. In order to manage and counteract the ballistic fallacy that the results would communicate themselves, these dialogues once again focussed on the Class 1 indicators (realism). This time we employed the indicators to study the implicit scripts embedded in the research methodology. In other words, across what sort of public domain would the evidence be generalisable? What were the perhaps taken-for-granted, yet not explicit, limitations of the usefulness of the field trial?
374
The Innovator replied that, of course there are limitations of a field trial — any field trial — as a research setting for assessing the risks of GM trees. A three-year long field trial provides a perspective that is, by definition, limited by time. Although the rationale was to use the field trial to expose the chitinase birches to seasonal changes and natural weather fluctuations, there is no guarantee that the conditions actually experienced would be representative of the climate over a longer time. Therefore, the trial may be uninformative about the capacities of the plants to survive under other environmental conditions. On the other hand the Innovator noted that the project aimed merely to provide a starting point for scientific risk assessments of GM trees. Another complication was that not all of the GM trees grew well under field trial conditions. Many of them were short and branched: the gene transfer probably had some side-effects (Pasonen et al., 2008). Although these side-effects were in accordance with theoretical understandings and with the results of other studies, it was difficult to be certain, without further genome-scale analyses, about what caused them in this particular case. It could have been because some lines were unstable in some way. Given that the underlying dynamic was unclear, the only way to select stable trees from stable lines for further research would be to wait and see which ones turned out to grow and act as expected. The problem was that the chitinase birches growing in the field trial had not had — and probably never would have — the opportunity to fully express their (in)stability. At the time the field trial was terminated, it was simply impossible to say which of the lines growing in the field trial would be suitable for further utilisation. Moreover, this was not even the project’s ultimate goal. This mismatch between the short testing period and the long physiological aging of the trees highlighted the point that the results achieved in a field trial would allow one to conclude little about the trees as such. Trees, with the exception of some fastgrowing species such as eucalyptus and poplars, grow slowly and therefore assessing them in a field trial is not as straightforward as the assessment of herbaceous plants, for example. It would take a testing period of up to perhaps one tree generation — decades, ranging up to a century in Finnish conditions — to determine whether or not a GM tree is what the Innovator called stable. The long regeneration time made it difficult to draw conclusions about the chitinase birches and treat them as objects with a definite identity. The Innovator was keen to adhere to the centrality of time: only if the trees were allowed to grow for long enough would a field trial reveal which ones were stable and which were not. However, lengthening the field trial would not address its spatial limitations. A field trial is a specific environment — a habitat. When the Evaluator queried the apparent marginality of the role given to the environment, the
Science and Public Policy June 2010
Doing research, creating impact
Acknowledgement of the implicit assumptions about the spatio-temporal specificity of the field trial opened a space for a further dialogue about the pre-conditions for the stable growth and development of chitinase birches
key methodological choices. Nonetheless, despite its limitations, the Innovator remained optimistic about the potential impact of the findings from the field trial, insisting that: If one carries out a three-year-long field trial such as ours, it is inevitable that much will remain open and inconclusive. This will always be the case for GM trees. Nonetheless, we did learn something about interbiontic processes and we can offer some references for the debate. Redescription 3: A strategy for engagement
Innovator replied — a bit frustrated — that naturally no one would plant a birch tree in a heath or in any dry environment, for example, and then expect it to grow well! He took it for granted that the findings of a field trial would only be valid for environmental conditions suitable for the organisms. Acknowledgement of the implicit assumptions about the spatio-temporal specificity of the field trial opened a space for a further dialogue about the preconditions for the stable growth and development of chitinase birches. What came next into sight were the humans working and co-existing with the field trial. This dialogue brought about an acknowledgement that the field trial represented a world made viable by very specific human practices. Were the humans to defect and leave the trial to survive alone, the interbiontic interactions would be affected too. Forestry practices affect how trees grow and which part of their DNA is actively read during physiological maturation; wood quality, for example, is sensitive to tree density. A lignin-modified tree may produce good-quality fibres when young, but little is known about what would happen when the forest ‘closes’ and less light become available. At this point the Innovator noted that if the aim was the practical utilisation of GM trees, risk assessment should be done under the conditions in which they would be expected to grow when adopted for commercial use. He also pointed out that the project’s field trial did not even attempt to test hypotheses related to commercial use. Actually, he continued, the topic can be viewed from a reverse angle. The project was focused on analysing the interbiontic consequences resulting directly from the genetic modification. It can be assumed that many of the environmental risks of GM trees are likely to result from changes in forestry practices. In the first instance, commercial reproduction of GM trees will most probably be based on vegetative propagation. Such cloning in forestry tends to be economically feasible only if carried out in big plantations which are intensively managed. At the time the dialogues took place, although the studies conducted in the field trial were still under analysis, the field trial itself was over and the site had been dismantled. Hence there was no way for the dialogues to make a difference to the project’s
Science and Public Policy June 2010
When redescribed as above, the field trial would seem to have always been inherently incapable of providing representative information about the qualities of the chitinase birches. Nonetheless, the Innovator remained optimistic about the relevance of the project’s findings for the public debate and committed to the project’s objective of making a difference to it, for example, through a written paper on the limitations of field trials. Therefore, the dialogues continued, guided by the indicators in Classes 2 (strategy) and 3 (falsifiability), towards the development of a communications strategy for contributing to the debate, and trials that would put this strategy to the test by exposing what was at stake. After the potential barriers to enrolment and communication had been discussed on a general level, it was to time to specify to whom exactly the project’s findings should be communicated. It was expected that the challenges of communication would vary from one group to another. Therefore, the Evaluator asked the Innovator to imagine audiences and the reasons why they might not listen or be convinced. According to the Innovator, the relevant audience was made of the two sides of the polarised public debate: the academic community on the one hand, and those opposed to GM trees on the other. There were also civil servants responsible for assessing applications for deliberate releases of GM plants, but since they are influenced by and dependent on scientific evidence and expertise, they could be included with the academic community. The academic community is only influenced by published results; therefore, the primary way to enrol it would be through peer-reviewed publications in scientific journals. However, enrolment of those opposed to GM trees would be likely to require more than scientific papers. The anti-GM lobby might not trust, or even accept, scientific arguments indicating that the GM trees pose no risk to the environment. As the Innovator put it: The opponents may think that we, as scientists in the project have a hidden agenda, which was to use the field trial to specifically demonstrate that the GM trees are riskless.
375
Doing research, creating impact
Moreover, since the lobby is opposed to field trials, it would be difficult to persuade it with one. It was reasoned that the anti-GM lobby would be unlikely to treat the project’s methodological suggestions and findings as neutral inputs into the debate. Given its lack of trust of scientists and its opposition to field trials, were there any conceivable ways to persuade the anti-GM lobby to engage with the project? What would the project have to do to communicate with it? The Innovator anticipated that enrolment of the opponents would not succeed unless their cause would benefit from the project’s message. ‘The opponents will listen only if the message is what they want to hear.’ The dialogue ended with the Innovator’s suggestion that the project should organise and participate in an open workshop with both sides of the debate on the pros and cons of field trials. Together with scientific papers, the workshop would test the project’s claims that it could and would make a difference to the polarised public debate by providing some common ground. Redescription 4: Closure
Several months passed, and when the dialogues resumed the Innovator declared that, apart from writing academic papers, he was unwilling to become engaged in the public debate. He had moved to a new position and was working only part-time with SUNARE. Moreover, the Innovator felt that in order to communicate the results in any other way he would have to become involved, be forced to locate himself within the debate, to take sides on the side of the anti-GM lobby, a position he was unwilling to adopt. I really do not know what to think about this debate. I have mixed feelings. So that … if I now were to participate in the discussion, I should have to put my values to some order which seems unfamiliar. I really wanted to set the debate in a scientific context. Without taking sides. But this communication strategy we have planned in a way forces me to take sides. And I am not very keen to do that. Thus the idea of the workshop as a way of engaging with the anti-GM lobby was abandoned. But what about engaging with the anti-GM lobby through academic publications? Surely the lobby reads scientific papers? Although the preliminary results of the project indicated that the genetic modification did not affect interbiontic interactions, it also seemed, as already discussed, that some of GM tree lines were unstable or at least under stress. Perhaps this message could be the one that the opponents would like to hear? Maybe the ‘instability’ of the trees could be turned into an asset? Maybe yes, responded the Innovator, but adopting this strategy could be detrimental to existing
376
relationships. A respected colleague had questioned the wisdom of even discussing the apparent ‘instabilities’ of the birches. The Innovator assumed that the colleague in question viewed this topic as a potential threat for future funding prospects. Whatever the reason, the somewhat unstable nature of the GM trees was, in the end, given little attention in the project’s communications outputs (see, however, Pasonen et al., 2008). Although the workshop — a specific kind of a trial — remained a mere thought experiment, it was nevertheless informative. The lessons learned from thinking it through forced the Innovator to abandon his objective of mediating in the debate. To have done otherwise would have been against what was viewed as neutral, and would therefore have been too risky for a professional scientist. Different messages, relating to different publics, turned out to be incommensurable. Choosing a message potentially interesting from the point of view of the anti-GMlobby and a medium in which to deliver it might have threatened the impartial position of the project and its associates. The Innovator tried hard to think how the same topics and interbiontic interactions could have been examined in closed conditions, and imagined experiments in large, roofless greenhouses, but these visions did not release the gridlock over the project. There seemed to be little potential for the project to mediate the conflict over gene technology in Finland, and the PROTEE experiment was brought to a close. Discussion
During the PROTEE evaluation process, the images of the public world embedded in, and enacted by, the SUNARE project became more explicit and discussable. Most importantly, the process provided an opportunity to view the project as a set of strategic choices already made, and to be constantly remade. The consensus story had presented the field trial as a key strength of the project, as a key attractor of funding. Moreover, the planting of the chitinase birches in a field had seemed a natural step to take. The fact that the problems of communication and enrolment were attributable to the use of a field trial
The fact that the problems of communication and enrolment were attributable to the use of a field trial showed that the project had, as anticipated, chosen a risky strategy
Science and Public Policy June 2010
Doing research, creating impact
showed that the project had, as anticipated, chosen a risky strategy. However, the risks taken were not only the ones anticipated at the time when the field trial was set up. The questions inspired by the ‘realism’ indicators helped to focus attention on the key ontological and epistemological assumptions of the research project. Identifying potential barriers to persuasive communication brought out additional capacities and limitations of the field trial. Although it had always been self-evident for the project team that the transgenic trees could only gain capacities within the limits of the field trial, the impact of the research strategy on the generalisability of the results had not been systematically discussed. While some outcomes are sensitive to shifts and perturbations in test conditions, other potential risks of GM trees will always be out of reach of field trials. The future movement of the trees to explicit imagined plantation(s) led the project to acknowledge this. It was, nonetheless, open to our Innovator and his group, however, to discount the big and problematic generability question, and draw conclusions from within the analytical limits of their chosen experimental setting. Thinking through a communications strategy made the Innovator more sensitive and responsive to the project’s imagined publics and the realisation that different publics would need different messages and different media. In particular, it became evident to the Innovator that the field trial itself was not a neutral tool. Through the dialogues, the project redescriptions had progressed from ignoring the implications of the anti-GMO opposition to the field trial to building it into a strategy for engaging with them. Indeed, the Innovator’s proposal to hold a workshop demonstrated his ability to view the anti-GM lobby not only as a potential nuisance, but also as a strategic resource that could help the project to realise its societal relevance. Developing the communication strategy led the Innovator to devise a trial which would put the project’s claims to be able to mediate in the debate to the test by engaging with critical audiences and necessary collaborators. However, unfortunately neither the SUNARE researchers nor their financiers had a realistic image of the public world of the project, of its political context. In PROTEE terms we can conclude that the project lacked a coherent strategy. Little thought was given to the way the project was to answer to ‘acute political questions concerning the use of genetically modified organisms’. Because the opponents of gene technology do not approve field trials of GMOs, it was always going to be difficult, if not impossible, to enrol them by the means of one. The opposition with its banners was there, but their voices did not affect the objectives set for the project. The assumption of the SUNARE project team was that facts would speak for themselves, regardless of the credibility of the strategy by which they were produced or to whose arguments they would favour.
Science and Public Policy June 2010
Many of the ‘opposing forces’ had been recognised previously. However, because these hybrid opponents do not add value to the conduct of the specific analyses at hand, but instead appear as potential disrupters of smooth progress, no relationship with them was created. What PROTEE encouraged the Innovator to do was to rethink strategically whether such disentanglement might turn out to be disadvantageous for the SUNARE project. The Innovator gave the matter a great deal of thought, but in the end concluded that entanglement might cause more harm than benefit. The SUNARE project was shaped by postulates and axioms which left little room for reflexivity. Instead, the scientific domain seems indeed to be bounded, as Wynne (2005) argues, by certain musts, heuristic and practical choices that a scientist is expected to make in order to get funding. However, SUNARE was exceptional in the sense that the Innovator genuinely wanted the project to lead somewhere, to have a public impact. His very willingness to participate in the PROTEE experiment is one clear indication of that. Moreover, during the PROTEE evaluation the Innovator was ready to critically assess his own starting points and to redescribe the public world of his project. Evidently the goals of the research project were not empty words, written only to attract the interest of the financiers — and this is what made the project unusual. Moreover, the choices ultimately made by the Innovator are in many respects understandable. The project could not attract funds, generate knowledge and make it matter without somehow taking into account the dominant and legitimate ways to practise science and to understand risk. In so doing, however, the project effectively blocked one of its (apparently secondary) goals. Its alignment with the norms of regulatory science turned into obstacles, ironically probably reducing the policy-relevance of its results. The project’s need to demarcate between science and politics, and to stick to a specific kind of scientific practice, provided it with few options for engagement in ways that could have supported conflictmediation. Moreover, it was not only the norms of science policy and scientific practice that conditioned the Innovator's decisions. The scope for action was also shaped by the nature of the Finnish GM-debate: it was not that tempting for the Innovator to participate in a stagnant discussion that was in polarised dead-lock. Conclusions: towards policy-sensitive research?
The case study analysed in this article suggests that achieving ‘policy-relevance’ is not a straightforward exercise, but demands experimentation in, and engagement with, the realities in which the research results are expected to associate. Paradoxically the
377
Doing research, creating impact
very same choices which allow a research project to claim that it is policy-relevant may end up undermining its public role. A research project must align and make allegiances with what is already there; the alternative is to challenge the premises circumscribing agency and thereby put itself at risk. From this perspective any regime of research management that fails to test the conditions of possibility of a project and to move it to acknowledge and declare its loyalties is insufficient. PROTEE-supported redescriptions can contribute to science policy by pointing out systematic weaknesses in past projects. However, PROTEE is designed to make a difference to research project management in real time. Our PROTEE intervention was in real time, but it did not take place until SUNARE was about to come to an end. Therefore we did not expect our PROTEE dialogues to have the potential to bring about radical shifts. Nonetheless, SUNARE was at the time considering its findings and drawing conclusions. As it turned out, and as our PROTEE redescriptions illustrate, this phase of work does not proceed as straightforwardly that one might think. Even in this phase strategic decisions, enacting very particular
realities and supporting particular societal trajectories, are yet to be made. Although the experiment analysed cannot be considered a success story, it still seems to us that it indicates that the PROTEE methodology has much to offer for the management of research projects. While it may be very hard to change the public imaginations stabilised in knowledge practice, PROTEE is useful is the sense that it makes the lack of learning and reflexivity explicit. Our experiment indicates that the PROTEE approach does indeed encourage disclosure of uncertainties, contradictions, oppositions and failures. By encouraging these types of disclosures, PROTEE could be useful for learning about the challenges of trying to achieve scientific excellence with impact. However, the project redescriptions that PROTEE elicits would be ‘risky’ for most innovators to disclose to their funders. Under existing project appraisal regimes, the danger is that projects with such risky profiles would jeopardise their continued funding, if they were funded in the first place. This perhaps explains why PROTEE is itself something of a hopeful monster, still awaiting adoption for use as envisaged by its creators.
Appendix. Project redescription summary sheet Use the sheet to record the summary of ONE redescription of the project Project name Date of redescription:
............................................................................................................................. ................................................................................................................
Summary by Innovator/Evaluator*:
...........................................................................................
*delete as appropriate
Class
Record the evaluation of the redescription, for example, by making a mark along the scale 0
1
2
3
4
5
1. Realisability (Anti-ballistic)
a. Redescription is very poor b. Elements are homogeneous c. Redescription is necessary d. Uncertainty is totally undifferentiated e. Trajectories are totally unbranched
Redescription is very rich Elements are very heterogeneous Redescription is highly contingent Uncertainty is highly differentiated Trajectories are highly branched
2. Negotiability (Anti-paranoia)
a. No antiprograms b. Antiprograms are incoherent c. No flexibility for negotiation d. Pros and cons are incoherent
3. Falsifiability (Anti-manipulation)
a. Experts are homogeneous Experts are very heterogeneous b. Judgements are arbitrary Judgements are based on relevant evidence c. Alternatives: Pros and cons incoherent Pros and cons very coherent d. Trials are irrelevant Trials are highly relevant
Very many antiprograms Antiprograms are very coherent Lots of flexibility for negotiation Pros and cons are very coherent
Source: McNally and Woolgar (1999)
378
Science and Public Policy June 2010
Doing research, creating impact
Acknowledgements The authors gratefully acknowledge the support of the UK ESRC Sustainable Technologies Programme which funded the UK collaboration on this project through the award number RES-338-250013; ESRC Centre for Economic and Social Aspects of Genomics at Cardiff and Lancaster Universities, UK; and the support of the Academy of Finland (Project 209197). We also acknowledge Fred Steward and Anne-Marie Coles for their contributions to the experiment.
Notes 1. It was funded under the Transport Research and Technological Development Programme of the 4th Framework. PROTEE is an acronym for: ‘PROcedures dans les Transports d’Evaluation et de suivi des innovations considerées commes des Experimentations collective’. 2. The term ‘hopeful monster’ was coined by geneticist Richard Goldschmidt to account for the origin of new species through large-scale mutation. 3. There is also a fourth summary class of PROTEE indicators called ‘innovativeness’. 4. See Appendix for the sheet designed for this purpose. 5. Fred Steward, then of Brunel University, also participated in this meeting. 6. At the time of writing, some of the data gained from the field trial are still under analysis. 7. For a description of this debate, see Rask, 2006. 8. See CEC Directive 2001/18/EC, preamble 24. See also McNally, 1996; Valve and Kauppila, 2008; Wheale and McNally, 1990. 9. However, what had remained unwritten was an analysis of brainstorming sessions carried out by the project team. These sessions aimed to identify potential problems of the GM trees.
References Akrich, Madeleine 2000. The de-scription of technical objects. In Shaping Technology/Building Society, eds W Bijker and J Law, pp. 205–223. 3rd edn., first published in 1994. Cambridge and London: MIT Press. Freeman, Chris and Luc Soete 2000. The Economics of Industrial Innovation, 3rd edn. Continuum, London and New York. Gomart, Emilie and Maarten Hajer. 2003. Is that politics? For an inquiry into forms in contemporary politics. In Social Studies of Science and Technology: Looking Back Ahead, eds B Joerges and H Nowotny, pp. 33–61. Dordrecht: Kluwer. Hommels, Anique, Peter Peters and Wibeke Bijker 2007. Techno therapy or nurtured niches? Technology studies and the evaluation of radical innovations. Research Policy, 36, 1088–1099. Höyssä, Maria and Sampsa Hyysalo 2009. The fog of innovation: innovativeness and deviance in developing new clinical testing equipment. Research Policy, 38, 984–993.
Science and Public Policy June 2010
Jolivet, E, P Laredo and E Shove 2003. Managing breakthrough innovations: theoretical implications from – and for – the sociology of science and technology. Paper presented at the ASET Conference, Manchester. Laredo, Philippe 1999. Learning from real-time case studies. A synthesis of lessons for the PROTEE instrument. PROTEE Final Report, European Commission, Framework 4, Transport Programme Contract N ST-97-SC.2093. McNally, Ruth 1996. Political problems, genetically engineered solutions: socio-technical translations of fox rabies. In Coping with Deliberate Release: the Limits of Risk Assessment, ed. Ad van Dommelen, pp. 103–119. Tilburg and Buenos Aires: International Centre for Human and Public Affairs. McNally, Ruth and Uwe Sondermann 1999. PROTEE as SocioTechno Therapy: Case Study on the Krupp Fast Handling System. PROTEE Final Report, European Commission, Framework 4, Transport Programme Contract N ST-97-SC.2093. McNally, Ruth and Steve Woolgar 1999. Learning from the Retrospective Case Studies: a Synthesis of Lessons for the PROTEE Instrument. PROTEE Final Report, European Commission, Framework 4, Transport Programme Contract N ST-97-SC.2093. Mol, Annemarie 2002. The Body Multiple: Ontology in Medical Practice. Durham: Duke University Press. Pappinen, Ari, Yeshetila Degefu, Leena Syrjälä, Kaija Keinonen and Kim von Weissenberg 2002. Transgenic silver birch (Betula pendula) expressing sugarbeet chitinase 4 shows enhanced resistance to Pyrenopeziza betulicola. Plant Cell Reports, 20, 1046–1051. Pasonen, Hanna-Leena, Sanna-Kaisa Seppänen, Yeshetila Degefu, Anna Rytkönen, Kim von Weissenberg and Ari Pappinen 2004. Field performance of chitinase transgenic silver birches (Betula pendula): resistance to fungal diseases. Theoretical and Applied Genetics, 109, 562–570. Pasonen, Hanna-Leena, Liisa Vihervuori, Sanna-Kaisa Seppänen, Päivi Lyytikäinen-Saarenmaa, Tiina Ylioja, Kim von Weissenberg and Ari Pappinen 2008. Field performance of chitinase transgenic silver birch (Betula pendula Roth): growth and adaptive traits. Trees – Structure and Function, 22(4), 931–1890. PROTEE 2000. Procedures dans les transportes d’évaluation et de suivi des innovations considerées comme des experimentations collectives. Final Report for Publication. Brussels, European Commission. Transport RTD Programme. 4th Framework Programme. Contract No ST-97-SC.2093. Rask, Mikko 2006. Closure in risk debates. Pre-assessment of Finnish forest biotechnology. Technological Forecasting and Social Change, 73, 705–730. Valve, Helena and Jussi Kauppila 2008. Enacting closure in the environmental control of genetically modified organisms. Journal of Environmental Law, 20, 339–362. Von Weissenberg, Kim, A Pappinen, J Rikkinen and R Koivisto 2001. Interbiontic processes between modified trees, forest pests and fungi. Development of risk assessment procedure. Unpublished proposal. Wheale, Peter and Ruth McNally 1990. Genetic engineering and environmental protection: a framework for regulatory evaluation. Project Appraisal, 5(1), 1–16. Wynne, Brian 2005. Reflexing complexity: post-genomic knowledge and reductionist returns in public science. Theory, Culture and Society, 22(5), 67–94.
379
