4. Improve local, national and international exchange, learning and ..... 218â233; available online at: ftp://ftp.ige.unicamp.br/pub/CT010/aula%201/Fagerberg.pdf.
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES Grant Agreement Number: 245319
Learning Resources for the Assessment of Local Innovation
Concepts and Tools for Local Assessment Teams
Deliverable N° D3.1b
Nour Eddine Sellamna (ICRA) Final Version February 2014
JOLISAA operated from February 2010 to July 2013. Its overall goal was to assess how smallholders’ innovativeness, knowledge, capacities and other resources can be tapped into, strengthened and linked effectively to those of other stakeholders – public or private, local or global – to contribute to reducing rural poverty and improving food security in Africa. To this aim, JOLISAA had six specific objectives: 1.
Incorporate the concepts and approaches related to Innovation Systems and local knowledge into a framework and an operational approach well-suited to assessing systematically and collectively experiences about agricultural innovation across Africa; Assess in a comparative and collaborative manner a series of carefully selected past and ongoing cases of agricultural innovation involving multiple stakeholder and knowledge sources; Strengthen the capacities of members of existing innovation platforms in three African countries to assess, learn collectively from and facilitate multistakeholder innovation approaches building on diverse knowledge sources; Improve local, national and international exchange, learning and networking among diverse stakeholders and constituencies about innovation experiences and approaches; Identify, synthesise and validate collectively the corresponding results, lessons and challenges; Propose to the EC and other interested national and international bodies an itemised, collectively validated agenda for future research, practice and policy related to multistakeholder innovation approaches and local knowledge in Africa.
2. 3.
4. 5. 6.
JOLISAA formal Consortium partners: • • • • • •
•
CIRAD LEI Wageningen UR ETC ICRA UP KARI UAC-FSA
www.cirad.fr/en www.wageningenur.nl www.etc-international.org/ www.icra-edu.org http://web.up.ac.za/ www.kari.org/ www.fsa.uac.bj/
More information about JOLISAA: www.jolisaa.net/
Correct citation of this report: Nour Eddine Sellamna (2013): Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams, ICRA, Montpellier and Wageningen, pp.168.
ii
Acknowledgements This work was carried out as part of the EU-funded project Joint Learning in and about Innovation Systems in African Agriculture (JOLISAA). The opinions expressed herein are the sole responsibility of the author. Thanks are due to ICRA for allowing the author to adapt part of its training materials for this publication, especially the Innovation Assessment Tools and the Procedure for Assessment Teams. Thanks also to the JOLISAA Country Teams for their support during the National Workshops, and the test of some of the assessment tools presented here.
iii
Table of Contents Introduction............................................................................................................................................. 1 1
INNOVATION ASSESSMENT CONCEPTS ....................................................................................... 2 1.1
1.1.1
Assessment of innovation: What innovation? ............................................................ 3
1.1.2
Innovation Systems: What systems? ........................................................................... 7
1.1.3
Assessment of Processes: What assessment? What processes? .............................. 15
1.1.4
The Importance of an Approach to Assessment ....................................................... 19
1.2
2
3
Assessing Innovation Processes .......................................................................................... 3
Approaches to the Assessment of Innovation .................................................................. 27
1.2.1
Innovation Histories .................................................................................................. 27
1.2.2
Social Network Analysis ............................................................................................. 31
1.2.3
Institutional Analysis ................................................................................................. 36
1.2.4
Benchmarking Analysis .............................................................................................. 42
A PROCEDURE FOR ASSESSMENT TEAMS ................................................................................. 46 2.1
Organize A Local Team ...................................................................................................... 48
2.2
Analyse The Situation and Clarify The Issue ...................................................................... 50
2.3
Define The Operational System......................................................................................... 52
2.4
Describe the Innovation and Identify Actors..................................................................... 54
2.5
Analyse Actors ................................................................................................................... 56
2.6
Plan Action......................................................................................................................... 58
2.7
Reflect ................................................................................................................................ 60
INNOVATION ASSESSMENT TOOLS ........................................................................................... 62 3.1
TOOLS FOR TEAM WORK ................................................................................................... 62
3.1.1
Brainstorming ............................................................................................................ 63
3.1.2
Team Facilitation ....................................................................................................... 65
3.1.3
Team Work Planning ................................................................................................. 69
3.1.4
Visualization .............................................................................................................. 71
3.2
TOOLS FOR MAPPING AND TIMELINES ............................................................................. 73
3.2.1
Activity Timelines ...................................................................................................... 74
3.2.2
Farming System Models ............................................................................................ 78
3.2.3
Innovation Timeline ................................................................................................... 82
3.2.4
Mind Mapping ........................................................................................................... 85
3.3
TOOLS FOR STAKEHOLDER ANALYSIS ................................................................................ 88
3.3.1
Actor Influence Analysis ............................................................................................ 89
3.3.2
Competing Values Framework .................................................................................. 94
3.3.3
Innovation Capability Assessment........................................................................... 104
3.3.4
Organisational Linkage Matrix ................................................................................. 108 iv
3.3.5
Stakeholder Identification Matrix ........................................................................... 110
3.3.6
Stakeholder Objectives Analysis .............................................................................. 113
3.3.7
Venn Diagramme ..................................................................................................... 115
3.4
TOOLS FOR SURVEYS ....................................................................................................... 118
3.4.1
Secondary Data Analysis.......................................................................................... 119
3.4.2
Semi Structured Interviews ..................................................................................... 125
3.5
TOOLS FOR STRATEGY & DECISION MAKING .................................................................. 128
3.5.1
Action Planning........................................................................................................ 129
3.5.2
Cause & Effect Diagramme ...................................................................................... 132
3.5.3
Context analysis ....................................................................................................... 134
3.5.4
Decision Trees ......................................................................................................... 137
3.5.5
Driving Forces .......................................................................................................... 139
3.5.6
Force Field Analysis ................................................................................................. 142
3.5.7
Policy Analysis Matrix .............................................................................................. 144
3.5.8
Ranking techniques ................................................................................................. 148
3.5.9
Scenario Building ..................................................................................................... 151
3.5.10
Scoring Techniques .................................................................................................. 158
3.5.11
SWOT Analysis ......................................................................................................... 161
3.5.12
Visioning .................................................................................................................. 165
v
Introduction Who are these resources meant for? These Learning Resources for the Assessment of Local Innovation have been produced in the framework of the JOLISAA project, in support to capacity strengthening efforts. There are meant for several types of users engaging into work on local innovation: • Professionals (researchers, consultants, educators); • Students interested in knowing more about rural innovation; • Local development workers (extension and advisory work, NGOs, government organizations); • Trainers wishing to design courses on innovation.
What do we mean by assessment? The general idea underlying these resources is that there can be no universal definition of “innovation” and thus there is no standard methodology to analyze it. These resources are meant to provide users with an overview of innovation assessment issues These resources –as the Tools section indicate- are clearly oriented towards a qualitative type of assessment, conducted by a multi-stakeholder and inter-disciplinary team, with an action research philosophy. How are these resources organized? The resources are organized in three parts: • The first part gives an overview of issues raised by the assessment of local innovation systems. The overview is simply presented as an exploration of each concept: innovation, assessment, processes and systems. This overview is concluded by a presentation of some basic research concepts and principles to follow in the design of an assessment methodology. Finally, four general approaches to innovation assessment are presented, similarly organized in “What is it?”, “What is its relevance for innovation assessment?”, “How to do it?”, “Potential and limits” and “Links to further information”. Some tools are indicated for each approach. • The second part presents a general team procedure to serve as a structure to the assessment. • Finally, a Toolbox is proposed, with 29 tools arranged in five sections: Team Work, Mapping & Timelines, Stakeholder Analysis, Strategy & Decision Making, and Surveys. How to use these resources? These resources can be used in various ways. • For researchers and other people professionally engaged in research activities, they can be printed out. The tools presentation can be used as a “manual”. • For students, they can be used as separate handouts on different topics of interest to them. Those students engaged in fieldwork can, specially, benefit from the Toolbox. • For trainers, they can be easily adapted as training materials, either for a course on innovation assessment or for more specific courses / modules on the topics covered by the resources. An interactive –and extended- version of these resources will be hosted as a “JOLISAA Learning resources Bank” on the JOLISAA website (www.jolisaa.net/). Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 1
1 INNOVATION ASSESSMENT CONCEPTS
Innovation and innovation systems have become a hot topic. Yet the idea of innovation is almost as old as the idea of development itself. In most developing countries, innovation has been a central concern of development thinkers and field workers, mostly through research and technology transfer activities. The concept of innovation has however come to new prominence over the last ten years, both in the North and in the South. In the North and some emerging economies in the South, business enterprises focus increasingly on their ability to innovate, as a condition of survival. In politics, all levels of government try to design policies that stimulate innovation, to foster economic growth, competitiveness and job creation (Fagerberg and Verspagen, 2009 1). In rural development, in the South, current interest in innovation systems is the outcome of a comeback of agriculture on the development agenda, after a “fall from grace” during the eighties (Hall, 2007) 2, with an important shift in development strategy. As Hall puts it, “If we are interested in development, and if we agree that development is about change, let us worry less about the supply of new knowledge and technology from research and concentrate instead on the conditions needed to demand and use knowledge to bring about that change. The following sections attempt to give researchers, trainers and learners an overview of the conceptual and methodological issues raised by the shift to an innovation systems perspective. They will review successively the concepts of innovation and innovation systems, before concentrating on the notion of “assessment” of these systems and making some recommendations on how to better select an assessment procedure.
1
Fagerberg, J. and Verspagen, B. (2009): Innovation studies - The emerging structure of a new scientific field; Research Policy 38 (2009) 218–233; available online at: ftp://ftp.ige.unicamp.br/pub/CT010/aula%201/Fagerberg.pdf 2 Hall, A. (2007): Challenges to Strengthening Agricultural Innovation Systems: Where Do We Go From Here? UNU-MERIT, Working Papers Series; available online at: http://www.merit.unu.edu/publications/wppdf/2007/wp2007-038.pdf Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 2
1.1 Assessing Innovation Processes
1.1.1 Assessment of innovation: What innovation? The meaning of innovation Innovation has stimulated a considerable amount of research since the Second World War. A simple search on the internet yields thousands of results. Yet this wealth of information adds to confusion, as the concept is defined in multiple ways with varying emphases on a number of dimensions across disciplines, units of analysis, professional domains, geographical scales, organisational contexts, etc… As many authors have acknowledged, there can be no consensus on a single definition of Innovation. Perhaps the only shared idea, across definitions, is that innovation refers to something “new”, whether an idea, a product, a form of organization or process. But even the idea of “new” can be differently interpreted: New where? New when? New how? New for whom? Newness does not mean new knowledge; it can also be elaboration on and modification of existing knowledge. To illustrate the ambiguities surrounding the concept of innovation three simple sources of misunderstanding can be pointed out. • First, “innovation” can be interpreted as a noun (one particular technology, product, etc…) or as a process (a sequence of steps). This means that the concept of “innovation” can cover both a result and the process leading up to that result, whereas the two are analytically different. As a matter of fact, the “result” can itself be a “process”, which means that innovation, like a snake biting its own tail, can be considered as an endless process. • Second, “to innovate” is an intransitive verb, which means that we cannot “innovate something”; we can only “improve” on something. But can innovation be equated with improvement? No sure: ideas, products, processes can be transferred from one context to another without much of an improvement and still be considered “innovations” in their new context. • Finally, “innovation” almost always refers to something ‘positive” (improvement), successful, whereas the result of a process can, in fact, be “negative”, ie, produce something new that has a negative impact. But usually negative results are called “mistakes” not innovations. These ambiguities, among others, contribute to turning innovation into something of a “polymorphic” concept, one that takes a different meaning depending on the perspective of the viewer: their discipline, their professional background, their institutional interests, their beliefs and attitudes, etc… The only consensual definition that can be arrived at, by combining the minimal points of agreement is that innovation is something that is perceived as “new” and “successful”, and
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 3
represents an “improvement” of something older. In this sense, West (1990) 3 has perhaps provided the best definition by stating that innovation is the intentional introduction and application within a role, group, or organization, of ideas, processes, products or procedures, new to the relevant unit of adoption, designed to significantly benefit the individual, the group, or wider society. These three features can be consensual: novelty, application and benefit to someone. The novelty of the innovation can be only relative. First, as already mentioned, it may or may not be based on an “invention” (something that did not exist before). Second, the innovation can be new only in a particular context or locale; it may have already been in use somewhere else. This means that what is called an “innovator” can be, in fact, simply an “adopter” or “imitator”. But this is not as simple as that because the process of adoption may entail adaptations that can be considered innovations; and the imitator being in the same position as the persons he imitates (introducing something new) can also be rightfully considered an innovator (Fagerberg, 2003 4). For discussion on innovation to have any meaning there is a need to define and agree on some types of innovation5. One of the most recent sources of definition is the Oslo Manual, developed by the OECD and the EU (2005), which defines 4 types of innovations (see box).
Product Innovation: it is the process of developing and diffusing a new or adapted version of a product or service. The adaptation to the product may be a change of design, technical specifications, use of new materials and components in its manufacture that enhance its performance. In the industrial and services sectors, the “diffusion” of the product means its release on the market. Process Innovation: it is the implementation of a new or significantly improved production or delivery methods. These methods may involve changes in equipment or production organization or both, and may be intended to produce new or improved products, which cannot be produced using conventional production methods, or essentially to increase the production efficiency of existing products. Marketing Innovation: it is the implementation of a new marketing method involving significant change in product design or packaging, its placement, promotion or pricing. The type of innovation aims at better addressing customer needs, opening new markets or positioning a product on the market. Organizational Innovation: it is the implementation of a new organizational method in the organization’s business practices, workplace organization or external relations. It is intended to increase the organization’s performance by reducing costs, improving workplace satisfaction and productivity and gaining access to knowledge.
This largely accepted typology of innovation describes the innovation but does not tell much about its degree of novelty or the extent of the changes its brings about. For this, Henderson and Clark’s 3
West, M.A. 1990. The Social Psychology of Innovation in Groups. In M.A. West and J.L Farr (Eds.), Innovation and Creativity at Work: Psychological and Organizational Strategies; Chichester, UK: Wiley, pp. 309-334. 4 Fagerberg, J. (2003): Innovation: A Guide to the Literature; Centre for Technology, Innovation and Culture, University of Oslo; available online at: http://folk.uio.no/janf/downloadable_papers/03fagerberg_innovation_ottawa.pdf 5 There are many typologies possible. The Oslo Manual’s is based on the nature of the innovation. The earliest typology of the nature of innovation is probably that proposed by Schumpeter: new goods, methods, market, inputs / materials, and work organization. Other typologies are, for example, those based on the complexity of the innovation (simple, systemic, radiating), the origin of the innovation (invention, transfer, replication), the innovation domain (technology, services, logistics, market). Innovations have also been labeled cumulative (more than one innovation at the same time), frugal (when reduced to their bare minimum functions), inclusive (when adapted to local conditions), open (a mode of innovation that is based on sharing and exchanging, and cooperation between actors), social (innovation strategies that aim at meeting social needs), spasmodic (episodic innovations in an otherwise stable environment), vertical (when a technology is built top of another technology by adding a new element to it, in a process of vertical integration). Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 4
(1990) 6 analytical framework provides a useful typology (see box for descriptions and implications) that characterizes innovations according to their impact on the system into which they are integrated. The typology represents the extent to which the innovation is a departure from existing practices.
COMPONENTS/ CORE CONCEPTS REINFORCED OVERTURNED UNCHANGED
Incremental Innovation
Modular Innovation
CHANGED
Architectural Innovation
Radical Innovation
SYSTEM/ LINKAGES
The difficulty to find a consensual definition should not prevent people interested in innovation study to proceed. If one needs to study and solve a problem, one has first to define one’s terms. The following terms have, in general, been accepted by all innovation researchers: • Innovation is something new, but this “something” can range from a product or a technology, to a process, a form of organisation, a new opportunity, etc… or even a combination of all these. Similarly “new” does not mean “original”; • Innovation is different from invention, but that “invention” can be a minor improvement of something already existing, or indeed the replication (imitation) of something existing in another context. Sometimes invention and innovation can be so closely linked that it becomes difficult to distinguish one from the other; • Innovation may not follow immediately on an invention; the time lag between the two can be several decades; • Innovation is a continuous process, so that what we think of as a single innovation may, in fact, be the result of a lengthy process involving many interrelated innovations. Innovation types - Incremental Innovation: is where there is a series of small improvements to existing components … - Modular Innovation: is where the architecture of a construction (technological system, production system) is maintained and the components (modules) are modified. It is considered as a form of “incremental” type of innovation. - Architectural Innovation: is where the architecture of the construction is itself changed, often without change in the components. But it reconfigures these in ways that make it “revolutionary”. - Radical Innovation: is based on different principles. The changes entailed in radical innovations mean that existing technologies, production systems and organizational structures become obsolete. Existing organizational knowledge and competence are rendered irrelevant.
•
Innovation is not simply about the creation of new products but also new processes; these processes may be “technological process innovations” (leading to new products) or “organizational process innovations” (leading to new ways to organise things). But these organizational innovations are not limited to the process of production within a given unit (factory, farm, institution), they may concern entire economic sectors (industry, agriculture,
6
Henderson, R.M. and Clark, K.B. (1990): Architectural Innovation: The Reconfiguration of Existing Product Technologies and The Failure of Established Firms, in Administrative Science Quarterly; 35, 1; available online at: http://dimetic.dimeeu.org/dimetic_files/HendersonClarkASQ1990.pdf. Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 5
•
•
services). Indeed, some of the most radical societal changes have been brought about by such economic sector – wide innovations. Innovation capacity is not limited to scientists or industrialists, nor is it a privilege of scientific knowledge. There are local innovations, based on local, non-formalised, knowledge. But local innovation processes are overlooked in the search for new solutions. Three reasons are: that these innovations are mostly incremental and go unnoticed; that there is little knowledge sharing, due to lack of documentation and isolation; and researchers often stress the importance of market or economic outcomes of innovations when local innovations may not have a noticeable market impact. No single discipline can deal with all aspects of innovation. Interdisciplinarity is therefore required.
Innovation models Our understanding of innovation and innovation processes shapes the way we analyse them and, consequently, our strategy to support them. Yet in research and development, understanding of innovation has considerably changed overtime, and mental models of innovation have evolved from simple, linear models to more complex, systemic models. Rothwell for example identifies 5 “generations” of innovation models: -The first two are linear models, whereby innovation is seen as the outcome of ‘need pull’ (demand driven) and, second, “technology push” (supply driven) which prevailed until the seventies. • The third is a model whereby technological innovation arises from the matching (‘coupling’) of markets needs and technological opportunities, through interaction between different elements and feedback loops between them. • The fourth model, prevalent from the eighties, emphasizes alliances, linkages and integration within the organisation (in this case the firm), downstream with customers and upstream with suppliers. • The fifth model, influenced by both resource constraints and the expansion of knowledge sources, moved on to a larger system integration, extensive networking and partnerships, flexible and customized responses, fast and continuous innovation to respond to rapid changes. Conceptual models of innovation have therefore moved from simple, linear, ideas on to more complex constructions that recognize the “messy” nature of innovation and the role of many actors up and downstream. Innovation is now recognized as an iterative process, in which there are ‘loops’, trials and errors before getting to the final outcome -and this outcome is sometimes not exactly the one originally aimed at. These conceptual changes–originating in industrial innovation thinking and spreading to other fieldshave been greatly influenced by changes in the markets in which industrial firms operate (economic growth, industrial expansion, more intense competition, inflation, unemployment and resource constraints): from the post-war belief in the power of scientific and technological breakthroughs to solve society’s problems (technology push), to the demand of expanding markets (need pull), to the realization that innovation requires adaptation of technology supply to market demand and, finally, that innovation is part a ‘system’ larger that what one organization can achieve. Consequently, the role of research and development (R&D) from being conceptualized as the main source of innovation and often organized separately from other departments has also gradually changed. Considered first as sole source of innovation, it was linked to other functions, specially Marketing (to respond to market needs), then integrated into the whole organization, recognizing Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 6
the need to innovate also in organizational processes and not only in products, to finally be encompassed in wider networks that extend outside the organization. In rural development, there have been similar conceptual changes. Although not exactly in the same terms, but under the same broad economic influences, rural innovation processes have also been successively conceptualised into a linear ‘transfer of technology’, ‘pipeline’ or ‘treadmill’ model, a ‘participatory technology development model’, a ‘farming system research and extension’ model, a ‘participatory learning’ model, etc….The role and contribution of projects and actors operating outside the formal ARD sector has been recognized. Under different names, the same central idea remains: an integrated and dynamic systemic model of rural innovation is made necessary by the changing nature of agriculture because, increasingly, farmers are facing complex issues (sustainable resource use in the face of climate change, demography; demand for product quality and standards; increasing market competitiveness, etc…). These new challenges (collective management of natural resources, chain management, collective marketing and input supply, organization building) transcend the level of individual farms and force us to move to: • A model of co-innovation in new, dynamic and interactive multi-actor networks / platforms that encourage collective learning and a pooling of resources for knowledge and action • Favouring approaches that facilitate the emergence of context-specific solutions, building on farmers’ innovation practices rather than centrally-imposed blueprints • Strengthening local capacity to constantly innovate and adapt to changing economic, natural and political environments; innovation processes become therefore also social learning processes meant to enhance the innovation capacities and knowledge bases of stakeholders and of the local, regional and national innovation systems
1.1.2 Innovation Systems: What systems? In innovation studies, the concept of innovation “system” dates back to the 1970s, and was originally related to technology and Research and Development in industrialized countries (Godin, 2010 7). At its most basic, the concept means that innovation and technology development are the results of complex sets of relationships among various actors, including enterprises, universities, public and private research institutions, etc…; they are no longer viewed as the outcome of single actors.
7
Godin, B. (2010): “Innovation Studies”: The Invention of a Specialty (Parts I and II); Working Papers No. 7 and 8, Project on the Intellectual History of Innovation, Montreal : INRS; available online at: http://www.csiic.ca/. Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 7
The growing importance of the concept is perhaps best exemplified by the adoption of the national innovation systems (NIS) framework for the conduct of national science and technology reviews by countries members of the Organization for Economic Co-Operation and Development (OECD) and SOME DEFINITIONS Innovation System: Network of organizations, enterprises, and individuals focused on bringing new products, new processes, and new forms of organization into economic use, together with the institutions and policies that affect the system’s behavior and performance (World Bank 2006b) Agricultural Innovation System (AIS): a collaborative arrangement bringing together several organizations working toward technological, managerial, organizational, and institutional change in agriculture. Such a system may include the traditional sources of innovations (indigenous technical knowledge); the modern actors (NARIs, international agricultural research institutes, and advanced research institutes); private sectors, including (local, national, and multinationals) agro-industrial firms and entrepreneurs; civil society organizations (NGOs, farmers and consumer organizations, and pressure groups); and those institutions (laws, regulations, beliefs, customs, and norms) that affect the process by which innovations are developed and delivered (Anandajayasekeram, 2011) National Innovation System (NIS): the elements and relationships which interact in the production, diffusion and use of new, and economically useful, knowledge ... and are either located within or rooted inside the borders of a nation state.” (Lundvall, 1992) Local Innovation System: a set of localized network of actors (firms and organizations) devoted to generate, transform and diffuse knowledge. In industry, a local innovation system has been defined as a “spatial concentration of firms (including specialized suppliers of equipment and services and customers) and associated non-market institutions (universities, research institutes, training institutions, standard-setting bodies, local trade associations, regulatory agencies, technology transfer agencies, business associations, relevant government agencies and departments, etc…) that combine to create new products and/or services in specific lines of business”. Regional Innovation System (RIS): A set of interacting private and public interests, formal institutions and other organizations that function according to organizational and institutional arrangements and relationships conducive to the generation, use and dissemination of knowledge …that encourage firms within the region to develop specific forms of capital that is derived from social relations, norms, values and interaction within the community in order to reinforce regional innovative capability and competitiveness (Doloreux). Sectoral Innovation System (SSI): A set of new and established products for specific uses and the set of agents carrying out market and non-market interactions for the creation, production and sale of those products. Sectoral systems have a knowledge base, technologies, inputs and demand (Malerba, 2002). Technological Innovation System: A dynamic network of agents interacting in a specific economic/industrial area under a
particular institutional infrastructure and involved in the generation, diffusion, and utilization of technology (Carlsson and Stankiewicz, 1991)
some industrializing countries from the mid-90s, as well as the production in 1997 of a manual (the “Oslo Manual”) which has become the primary source of reference with regard to the innovation system framework. The growing literature on innovation systems presents a diversity of conceptualizations. Depending on their point of departure, (a particular technology, geographical area, research theme, problematique), the system has different boundary and components. Thus, reference is made to different types of innovation systems: local, regional, national, territorial, socio-technical, sectoral, etc…
Systems within systems In developing countries, perhaps the strongest influence from innovation studies in the industrialized North has been the increasingly prominent focus on the importance of Science, Technology and Innovation (ST&I) in rural development and, with it, the system view of innovation. More indirectly, this influence is also represented by the rapid spread of projects focusing on value chains, themselves strongly influenced by Michael Porter’s theory of the role of innovation in achieving competitive advantage. But this influence lies more in the new focus on “innovation” than on the Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 8
systemic approach. The latter has a “tradition” in rural development, dating back to at least the 1970s. In rural development, the systemic approach has developed gradually by changing levels and focus of conceptualization. Applied to farming (Cropping systems, Farming systems, Livestock systems) in the 70s, the systemic approach aimed at integrating the complex interactions between all the elements that influence the rural household’s agricultural activities. This required a move from a focus on individual crops to a larger focus on the whole range of farming enterprises, and therefore from disciplinary to interdisciplinary analyses. Even broader frameworks are the “agrarian system” and the “livelihood system” approaches, that include, among others, land tenure systems, the role of institutions, the complementarity between on-farm and off-farm activities and sources of income, the key role of women, etc… A careful system analysis ideally leads to a typology of different systems which share the same characteristics; this helps better target development action and policy support. Outside agricultural production, the systemic approach has also been applied to the analysis of single commodities (Commodity systems / chains, filières), integrating all the components bearing on market performance, taking the commodity as the starting point, to include the supply of inputs, through to the farm production process to food processors, distributors, retailers and consumers. All these approaches share some common principles, namely the importance of: • Interactions between system components • Interdisciplinary and inter institutional analysis of the system • The influence of the context (markets, policy, institutions, etc…) • Participation of system actors in analysis and action planning At the more institutional, or support to farmers level, system analysis emerged mostly as a critique, and alternative to conventional, linear or “treadmill” approach to research and development (Röling, nd) 8. The National Agricultural Research System (NARS) framework, by putting research at the centre of the system, clearly equated knowledge generation process with innovation process (Assefa, Waters-Bayer, Fincham and Mudahara 9 (nd). The implication of this is that optimizing public investments in research organizations is the right means to develop technologies to foster agricultural transformation and rural development (Ugbe, 2010) 10. Taking a broader view, the Agricultural Knowledge and Information System (AKIS) framework (developed in the late 1980s by researchers at Wageningen Agricultural University in the Netherlands) sees all actors as capable of producing, adapting and disseminating knowledge. In AKIS, research and extension should not be seen as separate processes involving distinct institutions that must somehow be linked. Rather, scientists, specialists, extension workers, consultants and producers should be seen as participants in a single Agricultural Knowledge and Information System (Kaine, Doyle and Lees, 1999) 11. A shared responsiveness of all AKIS actors to rural people and an 8
Röling, N. (nd): Conceptual and Methodological Developments in Innovation; Available at: http://www.worldfishcenter.org/sites/default/files/Conceptual%20and%20methodological%20developments%20in%20inn ovation%20shared%20by%20Ann%20Waters%20Bayer.pdf. 9 Amanuel A., Waters-Bayer, A., Fincham, R. and Mudahara, M. (nd): Comparison of Frameworks for Studying Grassroots Innovation: Agricultural Innovation Systems (AIS) and Agricultural Knowledge and Innovation Systems (AKIS); In: Sanginga P, Waters-Bayer A, Kaaria S, Njuki J & Wettasinha C (eds), Innovation Africa: enriching farmers’ livelihoods (London: Earthscan), pp35–56; Available at: http://www.cgiar-ilac.org/files/Assefa_Comparison.pdf. 10 Ugbe, U.P. (2010): What Does Innovation Smell Like? A Conceptual Framework for Analysing and Evaluating DFID-RIU Experiments in Brokering Agricultural Innovation and Development; DFID‐RIU Programme, Nigeria; available online at: http://r4d.dfid.gov.uk/PDF/Outputs/ResearchIntoUse/riuinnovationsmellutiang.pdf. 11 Kaine, G., Doyle, B., Reeve, I. and Lees, J. (1999): Agricultural Knowledge and Information Systems: A Network Analysis; Paper presented to the 43rd Annual Conference of the Australian Agricultural and Resource Economics Society, Christchurch, New Zealand; Available at: http://ageconsearch.umn.edu/bitstream/123823/2/Kaine.pdf. Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 9
orientation towards their goals promote mutual learning and ensures synergies that enhance innovation. At its basis, the AKIS approach asserts that innovation is a collective competency. As Spielman and Birner (2008) 12 define it, “the Agricultural Innovation System (AIS) framework represents a move away from a more linear interpretation of innovation as a sequence of research, development, and dissemination, to an interpretation that recognizes innovation as a complex web of related individuals and organizations—notably private industry and collective action organizations—all of whom contribute something to the application of new or existing information and knowledge. The framework addresses novel issues such as the capacity of individuals and organizations to learn change and innovate, the nature of iterative and interactive learning processes among innovation agents, and the types of interventions that enhance such capacities and processes”. An attempt has been made to compare and contrast the three main systemic frameworks, in the table below. The main characteristics of the three main frameworks used in promoting and investing knowledge in the agriculture sector Defining feature Actors
NARS Research organisations
AKIS AIS Farmer, research, Wide spectrum of actors extension and education Outcome Technology invention and Technology adoption and Different types of technology transfer innovation innovation Organizing principle Using science to create Accessing agricultural New uses of knowledge for new technologies knowledge social and economic change Mechanism for Technology transfers Knowledge and Interaction and innovation innovation information exchanges among stakeholders Role of policy Resource allocation, Linking research, Enabling innovation priority setting extension and education Nature of capacity Strengthening Strengthening Strengthening interactions strengthening infrastructure and human communication between between all actors; creating resources actors in rural areas an enabling environment Source: Rajalahti, R. (2009): Promoting Agricultural Innovation Systems Approach: The Way Forward
In their rejection of the idea of the linear model of production and transfer of technology inherent in the NARS framework, and their emphasis on collective learning as the basis for innovation, AKIS and AIS are very close 13 but not immune from criticisms. The research-based approach to innovation, represented by the NARS framework has been criticized for being too linear, and neglecting the innovativeness of actors other than researchers. Similarly, the idea that strengthening research institutions will promote technological progress and innovation has not been validated, despite important funding efforts made in building the research capacity of national research institutions. Greater numbers of researchers and increased supply of knowledge are not in and by themselves sufficient conditions for enhanced agricultural innovation. Similarly, the
12
Spielman, D.J. and Birner, R. (2008): How Innovative Is Your Agriculture? Using Innovation Indicators and Benchmarks to Strengthen National Agricultural Innovation Systems; Agriculture and Rural Development Discussion Paper 41; The World Bank; Available at: http://www.share4dev.net/ffsnet/documents/4222.pdf 13 So close in fact that the two have been combined into the Agricultural Knowledge and Innovation Systems framework by the European Commission’s Standing Committee on Agricultural Research (SCAR). The Royal Tropical Institute (KIT) in Amsterdam has also developed an Agricultural Innovation System Analysis (AISA) methodology, based on RAAKS, a methodology originally meant for the analysis of AKIS. Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 10
AKIS framework 14, despite its inclusiveness, has been criticized for failing to take into account the diversity of research actors (Spielman, 2005) 15. Articulation between various approaches to innovation systems
NATIONAL AND GLOBAL INNOVATION SYSTEMS National Agricultural Innovation System National Agricultural Information System
Agricultural Knowledge and
Livelihood System Promotion of knowledge and information exchanges to foster innovation
Agrarian System
Farming System Cropping System
Non-farm activities
Food System
Research System
Livestock System
Institutional system
Production and transfer of technology
Land tenure system
Value Chains
The figure above attempts to represent how different types of systems are articulated. This representation is also that of the historical shift in thinking, over the last 40 years, from lower levels of system definition, with a focus on production and technology, to broader institutional and knowledge systems. What the figure does not clearly show is that these broader institutional and knowledge systems are not mutually exclusive with the lower systems. Indeed the broader systems can be interpreted as either encompassing all the lower systems or as focused on each of them (an innovation system of farming or livestock for example). The move from research to knowledge and information and on to innovation is, at the same time, an attempt to shift to a clearer focus on development impact through continuous innovation and a higher level of conceptualization. Being successful in the achievement of this dual objective is the challenge that the innovation system framework must still demonstrate: bringing new insights for policy making without falling into academia, ie, losing sight of its claim to better and more focused action to bring about significant change. As Spielman (2005) observed, although it may contribute to changing outlooks of researchers and policy makers, “there is little evidence to suggest that the application of the innovation systems framework to developing-country agriculture is, in fact, providing real solutions to many of today’s challenges”. The framework is yet trying to develop methodologies to live up to its ambitions of helping shape better policies and guide practical action.
14
The AKIS framework, like all “system” analyses, has been strongly criticized, notably by socio-anthropologists. In the case of AKIS, in the early 90s a debate opposed AKIS proponents, mostly from agricultural extension, and sociologists in the University of Wageningen. The criticisms revolved around a main issue: the system view of knowledge ignores the fact that knowledge is a social and cultural product, and thus knowledge processes are also social processes, AKIS therefore ignores issues of power and social conflict. 15 Spielman, D.J. (2005): Innovation Systems Perspectives on Developing-Country Agriculture: A Critical Review; ISNAR Discussion Paper 2, IFPRI. Available at: http://www.ifpri.org/sites/default/files/publications/isnardp02.pdf. Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 11
Local innovation systems: How local? Local innovation systems have been conceptualized from a variety of perspectives (local, regional, territorial) and disciplines (geography, economics, politics). In economic geography, regional development and other fields of research, the innovation focus has shifted to lower levels. One of the main arguments is that in an era of economic globalization and removal of national boundaries, the national level (nation-state) has lost some of its strategic economic capabilities. “Regions, especially when they have developed clusters and appropriate administrative machinery for supporting innovative enterprise, represent more meaningful communities of economic interest, define genuine flows of economic activities and can take advantage of true linkages and synergies among economic actors » (Cook and Memedovic, 2002) 16. It is at this level that social learning, social capital and other local specificities become more and more determining factors in innovation. The term “regional” is often loosely defined as smaller than the “national” level but larger than the level of a single organization or town. From an economic perspective, the concept of “regional cluster” is sometimes used. “Clusters can be characterized as a dense network of economic actors, who work together very closely and who have intensive exchange relationships. All economic actors who directly contribute to the dominant production process of a region are partners in this network” (Cook and Memedovic, 2002). This imprecision is inherent in the regional approach to innovation system study. What represents a region differs from one country to another and from one type of economic activity to another. It also depends on the issue under study; some issues call for larger boundaries than others. In any case, “regions” here do not necessarily coincide with administrative units. This means that depending on economic activities and researched issues, the distinction between “regional” and “local” innovation systems becomes blurred. Despite the imprecision, geography’s spatial approach to innovation has much to offer the study of rural innovation. It is based on the idea of a spatial polarization of knowledge production, diffusion and innovation activities. It argues that despite economic globalization, innovation is not randomly distributed over space, but concentrates in certain locations. This spatial concentration of innovation activities lies in the ability of geographical proximity to facilitate interactive learning and knowledge exchange (Martin, 2012) 17. Physical proximity, shared history, attitudes and outlooks, continuous interactions all contribute to forming a local innovation system. Thus, as Asheim and Gertler (2006) 18 argue, “geography is fundamental, not incidental, to the innovation process itself: one simply cannot understand innovation properly if one does not appreciate the central role of spatial proximity and concentration in this process”. National or Regional Innovation Systems are probably better levels to assess innovation processes when we aim at policy formulation. But it is at the local level that the concept of innovation system can best be operationalized and innovation processes and diffusion assessed, for two reasons. First, as geographers and most anthropologists emphasize, agricultural innovation starts locally and its diffusion is locally polarized. Second, the notion that the system’s capacity to innovate depends on the density and quality of interaction between innovation actors is best checked at the local level. It is at that level that physical proximity allows joint learning, a condition for continuous innovation. 16
Cook, P. and Memedovic, O. (2002): Strategies for Regional Innovation Systems: Learning Transfer and Applications; UNIDO; Available at: http://www.unido.org/fileadmin/user_media/Publications/Pub_free/Strategies_for_regional_innovation_systems.pdf . 17 Martin, R.(2012): Knowledge Bases and The Geography of Innovation; Lund University; Available at: http://www.lunduniversity.lu.se/o.o.i.s?id=24732&postid=3049932 18 Asheim, B. T. and Gertler, M. (2005): The Geography of Innovation: Regional Innovation Systems, in Fagerberg, J., Mowery, D., and Nelson, R. (eds.), The Oxford Handbook of Innovation. Oxford University Press, Oxford, 2005, 291-317. Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 12
Boschma (2005) 19 argues that geographical proximity, as such, is not a sufficient condition for innovation. There are other types of proximity (table below). However, geographical proximity allows a better flow of local / tacit knowledge, which is not easily coded and transferred. Five forms of proximity: some features Too little Too much Key dimension proximity proximity 1. Cognitive Knowledge Misunderstanding Lack of sources gap of novelty
2. Organizational 3. Social 4. Institutional
5. Geographical
Control
Opportunism
Trust (based on Opportunism social relations) Opportunism Trust (based on common institutions) No spatial Distance externalities
Possible solutions Common knowledge base with diverse but complementary capabilities
Bureaucracy
Loosely coupled system
No economic rationale
Mixture of embedded and market relations
Lock-in and inertia
Institutional checks and balances
Lack of geographical
Mix of local ‘buzz’ and extra-local
Source: Boschma (2005)
Similarly, it is perhaps at the local level that routines, established practices, rules and regulations influencing interaction between individuals and groups –that is all the “institutional” dimensions of innovation- are to be found. This obviously does not mean that local actors are better innovators. Dense social ties can enhance innovation by facilitating interactive learning and strengthening the other dimensions of proximity. They are also important in the transfer of complex knowledge because of the need for feedback. But dense ties can also stifle innovation when close relations lead to “lock-in” of exchange and to inward-looking systems. In this case, power asymmetries between actors create relations of dependencies and limit access to the new knowledge and information that innovation requires. Todoa et al (2011) 20 for example show that, in Ethiopia, geography affects the formation of social networks, but that networks based on frequent meetings promote knowledge diffusion but does not necessarily promote adoption of new technologies. Economic geography offers many insights on “geographic clustering of innovation”. For example, Stiller-Shulman, Lybecker and Johnson (2010) 21 review the debate about this “clustering” and point out several causes. • •
First, one of agriculture’s main features is environmental variability, which inhibits large-scale diffusion of new technologies. Even when successful, innovations may be suited to a particular region but useless in another. Another feature is the unpredictability of events (such as frost, drought, pest infestation, etc…) which increases risks and reduces expected revenues. The threat of going bankrupt at any time because of external factors discourages innovation strategies. As the authors
19
Boschma, R. (2005): Proximity and Innovation: A Critical Assessment', Regional Studies, 39: 1, 61-74; Available at: http://www.drkresearch.org/resources/boschma_proximity.pdf 20 Todoa, Y., Yadateb, D.M., Matousc, P. and Takahashid, R. (2011): Effects of Geography and Social Networks on Diffusion and Adoption of Agricultural Technology: Evidence from Rural Ethiopia; Available at: http://www.csae.ox.ac.uk/conferences/2011-EDiA/papers/407-Todo.pdf. 21 Stiller-Shulman, A., Lybecker, K. M. and Johnson, D.K. N. (2010): Fruits of Their Neighbors - The Role of Geography in Agricultural Innovation; Available at: http://ssrn.com/abstract=1674595 Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 13
• • • •
summarize it “it takes the “nothing ventured, nothing gained” mentality and flips it on its head. In agriculture, the saying should be “nothing gained, everything ventured.” Innovation is driven by market conditions, and agriculture is perhaps the industry closest to conditions of perfect competition. But if this gives agriculture the drive to innovate, it has few resources to do it. Although agriculture in the North and in part of some Southern countries has been transformed by science based technological progress, the agricultural sector is still reputedly resistant to technological change; this deters innovators from producing new technologies. Population growth stimulates demand for innovation. Innovation is reputedly correlated to population density, but rural areas have lower densities. Finally, agriculture has suffered from much less protection in terms of intellectual property rights.
Traditional Knowledge “Traditional knowledge refers to the knowledge, innovations and practices of indigenous and local communities around the world. Developed from experience gained over the centuries and adapted to the local culture and environment, traditional knowledge is transmitted orally from generation to generation. It tends to be collectively owned and takes the form of stories, songs folklore, proverbs, cultural values, beliefs, rituals, community laws, local language, and agricultural practices, including the development of plant species and animal breeds. Traditional knowledge is mainly of a practical nature, particularly in such fields as agriculture, fisheries, health, horticulture, and forestry.” (Convention on Biological Diversity, 2006) Indigenous Knowledge “..Commonly refers to a tacit knowledge built over time by a group of people in a specific locale. This includes a system of classification, a set of empirical observations about the local environment, a system of self-management that governs resource use, and means for the passing of knowledge from one generation to the next (Vazquez, 2011). Local Knowledge Local knowledge is a collection of facts and relates to the entire system of concepts, beliefs and perceptions that people hold about the world around them. This includes the way people observe and measure their surroundings, how they solve problems and validate new information. It includes the processes whereby knowledge is generated, stored, applied and transmitted to others. (Warburton and Martin (1999.)
Local innovation and local knowledge Assessing local innovation systems is akin to assessing processes of knowledge production, transformation and diffusion. If the notion of “local innovation system” is conceptually imprecise, that of “local knowledge” is no less difficult to define. To define such knowledge, concepts like “traditional knowledge” and “indigenous knowledge” are often used inter-changeably. Yet these have distinct meanings and come from different schools of thought (see Box). Local knowledge is the knowledge that people in a given community have developed over time, and continue to develop. It is the basis for local-level decision making in agriculture, health care, food processing, education, natural resource management, etc… in rural communities. It is based on experience and tested over long periods of time. As such, it is: • Adapted to the local culture and environment • Embedded in community practices, institutions, relationships and rituals • Held by individuals or communities • Dynamic and changing These general features make the concept of “local knowledge” more inclusive than other definitions. “Local knowledge is not confined to tribal groups or to the original inhabitants of an area. It is not Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 14
even confined to rural people. Rather, all communities possess local knowledge - rural and urban, settled and nomadic, original inhabitants and migrants. There are other terms, such as traditional knowledge or indigenous knowledge, which are closely related, partly overlapping, or even synonymous with local knowledge. The term local knowledge seems least biased in terms of its contents or origin. As it embraces a larger body of knowledge systems, it includes those classified as traditional and indigenous” (FAO). What to make of systems? The figure below attempts to visualize the inter-connectedness of the different systems discussed above.
Figure 1: Embeddedness of innovation systems
Integrating these embedded systems –of different levels and of different nature- into an assessment methodology is a daunting task. But these systems are essentially “human” (soft) systems, not mechanical or biological ones. Like all human systems, they have no existence of their own, they are socially constructed. “Instead of seeing (the concept of innovation system) as a new, and perhaps, competing approach, we should view it as a metaphor for innovation diversity. In order to deal with the shocks and opportunities that the modern world throws at us, we need different approaches to innovation; different ways of bringing together ideas and technology” (Hall, 2007). But viewing systems as metaphors is not simply for the representation of innovation diversity. It is also for methodological pragmatism. In assessment, they should be approached as frameworks of analysis rather than actual –existing- entities. The type(s) of systemic approach that is selected depends on the questions to be answered by the assessment, and its objectives.
1.1.3 Assessment of Processes: What assessment? What processes? In the dictionary, assessment is synonymous with appraisal or evaluation. Assessment is “to estimate or judge the value, character, etc., of something”, “the classification of someone or something with respect to its worth”. To evaluate is “to assess one's efforts”. The notion of assessment is therefore Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 15
akin to the notion of value judgment. Not surprisingly, it is very often used in the fields of education (evaluation of a student's achievement), finances and taxes (valuation set on taxable property, income, etc). In innovation studies, the notion of “assessment” is ambiguous. Is it “evaluation”? What is to be “assessed” exactly? Is it innovation performance? If it is performance, are we speaking of economic or social performance? Does it imply “measurement”? Looking up definitions offers little in the way of clarification. •
“Research” is the investigation of a particular topic using a variety of reliable resources, with the aim to establish facts, analyze information, reach new conclusions or devise new applications. Innovation research has extensively dealt with the analysis of innovation, sometimes using deductive reasoning – using “upstream” proxies for innovation such as investment in research and development and counts of innovation patents-, sometimes more empirical analysis –using “downstream” indicators such as commercialization activitiessometimes focusing on the innovating unit.
•
"Evaluation” has been defined as “a process that seeks to determine as systematically and objectively as possible the relevance, efficiency and effect of an activity in terms of its objectives, including the analysis of the implementation and administrative management of such activities” (Papaconstantinou and Polt, 1997) 22. As Ekong (2013) 23 points out, evaluation “conjures up the idea of an external ‘judgement’ of performance after a number of years of programme resulting in external recommendations for improvement”. “Measurement” is the assignment of numbers to objects or events, in order to increase precision and make comparisons between observations.
All these concepts can be said to be forms of assessment in a general sense. Yet an assessment need not have such formalism. Innovation activities do not necessarily have “objectives” or “administrative management” procedures that can be evaluated. This difference points to a more important one: the context of the innovation to be assessed. Evaluation procedures have been developed to assess projects and programmes, when local innovation processes may exist outside the framework of projects or programmes. Assessment of local innovation processes needs therefore to avoid turning into a project evaluation. The difference between these different forms of assessment lies in the context in which the assessment takes place (research project, project evaluation, economic analysis, etc…). The meaning of “assessment” is, eventually, decided by those who conduct it, depending on context and objectives. Here we can take “assessment” to have the same general meaning as research or evaluation –without necessarily being conducted in the formal context of professional research or evaluation. With them, it shares the need to have a formal framework –or an “approach”- which gives a rationale to the assessment exercise and a justification to the choice of methods used. A formal framework is also necessary when comparisons or cross-analyses are needed between several assessment exercises. But assessment does not require performance “measurement” or quantification. Neither does it require formal research hypotheses to be tested through sophisticated procedures, if the assessment does not take place in the context of a formal research process. As Ekong (2013) says, assessment “should be an iterative learning process, primarily for the
22
Papaconstantinou, G., Polt, W.(1997): Policy Evaluation Innovation and Technology: an Overvie/ In “Policy Evaluation in Innovation and Technology: Towards Best Practices” OECD. 23 Ekong, J. (2013): Grasping Complexity – Assessing Capacity Development in Agricultural Innovation Systems; Keynote paper for FARA Side Event: Strengthening Capacities for Agricultural Science and Innovation: From Framework to Networks and Impacts, ICRA. Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 16
stakeholders in that process. Indeed, the assessment process should itself contribute to the capacity development effort”. Assessment of innovation systems can take different forms and focus on different levels, or integrate all levels (cf. next section), and pursue different objectives: • Assessment of single innovation: the process, impact, performance, etc… in order to facilitate processes for example. • Assessment of innovation capacity: the performance of actors in initiating / conducting the innovation process in order to pinpoint weaknesses and build individual and collective capacity? • Assessment of innovation system efficiency: the performance of the whole system, in order to prioritize investments in R&D. In this case, the assessment can take the form of ex-ante or ex-post evaluations 24. This complexity – finding a working definition of innovation, deciding on a level of assessment, covering individual and institutional dimensions- suggests that no single assessment can give a good account of innovation processes at a systemic level. Kuhlmann (2003) 25, for example, reviews the German experience in the evaluation of research and innovation policies and describes a “shell model” of evaluation with three layers: at the centre, the first layer evaluates the individual research performance through a classic peer review procedure, seconded by individual and group research performance; the second layer is broader impact analyses of research and innovation policy programmes, using approaches of policy analysis; and the third layer is the evaluation of research institutions.
What processes? A dictionary definition of a process is that of “a series of actions, changes, or functions bringing about a result”. Such a generic definition does little to help understand the meaning of innovation processes, because of the “polymorphic” meaning of innovation. Like in the famous Russian dolls, there are innovations-within-innovations, and therefore processes-within-processes. The nature of the processes varies according to the level at which innovation is considered. At each level, the issues, actors, competencies, etc… are different. This has implications for the focus of assessment, its methodology and analysis. Some levels at which innovation is often defined are: • Innovation as the adoption of new technologies / inputs / forms of organisation : in this case, the process implied is the generation, development, diffusion of one particular innovation; ie, butterfly farming, drip irrigation, new seed, variety, fertilizer, etc. that gives a comparative advantage on the market (costs, quality). • Innovation as the development of a particular sector / activity : the process is the introduction, development, diffusion in a particular crop / domain /theme; ie, cotton value chain, water resources management, integrated soil fertility, etc… 24
Hambly Odame, H. (2012): Assessing Innovation for Prioritizing Investment. In: Agricultural Innovation Systems: an Investment Sourcebook. Washington, D.C.: World Bank. Hambly proposes that ex-ante assessment can have the following objectives: Analyze the likely impact of a proposed intervention, Identify the optimal portfolio of investments according to specific programs or projects, Establish a baseline or framework to collect and compare information for ex post evaluation. Ex-post assessment’s objective can be to: Assess impact, Support accountability reporting to investors, Justify allocation of resources, Generate lessons, and Identify recommendations for future interventions. 25 Kuhlmann, S. (2003): Evaluation of research and innovation policies: a discussion of trends with examples from Germany; International Journal of Technology Management, Vol. 26, Nos. 2/3/4. Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 17
•
•
Innovation as a process of local socio- economic change : the process is the generation, development, diffusion of innovation in general in a particular community; ie, modernization of agriculture, change in local farming systems, sustainable livelihoods development programmes, etc… Innovation as a development process : the process of change in a region /country/industry over a long period of time; ie, modernization / mechanization of agriculture in Europe, emergence of part-time farming, contract farming, industries, etc…
As the level at which innovation is conceptualized gets higher, the definition becomes increasingly synonymous with “social change” and “development”, and with it the focus of assessment. Actors to be investigated change from individuals to collective actors (groups, organizations). Fields of assessment change from individual decision making to policy making and from technical issues to political and strategic issues.
The problems in assessing innovation processes Assessing innovation is fraught with difficulties. These are summarized by Unger (2005) 26: “the methodological problems are intractable, because there is one major problem: determining whether something is an innovation or not can only be evaluated long after it has happened…One cannot measure innovation as it happens”. Besides, “…many actors suffer from a time bias… they tend to identify many more innovations within the most recent decade than in the two or three preceding decades…they only remember the most successful innovations from long ago, and alternatives that have been overtaken are forgotten”. She suggests, after Debresson, that it is easier to measure “innovative activities”, endeavors and attempts, than “innovations” (p.32). When innovation assessment is conceived of as an evaluation, especially economic evaluation, other difficulties arise (Georghiou, 1997) 27. A Return on Investment (ROI) approach “presumes a linear/sequential model of innovation whereby the benefits of a single research project lead to or are captured by specific innovations… which is rarely the case, particularly for collaborative research”. There are also side effects such as individual and institutional learning, unaccounted for in evaluation, which add to the direct socio-economic effects accruing from innovation. These sideeffects –accruing from partnerships and networking for example- influence norms and standards and contribute to the development of skills. Finally, there is the recurrent “problem of attribution”, ie, the fact that the impact assessed may not be due to the processes of one particular innovation, but to other activities conducted prior to the innovation project.
Innovation processes and project processes There is an abundant literature on the subject of projects in industrial innovation, but little on the interaction between agricultural innovation processes and project processes. In rural development, much has been written about projects, and their impact –or lack thereof- on development. This issue is particularly relevant for the assessment of innovation processes.
26
Unger, B. (2005): Problems of Measuring Innovative Performance; in Casper, S. and van Waarden, F.: Innovation and Institutions, A Multidisciplinary Review of the Study of Innovation Systems; Edward Elgar Publishing Limited. 27 Georghiou, L. (1998): Issues in the evaluation of innovation and technology policy, Evaluation, Vol. 4, pp.37-51; Available at: http://www.oecd.org/science/inno/1822541.pdf. Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 18
•
Development efforts have been increasingly ‘projectified’, ie, conducted in the framework of projects. This framework has some specific characteristics: it is time-bound (the duration of the project is specified); its procedures are formalized in the form of a plan and a management structure; its targets are specified; etc… These characteristics impact on innovation processes by pre-defining the context in which they take place.
•
Many, if not all, rural development projects can be considered innovation projects. They are so either because they aim at transferring innovation packages (in a transfer of technology mode), supporting innovation, or because they can in themselves be considered as a type of innovation.
It is therefore always difficult to assess innovation without taking into account project strategies and their management processes. Fillipov & Mooi (2010) 28 summarize the issue thus: “traditional project management is shaped by the precision, accuracy and optimal use of resources. However, innovation by its definition is a creative process coupled with uncertainty and a need for slack resources”. Other differences between the two are summarized in the Table below: Innovation processes and Project processes TIME PROCESS OBJECTIVES RISK COSTS MANAGEMENT
INNOVATION PROCESSES Long Iterative, improvised, exploratory Undefined or loose objectives High risk, risk management through continuous strategizing Value / costs difficult to estimate ex-ante Creative process coupled with uncertainty and a need for slack resources
PROJECT PROCESSES Short, planned Planned, managed Clearly defined objectives and processes Low risk, risks reduced at the formulation stage Value / costs defined before the start of the project Shaped by the precision, accuracy and optimal use of resources
In this context of “embeddedness” of innovation in project processes, the risk is to confuse innovation assessment with project assessment.
1.1.4 The Importance of an Approach to Assessment Given the complexity of defining and operationalizing it, some researchers doubt whether a “fuzzy” concept like innovation can really be measured (Godin, 2010) 29. Yet there is an abundance of documentation on the subject and countless studies of innovation systems, which suggest that the issue is more one of “measuring” than of assessing and, perhaps, one of the relevance of assessment methodology. The difficulty is again perhaps best summarized by Unger (2005) 30: “the innovation process is often not directly observable; it is a transformation process of an idea into an outcome, a complex organizational link between different departments, firms and the environment. Therefore, 28
-Filippov, S. and Mooi, H.(2010): Innovation Project Management: A Research Agenda; RISUS. Journal of Innovation and Sustainability, Vol 1, No 1; Available at: http://revistas.pucsp.br/risus. 29
Godin, B. (2010): National Innovation System A Note on the Origins of a Concept; Project on the Intellectual History of Innovation. Available at: http://www.csiic.ca/PDF/IntellectualNo4Note.pdf. 30 Unger, B. (2005): Problems of Measuring Innovative Performance; in Casper, S. and van Waarden, F.: Innovation and Institutions, A Multidisciplinary Review of the Study of Innovation Systems; Edward Elgar Publishing Limited. Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 19
the perception of what an innovation is and what one should study, depends on the theory one has (explicitly or mostly implicitly) in mind… and the perception of an innovation is shaped by the research questions formulated by the researcher” (p.33). In this sense, one risk is that what is eventually assessed may have little to do with reality, and “innovations” can be fabricated by the researcher’s own conceptual framework; hence the importance of a careful consideration of the assessment approaches. Such risk is essentially one of “confirmation bias”: we all tend to selectively filter and pay more attention to information that confirms our opinions, and ignore or rationalize away the information that does not.
Some basic research concepts Careful consideration of assessment methodologies can be helped by a review of some key research concepts: Ontology, Epistemology, Methodology, Method, Sources of information. The following definitions can be useful in thinking through the logic of an assessment approach. Ontology
Epistemology
What’s out there to know?
What and how can we know about it?
What is an innovation? What models of innovation exist?
What knowledge is relevant to assess innovation?
Source : Adapted from Hay (2002)
Methodology
How can we go about acquiring knowledge?
What is our general approach to acquire that knowledge? What methods exist and what are their theoretical underpinnings?
Methods
What procedures can we use to acquire it?
What information do we need to know more about innovation? What tools and techniques do we use to get it?
Sources
Which data can we collect?
Who and where do we get information about innovation?
31
Two important concepts must first be understood: • “Methodology” –used here interchangeably with “approach”- is different from “methods”. Methodology is the systematic analysis of methods applied to a domain of study, here innovation, and the concepts and theories behind them. A methodology provides the theoretical framework for the assessment, which ensures that there is continuity and coherence in the assessment. Methods are the set of instruments selected to do the assessment. Methods can be used in the context of different methodologies. • An assessment should not be ‘method-led’ but ‘question-led’. It is the questions that point to what information is needed, that is the most appropriate method. Without a fit between the 31
Hay, C. (2002): Political Analysis. A Critical Introduction, Basingstoke: Palgrave. Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 20
questions (the “what?”) and the methods (the “how?”), there is no coherence in the assessment. Interesting information may be collected, but not necessarily relevant to the objectives of the assessment (the “why?”). In innovation assessment, as in the assessment of any other issue, there is no standard methodology. It is difficult to decide on an approach unless we have clearly defined what is to be assessed exactly and why the assessment is needed. Once this is done, it becomes possible to define a context for the assessment and start thinking about the approach. What is precisely meant by an “approach” to assessment? Approaches are like Russian-dolls concepts. There are approaches within approaches. The two most commonly referred to are quantitative and qualitative approaches to research. Quantitative approaches: an explicit aim of the approach to research is to be as objective as possible. To that end, it usually involves collecting and converting data into numerical form through strict procedures so that statistical calculations can be made and conclusions drawn. A quantitative researcher takes great care to avoid affecting the results by their own presence, attitudes and behavior. They also critically examine their methods to pinpoint and correct any bias. Qualitative approaches: Like quantitative research, qualitative research seeks to answer questions through a rigorous procedure. The difference between the two is the emphasis on how people experience an issue. It is a common type of research in the social sciences and market research, where people’s motivation, attitudes, behavior, emotions and beliefs are important. Qualitative research gives priority to fine textual analyses over quantitative data. When combined with measurement procedures, it can help interpret and better understand the complex reality of a given situation and implications which quantitative data alone cannot interpret. Within these broad research approaches more specific approaches can help in thinking through innovation assessment. Here are some of them: Pragmatic approach: this approach is based on the notion that we do not need to be over-concerned by philosophical debates about methods; we should use the method we feel is best suited to the research problem. Pragmatist researchers recognize that every method has its limitations and that different approaches can be complementary. It is therefore possible to combine methods, techniques and procedures whether quantitative or qualitative. It is essentially a multi-methods approach. Narrative approach: this approach is essentially a qualitative methodology, used in anthropology, organizational studies, sociology, and education science among others. Story telling is typically a narrative approach. The approach puts human beings at the centre of the research procedure as narrators of their experience, produced as a text to be interpreted by researchers. In innovation studies, innovation histories can be considered as a comparative approach. Comparative approach: this approach is used in a wide range of disciplines: biology, linguistics, politics, population studies, anthropology, etc… It is used to identify, analyze and explain similarities and differences across communities, societies, languages, political systems, etc… Comparison can be across systems, regions, nations, through time depending on the object of the study. Basically it is based on the notion that comparison allows a researcher to highlight what is specific to the object of the study, and thus to gain a better understanding of their structure, institution, functioning, etc… In innovation studies, benchmarking can be considered as a comparative approach. Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 21
Emancipatory approach: Under this broad label, there are in fact several approaches: action research, participatory action research, cooperative inquiry, etc… They are all based on a criticism of the political neutrality of ‘conventional’ or ‘traditional’ research. Emancipatory research seeks to empower the subjects being research, as a condition for change. It recognizes power imbalances and aims to empower people through research. One of its main concepts is that people’s consciousness can be raised through the collective production of knowledge, and reflection throughout the research process. Institutional approach: the basic premise of this approach is that institutions shape and constrain human interaction. It argues that social processes (community networks and innovation processes for example) and development outcomes are largely the product of the political, legal and institutional environment. By focusing on this institutional environment, we can therefore explain these outcomes. These various specific approaches can be used singly or in combination, even merged, as for example in comparative institutional analysis. Some methodologies to innovation assessment In a seminal paper on what they call “product development”, Brown and Eisenhardt (1995) 32 review the literature on innovation research and identify three different approaches. The “rational plan” approach, which emphasizes the careful planning and execution of the innovation process. In this approach, a “superior” product aimed at an “attractive” market is successfully developed when it is well planned by a competent cross-functional team supported by senior management. Research in this approach is about discovering which among many independent variables explains the financial success of the innovation. Methodology is based on interviews –with questionnaires or otherwise- of well placed individual informants. The “communication web” approach focuses on a single variable: communication. In this approach, internal and external communication is critically important in the process –as is the role of “gatekeepers”-, the assumption being that “the better members are connected with each other and to key outsiders, the more successful the development process will be” (Brown and Eisenhardt, p.354). Methodology is more broad-based, investigating several aspects of the same variable – including political and information processes- and multiple informants. The “disciplined problem solving” approach, in which the innovation process is the result of a balancing act between autonomous problem solving by innovation teams and the discipline imposed by leaders with an overarching vision of the product they want. Methodology here is based on case studies focusing on management practices, training strategies, team problem solving strategies, communication between team members and information flows within and outside the teams. Innovation as a conscious human endeavor planned and organized (or “orchestrated”) as in the “rational plan” approach; Innovation as the result of collective exchanges, as in the “communication web” approach; Innovation as the result of juggling between individual efforts and demands from leadership as in the “disciplined problem solving” approach. In essence, these general models of innovation assessment point to the main variables to focus on, which can be derived almost 32
Brown, S.L. and Eisenhardt, K.M. (1995): Product development: Past research, present findings, and future directions. The Academy of Management Review Vol 20 N°2, pp 343-378; available online at: https://faculty.fuqua.duke.edu/~moorman/Marketing%20Strategy%20Course%20Materials/Brown%20&%20Eisenhardt%2 0-%20Product%20Development.pdf Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 22
intuitively when we think of investigating innovation. These variables are, so to speak, different plots making up the same story: innovation as something new is the result of efforts made by individuals living and working within a community –professional, social, economic, organizational-. Within this community, individuals interact and exchange information. When this community is an organization, individuals function within a power structure determined by economic interests, political objectives, and rules and regulations. What makes the difference between the approaches is therefore the plot they each choose to emphasize: individual endeavor, interactions and information exchange with others, power structure and conflicting objectives. Before proposing an integrative model, Brown and Eisenhardt point out the shortcomings of the three approaches they outlined. They say that the rational plan tends to include too many variables and to rely on single informants. The communications web theory tends to be focused on a single variable and rely on subjective criteria to measure success. Problem-solving is too focused on complex and vague concepts. This typology of innovation research methodologies is based on a review of product development in the corporate world (automobile and computer industries for example), and may seem far from the world of rural innovation. Yet there are strong parallels in emphases with the experience of innovation studies in rural development, at least for the first two approaches. The “rational plan” approach is similar to the “research” approach criticized by Paul Richards (1985) 33. To Richards, farmers are good innovators, whose knowledge of their environment enables them to invent, experiment and adapt continuously, even in very difficult environments. However their innovativeness is mobilized in pursuit of their own needs and interests in a changing environment. Development organizations fail to recognize their knowledge and support their innovativeness. Richards strongly criticized agricultural research for failing to integrate this innovativeness into its work. He contrasted farmers’ and researchers’ worldviews by highlighting the dynamic nature of farm management and the complex circumstances (environmental, institutional, cultural, and economic) under which agricultural decisions have to be made. Farmers do not simply acquire and mechanically use technologies; they do so under changing conditions. Based on his experience with rice farmers in Sierra Leone, Richards (1989) defined their view of “agriculture as a performance”, whereby they responded to the unfolding risks and uncertainties of a cropping season (early or late rains, poor burn of the bush, accidents, sickness, etc..) by drawing on a wide repertoire of resources and skills (knowledge, social and technical skills, etc…. In short, they improvise and “strategize” as they go along, in what is more than a simple routine technical response to biophysical conditions but a skilled and knowledgeable performance. This performance is determined by a range of factors, including the ability to mobilize, organize and motivate agricultural work groups in peak labour periods. This is in sharp contrast to the researchers’ worldview, in which agriculture follows a “plan” in which a pre-defined strategy is to be executed through the deployment of a set of activities. As Richards argues, attempts to reproduce farmers’ conditions in research experiments miss the point that the most important in farmers’ performance is not the ‘spatial’ arrangement of crops but the continuous adaptation to contingencies, ie, the ‘temporal’ sequence of improvised decisions.
33
Richards, P. (1989): Agriculture as a performance. In R. Chambers, A. Pacey and L. Thrupp (Eds.), Farmer First: Farmer Innovation and Agricultural Research. London: Intermediate Technology, pp. 39-42. Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 23
The “communication web” approach is, in essence, inspired by the diffusion of innovation approach which prevailed in rural extension in the 60s and 70s 34. At its simplest, the diffusion of innovation theory, applied to agriculture, is based on the notion that, over time, innovations diffuse through certain communication channels among the members of a community. This short definition encapsulates the main elements of diffusion research, which can enlighten innovation assessment: - an innovation (an idea perceived as new by the adopter): Here the innovation is almost always assimilated to a technology (new seed, fertilizer, machine, etc…) produced by research. The characteristics of the innovation 35 are important in shaping the potential user’s opinion and influence his decision to adopt or not. - communication channels: these channels are mainly fellow farmers, trusted neighbours who have first experimented with the innovation. These trusted channels provide information that influences the knowledge and assessment of the innovation. Whereas mass media are more effective in creating awareness of an innovation, personal contacts are more effective in forming an opinion about it, especially if the persons who provide the information are similar to the receiver in certain attributes. - time: the time period needed for the individual to go through stages from, first, awareness to persuasion, to decision, to implementation, and finally confirmation of adoption. Each stage is supported by information seeking, to form an opinion. - a social system: the inter-related individuals involved in communication and information sharing in order to solve a common problem or to reach a common goal. Adoption is, first of all, a social process, and is influenced by the system’s social and communication structure. This structure may facilitate or impede the diffusion of innovations, depending on the norms, rules and traditions that regulate the individuals’ behaviour patterns. One important element is the role that opinion leaders play in influencing system members. One important methodological implication when assessing innovations from this perspective is the central place of these elements, especially the roles of diffusion networks and opinion leaders. Many criticisms have been leveled at the diffusion of innovation theory 36 and have led to continuous adaptations. One of the most important is the recognition that technologies are not static. There are adaptations to the innovation all along the adoption curve, which are themselves ‘innovations’ around the ‘generic’ innovation. This latter criticism of the diffusion of innovation theory is the basis for much of what is now recognized: that a fully packaged innovation is ‘dismembered’ by farmers and adapted to their own needs and resources. Farmers’ innovation potential expresses itself when options presented to them offer an opportunity for which thought and innovation on their part are expected.
34
Since the first study investigating the diffusion of Maize in Iowa in 1941, studies of the diffusion of innovations are counted in the thousands. This simple and compelling notion has been theorized in the sociologist Everett Rogers’ seminal work on diffusion of innovations (1962, 1995) which had an enormous influence on extension research and practice. 35 These are: The relative advantage (whether the innovation is perceived as superior to the previous idea; The compatibility (whether the innovation is perceived “consistent with the existing values, past experiences, and needs of potential adopters”; the complexity ( the perceived difficulty to understand and use the innovation); The trialability (“the degree to which an innovation may be experimented with on a limited basis”); and the observability (how the results of an innovation are visible to others). Additionally, modifiability (whether an innovation can further be changed or modified by a user). 36 Rogers himself puts them into 4 categories: Pro-innovation bias: the innovation in itself is not a problem, it should be adopted by all members of the social system; Individual blame bias: tendency to blame individuals for their non-adoption; Recall problem: the overdependence on memory to recall innovation; Issues of equality: socio-economic gaps between the members of a social system. Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 24
The need to be pragmatic: some general principles In a rural development context, being pragmatic means at the same time: First, defining clearly what the assessment is all about, and not adopting an overly holistic definition: • • •
Defining the stage of the innovation process: if one cannot evaluate innovation as it happens, innovation can be studied after it happened. Defining the type of innovation: product, organization, process, etc… The important thing here is to arrive at a consensual definition of the innovation and the stage of its process. Defining and agreeing on an approach, with awareness of its conditions of validity and limits: innovation processes are, almost by definition, learning / learning by doing / learning through iterations processes. These processes are also processes of social, economic and technological change. Assessment, when not assimilated to -and restricted by- research and / or M&E rigid standard procedures- should mesh with these processes, and aim at enhancing them rather than simply reporting on them, to become, in the words of Guijt, "a purposebuilt assessment (Guijt, 2007) 37.
This pursuit of transformation and social change through pragmatism requires communication efforts and participation in a participatory assessment. Assessment and learning require more than a method. To be effective, frameworks, values and skills need to merge with a method to construct an appropriate assessment process, with investments in “soft skills” such as facilitation. When assessment pursues broad objectives of facilitating innovation and not simply documenting it, assessment methodology should reflect these broader objectives and aim at making their achievement possible. The literature on Social Learning (Lotz-Sisitka, 2012 38; Gerger Swartling, Lundholm, Plummer and Armitage, 2010 39 ; Kummer, Aigelsperger, Milestad, Chowdhury and Vogl, 2010) 40, Assessing for Social Change (Guijt, 2007), Learning for Social Change (Taylor, Deak, Pettit and Vogel, 2006) 41, Theory of Change (Keystone, 2009; Organizational Research Services, 2004 42) and Adaptive Management (Fisher, Prabhu and McDougall, 2007 43; Fazey and Fazey, 2005 44; Williams, 2011 45; Allan, 2007 46; Ojha, Hall, and Sulaiman, 2011 47; Stankey, Clark and Bormann, 2005) 48 offers good insights.
37
Guijt, I. (2007): Assessing and Learning for Social Change: A Discussion Paper; Institute of Development Studies; Available at: http://www.ids.ac.uk/files/dmfile/ASClowresfinalversion.pdf. 38 Lotz-Sisitka, H.B. (Ed.) (2012): (Re) Views on Social Learning Literature: A monograph for social learning researchers in natural resources management and environmental education. Grahamstown / Howick: Environmental Learning Research Centre, Rhodes University /EEASA / SADC REEP; Available at : http://arjenwals.files.wordpress.com/2013/01/reviews-onsocial-learning-literature.pdf. 39 Gerger Swartling, A., Lundholm, C., Plummer, R. and Derek Armitage, D. (2010): Social Learning and Sustainability: Exploring Critical Issues in Relation to Environmental Change and Governance Workshop proceedings, Stockholm Resilience Centre, Stockholm, Sweden ; Available at: http://www.sei-international.org/mediamanager/documents/Publications/SEIProjectReport-Swartling-SocialLearningAndSustainability.pdf. 40 Kummer, S., Aigelsperger, L., Milestad, R, Chowdhury, A.H. and Vogl, C.R (2010): Knowledge systems, innovations and social learning in organic farming; 9th European IFSA Symposium, Vienna; Available at: http://ifsa.boku.ac.at/cms/fileadmin/Proceeding2010/2010_WS1.8_Kummer.pdf. 41 Deak, A., Pettit, J., Taylor, P. and Vogel, (2006): Learning for Social Change: exploring concepts, methods and practice. 42
Organizational Research Services (2004): Theory of Change: A Practical Tool For Action, Results and Learning; Annie E. Casey Foundation; available online at: http://www.aecf.org/upload/publicationfiles/cc2977k440.pdf. 43
Fisher, R., Prabhu, P. and McDougall, C. (2007): Adaptive Collaborative Management of Community Forests in Asia Experiences from Nepal, Indonesia and the Philippines; CIFOR; Available at: http://ibcperu.org/doc/isis/8740.pdf
44
Fazey, I., J. A. Fazey, and D. M. A. Fazey (2005): Learning more effectively from experience. Ecology and Society 10(2): 4; Available at: URL:http://www.ecologyandsociety.org/vol10/iss2/art4/ 45
Williams, B.K. (2011): Adaptive management of natural resources - framework and issues; Journal of Environmental Management 92, 1346 -53; Available at: http://www.webpages.uidaho.edu/fish510/PDF/Williams%202011%20adaptive%20mgt.pdf. Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 25
Sample of Methodology; A Collaborative Innovation Assessment in Kenya Prosopis juliflora was introduced into Baringo county for re-vegetation of bare lands in the 1980s as a collaborative venture between the local community, Kenya Forestry Research Institute (KEFRI) and the Food and Agricultural Organization (FAO). Re-vegetation was successfully accomplished the invasive nature of prosopis soon proved to be a challenge which was exacerbated by the El Nino rains of year 2000. The plant invaded grazing lands, choked waterways, blocked paths and even caused the relocation of Endao shopping center. A section of the local community initiated a legal campaign against the government where they claimed compensation for damages incurred and total eradication of the tree species in the region. The management of Prosopis innovation case describes how diverse governmental and nongovernmental organizations and the private sector worked with the local community to curb adverse effects of prosopis in Marigat District of Baringo County. Various technical and socio organizational innovations were introduced to the community who also innovated and led to development of new innovations. A key policy innovation was the revision of the charcoal burning act by the Kenya Forestry Service that legalized burning and sale of Prosopis charcoal. This became a major source of income for some households in Marigat. Pruning of the scrub like young tree led to development of strong and fast growing trees that soon yielded wood for charcoal, posts and timber. The general objective of the Prosopis Collaborative Case Assessment (CCA) was to assess and understand how innovation processes within the case unfolded over time with a focus on multi-stakeholder interactions, innovation processes, the role of local knowledge, drawing of lessons and recommendations for research, policy and practice. Prosopis is a plant More specific objectives of the case assessment were: • To document and assess generic and specific aspects of innovation experiences within the Prosopis case and their recent dynamics, •
To provide answers both to JOLISAA research questions and to locally-identified and negotiated questions / concerns;
•
To illustrate and feed into JOLISAA key lessons, messages and recommendations to policy makers, practitioners and researchers;
•
To contribute to joint learning among the various partners of JOLISAA at local, national, regional and international levels, and to create awareness about innovation systems.
The assessment methodology chosen by the team was: - A Collaborative Case Assessment inception meeting - Community sensitization meetings - Desktop study to review secondary information on the innovation case, with project assessment reports, workshop presentations and correspondence in order to establish the institutional/organizational pathways of Prosopis introduction, population trends, follow up strategies, and challenges faced by relevant implementing agencies. - Development and pre-testing of study instruments, mainly checklists for semi-structured interviews -General and Focus Group discussions conducted in 2 locations with the help of a checklist of 11 questions covering different topics: Farmers’ livelihood, uses of Prosopis, stakeholders in Prosopis management and their roles, management and harvesting of Prosopis, marketing of Prosopis and informal trade, quality issues in Prosopis charcoal production, elementary innovations and source of knowledge - Key informant interviews: in total 14 persons of different backgrounds -crop and livestock producers, beekeepers, self help groups, local leaders, charcoal burners and transporters, staff from government organizations such as the Kenya Forestry Service and Ministries. - Data processing/draft report writing 46
Allan, C. (2007): Adaptive Management of Natural Resources, in Wilson, A.L., Dehaan, R.L., Watts, R.J., Page, K.J., Bowmer, K.H., & Curtis, A. (Eds). Proceedings of the 5th Australian Stream Management Conference. Australian rivers: making a difference. Charles Sturt University, Thurgoona, New South Wales ; Available at : http://www.csu.edu.au/research/ilws/news/events/5asm/docs/proceedings/Allan_Catherine_1.pdf. 47 Ojha, H.R, Hall, A. and Sulaiman, R. (2011): Can Learning Based Approaches Take Root in Natural Resource Management Reflections from the world of practice; Research into Use; Available at: http://www.researchintouse.com/resources/learning21final.pdf 48 Stankey, G.H.; Clark, R.N.; Bormann, B.T. (2005): Adaptive management of natural resources: theory, concepts, and management institutions. Gen. Tech. Rep. PNW-GTR-654. Portland, OR: U.S. Department of Agriculture, Forest Service, Pacific Northwest Research Station. 73 p.; Available at: http://www.fs.fed.us/pnw/pubs/pnw_gtr654.pdf. Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 26
- Feedback workshop to validate the findings of the assessment, gather information to fill existing gaps and determine the way forward for Prosopis management and marketing - Workshop report writing - Second data collection to fill gaps - Data processing/Final report writing Source: Prosopis CCA report; JOLISAA Project, 2013
1.2 Approaches to the Assessment of Innovation
The following are four general approaches to the study of innovation that can inspire teams engaging in innovation assessment. Social Network Analysis (SNA) and Institutional Analysis (IA) can be considered “generic” approaches in the sense that they represent each a vision of what determines innovation. These are not the only approaches to innovation assessment, but they are representative of the assumptions that underlie most types of assessments. SNA is a “communication-web” type of approach, based on the premise that innovation is the outcome of social interactions. IA is based on the assumption that innovations are determined by the institutional context –social, cultural, economic, political. Innovation History is more of a narrative approach which gives precedence to innovation actors’ own perceptions of what the innovation process was like and what influenced it. Benchmarking is a comparative approach which does not make assumptions about what drives innovation but attempts to better understand it through comparisons between different experiences. All four approaches have their strengths and weaknesses. They all can combine qualitative and quantitative methods, to mix and match tools –tools are suggested for each phase- to adapt to the situation. Done in a participatory fashion, all four approaches can be seen as opportunities where innovation actors meet to learn jointly and reflect on their experiences, in a process where assessment focuses on strengthening innovation capacity.
1.2.1 Innovation Histories What is it? An Innovation History is a ‘monograph’ of a particular innovation. It gives a historical account of the birth, development, diffusion (and possibly death) of an innovation. Innovation histories are probably the simplest and oldest forms of reporting on an innovation. They are also –as the name indicatesthe approach most specific to innovation. There is however ambiguity about the term “innovation history”. It can be used in a very general sense with no claim to a “method” (there is an abundance of “innovation histories” and “innovation stories” on the web, ranging from histories of education experiments to students’ innovative projects). There are much fewer references to “innovation history” as a structured methodology of investigating innovation processes. Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 27
What is its relevance for innovation assessment? An innovation history, in the strict sense, is not necessarily a thorough ‘assessment’ of an innovation. But as the telling of the “history” can take diverse forms and range from a simple short narrative description to a detailed study of the development of an innovation, history and assessment cannot be easily separated out. One major difference is the extent to which the building of an innovation history is seen as a research effort, led by researchers working in innovation research, or as a collective learning opportunity, and therefore closely involving innovation actors. Whatever the case, innovation histories are a very good way of documenting an innovation process. The history thus compiled can stand on its own, or serve as the backbone for more detailed studies (case studies, social network analysis, and institutional analysis). It is also flexible and allow, around the chronology of an innovation, to add other layers of information to make it more comprehensive. How to do it? There is no standard methodology to record an innovation history, although the one proposed by Douthwaite (2006) seems to be most popular in rural development. Methodologies range from detailed case studies using a variety of tools from social science research, including secondary data analysis (Seyfang and Longhurst, 2012; Von Hippel, 1988) to multi-stakeholder workshops using facilitation techniques and PRA-type visualisation tools (Douthwaite) 49. The one described here draws heavily on Douthwaite, because it is more adapted to assessments conducted by local teams of professionals, in a joint multi-stakeholder approach. The methodology outlined here is based on the view of innovation history building as a process meant to stimulate reflection and joint learning. Reports of innovation histories, as the name suggests, follow generally a chronological order in the recording of an innovation. But the order in which the stages of the innovation assessment process are used does not necessarily follow that chronology. The assessment can follow any order (for example, the end or the middle, first); the chronological development of the innovation is then reconstructed later for the report. Phase 1: Preparation The process of development of an innovation generally leaves records in the form of reports, publications, meetings, newspaper articles, etc… It makes sense therefore to begin with studying these as much as possible. The following questions can guide the preparation process: • What are our objectives? Why exactly are we producing this innovation history? How are we going to use it once recorded? • What is it exactly that we will be recording? What is the innovation? 50 • What is the already recorded history of the innovation? What do we already know about it? • What tools are we going to use? Do we know how to use them? • Who are the stakeholders to involve in the building of the innovation history? Phase 2: Information gathering The overall process of documenting an innovation history can be visualized like in the figure below.
49
CIAT who pioneered the development of innovation history methodology, lists the following objectives to guide the recording process: What happened and when did it happen? Why? Who made it happen? How? Where? What were the results? What were the motives? http://webapp.ciat.cgiar.org/ipra/ing/news_innovation_histories.htm 50 This question is essential. Innovation histories seem more adapted to single, clearly delineated, innovations. Complex innovations (innovation bundles) will be more difficult to record. Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 28
The questioning leading the process follows the classic “What, Who, When, Why, How” model. Other questions can be added depending on the objectives, for example “with whom?”, “conflicts arising?” etc… The information is generated through a series of workshops and/or individual interviews with actors. Given the differences between contexts (type of innovation, type of actors, type of professional situation, type of objectives, resources available) the methodology must be approached in a flexible manner. Two options can be considered:
•
Start with individual interviews to generate the information, and then reconstruct the innovation history preferably in a visual form, to be validated / amended during a multistakeholder workshop. In this option, differences between individual accounts (on dates, actors, roles, relations, etc…) can be listed and used as topics for discussion during the workshop. The option is very useful in highlighting differences in individual interpretations, which can help mitigate the “group think” effect and subtle power relations at play in multistakeholder workshops, which lead to some stakeholders imposing their own views.
•
Generate the information entirely during one or several multi-stakeholder workshops. In this option, several approaches can be considered, depending on context: → Ask stakeholder to come to the workshop prepared, by asking them to write down a timeline in advance; in this case, the workshop can be combined with a writeshop during which the stakeholders write collectively the innovation history. → Organize a workshop for each information topic: innovation timelines, innovation network, assessment of innovation partnerships, etc… → Organize one single extended workshop, working with different groups on different themes → Organize joint workshops with stakeholders and policy makers to combine assessment and advocacy
Usable Tools: Innovation Timelines; Actor Network Matrix; Interorganisational Linkages Matrix.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 29
Phase 3: Writing up the learning history Writing up the innovation history is the production of the record. It can be done in two ways. • As a report written by one or several persons, circulated, commented upon and amended by the stakeholders; • As a collective report written during a writeshop. There is no standard structure for the report, but the narrative can be structured on the chronology of the process, answering at each stage the questions asked during the process. Phase 4: Using the innovation history The use of an innovation history depends on the objectives set at the beginning. For example, it can be: • A standalone assessment of an innovation; • Used as the backbone of a case study; • Used as a means for advocacy and consciousness raising; • As a benchmark study of an innovation, for future reference. Potential and limits Innovation histories are well adapted assessment methods. • One risk associated with building an innovation history, highlighted by von Hippel, is that relying on a single source may introduce a bias. As he says, “success has many fathers”, and that source may over-emphasize their own role, and therefore minimize that of others. Similarly, the source used as an “entry point” may influence the choice of other sources of information, by favoring and/or over-emphasizing the role of some of them. It is therefore important to look for individual or organizational actors who played a role in the innovation process without privileging any of them. • Another issue is how far back the assessment of the “history” needs to go. If innovation “success has many fathers”, its history has also many “fathers” -diverse actors in the past who contributed to the process-, of whom present day stakeholders may not be aware. In an action-research mode, this may not be important, because the recording of “past” history is but a planned opportunity to chart paths for “future” action –the aim is not to be accurate or comprehensive. But in a more conventional “knowledge production” mode, the history may be arbitrarily formatted by the dates remembered by actors. This is all the more likely as actors have a tendency to better recall recent events, and forget old ones. Links to case studies and other useful materials - The Grassroot Innovations website presents a series of innovation histories around research projects on grassroots innovations, based at the University of East Anglia and University of Sussex. http://grassrootsinnovations.org/ http://grassrootsinnovations.org/category/publications/innovation-history/ - CIAT: http://webapp.ciat.cgiar.org/ipra/ing/news_innovation_histories.htm - von Hippel, E. (1988): The Sources of Innovation; Oxford University Press. Available at: http://web.mit.edu/evhippel/www/sources.htm - Boru Douthwaite, Alok Sikka, Rasheed Sulaiman, John Best and John Gaunt (2006): Learning with Innovation Histories; LEISA Magazine • 22.1 • March 2006 Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 30
http://www.agriculturesnetwork.org/magazines/global/documentation-for-change/learning-withinnovation-histories/at_download/article_pdf - Douthwaite B. and Ashby J. (2005): Innovation histories: A method for learning from experience; ILAC Brief 5; http://ageconsearch.umn.edu/bitstream/52515/2/ILAC_Brief05_Histories.pdf - Seyfang G. and Longhurst N. (2012): Grassroots Innovation for Sustainability: A Niche Analysis Of Community Currencies; University of East Anglia; 3S Working Paper 2012-10 http://grassrootsinnovations.files.wordpress.com/2012/10/3s-wp-2012-10-seyfang-longhurst.pdf .
1.2.2 Social Network Analysis What is it? Social Network Analysis (SNA) is used in different domains of research (sociology, anthropology, communication studies, mathematics, etc…). It is based on the assumption that social relations contribute to shaping the behavior of individuals. The study of relationships between individuals is therefore important when we analyse their behavior in domains such as decision making in organizations, health behaviour, lobbying and influence, adoption of innovations, etc… Through the mapping and measuring of social interactions, SNA helps us to visualize patterns of behavior that are more difficult to detect through the study of individuals in isolation. In short, SNA rests on the following principles: A focus on relationships between actors rather than on their individual attributes A view of actors as interdependent rather than independent The view that social structure affects outcomes The view that there are emergent effects that can be revealed through the study of a network What is its relevance for innovation assessment? SNA is especially relevant for the study of innovation processes. Innovation is rarely achieved by one single individual. It is result of interaction / cooperation between various actors at different stages of the process, with different resources and contributions. The representation of these interactions is therefore very useful to understand the composition of the networks involved and the unfolding of the innovation process. How to do it? There are two different types of network analysis, depending on the perspective taken for the analysis. This analysis can be centered on one individual (ie, “ego network analysis”) and conducted in the context of a conventional survey, for example by including questions on the relationships each respondent has with other actors on specific issues. The result gives an idea of “personal” networks. On the other hand, the analysis can map out all kinds of relationships between a set of respondents – for example all farmers in a specific area (ie, “complete” or “comprehensive network analysis”). This latter type is what is usually referred to as social network analysis. Although the general principles are the same, SNA sophistication can also vary according to the domain of the study and its objectives. It can range from highly stylized and quantitative representation with the use of specialized software, to more descriptive and qualitative Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 31
representation and analysis. At its simplest, it can be reduced to a “stakeholder mapping” using simple visualization tools, such as those in Participatory Rural Appraisal (PRA). Since networks consist of people and their interactions, SNA represents these as “nodes” and “links”. The nodes are the interacting units (individuals, organizations, groups); the links are the relationships and flows of information, knowledge, money, etc… between them. The visualization of all nodes and their links enables us to assess the location of the node in the network (its “centrality”), its number of direct connections and where those connections lead to. It shows also smaller networks within the larger network (“clusters”, “cliques”). It can therefore reveal leadership patterns through the centrality of some nodes, for example, their influence, brokerage role, etc… There is no standard procedure for the conduct of SNA. The underlying patterns of networks can be identified through the use of various techniques, selected on the type of information needed, itself depending on the research questions. The following generic steps can be used. Preparation a. Clarification of topic, objectives and scope of analysis This is the starting point that determines decisions made during the following steps. At this stage, questions such as the following should be answered: What is the focus of the analysis (network of what exactly)? What is the purpose of the SNA? How is it going to be used? How are the beneficiaries and other stakeholders likely to use and benefit from the results? What specific outputs are expected? What activities are necessary to achieve these outputs? What competencies are necessary to conduct, analyse and interpret the SNA? Who will do what? Who is responsible for what? In this step, the SNA team looks at policy issues, markets, institutional issues and other macrodevelopments in and outside agriculture that may have an influence on the innovation process. The team also identifies important stakeholders. The team learns to see the innovation process as the result of the functioning of a broadly defined innovation system. Ideally, this first step should result in the following outputs: A (re)definition or further elaboration of the “problematique” (the problem situation that is being investigated) A preliminary demarcation of the network to be analysed A preliminary formulation of hypotheses and questions to be answered Usable Tools: Rich Picture; Concept Mapping; Mind Mapping b. Development of a methodology The methodology used must enable the SNA team to reach the objectives defined in the previous step. Questionnaire surveys are often used because they allow a degree of standardization of information that makes network visualization easier. But any other tools and techniques suitable to identify social relationships with regard to the defined scope can also be used: interviews with resource persons, focus groups, observation, secondary data, etc… c. Identification of network members The identification of network members is a crucial stage because it determines the basis for the analysis. It depends entirely on the clarification of topic, objectives and scope of analysis (first step). People can be –and often are- members of several networks, SNA can therefore only make sense if its topic and purpose are clear enough to identify relevant members and exclude the others. Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 32
It is important, in the study of agricultural innovation, to make an analytical distinction between “stakeholders” and “network members”. Stakeholders are all those people who have a stake (or share) in a particular issue or system. Stakeholders can be groups of people, organisations, institutions and sometimes even individuals, located at any level or position in society, from the international to the national, regional, household or intra-household level. Thus, stakeholders include all those who affect and are affected by policies, decisions or actions within a particular system. They may or may not be members of a network. In the context of rural development, if SNA has an objective of empowerment, a participatory approach to network members’ identification is more appropriate. The identification process is then used for facilitating dialogue and negotiating among the actors of innovation. At this stage, questions such as the following should be answered: • Can all network members be considered as “stakeholders” in the innovation process? • What is their relation to the issue being investigated? • What important roles they play in innovation processes? Ideally, this second step should result in the following outputs: • A preliminary list of network members, with a short description and background information • In the case of organizations, a definition of how and from whom exactly data on relationships will be gathered Usable Tools: Stakeholder Identification Matrix; Interviews with Resource Persons Data collection and analysis a. Data collection The data to be collected will depend on the objectives, questions and scope of the SNA. Consequently the tools to be used will also be determined by the type of data (quantitative or qualitative methods). Survey data on social relations that can be collected and represented in SNA are often of the following types: • Kinship: brother of, father of, cousin of, in-law of… • Social Roles: boss of, friend of, etc… • Affective and friendship: likes/ respects/ hates, etc… • Cognitive: knows this, views as similar, etc… • Actions: talks often to, goes to markets with, goes to church with… • Flows of material resources: number of cows moving from to, fertilizer and manure, exchanged business transactions, lending, etc… • Flows of non-material resources: information, advice, etc. • Distance: number of miles between… • Similarities and associations: produces the same crop and/or uses the same techniques as, is in the same cooperative / group as, has the same trade relationship with, sells on the same market as… b. Data analysis
Figure 2: Advice seeking network of teachers and staff (Penelope Hawe and Laura Ghali, 2008)
Analysis of the data is typically done on the basis of a visual map representing the information about relationships between network members. This map can be generated with the use of specialized software, in which case the information has first to be coded and entered. It can also be produced manually with a drawing. The types of relationships are differentiated with the use of colour codes, icons, thick / broken / continuous lines, etc…
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 33
Manual maps are very common in Participatory Rural Appraisal. Several tools can be used for SNA (Venn Diagramme, Stakeholder Maps, etc…). But these are usually adequate for simple (often unique and one-dimensional relationships), with a limited number of actors. The map will quickly become cumbersome and will probably need to be adapted to the size and complexity of the network being mapped. When done manually, the map can be drawn by the SNA team, then presented and discussed with Figure 3: Map from Group Mapping Exercise (Marites actors. It can also be drawn with the actors Tiongco) themselves, with the SNA team acting as facilitators. In this case, they can better understand how well they are organised to deal with innovation processes, or not. It also allows them to develop commitment for collective action in the future. Usable Tools: Specialized software; Various PRA Mapping Tools Data interpretation and action planning a. Interpretation of results of analysis Once the network map is finished, it is reviewed in light of the research questions and hypotheses formulated during the first step. These questions can be very different from one SNA to another. Conventionally, SNA can help answer questions on innovation processes such as: • How dense is the actor network? How far does it reach? • Which actors have a central position? Who knows what? Who dominates the exchanges? What is their occupation profile? Who are the leaders? Who are the knowledge brokers? • Which actors are isolated from exchanges? Who are the peripheral players in innovation? • Where are the information breakdowns and bottlenecks? • Are there cliques / sub-networks separated from the others? • Etc… In a participatory SNA, reviewing the map and interpreting it collectively with the actors themselves is a powerful way to create awareness about the problems and opportunities shown by the map. b. Design and planning of interventions The next step depends on the objectives assigned to the SNA. Ideally SNA should not be simply a way of accumulating knowledge, but also a tool to plan action. Two areas of action can be considered: • To improve innovation and learning by actors by improving the effectiveness of formal and informal communication channels, developing personal competencies and expertise, integrating hidden expertise, integrating marginally connected members • To develop strategies to leverage support for local innovation. To help improve the efficiency of innovation networks, Stopper (2004) proposes 5 important behavioural changes that can be used as guidelines in the design and planning of interventions. These behavioural changes, suggested for business enterprises, are slightly adapted here for agricultural innovation networks:
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 34
Sharing, ie …”establishing a flow of information within the network. For the innovation network to be most effective, information should flow freely... (actors) should share their vision for future markets with suppliers, and ask how suppliers can help fulfill this vision by becoming a part of it. (They) can ask suppliers for their view of future markets and how the supplier sees its position. .. Trust is critical for promoting sharing ». Influencing. Once sharing has started, (actors) can influence each other’s technology and product plans. Influencing the plans of suppliers can bring tremendous value … by focusing the supplier’s resources on those projects of most value to the (farmers). The supplier’s influence on the (farm) is no less important. Influence is impossible without sharing. Aligning. « Aligning occurs when influence is strong enough to justify a major redirection in supplier or (farmers) plans. This often requires sharing of marketing and business development information for the other party to understand and accept the business case or vision to which it is aligning. Often it involves a redirection in vision, business objectives, product plan, technology plan and investments. It is at this point that economies of scope and synergies begin to appear ». Collaborating. « Once aligning is present, the network has a set of common objectives and values, and collaboration can begin. Network collaboration occurs when the network operates with a common set of objectives. Network members begin operating as a team. This brings a level of commitment and ownership from all parties. The most advanced forms of collaboration are present when strategic planning and business development begin with network participation, initiated by any network member. … Achieving this behavior within the network is a (great) challenge”. Performing. « Once collaboration is present, performing can begin. Performing occurs when the network becomes effective in producing innovation as a product of an aligned and collaborative effort. A performing network creates and captures economic value that otherwise cannot be created and captured by members alone ». Usable Tools: Scenario Building, Visioning, Action Plans Potential and limits Despite its proven usefulness in the study of innovation processes, SNA has been the object of several criticisms. The main one is about network boundaries. A network necessarily consists of a finite set of actors, which represent the social, physical, and/or thematic boundaries of the network being analysed. These boundaries can result from the choice of the SNA team (their interests, topic of research, etc…) or to methodological problems (how to sample a group representative of the larger population). Although these boundaries can be changed in an iterative way as the SNA process goes along, they remain to some extent arbitrary because there are always people / organizations that remain outside the network (sometime far removed) and who have an impact of some network members. Links to case studies and other useful material - Fabrice Coulon (2005): The use of Social Network Analysis in Innovation Research - A Literature Review; http://www.druid.dk/conferences/winter2005/papers/dw2005-305.pdf - Monique Mikhail, Amanda Fencl, Sopon Naruchaikusol, Eric Kemp‐Benedict (2010): Innovation and diffusion of sustainable agricultural water resource management in a changing climate: A Case Study in Northeast Thailand; Stockholm Environment Institute, Project Report; http://sei-us.org/Publications_PDF/SEI-2010-innovation-diffusion-agriculture-Thailand.pdf - Ira Matuschke (2008): Evaluating the Impact of Social Networks in Rural Innovation Systems - An Overview ; IFPRI Discussion Paper 00816; http://www.ifpri.org/sites/default/files/publications/ifpridp00816.pdf - Boru Douthwaite, Andrea Carvajal, Sophie Alvarez, Elías Claros and L.A. Hernández (2006): Building farmers’ capacities for networking (Part I) Strengthening rural groups in Colombia through network Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 35
Analysis ; KM4D Journal 2(2): 4-18; http://journal.km4dev.org/index.php/km4dj/article/viewFile/63/116 - Uwe Cantner & Holger Graf (2004): The Network of Innovators in Jena: An Application of Social Network Analysis; http://www.wiwi.uni-jena.de/Papers/wp-sw0404.pdf - Tom Bartholomay, Scott Chazdon, Mary S. Marczak & Kathrin C. Walker (2011): Mapping Extension's Networks: Using Social Network Analysis to Explore Extension's Outreach; The Journal of extension, Volume 49 Number 6; http://www.joe.org/joe/2011december/pdf/JOE_v49_6a9.pdf - Rick Davies (2009): The use of social network analysis tools in the evaluation of social change communications; http://www.communicationforsocialchange.org/pdfs/2nd%20draft%20the%20use%20of%20social% 20network%20analysis%20tools%20in%20the%20evaluation%20of%20social%20change%20commun ications%20b.pdf - Louise Clark (2006): Network Mapping as a Diagnostic Tool Manual; Centro Internacional de Agricultura Tropical – CIAT; http://www.dfid.gov.uk/r4d/PDF/Outputs/FIT/networkmapping_LC06.pdf - Robert A. Hanneman and Mark Riddle (2005): Introduction to Social Network Methods. This online book introduces the basics of formal approaches to the analysis of social networks. http://faculty.ucr.edu/~hanneman/nettext/ - Tobias Müller-Prothmann (nd): Social Network Analysis: A Practical Method to Improve Knowledge Sharing. This is a quick and well presented general introduction. http://www.knowledgeexperts.com/files/kb_ebook02_social_network_analysis_knowledge_sharing.pdf - Marites Tiongco (nd): Net-Map analysis of Value Network for Maize and Aflatoxin Information Flow in Kenya; http://programs.ifpri.org/afla/pdf/Kenya_netmap.pdf - Geoff Kaine, Brendan Doyle, Ian Reeve and Jim Lees (1999): Agricultural Knowledge and Information Systems: A Network Analysis; Paper presented to the 43rd Annual Conference of the Australian Agricultural and Resource Economics Society, Christchurch, New Zealand; Available at: http://ageconsearch.umn.edu/bitstream/123823/2/Kaine.pdf.
1.2.3 Institutional Analysis What is it? The institutional analysis approach to the assessment of innovation is based on the assumption that institutions are the key to explain economic performance and a society’s innovativeness. Institutions are the framework created by people to exchange, do business and innovate. Therefore, they determine the feasibility of technological improvement and engagement in economic activities. Similarly, institutional change can play a role in mobilizing human creativity and energy and harness them to shared purposes (Ferejohn, 2003). There are different ways to approach institutions, depending on the domain of study. They can be considered as all relatively stable social units / structures (ie, government agencies, non-government organizations, business enterprises); as all the laws, rules, conventions, and regulations that guide individual and collective action in a particular area (ie, land tenure); as the system of relations, customs that govern social life (ie, marriage, family, religion). In local innovation processes, there are elements of all these, both formal (organizations) and informal (social norms and relations). Sometimes, the literature makes an analytical distinction being organizations and institutions, the
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 36
former being the structure, the second being the norms and rules that operate the structure 51. Innovation assessment from an institutional perspective will therefore touch on each. What is its relevance for innovation assessment? Innovation, like all human endeavours, is influenced by institutions; it is therefore amenable to an institutional analysis. Some of the central hypotheses of the institutional analysis of innovation are: • No innovator is an island, innovation actors influence each other; • Different institutions have different ‘cultures’ that may or may not coincide; • Different actors within different institutions compete for power and resources; • Decisions made in central hierarchies are modified at the local level. These hypotheses suggest that conflicts may emerge out of the interplay between institutional strategies and impinge on innovation processes. This strongly relates to the concept of ‘innovation system’, ie, the idea that: • Institutions jointly and individually contribute to innovation, through the development and diffusion of technologies, the creation, storage and transfer of knowledge and skills necessary for innovation • Institutions are the framework within which governments develop and implement policies that influence the innovation process. The institutional approach is therefore important in highlighting this interplay between institutions, be they local, regional or national, formal or informal, public, private or non-governmental. How to do it? The concept of institution is not always easy to grasp. Added to the complexity of institutional roles, relations and interactions and the sensitivity of issues, institutional analysis can therefore very long and complex. Its application to the study of local innovation processes should not aim at a full blown analysis, but at a qualitative assessment. There is no standard methodology for the conduct of an institutional assessment of innovation. At its most complex, it can be thorough political, sociological or anthropological studies of institutions around a particular theme (technology, land tenure, agricultural credit, policy making, etc…). At its simplest, institutional analysis can be conceived of as a qualitative extended “stakeholder analysis” around a well defined innovation. To keep the institutional analysis of local innovation processes realistic, the following objectives can be considered: • Identify and analyse institutions that have an impact on innovation processes • Analyse their interactions and inter-dependencies in support of innovation • Establish the alignment of their strategies with those of different innovation actors The following are intuitive steps that can be adapted to local contexts.
51
A useful typology of « institutions » in this regard is the one proposed by Scott (2001): Regulative institutions are the laws, formal rules, decrees, codes of conduct; they determine what is allowed and what is not allowed and formally sanction divergent behaviour. Normative institutions are the societal norms, values, standards of good practice, work norms, conventions. These are not backed by formal sanction, but are socially expected behaviours. Normative institutions set out what is right and what is wrong. Cultural-cognitive institutions include cognitive frames, mental paradigms, visions, expectations, shared role models, perceptions. They shape what is thinkable and what is unthinkable. Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 37
Phase 1: Preparation The preparation of the institutional analysis should answer the following questions: • What is being researched? What should we focus on? • What are the institutions to be analysed? Which ones are the most important? With whom are we going to analyse them? These two questions are important because their precision determines whether or not an institutional analysis can be conducted in a reasonable length of time. The broader the question and the scope of investigation, the more complex is the institutional system to be analysed. The group conducting the analysis is essential; institutional analysis aims at bringing about institutional change –in itself an innovation- in support of local innovation processes. It is therefore essential that the analysis involves as much as possible those institutions. • How are we going to collect the information? What tools are we going to use? How do we plan to analyse the information once collected? The latter point is of paramount importance. Like in all assessment processes, the choice of methods –and of the type of information collected- must balance out constraints such as the quality of information needed (detail, precision, quantification) and the costs of the assessment (time, finances, need for external expertise, etc…). Phase 2: Identification of institutions This phase is, broadly speaking, about mapping the institutional landscape which forms the innovation background. The objective is to understand the institutional structure underlying innovation processes, the context in which decisions are made that affect innovation, especially those relating to policy and resource allocation. This can also be considered as a mapping of the innovation system, that is “ the study of sets of interrelated actors who interact in the generation, exchange, and use of agriculture-related knowledge in processes of social or economic relevance, and the institutional context that conditions their actions and interactions” (Spielman and Birner 2008). Adopting this systemic perspective means considering farmers as active participants in innovation processes, hence analyzing their own institutions, their rules and norms. This means also identifying the social units, within the farming community, at the level of which decisions are made that shape innovation processes. The procedure can start with a first selection of those institutions that have reputedly the highest impact on local innovation processes, identified with the help of key informants can be used (researchers, extension agents, farmers leaders) and complete the list gradually and iteratively during the information gathering process as new important institutions come to light. Usable Tools: Stakeholder Identification Matrix; Innovation System Mapping; Diagramming; Key Informants Interviews; Institutional Histories Phase 3: Analysis of institutional rules, relations and strategies The objective of this phase is to describe the institutional context as accurately as possible, its structure, actors, their roles, strategies, interests and rules. One way to synthesize the institutional analysis, and to compare institutions, is by describing their features in the following matrix.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 38
Institutional Analysis Matrix Farmers
Farmers’ Organisations
Research organisations
AgriBusiness
NGOs
Etc…
Strategies Laws, rules and regulations Values, norms and culture Interactions and networks Capabilities Market orientation Infrastructure Adapted from Frans Hermans, Laurens Klerkx & Dirk Roep
• • • • •
• •
Strategies: This relates to the institutions’ plans, programmes and decision making processes, their alignment with each other, and their support, or lack of, to innovation. Laws, rules and regulations: This concerns formal regulations likely to support or hamper innovation, such as environmental laws, national market protection, subsidies, etc… Values, Norms and Culture: This refers to what determines the way organizations and individuals go about their business: relation of trust, worldviews, cooperation, taboos, etc… Interactions and Networks: This describes the way actors are connected to each other, or the characteristics of the social networks they are part of, and how these influence innovation. Capabilities: This describes the technical and organisational capacity of the actors in the local innovation system to produce, adapt to and manage new technology and organisational innovations: their networking skills, their entrepreneurship, their technical and organizational knowledge, their leadership capacities, etc. Market orientation: This refers to the position of institutions in relation to markets: knowledge about the functioning of markets, market information (prices), market demand, market actors, etc… Infrastructure: This concerns both the physical infrastructure (roads, telecommunications, etc…) and financial and knowledge infrastructure (public and private investment, access to credit, education, etc…). Do these institutions benefit from such infrastructure?
Another way to analyse institutional impact is to describe the role of institutions in the functions of the innovation system. The functions are derived from recent work on technological innovation systems (Hekkert & Negro, 2010; Rohracher, Truffer & Markard, 2008). These functions represent all activities that contribute to the development, diffusion, and use of innovations. The analysis of these functions consists of determining whether they were fulfilled, who contributed to fulfilling them, and what were the driving forces behind them (‘drivers’) and the forces impeding them (‘barriers’). Role of institutions in the functions of the innovation system Drivers
Barriers
Roles of organisations
Functions Entrepreneurship Knowledge Development Knowledge Diffusion and Exchange Guidance of the Search Market Formation Resource Mobilisation Advocacy Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 39
Entrepreneurship: Activities promoting changes that facilitate the innovation process and demonstrating its usefulness, by mobilizing institutions or in market initiatives Knowledge Development: Activities promoting collective learning, on technologies, processes, markets, networks, users, clients, etc… Knowledge Diffusion & Exchange: Activities promoting partnerships and interactive learning between actors (for example technology developers), but also meetings, workshops and conferences. Guidance of the Search: Activities that shape the needs, requirements and expectations of actors with respect to their (further) support of the emerging technology Market Formation: Activities that contribute to the creation of a demand for the emerging technology (financial support for its use, barriers to the use of competing technologies, etc..) Resource Mobilisation: Activities that provide resources helping the innovation process, such as allocation of financial (investment, subsidies), material (infrastructures) and human capital (education) Advocacy : Activities aimed at creating legitimacy for the innovation and/or counteracting resistance to change. Usable Tools: Decision Trees; Social Maps; Venn Diagramme; Network Maps; Organisational Linkage Matrix; Stakeholder Influence Matrix; Phase 4: Assessment of institutional efficiency in supporting innovation Assessing institutional efficiency is a pre-requisite for action to help institutional change. This assessment is particularly difficult because the notion itself is not clear to all 52. Often the rules underlying the functioning of institutions are so deep-seated that actors themselves do not question their relevance and cannot conceive of the necessity of institutional change. An outsider’s view is therefore essential. Inversely, the subtleties of local knowledge and people’s sensitivities, habits, routines, norms, values, will be difficult for an outsider to grasp in a short period of time. An insider’s view is therefore required. Thus an assessment needs both views to move from analysis to realistic proposals for action. The key question to be answered at this stage is how the different institutions assessed (formal and informal) affect the innovation process under study, in the context of different interests of different people. The dimensions/criteria along which institutional support to local innovation processes can be assessed are very diverse. Some of them are: • Activities supportive of local innovation • Formal institutional goals, mission, and other strategic objectives • Financial support • Technical support • Business efficiency in the case of commercial organisations • Staff competencies • etc… The best way in a qualitative and participatory institutional assessment is to jointly list and review al criteria before selecting a few that are acceptable to all. Efficiency can be described in a narrative form or as a ranking / scoring format, or both.
52
The concept of efficiency itself is complex. In the field of project evaluation, there is formally a difference between ‘effectiveness’ -the congruence between outputs and goals or other criteria-; and ‘efficiency’, which links outputs with inputs. The concept here is taken rather loosely as meaning the ability to provide support to innovation processes. Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 40
Usable Tools: Brainstorming; Ranking and Scoring Matrices; Competing Values Framework; Innovation Capabilities Assessment Phase 5: Development of a plan for institutional change in support of innovation The final stage of the analysis is to propose a strategy likely to achieve long-term changes. These changes can be new arrangements, policies and legal frameworks that could improve the performance of institutions in their support to local innovation systems. Arrangements can concern individual and organizational linkages, resource mobilization and use, planning, capacity building, etc… The strategy is necessarily beyond the reach of individual institutions and projects and must therefore be acceptable to all. Usable Tools: Visioning; Scenario Planning; Potential and limits Institutional analysis has many critics, pointing out two points, which are relevant for the study of innovations. As Hollingsworth (2000) observes, almost “every social science discipline …has at least one distinctive strategy for doing institutional analysis. And it is because of the lack of consensus as to the appropriate boundaries and content of institutional analysis that we have limited ability to make theoretical advances in understanding how the institutional makeup of a society impacts on its innovativeness”. Even in innovation system studies, there are different interpretations, notably because there is no consensus on what the components of this system are. Despite efforts made to arrive at a common understanding, this conceptual vagueness is unlikely to end soon. Institutional analysis often considers institutions as static entities that need planned external intervention to change and adapt. But as has sometimes been observed “only dead institutions do not change”. Institutional stability is never absolute. Not only do institutions change, they are also dynamic, and are the result of the continuous interplay between existing rules and regulations and actors’ interventions to change them. As actors become successful in innovation, they engage in individual and collective action to change institutions to improve their innovativeness and competitiveness. Institutional change is therefore the outcome of multiple factors: the institutions’ own capabilities, the global environment, actors ‘interventions, all feeding into one another. Links to case studies and other useful materials - Harald Rohracher, Bernhard Truffer, Jochen Markard (2008): Doing Institutional Analysis of Innovation Systems - A conceptual framework; Paper presented at the DIME conference, Bordeaux ; http://www.dime-eu.org/files/active/0/Truffer_Institutional%20Analysis_Aug08.pdf - David J. Spielman and Regina Birner (2008): How Innovative Is Your Agriculture? Using Innovation Indicators and Benchmarks to Strengthen National Agricultural Innovation Systems; The International Bank for Reconstruction and Development/ The World Bank. http://siteresources.worldbank.org/INTARD/Resources/InnovationIndicatorsWeb.pdf - John Ferejohn (2003): Why Study Institutions ? www.yale.edu/coic/ferejohn.doc Smajgl A, Leitch A, Lynam T (Eds.)(2009): Outback Institutions: An application of the Institutional Analysis and Development (IAD) framework to four case studies in Australia’s outback. DKCRC Report 31. Desert Knowledge Cooperative Research Centre, Alice Springs. http://www.desertknowledgecrc.com.au/resource/DKCRC-Report-31-OutbackInstitutions_Application-of-the-IAD-framework.pdf Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 41
- J. Rogers Hollingsworth (2000): Doing institutional analysis: implications for the study of innovations; Review of International Political Economy 7:4 Winter 2000: 595–644 http://faculty.history.wisc.edu/hollingsworth/documents/Hollingsworth.J.Rogers.Doing_Institutional _Analysis-Implications_for_the_Study_of_Innovations.pdf - Terje Grønning (2008): Institutions And Innovation Systems: The Meanings and Roles Of The Institution Concept Within Systems Of Innovation Approaches; http://www2.druid.dk/conferences/viewpaper.php?id=3785&cf=29 - Raymund Werle (2011): Institutional Analysis of Technical Innovation, A Review; Discussion Paper 2011/04, Institute for Social Sciences Organizational Sociology and Innovation Studies, University of Stuttgart; http://www.uni-stuttgart.de/soz/oi/publikationen/soi4_werle_institutional_analysis.pdf - Scott, W.R. (2001): Institutions and organizations. Sage Publications, Thousand Oaks - Frans Hermans, Laurens Klerkx, Dirk Roep (nd) : Structural conditions for dynamic innovation networks: a review of eight European Agricultural Knowledge and Innovation Systems. http://ifsa.boku.ac.at/cms/fileadmin/Proceeding2012/IFSA2012_WS2.2_Hermans.pdf - Marko P. Hekkert & Simona E. Negro (2010): Functions of innovation systems as a framework to understand sustainable technological change; Innovation Studies Utrecht. http://www.geo.uu.nl/isu/pdf/isu0810.pdf - C. Shambu Prasad, Andrew Hall and Laxmi Thummuru (2006): Engaging scientists through institutional histories; ILAC Brief 14 ; http://www.cgiarilac.org/files/publications/briefs/ILAC_Brief14_institutional.pdf - Andy Hall, Norman Clark and Guru Naik (2007): Technology supply chain or innovation capacity?: Contrasting experiences of promoting small scale irrigation technology in South Asia; United Nations University - Maastricht Economic and social Research and training centre on Innovation and Technology; Available at: http://arno.unimaas.nl/show.cgi?fid=8651.
1.2.4 Benchmarking Analysis
What is it? Benchmarking is an ongoing, systematic methodology for identifying, measuring, and comparing the work processes or functions of an organisation –firm, project, farm, etc…- from period to period and making comparisons with one or more other entities in order to bring about internal improvements. These entities can be different depending of the type of benchmarking. “Internal benchmarking” is the choice of performance measures that help track the progress made and show whether the desired results have been achieved within an organisation. It can also be the comparison of performance between different services / departments within the same organization. « External benchmarking » is the comparison of performance between different organizations. This comparison can be “broad” –looking at a whole range of organizations- or “narrow” — looking only at specific organisations that display similar features to one’s organisation (size, products, technologies used, etc…). It can be a comparison with local, regional, national or international organisations. Similarly, benchmarking can be ‘vertical’ (focused on one particular function: ie, irrigation, seed production, marketing, processing, etc.) or ‘horizontal’ (covering several functions). Benchmarking analysis is akin to an analysis of “best practices”. It helps, through comparisons and adjustments, to improve processes, production performance and quality, and hence competitiveness.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 42
What is its relevance for innovation assessment? Benchmarking is essentially a performance gap analysis. Used creatively, benchmarking has several potential uses in agricultural innovation assessment: • The comparison between different farms can help identify innovations that give competitive advantages, thus it shows areas in which innovation is needed, costs reduced or activities changed. • The comparison between farms can help map the processes in different farms and thus highlight differences in the uses/performance of a particular innovation • The comparison between farms can highlight differences in innovative practices • Developing an innovation agenda / action plan can use benchmarks to help monitor progress. How to do it? Benchmarking methodology is different depending on the type of comparisons that are aimed for: internal or external; broad or narrow; horizontal or vertical. The following steps can be considered as generic steps that would need adjustments depending on the objectives of the benchmarking. One important principle is to approach benchmarking as a process of discovery learning and analysis of all that can contribute to improving performance and achievement of objectives. The process of learning can be conducted by a local research team, or by a group of stakeholders, for example rural advisory forum, innovation platform, farmers’ learning groups, farmers’ field schools, “farmers’ study groups”, etc.. In the latter case, farmers are engaged in a process of learning from each other. Ideally, benchmarking is not a one-off analysis. As processes and methods evolve with time, benchmarking should be a recurrent analysis, helping to (re)evaluate performance and adjust methods to objectives. Phase 1: Planning This phase starts the process by preparing as much as possible the benchmarking. It should answer the following questions: • What is being researched? What should we focus on? • What are we comparing against? What other organizations / functions / innovations, etc… are we comparing ours against? • How are we going to collect the information? What tools are we going to use? How do we plan to analyse the information once collected? The latter point is of paramount importance. Like in all assessment process, the choice of methods – and of the type of information collected- must balance out constraints such as the quality of information needed (detail, precision, quantification) and the costs of the assessment (time, finances, need for external expertise, etc…). Usable Tools: Mind Maps; Concept Maps; Group model building; Joint Visioning
Phase 2: Analysis of information Analysis in benchmarking consists of a review of activities, methods and processes, evaluation of their strong and weak points and conclusions on how they contribute to performance. The review covers both farms /
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 43
A full benchmark analysis covers both analysis of information and setting standards for the future, ie, performance thresholds. - The analysis of information should attempt to answer straight questions: • What is the gap of performance between the two entities compared? • What processes, activities, methods explain this gap? • What are the costs implied in the achievement of the best performance? This gap can be positive –in favour of our own entity- or negative – in favour of the other entity. There is no standard limit to the types of analyses that can be conducted. Here are a few: • Process Mapping: this is the diagramming of any type of process, visualizing the flow of activities, information, inputs, responsibilities, etc… • Value Added Analysis: this is reviewing every activity to decide what value it adds to a process, and if it can be eliminated. • Productivity Measurement: this is to find out the ability to provide services at minimal expense • Quality measurement: this is to find out if the quality of a product meets or exceeds expectations cost effectively. - Setting standards should attempt to set objectives to be more competitive: • If the gap is negative, ways should be defined to bridge the performance gap. • If the gap is positive, ways to capitalize and/or improve of them can be devised, so as to keep up the performance. An important principle is to be realistic, ie, set few objectives and translate them into operational terms: what methods exactly are to be introduced? What costs are implied? What are the roles expected from each? The more accurate the description, the better it is to mobilize actors for change. Usable Tools: Process Mapping; SWOT Matrix; Force Field Analysis;
Phase 3: Action The conclusions of benchmarking and the action to be taken must be translated into detailed plans, to be evaluated and adjusted regularly to assess progress. Action following benchmarking is at two levels: Preparing the change process : If the benchmarking is done as a collective, multi-stakeholder assessment, and followed by action planning, its outcomes and proposals have first to be communicated, so as to have buy-in from all stakeholders, especially those who have a role in the implementation of changes. Action planning to implement the changes: Planning the changes follows the standard “what”, “who”, “when”, “how” This consists of determining the tasks, roles and responsibilities, expected results and needed resources, as well as the means of verification. Usable Tools: Action Plans;
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 44
Potential and limits Benchmarking has been developed and used for a specific context: industry and services in the Western world. This context is characterized by a high degree of control over processes, specially the physical environment and logistics. Applied to agricultural innovations in developing countries, benchmarking takes place in a radically different context: uncertainty of the natural environment, of the political environment, of logistics and access to markets, differences in resource base (soil fertility, topography, water, etc…). Comparing two situations with very different environments, resources and access to markets is interesting only if these differences are well taken into account throughout benchmarking. Links to case studies - CQ University of Australia: Benchmarking Principles; policy.cqu.edu.au/Policy/policy_file.do?policyid=2224 - Tor Guimaraes, Kathryn Langley, (1994),"Developing Innovation Benchmarks: An Empirical Study", Benchmarking for Quality Management & Technology, Vol. 1 Iss: 3 pp. 3 – 20; http://dx.doi.org/10.1108/14635779410073283 - Benchmarking for Innovation and Improvement; Innovation Insight Series Number 4 http://www.psu.edu/president/pia/innovation/ - Lisa Jack: Benchmarking in Food & Farming, Creating Sustainable Change; Book Chapter; https://www.ashgate.com/pdf/SamplePages/Benchmarking_in_Food_and_Farming_Ch1.pdf - David J. Spielman: How Innovative Is Your Agriculture? Using Innovation Indicators and Benchmarks to Strengthen National Agricultural Innovation Systems; Agriculture and Rural Development Discussion Paper 41; http://siteresources.worldbank.org/INTARD/Resources/InnovationIndicatorsWeb.pdf - Sudarma Samarajeewa, Maury Bredhal, Getu Hailu and Daniel Fisk: A holistic approach to performance benchmarking of farming practices in Ontario; http://www.ccaontario.com/FCKEditor/File/Samarajeewa.pdf
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 45
2 A PROCEDURE FOR ASSESSMENT TEAMS
Assessing local innovation systems requires wide ranging professional knowledge and skills. The following procedure –inspired by the Agricultural Research for Development (ARD) procedure designed by ICRA involves a series of iterative steps directed by an interdisciplinary and interinstitutional team engaging the involvement of various innovation actors. The five steps (visualized in the Diagramme below) and outlined in the next sections guide the assessment process and provide for a confrontation of disciplinary insights and innovation actors’ interests related to the issue being assessed. Throughout the process, team members mobilize their knowledge and skills to contribute to the assessment, keeping in mind these three general principles. • The procedure distinguishes yet combines elements of verifiable fact finding and analysis with participatory analysis and decision making. The required analysis involves the integration of professional/scientific and indigenous/local knowledge; by itself, neither of these two types of information is sufficient. This knowledge can come from previously published data, individual or group interviews, participatory exercises such as mapping, model building or matrix ranking etc. Judgements have to be made about the relevance and reliability of all the information required and collected. • Participation of rural people in decision making is never total and never easy: increasing the degree of participation of others inevitably involves a loss of control by the team (over direction, time frames). Power differences within societies guarantee that certain actors are heard and have more influence than others. • At almost all steps, different methods and tools can be used to gather and analyse information, and how the tools are used affects the degree of participation of other actors in the process. At each step of the procedure therefore, the team has to decide on the methods and related tools that will best provide the information required, or make the best decision, given the manpower, financial resources and time available.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 46
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 47
JOLISAA LEARNING RESOURCES BANK
Assessment of Local Innovation Processes - Team Procedure
2.1 Organize A Local Team
What is it? Assessing a local innovation system is almost by definition an inter-disciplinary endeavor. This must be represented in the composition of the team conducting the assessment. Interdisciplinary and inter-institutional teams are meant to cover the disciplinary competences and interdisciplinary perspective needed to assess the often complex and multi-faceted innovation processes. In this step, the team must arrive at a common understanding of what they have to do together, how they are going to do it and who does what: • Review their terms of reference –if they exist-, clarify and elaborate on them, and define as clearly as possible what is expected of them, deadlines, outputs, etc… • Organize themselves as a team. For efficient team work clear planning, roles, ground rules, and agreed mechanisms of decision-making are necessary. There must be working rules for common tasks while other tasks have to be divided individually or within sub-groups. Team work fails, if it is not implemented with a clear understanding of common objectives. • Agree on facilitation principles. A main facilitator or, alternatively, rotating facilitators should be designated for the conduct of the assessment. • Plan the assessment activities. The assessment will necessarily need continuous re-planning. But the team needs some preliminary work plan to get them started. How to do it? In this step, the assessment team discusses among themselves and consults with relevant actors to come to an agreement. The following questions are proposed, to lead the process. Why are we working together and what are we aiming to achieve? • What goal do the team members want to achieve together? • What is the purpose of the assessment? How are the client institutions, beneficiaries and other stakeholders likely to use and benefit from the results? Who are we and what are our expectations? • Who are the stakeholders involved in the assessment? In the team? Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 48
• • • •
What are our interests in joining the team? How does the assessment relate to our organisation’s goal? What are each member’s expectations and fears concerning the expected outputs and the team process? What can we contribute to the assessment? Are there any other organizations that should be included in the team?
What exactly are we trying to achieve together? • What are the outcomes which will contribute to our common goal? • How will these outcomes contribute to our own organisation’s goals? • What innovation case studies are we going to work on? For whom are we doing this? • What contractual arrangements are we entering? • Who are the clients and beneficiaries of the assessment? When are we planning to do this? • What more concrete outputs will we produce to achieve these outcomes? • What main assumptions are we making in deciding on these outcomes? • What activities are necessary to achieve the outputs identified? When? • Who is responsible for the different outputs or activities planned? • What activities will we carry out to monitor and evaluate the achievement of the agreed activities? • What activities will we carry out to monitor and evaluate the process of working together? • How will the lessons learned be used? How are we going to do this? • What resources are available to carry out our activities? • What methods are we going to use? • Which roles are necessary for the team to function efficiently? Is each person’s role clearly defined? Are the team members happy with their interaction? Do all members actively participate? Are members happy with the facilitation/leadership within the team? Are there accepted rules of conduct for team members? • Which members have the necessary mandate and competencies (experience, disciplinary expertise)? • What competencies still need strengthening? • Who will facilitate team work processes? What should be the outputs? • A common definition of the clients, beneficiaries, objectives, tasks and expected outputs of the assessment, including indicators for verifying achievement. • A written definition of the problem as understood by team members. • A work plan, showing the different field phases, the proposed activities, methodology, a timetable, interim outputs in terms of data analyses completed and decisions made (including deadlines). • Defined roles for each team member, including the team’s ground rules written as a “team contract”. What tools can be used? Visualization – Brainstorming - Mind Mapping - Team Work Planning - Team Facilitation
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 49
JOLISAA LEARNING RESOURCES BANK
Assessment of Local Innovation Processes - Team Procedure
2.2 Analyse The Situation and Clarify The Issue
What is it? Context analysis –or ‘situation’ or ‘situational’ analysis- and defining the system to be assessed are done iteratively. In innovation assessment, a situation analysis is an exploration of the broad context or external environment in which innovation processes take place. There are two reasons to conduct a situation analysis as a preliminary step in innovation assessment: • In an assessment perspective –with a view to understanding the situation-, it is about identifying the factors which have an influence on innovation performance • In an action research perspective –with a view to changing that situation- the analysis is about exploring the context in which collective learning for innovation will take place in the future, and better targeting the action plan. Situation analysis includes aspects such as: • An overview of the state and condition of innovation actors, farming systems, ecosystem, local economy, political system, etc… (including identification of trends and pressures) • Identification of major issues –economic, social, cultural, and technological- impinging on innovation processes that require attention; At this stage, there is no exhaustive analysis –this will be done at later stages of the assessment. It is a simple identification of issues that the assessment team and innovation actors think should be on the assessment “agenda”. How to do it? In this step, the assessment team looks at policy issues, markets, institutional issues and other macro-developments in and outside agriculture that may have an influence on the issue and on attempts to solve it. The team also identifies possible actors and interviews them to gain an insight into the different perceptions of the issue. The team learns to see the issue as the result of the functioning of a broadly defined “soft system”. This step of putting the issue into a broader development context and integrating actors’ perspectives leads to: • a redefinition or further elaboration of the “issue” • a demarcation of a local innovation system that the team will analyse in more detail (see next step)
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 50
These are questions that can lead the process: • What is exactly the issue the team is supposed to assess? • What micro and macro factors in and outside agriculture may have an influence on the issue? • What has been done in the past to solve the issue? • Who are the stakeholders and what are their perspectives regarding the issue and the assessment being conducted by the team? Context analysis can easily be performed during a stakeholder workshop, in a participatory fashion, with the assessment team acting as facilitators. What should be the outputs? A mapping of the context both visual (as a mind map for example) and textual (as a short report for example). The visual mapping will be very helpful to delineate the operational system (see next step). This mapping should feature topics like: • All themes, issues, questions, etc… from the point of view of the different disciplines in the team and from the stakeholders • A redefinition or further elaboration of the issue • A central research question which the assessment of the local innovation system is supposed to answer at the end of the study • Secondary questions which contribute to the main central question • A list of stakeholders The context analysis is also an opportunity to develop a preliminary plan for the assessment with: • the establishment of a stakeholder task force working with the assessment team; • the identification of stakeholders/partners to be mobilized by the assessment team; • the development of a stakeholder involvement plan; • the selection of experts to consult during the analysis; • the list of potential key informants; What tools can be used? Secondary Data Analysis - Defining an operational system - Context Analysis - Mind Mapping - Driving Forces
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 51
JOLISAA LEARNING RESOURCES BANK
Assessment of Local Innovation Processes - Team Procedure
2.3 Define The Operational System
What is it? The ‘operational system’ is the system on which an assessment team will zoom in its analysis and identifies strategies to support local innovation processes. As the purpose of the team is to promote a change process that contributes to enhancing innovation processes, an ‘operational system’ reflects a desirable ideal situation and the need for a transformation process to arrive at that ideal situation. The ‘operational system’ is the system that the stakeholders and the team want to change in order to address the issue at stake. The desired ideal situation will be defined in relatively broad and abstract terms that do not bias the analysis of the operational system towards a narrow range of solutions. Moreover, the operational system must be at a high enough level in the systems hierarchy to include all the required elements to ‘manage’ the change. But it must not be at a higher level (and include more elements) than necessary, as that would be inefficient use of resources. The choice of the ‘operational system’ should also take into account that it must be possible to analyse the ‘operational system’ and propose solutions in the time and with the resources available. Why define an ‘operational system’? At its basics, defining an operational system means simply recognizing that there are so many levels – spatial, thematic, social, etc…- innovation can be assessed that it becomes an impossible task. Similarly, trying to analyze too complex a system may yield rich and diverse information, but one that runs the risk of turning the assessment into an academic exercise. “Operationalizing” the system means focusing on those elements that local stakeholders can act on and bring about change. How to do it? Two things must be done before engaging in the definition of an operational system: • Clarifying the issue: What is it exactly that we want to assess? For example, do we want to know why integrated soil fertility management has met with very limited success only? Why zero tillage has been adopted by few farmers and abandoned after a few years? Why the adoption of Prosopis had negative effects and whether the innovation package can be redefined to become more attractive? Etc… • Defining the objectives of the assessment: Why do we want to do the assessment? For whom and to what end? Is it simply a diagnostic study? Is it a pre-requisite for an action plan? Etc…
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 52
The exploration of the issue forms the main basis for defining the ‘operational system’. But turning the two points above into research questions will provide the information to decide what elements should be included in the system to be assessed. 1) First, the team reviews and discusses all the elements of the issue they have defined. They then ask themselves whether the issue is adequately represented or whether they should redefine it by adding more elements. 2) Second, the (re)defined issue is thoroughly discussed using two questions as guide: • Does this representation of the issue cover the concerns of all stakeholders or just a few? Is there a need to enrich it with other elements? • Does this representation go beyond the mandate and capacities of the assessment team and stakeholders involved? Is it too narrowly defined or is its scope too limited? 3) Then, the visual representation is turned into an operational system by selecting those elements they agree should be included in the assessment. By drawing a boundary around this selection, the team distinguishes it from other elements. It is a way of saying “all those elements are interesting to study but these which we leave out are those that are not essential to solve the issue being assessed”. It is thus an exercise of priority setting. By exploring collectively the issue, and looking at its wider context, insight is gained on questions such as ‘whose problem are we talking about’, ‘who are involved in the problem situation and how’, ‘how do stakeholders perceive the problem’, ‘what socio-economic, ecological and political factors are involved’, etc. Defining an operational system is also an iterative activity in innovation assessment. The first definition of the operational system, formulated right after the clarification of the development context, may need to be reviewed or refined as the team progresses with their analysis. By way of summary: • An ‘operational system’ reflects the interests and perceptions of various stakeholders • The interventions which the assessment team undertakes are at a level that is high enough to contain the elements needed to address the issue (e.g., supporting the Prosopis innovation process) • It is focussed enough for the team and other stakeholders to be able to further analyze it; • It is focussed enough for the stakeholders to manage the necessary change by implementing the ‘solutions’; • It is within the mandate of the stakeholders and the team to participate in the further analysis and implementation of strategies; • It is within the capacity of the team to deal with; What should be the outputs? • •
A visual representation of all the elements that the team has identified as the “system” to be assessed and the links between them; A list of all these elements with information available on them, and research questions.
What tools can be used? Context analysis - Mind mapping.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 53
JOLISAA LEARNING RESOURCES BANK Assessment of Local Innovation Processes - Team Procedure
2.4 Describe the Innovation and Identify Actors
What is it? An important step, once the assessment team is organized and the issue clarified –why are we doing this assessment? - is to describe the innovation. As innovation processes are also social processes, this step is also about identifying innovation actors, who will be analyzed in the next step. How to do it? There is no standard way of describing an innovation. The way we describe it is influenced by the assumptions we make as to what an innovation is, what causes it and how it proceeds. Description can therefore focus on some factors rather than on others. Preliminary documentation on the study of innovation can therefore be helpful. The Section on Approaches to Innovation described above give a short introduction to different approaches to innovation assessment. - Innovation Histories - Institutional Analysis - Social Network Analysis - Benchmarking Analysis A wider list of references is proposed in the Assessing Innovation & Innovation Capacity section of the Innovation Bibliography. A preliminary description can be structured by answering the following questions: • What is its purpose and how does it work? • How did it come about? Who had a significant role in its development? • What are its benefits and limitations? This description can then lead to more specific questions to be investigated further. What should be the outputs? At this stage, no in-depth analysis of innovation processes is expected. The output is a description of the innovation as accurate as possible, which maps out the innovation and generates specific questions. Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 54
Ideally, the output should be a preliminary report to be handed out to stakeholders and serve as a basis for discussion and identification of issues to be explored further. What tools can be used? Context Analysis - Mind Mapping - Stakeholder Identification Matrix - SWOT Analysis - Innovation Timeline - Secondary Data Analysis - Focus Groups Interviews - Farming Systems Models - Semi Structured Interviews
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 55
JOLISAA LEARNING RESOURCES BANK
Assessment of Local Innovation Processes - Team Procedure
2.5 Analyse Actors
What is it? Innovation is first of all a process involving people and organizations. Analyzing actors is therefore one of the first steps that should be taken in any innovation assessment. This analysis answers several concerns: • Studying the role of actors in innovation process in the past • Describing their role in the present • Providing information that can help recommend what could be their role in the future, individually or collectively. This projection into the future is also a means to mobilize and create incentives for them to get involved. For this, their participation in the analysis is crucially important. How to do it? At this stage of the procedure, the assessment team should identify everyone with a “stake” in the innovation, whether as initiator, contributor, beneficiary, etc… These actors can be farmers, researchers, extension agents, project workers, policy makers, traders, etc… There is a wide array of tools to do the analysis. Most of these tools are participatory and use visualization (diagrammes, matrices) and can involve people of different backgrounds and education levels. The following questions can lead the process of actor analysis. • Who are the actors –or group of actors- who have been involved in the innovation process? • What have been their roles? • How are actors organized to lead the innovation process? • Who is likely to benefit or lose from the innovation? • Are there any mechanisms to coordinate innovation activities? What should be the outputs? • •
A description of the strategies and objectives of the different stakeholders in relation to the innovation A description of their role in the innovation process and the extent of their involvement
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 56
What tools can be used? Stakeholder Identification Matrix - Stakeholder Objectives Analysis - Actor Influence Matrix Innovation Timeline - Organisational Linkage Matrix - Venn Diagramme
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 57
JOLISAA LEARNING RESOURCES BANK
Assessment of Local Innovation Processes - Team Procedure
2.6 Plan Action
What is it? In this last phase of the innovation assessment procedure, the assessment team develops an Action Plan to follow up on the assessment. Action Plans, in general, try to outline what concrete actions should be taken to reach some agreed objectives. In a participatory process, through the Action Plan, the assessment team should ideally try to formalize a partnership to support innovation. The Action Plan becomes then a strategy document outlining what partners commit themselves to doing, individually and collectively, to lead the innovation process to a positive outcome. How to do it? Action planning is about identifying what steps should be passed to reach the objectives. It formulates each step and the resources needed to move from the present situation to the desired situation. More specifically, it should state what should be done, the way it should be done and by whom, as well as monitoring and evaluation procedures. The Action Plan can be presented in a table or chart format and gives enough information and guidance to get people working on the actions they have collectively decided on. The format can be organized in various ways: topics, dates, priorities, responsibility, location, etc... Each step within the plan clearly defines the components of what, who, when and how. These are some key questions to guide the process of action planning: • • • • • • •
What is your understanding of the present situation concerning the innovation process? What are the concrete steps needed to bridge the gap between present situation and desired situation (the objectives decided upon)? What are the resources needed? Who is responsible for leading / overseeing each step? What time is needed for each step? What support is needed? What are the milestones / reference points which will be help decide whether the Action Plan is on a good path?
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 58
What should be the outputs? • • •
Selection of the best strategy to support the innovation process, based on partners’ organisational mandates, resource base, livelihood strategies and vision, and projections of future trends of driving forces. Identification of the steps to realize the Action Plan and activities to be carried out in support of the innovation process. Key indicators that can be used to monitor and evaluate the implementation of the Action Plan.
What tools can be used? Team Work Planning - Developing an Action Plan - Scenario Building -Visioning
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 59
JOLISAA LEARNING RESOURCES BANK
Assessment of Local Innovation Processes - Team Procedure
2.7 Reflect
What is it? Reflection is a central part of an action research process. In innovation assessment, its main goal is to facilitate individual and collective learning through discussion after all activities. It does so by allowing people to closely review their individual and collective experiences to expose and challenge assumptions, to provide new insights into self and others and draw lessons that help understanding and adaptability. Reflection is an essential component of continuous learning. The goal of reflection is not to solve problems but to draw lessons from action by reviewing critically what has been done. Reflection is mainly about processes, not about individual performance, although the review of processes may shed light on performance. There are three aspects to reflection: • Return to experience: recalling and describing salient events in detail, in order to document and analyse processes • Attend to and connect with feelings: recalling feelings and sorting them out in order to use helpful ones and contain or remove obstructive ones. • Evaluate experience: re-examining experience in the light of objectives and knowledge in order to integrate this new knowledge into the conceptual framework How to do it? Reflection is best done rapidly, if possible immediately after activities, when the events are still fresh in the mind. Reflection should involve all those engaged in the activities being reflected upon, in a group session. Since reflection depends entirely on the goodwill of participants, an important principle is for each participant to adopt an attitude of openness. This means: • To focus on constructive comments • To consider all participants as equal The following questions can help conduct the reflection process: • What did we plan to do and why? • What really happened and why? Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 60
• What were the differences and why? • What went well and why? • What went wrong and why? • What lessons can we draw from this experience ? How can we improve next time? In addition to team reflection, individuals can also reflect and document their thoughts individually in the form of personal memos, diaries, learning logs, etc… to improve their own learning. What should be the outputs? Reflection can be documented in any form deemed adequate by the team: short notes, reports, memos, etc… These documents are not “formal”, but they can be shared and discussed with others who can benefit from them. What tools can be used? Brainstorming - Mind mapping - Force Field Analysis
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 61
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3 INNOVATION ASSESSMENT TOOLS
3.1 TOOLS FOR TEAM WORK
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 62
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.1.1 Brainstorming
Importance Brainstorming uses the mind’s power of association, one idea triggering another. It stimulates a maximum of ideas in a short time and improves participation. Brainstorming is a good way of producing a lot of ideas in a short time by mobilising a group’s creativity. Mind mapping helps teams summarize information and ideas in a non linear fashion by visualizing the relative importance of each idea and connecting ideas to each other. Il allows gradual improvement of the map by addition of information from different sources and thus fosters the process of assimilating and connecting. This is a continuous process, as new ideas are added on. Steps Collecting ideas • Make the question asked as specific as possible: the more specific the question the more productive and focused the brainstorming; • Make sure everybody has understood the question to be brainstormed; • Remind the team that in brainstorming all ideas are taken and none censored; • Set a time limit for the brainstorming session, but extend it if necessary; • Once all cards collected, go around the group and try to get any last ‘repressed’ ideas.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 63
Processing ideas • Go over each idea in turn and clarify it. What does it mean? What are its pros and cons? Does the team want to keep it? • Repeat the process until the list is small enough for the team to agree on what ideas are worth following through • Express the ideas and information in short phrases or, better, in single words to avoid adding, • Use print letters to make the map more legible, • Group similar or related ideas in clusters.
Caution •
•
Not blocking brainstorming’s power of association by criticising each idea as soon as it is expressed (« killer phrases » like: « that’s dumb », « it won’t work », «it’ll cost too much », « we’ve tried that before », etc…); this would discourage team members and introduce an element of self censorship. Not considering the output of brainstorming as more than a first go at an issue. The output must be examined closely and analysed. Ideas that are clearly not backed by evidence and proposals that are not feasible should be eliminated. The team must avoid to keep poor ideas solely not to hurt somebody’s feelings.
Information - Website: http://www.brainstorming.co.uk/ - MindTools: Brainstorming; available online at: http://www.mindtools.com/brainstm.html - Business Balls: Brainstorming process; available online at: http://www.businessballs.com/brainstorming.htm
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 64
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.1.2 Team Facilitation Importance Most team interaction takes place during meetings, for planning, decision making, analyzing, etc… Steps Preparing the meeting • Be as familiar with the agenda as possible; • Prepare the logistics needed: materials, equipment, drinks, etc…; • Coordinate with individuals about the progress of their tasks; • Determine the steps of the meeting and select a method for each step • Plan approximatively the time needed for each step • Make time for changes and pauses • Decide how you are going to open and close the meeting Facilitating the meeting Questions that may help prepare the agenda of the meeting are: • What are the issues to be discussed? What is the expected outcome of the meeting and what is it needed for • Who in the team has background information on these issue(s)? • What needs to be prepared by individual members or sub-teams before the meeting? • What actions are likely to be needed? • What are the likely time constraints or expertise requirement? State meeting objectives and have agreement on agenda and timing • Make sure there are helpers for the meeting (note taker, time-keeper, etc.); • Ask everybody about their understanding of the issues under discussion; • Ask who has background information on the issues and have everybody listen; • Find out whether there are questions; • Adapt to meeting situations. EXAMPLES on meeting situations - The members of the team are keeping quiet Try to answer the question: why are they silent? Perhaps they have not understood the task: • If it’s because they are tired, then you can suggest a pause or a physical activity; alternatively change the subject of discussion to renew their interest; • If it’s because they think their work is finished, then try to have them sum up their main conclusions and get the ball rolling again, either by pointing out unanswered questions or by evaluating the meeting and setting the agenda for the next; • If it’s an extended lull, try asking challenging questions, using open-ended questions; alternatively you can pick on a point made before and ask them to elaborate.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 65
- Team members do not understand the task Find out what has not been understood and clarify it; give the team the information you have about the task; • Reformulate the question, or break down the task into simpler ones; • Reallocate the task to several persons if it’s too complicated for one. - Some team members do not participate Try to answer the question: why are they silent? • If it’s because they have a problem, try to find out about it privately and solve it; • If it’s because they are of the silent type and don’t generally say much, do not try to force them to talk at all cost, that would only cause embarrassment ; • Give them a task where they can participate without talking (note taking, time keeping, etc…); • Put them in a sub-team with other members who do not intimidate them; • Call on them on issues where you know they have good experience or expertise. - Some team members monopolize speech • Do not interrupt them abruptly, wait for an appropriate moment, then summarize what they said to show that you are listening; • Choose an appropriate moment and suggest a procedure to the team that limits speech time (no more than one minute, draw a list of speakers, visualizing ideas on cards…); • On occasions, ask the speech monopolizers to summarize what others said, to force them to listen as well as speak; • As a last resort, give them a role (visualizer, note-taker) where they have to listen as well as speak. - Some team members are too aggressive Don’t try to argue with them (this will only upset them) and don’t let yourself be drawn into the conflict. • Subtly point out the emotional content of the aggression (‘I can see that you are upset… ‘); • Show that you’ve understood their grievances by summarizing them; • Ask them to be more specific by giving examples; • If everything fails, put the problem to the team: What do they think about it? • Remind everybody of the rules of communication and of their team contract. If they are aggressive towards you, remain calm but be firm and don’t allow further aggression, this will undermine your own role. - The team is faced with a question they cannot solve • State the question blocking the team and make sure everybody has the same understanding of it; it may be that the question has been badly formulated; • State whatever options have been suggested by the team; • Suggest to split the team into sub-teams to explore the implications of each option. - You’re asked a question you cannot answer • First check with that you understood the question correctly by asking for clarification; this will also give you time to think; • If you really don’t know the answer, say so frankly and promise to check it out and give the answer as soon as possible; • And do it; • Alternatively, ask other participants whether they know the answer; Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 66
• •
The important thing is, first, to be honest and, second, not to let an unanswerable question block work team; Write it down on the board so that you can come back to it later.
- The team has discussed an issue and is getting nowhere Several options: • Suggest to the team to have a break to think individually about new perspectives on the issue; • Suggest to split into sub-teams to explore the issue; • Make suggestions of your own if you have expert knowledge of the issue, but do not take over the meeting! • Guide the team by asking critical questions; • Suggest to the team to pause and consult with outsiders who have expert knowledge. - Tempers are flying high First of all, do not take sides; this will alienate the ‘wrong’ side. • Reduce team stress by putting the cause of conflict into perspective; emphasize the common goal; • Refocus team attention away from people and onto the issue; • Provide the team with something to motivate them by emphasizing the areas of agreement and pointing out the potential solutions to the issue. - The team cannot reach a consensus • Make the team sort out the options: what type of consensus decisions have been proposed? • Find out who objected to them and why: was the objection based on formulation or on principle? • If on formulation, suggest to change it in a way acceptable to a majority of members; • If on principle, make the team pause and reflect on what it is that they are trying to achieve: what type of consensus decision? • If no consensus option is acceptable to a majority, explore with the team what more creative ways can be used to deal with the issue; • As a last resort ask the team whether they are willing to put the issue to the vote. - The team is split into sub-teams Make sure the tasks have been well understood by everybody, use action verbs to remind responsibilities • What is to be accomplished? • When, how, by whom? • When is it due? • When is the team meeting to reconvene? Closing the meeting • Review the meeting by going over each point of the agenda: where they covered and can everybody live with the decisions made? Is everybody happy with the way the meeting went? What could be improved? • Allow everybody to say a last word on the meeting; • Set date, time, objective, responsibilities for next meeting.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 67
Caution A facilitated session is not an improvised or chaotic session but a meeting which is fairly structured. The facilitator has the responsibility of providing the atmosphere, tools and techniques necessary to reach the objectives of the meeting within the time available. He/She must therefore start with making the objectives of the session as precise as possible. In general, the facilitator should not influence the content or product of the team, he/she must be aware of the team’s knowledge limitations. It may sometimes be necessary to provide the team with guidance in areas such as data analysis tools and problem solving techniques. Information - Johnson, D. W. and Johnson, F. P. : Joining Together: Group Theory And Group Skills. Allyn and Bacon, Needham Heights - Clark, N.: Team Building: A Practical Guide For Trainers. McGraw-Hill, Maidenhead Bentley, T.: Facilitation: Providing Opportunities for Learning. McGraw-Hill, Shoppenhangers Road, Maidenhead. - Oomkes, F.R. and Thomas, R.H.: Cross Cultural Communication: A Trainer’s Manual. Gower Publishing Aldershot.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 68
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.1.3 Team Work Planning
Importance Planning anticipates and guides action but it requires an objective. A common objective is essential to team work planning. Most teams have externally-defined objectives, in the form of terms of reference, mission statement, job description or other. To plan for an improved performance, members must arrive at a common definition of these objectives, prioritize them and identify the activities need to achieve them. Steps Define objectives In team work there is almost always a hierarchy of objectives to consider, each contributing to the achievement of the other. There is also a confusion between closely related terms: goals, objectives, purpose. It is essential that the team arrives at a common vocabulary and a common understanding of what is its own specific objective within that hierarchy. In the common language, ‘objective’ is used to describe almost anything one is aiming for: getting food, earning money, buying a car, making friends, becoming a president. Yet objectives are not of the same hierarchical level. This may have no consequence in normal life. It may create serious misunderstandings among team members. The convention in project planning is to use ‘goal’ to describe a higher level objective to which a ‘purpose’ contributes. Both are ‘objectives’ of a different level. Thus a team purpose of "Assessing the process of Gadam Sorghum Private Public Partnership Commercialization innovation" is not an end in itself. It will contribute to a higher objective (“goal”) that needs to be explicited. • •
• •
A ‘purpose’ is the objective to which team work is directly related, as contrasted with a broader ‘goal’ to which team work is expected to contribute but which is beyond the team’s capacity to achieve. Ideally the team should be involved in its definition. Unfortunately, the team’s purpose is generally stated for them by somebody else (generally their employers or a project leader), with or without collaboration with the beneficiaries of the team’s outputs (stakeholders). It may not have been clearly defined, or it may simply be hinted at in the results expected from the team. The procedure for its definition is through the clarification of the team’s terms of reference. Careful reading of the terms of reference followed by a visualized discussion using a diagramme can help the team arrive at a common understanding of it. Once this common definition is arrived at, the purpose should be written up and justified.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 69
Define expected results • The results are the products expected at the end of a given period of time, in order to achieve the purpose. They are generally specified in the team’s terms of reference and need to be understood by all members. Their relation to the purpose must be also clarified: logically if all results are achieved, the purpose should be also achieved. • The procedure for the identification of results is also through a visualized discussion of the terms of reference, followed by the construction of a simple tree diagramme, where the results are the ‘causes’ and the purpose is the ‘effect’. The results expected from the innovation assessment team may be different from those of whoever commissioned the assessment. This difference should be made explicit because one contributes to the other. Knowing how their expected results are meant to contribute will help the team make necessary adjustments when decisions are to be made during the assessment. Define activities • The activities are the concrete, measurable events that must occur in order to produce the expected results. In practice, activities are what the team must do, or ensure it’s done by somebody else, so that its results are achieved. • The nature of the result (material or intellectual) will determine the type of activity (action, data collection, etc…); • The team must ask themselves: what is already available (information, materials) for the achievement of this result? What more do we need to do to achieve this result? • Once activities are identified, they must be plotted in a calendar to check for the feasibility of the Work Plan. Caution Under time constraints and especially in an experienced team, some of the members might often start the first steps of implementation while others are still completing the planning. Such a division of labour saves time but demands a highly effective coordination and a good deal of group discipline. No planning is static or final, especially in a participatory assessment, when the team has to take into account other people’s constraints. Ideally a Team Work Plan is a simple map of the activities, which needs to be constantly adjusted to these constraints. Information - NORAD (1999): Logical Framework Approach: handbook for objectives-oriented planning; available online at: http://www.norad.no/en/tools-and-publications/publications/publication?key=109408 - Greta Jensen (2010): The Logical Framework Approach - How To guide; available online at: http://www.dochas.ie/Shared/Files/4/BOND_logframe_Guide.pdf
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 70
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.1.4 Visualization
Importance Team tools often use visualization to: • Stimulate creativity through brainstorming and association of ideas; • Stimulate discussion and lead to a deeper understanding of a subject; • Allow negotiation between team members over ideas and perspectives; • Be both a process and product ; they can also be used as a presentation tool once finished and redrawn; • Promote more equal participation in discussion; • Structure a discussion process; • Document clearly discussion results; • Promote understanding of information. Steps Gather materials needed
Discuss rules Discuss and adopt rules for visualization: • Only one thought per card ; • Maximum Three Lines per card ; • Print clearly (no handwriting); • Close pens (they dry fast!); • Use full Size of card; • Use coloured cards for structuring. Procedure • Write topic for discussion on card and place at the top of the pin board. • Each participant takes cards and writes ideas following rules for visualizing. Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 71
• • • •
Collect all the cards Place all the cards on the pin board Discuss each card and arrange them into subject areas Write new card for each subject area.
Example of visualization with cards
Caution On ambiguities Visualisation as a method of team work greatly facilitates communication. It enables the least articulate members to express themselves. It also enables the team to deal with issues systematically and transparently. However visual representation of ideas may be just a shortcut for more complex thoughts. There is therefore plenty of room for ambiguities that need to be clarified and ideas to be developed. Specially in brainstorming, team members must keep in mind that ideas may change overtime. The product of brainstorming must therefore be considered as a simple “photograph” of team thinking at a very specific time, that need to be elaborated on and changed if need be. Information - UNICEF (1993): Visualisation in participatory programmes, a manual for facilitators and trainers involved in participatory group events. UNICEF Bangladesh, Programme Communication and Information Section
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 72
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.2 TOOLS FOR MAPPING AND TIMELINES
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 73
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.2.1 Activity Timelines Importance Timelines capture changes in the farming system over time. Changes occur with the seasons or over longer periods: • Farms may have two or three cropping seasons in a calendar year. e.g. in semi-arid areas of India two cropping seasons are recognised: Karif and Rabi. • Many farmers practice crop rotations which take three to five years. Typically land is left fallow, then used for cereals followed by root crops before being left fallowing again. Timelines help us understand relationships between climate, market prices, labour availability and farming activities. Timelines can help us understand the decisions that farmers make. They can help identify opportunities for improving the system and avoid inappropriate interventions. It will help identify conflicting demands on the farming system. For instance a new technology might be very labour intensive at a time of year when the farmer is busy with other things. Steps (Cropping Calendar) Deciding on important changes Before starting to build a timeline, decide what enterprise to look at. Encourage farmers to discuss how they perceive time passing. It will often be as a calendar year or by season. Include factors that affect the enterprise. e.g. A cropping calendar will often show rainfall or irrigation supply. There will be different labour requirements during the year.
done
Some enterprises are integrated with others, e.g. crop and livestock enterprises. Include the relevant activities on the calendar. Drawing the calendar Once everyone has agreed what to focus on, draw the time line on the ground. Using seeds, stones or symbols to represent key activities, encourage farmers to show when these occur. E.g. when each crop is grown and livestock management activities take place or what labour is needed. You may need to start this process but the farmers should soon take over. Show differences, like amount of rainfall or number of laborers, this can be shown by changing the number of stone/seeds, etc. More on the wetter Source: Pretty, Guijt, or busiest months and less on the others. Thompson & Scoones, 1995
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 74
Sharing the calendar The calendar prepared by the farmers should be shared with other groups in the community. This will improve the accuracy of the calendar. Sharing the calendar means sharing the information. All can benefit from this knowledge and help identify new opportunities to improve the system. Sometimes later or earlier planting might be an advantage or new varieties could be used. Example of a cropping calendar
Cropping Calendar in Wollo, Ethiopia. Source: Ethiopian Red Cross Society, 1988
Steps (Labour Calendar) Deciding on important changes Before starting to draw out the calendar it is best to discuss and clarify what activities are to be depicted. Decide whether it is important to distinguish between men's activities and women's activities. Once these decisions are made then a timeframe must be decided along with the time unit to be used. One common timeframe is the year with monthly units. But timeframes for labor can be as short as one day as well. Drawing the calendar Start the diagram by drawing on the ground a time line covering the designated time frame and divided into the time units to be used. Farmers are then asked to indicate with seeds or stones or other symbol at which time the activity occurs. The months in which key activities are undertaken are indicated with appropriate symbols. If a degree of change, like amount of labour use, is to be indicated then more stones can be placed on the busier months than on the slack months. Months with high labour demand receiving more stones than slack months. If it is important to show whether men or women are doing the activity then some symbol for gender should be placed on the calendar. Sharing the calendar The calendar prepared by the farmers should be shared with the wider community. Care should be taken to make sure all stakeholders get an opportunity to learn from and comment on the calendar. During this process not only will the richness and accuracy of the calendar be improved but also
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 75
insights will emerge on potential conflicts in labour use, and potential improvements to the way labour is used. Example of a labour calendar
Steps (Trend Lines) Deciding on important trends Sometimes it is important to understand how the value of something changes over time. e.g the prices of commodities. Comparing the changes in the value is known as a trend. Before plotting changes over time, decide what trend you want to look at. Farmers should be encouraged to decide what time frame to use. Some changes may be too small to see and others may be too big. This will affect what unit is used to measure it. Choosing the timeframe will also affect the plot. E.g. Differences in the price of a commodity may be very large between growing seasons but very small within a single growing season. Looking back to the past by plotting trends between years may help us see what might happen in the future. Plotting the trends With farmers, decide what units to use. e.g. days, months, money or qualitative statements like high, medium and low. Where qualitative statements are used need to be clearly understood by everyone. Draw out the x axis or time frame and mark out the agreed units of time usually months or years. Now draw out the y axis. Mark out the values to be used starting with the smallest.
they
At each point in time, ask farmers to show the value. This could be by drawing a cross or making a collection of stones that represents the value. Complete the plot by drawing a line between all the marks made.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 76
Sharing the plots Sharing the plot with other members of the community will help to improve its accuracy. Sharing the plots also means sharing the information. All can benefit from this knowledge. It is also important to find out how others perceive the trend, particularly if they were not involved in drawing it, e.g. women, different social groups, etc. Example of a Plotting trend lines
Caution Try not to show too many trend plots on one diagramme. Keep to one or two trends, and make a new diagram for more trends. Do not begin a drawing until agreement has been reached on the focus of the diagram and what units should be used. Drawings should start by tracing out the axes on the ground or on a sheet of paper. The values and how to show them, e.g. as stones, icons or numbers, understood before starting on the calendar. Information Internet Sites: - http://www.ids.ac.uk/eldis/pra/pra.html - http://www.oneworld.org/odi/agren/index.html - Pretty, J., Guijt, I., Thompson, J. and Scoones, I. (1995): Participatory Learning and Action: A Trainer's Guide. IIED Participatory Methodology Series. London: IIED. 267p - Mikkelsen, B. (1995): Methods for Development Work and Research: A Guide for Practitioners. Sage: India. 296p.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 77
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.2.2 Farming System Models Importance Farming system models are important because they help people to understand the complexity of farming systems. They can also; • Improve communication between stakeholders • Provide a common focus for discussion • Help identify new solutions to old problems • Discuss the implications of an innovation for the system • Give information on new research and development needs • Help in planning agricultural support services • Can inform agricultural policy • The farming system model is made up of components and its inputs and outputs. The boundaries differentiate system components from its context or environment. A more complex model would include more detail of the linkages or flows between components. e.g how cash, labour or products flow. • Farmers' indigenous knowledge and perceptions are important. It is important that they participate in the construction of models. They have a lot to contribute to the development of future farming systems. • Knowing what kind of farming systems farmers want in future can inform research priority setting exercises and agriculture policy. Farming system models provide information on research and development needs. Similarly, agriculture support service organizations can learn what inputs and information support is needed. • System models can be used to identifying new opportunities to solve existing problems. Steps Identify system components Start by encouraging farmers to talk about the enterprises in their farming systems. An earlier agroecosystems analysis could be a useful starting point. Using symbols or icons to represent the different enterprises, draw them on the ground or a piece of paper. Put a circle around them. Try to put all sub-components together under one component or enterprise, e.g. individual crop species together under ‘crops. Sometimes components overlap, e.g. multi-purpose tree species may be used for livestock feed and also mulch for crops. Try to match the size of the circle drawn around the component with it’s importance. People are likely to interpret it this way.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 78
Identify system boundaries The boundary can be drawn around those components or enterprises that farmers control. The enterprises inside the boundary make up the farming system. e.g. privately owned cropland, vegetable gardens and livestock are inside the farming system. Open access forests or common property grazing is important but farmers may have little or no control over them. Therefore they are outside the system. Components outside the main system boundary represent components of the environment. Identify system inputs and outputs Once the system boundary is clear, encourage farmers to identify what inputs come into the system and what outputs leave the system. These may come from local traders or the market, e.g. Fertilizers, pesticides, drugs, seeds or new livestock. Other external inputs may be from open access areas. e.g. fodder grasses cut from roadsides or swampland to feed the cows. Show these inputs on the left hand side of the model. Draw an arrow to show that they are entering the system. System Outputs: Outputs from the system are all those products leaving the farm. This includes all products sold in the market as well as those sold, exchanged or given to neighbors. Put outputs on the right hand side of the model. Draw an arrow to show that they are leaving the system. Complete a simple system model Sharing the model with other members of the community will help to improve its accuracy. Sharing the model also means sharing the information. All can benefit from this knowledge. It is also important to find out how others understand the model, particularly if they were not involved in drawing it. e.g. women, different social groups, etc. Sharing will help show what components of a farming system are common to many farmers and which components differ between farmers.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 79
Example of a simple model
Moving on to flow models Decide what flows to model Flows add another dimension to the simple agricultural system model. The most common flows are labour flows, cash flows, and product flows. Use the farming system model that farmers have identified to decide which flows to model. Some flows may impact on the components or boundary of the system: • labour flows may include off-farm activities as well as non-farm activities. These may be inside and outside the farming system. • product flows may mean including different markets in the model. Putting too many flows on one model can make it messy and difficult to understand. Where models are to be used to communicate with others it may be better to draw a separate model for each flow. Identify flow direction After deciding what flow to model, encourage farmers to discuss the flow in more detail. It is important to find out what is happening, and where does it go/come from. For example, in labour flows, e.g for activities like harvesting crops, it is important to find out the gender and age of the person carrying out the activity. Are they family labour or were they hired? Complete the flow model Sharing the model with other members of the community will help to improve its accuracy. Sharing the model also means sharing the information. All can benefit from this knowledge. It is also important to find out how others understand the model, particularly if they were not involved in drawing it. e.g. women, different social groups, etc. Sharing will help show what components of a farming system are common to many farmers and which components differ between farmers. Sharing the model can facilitate discussion on future improvements to the farming system. Flow models are easily understood by extension staff as well as researchers. Models provide a common understanding between farmers and other stakeholders.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 80
Example of a flow model
Caution On boundaries: Where to draw the boundary around a system will depend on the perspectives of those involved. e.g. The boundary around women's enterprises and activities are often different from those of men. It is possible to draw different system models of the same situation by placing the boundary in a different place. Where ever the boundary is drawn, make sure it is clearly. On drawing: • Leave plenty of space within your model. The components will be easier to see and more can be added • Use ovals and circles. They will be easier to see than boxes or rectangles • Do not overlap any of the components unless they share sub-components • Make the farming system boundary stand out from any others. Use a thicker line or different colour. On perspectives: • The model is a qualitative tool. It reflect the knowledge and understanding of the people who helped make it; • Sharing the model with others will add value. Include these different perspectives into the model; • Farming systems can change with the seasons or different circumstances. The model may not show these changes. It may be necessary to draw different models for different circumstances. Information Websites - http://www.ids.ac.uk/eldis/pra/pra.html - http://www.oneworld.org/odi/agren/index.html - http:www3.open.ac.uk/courses/cframedes/t860.html - Pretty, J., Guijt, I., Thompson, J. and Scoones, I. (1995): Participatory Learning and Action: A Trainer's Guide. IIED Participatory Methodology Series. London: IIED. 267p. - Lightfoot, C., Feldman, S. & Abedin, M.Z. (1991): Households, Agroecosystems and Rural Resources Management. ICLARM Educ Ser. 12. 80p
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 81
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.2.3 Innovation Timeline Importance An innovation timeline is a chronological account of the innovation process as it unfolded. It documents the development of innovation, with the major events, the actors involved, the results, etc… The timeline can be that of a single innovation (ie, a new product, a new type of organisation, etc…). It can also be that of innovation processes in a general sense (ie, innovation in an economic sector, in a particular region, in an organisation, etc…). In this case, the timeline does not so much describe how one single innovation came about, but more the succession of innovations, ie, the process of change. The tool is much used in compiling an innovation history. The timeline is built with actors who have been involved in the innovation process, in various capacities: farmers, processors, extension agents, researchers, traders, etc… It is generally built with a group representative of these actors. But it can also be done with each actor separately and then compiled into a single timeline. Steps Explain the purpose of the timeline As in any exercise of visualisation, the use of a timeline can be sometimes bewildering to a group if they do not understand exactly what is expected from them. The purpose, method of the innovation timeline and the use of its results must be explained. Similarly the subject being recorded must itself be very clear to people. The concept of innovation and the nature of the innovation to be recorded must equally be clear. Work out a provisional timeline Once there is common understanding on what is being raw a line on the ground or on a large sheet of paper and draw a line across it. Ask the group when it is best to start the record: do they remember / have a date that could be considered the start of the innovation process. Put that date at one end of the line, and the current date at the other end, to mark out a provisional timeline. This timeline can be changed later on, during the recording process, as new information comes out. Fill in the timeline The recording process can start, basically as a focus group interview, guided by the following questions. • How did it all start? Where did the innovation originate? • What events do they remember? When did they happen? • How did the innovation evolve at each stage? Who played a role for each event?
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 82
As the interview proceeds, answers are written down and put along the timeline. The number of guiding questions can be as high as the process facilitators want it, with the timeline serving as a visualisation device. Examples of Innovation Timeline
Tidy up and finalise the timeline
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 83
Once the interview is completed, the timeline is tidied up and reported on a document. There is no standard way of finalising the results of a timeline exercise. The final format will depend on the use for which the timeline is intended: • -It can be reported in the visual format into which it was constructed, if it is meant as a simple support for discussion and/or distribution to stakeholders • -its contents can be written up in a narrative format, if it is meant as a simple support to facilitate the group interview. Caution On sources of bias Like in all Participatory Rural Appraisals using group interview, there may be biases in the process of building a timeline. Some possible sources of bias: • If the group interview is not representative of the diversity of stakeholders, some important aspects of the innovation process will be missing • If some group members are more articulate / dominant during the group discussion, their views will shape the timeline • The simple structure of the timeline can itself be a source of bias, as it does not represent easily iterations in the process. On the use of innovation timelines The innovation timeline provides inputs to innovation assessments, but cannot be considered as an assessment in itself. Its chronological structure is similar to that of innovation histories, but the latter is more thorough and needs more information, using other tools more adapted to that information.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 84
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.2.4 Mind Mapping Importance Mind Maps are a valuable tool for exploring people’s understanding of a problem or subject. Unlike writing notes, mind mapping allows you to capture ideas and information on paper without needing to know how they will fit together. Using the diagrams, you can explore the subject together without being distracted by a rigid structure. You can use mind mapping to work: • by yourself, on your own personal understanding of a complex situation, • to interview someone else on their understanding of a complex situation, • or even within a group, to facilitate an open exploration of a complex situation. What it is • Mind mapping is a way of exploring what people think is associated with a central theme, topic, problem or situation. It includes all ideas and details in an open minded way. What it is not • mind mapping is not a way of diagnosing a problem or analysing a system. It does not show the relationships between ideas or try to integrate them. Why use mind mapping To explore and capture many ideas associated with the topic of enquiry.
This can help you gain new insights on old problems.
To capture different points of view.
This can help you understand the different perspectives people have on the same situation.
To actively listen while interviewing.
As well as helping you, this also shows people that what they say is important and that their knowledge is valued
To unravel complex, messy situations especially those that involve lots of different stakeholders.
This can help you resolve conflicts between different groups of people and get a common output from a divergent group.
To make sure that important factors are not left out of your analysis of complex situations.
This can help you avoid ‘unintended’ consequences of development interventions and give you more confidence when it comes to action.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 85
Steps How do I begin? Write down in the middle of the page the central idea you wish to explore. Leave plenty of space all around it What do I do next? Let the interviewee or group explain how they understand the central theme. As they speak, write down the ideas they associate with the central theme using key words or phrases. Connect each idea to the central theme with a line. Prompt the exploration of the theme with open questions like: Who? What? Why? When? Where? This will also help to encourage the flow of conversation, if you are having difficulties. How do I keep going? One at a time, explore each idea that is associated with the central theme (the first ‘layer’). Ask the who? what? and why? Questions again. When you find more ideas emerging write them near their appropriate (single layer) idea and link the new ideas with a straight line. Do not link these second layer ideas directly to the central theme. When do I stop? You can continue to mind map through third and fourth layers of ideas. Mind mapping could carry on forever! Deciding when to stop will depend on whether you are getting any more useful information for your defined purpose. Also, the diagram may have become too crowded. Mind maps lose their value when they are hard to read or interpret.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 86
Examples of mind maps
Caution On ease of use Beware! Mind mapping is not always quick. It takes time to handle disagreements, but this is valuable time as it builds common understanding. It also takes time for you to develop the skills needed to capture each idea as a keyword(s) or short phrases. The layout of the diagram is also important. It should not look messy or confused. On Drawing Beware! The lines that indicate connection or association between ideas must not become confused with cause and effect relationships. If key words and short phrases are too general, they can lose their meaning. The layout of all the ideas and their connections can become very messy. Information Internet sites : - http//www.cpsc.ucalgary.ca/~couprie/623/ssmfinal.html - http//www.csu.edu.au/ci//vol11/Andrew.Finegan/paper.html - http//www-tec.open.ac.uk/ccc/sysdis.html - Checkland P. and Schooles J. (1990): Soft Systems Methodology in Action, Wiley & Sons, Chichester UK, pp 13-27.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 87
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.3 TOOLS FOR STAKEHOLDER ANALYSIS
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 88
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.3.1 Actor Influence Analysis Importance One way to analyse actors is by assessing their influence in various situations: their potential contribution to activities and results of a project, their access to benefits, their role in a partnership, their role in an advocacy programme, their position in social networks, etc. Knowing the influence of actors helps identify risks linked to isolated actors or to potential coalitions. It also helps plan and mobilize relevant actors to ensure success of a project. Influence is the extent to which the actor is able to persuade or coerce others into decision-making and/or implementation of actions. There are different sources of power and influence, some more difficult to assess than others: • Administrative or legal hierarchy (command and control, budget holders); • Authority of leadership (charisma, political); • Control of strategic resources for the project (e.g. suppliers of hardware or other inputs); • Possession of specialist knowledge; • Negotiation position (strength in relation to other stakeholders in the Project. There are many types of matrices used to assess actors’ influence. Actors’ influence analysis is generally conducted in the context of a research and / or development project,, to help develop its stakeholder involvement strategy. In innovation assessments, it can help • Analyze actors’ potential for action and how they can be involved in the assessment • Analyze their role in innovation processes • Analyze their potential role in the future, as part of action planning In innovation assessments influence analysis can be very useful, for example: • To see how project-based innovation processes were influenced by some actors • To assess the influence of actors during an actor network analysis • To identify actors to be targeted during the planning of an advocacy programme Types of matrix analysis For example: • to classify actors: Actors Typology Matrix • during the planning of the assessment: Power and Interest Matrix • to analyze the source of actors’ power: Power Analysis Grid • to analyze actors’ influence over an issue: Stakeholder Influence Mapping • to analyze actors’ relationships over an issue: Conflict and Partnership Matrix • during the planning of advocacy: Political Process Mapping and Political Context Mapping • during action planning: Policy Attractiveness Vs Stakeholder Capability Grid and Readiness for Change Matrix
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 89
Example: Political Process Mapping
Example: Political Context Mapping
Source: ODI, Civil Society Partnerships Programme
Example: Power and Interest Matrix
Examples: Actors' Typology
Example: Power Analysis Grid
Example: Stakeholder Influence Mapping
Adapted from James Mayers and Sonja Vermeulen (2005): Stakeholder influence mapping; IIED
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 90
Example: Conflict and Partnership Matrix
Example: Policy Attractiveness Vs Stakeholder Capability Grid
Example: Readiness for Change Matrix
Steps Clarify the issue and select a type of analysis An analysis of actors’ influence implies that those actors have already been identified. Stakeholder Identification and Stakeholder Objectives Matrices can be used to that effect. Clarifying the issue of actors’ influence means defining the context: • Deciding what kind of influence and influence on what? Is it influence on the assessment process? Is it influence on policy decisions? • Deciding what is the analysis conducted for? What is its objective? Once this is done, a specific tool can be selected among those presented Determine actor level of influence Before deciding on actors’ influence, these preparatory steps can help: • Make a list of all actors and write the name of each one of them on a post-it note or index card • If there are too many actors, you can reduce their number by ranking each of them on a scale of one to five and keep only those who rank higher. Deciding an actor’s level of influence, particularly on complex issues, can never be completely objective. Issues that make it difficult are: • Transparency of decision processes: there are formal and informal decision processes. Decisions are not always made as we think they are. • Complexity of processes: on any issue there may be several influences from various sources, and it is difficult to attribute them to only one actor.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 91
The best way is to map actors’ influence by looking at it from different perspectives. This means conducting the analysis collectively and seeking consensus. Determining actors’ level of influence, with most visual tools, is done concurrently with their positioning on the matrices and grids. The exercise combines therefore production and analysis of information. Deciding on actors’ influence can be done using several group techniques, depending on the tool used: • Open discussion between assessment team members • Focus group discussion, with some actors knowledgeable about the issue Group ranking / scoring of actors’ influence Locate each actor on the plot Locating each actor means plotting them against the variables of the matrix. This goes along with explaining why they were located there. This can be done by the assessment team alone, or by a larger group of stakeholders in a workshop fashion, with the team acting as facilitators.
Caution Like all visual tools, those used for influence mapping is useful to compare and contrast the information available about different actors. However a visual representation carries some risks: • When done collectively, it carries the risk of reflecting “group think”, that is a desire of group members to reduce conflict and achieve harmony. The analysis becomes therefore biased by this search for consensus. This is a risk especially in a politically sensitive topic like power analysis. • A visual representation is just that: a representation. It helps understand but it is not self explanatory. It should therefore be supported by an argumentation, helped by a documentation of the process and a recording of the group discussions. Information - Robert Nash, Alan Hudson and Cecilia Luttrell (2006): Mapping Political Context: A Toolkit for Civil Society Organisations, ODI, London; available online at: http://www.odi.org.uk/sites/odi.org.uk/files/odi-assets/publications-opinion-files/186.pdf - John M. Bryson (2004): What to do when stakeholders matter –
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 92
Stakeholder Identification and Analysis Techniques; Public Management Review, Vol. 6 Issue 1, pp 21-53; available online at: http://www.hhh.umn.edu/people/jmbryson/pdf/stakeholder_identification_analysis_techniques.pdf - Anne Marie Groot (nd): Stakeholder Matrices – Guidelines, ICRA learning resources; available online at: http://www.icra-edu.org/objects/anglolearn/Stakeholder_Matrices-Guidelines%28new%29.pdf - Knowledge Co-Creation Portal Multi-Stakeholder Processes - Tools & Methods, University of Wageningen; http://www.wageningenportals.nl/msp/tools http://www.wageningenportals.nl/msp/tools
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 93
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.3.2 Competing Values Framework Importance The Competing Values Framework is a model developed originally to identify indicators of organisational performance. It describes the approaches to thinking, behaving and organising associated with human activities in an organisation, that determine its effectiveness. It can help assess various domains associated and/or influenced by organisational culture: • Leadership roles and competencies • Innovativeness • Workers’ health • Information processing • Etc… It is therefore particularly relevant for the assessment of organisational impact on innovation. The Competing Values Framework distinguishes organisational cultures according to two dimensions: • One dimension assessing flexibility along a continuum going from an emphasis on organisational flexibility, dynamism and discretion to an emphasis on stability, order, and control. • Another dimension differentiating an internal orientation with a focus on integration, collaboration, and unity from an external orientation with a focus on differentiation, competition, and rivalry. Using these two dimensions, the Competing Values Framework identifies 4 models describing the overall cultural profile of the organization and its dominant characteristic traits. The Clan Model is a human relations, “family” type, model. Its main characteristics are team building & collaboration, cooperation, relationship development, shared values, harmonious work environment, development of personal and collective competencies. “The clan culture …is typified by a friendly place to work where people share a lot of themselves. It is like an extended family. Leaders are thought of as mentors and perhaps even as parent figures. The organization is held together by loyalty and tradition. Commitment is high. The organization emphasizes the long-term benefit of individual development, with high cohesion and morale being important. Success is defined in terms of internal climate and concern for people. The organization places a premium on teamwork, participation, and consensus” (Kim S. Cameron & Robert E. Quinn).
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 94
The Hierarchy Model is a model focused on internal process. Its focus is on continuous improvement, integrity, organizational learning “The organizational culture compatible with this form … is characterized by a formalized and structured place to work. Procedures govern what people do. Effective leaders are good coordinators and organizers. Maintaining a smooth running organization is important. The long-term concerns of the organization are stability, predictability, and efficiency. Formal rules and policies hold the organization together… Large organizations and government agencies are generally dominated by a hierarchy culture, as evidenced by large numbers of standardized procedures, multiple hierarchical levels, and an emphasis on rule reinforcement… Key values center on maintaining efficient, reliable, fast, smooth-flowing production” (Kim S. Cameron & Robert E. Quinn). The Adhocracy Model is an open systems model. Its focus is on creativity, shared vision, energetic and positively affirming culture. It is characterized as a dynamic, entrepreneurial, and creative workplace with a high-risk orientation and a commitment to experimentation and innovation “The root of the word adhocracy is ad hoc—implying something temporary, specialized, and dynamic… Adhocracies are similarly temporary. They … can reconfigure themselves rapidly when new circumstances arise. A major goal of an adhocracy is to foster adaptability, flexibility, and creativity where uncertainty, ambiguity, and information overload are typical. (A typical example of adhocracy is a consulting organization). An important challenge for these organizations is to produce innovative products and services and to adapt quickly to new opportunities. Unlike markets or hierarchies, adhocracies do not have centralized power or authority relationships. Instead, power flows from individual to individual or from task team to task team, depending on what problem is being addressed at the time. Emphasis on individuality, risk taking, and anticipating the future is high as almost everyone in an adhocracy becomes involved with production, clients, research and development, and other matters…Each different client demand in a consulting firm is treated as an independent project, and a temporary organizational design is set up to accomplish the task. When the project ends, the structure disintegrates” (Kim S. Cameron & Robert E. Quinn ). The Market Model is a rational goal model. Its basis is rewarding high performers, transforming problems into opportunities for abundance, creating clear goals for success and encouraging their pursuit. The term market in this model refers not to the marketing function but to a type of organization that functions as a market itself. It is oriented toward the external environment instead of internal affairs. It is focused on transactions with (mainly) external constituencies such as suppliers, customers, contractors, licensees, unions, and regulators. And unlike a hierarchy, where internal control is maintained by rules, specialized jobs, and centralized decisions … The core values that dominate market-type organizations are competitiveness and productivity. The basic assumptions in a market culture are that the external environment is not benign but hostile, consumers are choosy and interested in value, the organization is in the business of increasing its competitive position, and the major task of management is to drive the organization toward productivity, results, and profits. It is assumed that a clear purpose and an aggressive strategy lead to productivity and profitability. A market culture… is a results-oriented workplace. Leaders are hard-driving producers and competitors… They are tough and demanding. The glue that holds the organization together is an emphasis on winning. The long-term concern is on competitive actions and achieving stretch goals Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 95
and targets. Success is defined in terms of market share and penetration. Outpacing the competition and market leadership are important. The CVF in short…
CVF and Innovation By revealing the main characteristics of organisational culture, the CVF can be very useful in an institutional analysis of innovation. Organizational culture -also covering the shared values, regulations, assumptions, beliefs, practices, formal and informal rules and attitudes - is one of the key elements in both enhancing and inhibiting innovation. Change is hampered by a legacy of established behaviors, with an over-reliance on rules and a reinforcement of old patterns of behavior. Findings of analyses of organisational culture provide evidence about this proposition: adhocratic cultures seem to enhance innovation; hierarchical cultures seem to inhibit it. Dimensions of CVF The organizational culture assessment in the Competing Values Frameworks is usually done along 5 dimensions: 1. Dominant characteristics: what best characterizes the organization 2. Organizational leadership: what best characterizes the managers 3. Organization glue: what holds the organization together 4. Strategic emphases: what is important for the organization 5. Criteria of success: how is performance evaluated Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 96
Steps Select an assessment methodology Although the Competing Values Framework has been designed for a quantitative organizational culture assessment , it can also be assessed using qualitative tools The quantitative assessment is done with the help of a set of questions investigating each of the 6 dimensions, and the results reported on the CVF model The qualitative assessment can be done using available information on the organizations, supplemented by individual or group interviews with their staff, using open ended interview guidelines. Alternatively the group participating in the assessment can adapt the interviewing instrument and fill in directly the model in a participatory fashion, as in other visual techniques (Venn Diagramme, Ranking and Scoring Matrices, etc.). The assessment process can also combine both by measuring quantitatively the cultural dimensions and qualitative methods including stories, incidents, and symbols that represent the unmeasurable ambience of the organization. The CVF is both a diagnostic and a planning tool for organizations willing to engage in a process of change. When planning is envisaged as a follow-up to diagnostic, the value statements in the assessment process are answered in two parts: “current situation” and “preferred situation”. The “preferred situation” is then used as a reference point, and the plan is developed to reach it. Only the diagnostic assessment is described here, ie, the “current situation Select the organisations In the use of CVF for an institutional assessment of innovation, organizations are selected on the basis of their perceived importance in innovation processes. Their selection can follow on a stakeholder identification and analysis process, as is generally done in innovation assessments. The selection can then follow several stages: • First a purposeful sample from the stakeholder analysis • Then a smaller sample • Then a selection of staff from various levels of the organisational structure to conduct the interviews Assess each dimension Example 1: Dominant characteristics: what best characterizes the organization
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 97
Example 2: Organizational leadership: what best characterizes the managers
Example 3: Organization glue: what holds the organization together
Example 4: Strategic emphases: what is important for the organization
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 98
Example 5: Criteria of success: how is performance rewarded
Reporting of interviews The answers can be reported using several methods 1. A Likert Scale method: Interviewees rank their level of agreement with each value statement from 1 (strongly disagree) to 5 (strongly agree) 2. A rank sum assessment: Interviewees are asked to distribute 100 points among the value statements according to their relative importance. Example: Statement A: 30; Statement B: 40; Statement C: 10; Statement D: 20. 3. The interviews can also be conducted informally, in an open ended fashion, using the questions as a guide. Notes are taken, followed by a content analysis and summaries. Results are then discussed, perhaps triangulated with other sources of information and the final score reported on the model. To help prepare the plotting of answers, participants to the assessment can first give their answers individually or collectively in the following form: Statements
Assessment 1
2
3
4
5
What evidence supports your assessment?
Others comments, observations on the implications of the assessment
A. The organisation is a very personal place. It is a lot like an extended family. People seem to share a lot among themselves. B. The organisation is a very dynamic and entrepreneurial place. People are willing to take risks. C. The organisation is a very formalized and structured place. Bureaucratic procedures generally govern what people do. D. The organisation is very production oriented. A major concern is with getting the job done. People aren’t very personally involved. etc….. Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 99
Plotting the responses The final scores are finally aggregated by adding up all first, second, third and fourth questions and working out the average
Each of the alternative scores represent a culture type: • Score A: Clan culture • Score B: Adhocracy culture • Score C: Market culture • Score D: Hierarchy culture The scores are plotted on the two diagonal lines crossing the 4 quadrants representing the 4 culture types. Reporting the results in the CVF quadrants
The results are reported on a two axis diagramme on which the scores are plotted
Examples: Average Culture Profiles from various industry groups
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 100
Example : Current & Preferred cultures
Interpreting the results – Dominant culture The type and strength of culture that dominates the organizations: the quadrant in which scores are the highest indicates the dominant culture in the organization. The assessment team can collectively interpret the results for each organization and across all organizations assessed.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 101
These questions may help: • With the type of culture come the values, leadership styles basic assumptions, styles, and values that predominate. They strongly influence organizational success. Does this culture match the demands of the competitive environment? Does it foster innovation? • Are different aspects of organisational culture aligned on each other? Do different levels of the organisation adhere to the same values? • What challenges face the organisation ? What aspects of its culture are important to nurture? What aspects need changing? Culture, Value Drivers, Leadership and Effectiveness
Caution The “models” of organisational cultures in the CVF, like any other models developed in other approaches, are not limitative. They cannot cover the diversity and fluidity of notions like culture. Instead they provide. No model of organisational culture can be considered “unique” or “best” for innovation. There is an element of all models in an organisation, with a “tension” between the dominant model and the other models. “It takes both leadership and management to strengthen, maintain, change, or create a culture in any of the quadrants. Leaders who are not managers are bound to fail, just as managers who are not leaders are bound to fail. Change without stability is chaos. Innovation without productivity is pie in the sky…. All four culture types (and the management competencies that accompany them) are valuable and necessary. None is better or worse than the others” . The CVF is both a diagnostic and a planning tool for organizations willing to engage in a process of change. When planning is envisaged as a follow-up to diagnostic, the value statements in the Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 102
assessment process are answered in two parts: “current situation” and “preferred situation”. The “preferred situation” is then used as a reference point, and the plan is developed to reach it. Information - Tianyuan Yu & Nengquan Wu: A Review of Study on the Competing Values Framework; International Journal of Business and Management Vol. 4, No. 7 - Kim S. Cameron & Robert E. Quinn (2006): Diagnosing and Changing Organizational Culture Based on the Competing Values Framework; Jossey-Bass, San Francisco - Jung, T, T Scott, HTO Davies, P Bower, D Whalley, R McNally, and R Mannion (2007), Instruments for the Exploration of Organisational Culture, Working Paper, Available at http://www.scothub.org/culture/instruments.html
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 103
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.3.3 Innovation Capability Assessment Importance Innovation capability is all the characteristics that support and facilitate innovation strategies. It covers capability to invent, capability to innovate and capability to improve existing technology. In market oriented organisations, innovation capabilities are what allow them to obtain and sustain competitive advantage In innovation support organisations (rural advisory services, research organisations, credit services, etc…), these capabilities are what enables them to reach out and adapt to the needs of their clients and partners to facilitate innovation. Innovation capability can be assessed at different levels: • System level: local, regional, national • Organisational level • Individual level Innovation capability can be assessed quantitatively (using metrics) or qualitatively (making group assessments). The following guidelines are more suited to a qualitative assessment at the organisational level Steps Identify the capability factors The factors that make up innovation capability are those factors that enhance innovativeness and facilitate innovation processes. The most common are: • Innovation Space • Leadership • Learning Network • Implementation Discuss and assess each innovation factor Innovation Space Innovation space relates to the conditions that constrain or enable innovation within an organisation: • Organisational procedures: are they flexible enough to allow quick decision making? Do they allow collaboration within and outside the organisation? • Freedom for initiative and experimentation: do people have room for manœuvre? Are they allowed / encouraged to take initiative? Is initiative rewarded? • Spirit of entrepreneurship: how conservative is the organisation? Leadership Leadership refers to the existence of people within the organisation who are able to promote creativity and innovation by, for example: Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 104
• • •
Encouraging people to take initiative Leveraging support for them Coaching them
Learning Network Learning network relates to the learning environment in which people work and the existence of collaborative learning processes. For example: • Is there collaboration with people in other functions within the organisation? • Are people able to mobilise work group support? • Are they part of professional –formal or informal- networks? • Do they participate much in outside events to acquire new knowledge and skills? Implementation Even when the creative and inventive stages of innovation exist, the absence, or weakness, of innovation process management can prevent desired returns. Implementation is the ability to concretize and create new value. It is the materialization of creativity into real innovation, to turn ideas into products, processes, and services that can be of benefit to the organization and the wider community. *** There are no definitive sets of innovation capability factors. These 4 factors can be reviewed and validated by the group conducting the assessment. Other factors that can be considered are: • Staff competencies and training / development • Importance of innovation in the organization’s strategy • Existing performance measures • Communication strategy and infrastructure • Sufficient organizational resources • Knowledge of “clients” and market needs The assessment of the innovation factors can be done through different methods: • Individual interviews with the help of guidelines • Group interviews using visual methods • Review of documentation, reports, etc… • Role plays • Story telling • Etc… Report the result of the assessment Once each factor has been discussed, it is assessed in relation to how positively it ranks in the organization. For example: • How much space is available for innovation in the organization? • How much is leadership supportive of innovation? • How involved are members of the organization involved in learning activities with others inside and outside the organization? • How good is the organization in translating ideas into practice? Assessment is mainly through group discussions. It is summarized in a score along a 1 to 5 scale, with a group statement. For example, some alternative statements in the case of innovation space: •
1: Very bad: The organization is totally closed to new things, it has strict rules and regulations and allows no freedom to improvise. It sticks to standard practice, and is not reactive to new needs and demands from outside Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 105
•
2. Bad: The organization is not open to new idea; it rather values conformism and adherence to standard practices and staff are encouraged not take risks.
•
3. Unclear: It is difficult to assess the organization for different reasons: we don’t have the information needed; on the face of it, the organization is open to innovation, but we are not sure its performance is good.
•
4. Good: The organization is fairly open to new ideas. There are rules, but they are never obstacles, if managers are convinced of the relevance of new ideas, they find a way around rules.
•
5 Very good: The organization is very open to new things, staff are encouraged to experiment; decision is quick, and new ideas are easily accepted.
Once a consensus is reached on a scored for each factor, the result is reported on a kite diagramme, which helps visualize the general shape of the assessment. This visualization serves two purposes: • To draw conclusions on the assessment of each organization • To compare and contrast several organizations Example
Interpret the results of the assessment There are no standard methods for interpreting innovation capability. The following hypotheses, derived from several innovation research studies, can help the group interpret the results within and across organizations: • Staff empowerment and participation is conducive to a greater innovation capability ? • Collaborative learning processes, and group work, strengthen organizational learning and innovation • Younger staff and smaller organizational size lead to more innovative practices. • Business and consultancy-type of organizations have more innovative practices. • Innovative organizations expand their activities and grow significantly faster than other organizations.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 106
Caution On the notion of capability Having a strong innovation capability does not mean being innovative. All capability factors can be present without organizations being very innovative. It is important to keep two points in mind: • Organizations work in a changing environment that is constraining them • Assessments are a snapshot at a specific time in the life of an organization. In a changing environment, it is essential for organizations to continuously build and improve innovation capabilities that are holistic and process oriented to deal with changes. On the use of the results The Innovation Capability Assessment is both a diagnostic and a planning tool for organizations willing to engage in a process of change. When planning is envisaged as a follow-up to diagnostic, the results of the assessment can be used an in input to conduct a participatory strategic planning of the organization to make it more geared towards innovation. In this case, this qualitative assessment can be deepened and improved through a wider participation. Information - Tom Freeman, (1999),"Assessing the innovation capacity of the consortium: an evaluation of the CAM-I cost management systems program", Journal of Knowledge Management, Vol. 3 Iss: 1 pp. 61 – 65; Available at: http://dx.doi.org/10.1108/13673279910259402 - Andy Hall, Lynn Mytelka and Banji Oyeyinka (2006): Concepts and guidelines for diagnostic assessments of agricultural innovation capacity; Working Paper 17; United Nations University – MERIT; Available at: http://www.merit.unu.edu/publications/wppdf/2006/wp2006-017.pdf - The Conference Board of Canada : The General Innovation Skills Aptitude Test (GISAT); Available at: http://www.conferenceboard.ca/topics/education/learning-tools.aspx
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 107
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.3.4 Organisational Linkage Matrix Importance A linkage matrix can show the relationships between organisations. Looking at the interaction between organisations often provides important insight in why service providers and users are not very well organised. With this information, it is possible to suggest ways in which relationships could be improved. Steps Identify relevant organisations Start by identifying all the organisations who affect, or are affected by the issue being considered. Draw a matrix of organisation against organisation. Develop criteria for assessing links Discuss with key informants or focus groups what criteria they would use to judge their relationships. Do this by asking questions like: • Who do you have contact with? • How do you make contact? • What are you in contact about?”
Fill out the linkage matrix Using the criteria they have developed, discuss the value they give each existing relationship. Simple scores of bad, moderate, good, and very good can be used. The value given to the linkage is written in the appropriate cell of the matrix along with the reason for giving that score.
Organisations A B C
A
B score (reason)
C
Others
score (reason) score (reason)
Others
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 108
The matrix facilitates discussion between organisations on how weak linkages can be strengthened and how new linkages might be established. Example of Linkage matrix
Caution On Linkages Weak linkages are not necessarily bad. Sometimes users appreciate weak linkages but not the strong ones. This will depend on their expectations and needs. On Balance Often only the problem areas are indicated in the matrix. However, we can learn from good linkages as well as problem linkages. Ideas from good relationships can be used to help improve the poorer ones. It is important to explore strong and weak linkages. Information - Engel, P. and Salomon, M. (1997): Facilitating Innovation for Development. A RAAKS Resource box. Amsterdam: KIT publications - ICRA (1998): A Participatory Systems Analysis of the Termite Situation in West Wollega, Oromia region, Ethiopia. Working Document Series 68. ICRA, Wageningen, The Netherlands - Ramirez, R. (1997): Understanding Farmers’ Communication Networks: combining PRA with agricultural knowledge systems analysis. Gatekeeper Series no.66. London: IIED.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 109
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.3.5 Stakeholder Identification Matrix
Importance
Steps List all potential stakeholders Working from secondary data, TOR's and rich pictures: list out all the persons, groups, organisations who might be potential stakeholders. Working from interviews with key informants add to this lists other actors who play a significant role in the system. Try to think beyond the classical roles of research, extension and farming.
Differentiate and group stakeholders Working from the long list of potential stakeholders try to identify groups of stakeholders. This could be through pooling resources together or the way they talk about the issue of concern. Sometimes it might be necessary to differentiate subgroups within a stakeholder group. For example "Farmers" or “communities” are not homogeneous entities but can be differentiated based on gender, age, tribe or cast, or economic class.
Brainstorm who are the stakeholders • Brainstorm with key informants or focus groups and discuss the importance of each stakeholder. Generate as many ideas as possible, don’t try to limit yourself too much in this phase.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 110
•
The ideas can be clarified but not criticised in these brainstorming sessions. They are all important. The session should identify which stakeholders are considered important and why.
EXAMPLE of Brainstorming who are the stakeholders
Stakeholder identification table: A study of farmers’ and scientists’ perspectives on soil classification in Kenya. (source: ICRA 1998)
Example of Stakeholder Identification Matrix
Caution On iteration After all research, including stakeholder analysis, is an iterative process. Remember that at this stage, you do not have all the answers. You can always come back at a later stage and review the issues this tool is dealing with.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 111
On inclusion Including or excluding a stakeholder is a matter of careful consideration. In this early stage the team should not focus too narrowly. Usually the terms of reference indicate some of the actors. The study of secondary data will usually also lead to the identification of some new stakeholders. Try to include the stakeholders as much as possible in the assessment. Information - Salomon, M. & Engel, P. (1997): Networking for Innovation: Windows and Tools. In: Engel, P. & - Salomon, M. Facilitating Innovations for Development. Amsterdam: KIT, STOAS and CTA. - ICRA, 1998: Participatory Approach to Soil Mapping and Management. Working Document Series 70. ICRA The Netherlands/KARI Nairobi, Kenya
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 112
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.3.6 Stakeholder Objectives Analysis Importance A participatory objectives analysis can help stakeholders explore their differences and common ground. The reasons they put forward to justify the importance of their objectives or interests provides insights into their own values and principles. By sharing this, stakeholders can start to identify the possibilities and constraints for collective action. Often conflicts found in research and development projects are related to the conflicting objectives among stakeholders. Analysis can help identify these conflicts and find ways to resolve them. Identifying shared objectives can build collective action. Steps Clarify key stakeholders From the list of key stakeholders, identify and interview key informants from each group to determine their objectives or interests. Agree among stakeholders what they see as their objectives or interests. This will depend on their own perceptions. Clarification can be sought from organisations documents, secondary data or the TOR.
Form pairs of stakeholders From the list of stakeholders and their objectives, organise the stakeholders into pairs. They can now be compared to see if they have common activities or can share resources.
Identify shared and conflicting objectives Use key informants or focus groups to represent the stakeholders. Brainstorm on their objectives and interest. Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 113
Identify the stakeholders with conflicting and shared objectives. Use the objective table to summarise the outputs of these discussions. In these discussions, stakeholders should agree on the potential and constraints for collective action.
EXAMPLE: Objectives matrix for improved soil management
A study of farmers’ and scientists’ perspectives on soil classification in Kenya. (ICRA, 1998). Caution On time planning It is very difficult to predict how much time will be needed for an objectives analysis. Participants in the discussion should be warned at the beginning. They will need to be flexible. Allow more time than you think for this work. On brainstorming Be careful if there are only a few representatives of each pair of stakeholder taking part in discussions. They may feel vulnerable and become defensive over issues of conflict. When emotions run high, remind participants that the discussion is only meant to identify conflicts, not resolve them. Information - Salomon, M. & Engel, P. (1997). Networking for Innovation: Windows and Tools. In: Engel, P. & Salomon, M. Facilitating Innovations for Development. Amsterdam: KIT, STOAS and CTA. - ICRA,1998. Participatory Approach to Soil Mapping and Management. Working Document Series 70. ICRA The Netherlands/KARI Nairobi, Kenya).
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 114
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.3.7 Venn Diagramme Importance A Venn Diagramme is a tool used to identify and analyse institutions, relationships, using symbols or circles to represent individuals, groups or organizations and their perceived importance by a community or a given group of people. The size of the symbols/objects or circles indicates their perceived importance The positioning – overlapping, touching or separate – indicates their degree of interactions. The diagramme can thus illustrate the relationships between several different institutions and the community – and provide entry point for discussing ways of improving relationships between such institutions. Steps List the organisations The first step is to ask the group what are the main outside organisations working with them. For example, what organisations work with the group on anything to do with Cotton, or Soybeans, or soil fertility management, etc… Once this is done, discuss how important for them is each organisation, in general and in relation to each other. Use an object (round piece of paper, paper plate, etc… ) to represent that importance; the most important, the bigger the object. Write the name of the organisation on it. Objects can also be of a different color. Visualise the relationships The second step is to visualise the relationships between the organisations and the group, and between the organisations themselves. Draw a large circle on the ground, representing the village for example, then put a circle representing the group at the centre. Ask the group then to place each organisation close or near the group circle, to represent how closely they work together. Ask them also to place them close or far from each other, to represent how closely they work together. For example: • Circles removed from each other: No perceived collaboration between them • Circles touching each other: Information exchange between them • Circles overlapping a little: Some collaboration between them • Circles strongly overlapping: Strong collaboration between them.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 115
Visualise the relationships - Examples
Source: Guide pratique de suivi-évaluation des projets de développement rural FIDA 2003
Example of a Venn diagram showing the relationships between stakeholders in an information project http://www.ink4dev.net/?q=node/422
Discuss the meaning of the diagramme The process of visualising the sometimes complex relationships can be very long, as not all group members may not agree, or simply have the information, to decide. The circles may be placed, removed and placed again at different places before agreement is reached. In this process, it is important not to lose sight of the purpose of the exercise. These key questions may help the team to lead the process: • Which organisations/institutions/groups are working in or with the community? • Which institutions/groups do the villagers regard as most important, and why? • Which organisations are addressing the issue being investigated (innovation processes, food security, soil fertility, etc…)? • Which organisations work together? • Are there organisations that address the needs of specific groups (women, the young, the landless, etc…)? • Are some particular groups excluded from being members of or receiving services from certain organisations? The discussion around the meaning of a Venn Diagramme can be as deep and extended as the team wants it to be, depending on their objective. For example, further visualisation on the diagramme can be on characterising the relationships between the organisations. For example: • What is the object of the relationship (credit, control, agricultural advice, etc…); • What is the nature of the relationship (forced by the organisation, essential, important, not so important, etc…); • What is the intensity of the relationship (regular, exceptional, etc…); • What is the quality of the relationship (mutually profitable, competitive, difficult, unilateral...).
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 116
Caution On mixed groups Like with all tools and techniques used with groups, the users must be very careful about differences within the group: men, women, young, poor, big and small farmers, poor and rich people etc… Each of these groups may have a different perception of the importance of organisations and their relationships. For this reason, it is best to work with each group separately. On managing group process Even when groups are relatively homogeneous, there are always differences in how articulate people are. Dominance by a few people may strongly influence the results, by emphasizing some organisations or relationships that represent the personal views and interests of only a few persons. One of the most important role of the team facilitating the Venn diagramming session is to observe the dynamics underway and make sure that everybody is participating. Information - Diagramme de Venn/Chapati (MARP-ASEG); PSEP, WUR; Available at: http://portals.wi.wur.nl/ppmefr/?Diagramme%20de%20Venn/Chapati%20%28MARP-ASEG%29 - Venn Diagramme on Institutions: An Example from a Participatory Household Food Security and Nutrition Project in Ethiopia; FAO Field Tools for Participation; Available at: http://www.fao.org/Participation/tools/venndiagram.html - Silvan Widler: Venn Diagrams; SSWM; Available at: http://www.sswm.info/category/planning-process-tools/decision-making/decision-makingtools/situation-and-problem-analys-4
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 117
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.4 TOOLS FOR SURVEYS
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 118
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.4.1 Secondary Data Analysis Importance Improve our understanding In order to improve and expand our knowledge, we must begin where others have ended and not repeat what has already been investigated. Therefore, an essential part of doing research consists of studying the results of other researchers and to apply these in order to produce new results. All information (in databases and publications) which already exists, whether published or unpublished, is considered as secondary data. Relate to existing knowledge The use of secondary data is important, because it enables us to: • relate existing knowledge to research questions and to basic hypotheses; • have information at our disposal which we could never acquire ourselves. We simply do not have the time or the means to gather this information, especially in case of short-term studies. • understand the origin of and the reasons behind the terms of references. Many advantages Using secondary data well has the following advantages for a survey: • avoids repetition or research of subjects which are already investigated; • provides answers and enables us to elaborate hypotheses; • inspires us to produce new hypotheses; • is more cost effective and provides for a better use of time than collecting primary data; • is quickly available; • enables us to make comparisons at different levels; • is complementary to collecting information. Steps Define the type of information needed Close reading of the terms of reference enables you to identify the subjects to which the collection of secondary data should be oriented. A clear definition of the study objectives enables you to know exactly what you want to do and why. There are qualitative and quantitative secondary data. But which are the most useful secondary data?
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 119
- General topics
- Specific topics
- Visual material All visual material containing non-written information is very useful. We should, however, bear in mind that this material has not been designed nor elaborated for the specific needs of our study. Unfortunately, this type of material is not always available.
Where to find the data?
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 120
Screen the documents When working in a team, it is not necessary (or possible) that everyone reads all the material. Dividing the work is much more efficient: • starting with the most recent documents; • using all available competence in the team; • according to each individual focus of interest. With regard to written documents (reports, articles, books) a first selection can be made on the basis of: • Key-words; • Tables of content; • Index; • Summaries. Through the study of aerial photographs, the team can quickly assess the shapes and dimensions of the landscape, the habitat and the size of forests, grazing areas and cultivated areas. Organize documentation system The objectives of organizing a documentation system are to: 1) Make information accessible and share it 2) Maintain a common level of knowledge among the members of the team 3) Retrieve the original source of any information Make sure you keep the list of consulted and selected documents up-to-date; mark the most important documents; it is not useful to keep too many documents concerning the same subject. Reserve and plan special periods to work on documentation and organise internal lectures, so that every member is able to inform the entire team about the results of his readings. Provide for a budget for purchasing and copying documents (however, do not copy entire documents, that would be too heavy. You must still be able to retrieve the information and quote or mention it). Using the secondary data You should be able, before going to the field, to have an established idea about the region under study, in terms of: • Description of the local conditions • Structural characteristics • Functional characteristics • Trends of development and opportunities, possible evolution • Comparisons between different regions - Presenting data Managing and presenting data according to topics.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 121
- Linking data • Establish links between documents in which the same topics are discussed. • Identify which differences / analogies exist between the selected data and verify to which extent the results can be compared with each other • Establish hypotheses about the differences in point of view.
- Reading notes
Linking secondary data to field work Objective: to establish a link between the secondary data and observations. Use of data through analogy, during the research: elaborate conclusions which are formulated partly on the basis results obtained by others, mixed with our results. Look, in the approaches followed by other researchers or in other disciplines, for those things which could be used, through analogies, in other approaches. Search for a specific data / missing information in the collected data, in a publication concerning the area of study or a similar area and use this value/information while explaining that it concerns an approximate value which needs to be confirmed, but which seems acceptable. Visiting places, which have been subject to a thorough analysis, enables us, by observing and by interviewing people, to improve our understanding of what exactly the writer wanted to say. Once the analysis has been accomplished, the secondary data provide for elements which can be compared with the obtained results and which enable us to explain described trends.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 122
EXAMPLE of Socio-economic data
EXAMPLE of Farming system data
Caution On reliability of secondary data With regard to the subject of study, secondary data are only indirect measures of research problems / questions. Do not forget that you have no control over the collecting, processing or accurateness of the data. It may prove difficult to determine the reliability of some data; some data may be relatively old that is why one has to take some precautions during their analysis. Some of the more common problems are: • difficulties in comparing data from different sources; • generalisation of data is risky; • difficult interpretation of raw data; • too many data are difficult to digest; • language: impossibility of translating a high number of documents; • relevance and checking of selected criteria. On selecting the type of data Which type of secondary information should be gathered depends on the kind of study which will be executed.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 123
Most often, the need for information changes along with the research progress. However, using secondary sources remains necessary until the research has been accomplished (even during the final reporting). Some data will be used during the preparation and planning phase of the research process. Other data will not be of use when preparing the study and will only become useful or available in the field. Information - Agel, C. & al. (1981): Guide des enquêtes statistiques pour le suivi des opérations de développement rural. Coll. Méthodologie de la Planification. Ministère de la Coopération, Paris - Billaz, R. & Diawara, Y. (1981): Enquêtes en milieu rural sahélien. Coll. Techniques Vivantes, ACCT. PUF, Paris - Gastellu, J-M. (1978): Mais où sont donc... Cahiers des Sciences Humaines, vol XVII, n°1-2, ORSTOM, pp 3-11. - Gouet, J-P. (1978): L'élaboration d'un protocole d'enquête. ITFC, Paris. - Mettrick, H. (1994): Recherche Agricole orientée vers le développement: Le cours ICRA. Chapitres 6 et 7. CTA-ICRA: Wageningen. - Website in English: http://www.uea.ac.uk/care/elu/Research/Issues/Education - Website in French: http://www.unites.uqam.ca/sqsp/TEXTES/GUIDE - Murphy, J. & Sprey, L.H. (1984): Introduction aux enquêtes agricoles en Afrique. ILRI, Wageningen. - Ghiglione, R. & Matalon, B. (1991): Les enquêtes sociologiques - Théories et pratique. Collection U, Armand Colin.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 124
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.4.2 Semi Structured Interviews Importance “Semi-Structured Interview” is a broad concept associated with qualitative research, often contrasted with the “Structured Interview” typical of questionnaire surveys, in which the interview is structured in a sequence of standardized questions to be asked in the same way to all interviewees. Unlike formal ‘structured’ interviews in which all the questions are pre-determined and allow little diversion, the SSI is an informal, purposeful, flexible and open ended dialogue in which questions are meant to prompt discussion. It is sometimes called “informal” interview. It allows for flexible and sensitive questioning and helps the interviewer engage the interviewees. This allows the interview to be shaped by the interviewees’ own understandings as well as the interviewer's interests. Steps Define the themes and purpose of the SSI The structure of a semi structured interview is usually organized around an aide memoire or interview guidelines. The contents of these guidelines are not chosen haphazardly, they represent what the interviewer wishes to achieve. Two questions need therefore answering: • What do we want to achieve through this Semi Structured Interviews? What information do we want to collect? • Why do we want that information for? What is the role of the Semi Structured Interview in the process? Once these questions are answered, the assessment team will be in a better position to select the interview topics and identify the people that need interviewing. Identify the participants An SSI should be done with: • A broad cross-section of people -to cross-check views and opinions. • A relatively small sample –to avoid complexity of data analysis. The composition of the sample of interviewees will vary depending on what is exactly the purpose of the interview. In innovation assessment for example, interviewees can be all people involved –or who know about innovation processes: community leaders, elders, local officials, individual farmers, researchers, etc… It is equally important to achieve a good balance between men and women, old and young, rich and poor.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 125
Work out the SSI guidelines The SSI guidelines list all the topics that the interviewer wants to explore during the interview. They should work out well in advance (especially during preliminary interviews during the preparation phase). There is no standard guideline format. • The guidelines can be just a list of topics and sub-topics; • They can be formulated as general questions; • They can be a combination of the two. The interviewer can ask in different ways for different participants. Interview guides help researchers to focus an interview on the topics at hand without constraining them to a particular format. This freedom can help interviewers to tailor their questions to the interview context/situation, and to the people they are interviewing. Conduct the interview - Building rapport The general objective of the SSI is to understand the interviewees' point of view rather than make generalizations about their behaviour. Here are some points to keep in mind: • It is essential to build rapport with the interviewee before getting into serious conversation. For this: → Make an appointment for the visit and be punctual → Explain the purpose of the visit and why it is important → Be nice and friendly (greetings, introductions, etc...) - Asking questions • Ask ‘open’ questions and follow-up through probing: → Listen carefully, give feedback and crosscheck if in doubt → Do not interrupt interviewees → Questions are asked when the interviewer feels it is appropriate to ask them. They may be prepared questions or questions that occur to the interviewer during the interview. For probing, the usual six helpers -What? Why? When? Where? Who? How?- can serve as a guide. → Conduct the interview informally, like a conversation: “Tell me about…”, “You said a moment ago…can you tell me more?”. → The wording of questions will not necessarily be the same for all interviewees - Managing the interview • If the SSI is conducted in a group context, find ways to manage ‘gate-keepers’ who monopolize the discussion and guide the others; • Take notes accurately and sensitively not to interfere and/or slow down the interview. Ideally, have a note taker separate from the interviewer. • End the interview as naturally and in as friendly a manner as possible, to allow for follow-up visits. Caution The open-endedness of the SSI has both strengths and weaknesses. • The SSI requires highly skilled and/or trained interviewers and note-takers, and may be intrusive, especially if done hurriedly or with little tact. It requires considerable amounts of time and energy for information management and review. • The outcome of SSI depends greatly on the skills of the interviewer, notably his/her ability to build rapport with the interviewee and to think of questions during the interview, to keep Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 126
•
the interview relevant for the topics being discussed and, at the same time, not guiding the interviewee too much, which would defeat the purpose of the SSI. The personal and adaptive nature of the SSI makes it difficult to repeat exactly the same interview from one person to another. This is an important issue when the SSI is used to investigate issues across several locations and across different samples - interviewees may effectively be answering different questions. It is therefore difficult to analyze the results and generalize the findings from a SSI.
Information - Nigel Newton (nd): The use of semi-structured interviews in qualitative research: strengths and weaknesses; available online at: http://www.academia.edu/1561689/The_use_of_semistructured_interviews_in_qualitative_research_strengths_and_weaknesses - The Evaluation Toolbox: Semi-Structured Interview; available online at: http://evaluationtoolbox.net.au/index.php?option=com_content&view=article&id=31&Itemid=137 - Margaret C. Harrell, Melissa A. Bradley (2009): Semi-Structured Interviews and Focus Groups; Training Manual; The Rand Corporation; available online at: http://www.rand.org/content/dam/rand/pubs/technical_reports/2009/RAND_TR718.pdf - World Health Organization: Semi-Structured Interviews, in ow to Investigate the Use of Medicines by Consumers; Available online at: http://apps.who.int/medicinedocs/en/d/Js6169e/5.4.html#Js6169e.5.4
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 127
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.5 TOOLS FOR STRATEGY & DECISION MAKING
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 128
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.5.1 Action Planning Importance Innovation assessment conducted by a team of professionals, in a participatory fashion, is not aimed simply at producing new knowledge –which is only a first stage-, but also at initiating action to support local innovation. Once that vital preliminary stage is over, it is time to identify actions that move people into the results stage. An action plan guides the group for getting things done and realizing their goals and ultimately their visions. There are usually two ways to look at action plans: • As a product that formalizes commitments for action and provides a plan to guide it • As a learning process that allows stakeholders –through the exercise of planning- to arrive at a consensual definition of their common problem. There is no contradiction between the two if we consider Action Planning not as a rigid blueprint for action but as a continuous process. By leading a group of stakeholders through a process of collective reflection on a specific issue that is common to all, each of them can become aware of their capacities to change the situation. Steps Mobilize an Action Planning group Action planning is almost by definition a collective process in which participate those meant to carry out the action. A first step is therefore to mobilize key stakeholders. Identifying them will be helped by the stakeholder analyses carried out in the earlier stages of the innovation assessment. The action plan should build on work already done with those stakeholders because ideas have been already discussed and issues explored. It is essential that people taking part know what are the conditions of their engagement in the action planning exercise: financial, professional and otherwise so as not to create false expectations. Define the issue Ideally, action planning should start with, and follow on, a review of the situation, which will provide the “reference” point for planning. This review can be supported by any previous analysis conducted by the assessment team, for example: SWOT Analysis - Context Analysis - Innovation Timeline - Policy Analysis The outcome of this review is a clear statement of what it is that action planning is meant to change, and will be helped by a visioning exercise. Develop a common vision Developing a common vision is about laying the groundwork for the planning process. It answers simple questions: Where do we want to go? What is it that we want to change? How do we go about it? It is therefore also a consensus building step about the ambition of the Action Plan. Tools that can be used are: Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 129
• •
Visioning: a relatively simple procedure Scenario building: a more sophisticated procedure that requires more expertise and knowledge about driving forces.
Define the actions, roles and responsibilities There is no standard format for an Action Plan. Some are proposed as examples. The contents should however cover the following: • What are the group of activities to be conducted to achieve the objectives • What are the tasks / specific activities necessary for completion: seeking funding, technical recommendations, acquiring resources, making contacts, etc. • Which actors will be leading the activities? • What is the timeline and/or deadlines for activities. Dates can be precise or given as a sequence, ie, "immediate“, "later“, "near-term“, "long-term”. This allows more flexibility in the action planning process, but specific dates should be added later. Example of Action Plan format
Example of Action Plan format
Source: Adapted from IFAD (2003
Caution •
Avoid the temptation to overdo things or be too ambitious. Do not make the scope of the action plan too large. Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 130
•
The more concise and targeted the action plan, the more likely people will appropriate it and volunteer to get the work accomplished
Information - The Community Tool Box: Developing an Action Plan; available online at: http://ctb.ku.edu/en/tablecontents/sub_section_main_1089.aspx - Centers for Disease Control and Prevention: Build the Community Action Plan; Tools for Community Action; available online at: http://www.cdc.gov/healthycommunitiesprogram/tools/ - UN-Habitat: Community Action Planning, in People's Process in Post-disaster and Post-conflict Recovery and Reconstruction; available online at: http://www.fukuoka.unhabitat.org/docs/index_en.html - River, Trails and Conservation Assistance Program: Action Agenda; Community Toolbox, National Parks Service; available online at: www.nps.gov/ncrc - IFAD (2003): Guide pratique de suivi-évaluation des projets de développement rural.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 131
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.5.2 Cause & Effect Diagramme Importance The cause-and-effect diagramme (also called root-cause analysis, fishbone diagramme because of its shape, and Ishikawa diagramme because of its creator) visualizes the factors that affect a particular problem or outcome. • It concentrates team members on a specific problem and helps them construct a comprehensive picture of it. • It focuses team thinking and allows interdisciplinary exchanges and team reflection • It forces the team to consider a problem in all its complexity, ordering the factors that contribute to it, before deciding on how to deal with it. Cause and effect diagrammes can also be drawn as tree diagrams. The tree structure has an advantage over the fishbone-structure. As a fishbone diagramme becomes more and more complex, it becomes difficult to find and compare items that are the same distance from the effect because they are dispersed over the diagramme. With the tree structure, all items on the same causal level are aligned vertically. Steps Collecting the information • Brainstorm to identify the central problem being investigated and put it at the right end of a horizontal line; once the problem defined, brainstorm on its main causes, then decide which are the most important; • Brainstorm for the main causes of these categories ; select the most important of these secondary causes and add them on as further diagonal lines branching out; • Discuss the remaining causes and decide whether to analyse them further ; if so, add them on as tertiary causes; • Draw diagonal lines branching out of the horizontal line, each representing a major cause of the problem investigated; these problems can also be grouped and discussed under main category headings. Processing the information If the Cause and Effect diagramme is too complex and messy, the team can split it into several simpler diagrammes, each developing a particular problem. Once several underlying causes have been uncovered, the team can: • Identify the causes that need further investigations; • Discuss the ways to solve them.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 132
Example of Cause & Effect Diagramme
Caution • • • •
The value of the diagram is as good as the initial problem statement. Teams should therefore make sure that this problem is formulated as accurately as possible and that there is consensus about it; It is easy to confuse “problem” with “missing solution” as the later can also be interpreted as a problem. Teams should make a special effort to go to the root cause and not identify only symptoms; The rationale of the cause and effect diagram is that there are specific lines of causality. This is linear thinking. In real life, things are more complicated and there may be several lines; Trying to represent all causes and all inter-relationships between them can become unmanageable. Teams have to reach consensus about what are the most important ones.
Information - Harris, Robert (1998). Decision making techniques. Available at: http://www.sccu.edu/faculty/R_Harris/crebook6.htm.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 133
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.5.3 Context analysis
Importance Context analysis is important when you want to show a complex situation in a way that is easy for others to understand. They capture a large amount of objective information but also include peoples perspectives (subjective information), how they see the situation. The context analysis should not be your interpretation of other people's views. You can use context analysis: • to communicate your own understanding of a complex situation to others, • or to help a group to build a common understanding of a complex situation. Context analysis uses simple words and cartoon-like images, or icons, to represent all the features that are important to the central theme. In this way, they can capture the context or nature of a situation. They are not system models or diagrams of cause and effect relationships. Characteristics of a good context analysis: 1. It should show a broad range of subjective and objective information. i.e. peoples views and feelings as well as institutes/organisations and activities. 2. It should have a clear central theme that communicates, without additional explanation, exactly what the context analysis is about. 3. Ideas and relationships must be clearly laid out so that the whole picture can tell the story. The images and icons also tell an important part of the story. Linkages should show the nature of the relationship. Is it good or bad? Does it work or not? Why use context analysis?
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 134
Steps How do I begin? Start by ‘visualising’ the central theme in the middle of a large sheet of paper. Take care that the idea is clear and easy to understand. You may have to use a few words to help. What do I do next? Take any one of the ideas directly associated with the central theme. These are usually ideas mentioned in the first level of your mind map. Start to express each idea in a visual way. The context analysis is not a formal diagram. You can use symbols, icons, words, pictures or sketches to show what you mean. How do I keep going? Move from one idea to the next until all the first level ideas from the mind map have been covered. Do not forget to include information on the relationships and linkages between elements. Show whether they are working or not. Also try to capture what activities are taking place and the stakeholder’s different perspectives. When do I stop? You could go on and on adding to a context analysis. Deciding when to stop is not easy. To help decide, the following questions can be used as guide: • Have all the important elements been covered properly? • Is the picture still clear and easy to understand? • Has the picture captured the context surrounding your central theme? Example of a context analysis
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 135
Caution Context analysis includes more than just factual information. It should also include subjective information like the perceptions of different stakeholders. Preparing a context analysis is not a quick process. Managing discussion and disagreements takes a lot of time. Nevertheless, this is valuable as it builds common understanding. Context analysis contains a great deal of information. They often appear confusing and require thoughtful study rather than a quick glance. It takes time to develop the skills to draw meaningful icons and lay out a context analysis so that they do not look messy. Be prepared to redraw your picture several times so that it can communicate well to others. Information - Internet Sites: http//www.cpsc.ucalgary.ca/~couprie/623/ssmfinal.html http//www.csu.edu.au/ci//vol11/Andrew.Finegan/paper.html http//www-tec.open.ac.uk/ccc/sysdis.html - Checkland P. & Schooles J. (1990): Soft Systems Methodology in Action, Wiley & Sons, Chichester UK, pp13-27. - Couprie D., Goodbrand A., Li B., Zhu, D. (1998): Soft systems methodology. Univ of Calgary, Canada. 19p.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 136
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.5.4 Decision Trees Importance Decision trees provide a structure in which decisions and their implications are formulated and evaluated. • They give an idea of the implications -risks and rewards- of a decision relating to the resolution of a specific problem. • They allow the team to bring in different perspectives on the likely outcome of a particular choice and the probability of achieving the desired result. • They help the team make the best decisions on the basis of existing information and best guesses. Steps Write the decision Start by writing what decision needs to be made in a box at the left of the chart and develop the tree to the right; alternatively you can start up and develop it downward. Exploring the implications • Branch out from each outcome by writing the implications : in a circle if it is a definite state of affairs or an uncertain result, in a box if it is another decision; • Branch out from this decision by writing each option / solution into a separate box; • Branch out from each option by writing likely outcomes into separate circles ; if the result is another decision to be made, write it in a box ; • Follow on the remaining decisions until the end ; • Write a short note about what every final outcome means Example of decision tree
Caution • Identify as precisely as possible the problem the decision is supposed to help solve (although the process of building the decision tree does also help see more clearly the nature of the problem). Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 137
•
The rationale of the decision tree is that there is a unique path in the tree from the initial decision to each of the implications. This is linear thinking. In real life, things are more complicated and there may be several paths. The tree should therefore be seen more like a means of negotiation between members to develop insights into the issue being examined.
Information - Mind Tools: Decision Trees; available at: http://www.mindtools.com/dectree.html - Centre for The Study of Complex Systems: Decision Trees; available online at: http://vserver1.cscs.lsa.umich.edu/~spage/ONLINECOURSE/R4Decision.pdf
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 138
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.5.5 Driving Forces Importance A driving force is a conceptual tool that helps understand deep trends at work that have an impact on a focal issue. Driving forces are those factors -external and internal- that determine or influence outcomes, be they organisational, environmental, societal, community or economic outcomes. In rural development, more often, they are considered as simply the external factors that are causing, or might cause, changes in livelihood systems and rural practices. These factors are external to the system in which stakeholders operate and are considered to be beyond the control of these stakeholders. The type of forces –internal and external- will depend on the domain and use intended. Driving forces are explored in various domains and disciplines: climate change, population growth, organisational development, innovation studies, etc… They are used in strategic planning, policy formulation, innovation management, etc… In innovation studies, these are common examples of driving forces: Internal forces: knowledge, competence, level of satisfaction with the present situation, motivation , technological capacity, organizational culture, management systems,. External forces: competitive pressure, economic incentives, political incentives, demographic changes, technological evolution, social trends • Demographic factors (population increase, in/out migration, changing age/gender structure, etc). • Laws and regulations (affecting land ownership, labour relations, environmental protection measures, etc). • Policies (subsidies, price controls or guarantees, import/export controls, quotas and tariffs, exchange rates, etc). • Markets and competition (size of or access to, local, national and international markets; growing competitiveness of producers in other regions, countries etc). • Technology (availability of new genotypes, machinery, etc). • Institutions (new actors, influences, social organization). • Information (availability, communications technology). • Non-agricultural employment opportunities (processing, manufacturing, business, services, migration/ remittances, etc.) • Natural resources (characteristics of watershed, water availability, groundwater levels, land degradation, etc).
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 139
Steps Clarify the issue Innovation driving forces can be very different depending on: • The domain being investigated: agricultural development strategies, livelihood strategies, • The issue being investigated: technological innovation; organisational innovation, marketing innovation, etc… • The system level at which it is investigated: for example local, regional or national level; individual or network level; farm or value chain level. To have a common understanding of the issue and the level at which it is investigated is therefore necessary before starting to identify driving forces. Identify the forces Identifying driving forces is essentially an investigation of the causes of change. Broadly speaking, driving forces can be divided into political, technological, social/demographic, economic and technological forces. In the case of innovation assessment, identifying precisely what forces have been at work requires some research work (bibliography, interviews with resources persons) and group work. After deciding on the issue and the decision focus, driving forces can be identified through a brainstorming session. This consists of asking the group participating in the exercise to list key trends that had an impact on the issue. Prioritize the forces Prioritizing involves identifying the small number of driving forces that are likely to have greatest impact on the issue. The following procedure can help in the prioritization process. • Select only those forces that had an impact on the issue, and eliminate the others: although forces’ impact cuts across a sector of economic activity, some forces may have an impact on a specific issue and not on others • Group trends, or combinations of trends that combine to create a force: a force can be a combination of different elements. Make sure not to select forces that overlap too much. This is difficult because forces are rarely completely independent from each other. Use of driving forces in innovation assessment In innovation assessment, driving forces can be identified and analyzed as part of broader analyses which make use of its results, for example: Context Analysis: To describe the complex situation of innovation processes and the forces that influence it. Force Field Analysis: To identify the forces that influence innovation plans and projects and decide how to use or counter them Scenario Building: To identify forces that we have to deal with in innovation processes in the future and plan ahead Innovation Timelines: To identify what forces have shaped an innovation process overtime The procedure to select forces can be through: • Simple discussion and consensus between team members • Scoring or ranking techniques. The number of forces to select depends on the use to which they are put; for example, analyses based on the study of interaction between forces cannot accommodate too many forces: Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 140
•
For Scenario Planning or Force Field Analysis, the number should not exceed 4 to 5 forces, to remain manageable For Context Analysis or Innovation Timelines, more forces can be selected.
• Caution
The identification of driving forces is often biased when conducted by people who have the same profile (professional, educational) or who work for very similar organizations, because they share the same outlook and experiences. The process should therefore be inclusive and participative. Care should be taken to seek the opinion of a wide range of professionals, including farmers, women, traders, etc… Information - Sellamna, N.: Driving Forces, Scenarios and Strategies - Key Concepts; ICRA Learning resources on ARD; available online at: http://www.icra-edu.org/page.cfm?pageid=anglolearnicrahandouts - Synnot, B. & Associates: Identifying Driving Forces; The Toolbox for Change - A Practical Approach; available online at: http://www.billsynnotandassociates.com.au/centre-of-excellence/toolbox-forchange.html
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 141
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.5.6 Force Field Analysis Importance The Force Field Analysis (also called pros and cons analysis) aims at drawing a picture of forces that work in favour or against a plan or project. The analysis will provide the assessment team with information to help make decisions that accommodate the interests of all forces. It helps plan actions that reduce the impact of opposing forces and maximize the support of favourable forces. Steps Collecting the information • List all the forces likely to manifest themselves if the plan were to be implemented ; • Order them in two categories (for and against); • Assign a score to each force -from 5 (stronger) to 1 (weaker); • Draw a diagramme representing the forces; alternatively visualize the strength of the force by giving it a larger size. Processing the information Once the information is visualized, the team has to decide on ways to create the best conditions for the plan to succeed. Ideally suppressing the negative, or restraining, forces should be avoided: trying to impose change will only alienate them further and turn them into active opposition. Ideas as to how to alter the balance of forces can be done through a brainstorming. Some questions that could be answered: • How could we reduce the weight of restraining forces by winning them over to our plan? • Could we add more positive forces to alter the balance of power? Examples
Source: JOLISAA Project (2011): Collaborative Case Assessment in Benin
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 142
Caution The rationale of Force Field Analysis is that the outcome of a future plan or project is determined by forces acting for or against it. The assessment team must remember that: • They must reach consensus on what is the future “desired” state if they are to agree on what are positive or negative forces; • Some actors may view a force as favouring the desired future state, while other people may view the same force as maintaining the status quo; some forces do both; the team must reach agreement as there is no ‘middle way”. Information - Mind Tools: The Force Field Analysis Techniques; available online at: http://www1.umn.edu/ohr/prod/groups/ohr/@pub/@ohr/documents/asset/ohr_asset_310480.pdf - Somesh Kumar (1999) : Force field analysis: applications in PRA; in PLA Notes Issue 36, IIED London; available online at: http://pubs.iied.org/pdfs/G01849.pdf - Training Material in Creativity and Innovation for European R&D Organisations: Force Field Analysis;available online at: http://www.train4creativity.eu/dat/77F14335/file.pdf?635101004008370000
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 143
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.5.7 Policy Analysis Matrix Importance Innovation processes are influenced by policies, whether local, regional or national, made by governments or donors. These policies can hinder or help innovation. • In assessing innovation processes, policy analysis can help understand the framework into which innovative activities take place, and individual and organisational decisions. • In developing innovation projects, policy analysis can help plan more realistic objectives and activities. • In developing advocacy programmes, policy analysis can help better target messages and people to influence. Depending on objectives, policy analysis can focus on one of three areas: • Policy formulation: How are political decisions made? By whom? • Policy implementation: How are political decisions put into practice? • Policy analysis: What are the policy choices? What is their impact? Does the impact comply with the policy’s objectives? A further area is the formulation of policy alternatives, which is not dealt with here. The three types of analysis can be carried out together, but focus will depend on the objectives of the analysis and its context. Analysis within the framework of an advocacy initiative will for example look closely at the first area. Analysis in the context of innovation assessment will need to look at the third one. What is the Policy Analysis Matrix? The Policy Analysis Matrix is, in fact, a series of matrices helping the analysis by: • Structuring the process through a series of questions • Exploring the objectives, processes, causes and consequences of the policy The methods needed to produce the information for the analysis are not provided by the Matrix. These need to be decided by the team carrying out the analysis. Steps Clarify the issue Clarifying the issue means determining: • Why is the analysis conducted? What problem it is supposed to solve? What information it is expected to yield? • What policy is to be analysed (Research? Extension? Agricultural subsidies?) and how is it related to the issue? • How is the analysis going to be conducted? With what methods? By which persons? With the help of what resource persons?
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 144
Analysis of policy objectives Once the policy is clearly identified, its objectives are sought out and analysed. Key Questions Objectives
What are the stated objectives of the policy?
Expected results
What is the policy expected to produce? What is the intention of the policy makers?
Example of analysis of policy objectives
Objectives Expected results
KEY QUESTIONS The policy creates a national tax on grazing; it disengages the State from input distribution and privatises its infrastructures. The tax on grazing will change the economic context of herders’ decision making and will lead to a reduction of herd size, and thus to an improvement of livestock and pasture management. It will also a better control over local resources. The withdrawal of the State will encourage the participation of the private sector in the agricultural inputs market, and thus a greater availability of inputs for farmers and less pressure on the State’s limited resources.
Analysis of causes and consequences of policy Once the policy and its objectives are clarified, their causes and consequences are analysed following the questions below.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 145
Example of causes and consequences of policy
Analysis of policy processes The last stage of the policy analysis is to analyse the results of the previous two stages, to put them into perspective and establish causal relationships. Two broad questions can help structure this analysis: • What are the relationships between the promotors of the policy, its beneficiaries and the losers? Are they professional, private, ethnic relationships? Are they conflictual relationships? • What is the nature of the process used by the initiators to promote the policy with formal decision makers? Is it legal and transparent? Was there an exchange of favours? • What incentives for change does the policy offer? Here are some factors that can help the analysis: • The perception of the question’s importance • The importance of political support to the expected change • The importance of the participation necessary to its implementation at the central and local levels • The importance of benefits, as perceived by actors • The costs of opposition to change, as perceived by actors • Is the distribution of benefits limited to those who contribute to resources? • Is political reform a condition for additional funding by donors? Caution Policy analysis is always a difficult task because it requires a considerable amount of information, both formal and informal, and both on formal processes and informal processes. Political decisions do not always coincide with formal decision making processes. Some information is kept secret, sometimes hidden. The implications are that: • Policy analysis, to be useful, has to go beyond official and public discourse to explore deeper causes and unreported consequences.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 146
•
For this, the group carrying out a policy analysis must go beyond their own limited knowledge and mobilise specialised / confidential knowledge of resource persons.
Information - Daniel Gustafson and Marcus Ingle (1992): Policy Characteristics Analysis, Technical Notes n° 3, USAID Implementing Policy Change Project. http://www.msiworldwide.com/wpcontent/uploads/2011/07/IPC_Policy_Characteristics_Analysis.pdf - Daniel Start and Ingie Hovland (2004): Tools for Policy Impact, a Handbook for Researchers, ODI. http://www.odi.org.uk/sites/odi.org.uk/files/odi-assets/publications-opinion-files/194.pdf - Stephen Sandford (1984): Better livestock Policies for Africa; African Livestock Policy Analysis Network; International Livestock Centre For Africa (ILCA) http://www.fao.org/wairdocs/ILRI/x5499E/x5499e00.htm#Contents .
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 147
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.5.8 Ranking techniques Importance Aid to decision making Ranking can be used to explore the perceptions of stakeholders, or different groups of stakeholders, and aid the decision making of research and development agencies. Factors commonly ranked are shown below. Ranking can be used to explore the perceptions of stakeholders, or different groups of stakeholders, and aid the decision making of research and development agencies. Factors commonly ranked are shown below.
Use of multiple criteria Use of multiple criteria for decision making. In development oriented research, the use of single criteria (e.g. “yield”) is not sufficient for technology assessment. The increasing scarcity of research funds, and the development orientation of agricultural research, means that research priorities have to be determined on the basis of stakeholders’ multiple needs and objectives. Steps Determine options and criteria • Provide the individual or group with a list of options - or ask them to draw up the list. • Represent these items on cards, as words or drawings). Generally speaking, the list should not exceed about 6 or 7 options. • Instead of written words, it can often be helpful to choose objects (leaves, wood, fruits, grain, coins, etc) to represent the options to be ranked, or the criteria used to rank. Determine relative rank • Start by asking the respondent(s): “Which is the best?” (“most important”, “most useful” etc). “The next best?”. “The worst?”. “Next worst?”. “Of the remaining, which is best?”. “Which worst”, etc. • Ask why. “What is good about it?”. “What is bad about it?” These criteria can be used in a matrix ranking as well. Include views of key stakeholders The responses of different groups or types of stakeholders can be compared.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 148
Ranking are normally carried out by semi-structured interviews, either with individuals or with groups of respondents. Ranking with groups generally allows quicker progress, and promotes discussion. Examples of ranking In Tanzania, agropastoralists were asked to rank problems, and the result was compared with researcher perceptions. (Ranking was also done with crop Production Ranking by Ranking by farmers, and mixed farmers; but those results are not Constraint Researchers Farmers shown here). Lack of water 4 1 for livestock Researchers ranked the problems according to their Crop water 2 2 perception of distribution (% households affected), loss of stress Livestock 1 3 potential income per household, and frequency of Diseases occurrence. Crop pests 6 4 Farmers argued that water was the most critical Low soil 2 5 Fertility constraint, because they could do something about Lack of 5 5 disease, but nothing about the lack of water. Firewood Weeds 5 Source: Baral et al, 1993 Lack of fodder
Preference ranking was used to obtain the views of local people on the virtues and drawbacks of local species of firewood. By repeating the procedure, the knowledge of different locations and user groups was compared. As with all ranking exercises, the value is not in the final product, but in the discussion that is created in the process. The knowledge that is shared can be a starting point for a more in-depth investigation of perceptions and criteria held by different types of stakeholder. Source: Inglis, 1991
Plum Tik Spice Tik Monkey Apple Mangrove
-
8
Ranking PT PT ST PT MG MG
1 2 3 4 5 6
Fambul tik Plum tik Black tumbla Mangrove Spice tik Monkey apple
Fambul Tik
FT FT FT FT
Black Tumbla
FT BT BT BT FT PT
ST
MA
MG
FT
BT
Notes: 1 = most preferred. 3 = least preferred
Local
ICPL 84060
ICPL 332
Leaf production
3
1
2
Flower production
3
1
2
Green pod production
3
2
1
Pod filling
3
1
2
Pest resistance
3
2
1
Seed yield
3
2
1
Taste
3
1
3
Wood production/quality
3
1
2
Market price
1
2
2
More on preference ranking In a pairwise comparison, ask the informant(s) “which of these do you prefer to grow” (or “gather”, “eat”, etc.). Record the answer in the matrix. Then ask the informant(s) to give the reasons for the choice; the good and bad points about each alternative.
After all possible combinations have been discussed and recorded, a ranking can be constructed according to the frequency of occurrence for each item. As in simple ranking, present an individual or group with the list of options - or ask them to draw up the list. Represent the items on cards, as words or drawings). Generally speaking, the list should not exceed about 6 or 7 options.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 149
Ask the individual or group to then indentify those factors which are important in deciding which of the problems or options are important. These criteria are the value judgements people make when analyzing the relative merits of the options, and this discussion can be important as the overall result of the matrix ranking exercise. Again, there should not be more than 6 or 7 criteria. Arrange the options across the top of the table in the columns (the x-axis). Arrange the criteria down the side of the table (i.e. the rows or y-axis). Ask the individual or group to rank each option according to each criteria. The matrix ranking of pigeon pea varieties was constructed by women farmers in 4 villages in Andhra Pradesh, India. Farmers were previously asked to grow the genotypes on large plots. After harvest, a matrix ranking was then conducted in semi-structured interviews with groups of 10-15 women per village group, using leaves, coins, seeds, etc. to represent the criteria, and bean seeds to represent ranks, in a matrix drawn on the ground. The criteria defined by farmers for evaluation were different than those normally used by ICRISAT researchers (days to maturity, grain yield, seed colour, etc). The women rated ICPL 332 equal or higher than the local variety on 7 of their 10 criteria. Nevertheless, they said that overall they preferred the local. ICPL 332 had been officially released in Andhra Pradesh, but women farmers said they would not grow it because of it’s bitter taste - even though it yielded far more, and was more resistant to the prevalent Helicoverpa armigera pest. Caution On disadvantages • Manipulation of numerical data in an over-simplified and incorrect manner (e.g. adding/averaging ranks). • Biased results from small or non-representative groups. • The image of ranking techniques (especially in the case of technology assessment) as not “scientific” enough by scientists to allow publication in peer-reviewed journals. Information - Inglis, (1991): Harvesting local forestry knowledge: a comparison of RRA and conventional surveys. RRA Notes 12, p 32-40. - Maxwell, S. & Bart, C. (1995): Beyond ranking: exploring relative preferences in P/RRA. PLA Notes 22: 28-34. International Institute for Environment and Development, London. - Pretty, J.N., Guijt, I., Thompson, J. & Scoones, I. (1995): A trainer´s guide for participatory learning and action. International Institute for Environment and Development, London.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 150
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.5.9 Scenario Building Importance Scenario building is typically a strategic planning tool. Scenarios help order our perceptions about alternative future environments in which today's decisions might play out. In practice, scenarios resemble a set of stories, written or spoken, built around carefully constructed plots. Stories are an old way of organizing knowledge, and when used as planning tools, they defy denial by encouraging--in fact, requiring--the willing suspension of disbelief. Stories can express multiple perspectives on complex events; scenarios give meaning to these events. The point of scenario planning is not to produce accurate pictures of the future, an impossible task. It is to make strategic decisions that will enable stakeholders to coordinate and orient their action. A good scenario is one that enables them to design a strategy that can adapt to several possible futures. For this, identifying strategic ‘heavy’ driving forces and most likely zones of uncertainty is enough. Steps Clarify the issue and describe the present situation • Define what issue is being explored through the scenarios Defining the issue is of paramount importance, because scenarios will be focused on that issue: what is it that we want to explore the likely evolution in the future? Hopefully, the work of the team conducting the scenario building exercise prior to the scenario planning phase has already produced such a description of the issue. Instead of simply identifying a “problem”, stakeholders can, alternatively, identify the decision to be made in order to solve it (How to better manage soil fertility? How to organize a commodity market? How to organize the local research and extension system? How to manage the local land use system? How to conduct an innovation process? Etc.). • Select the stakeholders to Involve in scenario building Stakeholders participating to scenario planning can be identified before or after the identification of the issue. Depending on the nature of the issue, the team seek validation by the stakeholders. Whenever possible, stakeholders should be involved in the issue definition phase. The type of stakeholders involved will differ from situation to situation. The team can start with a limited number of stakeholders and expand to others as the analysis progresses. • Decide the time horizon Depending on the time horizon (5 years, 10 years, 50 years), scenarios will be very different. The time horizon needs to be negotiated to fit the requirements of all stakeholders.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 151
Stakeholders’ mandates, interests and potential for action may not coincide in the same time frame. Farmers have more immediate concerns that need to be accommodated in shorter term scenarios. On the other hand, the development of some problems needs to be explored in a longer time frame. Past trends can give an idea on what time perspective to take. For example: in situations of strong population pressure, problems of soil fertility are better envisaged in a long perspective, a generation or more (25 years). Identify driving forces Driving forces are those predetermined elements whose dynamics is unpredictable or is underway and cannot be significantly changed in the short term. They can be of a different nature. • Political Driving Forces → How is the political situation likely to evolve: will the regulatory policies change (land tenure laws, economic regulations, international exchange laws) and how will it affect the local situation / the issue? → Will locally produced commodities be competitive? → Is the country likely to introduce subsidies in the future? • → → → →
Economic Driving Forces What are the macro-economic trends: how is the national market evolving (private, public)? Will the region be more industrialized and urbanized? Will non-agricultural employment increase? What are the micro-economic trends: What are the sections of the farming population most affected by the issue? → Is there a clear typology of people affected by the issue? → What are the present sources of farmers’ income? How are these sources evolving? → What is being invested in the farm to solve the issue? • Social and Cultural Driving Forces → What will be the demographic situation in the future (specify the horizon) : what trend does quantitative data show? → How is the balance of population likely to evolve: will the youth still represent the same ratio? → What will consumption models be like? → Will they be interested in agricultural activities? → Will the agricultural population increase or decrease? → Will women farmers have access to land ownership? • → → → →
Technological Driving Forces What is the state of research pertaining to the issue? What has been the response of farmers to present technological solutions? Are there any promising new technologies being developed? Who is most likely to adopt them?
• Environmental Driving Forces → What are the broader environmental factors affecting the issue and how are they evolving? → Are there any unpredictable aggravating factors (floods, droughts)? The above factors must be taken as given in the building of the scenarios, and their implications built into the alternatives. Prioritize the driving forces Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 152
In a complex situation, there is, by definition, no limit to the number of driving forces. The point in listing all the driving forces is to try to understand them and analyse how they interact with each other. Scenario builders will realize that, often, there is a hierarchy of driving forces which limits the key forces to two or three (normally of a broader political or economic nature). Once this is done, no more than four of them should be retained to build the scenarios. More forces will render the scenario analysis too complex and will miss the point of the exercise. Planners can use tools such as problem trees, or matrix ranking or scoring or simple group discussion for prioritisation. • Discuss the implications Once the issue and its driving forces have been defined and prioritised, the planning team can engage in a discussion of their implications. • Facilitation is essential for the process to run smoothly. • Visualization is a good way to plot possible scenarios. An important principle is for the team to constantly question the plausibility and consistency of the phenomena they are describing. • Decide on the likely evolution of driving forces Once the driving forces have been prioritized, the team can discuss in what way they are likely to evolve in the future, so as to plot the scenarios and describe how they will affect the present situation. In principle, there are 3 possible states in relation to their present situation: • Status quo: they are unlikely to change • Better: they will evolve in a direction that is more positive for the situation (for example: the price of fertilizers goes down, in the case of a soil fertility issue) • Worse: they will evolve in a direction that is negative for the situation (for example: population pressure will increase in the case of a land shortage issue). The likely future states of the driving forces can be considered as “hypotheses”, and like all hypotheses they must be carefully examined: • Check whether the hypotheses of each scenario are internally consistent. Is each hypothesis backed by evidence? For example: is it likely that fertilizer prices will be significantly higher in the time horizon of your scenario? What do past trends show? • Check whether the hypotheses are consistent with each other. Can the assumption you make about one driving force generate the behaviour you describe in another? For example: is it likely that in a situation of very high population pressure and low income farmers will still find land and cash resources to grow non-food crops and raise cattle? Visualise and describe the scenarios Kite Diagramme When there are more than two driving forces (three, four, five), they can be represented in a diagramme, where each force starts from the centre, reaching out in importance. Example in the case of an issue of acute soil fertility decline. The driving forces identified are the likely evolution of agricultural population, fertilizer prices, household income and alternative sources of soil nutrients, leading to 5 possible scenarios.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 153
Orthogonal Axes When the uncertainties can be reduced to two single elements, these can be visualized as two axes crossing, with each corner of the quadrant representing a possible future. Each axis represents a spectrum of uncertainty along which the future will play out. There is, of course, an infinity of combinations of forces along the two spectrums, but scenario builders must resist the urge to explore more than the four and not lose sight of the point of the exercise.
Matrix Presentation
A good way to identify scenarios is to give them each a name which conveys its main logic. Writing down the story is formulating in a narrative form the vision outlined by each scenario. This formulation should expand on the visualized form by explaining the dynamics which led to the situation described by the scenario. The story must attempt to show the consistency of the scenario. Two main questions will need answering: • Is there an empirical basis behind each hypothesis: what allows the stakeholders to think that the situation described by each hypothesis is likely to happen? • What is the relationship between hypotheses: How is the situation described by each hypothesis supposed to lead to the situation described by the others? Point Break Scenario • In this scenario, the agricultural population has kept its upward trend. • Land pressure has reached its limit. • No alternative employment sources are available, household incomes remain very low and no resources can be devoted to soil fertility maintenance. • Farm size has gone down and can barely sustain a household, let alone produce alternative soil nutriment sources.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 154
•
Liberal policies have eliminated what remained of state subsidies and fertilizer prices have considerably increased
Diversification Scenario
• •
• In this scenario, the agricultural population has stabilized around its present level, mostly due to out migration. • Country policies are beginning to bear fruit and some local industries have started in the region, allowing households to diversify and considerably increase their incomes. • Few migrant children send in money but at least they are no longer a burden on their families. The international fertilizer market has also stabilized and prices have not increased, still with little subsidies from the government. The increased purchasing power of farmers has made the search for alternative sources of fertility maintenance little interesting.
Differentiation Scenario
• •
In this scenario, country policies are beginning to bear fruit. • New market opportunities for products like vegetables and fruits, allowing households to diversify and considerably increase their incomes. • The international fertilizer market has also stabilized and prices have not increased, still with some subsidies from the government. • The agricultural population has stabilized around its present level , mostly due to out migration, but the number of farms has diminished. The increased purchasing power of farmers has been mostly used to expand their land holdings and diversify their production. The practice of fallow has reappeared and the number of cattle increased, with a significant production of animal and green manure.
Trimming Scenario
•
• In this scenario, the agricultural population and household incomes remain approximately at the same level. • Fertilizer prices have considerably increased, but this had little impact since their use has always been low. • Those who have been affected are mostly the middle sized farmers and farmers who had other sources of income to pay for fertilizers, which have now virtually disappeared. No significant alternative has been found for lack of investment and scarcity of land.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 155
Greening Scenario
•
• In this scenario, the agricultural population has kept its upward trend. • Land pressure has reached its limit. • No alternative employment sources are available and household incomes remain very low. • Liberal policies have eliminated what remained of state subsidies and fertilizer prices have considerably increased. • But a radical change in farming has led to a zero grazing system, with a considerably increased fodder crop production and cattle population. Green and animal manure provide an alternative source of soil nutrient and help maintain, and in some cases, increase soil fertility.
Analyse the scenarios and choose a strategy The analysis of the scenarios will depend on the objective set at the beginning. It will however aim at examining the basic elements for the formulation of a strategy to solve the issue. The following are standard questions to answer in the analysis.
Who are the winners and the losers in the scenario? Who are the winners and the losers in the scenario? In situations where resources are limited (ie, land, water) there is bound to be social and economic tensions. Who stands to gain most from each scenario? Who stands to lose? For example, if the research team has a typology of farmers, what would be the impact of each scenario on each type: how would each type contribute to each scenario and would this impact on other types? What type is most likely to disappear from the picture? In the case of an ongoing innovation process, who is likely to gain from the innovation once the process achieved? How are the winners and the losers likely to react? Strategic decisions must be made as to how best to support that process. What are the implications for each scenario? • What are the alternatives? What scenario offers the best alternatives for stakeholders? Scenarios may not always be described as ‘good’ or ‘bad’, ‘desirable’ or ‘undesirable’. They contain elements of each. Scenario builders should decide what elements they prefer. • What are the strategic decisions? What impact will the decisions made today have in the future? Scenarios help put decisions into perspective; they make decision makers aware of the wider implications of their decisions. Most importantly, scenarios help determine what strategic decisions should be made now. • What are the indicators? What are the signs to watch for in the future?
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 156
New legislation, technological innovations, social, economic and cultural trends are signs to monitor as indications of what type of scenario is unfolding. • What new partnerships are needed? What institutional changes are needed to carry out the best alternatives? Collectively outlined scenarios need collective future action. The complexity of challenges ahead of stakeholders will necessarily require closer collaboration in the future. Caution Logical process: the future developments postulated must be logical and this can only be achieved if personal values and ideology are put aside; Open and informal process: for scenario building to be creative, stakeholders must feel free and comfortable; scenario building must be felt as exploring possible futures and not making commitments; Inclusive and holistic process: scenarios about the future only make sense if they encompass all aspects of a problem (social, economic, cultural, etc...); Constructive process: looking forward helps participants a common future and seek a common ground; Credible process: scenario builders must be seen as knowledgeable about the issues and neutral, not advocates of a particular outcome. Source: Adapted from http://www.gbn.org/scenarios/fleur/fleurIntro.html
Information - Global Business Network – Scenario Planning: http://www.gbn.org/scenPlan.html - Schwartz, Peter (1991) : The art of the long view, Doubleday Currency. Summary available at: http://www.gbn.org/scenarios/alv/ALVSummary.html - Wilkinson, Lawrence (1995) : How to build scenarios, in Wired, http://www.wired.com/wired/scenarios/build.html - World Water Vision http://www.watervision.org/ - ICRA Learning Resources: Scenarios and Strategies - Approaches (pdf, 146K)
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 157
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.5.10 Scoring Techniques Importance Aid to decision making Scoring techniques can be used to explore the perceptions of stakeholders, or different groups of stakeholders, and aid the decision making of research and development agencies. Factors commonly scored are shown below.
Participation of stakeholders Participation of many stakeholders in the assessment of development options. The concept and process are relatively simple (compared to many scientific methods), allowing rural people to understand and be directly involved in the process. It also allows stakeholders to introduce, discuss and use their own criteria for decision making. Use of multiple criteria Use of multiple criteria for decision making. In development oriented research, the use of single criteria (e.g. “yield”) is not sufficient for technology assessment. The increasing scarcity of research funds, and the development orientation of agricultural research, means that research priorities have to be determined on the basis of stakeholders’ multiple needs and objectives. Steps Determine options and criteria Scoring means assigning values from a convenient scale, within a matrix of alternatives and criteria, to arrive at an overall best option. Weighted scoring means assigning values within a matrix of alternatives, taking into account the relative importance of the different criteria, to arrive at the overall best option. Provide an individual or group with a list of problems or options - or ask them to draw up the list. Represent these items on cards, as words or drawings). Generally speaking, the list should not exceed about 6 or 7 options.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 158
Determine relative rank Start by asking the respondent(s): “Which is the best?” (“most important”, “most useful” etc). “The next best?”. “The worst?”. “Next worst?”. “Of the remaining, which is best”. “Which worst”, etc. Ask why. “What is good about it?”. “What is bad about it?” These criteria can be used in a matrix ranking later.
Include views of key stakeholders Ask the individual or group to place objects (stones, beans, etc) in each cell of the table according to their perception of the value of the option or problem according to the criteria If weighting is to be applied to the criteria afterwards (e.g. by ranking or scoring the criteria themselves), a fixed number of stones/beans (e.g. 10) should be distibuted across each row Alternatively, a fixed number of stones/beans (e.g. 50) can be distributed throughout the whole table: this automatically gives a weighting for each criteria, but may be more difficult for the group to manage. Example of scoring The women farmers of Andhra Pradesh were asked to compare two varieties by dividing ten bean seeds between the two varieties within a matrix drawn on the ground. This procedure gives the scientists useful information about varietal performance, and the women's perceptions of the varieties. No attempt was made to add the scores in each column (i.e. to get an overall score for each variety), because the criteria were not of equal importance to the farmers. Source: Pimbert, 1991
Caution On disadvantages • Manipulation of numerical data in an over-simplified and incorrect manner (e.g. adding/averaging ranks). • Biased results from small or non-representative groups. • The image of scoring techniques (especially in the case of technology assessment) as not “scientific” enough by scientists to allow publication in peer-reviewed journals.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 159
Information - Inglis, (1991): Harvesting local forestry knowledge: a comparison of RRA and conventional surveys. RRA Notes 12, p 32-40. - Maxwell, S. & Bart, C. (1995): Beyond ranking: exploring relative preferences in P/RRA. PLA Notes 22: 28-34. International Institute for Environment and Development, London. - Pretty, J.N., Guijt, I., Thompson, J. & Scoones, I. (1995): A trainer´s guide for participatory learning and action. International Institute for Environment and Development, London.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 160
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.5.11 SWOT Analysis Importance The Strengths-Weaknesses-Opportunities –Threats (SWOT) analysis is a strategic planning method used to summarize the key elements of a strategic environment, in various contexts and at different levels of analysis. For example: • In national development, SWOT can be used to analyse the environment of a whole economic sector (agriculture, education, etc..), to develop a strategy to improve its performance • In organisational development, SWOT helps develop a strategy to establish an organizational operating plan. • In value chain analysis, SWOT can help explore the economic environment of a commodity, to develop a strategy to improve its competitiveness. • In innovation assessments, SWOT can be used to analyse the environment of an innovation and develop an action plan to support the innovation process. What is it? The analysis is structured in a 4 segment matrix, each representing one component of the analysis
- Strength and Weaknesses refer to factors that are internal to the project / organisation being analysed. - Opportunities and threats are external; they refer to its environment.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 161
Steps Brainstorm Strengths, weaknesses, opportunities and threats are brainstormed, with a facilitator reminding the group to keep the focus internal for strengths and weaknesses and external for opportunities and threats. • Following the brainstorming procedure, all ideas (items) are proposed by all group members, without restriction. • Once all items are collected, they are listed and discussed, to clarify meaning, remove overlapping ideas and keeping /eliminating those on which there is a consensus Review and prioritise each category Once all ideas have been brainstormed, and disagreements solved: • Each component of the SWOT matrix is reviewed separately and items discussed in depth, drawing their potential implications for the project; • In each component, items can be ranked by order of importance; • Only 5 or 6 items deemed the most important are kept (it is very difficult to proceed with the analysis with a large number of items, especially if they overlap). Example of SWOT Analysis
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 162
Develop a strategy / Action Plan Once the SWOT Matrix is completed, several strategies can be worked out to deal with the issue at stake: • Converting weaknesses into strengths • Matching strengths with opportunities • Using strengths to avert threats.
Using the results of a SWOT analysis to develop a strategy can be structured around the answers to the following questions: 1. How can we take advantage of new opportunities, using our strengths? 2. What weaknesses can prevent us from seizing these opportunities? 3. What strengths do we have that can help us remedy to our weaknesses? 4. With what strengths it is possible to neutralize threats? 5. What threats, aggravated by weaknesses, do we need to be afraid of most of all? Caution On defining the context Although SWOT analysis can be applied to a variety of contexts, it has mainly been used to develop strategies for organisations and projects. Some contexts are neither of these. For example, how to define the context to be analysed in innovation assessments? The definition must be clearly understood by all involved in the analysis. For example: • Future support to local innovation processes can be considered as a “project” on which the SWOT analysis will be focused • In the case of a product innovation, the analysis can be focused on environmental conditions of the product (value chain, crop, new input, etc.). On knowledge and expertise required A basic assumption of SWOT analysis is that people involved in the analysis are a knowledgeable cross-section of individuals and interests is included and that they are honest and can back up their views with facts or representative examples. If this is not the case, the results can be biased, false or misleading. On different perceptions It is important to note that in a multi-stakeholder analysis, the perception of strengths, weaknesses, opportunities and threats can be different. What is a threat to someone can be perceived as an opportunity by another. Within the same organisation / project, what is seen as a strength can be seen as a weakness by another. The challenge of the analytical process is to make sure that everyone has the same understanding of each of the 4 components of the analysis and arrive at a common definition. On the implications for strategy Decisions made on the basis of a SWOT analysis are important and should be based on a solid analysis. A common mistake is to underestimate the downstream implications (e.g. the cost and complexity of overcoming weaknesses). The results of the SWOT analysis should be triangulated with those of other, more in-depth evaluation tools to help make final decisions. Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 163
Information - Harris, Robert (1998). Decision making techniques [Online]. http://www.sccu.edu/faculty/R_Harris/crebook6.htm
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 164
LEARNING RESSOURCES FOR THE ASSESSEMENT OF LOCAL INNOVATION PROCESSES
3.5.12 Visioning Importance Visioning (also called “future visioning”) is similar to scenario building. It is a process through which a group develops a vision (a single scenario) to define and help achieve a collectively desired future. Visioning exercises are used to develop strategic plans in a variety of contexts: urban planning, community planning, land use planning, organisational development, innovation impact assessment, action plans, etc... They allow the group to create images that help provide a frame for a strategy for the achievement of the collective vision.
General objectives of visioning In the context of community visioning, these objectives have been proposed by Ames: • better understand local strengths, weaknesses, and core community values; • identify outside forces, trends and issues that are shaping its future; • articulate a preferred vision to guide its future directions; and • develop the strategic tools to achieve its vision. Specific objectives of visioning The objectives of visioning can be very different, depending on contexts. In the context of a collective strategic planning process, the outcome of visioning is a long term plan, sometimes with a 20 to 30 year horizon. In the context of a community consultation, the outcome of visioning can be a set of priorities to be integrated into public planning and/or policies or to review existing programmes and policies. In the context of community action planning, for example to develop an innovation support plan, the outcome can be to formulate objectives and activities for the action plan.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 165
Steps Define the issue Visioning is about exploring the desired future of a particular issue. This issue needs therefore to be defined and made clear to all, so as to avoid misunderstanding during the visioning process. What is the issue we are planning for? • The desired future of a value chain? • The desired future of communal land? • The desired future of soil fertility management? • The desired future of fish farming in the area? • Etc… Map present situation The purpose of this first step is to describe as accurately as possible what exists in the present. What is described depends on the issue object of the visioning. In the case of community land planning, it is an assessment of community strengths and weaknesses: key characteristics of the area, such as geography, natural resources, population, demographics, local economy and labour force, political and community institutions, housing, transportation, education resources, cultural and recreational resources, shared values and beliefs, etc… In the case of innovation assessment, it can be a review of the state of an innovation process, or the first step of an action planning process. Useful tools are: SWOT Analysis - Context Analysis. Decide on a time horizon Depending on the time horizon (e.g. 5 years, 10 years, 50 years), a future vision will be very different. Time horizons depend on the type of planning, the issue and the planner. In rural development time horizons are shorter, perhaps 5-10 years for a strategic plan; shorter for an action plan. The time horizon needs to fit the requirements of all stakeholders. This is a difficult task because stakeholders’ mandates, interests and potential for action may not be the same. For example: farmers will have more immediate concerns that need to be accommodated in a short term vision. But it is often not possible to mobilize collective action to achieve too short term visions, especially if they are too ambitious. On the other hand, the development of some issues needs to be explored over a longer time frame. Past trends and driving forces can give an idea of what time perspective to take. For example: where there is high population pressure, problems of soil fertility are better looked at over a longer time frame, for example: a generation or more (25 years). But it is difficult to plan and motivate people for results expected beyond their own life span. Develop a future vision Visualization of present community resource map from Soroti District in Uganda. Source : Linked Local Learning Resource CD. ISG, 1999.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 166
• Where are we going? The purpose of the second step is to determine where we are headed if nothing is done to change course. It involves identifying: → Current trends and how they are likely to impact on the issue object of the visioning. For example: describe the state of value chain in 10-15 years as we can imagine it; describe the soil fertility issue in one generation from now and its impact on land productivity, etc… → How stakeholders are likely to benefit and/or lose. For example: what will be the situation of women farmers like? The situation of small food process businesses? • Where do we want to be? This step creates the actual vision for the future, describing what the stakeholders seek to become, what the desired future will look like if stakeholders react to trends in a proactive manner. For example: → Describe / Map the value chain in 15-20 years from now. How will it be organised? What roles will stakeholders play? Etc… → Describe / Map the community land in 15-20 years from now. How will the land be allocated? Where will this type of crops be grown? Where will new wells be dug? Where will new buildings be constructed? • How do we get there? Once the vision has been created, an action plan to achieve it is developed. The action plan should be as specific as possible: actions to be taken, responsibilities, new partnerships required, timelines, costs, etc... The following questions guide the process: → What strategy will we adopt? How will we move from the present situation to the desired situation? → What objectives do we seek to achieve?: What forces do we use? What weaknesses do we need to solve? What opportunities can we seize and how? What threats must we avert? → What activities do we plan? How will we achieve our objectives? EXAMPLE Visualization of new partnerships needed for future vision of natural resource management in Soroti District in Uganda in year 2020. Source : Linked Local Learning Resource CD. ISG, 1999.
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 167
Caution On the meaning of the vision A vision is a clear and succinct description of what stakeholders would like to see happening in the future after they have successfully implemented their strategy. It is therefore an ambition, a dream. Like all ambitions: • Its accuracy and relevance is determined by the accuracy and relevance of the information used to formulate it • It may change overtime, with the change in stakeholders’ circumstances, interests, and points of views Visioning should therefore be seen more as a learning process that enables stakeholders get together, exchange and analyse their situation than as a final project formulation. On community representation Whenever possible, stakeholders should be involved in the problem definition phase. Depending on the terms of reference for the study, the assessment team could define the problem themselves but should then seek validation by stakeholders. The type of stakeholders involved will differ from situation to situation. The team can start with a limited number of stakeholders and expand to others as the analysis progresses. Stakeholders will emerge either before or after the problem identification phase. Information - Steven C. Ames (2001): What Is Visioning? A Brief Introduction To The "Oregon Model“ "Rebuilding America's Schools“; OCRE Facilitator's Guide; Available at: http://www.ruralschools.org/takeaction/guide/page06.html - Gary Green, Anna Haines & Stephen Halebsky (2000): Building Our Future - A Guide to Community Visioning ; Wisconsin Extension Office; Available at: http://oconto.uwex.edu/files/2010/08/G3708-BuildingOurFuture-AGuidetoCommunityVisioning.pdf - National Civic League (2000): The Community Visioning and Strategic Planning Handbook; Available at: http://www.cpn.org/tools/manuals/community/pdfs/vsphandbook.pdf
Learning Resources for the Assessment of Local Innovation – Concepts and Tools for Local Assessment Teams 168
