Doing the right thing approximately not the wrong thing precisely ...

Overseas Development Institute

Doing the right thing approximately not the wrong thing precisely: Challenges of monitoring impacts of pro-poor interventions in tourism value chains Caroline Ashley and Jonathan Mitchell

Working Paper 291 Results of ODI research presented in preliminary form for discussion and critical comment

Working Paper 291

Doing the right thing approximately not the wrong thing precisely: Challenges of monitoring impacts of pro-poor interventions in tourism value chains

Caroline Ashley and Jonathan Mitchell June 2008

Overseas Development Institute 111 Westminster Bridge Road London SE7 1JD

An earlier version of this paper was prepared by ODI for the SNV/IFC Mekong Private Sector Development Facility (IFC MPDF) Workshop on Measuring and Enhancing Impact in Tourism Value Chains, Cambodia, December 2007. This paper aims to stimulate discussion and feedback is welcome. The authors can be contacted at: [email protected] [email protected] www.odi.org.uk/tourism SNV Asia: Contact Mr. John Hummel, [email protected], SNV Nepal, PO Box 1966, Kathmandu, Nepal. IFC Mekong Private Sector Development Facility: Contact IFC Advisory, Mekong. Mr. Hourn Thy, [email protected], IFC Mekong, 70 Norodom, Phnom Penh, Cambodia. The views expressed in the paper are those of the authors alone and do not necessarily reflect the views of ODI, SNV or IFC.

ISBN 978 0 85003 874 3 © Overseas Development Institute, SNV and IFC. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without the prior written permission of the publishers.

Contents List of Acronyms

iv

1.

Introduction and starting point

1

2.

The Value chain development approach in tourism, its implications and objectives 2.1 Emergence of pro-poor value chain approaches in tourism 2.2 Implications of adopting pro-poor value chain approaches in tourism 2.3 Who are ‘the poor’ in a value chain? 2.4 Potential impacts on the poor (development impact) 2.5 Types of change in value chain performance (intermediate outcomes)

4 4 5 8 9 10

3.

Diagnostic studies done to date

15

4.

Emerging lessons concerning what is measured and how

19

5.

Developing approaches for monitoring impact 5.1 Problem statement 5.2 Diagnostics versus M&E 5.3 Approaches to M&E in enterprise development and value chains 5.4 Emerging framework for M&E 5.5 Monitoring intermediate objectives 5.6 Attributing change to an intervention 5.7 Long-term versus short-term Impact 5.8 Geographic scope of the poor

28 28 28 30 30 34 34 35 35

6.

Conclusion

36

References

38

Annex: Useful reports on M&E, value chains, and pro-poor VCA in tourism

40

List of Tables Table 1: Constraints and opportunities to VC performance Table 2: Typology of pro-poor changes in value chain performance Table 3: Tourism studies using value chain analysis Table 4: Percentage of staff assessed as ‘poor’ in Cambodia, by sector Table 5: Estimates of additional tourist impact on LEI Table 6: Comparison of issues tackled in several pro-poor tourism value chain analyses Table 7: Key questions and indicators for monitoring value chain intervention impact Table 8: Data needs at enterprise and sub-sector level for calculating PPI and pro-poor changes

7 12 16 20 23 27 31 33

List of Figures Figure 1: Figure 2: Figure 3: Figure 4: Figure 5:

Key steps in program design for value chain development Value chain changes that increase pro-poor participation or returns Value chain for prayer beads in Bihar IFC MPDF estimates of PPI by market segment Economic, Socio-Cultural and Environmental/Infrastructural Benefits perceived by residents in Nabji Korhu

iii

5 12 21 23 24

List of Acronyms DFID EV F&B FRW HH IFC IFC MPDF LEI LPB M&E MSE NTFP ODI p.a. PDR PPI PPEI PPST rx SL SMME SNV UNIDO USAID VC VCA

UK Department for International Development explanatory variable food and beverage Rwandan Francs household International Finance Corporation International Finance Corporation Mekong Private Sector Development Facility local economic impact Luang Prabang monitoring and evaluation micro and small enterprise non-timber forest product Overseas Development Institute per annum (Lao) People’s Democratic Republic pro-poor income pro-poor employment income pro-poor sustainable tourism recommendations sustainable livelihoods small, medium and micro enterprise Netherlands Development Organisation United Nations Industrial Development Organization United States Agency for International Development value chain value chain analysis

iv

1

1. Introduction and starting point ‘It is better to be approximately right than precisely wrong’. John Maynard Keynes1 Approaches to tourism intervention are changing. Organisations that have been investing in pro-poor tourism, or community tourism, have found that their efforts remain in a niche and are too marginal in denting poverty levels. Others operating in mainstream tourism are realising that growth in arrivals is not all that counts – more has to be done to deliver impact for the poor from the growth of mainstream tourism. The focus today is on scaling up the contribution that tourism can make to poverty reduction. Amid disappointing and piecemeal data on the impact of pro-poor tourism initiatives to date, the International Finance Corporation (IFC), the Netherlands Development Organisation (SNV), the Overseas Development Institute (ODI), and other organisations are addressing the challenging question of how to scale up impacts of tourism (and their interventions) on poor people. They are increasingly adopting a ‘value chain’ (VC) approach, seeking to intervene at key points in the tourism value chain that can significantly expand income and opportunities for the poor, while working effectively within a highly commercial and sophisticated service sector. The driver of change here is the need to deliver results. Pro-poor policy rhetoric has been plentiful of late, but managers, funders, partners and of course the poor require delivery. Delivering results requires (inter alia) two things: accurate and informed diagnosis of problems and solutions; and an ability to measure impact by comparing baseline data, ongoing monitoring and ex-post evaluation of impact. There has been a recent flurry of pro-poor value chain analysis (VCA) in tourism. This has been mainly diagnostic work to assist in planning interventions, and has developed rapidly in the last two years, through the work of SNV, IFC and ODI. Now these organisations are turning to the complementary question – what information do we need to monitor and measure impact? Experience with diagnostic studies has already generated lessons about data to collect and how, emerging patterns of pro-poor impact, and some implications for design of intervention. Creating a framework for monitoring and evaluation (M&E) of impact creates new challenges, including how to balance the complexity of value chains with the need for affordable and manageable M&E. What information is needed for a baseline that will enable accurate impact assessment in years to come? Just how much detail is needed and is affordable? What indicators best serve on-going monitoring? To what extent should common indicators be used across countries and interventions to aid comparability, or be left to develop in situ? An IFC/SNV Asia workshop in December 2007 addressed such questions and highlighted the need for more work on developing a framework for M&E of pro-poor value chain interventions in tourism. The aim of this paper is to feed into this ongoing process. Enhancing, and thus measuring impact, is at the heart of the current shift in approach. SNV has adopted a value chain approach to a number of productive sectors, including tourism, as part of its increased emphasis on managing for results.2 IFC with partners in the Mekong Private Sector Development Facility (MPDF) is developing a value chain approach to planning tourism intervention in the Mekong, that differs from standard IFC tourism interventions. A detailed study of tourism, including VCA, has been done in order to identify investments that will generate a high rate of return in terms of impact on poverty relative to the investment. From ODI’s perspective, the adoption of a value chain approach to pro-poor tourism has been a natural next step in developing understanding of how small and large enterprises in mainstream tourism can boost pro-poor impact, and how markets in tourism, as in other sectors, can be made more accessible for the poor. 1 2

Courtesy of Conway and Sophal, 2007. SNV has invested considerable resources into developing a new framework for managing results over the period 2007 – 2015. It commits SNV to (1) prove – account for the result of SNV efforts; (2) improve – to learn from the results; and (3) move – to manage towards better results.

2

Different organisations in tourism or value chain development will find different uses for this paper. It is thus worth being explicit as to the starting point that SNV, IFC and ODI share, and thus wish is assumed in this paper. The development logic is as follows:

• •

•

•

The assumed aim is to intervene in the tourism value chain in ways that expand economic benefits and opportunities for poor people. Thus our diagnosis needs to identify where currently ‘the poor’ gain most benefit, and most importantly where there is potential to increase benefit: what kind of benefit, for whom, and how can it be increased? What constraints and bottlenecks in the value chain need to be unlocked? The diagnosis guides us as to where to intervene, ideally by comparing potential impact with cost and feasibility of intervention. This should indicate the likely return, in terms of sustainable benefits for whom. With this we want to design effective interventions that unleash that potential benefit for ‘the poor.’ Baseline data, regular monitoring data, and impact assessment should measure the benefits achieved and prove that we did what we wanted. In addition, we want the ongoing monitoring data to not only ‘prove’ the end result, but also to feed directly into project management to improve it as we go.

So in terms of information, at the start we need to know: The broad outline of the tourism sector, key components; How many poor people, and specifically women, currently engage in tourism and how? Current flows or pro-poor income (PPI) and non-financial benefits to the poor; Main potential PPI increases, given market demand, supply conditions, and policy context; Factors and particularly market failures that constrain greater benefits to the poor; and Feasibility of, and return to, tackling the said barriers. And to assess impact we will later need to know, at the minimum: Any increase in numbers of poor participants, including by gender; Any increase in PPI flows, and to whom; Any increase in non cash benefits or reduction in costs to the poor, including environmental and social costs, and costs to non-participants; Changes in factors that constrain or facilitate participation of the poor; and The extent to which any of the above can be attributed to the programme of intervention. Or why not? As is evident, the focus of this paper is benefits to the poor, rather than wider contribution to national development or macro-economic growth, with an emphasis (though not exclusively) on financial flows, or PPI. This paper is not a detailed guide to the M&E process, nor a checklist of data needs, nor a set of methodologies for impact assessment. It does seek, however, to provide some guidance on key questions to address in measuring the impact of value chain interventions, and the extent to which information needs differ for diagnostics, baselines, and monitoring. But because pro-poor value chain approaches in tourism are relatively new, it focuses as much on the context and intervention logic within which these intervention needs are framed. It is important to explore and understand the value chain approach, the type of interventions and impacts that might result, and thus purposes of gathering information. It is also essential to reflect on how value chain mapping has been used to date, and what lessons can be learnt from the methods applied.

3

The structure of the paper is as follows: Chapter 2 reviews the approach of pro-poor value chain development as it is being applied in tourism. It draws out the implications of this approach for what is done and not done, and for how we can measure results. Before focusing specifically on measurement, it addresses the prior questions of: how do we define the poor? What kind of changes in the value chain might be sought? And what type of impacts on poor people is intended? Chapter 3 briefly reviews value chain analyses that have been done to date, most of which are diagnostic studies. It touches on the information and issues covered, and lessons learned so far about scope, method, and analysis. Chapter 4 focuses on monitoring of value chain interventions, contrasting monitoring with diagnostic work, and proposing six key questions and a set of data needs that should be useful in monitoring most value chain interventions. Caveats are essential. This paper is merely a short-term input on a vast topic. In this time of evolution, learning is continuous. To try to assess monitoring frameworks before we have even crystallised lessons on diagnostic frameworks is brave but bound to be difficult. Anything produced in 2008 is only a start. As projects and monitoring systems are actually implemented, there will be much more to learn. There is a small, but growing and insightful, literature on pro-poor value chain interventions. There is a wealth of literature devoted entirely to issues of monitoring and impact assessment: techniques of appraisal, participatory versus extractive approaches, the project cycle, the problem of attributing causality. These are not explored here, as the paper focuses instead on the specific and new aspects of tourism VCA. But of course this rich literature should not be neglected by those working out monitoring frameworks in tourism. A few recent reports are highlighted in Annex 1. The paper includes candid reflection on VCA work done to date.3 The aim is to move forward, not criticise. We apologise for any errors of fact or interpretation.

3

The fact that we were authors or reviewers in many pro-poor value chain assessments, enables us to perhaps be more open in pointing out their (our) weaknesses.

4

2. The Value chain development approach in tourism, its implications and objectives 2.1 Emergence of pro-poor value chain approaches in tourism Value chains are a way of representing the series of transactions involved in providing a good or service, starting with the provision of inputs for production, and going through production, transformation, marketing to final consumption and subsequent recycling. Value chain analysis (VCA) is a tool that emerged from a number of quite different disciplinary traditions, including the French filliere approach, the need of international businesses to review and adapt their expanding supply chains, and, more recently, concern for the role of the poor in globalisation. So, for instance, when large British supermarkets began pushing their supply chains into Africa and former Eastern Europe, they employed business consultants to work out how these supply chains could function in a way that would deliver high quality produce at the right time to consumers and with a healthy margin for the retailer. But currently a broad range of social scientists including economists, sociologists and political scientists are applying value chain analysis in a development context, as a way of seeing how to maximise the value that can be captured by the poor producer in a value chain. Value chain analyses are not just about numbers (representing financial returns from a sequence of productive processes). They are also a tool to describe the inter-relationships between a range of functional activities, service providers, customers, supporting institutions and supply chains. The political approach to VCA (Gereffi et al., 2005) assesses governance by exploring power relations between the different parties involved in any transaction – a critical issue when looking at barriers to entry for poor producers. The rationale for doing all this assessment is to catalyse action to deliver change, in terms of efficiency, quality or profitability within these relationships. In the tourism field, there have been a few VCAs focused on competitiveness (e.g. Mozambique VCA by FIAS and OECD, 2006). The alternative type, that can broadly be called ‘pro-poor value chain assessments’ have so far concentrated on mapping the value chain and participation of the poor, and particularly estimating pro-poor income4 (PPI) flows. They have paid less attention to classifying types of governance and chain linkages, and thus to the politics of VC change. Tourism value chains are quite different to value chains for manufactured or agricultural commodities (such as textiles and coffee) because of the nature of the product. Tourism is a complex set of complementary services, including accommodation, transport, food, entertainment, cultural heritage, and shopping. Because services cannot be stored, production and consumption of services are usually simultaneous and take place at a specific geographical location – the tourist destination. In tourism, the market (tourists) move to the product (the destination) – the opposite pattern to that observed in conventional product value chains, where a product moves through different stages from primary production through export to final consumption. In primary product value chains, it is quite clear where the poor participate – usually in the primary production and initial processing of, for example, coffee beans or garment making. But in tourism, one of the reasons for undertaking VCAs has been to explore the different points in the chain where the poor participate, given the number of ‘sub-sectors’ or chain components that comprise the tourism product. In the past, pro-poor interventions in tourism tended to focus on boosting specific tourism products, such as community tourism, or addressing the most exploitative forms of tourism. Mapping the entire value chain has helped to demonstrate the need to work in the mainstream tourism economy, not just a niche, and provides a basis for identifying the most effective points of intervention for pro-poor impact. The shift to a pro-poor value chain approach among international tourism practitioners is driven by the need to scale up impact on poverty, beyond a few high-input local projects. It also stems from 4

PPI = pro-poor income is a shorthand for US$ per year flowing to poor participants in the value chain.

5

recognition that a major focus should be on helping the poor access international markets via mainstream tourism. SNV has adopted an approach known as Value Chain Assessment and Development, not only in tourism but in other productive sectors. The aim is to develop market-based solutions that boost production, income and employment opportunities for the poor through improved inclusion in global value chains (SNV, 2008). Figure 1 shows the basic 6 steps, and the inclusion of monitoring and evaluation of performance (step 6). Figure 1: Key steps in program design for value chain development

The traditional approach is to jump from Step 2 to Step 5, without investing in 3 and 4.

Source: Action For Enterprise

IFC operates at an altogether different scale, but with similar logic. For IFC in the Mekong, the aim is not just to invest in tourism development per se, on an assumption that tourism growth is good and more growth is better. Instead, IFC wants to identify how and where to invest in the tourism value chain, so as to deliver maximum impact for development, relative to the donor injection. 2.2

Implications of adopting pro-poor value chain approaches in tourism

There is sometimes confusion between value chains as a type of analysis, and value chains as the focus of an intervention. While the reports that get published are the value chain analyses, the programmes implemented are increasingly about altering the performance of the value chain. Box 1 highlights the difference.

6

Box 1: The difference between value chains as a tool and as an objective As a tool: ‘value chain analysis’ (VCA) is simply a tool for enhancing our understanding. Pro-poor VCA-type analyses, for example as done in LPB, The Gambia, Kratié and Route 9, focus on pro-poor issues, mapping flows to the poor and linkages between actors in the value chain. Alternatively VCA can focus on analysing sector competitiveness without delving into participation by the poor. VCA can inform many different kinds of intervention whether or not their overall aim is to enhance performance of the chain. As an objective: pro-poor ‘value chain interventions’ aim to intervene at key points in the value chain so as to change how they operate, and improve the performance of the chain from the perspective of the poor. Many different tools can be used in diagnostics and M&E, including but not limited to, value chain analysis. In this VCs are no longer just a tool. ‘Value chain development’ or ‘enhancement’ become core to achieving impact.

The adoption of value chain development approaches means much more than using value chain analysis as a mapping tool – a way to picture the sector. It means redefining the aim of intervention as one of making the value chain work better for the poor. This has three implications: (i)

The first is that a good understanding of the current value chain is needed, both to act as a diagnosis (to determine what to do) and as a baseline (for measuring future impact) – hence the flurry of VCA-type diagnostic studies.

(ii)

The second is that all options are open. There is no prior assumption that a certain kind of tourism intervention is best for the poor. You do not just focus on the specific products that are currently produced by the poor (such as cultural tours). Instead, you understand the overall sector, the linkages between players, different places where the poor fit in and obstacles that prevent participation, what revenue flows down to them and what their earnings depends upon.5 Resulting interventions may be at any point in the chain, in any sub-chain and with any stakeholders, if they increase access and returns for the poor.

(iii)

Thirdly, an outside intervention cannot just reshape a value chain. What an intervention can do is remove obstacles that enable the value chain actors to engage differently, and in this way enhance VC operation in a pro-poor way. Thus intervention will normally be targeted at constraints, or market failures. Diagnostic analysis thus needs to identify where in the value chain to focus, and also which obstacles to address. Table 1 provides a classification of different types of constraints addressed by Action for Enterprise in VC intervention.

Given that the whole point has been to scale up impacts on poverty, this also implies: Careful monitoring of poverty impact is essential. But VCA as a tool is not sufficient on its own for most VCA interventions. Tracking shifts in the performance of the value chain is essential for capturing actual change in, for example, poor people’s participation levels and their returns, and how they relate to other actors. But VCA analysis is not satisfactory for capturing other issues such as social and environmental costs and benefits to the poor or to society, dynamic impacts of tourism on the local or national economy. Therefore, other tools are needed.

5

See ODI Briefing Paper ‘Assessing how tourism revenues reach the poor’, June 2007: http://www.odi.org.uk/publications/briefing/bp_june07_tourism_vca.pdf

7

Table 1: Constraints and opportunities to VC performance Type Technological / Product development

− − −

Market access

− − −

Organisation and management

Regulatory (Policy)

− − − − − − − − −

Finance

− − −

Input supply

− − −

Infrastructure

−

Value chain constraint/opportunity small-scale farmers lack access to appropriate tools and machinery (technologies) which decreases their yield opportunity for equipment manufacturers to offer leasing of tools/machinery to farmers and thereby improve product quality lack of technical skills of growers to produce to European buyer specifications reduces their income and market access lack of linkages to large buyers decreases sales potential of producers lack of information on standards reduces farmers ability to produce to buyer specifications lack of marketing organisations or brokers limits market outlets for small enterprises high transportation costs increases the price of farmers’ production inability of producers to organise for economies of scale limits their opportunities to access higher value markets micro-enterprises lack ability and time to conduct accounting micro-enterprises lack skills to develop business plans high rejection rates result in loss of income for producers and buyers import taxes on inputs increases producer costs artificial price subsidies prohibit the emergence of small-scale producers export tariffs increase exporter costs and decrease global competitiveness of the value chain lack of government contracting procedures that favour micro-enterprises reduces their opportunity to engage in public sector bids farmers are unable to pre-finance improved inputs opportunity for exporters to access commercial funding and increase their purchases from small-scale producers inability of farmers to provide adequate collateral decreases their access to working capital loans high prices of inputs restricts use by small-scale producers use of poor quality raw materials by small enterprises results in inferior products unable to meet market demands farmers in remote rural areas lack access to inputs which reduces productivity poor roads, electricity, refrigeration facilities, telephones, etc., increases the price of the final products and makes competing with imports more difficult

Source: Lusby and Panlibuton (2007) Value Chain Program Design: Promoting Market-Based Solutions for MSME and Industry Competitiveness. Action For Enterprise (AFE).

It is worth being explicit on how a VC approach leads to a set of interventions that is likely to be quite different from previous standard practice, because this affects our approach to gathering information. It will vary by agency background, but to give an example in the case of SNV, the VCA approach alters: With whom SNV works: i.e. much stronger emphasis on private sector partners, tourists themselves, and a range of market actors that can provide market-based solutions, compared with traditional entry points with state and community actors; Where SNV works: i.e. less in rural villages and more where tourists are found, including mainstream tourist destinations; Types of project: less justification for small, isolated, product-orientated projects but more facilitating, brokering and adding value to existing processes; The evidence basis needed for interventions: interventions need to be justified by addressing a specific market failure that holds the poor back – and not just by their

8

distributional (e.g. reaching marginalised communities) and/or inspirational (e.g. increasing responsible tourism) qualities; Skills required: these different types of project will require brokerage and ‘deal making’ strengths – which are different from the skills required to manage community-based tourism projects; and The amount and type of information needed for assessing impact: capturing changes in the performance of a complex value chain, and how these affect the poor, is a challenging task. The intended development outcome of pro-poor value chain interventions is invariably similar: that tourism develops in a way that enables the incomes or numbers of poor participants to increase, and that other non-financial (social, environmental and dynamic) net benefits are increased. The means of doing this may be very extremely varied. Intervention can involve:

• • • • •

Working directly with poor producers to help them upgrade their product and better match demand; Working with hotels, tour operators, ground handlers, on their demand, procurement systems, pricing, so they can work more effectively with poor entrepreneurs; Working with local or national government on regulations and business conditions, to remove blockages to micro entrepreneurs; Engaging with tourists and those doing tourism marketing, to influence tourism behaviour and spending patterns; and Working with many different stakeholders in the tourism value chain to enhance communication and commercial linkages.

It is clear that what starts as a relatively simple intervention logic based on VC enhancement, soon leads in practice to an array of options that pose problems in terms of the information overload needed for diagnostics and monitoring. It also makes it hard to develop a single M&E framework that can fit intervention approaches as diverse as increasing length of stay in a destination, enhancing access to credit for small transport operators, building up capacity of provincial staff to develop a destination plan, and encouraging commercial operators to adapt their supply chain sourcing. We have established that enhanced performance of the value chain is central to achieving results in the new approaches. But what is meant by ‘the poor?’ And what exactly are the changes in the value chain and the possible impacts on the poor that we need to measure? These issues require exploring before the detail of value chain analysis and M&E requirements can be considered.

2.3 Who are ‘the poor’ in a value chain? One problem with a pro-poor VC approach is that it does not define the poor for you. Conventional approaches that began with a target community, such as rural village, or women crafters, and helped them to develop tourism products, did not face this problem. But the VC approach means starting with an assessment of the tourism economy, and working out where the poor already operate and where their participation can be increased. Thus inevitably the first question is ‘who counts as poor?’ The answer to this question affects everything: the results of diagnosis or impact assessment, the calculation of PPI, which sub-chains appear most pro-poor or most promising, whether the destination itself ranks well or poorly compared to others in pro-poor impact. Tourism is not a means to reach the poorest of the poor. It is a commercial industry offering opportunities for the economically active, and often supplying a livelihood that keeps families just above the poverty line. Taking a very narrow definition of ‘poor’ will mean that tourism appears not to benefit the poor at all. This puts all the policy focus back on to macro economic impacts such as foreign exchange. Taking a very wide definition of poor would encompass the majority of entrepreneurs and

9

staff in the sector, and provides an easy way to avoid having a specific pro-poor focus on assisting the disadvantaged. A middle ground, or a dualistic approach covering both, is essential but not easy. In most countries the national poverty line is much lower than the international poverty line (US$1 per person per day at 1995 purchasing power parity). So which should be used? Take the example of Rwanda. The Rwandan Government’s own upper poverty line (consumption of 250 FRW per adult per day, equivalent to just under US$0.50 per person) means that a breadwinner earning around $68 per month would just keep a family of five out of poverty. This is roughly the amount earned from an unskilled job in a mid market hotel in a rural area. If instead we take the international poverty line, this translates into a monthly income for a household of five of US$150. With this boundary, even reasonably well-skilled and well-paid non-managerial staff in tourism (such as waiters in a 4 or 5 star Kigali hotel, who earn more than teachers) can be included as ‘poor’. If we use the even higher $2 a day international poverty line, even the headwaiter’s family could count as poor if his is the sole salary. Although it would be ideal if we had one standard definition that could be applied in all destinations to allow comparisons, in fact, the appropriate definition varies by purpose of analysis. Discussion of ‘propoor tourism’ started with the assertion that tourism should not be shunned by development professionals, but should instead be harnessed. It was argued that it could be more pro-poor than other sectors, partly due to its labour intensity and employment of unskilled and female workers. When taking this perspective, considering the overall distributional impact of growth in tourism compared to other sectors or types of growth and engaging with macro-economists, it probably is appropriate to take a broad definition of poor. Thus from IFC’s perspective, it is useful to consider the ‘poorish’ as well as the ‘poor.’ On the other hand, take the example of SNV trying to identify key areas to engage for propoor impact along Route 9 in Lao PDR and Vietnam. The new VCA (Travers, 2008) is finding that the bulk of pro-poor flows are from employment in the accommodation sector. But as Travers points out, people with jobs in the service sector, earning on average $88 a month in Hue and $63 in Quang Tri are not poor compared with those working the land. In this instance, defining areas for SNV to leverage additional pro-poor impact, a focus on those earning less than $60 per month may be appropriate.

2.4 Potential impacts on the poor (development impact) Development organisations use different terminology for identifying levels of impact. For some the ultimate level, involving changes in the lives of the poor, is termed ‘impact’ while for others it is ‘outcome’.6 In this paper we will focus on: Changes in the functioning of the value chain and performance of actors, which are intermediate outcomes; and Changes in incomes and livelihoods of the poor, or other development objectives, which represent ultimate development impact. Starting first with impacts on the poor, a change in the operation of the value chain can benefit poor people in three different ways. These developmental impacts can be any or all of the following: Increased access to the tourism value chain for more poor people, thus creating new entrants – this may be by removing barriers to entry to the existing value chain, developing new links into the value chain, or expanding sub-sectors of the chain or indeed the total sector size;

6

SNV’s new strategy refers to: impact – the improvement of good quality basic services and income, production and employment and the related improvements in the well-being of poor people; outcome – improved performance of (groups of) client organisations and sector as a whole, in term of delivery of basic services and value chains for the poor, and its related improved enabling environment; and output – the quantity and quality of SNV services. By contrast, IFC refers to ‘development outcomes’ as the ultimate level of impact.

10

Increased income of the existing poor participants in tourism – this may be as a result of increasing their returns on current activities, helping them upgrade to new more profitable activities, or expanding their volume of sales; and Increased net non-financial benefits to poor households – such as reduced livelihood risk, stronger local institutions, cultural pride, greater gender equity, improved natural resource management or social capital, and dynamic changes in the local or national economy. The third may be caused by the first two but may also be achieved quite independently, and impact different poor groups. The rest of this paper focuses mainly on the first two impacts, and financial estimates of PPI flows. This heavy focus on economic participation is for three reasons. Firstly, although a narrow focus on PPI and numbers of poor people is too limited it does represent the common core of the new pro-poor VC approaches. Organisations vary in the extra factors that they prioritise: reaching particular target groups (women, minorities, the poorest), reducing social or environmental costs, building institutional capacity, generating a demonstration effect to influence others, generating long-term multiplier effects for the local economy, balancing returns to national economic growth and pro-poor income. Thus the trade-offs with other impacts will need to be addressed slightly differently each time. Secondly, the non-financial impacts will need to be measured separately, and then balanced against PPI flows. Interventions that generate reasonable returns in both areas can be compared against returns that are highly positive on one but negative on the other. We are not recommending a methodology that integrates all variables into one quantified result (as cost benefit analysis tries to do). Thus the approach outlined here for understanding the financial side does not change with the additional social, environmental or distributional perspective. The approach remains valid, but will need integration with other results. Our third reason is that it is in the identification and measurement of PPI and poor participation that most innovation has happened recently, and needs to be captured. Nevertheless, we indicate various points at which broader livelihood or environmental issues can be addressed.

2.5 Types of change in value chain performance (intermediate outcomes) What changes in the functioning of the tourism value chain might we be looking for? There are a wide range of types of interventions that can be seen to enhance a tourism value chain for the poor, ranging from integration of an entirely new product provided by poor entrepreneurs, to increasing total bed availability to expand all element of the value chain. Four different changes that can help current poor participants increase their earnings:

(i) Expansion of existing production activities: higher demand leads to higher price or volume of production. For example, as hotels thrive, seasonal workers gain permanent jobs and/or higher wages; (ii) Improvement of existing production: producers increase their profits or reduce their risks via improved quality and productivity, perhaps through investing in new techniques or equipment – for example, pavement food vendors add new product lines or a cool box to their offering; (iii) Greater contractualisation leading to changed terms of existing relationships i.e. higher prices or greater security – for example, farmers may form collective agreements with restaurant buyers and strengthen their bargaining position;7 and

7

This typology draws on, but differs from, a typology developed by Riisgard et al. (2007), in which the five upgrading strategies are (i) entering the chain; (ii) improving on existing production activities; (iii) adding value by taking on more functions; (iv) increasing contractualisation; (v) coordinating a chain segment. The last of these is not a separate category in our typology (it can be part of our 3 or 4) and we have added several others, particularly related to new entrants.

11

(iv) Moving up the value chain to create more value-added and enter higher-return activities – for example, silk weavers may start undertaking transport or retail. The second, third and fourth changes may combine together: informal vendors may organise themselves into an association, upgrade their product offering and language skills, and embark on new forms of marketing, all at the same time. There are a further four different changes that can assist new entrants to enter the value chain:

(v) Expansion of the overall tourism value chain, so that in all activities, more opportunities for the poor exist – for instance, year on year growth in arrivals requires more of all inputs;

(vi) Expansion of specific parts of the tourism value chain which are relatively important to the poor – for example, development of a new market expands the share of spending spent on locally produced artisanal products; (vii) Introduction of new links in the value chain, which create new opportunities for poor participants, such as developing a new cultural tour which is operated and hosted by poor households; and (viii) Reduction of barriers to entry so that poor people are able to access opportunities previously taken by the non-poor – for example, training of rural youth so they can qualify as registered guides or for hotel employment. Changes five, six, and seven are all very similar – broadly an expansion of value chain activities creating new openings. Change eight is quite different in that it is shifting access from non-poor to poor and thus changing the distribution of opportunity. There is some overlap between these two sets of changes: as existing producers move up the value chain (Change 4), opportunities for new entrants will expand. If parts of the value chain expand, this may both attract new entrants (Change 6) and generate higher returns (at least short- to medium-term until supply responds) for existing participants (Change 1). Some reforms in the operating environment have potential to expand opportunities for the poor in many ways, including the above, but are not specific to one kind of value chain change. Important examples would be development of new infrastructure (roads, markets, communications), financial services (banking facilities for tourists and small, medium and micro enterprises (SMMEs)), institutional development and capacity building. Figure 2 depicts the main options, and whether they are primarily increasing returns to the poor or increasing the number of opportunities, or a combination of the two. Table 2 shows how these eight changes/strategies can be grouped into three main areas of focus: expansion, increasing returns to activities of the poor, and removing barriers to participation. It also adds an important fourth area of activity, which is increasing benefits of non-participants. It gives examples of each, indicating whether they are likely to increase both the number of opportunities available to the poor and the financial returns to those who already participate.

12

Figure 2: Value chain changes that increase pro-poor participation or returns

Table 2: Typology of pro-poor changes in value chain performance Type of change in the tourism value chain (with cross-reference to Changes 1 to 8 above)

Example of an intervention

Increase income of current poor participants

Enable new entrants to enter the tourism value chain

Of the overall chain (Changes 1 and 5)

Supporting the Da Nang government in Central Vietnam to better manage coastal land supply to facilitate emergence of new luxury coastal resorts

likely

9

Expansion or creation of products/services heavily provided by the poor (Changes 6 and 7)

Developing new cultural tourism products, like the Outback Safari excursion pioneered by First Choice Holidays in the Dominican Republic

likely

9

possibly

Expansion:

Higher returns to existing production: Upgrading existing production by the poor (Change 2)

Improving the quality of service (and returns enjoyed) by juice sellers and local guides. An Association of Small Scale Enterprises in Tourism (ASSET) initiative in The Gambia

9

Contractualisation of existing producers for enhanced terms (Change 3)

Sandals resort initiatives in Jamaica to contract farmers ahead for vegetable supply.

9

13

Type of change in the tourism value chain (with cross-reference to Changes 1 to 8 above)

Example of an intervention

Increase income of current poor participants

Enable new entrants to enter the tourism value chain

Removing barriers to entry/progression; Existing producers move up to higher value-added functions (Change 4)

Investment in staff training so that local staff move from unskilled cleaner to departmental supervisor

9

likely

New entrants able to access previously ‘nonpoor’ activities (Change 8)

Affirmative hotel procurement to source from emerging business, such as outsourced laundry at Spier estate in South Africa. Helping poor farmers sell into the food and beverage (F&B) chain

likely

9

Enhancing the enabling environment and value chain linkages: Development of financial services that enable entrepreneurs to invest and/or tourists to spend more locally

SMME finance facilities that enable long-term supply contracts to be used as collateral for loans

9

9

Mechanisms for large and small enterprises to exchange information and market intelligence

New fora for chefs and farmers to discuss future supply and demand in an Oxfam Caribbean initiative

9

likely

Boosting benefits to the wider community: Development of infrastructure and services used by residents

Road improvements around Parc National des Volcans in Rwanda, linking farmers to town

Social organisation and institutional capacity

Community conservancies develop institutional capacity in Namibia via their decision-making roles in tourism

Preventing natural resource damage or cultural intrusion by tourists

Codes of conduct for tourist behaviour drawn up by residents and disseminated by hotels and tour operators

A mixture of non-financial and sometimes financial impacts for residents, including many who are not direct (economic) participants.

Reality often does not fit quite so neatly into these categories. For instance, the night craft market in Luang Prabang (LPB) in Lao PDR in South East Asia has actually generated many different types of change in the value chain, as shown in Box 2. This example illustrates some useful points for our planning of baselines and monitoring. Firstly, there is no point in being rigid about categorising interventions according to the different types of VC change. Boundaries are fluid. The reason for having these is to remind us not to ignore any of the changes, and thus to be ready to capture them in baselines and monitoring.

14

Box 2: Multiple types of value chain enhancement as a result of Luang Prabang Night Market

• First, traditional craft producers (primarily Hmong women villagers) have upgraded their returns to existing production (Change 2 above). Some have enhanced the quality of design of fabrics as they combine innovation with better knowledge of international tastes.

• Secondly, some have upgraded their position in the value chain (Change 4) now employing neighbours to do piecework tailoring (e.g. sewing the embroidered work into a duvet cover), and some now sell directly to tourists in town rather than relying on traders or shops.

• Thirdly, as the Night Market has become an unmissable part of the ‘LPB Prabang destination’, it seems clear that craft shopping and spending will have increased as a proportion of the overall tourism value chain (Changes 1 and 6). In 2006, craft expenditure alone accounted for some $4m, or 15%, of total tourist spending in the destination of $24m. The night market is not just a substitute for ad hoc sales in villages or sales via shops, but is a way to stimulate greater shopping and total craft spending by tourists. Both existing and new entrants benefit from this expansion.

• Fourthly, associated with the Night Market, new services that are strongly pro-poor have developed such as pavement kitchens around the Night Market area. These new services bring additional benefits to the poor (Change 7).

• Finally, the goods and services offered by the poor to tourists in LPB have become an integral part of the tourist product and brand for the destination. Craft is not only allowing poor people to gain a decent slice of the tourist ‘cake’ (estimated at about 25% in 2006), but is also expanding the ‘cake’ itself (Change 5). Overall, the successful craft chain in LPB is estimated to generate financial flows into poor communities of around $1.8m per year (Ashley 2006). We do not know how much of that $1.8m could be attributed to the development of the Night Market, but probably a fairly significant share.

Secondly, it is clear that there are not necessarily trade-offs between raising the incomes of current participants and involving new entrants: the successful Night Market has achieved both. Similarly it demonstrates synergy between expanding incomes of the poor and increasing the overall tourism ‘cake’. Existing assumptions and actual trade-offs need to be explored in situ through diagnostic monitoring. Finally, it is essential to capture incomes that flow to the poor via the supply chain, and not only from direct tourism expenditure. The greatest potential for expanding pro-poor impact of crafts in LPB does not come from further expansion of sales (for which there is probably little potential), but from increasing the share of silk that comes from farmers in Lao PDR, above the current estimated 50% (Ashley, 2006). Pro-poor improvements at the very bottom of the value chain may not affect the tourism product at all, but can have a big affect on poor incomes. We have found that in some destinations, about half of the pro-poor benefit of tourism is derived from indirect linkages (Mitchell and Ashley, forthcoming).

15

3. Diagnostic studies done to date The pro-poor value chain approach so far The majority of pro-poor VCAs done so far have been to assist in designing and planning interventions, rather than for providing baselines or assessing impact. They represent a substantial shift from previous diagnostics approaches, as highlighted in Box 3. Box 3: How VCA diagnostics go beyond previous tourism project planning approaches Classic approaches to planning tourism projects are quite different from the new generation of pro-poor value chain interventions: The traditional approach or ‘boosterism’ is based on the assumption that the contribution of tourism to national economic development is a function of the size of the sector. It follows that growth is good and rapid growth is even better. Project design is based on growth trends and assessment of the bottlenecks to sector growth (e.g. air lift, business climate etc.). In more refined approaches, a commercially-orientated value chain analysis (as was done for Mozambique by FIAS and OECD, 2006) that identifies constraints to sector competitiveness might be done to pinpoint a range of bottlenecks. This approach is usually about growth, not distribution, and does not include pro-poor measures. Conventional project appraisal methodology is based on cost benefit analysis, in which the economic effects of tourism, including multipliers, are compared with investment costs. The approach is simultaneously strengthened and undermined by a strict adherence to conventional economic theory. The benefits of this approach are that a clear market failure has to be identified to justify an intervention and there is a focus on assessing the full economic impact of tourism throughout the economy via indirect and induced effects. In addition, non-financial variables can be included in a cost-benefit analysis. The weakness of this conventional approach is that, even though some distributional assessment can be ‘bolted-onto’ the main analysis,8 there is a failure to measure benefit flows to the poor. The reason for this failure is conceptual. A cornerstone of conventional economics is that the goal is to increase the aggregate welfare of society as a whole. From this perspective, the notion of prioritising benefits to one group (like the poor) is not only unnecessary; it dilutes the potential benefits of a project. Community-based tourism projects have, at the other extreme, been driven by distributional objectives – the desire to assist a specific poor community – but with the lack of an economist’s attention to opportunity cost (i.e. whether intervention of donor resources elsewhere in the chain could deliver greater pro-poor impact, and whether the substantial investment likely to be needed is justified by the community’s returns). These projects have also suffered from a lack of attention to demand for the product and the functioning of the wider market. Projects that could be classed as nascent ‘tourism development with a pro-poor element’ have emerged recently, such as the IFC-supported MPDF in the Mekong, and an emerging World Bank project in Ethiopia. These are supporting development of the overall sector, but with specific attention to obstacles constraining greater participation by the poor. In the MPDF case, a wealth of data on local income generated by different tourism products and segments complements the more standard project planning information base. In Ethiopia, the value chain analysis conducted was not strongly focused on pro-poor issues, but a partial VC at one destination, Lalibela, clearly demonstrated the lack of local spending and linkage. A specific ‘linkage’ component is now being designed in tandem with the Ethiopian Sustainable Tourism Development Program.

Table 3 summarises most of the VCAs done so far, in terms of their purpose and findings. Table 6, below, will look in more detail what information or analysis is covered.

8 Tun Lin & Franklin Guzman (2007) Tourism for pro-poor and sustainable growth: Economic analysis of tourism projects Economic & Research Department, Asia Development Bank Technical Note Series No.20

Table 3: Tourism studies using value chain analysis The study Country/destination What, who, when

Main purpose of the VCAtype work

What was covered in the mapping/VCA

Examples of main findings

Examples of policy implication for poverty impacts

Lao PDR (Luang Prabang) Local economic mapping of tourism in LPB

Diagnostic: Identify opportunities for further propoor intervention by Provincial Government and SNV.

Estimated tourism expenditure within LPB in 4 sub-chains (accommodation, food, handicrafts, and transport/ excursions) and flows to the poor (PPI) within each.

Total direct and indirect earnings of the poor equivalent to 27% of tourist expenditure in the destination.

Explore food chain expansion with agriculture; protect and enhance shopping experience; assist rural entrepreneurs to capture revenue but do not rely only on rural tourism for propoor impact.

Analysed the high and low season value chains, returns to each stakeholder from total tourist spending (package plus discretionary), and incomes to the poor from each part of the value chain.

Over half of total tourism spending is captured in-country, of which about 14% is earned by the poor (mainly via retail sales, food supply, hotel jobs).

Tourism demand and trends. PPI from key sub-chains (accommodation, food, shopping, transport).

At least 26% of destination expenditure flows to poor people. Main poor groups are hotel/restaurant employees, tourist sector enterprises; and local crafters and farmers.

SNV/ODI (Ashley, 2006) The Gambia Analysis of the Gambian tourism value chain and its pro-poor impact.

Diagnostic: Assess tourism poverty linkages in The Gambia and advise on how to enhance pro-poor impacts.

ODI/ Commonwealth Secretariat (Mitchell and Faal, 2007) Vietnam Da Nang (Mitchell and Le Chi Puc, 2007)

Diagnostic: Assist local stakeholders to assess how to promote local economic development by harnessing tourism.

16

Earnings via the food chain are largest, crafts are second.

Higher-than expected linkages result from high out-of-pocket expenditure, and supportive action for the informal sector.

Expand total sector size while further strengthening local supply of food, boosting hotel sector wages, maintaining local retail sales and developing more ‘local excursions’.

Growing the tourist sector by encouraging upmarket beach resorts and increasing length of stay will increase pro-poor benefit flows the most.

The study Country/destination What, who, when

Main purpose of the VCAtype work

What was covered in the mapping/VCA

Examples of main findings

Examples of policy implication for poverty impacts

Ethiopia Value chain analysis of Cultural Heritage Tourism in Ethiopia

Diagnostic: to assess challenges in the tourism supply chain and make recommendations (rx) for a more competitive and sustainable tourism sector.

Allocation of expenditure for 4 different types of tourists (across accommodation, transport, food, excursion and overhead). Not PPI. Bottlenecks, international comparisons.

Discretionary spending is low, as is accommodation spend. Tour operator margins are high due to high risk.

Nothing specific on poverty impact.

Analysis of tourism to propose a viable strategy for growing tourism while supporting Government goals for poverty reduction.

Low level of local income flow. Survey of enterprise revenues in Lalibela. International comparisons from tour operators.

Nabji-Korphu Trek: The pilot community-based nature tourism project in Bhutan. Department of Tourism (Dorji, undated)

Impact and baseline: assess emerging impact of Year 1 of the trek. Make rx for future. Lay baseline. Develop method of assessment.

Mapping PPI to local people per trek was one small part. Also: tour operators’, tourists’ and residents’ views. Economic, social and environmental change.

Positive local impacts in Year 1: almost US$4,000 in local earnings, new enterprises started. No negative environmental and social impacts so far. In season 2, over $10,000 in local earnings.

Enhance local benefits via e.g loans, fruit and vegetable extension, sexual awareness education, craft branding, water supply, new institutional systems. Also rx to enhance product for tourists and tour operators.

Route 9 (Lao PDR / Vietnam)

Diagnostic and baseline: identify and quantify the current tourism market and impacts on the poor; give strategic advice to SNV on Pro-poor Sustainable Tourism (PPST).

Inventory of tourism supply and demand. PPI flows in accommodation, restaurants, shopping and excursions. Policy context.

In Lao PDR, PPI is low – $300,000 p.a. Main generator is the multiple restaurants. In Vietnam, PPI is around $5.36m p.a. but lower than potential. Hotels generate most PPI. PPI is low for poorest groups, particularly minorities.

To tackle problems of ‘corridor tourism’, need to broaden the product offering. Deepen supply chain, via SME and craft development, and particularly by involving minorities.

World Bank (GDS, 2006) and Towards a Strategy for Pro-Poor Tourism in Ethiopia World Bank for Government of Ethiopia (Mann, 2006)

SNV (Travers, 2008)

17

Weak and shallow supply chains due to multiple constraints (regulation, low quality and availability, and low discretionary spending).

Development of tourism demand and supply should include measures to boost local spending and linkages, such as support to handicrafts, local infrastructure, money changing facilities.

The study Country/destination What, who, when

Main purpose of the VCAtype work

What was covered in the mapping/VCA

Examples of main findings

Examples of policy implication for poverty impacts

Rwanda

Diagnostic: develop an overview of the tourism VC and PPI to make strategic rx for PPST, including further needs for baseline and diagnostic information.

PPI from business and gorilla tourism enterprises. Comparisons of business model and tour structure.

Importance of business tourism. Food chain is big but does it involve the poor? Very high level of donations. PPI depends on business model of hotel/lodge, and whether cultural tour is included.

Explore how to enable poor farmers to access hotel food chain. Encourage diversified product (supply and demand of cultural interaction). Encourage pro-poor business models including joint venture.

Diagnostic: quantify spending, domestic earnings and pro-poor flows from tourism; identify priority markets in terms of yield. Make strategic rx to Government on interventions with high development return.

Detailed calculation of yield (spending and profit per tourist), local economic impact (net of leakages) and PPI by types of tourist (per trip and per day) chain segment, and location. Poverty profiling of participants.

Overall share of expenditure reaching the poor via salaries (only) is low (PPI = 3 to 7% of spending, lowest in Siem Reap (primary destination), highest in Kratié (secondary destination). Group tourists generate more propoor employment income (PPEI) per night than independents. Spending in restaurants is only 12% of expenditure. High PPIgenerating markets include China, USA, France and UK. Ranking varies per trip, bed night, or year.

PPI will increase with growth, particularly visits to secondary destinations and visits by highPPI-yielding tourists. It is possible to boost PPI with less environmental impact by targeting high-PPI markets. Invest in training and enterprise, and in key market segments with PPI impact.

Distribution of expenditure of different types of tourism across the chain, from accommodation to foreign commission.

Competitiveness barriers include: costs of doing business; transport weaknesses.

None specifically on poverty impact.

SNV (Ashley, 2007)

Cambodia (Siem Reap, Phnom Penh, Sihanouk, Kratié) (IFC/MPDF, 2007) (IFC/MPDF, forthcoming)

Sector competitiveness VCA Mozambique Prepared for Mozambican government. (FIAS/OECD, 2006)

Diagnostic: the constraints and challenges that undermine growth of tourism in Mozambique. Aim: to increase Mozambique’s share of value added in tourism.

Factors that affect competitiveness, gaps to address.

18

Most inputs and skilled labour are sourced from abroad, handicraft spending is a fraction of that of Kenya/Tanzania.

Recommendations are on enhancing investment climate and improving competitiveness (e.g. visa, transport, marketing and investment issues).

19

4. Emerging lessons concerning what is measured and how A thorough overview of the methodology for doing a diagnostic VCA has not yet been written, though we are probably now approaching the point where a first draft could be done. This is not the place to document all the information needs of a diagnostic and all the methodological lessons learnt to date. Nevertheless, it is useful to draw out some pointers. The common feature of all VCAs is the tracking of income flows to the poor, or mapping PPI. While this is insufficient on its own, and there is a risk of excessive attention or effort going into PPI, this is a key element. Most VCAs have calculated PPI as a combination of wages paid to poor/low-paid or unskilled/semi -skilled staff in an enterprise, plus a share of the cost of inputs that can reasonably be assumed to accrue to poor producers, vendors, or raw material suppliers. While the latter estimates are very rough, they make a significant difference to PPI estimates. It is important to note that the IFC MPDF study in Cambodia uses a narrower definition of pro-poor income, more correctly termed Pro-poor Employment Income (PPEI). It measures only the wages accruing to staff who come from a poor background. It does not include a share of supply costs that might accrue to farmers or craft-makers, and does not include wages of staff who are low-paid but not from a poor background. In most, this PPI assessment is not just done per enterprise, but is aggregated up to calculate current PPI per sub-chain (or per ‘supply chain, meaning for example, the craft sector, the restaurant sector etc.) This is a major distinguishing feature of these VCAs compared to many other project appraisals in the past: invariably some assessment of income to the poor from a tourism enterprise is included, but not from the overall restaurant chain, or craft chain. Aggregating the data up to sub-sector level requires good comparable data, which is not always available. Ideally, enterprise analysis should generate different measures of PPI-per visitor, bed night, available room, meal taken or dollar of spending – as well as total PPI per year for an enterprise of certain size and type. Aggregation can be done by multiplying by the number of enterprises of that particular size and type, as well as by multiplying the total number of rooms, meals, bed nights, dollars spent, or visitors of that type. In Siem Reap, the aggregation has been done carefully with hotels grouped into several different types, and aggregation done both by PPI per enterprise and PPI per room, so that any discrepancy can be explored. Other VCAs have taken short-cuts: in LPB, the ratio of PPI to tourist spending at enterprise level was applied to aggregate tourist spending on that type of enterprise. In Rwanda, the lack of an inventory of enterprises of different types prevented any comprehensive aggregation. Only a very rough estimate, based on PPI per bed night in hotels and lodges could be done (which in turn relied on rough estimates of total bed nights in Rwanda). VCAs all face the problem of defining the poor. The LPB and Da Nang VCAs took a wide definition which was unskilled and semi-skilled (SS&US) participants, and let the nature of someone’s work define them as poor. Thus in LPB, all unskilled hotel employees, craft vendors, farmers and rickshaw drivers (probably the ‘top’) of the SS&US group counted as poor by definition. Most VCAs have since adopted a similar approach to letting job function define poverty. The IFC MPDF approach in Siem Reap and Kratié marks a step forward by using poverty profiles of workers, to assess what percentage of each functional group come from a poor background (e.g. 41% of hotel workers in Kratié come from poor backgrounds, based on poverty indicators such as roofing). This has a big impact on the resulting calculation of PPI. Table 4 shows the results across Cambodia, with workers who come from a poor background accounting for around half of employees – less in transport, and more in the souvenir sector and guest houses.

20

Table 4: Percentage of staff assessed as ‘poor’ in Cambodia, by sector

Sector

# of staff employed

Estimated number of staff from poor backgrounds

as % total number of staff

Earnings of poor staff as % total salary bill

Hotel

12,628

5,821

46%

22%

Guest house

3,489

2,346

67%

53%

Restaurant

8287

3,762

47%

33%

Souvenir shops

2,835

1,729

61%

32.5%

Market shops

889

nc

nc

nc

Roadside vendors for souvenirs

488

378

65%

41.0%

Motodop

6,220

1,963

32%

35%

Tuk tuk

6,100

1,786

29%

26%

Taxi

980

92

9%

11%

Cyclo

145

66

46%

66%

Boat

290

nc

Nc

Nc

Guide

2,917

52

2%

3%

Source: IFC MPDF data

Another main weakness is that most VCAs focus on value chain operations and how they generate dollars that flow to the poor, but provide much less information on the poor themselves, how they earn the dollars, and how many poor people are involved. The result is weak information on producers themselves, compared to the markets in which they operate. The whole thrust of this VCA approach has been to switch away from starting with a target group of producers to bring into tourism, and into starting with the market and from there identifying how to integrate more poor people. But perhaps, inevitably, we swung too far at first. This creates a number of problems. Firstly, if we do not know how many poor producers are involved, we cannot monitor changes in numbers of poor participants. And if we do not know how poor they are, we cannot monitor a reduction in the poverty gap. Secondly, to move beyond a focus only on PPI, we need to understand their perceptions of the chain, and the wider costs and benefits their households face. Thirdly, without information on how revenue is distributed amongst various poor earners, assumptions are made about PPI flows which may be wrong. For example, most VCAs estimate spending on domestically produced food, but do not then analyse the agricultural chain in detail to assess the roles and returns of farmers, wholesalers, transporters, etc. The LPB study simply assumed that all spending on food at the local market counted as poor because everyone from farmer to vendor was poor, except that 15% needed to be deducted for operational costs (e.g. fuel). This doubtless led to an over-estimate of PPI. The more recent IFC MPDF studies collected data on spending on domestic food, but were unable to work out PPI from this mainly because the food wholesalers were unwilling to share information. In the Rwandan VCA, whether or not suppliers of fresh food to Kigali hotels include the poor or not makes an enormous difference to calculation of PPI (Box 3). To fill this gap, VCAs need more input from agriculturalists and more focus on producers themselves. Two reports which use VCA but are not just VC analyses, provide a useful contrast, as they provide more insight from producers. The Nabji-Korphu trek report (Dorji, undated) is a socio-economic analysis, with considerable data from village participants in tourism about costs and benefits they

21

face, as well as their PPI. This gives a broader context for making recommendations. A scoping study done by the International Trade Centre for a World Bank project in Bihar (Wight, 2006) looks at five tourism-related product chains where intervention could benefit the poor. The overall report looks at a wide set of issues, such as tourism demand and community profile. It does not calculate PPI per sub-chain as most VCA reports do. But what is has that is very valuable is a mini-VC for the bottom of each sub-chain which explores the returns to the poor producers in relation to middlemen and retailers. This shows, for example, that poor producers are earning just 4% of retail prices of prayer bead chains (Figure 3). It thus highlights the considerable potential to strengthen their marketing capacity to achieve considerably higher returns. Figure 3: Value chain for prayer beads in Bihar

Fruit from Rajgir, Guruva or Barachatti Block@Rs20/kg (traders tell villagers they will sell for Rs1,200/kg)

100–150 families in Bodhgaya (Bhagalpur village)

Shell the seed Drill a hole Smooth the seeds Chain 108 beads Polish the chain

Production in Nepal

Wholesalers/retailers in Kathmandu

Nepali traders buy fruit @Rs25/Kg

Villagers sell to traders for Rs2 per chain

900,000 chains produced p.a.; cost price per set is Rs2 (labour) plus the raw materials; 1kg fruit makes 3 chains

Traders sell to wholesalers/ retailers for Rs30–40 per chain

Wholesalers /retailers in Bodhgaya, Rajgir, Delhi, and Bombay

Pilgrims to Bodhgaya. Retail price Rs50 per chain

Retail price of Rs50 per chain yields Rs45,000 p.a.

Source: Wight, 2006

While most pay enormous effort to counting and mapping current PPI, there is remarkably little on potential increases in PPI. Because there is a lack of explicit recognition that current PPI is not a useful guide to where to intervene, this is perhaps the major weakness so far. There is little ‘what if’ modelling to assess impact if key variables change: if 10% more fruit and vegetables were sourced locally, or if craft makers could get 50% not 20% of retail price, or if tours paid a fee for entering a village, then how would PPI change? Given the diagnostic purpose of most studies, this omission is surprising and clearly an area for development. The likely explanation (speaking from experience), is, firstly, that so much effort goes into collecting the raw data, that by the end of the exercise, there is too much data and insufficient energy left to interrogate the results thoroughly to draw out implications. Secondly, that identification of possible reforms that could be modelled is a somewhat different, additional, work stream, with different information needs, that has not always been included in terms of reference for VCAs.

22

The IFC MPDF study goes furthest in developing ‘what if’. Having analysed PPI and employment impact for different market segments and different sub-chains, it then uses this to calculate the number of additional tourists from each segment that would be needed to generate a certain level of additional employment or PPI (see Box 4). A more conventional approach to planning tourism development would compare an audit of the product (as done in VCA mapping) with trends in demand. A particular strength of the IFC MPDF VCA is the attention to tourism demand, and particular disaggregation of results by market segments (between country of origin and between group (GIT) and independent (FIT) tourists (see Box 4). This should make it easier to identify interventions based on the real market situation and tailored to specific client groups rather than recommendations based on unrealistic averages. While the importance of understanding demand is taken as given, the question of ‘demand for what’ needs attention. There is rarely a tourism study that does not give numbers for overall arrivals and growth rates. But if the focus of intervention is tomatoes or prayer beads, then it is the specific demand, of chefs or pilgrims, not total arrivals, that needs analysis.

23

Box 3: Disaggregation of results by market segment MPDF has analysed expenditure (yield), local economic impact (LEI, meaning revenue staying in the local economy), and PPEI, or salaries to those from a poor background) by market segment. While data are not yet finalised (MPDF 2008, draft), detailed comparisons are being drawn. Figure 4 shows how the date enables comparison between different types of tourist, in terms of their contribution to PPEI per visit.

Figure 4: IFC MPDF estimates of PPI by market segment

x-axis: PPEI per bed night for each market segment. PPEI is direct employment income for employees from poor backgrounds (only) y-axis: nights per annum for each market segment Size of bubble: total expenditure per year for each market segment G = group tourists. F = independent tourists.

The data is used to calculate the number of additional tourists that would be needed to generate one million in LEI and PPI in Cambodia, as shown below.

Table 5: Estimates of additional tourist impact on LEI and PPEI Tourists segment High PPEI generators USA – FIT USA – G Taiwan – FIT China – G China – FIT Medium PPEI generators UK – G Taiwan – G Japan – G Japan – FIT Low PPEI generators Korea – G Vietnam – G Thailand – G

No of additional tourists* needed to generate: $1 m of LEI** $1 m of PPEI 3,000 4,000 5,000 3,000 2,000

50,000 or less

3,000 5,000 5,000 6,000

85,000–120,000

6,000 8,000 6,000

160,000–170,000

*estimates assume nights per segment remain constant. **rounded to the nearest thousand. Data is not finalised. G = group; FIT = independent.

However, drawing policy implications from this is not easy, as there are several complexities to note. • There is not one segment that is ‘best’ for pro-poor impact. Highest levels of PPEI per person (i.e. per trip) are from Japanese, Chinese and American Group tourists. But in terms of PPEI per day, group tourists from Malaysia and China generate the most. In aggregate, it is USA and Chinese independent tourist segments that contribute the largest amount per year to PPEI. • There can be trade-offs between PPEI and LEI, and between PPEI per trip and market significance to Cambodia. For example, while Korean tourists appear to be generating low PPI per visit, they are one of the most significant markets (in terms of arrivals and investments). They have a high LEI per day, but PPEI is low because they are not visiting secondary destinations and high shopping expenditure is on items that do not generate salaries for the poor. • Those who spend less per day generally stay longer, and vice versa. Thus Japanese tourists show as only medium PPEI generators in the table above, due to their short length of stay. But analysis of the number of extra bed nights per market segment required for $1 m of PPEI would illustrate the high return to enhancing their length of stay. This also has environmental benefits: generating more PPEI from relatively short visits.

24

Most VCAs have been done in an extractive technical way: gather information from surveys and interviews, put it together in an Excel spreadsheet and produce a report. The Da Nang VCA of Mesopartner and ODI (Mitchell and Le Chi Phuc, 2007) show a way of doing VCAs more participatively with promising results in terms of mobilising stakeholders. The VCAs focus heavily on ‘the poor’ but little on distributional issues beyond that. Value chain maps can be used for illustrating distributional flows. For example, a map of the LPB value chain was reproduced three times, one showing where the poor participate, one showing ethnic minorities and one showing women. However, the geography of the supply chains was not addressed, while from a poverty perspective it would be useful to know whether silk and food are coming from the poorer regions. None of the VCAs have mapped the geography of participation by the poor. Where data permits, calculations of PPI could be developed further to separate out the PPI that flows to specific poorer groups (akin to using two poverty lines), or even to apply weighting when calculating benefits of intervention options. This has not been done yet. The Route 9 VCA demonstrates the problem well: it points out that poverty in the Savannaket Province of Lao PDR is more extreme in the East than the West, and that most SNV and Provincial Tourism Office attention to date has focused on the West. In mapping Route 9 tourism and VCA, the report finds that ethnic minorities operate few tourism businesses except non-timber forest product (NTFP) sales. But beyond these very relevant points, the mapping of PPI flows does not address this distributional challenge. The VCAs have not yet found a way to reconcile objectives related to PPI with environmental and social objectives, although that is the oft-stated intention at agency level. The obvious approach is to develop two-variable or three-variable matrices that balance PPI flow with environmental and social performance (i.e. a sub-chain or an intervention could be scored in environmental terms, and then ‘plotted’ as to whether it performs well on PPI and/or environment). Something akin to this is already done in the Nabji-Korhu Trek report as shown in Figure 5 which plots residents’ views of environmental and social issues. Figure 5: Economic, Socio-Cultural and Environmental/Infrastructural Benefits perceived by residents in Nabji Korhu 120 100

97.4 73.7

Percent

80

68.4

60 40 20 0 Economic

Cultural/Social

Environmental/Infrastructural

Benefits

We are beginning to be able to compare results across destinations and that is proving invaluable. For example, it is evident that tourism in Cambodia is not performing well on pro-poor impact by international standards, because total PPI as a percentage of tourist destination-level spending is much lower than in some well-linked destinations such as LPB and Da Nang. When findings differ from what is emerging as ‘typical’ internationally (based on a very small sample so far), it is useful,

25

firstly, for identifying where data may have gone wrong, and secondly, if data is correct, for identifying where there may be substantial potential for intervention to get closer to some ‘norm’. However, a very big caveat is needed here. The data is not comparable. Each VCA has adopted slightly different definitions, particularly of what counts as PPI and what counts as destination expenditure. Rough indications are useful, but for genuine benchmarking we do indeed need agreement on shared framework and method. Finally, the application of methodology requires refection. It is clear there is a vast amount of data to collect, merely to estimate PPI from different sources, let alone to add all the other aspects touched on above. Perhaps more challenging than to list out the data needs is to try to capture the data analysis process. It is a technical process in which one wrong assumption, one data error, or one momentary lack of attention as to whether data is per bed night or per visitor, can substantially distort the results. We have surely all found, in our own or each others’ work, errors that significantly changed the calculation when rectified. What we do not know is the impact of errors we have not found. On the other hand, we are getting quicker, learning some short-cuts, using each others’ methods and taking them further each time. There is a risk of excessive focus on data collection and information needs, over analytical needs. In the Route 9 survey, local partners in Lao PDR and Cambodia were charged with hefty data collection tasks from samples of hotels and restaurants. The result is an invaluable mass of data that is much more robust than the more ‘quick and dirty’ VCA such as the LPB and Rwandan VCAs. The challenge however is to be able to summarise, compare, and interrogate the results to make policy recommendations within a relatively limited consultancy. We do not yet have the documented tools to share with partners that could make this an automatic part of the process, so this is very time-consuming. The IFC MPDF surveys have generated an extremely impressive data set of well triangulated data. Again, the challenge is to maintain the energy to draw out findings that feed into policy. The Rwandan work was an attempt to do as much of a strategic-oriented VCA as possible in 10 days. On the positive side, it showed that, with increasing clarity on some key variables, it gets a little quicker to hone in on them. However, this VCA has the same problem as the IFC MPDF and Route 9 VCAs – more information was collected than could be processed – while also demonstrating the reverse problem – the risk of trying to make strategic policy recommendations on an evidence base that is weak. It is important that evolution of value chain approaches is driven by the desire to scale up impact, and not be dominated by formulas for value chain mapping. Experience with the wave of interest in Sustainable Livelihoods adds some context here. The sustainable livelihoods (SL) approach, developed by the UK Department for International Development (DFID) in the late 1990s, was an invaluable addition and complement to conventional approaches to poverty analysis by grounding analysis more heavily in the reality of poor people’s priorities, the complexity of their livelihoods, and the importance of their assets, not just income, to them. The focus on livelihoods not just income, just like our focus on PPI not just size of tourism, was, and still is, invaluable in pushing development practitioners to think more broadly and laterally. But over the years, inboxes and websites became cluttered with reports that used the SL categories (five assets, various strategies, policy environment) somewhat mechanistically, and struggled to draw out the policy relevance (though some reports remain analytical and useful). This mechanistic adoption is perhaps inevitable when a new diagnostic approach becomes popularised. It does not undermine the intrinsic value of the approach, but does mean that practitioners need to watch out for this in the roll-out process.

It is clear from the above that VCA has a lot to contribute, as a diagnostic approach that pays specific attention to benefits to the poor, and is also driven by a focus on scale of benefits. However, it has strong limitations. Some of these mean that the VCA approach needs to be enhanced, while others highlight that VCA must not be relied on alone, but must be incorporated with other techniques. There

26

are debates to be had about the scale of resources that can be invested in monitoring value chains, and also about the balance of effort to invest between data collection and interrogation. In order to turn this discussion back to the question of ‘what should we measure’, Table 6 compares a number of studies in terms of the issues they tackled. It is clear that even in diagnostics, we are still making progress and have gaps to fill.

Table 6: Comparison of issues tackled in several pro-poor tourism value chain analyses Emerging studies (MPDF) Kratié Cambodia

Siem Reap Cambodia

Define ‘poor’ against a poverty profile, not just wage level

9

9

Calculate pro-poor benefits ($ per supply chain)

9

Calculate PPI as % of turnover or expenditure per supply chain

Other existing pro-poor tourism value chain analyses Luang Prabang Lao PDR

The Gambia

Da Nang Vietnam

Bihar India

Rwanda

9

9

9

9

9

9

9

9

9

9

9

9

Estimate aggregate PPI from sample based on a supply chain inventory

9

9

9

9

Compare sub-chains by PPI $ p.a. and by % of turnover

9

9

9

9

9

Triangulate data from multiple sources

9

9

9

9

9

9

In part

In part

In part

In part

9

Track the production-wholesale-retail process to identify producer sale price (‘farm gate’) vs. retail price in key supply chains

9

9

Go beyond pro-poor flows to assess market linkages and bottlenecks that need tackling for the poor Estimate potential increase in pro-poor impact from example interventions

In part

9

Explore which ‘middlemen’ and retailers count as poor Estimate pro-poor impact by tourist segment

In part

In part

In part

In part

In part

9

In part

In part

9indicates it has been done, ‘in part’ means the issue has been addressed but not comprehensively. A blank cell means it has not been addressed, or very little. The Bihar entry is based on Wight (2006), Community-Based Tourism Opportunity Study, Bodhgaya, India, an International Trade Centre input to a World Bank project. This was not a value chain analysis but a scoping study for pro-poor intervention that included mini-value chains for products such as prayer beads. p.a. = per annum. 27

28

5. Developing approaches for monitoring impact 5.1 Problem statement M&E is never easy, but its application to value chain interventions is particularly challenging because value chains are complex and multi-layered, changes in performance are subject to many influences not just one project intervention. Three particular challenges highlighted in discussion at the December 2007 workshop in Cambodia on measurement of value chain impact were: (i)

How to maintain the right balance between resources allocated to baselines and monitoring versus resources allocated to action. Data needs for accurate assessment of value chain performance are so high, that there is a risk they will be disproportionate to the investment in creating change for the poor. On the other hand, measuring impact is essential. Each context and project is different, so needs different indicators, and yet a shared framework of common indicators would help enormously, both in allowing for comparisons across projects, and in generating efficiency savings when developing M&E. The right balance needs to be found. Social and environmental impacts are too important to be ignored. But they are more varied and less quantifiable, so are more difficult to capture in any ‘common framework’.

(ii)

(iii)

The question of resource allocation is a real problem with no single answer. It is clear that different organisations have needs for different levels of detail and statistically robust data. IFC needs to justify large investments of public money in an institution used to full economic appraisal of Internal Rate of Return, Net Present Value, and more. While SNV needs to assure its funders that it is not just ‘proving’ but is actually ‘moving’. To some extent, it is already proving possible to speed up the process and learn short cuts. Key data needed in any VCA is now fairly evident – but of course collecting the data locally, and the all-important interrogation and interpretation of data, will always require investment of time.

‘Not everything that can be counted counts: not everything that counts can be counted’ (Albert Einstein)

5.2 Diagnostics versus M&E A baseline assessment, a diagnostic analysis and regular M&E will all need to cover similar issues but they do have a slight difference of emphasis:

•

•

•

A diagnostic analysis is done to decide what action to take. The key elements are to find which linkages in the chain are subject to bottlenecks that can be influenced, and which sub-sectors in the value chain have potential to boost poor people’s participation. An understanding of the contextual factors and trends is also of course essential to make sensible choices. A baseline assessment should provide a fairly comprehensive snapshot of the current value chain for future comparison. In this, an inventory of different enterprises is essential, along with an indication of the numbers of poor people involved in different ways. A baseline also needs to cover who does and thinks what, who relates to whom, and how money flows through the value chain. It needs to include some way of indicating the status of processes and perceptions: policy maker attitudes, strengths of partnerships and attitudes towards pro-poor issues, plus priorities of the poor and their perceptions of the tourism economy. This provides a basis for future comparison. Monitoring is done to measure change that is occurring. Therefore it covers many of the same items but with the main focus on identifying trends and changes over time. These can be in anything from profit per artisan, numbers of village tours and products, or stronger alliances among pro-poor stakeholders. Monitoring aimed at establishing the impact of interventions

29

also needs to focus on causality between an intervention, a change in VC performance, and change in incomes of the poor. For measuring changes over time it is of course essential to maintain the same indicators. For diagnostic analysis, we have a good idea of what we need to measure, but still need to improve the methodology, particularly in order to get it done with reasonable resource investment, address current gaps, and to make best use of the data. Box 4 summarises some key information needs for diagnostic VCA. In deciding what we need for baseline assessments and monitoring, we are at the beginning. Compared to the diagnostic studies, the type of information needed is probably very similar, but the areas of focus and level of detail will differ, as will the purpose of some of the information. A diagnostic approach must look across the entire chain, in order to identify areas for intervention. It must be openminded about areas for potential increase. It will be impossible to go into full detail on all areas, but will need more detail on areas that are prioritised for action. A baseline and future M&E will need even more detail on those areas of intervention, and will need less on the other parts of the sector that are not targeted. For example, a diagnostic may identify that craft earnings are very low in a destination x. It would need some assessment of market size, returns, and bottlenecks for project planning. A baseline would need more detailed information on how many are earning how much from what activity, and the current poverty situation of households involved. M&E will need to monitor the overall shape and size of the craft chain, combined with more detailed work with producers and middlemen to spot changes in returns to the poor. Household views on the impact of financial and non-financial changes to them will be needed. The contextual information will be essential in both the diagnostic and the M&E, in the former to understand potential for action, in the latter to understand attribution. Box 4: Summary of suggested information for diagnostics Who counts as poor and method for defining them. Inventory of enterprises, type and number. PPI by type of enterprise: $ p.a. and % of turnover. PPI by sub-chain: $ p.a. and % of turnover/tourist spend. For enterprise and sub-chain: number and type of poor participants. For each sub-chain identified as of interest, PPI as wages, as profits from supply of inputs. How returns to poor producers compares with returns achieved by others in the supply chain. Producer level information: net returns, benefits to their household (HH), constraints faced, upgrading ambitions. Comparisons: by enterprise type and ‘business model, by market segment, between sub-chains, between different poor producers, between this destination and other ‘benchmarks. Key factors enabling high PPI or constraining it where it is low. Supply side, demand side, and missing markets. Overall context: tourism sector growth, policy environment (not just in tourism, but relating to enterprise, agriculture, land use, language skills, etc.). Views of producers, clients, tourists, etc. on bottlenecks and areas of potential. Social costs and benefits perceived by stakeholders. How these can be scored. Environmental costs and benefits perceived by stakeholders. How these can be scored. What-if scenarios. A process for identifying possible changes, and assessing their potential impacts. Plus their feasibility and cost.

30

5.3 Approaches to M&E in enterprise development and value chains So far we have focused entirely on what has been learned from the application of pro-poor VCA in tourism in the last two years. However, there is a wealth of work in other sectors on enterprise development or value chain approaches, and in the field itself of impact assessment. This should also be utilised. A comprehensive review cannot be provided here, but no M&E framework should be developed for a specific project without considerable attention to norms of project cycle management and M&E design. For example, most manuals on M&E or impact assessment, whatever the focus, have a similar table of contents, taking the reader through essential steps: the purpose of the assessment, who should monitor, what to monitor, the key issue of defining indicators, the choice of methods for collecting data, then analysing data, interpreting it, feeding back results, etc. (see as an example, SNV’s toolkit on Community Tourism Impact Assessment, Twining Ward, forthcoming). Whether the focus is an enterprise or a value chain, M&E should not be designed without attention to these issues. There are some more quantitative and rigorous approaches to impact assessment just as there are more quantitative conventional economic approaches to project diagnosis (see Box 1 above). The United States Agency for International Development (USAID) is investing considerable resources in impact assessment of an enterprise development approach in Zambia and elsewhere. It is a quasiexperimental design using a mix of methods including longitudinal surveys (meaning repeated measures with the same respondents over time). A sample of project clients and a comparable group of non-clients will be surveyed twice, with a two-year interval between surveys. Data from these surveys will be combined with qualitative information collected through interviews and focus group discussions. Impacts will be measured at the value chain, enterprise, and household levels. This approach differs from most in its use of longitudinal data, and a control group. While non governmental organisations (NGOs) often cannot afford longitudinal surveys, the principle of tracking change over time among a fixed group of respondents is important, even if done more as ‘case studies’ than as a robust survey. In the longest-running community tourism initiative in Luang Namtha (Lao PDR) monitoring was done with the same families over three years, to track changes in income, expenditure and participation. Household level monitoring was complemented by monitoring at village and enterprise level, plus tourist surveys. The original design for six-monthly surveys was too time and resource-intensive so was scaled back to annual assessments. The United Nations Industrial Development Organization (UNIDO) recommends a relatively participatory approach to impact assessment of ‘cluster development’. There are several parallels between ‘cluster development’ and value chain development. UNIDO (2004) focused on teasing out the specifically propoor potential of clusters, which had hitherto received little attention in the overall enterprise development drive. Taking a somewhat different approach to USAID, their paper recommends a participatory approach, in which impact assessment in used primarily for enhancing project management. Annex 1 provides a little more detail on some useful sources that discuss M&E from an ‘enterprise development’, value chain, or tourism perspective.

5.4 Emerging framework for M&E To measure impact, we need to know how the tourism value chain is functioning now compared with before, how many more poor people are earning how much more from any increased participation, and what other change in net benefits has occurred, what impact this has on household wellbeing, and what bottlenecks or opportunities are changing. No small task. There is so much that can be measured, so honing down on what is essential and will be used is key. The best place to start is probably not with a long list of data needs, but with the overall questions that monitoring is trying to answer on an on-going basis throughout a project, and at the end. If too much

31

information is collected, it will simply not be used. So it is always wise to start by defining the questions to answer (in the final report conclusion), before defining the questions to ask (in the field), so that the first drives the second, and those doing the work are able to prioritise. Data should be sought that helps answer these six questions:

(i) How is the overall size and shape of the sector changing? Why? (ii) Is there progress in creating an enabling environment for increased participation by the poor (iii) (iv) (v) (vi)

(public sector policy/regulation, private sector engagement, business environment, barriers to SMMEs, etc.)? What and how? Are there changes in the number of poor people participating in the tourism value chain? Who, how, where, why? Are poor participants in the tourism value chain experiencing any increase (or decrease) in income or livelihood? Who, what, and why? How does participation in the tourism value chain impact the livelihoods and wellbeing of the families concerned? Has this changed? What wider economic, social and environmental impacts is tourism generating that affect poor communities (not just direct participants)? Are they changing? Why and how?

The question of ‘why’ recurs throughout: understanding why a change is happening is essential to understanding the impact of an intervention. This can also be termed assessment of ‘explanatory variables’ – the key factors that cause change at each level – as in Table 7 below. If the form of intervention is specific and targeted to one element of the VC, then monitoring questions of course need to explore that specific segment of the VC in more depth, and the questions for the tourism sector as a whole in less depth. The specific indicators for each question will need to be defined, tested, and refined early on in the project. Table 7 uses these six questions as a basis for developing key indicators. Table 7: Key questions and indicators for monitoring value chain intervention impact Questions

Key indicators (for outcomes and impacts)

Notes on data gathering

(i) How is the overall size and shape of the sector changing? Why? (Important context, usually external to the project)

• percentage change in arrivals, length-of-stay, bed night • market segments • market trends • explanatory variables (EV)

National data and an informed knowledge of the industry

(ii) Is there progress in creating an enabling environment for increased participation by the poor? (this may be ‘context’ or may be a key measure of project impact, depending on the type of project)

Any changes in: • public policy, regulation • municipal/provincial action • private sector engagement and action • business environment • SMME constraints / constraints perceived by poor • HR investment • land/natural resource/other policy • EV for any change

Measure actions and attitudes of policy makers, private operators, and poor people e.g. a change in bottlenecks perceived by the poor is a significant change (particularly if attributable to the project) Known as ‘soft’ indicators (see Roduner, 2007)

32

Questions (iii) Are there changes in the number of poor people participating in the tourism value chain? Who, how, where, why?

Key indicators (for outcomes and impacts) • estimated numbers of new entrants • profile: gender, minority, poor/poorish • in what activities • why? expansion of the VC, of a segment? or reduction in barriers to access by the poor?

Notes on data gathering Less detail for overall VC context and more detail for areas of project intervention Data is mainly via enterprise analysis: poor per enterprise, and enterprises per segment (see Table 2 above for different changes that lead to increased numbers)

(iv) Are poor participants in the tourism value chain experiencing any increase (or decrease) in income or livelihood? Who, what, and why?

Any increased PPI via: • increased production volume • upgrading production/ enhancing market negotiation • moving up the VC to new activities • for whom? (gender, type of participant) • non-wage changes, such as job security, risk, increased influence over change • EV for any change

Any changes in: (v) How does participation in the tourism value chain impact • tourism income relative to poverty the livelihoods and wellbeing of levels and other household (HH) the families concerned? Has income this changed? • views of poor re: significance of their tourism participation • spending patterns of tourism income (meeting needs, and investing) • observable investments e.g. in HH or community assets • EV for any change

(vi) What wider economic, social and environmental impacts is tourism generating that affect poor communities (not just direct participants)? Are they changing? Why and how?

Any changes in: • observable social / environmental impacts e.g. water use, resource competition, availability of training, degree of local voice in decisions • perceptions of poor (and others) re: ranking of tourism impacts, and balance of positive versus negative

Requires analysis by sub-sector, mainly at enterprise level and with poor workers/producers (see Table 2 above for different types of change in the value chain for poor participants)

Requires HH level views and information on the role of their tourism earnings Also essential to gather info on other levels of income, poverty & expenditure to understand the significance of tourism Cannot be done for all poor participants, so choose key target groups and a few key indicators for regular monitoring Needs to focus on key social or environmental indicators defined early in the project e.g. water, sewage, skills development, participation, land-use, etc. Combine views of poor and of ‘experts’

We assume that monitoring and evaluation will attempt to keep track of PPI flows and numbers of poor participants. This is in keeping with the pro-poor value chain approach so far, but has significant implications for M&E. Tracking PPI across a sub-sector requires substantial information from specific enterprises, plus key data for aggregating up from some enterprises to a sub-sector aggregate. Table 8 thus provides some more detail on data needs at enterprise and sub-sector level for doing this.

33

Table 8: Data needs at enterprise and sub-sector level for calculating PPI and pro-poor changes Enterprise level data (e.g. restaurant, craft outlet, boat operator) Business profile

product(s), product prices, size (turnover, clients, beds/vehicles/tables, staff, etc.), location, position in the market

Income

sales volume, gross revenue, costs (thus profit), seasonality, trends and key variables.

Inputs (enterprise analysis may need to be repeated with suppliers to estimate participation and PPI in the supply chain)

main inputs, suppliers (poor/non-poor), prices paid, contracting process, trends, key variables

Staffing

number of staff, wages, other benefits, skills levels, recruitment, training and promotion, worker profile (gender/poverty), trends, key variables

Clients/ market

main markets, profile, expenditure per client/trip/meal/day, contracting relationship, trends and key variables, share of market related to tourism

Business operation

constraints on competitiveness, current expansion/ contraction, horizontal linkages with similar firms, vertical linkages for pricing, contracts, marketing, etc.

Explicit pro-poor issues

business commitment to pro-poor action, perceived constraints (e.g. on local supplies), social/environmental impacts recognised or mitigated/enhanced

Comparable variables

ensure key data collection to make comparisons with other enterprises and aggregate info. across the sub-sector, e.g. number of beds/clients and position in market and dependency on tourism

Sub-sector level information, needed to estimate PPI flows and trends across the sector Number of enterprises

divided by type: size, grading, product type

Volume of sales

preferably by several indicators: bed nights/meals, pax/clients, turnover, staffing; and using more than one to calculate aggregate PPI per sub-sector

trends

in sub-sector size and profile

sector demand

volume and prices, main segments and trends

key actors

their actions and opinions

This gives us the basic framework for the levels at which we need to collect and analyse information: The primary level is the level of the enterprise: activities, returns, poverty profile, supply links, influencing factors; Above that, a picture is needed of the sub-chain, its impacts, performance, and stakeholder roles. This comes partly from aggregating-up data from enterprises, and partly from additional information from key stakeholders; and Going down from the enterprise and PPI flows, we focus at the level of the household: how PPI changes household poverty, perceptions of non-financial benefits, and other factors changing HH poverty, etc.

34

Then for contextual information and understanding causality we need information on the wider tourism economy, and the non-tourism enabling environment that affects tourism enterprise (credit, transport, etc.). Given the need to be project and context specific, it is impossible to set out a more detailed framework that will be applicable to all project, precisely because if it is comprehensive enough for all projects under a tourism umbrella, it will be too comprehensive for any. Of course it is impossible to document every aspect of the value chain, how the linkages work and where funds flow. So it is vital to combine broad brush ‘big picture’ information of the overall value chain, with more focused information on those components where the poor are most active or interventions are focused. It is important not just to take a static snapshot. As M&E is not just to monitor impact but to feed back into strategy and action, it needs to identify bottlenecks and opportunities that affect participation of the poor over time (i.e. M&E must retain a diagnostic element as well). When planning M&E, it is as important to plan the analysis and feedback of findings, as it is to plan the data collection.

5.5 Monitoring intermediate objectives Often an intervention will not necessarily engage directly with the poor to change their value chain engagement, but will be targeted at changing practice by others. As, for example, SNV’s tourism component of the strategy for East and Southern Africa (2008–9) makes clear, it will engage via a number of mechanisms to achieve impacts at different levels, such as:

• • •

at macro level, such as inter-ministerial collaboration, increased policy dialogue at meso level, strengthened in country capacities for tourism development at micro level, increased linkages between tourism establishments and local economies

Therefore it is important to measure such intermediate objectives, or outputs, such as changes in capacity of partners, commitment of government and private sector stakeholders, and strength of alliances of change. Indicators for these need a fair degree of common sense and realism, but often include degrees of participation at meetings, types of communication, statements of policy, designation of officials or staff with specific new tasks, examples of collaborative action, actions taken by private sector operators in their business. These are also called ‘soft’ indicators (Roduner, 2007). Roduner emphasises that these early signs of future change are particularly important when the aim is long-term change of a complex system such as a value chain, rather than a specific change in one enterprise. Change in attitude and practice of others may be a necessary means, and thus precursor, to achieving any poverty impact. On the other hand, it could be an additional impact caused by the demonstration affect of initial success. Either way it needs to be monitored. Other literature on value chains (not specifically in tourism, see Roduner, 2007 in Annex 1) also highlights the need to assess changing mindsets, for example among ‘early adopters’ and the ‘early majority’, in addition to significant impacts that derive from change implemented by a few ‘innovators’.

5.6 Attributing change to an intervention Attributing observed changes to any particular cause (such as a development intervention) is fraught with difficulties. The social and economic systems are not closed – the tourism value chain will be affected by events literally all around the world. Establishing the impact of specific variables normally requires the use of a control area that is similar to the main study area in all respects except that it is not benefiting from the intervention. A control area is conventionally considered the only way to establish a plausible counter-factual scenario (i.e. what would have happened if the intervention had not gone ahead). Sometimes there may be a convenient relatively similar geographic area for an

35

approximate comparison, but often in practice it is simply a matter of extrapolating from a trend to identify whether the trend line has changed due to the intervention. However, adopting a value chain or market access approach does mean being part of a wider network of change agents, and thus generally means accepting that direct attribution to a specific stakeholders intervention cannot be fully attained. Ironically, intervening to scale up impact may make it harder to demonstrate that impact. Roduner (2007) argues against trying to circumvent the methodological problem of attribution with onerous statistical methods. Instead one should use plausible ‘rules of the game’ and field observations.

5.7 Long-term versus short-term Impact The short-term impact of opening a new hotel that employs 100 staff is to create 100 new jobs. The long-term impact may be quite different. The new hotel could, for instance, force the closure of a nearby competitor which has to retrench its workers. The capital used to finance construction of the hotel could have been used to develop an agro-processing factory that would have created 200 jobs. The reason for this is that, in most developing country contexts, factors of production like capital, land and skilled labour are not infinite. So, if they are applied to the tourist value chain, they are being diverted from another potentially productive use. The point is that the effect of increases in tourism expenditure is almost always more muted than the immediate impact of these changes. On the other hand, the development of roads, of skills, or new demands for services can have positive knock-on effects on many aspects of the local economy. Further thinking and alliances are necessary to measure dynamic effects, but the first priority is to – at least – identify and categorise the main types of dynamic impacts relevant to the destination.

5.8 Geographic scope of the poor Inbound tourism can impact on poor households that are distant from the tourist destination. This impact may be positive (for instance where park fees are collected by central government and enhance the general fiscus) or negative (where the ‘shock’ of a surge of foreign exchange from tourism can hit farmers who find food exports less competitive. Agencies tend to focus on ‘the poor’ in the area of their own operation or in the heart of a tourism destination, but thought should be given to how to measure impacts on more geographically distant poor. Particular challenges arise when gains to the poor in one country are at the expense of the poor in another, for example, if local farmers are able to grow mangos to replace imported supplies. For some agencies, this is a pro-poor gain, while from the perspective of others, it is merely a substitution.

36

6. Conclusion The shift to pro-poor intervention in tourism value chains marks a substantive departure from past practice. It is driven by a desire to achieve greater impact on the poor at scale. Therefore the question of what information is needed for accurate diagnosis, and what is needed for monitoring impact, is very important. The story so far is that we are now reasonably confident that we know what we need to measure for good diagnostics, but still need to develop the methodology, particularly in order to get it done with reasonable resource investment and to make best use of the data for decision-making. In developing an approach for baselines and monitoring, there is a long way to go. But the discussion above suggests that attention is particularly needed in five areas: (i)

(ii)

(iii) (iv) (v)

Who counts as poor? Does it vary by objective of the work, or do we need a rigid definition (e.g. US$1 per person per day) to allow benchmarking across destinations? Or can we manage to do both? PPI. How much is being earned by how many poor people? This question is at the heart of most VC approaches in tourism so far, and is central to measuring impact. This alone is a complex data task. Non-financial benefits and costs, including other livelihoods impacts, distributional priorities, social and environmental change. Functioning of the value chain, the bottlenecks, market relations and trends that are key influencing factors for understanding impact or assessing potential interventions. How the information will be used and analysed. What will be compared with what and will it be comparable? What aggregations will provide what? What questions are we ultimately trying to answer?

While monitoring impact varies enormously between projects, the following six questions are proposed as a common set of questions to answer – these will shape the data that is needed.

(i) (ii) (iii) (iv) (v) (vi)

How is the overall size and shape of the sector changing? Why? Is there progress in creating an enabling environment for increased participation by the poor? (public sector policy/regulation, private sector engagement, business environment, barriers to SMMEs, etc.). What and how? Are there changes in the number of poor people participating in the tourism value chain? Who, how, where, why? Are poor participants in the tourism value chain experiencing any increase (or decrease) in income or livelihood? Who, what, and why? How does participation in the tourism value chain impact the livelihoods and wellbeing of the families concerned? Has this changed? What wider economic, social and environmental impacts is tourism generating that affect poor communities (not just direct participants)? Are they changing? Why and how?

Measurement of PPI has been a major focus on value chain analysis done so far. It is an essential addition to the piecemeal evidence that existed before, but we need to be aware of its limitations too. Firstly, conventional VCA goes well beyond mapping PPI. It analyses linkages and blockages up and down the chain. Tourism pro-poor VCAs have not been strong on this to date. Secondly, however thorough a tourism VCA may be, it is a tool that is insufficient on its own for intervening in value chain development. A VC map is useful for assessing PPI, numbers of poor participants, and how these two relate to other actors. But VCA analysis is weak for capturing other issues such as social and environmental costs and benefits to the poor or to society, dynamic impacts of tourism on the local or national economy. Other tools are also needed.

37

A rigid M&E framework cannot be identified in the abstract. But the discussion here has identified the main areas and issues. For any specific project, these the need to be translated into indicators that are specific to those parts of the value chain that are tackled. Comprehensive monitoring of all parts of the chain is impossible. While M&E must be project specific, it merits some extra investment to enhance comparability wherever possible. This is because the spirit of testing and learning is enabling fast progress to be made amongst pro-poor VC practitioners. There is no doubt this learning and sharing will need to continue as we gain experience in the M&E of value chain interventions.

38

References Ashley, C. (2006) ‘Participation by the poor in Luang Prabang tourist economy’. ODI Working Paper 273. Ashley, C. (2007) Pro-poor analysis of the Rwandan tourism value chain: An emerging picture and some strategic approaches for enhancing poverty impacts. ODI report to SNV and ORTPN. Ashley, C. and Mitchell, J. (2007). ‘Assessing how Tourism Revenues Reach the Poor’. ODI Briefing Paper 21. Ashley, C. and Mitchell, J. (forthcoming). Pathways to prosperity: How can tourism reduce poverty? ODI. & World Bank. Conway. T. and Sophal, C. (2007) How does the choice of poverty measure shape our understanding of poverty levels and dynamics? Presentation to CDRI National Poverty Workshop, Phnom Penh, November 2007. Dorji, P. (undated) Nabji-Korphu Trek: The piloted community-based nature tourism project in Bhutan. Department of Tourism, Bhutan. FIAS and OECD (2006) Assessment of Mozambican tourism value chains. Prepared for the Government of Mozambique. GDS (2006) Value chain analysis of Cultural Heritage Tourism in Ethiopia. Prepared for the Government of Ethiopia and the World Bank. Gereffi, G., et al. (2005) ‘The Governance of Global Value Chains’. Review of International Political Economy 12(1): 78–104. GTZ and CTA (2007) Value chains for broad-based development: Summary of discussions and results of the conference. Value chains for broad-based development International Conference May 2007. IFC/MPDF and SNV (2007) Poverty Alleviation through Tourism – Impact Measurement in Tourism Chain Development. Proceedings of a workshop, Cambodia, December 2007. Lin, T. and Guzman, F. (2007) Tourism for pro-poor and sustainable growth: Economic analysis of tourism projects. Economic and Research Department, Asia Development Bank Technical Note Series No.20. Lusby, F. and Panlibuton, H (2007). Value Chain Program Design: Promoting Market-Based Solutions for MSME and Industry Competitiveness. Action for Enterprise (AFE), Virginia, US. Mann, S. (2006) Towards a Strategy for Pro-Poor Tourism in Ethiopia. World Bank for Government of Ethiopia. Mitchell, J. and Faal, J. (2007) ‘Holiday package tourism in The Gambia and the poor’. Development Southern Africa Special Edition on Tourism Vol. 24, No. 3. Mitchell, J and Le Chi Phuc (2007) Tourism value chain analysis in Da Nang, Vietnam. Report submitted to the Government of Vietnam and the European Commission (Vietnam Private Sector Support Programme). Mekong Private Sector Development Facility (MPDF) (draft, 2008) Tourism Yield Analysis: A Cambodian Case Study. Interim analysis. Nadvi, K. and Barriento, S. (2004) Industrial Clusters and Poverty Reduction: towards a methodology for poverty and social impact assessment of cluster development initiatives. Prepared by the Institute of Development Studies for UNIDO. Men, P. and Rossetto, A. (2007) Engaging the Poor in Tourism. Kratié. Cambodia. IFC, MPDF, SNV. Riisgaard, L., Bolwig, S., Matose, F., Ponte, S., du Toit, A., and Halberg, L (2008). A Toolbox for Action Research with Small Producers in Value Chains (draft). Danish Institute for International Studies, University of the Western Cape, University of Aarhus. Roduner, D. (2007) Donor interventions in value chain development. Swiss Agency for Development and Cooperation (SDC). Snodgrass, D. and Woller, G. (2006) PROFIT Zambia Impact Assessment: Baseline research design. Prepared for USAID. SNV (2008) Using the Value Chain Approach for Pro-poor Development: Experiences from Asia. Travers, R (2008). Tourism on Road 9 in Lao PDR and Vietnam: Identifying opportunities for the poor. A value chain appraisal. SNV Lao PDR with Savannakhet Management Institute and SNV Vietnam with Huế College of Economics.

39

Wight, P. (2006) Community-Based Tourism Opportunity Study, Bodhgaya, India. Prepared for the Export-Led Poverty Reduction Programme of the International Trade Centre (ITC), for input to the World Bank’s Bihar Rural Livelihoods Project. ITC, Geneva.

40

Annex: Useful reports on M&E, value chains, and pro-poor VCA in tourism Value chains (general) and impact of donor interventions There is a wealth of material on value chains from both academe and business. There is a smaller but growing literature looking at value chains as a means of donor intervention in development. GTZ (Deutsche Gesellschaft für Technische Zusammenarbeit GmbH) SDC (Swiss Agency for Development and Cooperation) and CTA (Technical Centre for Agricultural and Rural Cooperation) have been particularly active and co-hosted a workshop in July 2007. The discussions and material are not specific to tourism, but are very useful in providing a general overview of thinking on value chain interventions, and showing some of the parallels and contrast with the work that is confined to tourism. See www.value-links.de for several documents, including a VCA manual.

Donor interventions in value chain development , Daniel Roduner (2007) This is a short invaluable paper from the perspective of donor agencies that want to intervene in value chains. Three pages on impact assessment make several important points about the particular challenge (p. 20) of M&E in a context that is – by definition – dynamic and complex, and in which some change is direct and measurable, but same change will be broad and intangible. ‘The challenge is how to plan something that will be exposed to strong pressures for continuous dynamic change, and how to monitor and evaluate such dynamic processes.’

It is particularly useful on the need for ‘soft indicators’ (p. 20): ‘Results are to be measured at various levels and in different ways. Relevant issues regarding the result, outcome and impact measurement include: Tangible results at the level of the value chain actors (i.e. producers, traders, etc.) like: productivity, profitability and income. Qualitative or soft data at the level of the value chain actors and supporters (i.e. BDS providers) like: changes in behaviour or change in mindsets and attitudes, the absorption rate of new technology/knowledge, or the willingness to invest in (new) ventures. Indicators for sustainability may be (among others): – the commitment of the primary actors (fair and transparent agreements are an indicator in this respect) – the business growth potential for all actors (trends must be supportive for future growth of the business) – the cohesion of the support from the service sector (a structured and coordinated support).’

Soft indicators, or early signs of future change, are particularly important when the aim is long-term change of a complex system such as a value chain, rather than a specific change in one enterprise. Broad impact can occur by facilitating change within a broader population. Changes in attitude of the ‘early adopters’ and the ‘early majority’ need to captured (p. 21) The approach to the question of attribution in Roduner’s paper and others in this arena contrast sharply with conventional thinking on the need for control groups:

41

‘Instead of trying to circumvent the methodological problem of attribution at a higher level of impact assessment with onerous statistical methods with doubtful outcome, one should rather turn to identify plausible behavioural patterns or “rules of the game” for what one can observe at field level.’

The paper provides many useful sources of further information: Links to: Measuring and evaluating value chain development interventions • Altenburg T. (2006), Governance Pattern in Value Chains and their Development Impact. http://www.informaworld.com/smpp/content~content=a770704668~db=all~order=page • ILO, 2006. A Guide for Value Chain Analysis and Upgrading; Module 4: Implementation. http://www.value-chains.org/dyn/bds/docs/545/VCAU_9_MODULE_4.pdf • ILO, 2006. A Guide for Value Chain Analysis and Upgrading; Module 5: Impact assessment & strategy revision. http://www.value-chains.org/dyn/bds/docs/545/VCAU_10_MODULE_5.pdf • Outcome Mapping – Building Learning and Reflection into Development Programs; Sarah Earl, Fred Carden & Terry Smutylo; IDRC 2001. http://www.idrc.ca/en/ev-26586-201-1-DO_TOPIC.html • The Most Significant Change Technique – A Guide to its Use; by Rick Davis and Jess Dart (2005). http://www.mande.co.uk/docs/MSCGuide.pdf Links to: Analysing value chains • Participatory Market Chain Approach – User Guide (2006). http://papandina.cip.cgiar.org/fileadmin/PMCA/User-Guide.pdf • ILO guide for value chain analysis and upgrading (2006). http://www.valuechains.org/dyn/valuechains/bdssearch.details?p_phase_id=545&p_lang=en&p_phase_type_id=6 • CAPSA, Capacitating Sector Analyses (2004); A practical training methodology to analyse value chains. http://www.sdc-ruraldevelopment.ch/resources/resource_en_45.pdf • Clients first! A Rapid Market Appraisal Toolkit (2006), Helvetas. http://www.helvetas.ch/global/pdf/english/Professional_competences/Documentedexperiences/ resources/Clients_First_lowres.pdf • Mapping the market, ITDG / practical action (2005). http://practicalaction.org/docs/ia2/mapping_the_market.pdf • Promotion of commercially viably solutions to subsector and business constraints. http://www.actionforenterprise.org/paper0404.pdf • Gemini, 1991. How sub-sector analysis can help development interventions increase leverage to micro and small-scale enterprises: A Field Manual for Sub-Sector Practitioners. DAI/USAID, Washington DC. http://www.microlinks.org/ev_en.php?ID=7351_201&ID2=DO_TOPIC

International conference: Value chains for Broad-based development Summary of discussions and results of the conference. GTZ and CTA (2007) The summary of the July 2007 International conference, concludes that ‘value chain promotion has been established as a solid, widely used concept’. But a much greater challenge remains in ‘finding answers to the question how value chains work can become more socially inclusive benefiting a greater number of poor and securing their position in the market’. On this point, it concluded:

42

‘The value chain approach contributes to reducing poverty if it is employed strategically and concentrates on targeting the poverty problem. We have to overcome the bias towards the better off by consciously using the full range of options available to support the poor in value chains. This includes fostering associations, skills development and learning, facilitating contract arrangements and supporting information and service delivery. Often, it is necessary to combine value chain promotion with a livelihoods perspective, with local economic development or with vocational training so as to enable the poor to enter (and stay in) commercial markets. However, we need much better monitoring tools to guide pro-poor value chain promotion.’ (ibid, p.3)

This debate is a reminder that value chain approaches began as a way to help poor countries or regions enhance their competitiveness and integrate more productively in globalisation, rather than compete only in a ‘race to the bottom’ (via cost-cutting). The insertion of pro-poor concerns is more recent. In tourism, while a few ‘conventional’ VCAs have been done, many actors introduced a VCA-type component precisely so as to focus on flows to the poor. Thus VCA discussions in tourism are somewhat ‘home-grown’ and different to the wider debates.

Value Chain Program Design: Promoting Market-Based Solut ions for MSME and Industry Competitiveness. Lusby and Panlibuton, AFE (2007) This paper clearly lays out a step-by-step process for identifying value chains in which to intervene, diagnosing problems, and assessing market based solutions. It provides the basis for SNV’s six-step approach presented in Figure 1. The paper provides several examples of constraints to value chain performance that can be tackled through market-based solutions. PROFIT Zambia Impact Assessment : Baseline research des ign . Snodgrass and Woller (2006) prepared for USAID The PROFIT project is focused on competitiveness of medium and small enterprises rather than specifically the poor. It is notable for its attention to developing a comprehensive approach to monitoring impact. PROFIT is a five-year project that began in June 2005. It is funded at the level of $15 million, including $5 million for local grants. The goals of the PROFIT project are to: Improve the competitiveness of selected industries in which large numbers of micro and small enterprises (MSEs) participate and might benefit; Foster the sustainability of competitiveness to enable firms and industries to respond to market demands, both in the short- and long-run; and Increase the breadth and depth of benefits at the industry, MSE, and household levels. The approach is to identify key industries, assess their competitiveness, and design a commercial upgrading strategy for the value chain/industry to turn competitive advantage into competitiveness. In Zambia, several industries were scanned for their relevance to the development of medium and small enterprises, and their overall growth potential. The analysis showed that cotton and livestock provide the best potential returns. Non-timber forest products (NTFPs) and tourism were judged to provide reasonable potential returns, while high-value horticulture and small-scale mining were considered less promising (because of poor industry leadership and inability to achieve scale, respectively). Based on this analysis, PROFIT initially targeted cotton, livestock, NTFPs, and tourism activities. An ambitious impact assessment methodology has been designed. Like similar studies being conducted in other countries under the same programme, the PROFIT impact assessment will employ a longitudinal, quasi-experimental design implemented through a mixed-method approach. A sample of project clients and a comparable group of non-clients will be surveyed twice, with a two-year interval between surveys. Data from these surveys will be combined with qualitative information collected

43

through interviews and focus group discussions. Impacts will be measured at the value chain, MSE, and household levels.

Industrial Clusters and Poverty Reduction: towards a methodology for poverty and social impact assessment of cluster development initiatives. Prepared by Khalid Nadvi and Stephanie Barriento, Institute of Development Studies, for UNIDO (2004) This UNIDO report highlights the potential of cluster development for poverty impact. It does not focus only on income of the poor, but takes a wider ‘capability approach’ drawing on work of Amartya Sen. As illustrated below, they anticipate that cluster development can lead to a range of financial and nonfinancial impacts relevant to poverty alleviation (Table 1, p. 27) For example, expansion of business linkages can stimulate enterprise development, and increased household security through diversification, enhancement of producer (horizontal) linkages and institutional networks leading to increased social capital, increased responsiveness of local support institutions, and improved environmental conditions. UNIDO ‘maps’ clusters against poverty issues, in order to rate the relevance of different clusters relative to poverty reduction. The context, stakeholders, relations and particularly ‘poverty nodes’ within the cluster are mapped, partly using value chain techniques (on p. 28). Poverty nodes are specific points in the cluster where producers or workers vulnerable to poverty are located. This is analogous to identifying which parts of the tourism value chain are priorities for intervention, but is based on wider considerations than PPI. For example, clusters which are located in rural areas, are labour-intensive, involve informal and unskilled labour, women or child labour, have weak social provisioning, social capital and institutions, rate most highly in their potential for poverty impact. Several indicators are suggested for use in an impact assessment (Box 5, p. 36 and Box 6, p. 38), but as the authors point out, much depends on the context. Box 5. Examples of types of indicators or ‘impact criteria’ Stakeholders

Areas of change: positive poverty reduction impacts

Areas of change: negative poverty reduction impacts

Entrepreneurs

Increasing revenues Enhanced standard of living Reducing dependence on single trader/market Increasing formal training/skill Increasing access to credit Better information and contacts Less discrimination Greater participation in cluster Improved governance

Falling revenues Reduced standard of living Higher dependence on one trader or market No change in training/skill Poorer access to credit Isolation from information and contacts More discrimination Poorer participation in cluster Reduced governance

Workers

Increased wages Enhanced standard of living (e.g. housing) Longer periods/more stable work More skills training/experience Increased employment benefits (pensions, social security) Improved conditions of work (e.g. hours, contracts) Better health and safety (e.g.

Falling wages Poorer standard of living (e.g. housing) Shorter periods/less stable work Less skills training/experience Reduced employment benefits (e.g. pensions, social security) Poorer conditions of work (e.g. hours, contracts) Poor health and safety (e.g.

44

Stakeholders

Areas of change: positive poverty reduction impacts

Areas of change: negative poverty reduction impacts

chemicals, machines) Less discrimination (e.g. wages, jobs, training) Gender empowerment (e.g. more female employment) Freedom of Association

chemicals, machines) More discrimination (e.g. wages, jobs, training) Less gender empowerment (women have lost jobs) No freedom of association

Households

Increased and stable income Decent housing Access to childcare Social networks and support Equitable distribution within household (work, income, decision making)

Debt burdens Migration Loss of social capital/support networks; Lack of childcare Unequal household distribution (income/work/decision making)

Local community

Improved services Improved social capital Clean and safe environment

Reduced services Reduced social capital Environmental degradation

Extracted from pp. 36–37

This report is useful in that it draws on general M&E issues (such as proving versus improving, quantitative versus qualitative versus proxy indicators) while applying them to issues of enterprise development. On purpose: (p. 30) it highlights: ‘An important factor in the type of the impact assessment is whether it is aimed at: Proving impact—for example the upward accountability of a project to donors (or more recently downward accountability to beneficiaries). This puts greater emphasis on objective and accurate measurement of the impacts of policy interventions. It often involves a top down approach, carried out over a longer time frame, and using “scientific” research methods. Improving impact—using impact assessment as a learning process to enhance policy. It involves understanding the process of an intervention with the aim of improvement (even as the impact assessment itself is being undertaken). It uses a more bottom up approach, accepts a degree of subjectivity, and can be carried out over a shorter time frame (Bird, 2002).’

This Appendix provides some detail on steps of an impact assessment, including six steps from planning to feedback, selection of different participatory tools, and indicators for poverty mapping.

Economic analysis of tourism projects Tourism for pro-poor and sustainable growth: economic analysis of tourism projects. Tun Lin and Franklin D De Guzman (2007). ERD Technical Note No. 20. Asian Development Bank. This technical note on how to assess economic impacts of tourism is useful, because it outlines conventional economic approaches that multilateral donors use, but does it (a) with some recognition of the need for a pro-poor perspective and (b) by comparing two different economic approaches. It compares ‘economic impact’ assessment, which calculates multipliers in order to add indirect impacts of tourism to the direct (‘first round’) affects, with cost-benefit evaluation, which ascribes a value to non-market goods so as to integrate them into one calculation of whether costs exceed benefits. It chooses to develop the first of these, multiplier analysis, because it focuses on benefits at national level, which offers best fit with the ADB perspective, and because it has been used to date. Different techniques that can be used are reviewed. Additional analyses are suggested (one paragraph each) to

45

add distributional analysis, financial and institutional analysis, and environmental sustainability and preservation principle. The report is useful for illustrating the contrast with a VCA approach. The similarity between VCA and multiplier analysis is that both focus attention on the impacts of tourism that occur through the supply chains and across sectors, rather than on merely direct impacts on tourism service providers. There the similarity stops, as the multiplier approach aims to conclude with one or two numbers (ratios) that indicate the level of increased economic activity (or income or employment) deriving from each unit increase in final demand (a technical term related to tourist spending) in tourism. It does not measure flows to particular groups, nor does it assess linkages between them. But it does provide more of the ’big picture‘ of tourism impact, in a way that is easier to compare with other destinations or interventions.