Literature Review of Evaluative Approach - Unicef

Literature Review of Evaluative Approaches for Education Strategies for Peacebuilding and Approach for the Evaluability Assessment of UNICEF’s Peacebuilding, Education and Advocacy Programme

The KonTerra Group Randall Salm and Clarence Shubert December 2012

700 12th Street, NW | Suite 700, Washington, DC 20005 | t. +1.202.351.6826 | f. +1.888.755.8925 | [email protected] www.konterragroup.net

Table of Contents Executive Summary .................................................................................................................... 3 1.0 Background ........................................................................................................................... 7 1.1 Armed Conflict and Peacebuilding ................................................................................... 7 1.2 The UNICEF EEPCT and PEACC Programmes .............................................................. 7 1.3 UNICEF Evaluations of Education for Peacebuilding ..................................................... 9 2.0 Methodology for Literature Review ................................................................................... 11 2.1 Meta-Analysis Approach ................................................................................................ 11 2.2 Selection of Studies......................................................................................................... 11 2.3 Study Limitations ............................................................................................................ 13 3.0 Peacebuilding Models and Evaluations .............................................................................. 15 3.1 United Nations Peacebuilding Model ............................................................................. 15 3.2 Peacebuilding Model of Ramsbotham, Woodhouse and Miall ...................................... 18 3.3 Peacebuilding Model of Lederach, Neufeldt and Culbertson ......................................... 20 3.4 Evaluation models of peacebuilding ............................................................................... 22 3.5 Conclusion on Peacebuilding and Evaluation Related to Education Programmes ......... 25 4.0 Education for Peacebuilding Programmes .......................................................................... 27 4.1 EEPCT Programme and Evaluation................................................................................ 27 4.2 PEACC Programme ........................................................................................................ 33 4.3 Non-UNICEF Programmes and Evaluations on Education for Peacebuilding .............. 38 4.4 Academic Studies Related to Education for Peacebuilding ............................................ 42 4.5 Lessons Learned for M&E of Peacebuilding, Education and Advocacy........................ 46 4.6 Conceptual Model for Education Interventions and Peacebuilding ............................... 48 4.7 Ethical issues for Peacebuilding Evaluations ................................................................. 49 5.0 Approach to Evaluability Assessment for PEACC Programme ......................................... 52 5.1 Why an Evaluability Assessment is needed for PEACC Programme ............................ 52 5.2 Purpose of the Evaluability Assessment ......................................................................... 52 5.3 Scope and focus of the Evaluability Assessment............................................................ 53 5.4 Issues to be reviewed and questions to be addressed in the EA ..................................... 55 5.5 Monitoring and Evaluation System ................................................................................ 57 5.6 Methods to be used in the Evaluability Assessment ....................................................... 60 5.7 Process and schedule for Evaluability Assessment ........................................................ 61 5.8 Qualifications of Team and Firm to Conduct EA ........................................................... 62 5.9 Tools to Support the Evaluability Assessment ............................................................... 62 References ................................................................................................................................. 63 Appendix A: ToR for Literature Review and Approach Paper ................................................ 67 Appendix B: PEACC Proposal Results and M&E, Reporting and Research ........................... 72 Appendix C: Tools for Evaluability Assessment ...................................................................... 81 Checklist for Programme Evaluability.................................................................................. 81 Participation – Empowerment Index .................................................................................... 82 Participation - Empowerment Index ..................................................................................... 82 Appendix D: Conceptual Model for Education Interventions and Peacebuilding .................... 83

2

Executive Summary More than half of children out of school live in a limited number of conflict-affected states which threatens the achievement of the Education for All (EFA) goals and Millennium Development Goals (MDG). Education in conflict affected countries is particularly important because it provides a return to normalcy and psycho-social support in addition to its other benefits. However, providing education for children in countries affected by armed conflict has proven difficult. With the support of the Government of the Netherlands, UNICEF’s Education in Emergencies Post-Crisis Transition (EEPCT) programme began in 2006 to put education on a path toward quality basic education for all. The programme directly provided access to or improved education for up to 6 million children per year in 42 countries and was extended through 2011. EEPCT has served as a catalyst for Education in Emergencies (EiE) contributing to the role of UNICEF as a leading partner in education in emergencies and post-crisis transition contexts. Despite the many achievements of the EEPCT, two major independent assessments, the Programme Review and Evaluability Study (PRES) and the Progress Evaluation (PREV), revealed the lack of a coherent theory of change and results framework, inconsistency in fund allocations and an inadequate monitoring and evaluation system. Building on the experience with the EEPCT, UNICEF again received funding in 2012 from the Government of the Netherlands for a new Peacebuilding, Education and Advocacy in Conflictaffected Countries (PEACC) Programme focused on education for peacebuilding in 14 countries. Education for peacebuilding is a broad term that includes restoring basic and conflict-sensitive education, but it goes further to include peace education to influence political, socio-economic and cultural institutions to create the necessary conditions for sustained peace. Through conflict analysis based programming, education for peacebuilding will cultivate inclusion, tolerance, respect and conflict resolution. The PEACC programme “will strengthen policies and practices in education for peacebuilding” through five key outcomes: 1. 2. 3. 4. 5.

increase inclusion of education in peacebuilding; increase capacities to supply conflict sensitive and peace education; increase capacity of children and others to cope with conflict and promote peace; increase access to quality conflict-sensitive education; and generate and use evidence-based knowledge on linkages between education and peacebuilding.

Drawing lessons from the EEPCT, it is critical that there is a strong monitoring and evaluation system to determine the effectiveness of interventions in achieving the key outcomes. In this context, UNICEF has commissioned this consultancy to perform the following tasks:

1. Summarize literature on evaluative approaches for peacebuilding programmes that use education as a central strategy, including a review of the EEPCT programme results framework and how it was implemented in a number of EEPCT participating countries. 2. Distil key lessons to take into consideration when developing the M & E strategy for the Education and Peacebuilding Programme; 3

3. Study the proposal document, and programme documents from two countries, to determine whether there is a coherent theory of change, and whether a reasonable M&E strategy has been laid out in view of the proposed theory of change. 4. Proffer advice on ethical issues/considerations likely to arise in monitoring and evaluation of the programme, and actions to mitigate them. 5. Proffer an approach for evaluability assessment planned for the first year of the Education and Peacebuilding programme, including draft terms of reference. The initial review examines three peacebuilding models: the United Nations peacebuilding model, Oliver Ramsbotham, Tom Woodhouse and Hugh Miall’s transformative cosmopolitan peacebuilding model that specifically recommends peace education as a formative strategy for peacebuilding, and John Paul Lederach’s reflective, integrative peacebuilding model. For each model we identify the principal components of each model, key theoretical assumptions, and evaluative approaches. The findings significant for evaluation methodologies of education for peacebuilding are:

• •

•

•

• •

Newness: peacebuilding per se has a short history so models, practices, and evaluation systems are still being tested. Sphere: the United Nations work is primarily to develop state capacities to legitimately govern and provide services whereas academic models encourage interventions with a wider range of actors. Targeted interventions: agreement exists on the need to identify conflict drivers or causes, and develop interventions to address them. Therefore if the nature of the conflict is educational, learned or social in nature, the best intervention is via the education sector. Time frames: the U.N. currently emphasizes early interventions in the first two years after peace agreements, while others promote longer time frames of decades. Continuity and sustainability are critical, along with constant assessment and adaptation of interventions to meet changing conflict dynamics and needs over time. Education interventions and evaluations must be responsive to this reality. Peacebuilding processes: how peacebuilding occurs is as important as what outcomes develop. Participative and equitable processes are necessary also in education. Child focused interventions for peacebuilding: How UNICEF’s core mandate in basic social services, social protection, and child rights relates to broader peacebuilding efforts must be clarified as well as how education for peacebuilding integrates with other sectors.

The next section summarizes the literature on education programmes specifically promoting peacebuilding, focusing on evaluation issues and experiences. This section begins by looking closely at the EEPCT experience, then examining the PEACC programme, evaluations of other UNICEF and NGO education for peacebuilding experiences, and finally academic literature. Some lessons from this review are as follows:

• •

Education for peacebuilding is a new field and the concepts and linkages between education and peacebuilding are still not well defined. UNICEF and other actors are already conducting many programmes that could be considered education for peacebuilding (e.g. Timor-Leste, Nigeria). 4

• • •

•

• •

•

•

•

Evaluations should be systematically conducted and made readily available to expand knowledge generation for all stakeholders. The EEPCT experience showed large scale results with millions of children benefiting from the program yet an uncertain impact on existing armed conflicts or future peace. Political and funding issues that constrain planning should be documented to understand how they impact the programme. For example, PEACC goals and countries being agreed with the donor without significant field participation, circumventing critical planning steps and conflict analysis, an important lesson learned from the EEPCT. Broad conflict analysis is needed to assess the extent to which the drivers of conflict are education related and hence the size and role of education in the overall peacebuilding programme, but the conflict analysis also needs to be education-specific to develop a strategic approach to education for peacebuilding. Further research is needed on the relationship between education and peacebuilding to confirm fundamental assumptions underlying the PEACC programme. Education for peacebuilding should be conflict-sensitive but should go further in building toward a positive peace with the following characteristics: child protection, psycho-social support, reintegration of combatants, refugees and displaced children and communities, economic recovery and self reliance, and socio-political reform to reduce conflict drivers. Education for peacebuilding need not be limited to the formal school system and may include community-based education for reconciliation using theater, music and art or peace camps that bring together mothers and children from conflicting groups. Further research is needed on the forms of education that address the range of child development issues relevant for peacebuilding, including psychosocial development, personal and ethnic identity, interdependence, and tolerance. Given the newness and uncertainties in the concepts and practice of education for peacebuilding, the programming approach is necessarily broad allowing for great variation and experimentation and the monitoring, evaluation, research and knowledge management functions should be built in as core components of the programme.

The review then looked at a limited number of evaluations of education for peacebuilding. Some implications that can be drawn from these cases are: • • •

•

•

It is difficult to apply the results based programme approach and to develop a results framework that has clearly defined outcomes that are broadly applicable. It is also difficult to define the programme components or activities and the processes and linkages that lead to the desired outcomes. The differences between general education, conflict-sensitive education and education for peacebuilding are largely qualitative and difficult to define as indictors for purposes of monitoring and evaluation. The evaluations have been in specific contexts which enabled a more limited focus on the linkage between project actions and specific results related to one aspect of peacebuilding. For the PEACC programme, the M&E system must be much more complex. It has to be evaluated at the: 5

o global level in terms of effectiveness of policy advocacy, interagency coordination, and funding mobilization and allocation; o national level in terms of policy advocacy, institutional capacity building, budget and fund allocation, research and information management; o district, school and community levels in terms of effectiveness of project actions; capacity development of teachers, students and others; and increased inclusiveness, access, and participation. The final section of this report provides an approach to an evaluability assessment (EA) that focuses primarily on the requirements of the monitoring, evaluation and research functions of the PEACC programme. It begins with why an EA is needed, the purpose of the EA and the scope and focus. It then gives a detailed review of the issues and questions to be addressed at each level. Particular attention is given to sharpening the results framework and outcome indicators to make them more specific, measurable, achievable, relevant and time bound (SMART). They should also be relevant at different levels and be capable of being aggregated at the global level. At the country level a major focus will be on the processes of conflict assessment and programming and the availability of information sources and baseline information in conflict affected areas. The EA will also consult the regional advisors on the regional and cross border aspects of conflicts and how regional cooperation may enhance the country level programme implementation and particularly monitoring and evaluation. In order to enhance interactive learning, the EA will be done through an open consultative process concluding with stakeholder workshops at both global and country levels to present and discuss findings and recommendations.

6

1.0 Background 1.1 Armed Conflict and Peacebuilding Recent global and local events have led to an expansion of UNICEF’s education programming in support of peacebuilding globally. Since the early 1990s there is growing interest in peacebuilding in armed conflict or post-conflict settings as a means to address the complex underlying causes of recurring conflict that inhibit full human development, and to promote Millennium Development Goals and other global development objectives. These peacebuilding efforts by the United Nations and international development partners have predominantly addressed security, political, economic, and human rights concerns. To a lesser extent, the provision of basic social services such as education has been a part of peacebuilding as well. Armed conflict may impact education in many ways. Approximately 67 million children are out of school globally, of which 42% or 28 million live in countries affected by armed conflict. Many teens living in countries in armed conflict are similarly out of school. Armed conflict leads to destroyed schools and learning materials, cancelled classes, displacement of teachers, traumatized students, disrupted social structures, and lost learning among other outcomes. In severe cases it provides space for child soldier recruitment and conflict-promoting indoctrination (UNESCO 2011 EFA Global Monitoring Report; Bush and Saltarelli 2000). Girls may be particularly affected, through rape and sexual violence, which often leads to stigmatization and exclusion from school. Disruption of schooling for children and teens also violates basic human rights norms and inhibits their developing the skills to sustainably meet their future basic needs. 1.2 The UNICEF EEPCT and PEACC Programmes UNICEF has a lengthy track record of working on education in armed conflict, extending back to work in the early 1990s on peace education in armed conflicts. More recently, UNICEF has been involved in two major education programmes in support of peacebuilding, the Education in Emergencies and Post Crisis Transition (EEPCT) programme, which is just ending, and the Peacebuilding, Education and Advocacy in Conflict-affected Countries (PEACC) programme, which is just starting. One major difference between the EEPTC and the existing PEACC is that the EEPCT focuses on emergencies arising from natural disasters and conflict, while the PEACC only addresses post conflict peacebuilding interventions. While this is still a large purview, it is narrower than the EEPCT scope. Nevertheless, much of the EEPCT experience is highly relevant for the PEACC. The EEPCT operated in 39 countries using a more flexible and responsive model than previous models, which enabled faster funding of critical projects and better coordination with partners. UNICEF seeks to use the EEPCT experience to improve the design, implementation and evaluation of the PEACC. With principal funding from the Government of Netherlands, the EEPCT programme operated from 2007 to 2012 in 39 countries, although the length and intensity of each country programme varied considerably. Cross-cutting issues addressed by the EEPCT include human rights approaches, gender equity and empowerment, conflict and fragile state sensitivity planning, and improved coordination (PREV 2010). The EEPCT goals were: •

Overall Goal: To put education in emergency and post-crisis transition countries on a viable path of sustainable progress towards quality basic education for all. 7

• • • •

Designated Goal No.1: Improve quality of education response in emergencies and postcrisis transition countries. Designated Goal No.2: Increase resilience of education sector service delivery in chronic crises, arrested development, and deteriorating contexts. Designated Goal No.3: Increase education sector contribution to better prediction, prevention and preparedness for emergencies due to natural disasters and conflict. Designated Goal No.4: Strengthen evidenced-based policies, efficient operational strategies and fit-for-purposes financing instruments for education in emergencies and post-crisis situations.

The EEPCT programme followed the general UNICEF model, considering inputs, outputs, outcomes and impact (PRES 2010). The design of the EEPCT results framework followed standard UNICEF programme design, with an overall goal; strategic objectives or impacts (the four designated goals above) and associated impact indicators; thirteen intermediate results or outcomes with associated indicators; output indicators for each designated goal; problems and challenges; and interventions for each of the four sub-goals (PRES 2010 Appendix H). The framework further identifies pertinent levels of analysis for data collection, including global, regional, national, district (sub-national) and school or community levels, as well as the need for capturing pre-existing and or new baseline data for evaluating results and impacts, and factors that should be disaggregated, such as gender, urban/rural, and conflict status (PRES 2010 Appendix H). Specific EEPCT programme interventions included: Child Friendly School Initiatives that combine a human rights approach with more motivating learning environments; Accelerated Learning programmes for over-age students; Learning Along Borders (LAB4LAB) for children living in border areas of conflict countries; country specific peace education curricula; Talent Academies; Schools as Zones of Peace (SZOP); psychosocial support for trauma affected populations; and work with the larger local community and parents (PREV 2010). Both the EEPCT and PEACC are explained further below in the section on Education for Peacebuilding Programmes. Building on the experience and lessons learned from the EEPCT programme and support from the Government of Netherlands, UNICEF recently started implementing the PEACC programme to systematically expand education programmes in support of peacebuilding during the period from 2012 to 2015. The PEACC programme proposes sustained, intensive intervention for four years in up to 14 countries. The programme’s vision is to strengthen resilience, social cohesion and human security, with the strategic result of strengthening policies and practices in education for peacebuilding (UNICEF 2011 Proposal on Peacebuilding and Education). Its outcomes are: 6. Increase inclusion of education into peacebuilding and conflict reduction policies, analyses and implementation. 7. Increase institutional capacities to supply conflict sensitive and peace education. 8. Increase capacity of children, parents, teachers and other duty-bearers (such as guardians and local leaders) to prevent, reduce, and cope with conflict and promote peace. 9. Increase access to quality, relevant, conflict-sensitive education that contributes to peace. 8

10. Contribute to the generation and use of evidence and knowledge in policies and programming related to education, conflict and peacebuilding. 1.3 UNICEF Evaluations of Education for Peacebuilding One critical aspect of UNICEF’s education for peacebuilding work has been the development of systematic evaluation tools to determine the effectiveness of interventions. Two major evaluations were done on the EEPCT programme, the Programme Review and Evaluability Study (PRES 2010) and the Progress Evaluation (PREV 2010). Both the PRES and the PREV were global evaluations with a few country studies in each assessment. The PRES was completed in May 2010, late in the EEPCT programme cycle, by a team from the University of York’s Post-war Reconstruction and Development Unit. Using a mixed methods approach including a document review, semi-structured interviews, focus groups and community surveys, the PRES assessed the overall programme design and implementation, operational issues, and the results framework from an evaluability standpoint. Case studies were completed in Kenya, Nepal and Southern Sudan of the EEPCT (PRES 2010). The PRES (2010) acknowledged constraints with evaluability of programmes in crisis-affected countries and proposed an architecture with multi-level indicators for evaluability assessments of education in emergency programmes. In December 2010 the EEPCT Progress Evaluation (PREV) was completed by the Columbia Group for Children in Adversity at Columbia University. The purpose of the PREV was “to identify and assess progress in the strategic goals of the Programme and to enable systematic reflection that results in concrete programme improvements.” (PREV 2010:6). The PREV used similar mixed methodologies as the PRES, including a review of documents, semi-structured interviews, focus groups and community surveys covering all 39 EEPCT participating countries. In addition, EEPCT case studies were conducted in Angola, Colombia, Cote d’Ivoire, Liberia, the Philippines and Sri Lanka. Findings were identified for programme design and management, noting the original EEPCT programme design was both “ambitious,” “imprecise” in terms of results (PREV 2010:6), with problems related to tracking funding by goal or activity, consistency and timing of funding, communication regarding programme objectives, and prioritization of global programme objectives over country programme goals (PREV 2010). Building on the PRES and PREV and to assist the monitoring and evaluation of the PEACC, UNICEF has commissioned this consultancy to perform the following tasks (UNICEF 2012 Terms of Reference): 1. Summarize literature on evaluative approaches for peacebuilding programmes that use education as a central strategy for achieving intended outcomes, including a review of the EEPCT programme results framework put in place after the PRES and how it was implemented in a number of EEPCT participating countries. 2. From 1 above, distil key lessons to take into consideration when developing the M & E strategy for the Education and Peacebuilding Programme; 3. Study the Education and Peacebuilding Programme proposal document, and programme documents from two participating countries, determine whether there is a coherent theory of change (from both the peacebuilding and education practices) that influenced programme development, and whether a reasonable M&E strategy has been laid out in view of the proposed theory of change. 9

4. From 3 above, proffer advice on ethical issues/considerations likely to arise in monitoring and evaluation of the programme, and actions to mitigate them. 5. Proffer an approach for the evaluability assessment planned for the first year of the Education and Peacebuilding programme, including draft terms of reference (TORs) for the evaluability study. This paper addresses all five tasks, evolving around the literature review and evaluability assessment approach, leading to its TOR. This study’s purpose is to summarize past evaluations and research related to education for peacebuilding and provide guidance for effective monitoring and evaluation of the PEACC programme.

10

2.0 Methodology for Literature Review The methodology for this literature review includes a review of materials on evaluation methodologies of peacebuilding programs generally and education programs focused on peacebuilding specifically. The consulting team first met with the pertinent UNICEF headquarters evaluation officer and other key UNICEF staff to confirm the parameters for the study, the countries for the case studies, and outputs and formats needed in the literature review and approach paper so as ensure that this product met UNICEF’s needs. At the recommendation of the UNICEF education evaluation officer, this literature review and analysis started with a summary of several peacebuilding models and their evaluation methodologies. This analysis then proceeds to review the literature on education programs related to peacebuilding, focusing especially on evaluations of such programmes, including EEPCT, PEACC, and other, non-UNICEF programmes, and relevant academic studies. This is followed by the development of lessons learned for the monitoring and evaluation of the PEACC programme, a draft conceptual model for understanding the relationships between education sector development and peacebuilding, and an assessment of whether a coherent theory of change and feasible M&E strategy exists in the PEACC programme. The last section on education for peacebuilding identifies potential ethical issues that may arise with this work in the fields of education and peacebuilding. In addition to a review of literature on education for peacebuilding, we develop recommendations for an approach to the evaluability assessment of the PEACC in the last section of this document. This approach considers the lessons learned during the literature review and discussions with key UNICEF headquarters, regional and country staff. 2.1 Meta-Analysis Approach Using a meta-synthesis methodology1 with the goal of summarizing the literature on evaluative approaches for education in peacebuilding programmes, we framed this literature review around the research question: “What evaluative approaches have been used to evaluate education programmes for peacebuilding and what findings, challenges and strengths have developed from the use of these evaluative approaches.” This research question guided the scope of the research, focusing on those education in peacebuilding experiences that have been formally evaluated. A meta-synthesis analysis attempts to integrate and identify common themes across multiple studies to develop new conceptual frameworks. When possible, this literature review identifies theoretical models underlying evaluations of education for peacebuilding interventions, relevant theories of change, evaluation methodologies, and indicators used. 2.2 Selection of Studies 2. As required by the UNICEF TOR, the principal studies to be reviewed under task 1 include the EEPCT programme results framework with several EEPCT participating countries. The main selection criteria for these case study countries were whether evaluation documentation was available and geographical diversity. This led to the selection of Liberia, Myanmar and South Sudan as case studies of evaluation models under the EEPCT programme, mainly using 1

Walsh, Denis, and Soo Downe. 2005. “Meta-synthesis method for qualitative research: a literature review.” Journal of Advanced Nursing 50(2):204–211. Retrieved September 3, 2012.

11

country report documents found on the Back on Track website. For the study of the PEACC programme, selection of the cases studies was contingent on which countries had advanced the furthest in programme implementation, specifically the country-level conflict analysis on the education system and development of the Education and Peacebuilding programme design. UNICEF education staff provided guidance on this country selection, leading to the selection and analysis of the Sierra Leone and Liberia PEACC country programmes. The principal search methodologies for evaluations of education for peacebuilding were searches on Google, the Social Sciences Citation Index, and certain peacebuilding evaluation websites including ALNAP Evaluation Reports Database, CR Info – Completed Evaluations, DAC Evaluation Resource Center, Search for Common Ground Evaluations, and Design, Monitoring and Evaluation for Peacebuilding. We also were given access to the Education in Transitions (Back on Track) website used by the EEPCT program. We utilized a snowball technique, whereby after identifying a relevant source, it was examined for additional relevant resources. Relevant studies are described below in the education for peacebuilding section of this document. Search terms used for the Google and SSCI databases mainly included combinations of the terms evaluation, education, peacebuilding and armed conflict. The types of peacebuilding sector activities identified by Jim Rogan (Sept. 25, 2012 meeting presentation), such as basic education services, child protection, education capacity building, youth life skills and peace education, were also searched for on the various websites visited in this literature review. It should be noted that very few studies or evaluations were found that specifically address evaluations of education programmes oriented towards peacebuilding. While many education programmes exist in armed conflict and post-conflict settings, few specifically link their objectives to peacebuilding. For example, a review of education theme documents of the UNICEF evaluation database found evaluation documents on basic education in Myanmar, civic education in Serbia, Go to Schools in Sudan, and Child Friendly Schools globally in conflict or post-conflict countries, but none that directly addressed peacebuilding. Similarly, a review of the Back on Track website, which contains many EEPCT country progress reports, found limited information on evaluation of the programme. Below is a summary of key websites that were reviewed and the types of documents found. • • • •

•

Within the online UNICEF evaluation database, no pertinent documents were found. The Back on Track/Education in Transitions website contains EEPCT country reports and PEACC planning documents, but access is limited and evaluations were scarce. The DAC Evaluation Resource Center and CR Info websites contained no pertinent evaluations. Within the ALNAP.org website, evaluations were found of a 2007 CORD programme in Chad and Darfur, a 2004 Tanzania refugee camp education programme, a 2001 North Caucasus psychosocial training for education professionals, a 2001 UNICEF Yugoslavia comprehensive education programme and a 2001 mine risk education programme, but none had specific references to peacebuilding. The Search for Common Ground website contained evaluations on a 2007 Cote d’Ivoire civic education programme, a 2011 Indonesia religious freedom education project, a 2012 12

•

•

Rwanda conflict resolution project, as well as other similar documents which were reviewed. Only the Rwanda study made mention of peacebuilding, with the intention of the project to inform the larger field of peacebuilding, but no further references to peacebuilding existed in the evaluation. The Design, Monitoring and Evaluation for Peacebuilding website did not contain any pertinent evaluations of education for peacebuilding documents. Under the Good Governance/Education tab, several general education programme and evaluation design documents exist, but nothing specific to evaluations of education programmes for peacebuilding. Under the Tools/Civic and Peace Education tab, several education programme design and programme evaluation documents exist, but these had limited direct connection to larger peacebuilding efforts. In the Key Issues/Peacebuilding tab, most documents do not address education. Few academic articles address the niche field of evaluation, education and peacebuilding. A few of the relevant documents are noted below.

In effect, the search for documents was wide ranging, yet found limited material on formal or systematic evaluations of education programs directly oriented toward peacebuilding. The novelty of the work on education and peacebuilding may partially account for this gap. Similarly, the fact that grey material, whether by the United Nations, non-governmental organizations, and private research centers, is often unpublished, may account for the scarcity of material (Tomlinson and Benefield, 2005). The result is that while a fairly extensive search for relevant material was conducted, few studies were found that directly address evaluation, education and peacebuilding. 2.3 Study Limitations Several limitations existed for the review of existing literature and the EEPCT and PEACC programme documents. First, we acknowledge that the scope of review of literature on the topic was limited by the time allotted for this study (24 days total). Any such study makes informed decisions as to what should be included and omitted, based on the scope of work, relevance of material and potential benefit to the end product. While considerable time was given to searching for material, the limited time for research influenced search results. With more time, additional materials may be found. Secondly, subjective determinations of relevance were necessary to determine which documents were pertinent. For example, many documents exist on education and armed conflict, but few of these programmes address peacebuilding or evaluation, as found with many UNICEF and NGO post-conflict education programmes that have no explicit references to peacebuilding. The problem of reviewing multiple fields of work (education, peacebuilding, evaluation, armed conflict, etc.) required a persistent focus on this study’s research question and purpose. Third, the TOR and our research proposal mention review of EEPCT evaluation and PEACC planning documents, but only a few documents were available from UNICEF (the EEPCT and PEACC proposal documents, and the EEPCT PRES and PREV global evaluations). Documentation of important programme design aspects were difficult to find or access. For example, explanation of the EEPCT results framework was scarce, as is noted below. Likewise, for the review of the PEACC results framework in selected countries, the PEACC planning and 13

results framework documents for Sierra Leone and Liberia were early drafts, because the programme was in an intensive startup phase as this report was written. While we acknowledge the limited availability of relevant documents, we appreciate the assistance of UNICEF staff in facilitating access to the many UNICEF resources we reviewed. In effect, this study is limited by the lack of material on the nexus of evaluation, education and peacebuilding. The time limitations, novelty of the field, and lack of available EEPCT evaluation and PEACC results framework documents significantly limited the implementation and outcome of this study.

14

3.0 Peacebuilding Models and Evaluations This review of peacebuilding models examines three models selected because they represent the United Nations peacebuilding model or address the combined issues of evaluation, education and peacebuilding. The models selected include the United Nations peacebuilding model, Oliver Ramsbotham, Tom Woodhouse and Hugh Miall’s transformative cosmopolitan peacebuilding model that specifically recommends peace education as a formative strategy for peacebuilding, and John Paul Lederach’s reflective, integrative peacebuilding model. For each model we identify the principal components of each model, key theoretical assumptions, and evaluative approaches, if available. 3.1 United Nations Peacebuilding Model The United Nations peacebuilding model is a relatively new endeavor, with the initial impetus in 1992 with the Secretary General’s report An Agenda for Peace and gaining significant momentum in 2005 with the Secretary General’s proposal In Larger Freedom. To manage the UN peacebuilding efforts, an Addendum to the 2005 In Larger Freedom policy document proposed new units with peacebuilding responsibilities. These include the Peacebuilding Commission (PBC), Peacebuilding Support Office (PBSO) and the Peacebuilding Fund (PBF), which are meant to coordinate existing UN units working on peacebuilding, not create new implementation units. “The PBC was created to provide accompaniment to countries during the most vulnerable decade post-conflict, by mobilizing political, financial and technical support. The PBF was designed to provide fast and flexible funding, to catalyze peace processes and donor funds, and to fill emergency gaps in financing for peacebuilding. The PBSO was mandated by the General Assembly and by the UN Policy Committee to assist and support the Peacebuilding Commission, manage the Peacebuilding Fund and support the Secretary-General’s efforts to coordinate the UN system in its peacebuilding efforts.” (UNPSO Strategy 20122013:1). The UN visualizes peacebuilding as a coherent integration of security, political, economic, social and human rights efforts. The UN uses the following definition: Peacebuilding involves a range of measures targeted to reduce the risk of lapsing or relapsing into conflict by strengthening national capacities at all levels for conflict management, and to lay the foundations for sustainable peace and development. Peacebuilding strategies must be coherent and tailored to the specific needs of the country concerned, based on national ownership, and should comprise a carefully prioritized, sequenced, and therefore relatively narrow set of activities aimed at achieving the above objectives.” (UNPSO Peacebuilding Orientation, 2010:5). Peacebuilding involves preventing ‘negative peace’ dynamics, such as escalation of armed conflict and violence, and promoting ‘positive peace’ dynamics, as with meeting the full range of needs of individuals and groups. Since 2009 the UN has strengthened efforts to promote peacebuilding in post-conflict countries during the critical first two years after cessation of hostilities. As they note: “The immediate post-conflict period offers a window of opportunity to provide basic security, deliver peace dividends, shore up and build confidence in the political process, and strengthen core national capacity to lead peacebuilding efforts thereby beginning to 15

lay the foundations for sustainable development.” (UNPBC Report of the SG on Peacebuilding 2009:1). This emphasis on effective early intervention promotes sustainable peace and prevents conflict escalation. However, considerable literature on peacebuilding recognizes that it also requires long term interventions that can take decades and generations to fully implement. The primary features and implementation phases of the UN model are worthwhile to examine. Core features of the UN peacebuilding include national ownership by its citizens, national capacity building, and strategy coordination by all actors with clear priorities. (UNPSO Peacebuilding Orientation, 2010; UNPBC Report of the SG on Peacebuilding 2009). While the first phase of peacebuilding is the conflict assessment following the cessation of violence, the UN recognizes that some peacebuilding efforts may have occurred during the conflict as well. The second phase leverages public interest in rebuilding that exists after the peace agreement and involves a range of interventions depending on the country’s priorities and existing capacities and resources. Frequent early needs include safety and security (DDR and rule of law), political processes (elections, reconciliation and conflict resolution mechanisms), basic services (water, sanitation, health, education), reintegration of internally displaced persons and refugees, restoration of key government functions (public administration, finance) at national and subnational levels, infrastructure and economic rebuilding (employment, livelihoods) especially for youth and demobilized combatants (UNPSO Peacebuilding Orientation, 2010; UNPBC Report of the SG on Peacebuilding 2009). Dan Smith summarizes four primary purposes of peacebuilding: “to provide security, to establish the socio-economic foundations of long-term peace, to establish the political framework for long-term peace, [and] to generate reconciliation, a healing of the wounds of war and justice.” (2004:10). These phase and sector concepts could help guide evaluation processes and indicators. The UN uses a variety of planning documents for peacebuilding, and the UNPBSO recognizes the need for enhancing coordination across these documents (UNPSO Policy Issues). Such instruments include the Integrated Mission Planning Process, Post-Conflict Needs Assessments, UNPBC’s Strategic Frameworks for Peacebuilding, the UN Development Assistance Frameworks, Poverty Reduction Strategy Papers and Integrated Strategic Frameworks. The timing and sequencing of sector interventions requires careful planning and monitoring (UNPSO Peacebuilding Orientation, 2010; UNPBC Report of the SG on Peacebuilding 2009). There is recognition that peacebuilding efforts have both successes and failures. Criticism of the UN peacebuilding model includes the fact that often the immediate interventions are imposed by Western countries and reflect Western values and interests, such as the argument against the Washington Consensus (van Leeuwen et al 2012). Deeper criticisms concern the lack of consideration of the collective identity of the citizens of the country, uncritical implementation of free market policies, and an overrepresentation of national issues compared to local concerns (van Leeuwen et al 2012), and deeper forms of human emancipation, such as individual-level emancipation and social values (Richmond 2007; Ramsbotham et al 2011). As Richmond argues, the emancipatory approach to human security: …means that individuals are empowered to negotiate and develop a form of human security that is fitted to their needs – political, economic, and social, but also provides them with the necessary tools to do so. This, by necessity, focuses on a broad notion of human security and on its external providers, but is aimed at 16

local agency as its ultimate expression. Human security is therefore focused on emancipation from oppression, domination, and hegemony, as well as want. It is thought of as a universal project, but one that is capable of being shaped by, and reflecting, local interests and particularities (2007:461). In effect, this deeper level of human security could be interpreted as the end goal of peacebuilding, and identifies possible evaluation criteria for education in peacebuilding such as local agency, human needs, and removal of forms of oppression. Current priorities in 2012 for the UN Peacebuilding Commission emphasize “enhancing its impact in the field and strengthening its relations with key actors at the Headquarters.” (UNPBC Road Map 2012:1). They include such areas as impact in the field, Headquarters actions, working methods, policy development, and implementation and tracking. Two current priorities for UN peacebuilding efforts are resource mobilization and mapping of relevant actors. Resource mobilization involves expanding the range of resources, including finance and technical assistance, through better mapping of the available resources and then improved coordination. As of early 2012 the PBSO was implementing a pilot project titled Aid Information Management for Peacebuilding to coordinate, monitor and report on aid to postconflict countries (UNPBC, 2012 Resource Mobilization). These planning and information resources may provide valuable data collection sources for evaluation of education for peacebuilding efforts. In general there is little inclusion of education within overall UN peacebuilding efforts. A review of the most recent UN Peacebuilding Fund report from 1 July 2010 to 31 December 2011 found no mention of education activities (UNPBF Report of the SG 2012), although UNICEF project flows on peacebuilding from 2009 to 2011 were approximately US$27.4 million. The Peacebuilding Fund methodology measures project contributions to four outcomes: “a) support the implementation of peace agreements and political dialogue, b) promote coexistence and the peaceful resolution of conflict, c) revitalize the economy and generate immediate peace dividends, and d) establish or re-establish essential administrative services” (UNPBF Report of the SG 2012:8). Outcomes are defined “as those changed behaviors (individual or institutional) or changed perceptions (e.g. increased confidence in the Government) that are estimated to contribute to peacebuilding.” (UNPBF Report of the SG 2012:19). Given the nature of education and its focus on human development and social relations, the PEACC programme would seem to best fit logically with the PBF’s second outcome on promoting coexistence and peaceful conflict resolution, and to a lesser extent, developing essential administrative services. The major exception to inclusion of education within United Nations peacebuilding efforts has been the EEPCT programme, implemented primarily by UNICEF, but it was not integrated well into larger peacebuilding efforts by other UN agencies or external actors. The EEPCT work is discussed further below. Finally, this review did not find much consideration of explicit theories of change within UN peacebuilding planning frameworks. Documents such as the UNPBSO Peacebuilding Orientation (2012), UNPBC Report of the SG on Peacebuilding (2009) and the Integrated Missions Planning Process make no explicit reference to theories of change. Implicit theories of change may be inferred from some UN peacebuilding documents. As noted earlier when 17

considering frequent peacebuilding needs, several issues are seen as critical to peacebuilding, including creating security, public confidence in the political processes, provision of basic services or peace dividends, and national capacity building, which imply that these types of actions will lead to peace. Basic security and safety - whether provided by the State or with international assistance - are essential to the population and to create the needed political space, and to enable the delivery of international assistance. Supporting the political process and reconstituting a stable and peaceful political order have to be central goals. The post-conflict government needs to build core State capacities that will help to restore its legitimacy and effectiveness, including the capacity to provide basic services and essential public safety, to strengthen the rule of law, and to protect and promote human rights. Visible peace dividends that are attributable to the national authorities, including early employment generation and supporting returnees, are also critical to build the confidence in the government and the peace process. Jump-starting economic recovery can be one of the greatest bolsters of security, and provides the engine for future recovery (UNPBC Report of the SG on Peacebuilding 2009:6-7). The assumption is that political decisions, state capacities and national ownership are related to public perceptions of legitimate and effective governing structures, which in turn enable provision of more basic services, which reduces unmet human needs, which promotes peaceful social relationships. However, this theory of change is implicit and not clearly described in the UN documents found in this study. To help narrow the conceptualization of the relationships between peacebuilding and UNICEF’s work, James Rogan, chief of the UNICEF EMOPS Peacebuilding and Recovery Section identified UNICEF’s work on peacebuilding through the following sectors and programme areas (Workshop presentation Sept. 25, 2012): • • • • •

“Basic social services and peace dividends: CFS, life skills, peace education, back-toschool, community-based management of services Safety/Security: Mine risk education, child combatants, police/peacekeeper training Core Government Functions: Capacity development at ministerial, subnational levels Economic: Youth, life skills, livelihoods, social protection Political/Reconciliation: Youth leadership, peace education, advocacy.”

This identification of types of UNICEF activities across five sectors which can support peacebuilding may help inform programme design and evaluation. The challenge remains to make the fit with education programming. Rogan also noted synonyms for peacebuilding which may help conceptualization of evaluation of education for peacebuilding models: “social cohesion, resilience, human security, community development, community management, etc.” (Rogan 2012 meeting presentation). 3.2 Peacebuilding Model of Ramsbotham, Woodhouse and Miall A second model of peacebuilding is provided by Oliver Ramsbotham, Tom Woodhouse and Hugh Miall (2011), who urge a cosmopolitan conflict resolution approach to peacebuilding. 18

They define peacebuilding “as the project of overcoming structural and cultural violence (conflict transformation), in conjunction with peacemaking between conflict parties (conflict settlement) and peacekeeping (conflict containment)” (Ramsbotham 2011:199). They suggest peacebuilding from below must be coordinated with cosmopolitan conflict resolution so as to limit the impact of global and external forces on local communities and the local conflict. The peacebuilding from below model was the dominant theoretical model from the 1990s through the early 2000s, and is currently being replaced by more transformative models. They provide an interesting schema for peacebuilding from below which identifies grassroots, mid-level, and top leaders in four sectors: psycho-social, economic-social, military-security, and politicalconstitutional. This model builds on the work of Reynolds Levy (2004) and J.P. Lederach (1997) and provides a useful framework for peacebuilding programme design and evaluation. Referring to previous language for peacebuilding, they provide an intervention, reconstruction and withdrawal (IRW) model that combines phases of intervention with specific sectors. The three phases are nested in the sense that they should be implemented concurrently, with their main tasks below (Ramsbotham 2011:210-213): • • •

Intervention: simultaneous stopping violence and making self-sustaining peace through institution building. This involves peacekeeping and high level peacemaking. Stabilization: promoting political stability and government institutional development and legitimacy. This involves structural peacebuilding. Normalization: supporting and sustaining the full range of social institutions necessary for a peaceful society. This involves social and cultural peacebuilding.2

Crosscutting these three phases is a wide range of critical activities across five sectors: • • • • •

Security – disarmament, demobilization and reintegration, etc. Law and order – policing, justice, human rights, etc. Government – constitutions, elections, taxes, representative governing systems, etc. Economy – building the official economy, ending the war economy, building employment, service provision, etc. Social and civil society – containing intergroup antagonisms, supporting minorities, refugees, and peace and justice efforts

This phase and sector matrix provides one cogent model for managing peacebuilding efforts, from program design through monitoring and evaluation. Moving beyond the peacebuilding from below model, Ramsbotham, et al, (2011) recognize that, while empowering local communities is valuable, there may still be power disparities and local forces that promote conflict and violence, which must be contained and transformed toward peaceful change mechanisms. They suggest using a cosmopolitan conflict resolution approach for peacebuilding that provides a framework for global governance founded upon cosmopolitan human rights while also supporting local 2

It should be noted that some debate exists regarding the ability of peacebuilding efforts to generate full normalization of post-conflict societies, given the potential for conflict escalation and problems with conceptualizing or affecting such complex social issues as community empowerment, gender equality and reconciliation (Ramsbotham et al 2011:222).

19

political autonomy and cultural diversity (Ramsbotham, et al, 2011:237). The principal mechanism for building peace suggested by Ramsbotham, et al, (2011) is peace education. Ramsbotham’s (2011) theory of change reflects discursive and systems models for addressing complex conflict structures, and engages multiple actors, spaces, actions, and communication feedback mechanisms. Their theory of change may be best seen (although not explicitly stated) through the nature of their cosmopolitan conflict resolution and transformative cosmopolitanism models. Cosmopolitan conflict resolution refers to “constructive means of handling conflict at the local through global levels” and transformative cosmopolitanism to “a genuine and inclusive local-global effort to determine what contributes to human welfare in general and to human emancipation worldwide.” (Ramsbotham et al 2011:265). Much of this change effort rests on political and economic cosmopolitan models, such as those of David Held, Paul Collier, Daniele Archibugi, Amartya Sen, and Immanuel Kant. Such authors identify distinct, but interconnecting concepts that are relevant for peacebuilding, depending on the specific context, including a rejection of violence and war, new spaces for democratic interaction, confrontation of power and security structures, reform of international institutions, human emancipation, local participation, and human diversity. One primary change model that can be presumed from Ramsbotham et al (2011) work is the social and communal nature of conflict. They argue that most current armed conflicts can best be explained by Edward Azar’s Protracted Social Conflict model which has four main components: 1) communal nature of conflict with identity groups (racial, religious, ethnic, etc.) as a basis for analysis; 2) social and human needs deprivation as a main driver of conflict; 3) poor state governance and under development cause needs deprivation; 4) external political, economic and military factors impact the communal-state conflict. This change model highlights the importance of social and cultural factors (especially group conflict dynamics and basic human needs), state functions, and globalization forces.3 Part of Ramsbotham and colleagues (2011) theory of change recognizes special organizational principles and explanatory theories in different state contexts. In those societies still governed by pre-state structures (often fragile or failed states), charisma and traditionalist explanations of how things work are most applicable. In those states or societies governed by an international system of states, power and realism best explain human interaction and armed conflict. In states or societies that operate more as an international society of states, social order exists through reciprocity, mutual interests, cultural pluralism, and non-intervention. In states that operate at an international community level, legitimacy and solidarity are the organizing principles, with both state and non-state actors finally having equal footing. Finally, the most idealist form is the world community, which is based on social justice and cosmopolitan governing structures. This change model depends on how power is used and whether international institutions seek full emancipation, progressive change and cosmopolitan values (Ramsbotham et al 2011:268-271). 3.3 Peacebuilding Model of Lederach, Neufeldt and Culbertson The reflective and integrative peacebuilding model of John Paul Lederach and colleagues (Lederach 1997; Lederach, Neufeldt and Culbertson 2007) is founded on a view of conflict as being deep-rooted, and involving broken social relationships and sociological dynamics of 3

See Edward Azar, 1991. The Management of Protracted Social Conflict. Aldershot: Dartmouth Press.

20

interpersonal and intergroup conflict. Much of what occurs in these armed conflicts pivots on individual and group identities at the local level, where basic needs of security and identity are left unmet due to a breakdown of governing structures, lack of participatory democratic spaces, and inequitable resource distribution (Lederach 1997:8). Characteristics of such conflicts include marginalized populations, poverty, inequality, lack of development, group collective issues, acceptance of armed confrontation, availability of small arms, and direct conflict that is more communal than international (Lederach 1997:17-18). These communal conflicts involve an interaction between relations, power, and awareness of conflict (Lederach 1997:65). This understanding of conflict as being more social, communal, and needs or rights-based has significant implications for Lederach’s peacebuilding model. Lederach conceptualizes peacebuilding as “a comprehensive concept that encompasses, generates and sustains the full array of processes, approaches and stages needed to transform conflict toward more sustainable, peaceful relationships. The term thus involves a wide range of activities and functions that both precede and follow formal peace accords. Metaphorically, peace is seen not merely as a stage in time or a condition. It is a dynamic social construct.” (1997:21). His model encourages transformation of three interrelated arenas, structures, social relationships, and peace infrastructure, and relationship building through reconciliation activities concentrating on truth, justice, mercy and peace (1997:22, 29-34). Lederach conceptualizes targeted peacebuilding activities for different actors, with senior leaders working in high level negotiations on issues such as ceasefire agreements; mid-level leaders participating in problem solving workshops, conflict resolution training, peace commissions, and insider, partial roles in mediation or conciliation efforts; and low-level actors involved in local peace commissions, grassroots training, prejudice reduction, and psychosocial trauma work (1997:39). Lederach’s theory of change reflects this framework and conceptualization of conflict as a social process. It requires focused actions at different levels depending on the nature of the conflict, from issues at the most concrete level, to relationships, subsystems, and finally systems level (1997:56).4 He views training as a critical component of any peacebuilding intervention, entailing “a process of strategic capacity and relationship building.” [italics in original] (1997: 108). Capacity building entails empowering individuals and groups to affect change, a “process of reinforcing the inherent capabilities and understandings of people related to the challenge of conflict in their context, and [italics in original] to a philosophy oriented toward the generation of new, proactive, empowered action for desired change in those settings.” (Lederach 1997:109). Relationship building involves reducing stereotypes and building skills so as to enhance relationships with both in-group members and parties in conflict, highlighting the interdependence of group and community-level actors. Both capacity and relationship building should be strategic in the sense of considering the broader conflict setting and sustainable change mechanisms. To guide such change Lederach proposes a detailed training matrix incorporating outcome and time factors and an infrastructure matrix that guides the areas of design and inquiry needed on issues of crisis management, people, institutions and visions, moving respectively from short to long term (1997:111-115). In effect, he posits that change is contingent on the vision, process and context needed for long-term, sustainable, peaceful relationships. 4

Both Lederach and Ramsbotham et al (2011) build their nested models off the nested theory of conflict by Maire Dugan (1996).

21

Lederach’s later work (2007) with Neufeldt and Culbertson reinforces the importance of conflict transformation (as opposed to conflict resolution), identifying four dimensions of change that occurs in conflict transformation. Personal changes may include emotional, spiritual, attitude and behavior changes. Relational changes exist between people in direct contact with others, and may include changes during conflictive escalation of communication, stereotypes, polarization, trust, cooperation, decision-making and conflict management mechanisms. Structural and systems changes address the nature of social relations and power issues at the level of families, organizations, communities and societies, and consider social conditions, and procedural and institutional trends. Cultural changes may include beliefs, norms, the meaning of things, traditional conflict resolution mechanisms, authority structures, and processes (Lederach, et al 2007:17-23). This model of personal and interpersonal change may be especially pertinent for UNICEF’s PEACC programme, given the common role of education for individual human development and social skill building. Lederach, Neufeldt and Culbertson note that multiple theories of change can exist depending on which of the four previous dimensions are relevant in the peacebuilding intervention. These theories of change are often implicit in the programme design and simply need full articulation of how participants are expected to respond to the proposed peacebuilding activities.5 As they note, “making the theory of change explicit can help the group identify further activities that may be needed as well as other actors or institutions that need to be involved in the program.” (Lederach, et al 2007:27). Helpful techniques for identifying change theories include diagramming or modeling the steps which need to occur to get to the end goal; examining stories from actual peacebuilding field practice to identify change mechanisms; and involving reflective or inquisitive outsiders to help identify change mechanisms (Lederach, et al 2007:27-28). 3.4 Evaluation models of peacebuilding The United Nations has recently started to formalize the peacebuilding monitoring and evaluation system, best summarized in the Monitoring Peace Consolidation report (2010). This summary of UN peacebuilding evaluation models and methods is taken from this report, which is an outcome of a two year peacebuilding M&E development process. The peace consolidation report also identifies and clarifies goals, models, and processes for peace building. Peace consolidation is visualized as the end goal of peacebuilding and defined as “a process leading towards a self-sustaining peace.” (United Nations 2010:23). It notes that this requires changes at the regional, national and sub-national levels. The report clarifies the nature of goals, inputs, outputs, and sector-wide and system-wide outcomes and impacts, and their interrelationships. The report utilizes peace consolidation benchmarks as referent points for knowing when peacebuilding interventions have reached their goals, and such benchmarks are situations or conditions where peace exists and is self-sustaining. The report proposes a seven step process to establishing peace consolidation benchmarks (2010:6-12): 1. Prepare for benchmarking: start from the beginning of the peace operation, work with national government and civil society, identify core tasks, and allocate resources. 5

Lederach, Neufeldt and Culbertson describe theories of change as “everyday expectations of “how the world works.” It requires articulation of common sense notions or assumptions of social behavior and relationships during conflict (2007:25).

22

2. Establish benchmarks: identify solid, realistic, contextual benchmarks and indicators, bearing in mind the goal of monitoring the main factors affecting peace and conflict. 3. Establish the data collection system: map data sources across quantitative and qualitative indicators. 4. Attribute indicators to benchmarks: use multiple indicators for each benchmark; use objective and subjective indicators, and indicators representing positive (e.g. teachers trained) and negative forces (e.g. schools destroyed), and compare outcomes from all actors with those just of national actors. 5. Aggregate and analyze data: combine data from lower to higher levels of analysis, and assess validity and reliability of data. 6. Establish the reporting mechanism: identify standard formats that are user-friendly, multi-dimensional and fair, and transparently share reports. 7. Evaluate benchmarking process and adjust as needed: improve quality and utility, and share lessons learned. The UN Monitoring Peace Consolidation report (2010) notes that peace benchmarks should be broken down by sub-units using one of three methods: sector approaches, such as education; use of peace agreement planks as a framework; and peace priorities from local society or conflicting parties. Three benchmark frameworks are identified (strategic, sector-based and process-based). The report also provides guidance on data collection methods and sources, including the use of existing sources, and statistical and non-statistical data, evaluating the quality of data in postconflict, peacebuilding contexts, and the types of indicators that may best work for monitoring peace consolidation (2010). Several key points exist from the Monitoring Peace report for this consultancy on evaluation of education for peacebuilding. First it should be noted that this document only refers to education once, in recognition of the need to respond to the specific conflict context when developing the monitoring mechanism. Countries more advanced in the post-conflict rebuilding process should identify peace consolidation targets such as demobilized combatants’ employment, youth unemployment, and basic government services in health and education. Second, we believe that a combination of sector and process-based frameworks may be best for evaluating education for peacebuilding efforts, since it can focus directly on the most effective strategies in the education sector for building peace at multiple levels (e.g., child protection, peace education, quality education) and identify critical process outcomes (e.g. resilience, cooperation, tolerance, or communication) that are necessary for peaceful social relationships, and which are primarily developed or socialized through formal or non-formal educational processes. In 1997 Lederach argued for a strategic and responsive evaluation model, which he elaborated on in the 2007 Reflective Peacebuilding document with Reina Neufeldt and Hal Culbertson. His 1997 work identified evaluation and funding dilemmas, working assumptions based on his framework, and the types of evaluation tools needed for a strategic and responsive peacebuilding evaluation. Lederach’s evaluation model reflects key principles of the field of conflict transformation, with the need for clear goals and values, feedback mechanisms, explicit change theories, contextualization of measurement methods and standards, and participatory evaluation methods (1997:135). Lederach (1997:135-7) posits several working assumptions for peacebuilding evaluation: 23

• • • • • • • • • • •

Recognition of change as a dynamic social process involving changing relationships. Measurement is needed on multiple levels (personal, relational, structural and cultural) Long term vision and design for wide ranging changes, and an explicit theory of change Social conflict continues during relationships with escalation and de-escalation stages Long term violence requires long term peace efforts Social conflict and the broader environment are constantly changing Balance the vision of change and short term actions Long term continued actions to support vertical and horizontal integration Continuous efforts of action and reflection Measurement of obscure relationship processes Descriptive evaluations by identifying issue and relationship dilemmas

His early work identifies several tools, which start with vision mapping of the widest picture feasible, followed by a series of inquiries on goals, assumptions, design, theories of change, strategic indicators, vertical and horizontal integration, conjunctural capacity responsiveness, and transformative capacity responsiveness (1997:138-146). These tools often contain strategic questions to guide each aspect of the evaluation process. Lederach, Neufeldt and Culbertson developed a targeted toolkit for peacebuilding planning, monitoring and evaluation in 2007 which reinforces the importance of using evaluations as learning experiences for funders, implementers and participants (2007:63). Evaluations can be used to learn from and modify project objectives or activities. They identify the need to start evaluation planning early, by emphasizing learning opportunities during the project design phase and selection of appropriate evaluation objectives, of which they posit several (2007:64). Various learning opportunities exist, including instrumental purposes to guide program continuance or strategies, conceptual usages to influence how people think about the intervention, and ritual or symbolic usages that simply reinforce what is already known about the issues at stake. Instrumental and conceptual usages of peacebuilding evaluation findings may generate the most use for all participants and stakeholders (2007:65). Since conceptual models may be the most beneficial, they recommend using participatory evaluation methods, widely disseminating the results, reflecting on other peacebuilding evaluations, and consideration of evaluations of similar projects during the project planning stage (2007:65-6). In their brief discussion of evaluation, Ramsbotham et al (2011) identify a number of key issues for evaluating the effectiveness of peacebuilding efforts, but do not propose a specific evaluation model. They raise key questions for evaluation: what signifies the best interests of local parties, how is peace and peaceful change framed, who determines the outcomes, what criteria can be used, and how positive and negative impacts can be assessed. They also urge the use of peace and conflict impact assessments and a range of pre-, midterm, and post-evaluations. They note that while evaluation methodologies have improved, such problems exist as counterfactuals (if X had or had not done Y, what would have happened), causality assessment when multiple factors exist, positive benefits at one level which create negative impacts at another level, and divergent interpretations of the same peacebuilding event (Ramsbotham et al 2011). A number of post-war reconstruction evaluations, such as the work of Durch (2007), Doyle and Sambanis (2006), Lund (2003), Diehl (2008) provide methodologies and criteria for evaluating these types of 24

interventions, but reconstruction is just one subset of peacebuilding activities. They also identify a number of sources for additional peacebuilding evaluation material. In addition, Dan Smith’s 2004 review of European peacebuilding efforts of the Utstein project emphasizes conducting strategic impact evaluation assessments instead of project level assessments, and timely communication of results to key stakeholders. He argues for assessing outputs and impacts, but notes that confirming relationships between outputs and peace impacts is difficult. In the worse case, what may be found initially to be a positive impact can change to become a negative impact for peace. The recommended strategic impact evaluations are similar to policy assessment/evaluations methods. Appropriate impact criteria selection is important, along with a long term, cradle to grave assessment strategy. Finally, given the noteworthy problems of peacebuilding evaluations, Smith argued for expanded academic research, such as social change, game and chaos theories, even though it is slower (Smith 2004:-15). 3.5 Conclusion on Peacebuilding and Evaluation Related to Education Programmes A number of findings from this brief review of peacebuilding models and evaluation are significant for developing evaluation methodologies of education programmes for peacebuilding. •

•

•

•

•

•

Newness of peacebuilding: while peace efforts have existed for ages, peacebuilding per se has a very short history. Models, practices, and evaluation systems are all fairly new and still being tested. Education and its evaluation methods must monitor changes in the broader peacebuilding field and adapt accordingly. Sphere: the United Nations work is primarily focused at the state level, working to develop state capacities to legitimately govern and provide services, along with necessary political processes. Academic models encourage broader interventions with a wider range of actors. The role of states becomes an intervening variable (process) in evaluating education for peacebuilding endeavors. The roles of states and other actors, such as local communities, for peacebuilding education must be critically examined. Targeted interventions: agreement exists on the need to identify conflict drivers or causes, and develop interventions the directly address those issues. This implies that if the nature of the conflict is something that is educational, learned or social in nature, the best intervention is via the education sector. If the driver is not educational in nature, the solution should not primarily be educational. Time frames: the U.N. currently emphasizes strong early interventions in the first two years after peace agreements, while others promote much longer time frames of decades and generations. Continuity and sustainability are critical, along with constant assessment and adaptation of interventions to meet changing conflict dynamics and needs over time. Education interventions and evaluations must be responsive to this reality. Peacebuilding processes: how peacebuilding occurs is as important as what outcomes develop. Participative and equitable processes are necessary. Education interventions may require special processes as well. Child focused sector interventions for peacebuilding: UNICEF’s core mandate leverages such areas as basic social services, social protection, and human, economic and political development for children and youth. How each of these areas relates to broader peacebuilding efforts must be clarified, as well as how education can support each area. 25

How exactly should education for peacebuilding integrate with the other sectors is vital for internal UNICEF planning.

26

4.0 Education for Peacebuilding Programmes This section summarizes the literature on education programmes specifically focused on promoting peacebuilding, focusing on evaluation issues and experiences. We attempt to summarize theoretical models, theories of change, evaluation indicators, relevant findings and knowledge gaps of education programmes that are directly or indirectly related to peacebuilding. We divide this section by looking closely at the EEPCT experience, then examine the PEACC programme, followed by evaluations of other UNICEF and NGO education for peacebuilding experiences, and finally academic literature on the subject. This review seeks to be informative for the PEACC evaluation methodology. Possible theories of change may exist for education for peacebuilding programmes. This review could not find mention of explicit theoretical models for the design of the EEPCT, nor for a programme-specific applicable theory of change, which were also noted by the PREV (2010:34) and the EEPCT Synthesis Report (Novelli and Smith 2011:24). One implicit theory of change hinges on conflict as the main driver of change. This model is evidenced by the Reflecting on Peace Practice model of the CDA Collaborative Learning Projects (2009) and Causal Analysis model of the Course on Conflict Sensitivity and Peacebuilding of UNICEF (UN System Staff College), which were utilized during EEPCT training and programme design, and found on the Back on Track website. Utilizing a systems approach, this conflict as a change factor model suggests that a comprehensive conflict analysis will identify key conflict drivers, actors and dynamics, which can be used to determine program strategies. Such strategies include determining where and with whom to intervene, identifying the programme approach, and guiding implementation, monitoring and evaluation (CDA Collaborative Learning Projects 2009). However, in general we find a lack of a coherent theory of change. While we acknowledge the difficulty in developing theoretical models and theories of change that may work across the various conflict and cultural contexts that the EEPCT and PEACC work in, such conceptualization is essential for informing programme design and evaluation. 4.1 EEPCT Programme and Evaluation 3. As mentioned, the EEPCT programme was mainly funded by the Government of Netherlands and operated from 2007 to 2012 in 39 countries. The EEPCT programme varied considerably across the 39 countries, with diverse interventions including Child Friendly School Initiatives that combine a human rights approach with more motivating learning environments; Accelerated Learning programmes for over-age students; Learning Along Borders (LAB4LAB) for children living in border areas of conflict countries; country specific peace education curricula; Talent Academies; Schools as Zones of Peace (SZOP); psychosocial support for trauma affected populations; work with the larger local community and parents; and other activities (PREV 2010). 4. The general UNICEF model was followed for the EEPCT programme, considering inputs, outputs, outcomes and impact (PRES 2010). The design of the EEPCT results framework followed standard UNICEF programme design, with an overall goal; strategic objectives or impacts (the four designated goals above) and associated impact indicators; thirteen intermediate results or outcomes with associated indicators; output indicators for each designated goal; problems and challenges; and interventions for each of the four sub-goals (PRES 2010 Appendix H). The framework further identifies pertinent levels of analysis for data collection, including 27

global, regional, national, district (sub-national) and school or community levels, as well as the need for capturing pre-existing and or new baseline data for evaluating results and impacts, and factors that should be disaggregated, such as gender, urban/rural, and conflict status (PRES 2010 Appendix H). The PREV programme evaluation used such criteria as relevance/appropriateness, effectiveness, efficiency, coherence, coordination, impact and sustainability (PREV 2010). 5. Considerable education results were documented from the EEPCT programme intervention. According to the UNICEF 2012 EEPCT Final Synthesis Report, these include restored access or improved education quality for 37 million children, as well as 11 million indirect child beneficiaries through other education strategies. Also, 300,000 educators were trained, 30,000 schools and learning spaces built or improved, and preparedness and risk reduction strategies developed in 42 countries. Strategic education sector planning was improved through the Inter-Agency Network for Education in Emergencies and education clusters. Finally, a number of innovative education methods have been developed or expanded (UNICEF 2012 EEPCT Final Synthesis Report). These results are valuable for improving education for some of the most vulnerable children worldwide, as well as for its support for Education for All and Millennium Development Goals. 6. The ability to conduct rapid country level engagements and improved global and regional education sector coordination were seen as two positive results of the EEPCT programme, albeit with caveats (PREV 2010:7-10). One PEACC workshop respondent noted that the EEPCT funded quick fixes like building schools, school supplies and teacher training that are always needed in post-conflict situations, but may not have much to do with peacebuilding. Other EEPCT country programmes used the funds to do research and introduce reforms that were responsive to emergencies and conflicts. This supports the idea that there must be a clear PEACC global programme structure so that this new programme is not seen as a general purpose fund. This consultancy was specifically tasked with reviewing the EEPCT results framework from an evaluation standpoint, but we find considerable difficulty in doing so given the lack of direct reference to and analysis of the EEPCT results framework. First, there is little mention of the results framework in EEPCT planning and evaluation documents. The Programme Review and Evaluability Study (PRES 2010) provides the most information on the results framework and critical framework issues, with the lesser information found in the EEPCT Progress Evaluation (PREV 2010). However, there was minimal direct discussion of the design and effectiveness of the results framework (see EEPCT Final Synthesis Report 2012). Secondly, few EEPCT planning documents were found for analysis in this literature review. If such documents exist, they are not readily available to the public or external partners, which severely limits their potential for knowledge generation. While mention has been made of changes to the EEPCT results framework in the middle of the program (following the PRES), we found no available documentation to identify what changes were made and for what purpose. A number of learning and evaluation gaps were identified in the PRES and PREV. At the broadest level, the PREV and PRES noted inherent problems with how EEPCT goals were defined, which were more focused on broad priorities than clear outcomes or results. While the EEPCT’s flexibility was often noted as a programme strength, such flexibility also created problems for evaluation. Programme evaluations suggested developing a large number of 28

indicators for each goal to address this problem, even though it created extra problems for monitoring and evaluation (PRES 2010:68). As the PREV clearly noted: “The monitoring and reporting system is broken. Fixing it will require engaging country, regional and headquarter actors in the critically important task of collectively identifying and agreeing to a set of indicators that are sensitive to field realities while also responding to the need to compile results globally.” (2010:10). These challenges underscore the problems of effectively evaluating education programmes at both global and local levels. UNICEF country level implementation reflect some of the EEPCT evaluation challenges, with office staff more often assessing outputs than outcomes or impact (PRES 2010:50). Local staff also had considerable confusion about the main purpose of EEPCT and key evaluation issues. For example, only 62.5% of PRES survey respondents stated that programme indicators had been developed and only 33% stated that baseline data had been collected prior to programme implementation (PRES 2010:50). Reasons for early problems with EEPCT evaluations included: a) fragmentation of responsibility for evaluation between headquarters, regional and country level staff, b) contextual problems such as security, transportation and infrastructure deficits, c) lack of development or application of evaluation indicators, and d) a lack of capacity, time and interest at the country level by UNICEF staff for conducting evaluations and knowledge management activities (PRES 2010:51-52). This identification of inconsistencies in country level evaluation efforts is valuable for the PEACC evaluation endeavor, especially the need for staff training, standardized data collection methods, and coherent logical models. The PRES also noted problems with evaluation methodologies and data collection in 13 studies conducted by UNICEF, Save the Children and Norwegian refugee Council, of which a significant part is due to the inherent problems of working in post-conflict environments. While 100% of these evaluations described their methodology, only 92% described data collection methods, 85% sample size, 62% analytical framework or approach and 62% methodological weaknesses. Fewer reports included their data collection tools (46%) or sampling approach (39%). None of the 13 EEPCT evaluation studies were experimental, 69% were reflexive or reflective, 31% were theory based, with most being qualitative (62%) or mixed methods (39%) (PRES 2010:65). One example of evaluation problems existed in reporting on Child Friendly Schools, with confusion about the level of involvement of schools and lack of integration of Child Friendly Schools construction and rehabilitation practices by local governments. Possible causes of this failure include lack of state recognition of the benefits of Child Friendly Schools and inappropriateness of the Child Friendly Schools model in armed conflict contexts (PREV 2010:7, 48). This last example is noteworthy for the PEACC evaluation, since Child Friendly Schools was considered one of the most important education interventions for peacebuilding of the EEPCT. The general problem of lack of systematic monitoring and evaluation of postconflict education interventions, and challenges inherent with such data collection and analysis, are worrisome for the PEACC programme. Since there is considerable documentation on findings of the main EEPCT evaluations, we simply summarize the main PREV 2010 findings here. •

Significant conceptualization issues exist of peacebuilding and UNICEF’s work on peacebuilding. Should UNICEF be involved in conflict and social transformation, which 29

•

• •

• • •

raises political and neutrality issues? Should UNICEF conduct peacebuilding during armed conflicts or just post-conflict? Integration with other UN peacebuilding offices. For various reasons, integration has been limited. How should UNICEF’s education work be integrated with other UN peacebuilding interventions?6 Consequences for UNICEF mandate and positioning. UNICEF has special strengths for peacebuilding. How can they best be used? Changing UNICEF country level programming, by expanding work with clusters and strategic partners, ensuring critical conflict analysis, and understanding limitations of education sector work. A comprehensive UNICEF capacity development framework must be developed for the education sector. Differentiate peacebuilding monitoring and evaluation goals, indicators and methods from other types. Improve local and global evaluation processes. Promote research and knowledge generation apart from evaluation work, by ensuring that research is independent, critical, and formative.

Review of several EEPCT country case studies helps comprehend how the global programme design and evaluation becomes implemented at the local level. We attempted to review theoretical models underlying evaluations of education for peacebuilding interventions, relevant theories of change, evaluation methodologies, and indicators used for each of these three countries, but were only able to find information describing results. It should be noted that few country evaluation documents were found for these three case studies, requiring use instead of annual programme reports found on the Back on Track website, which do not significantly discuss monitoring and evaluation issues. In the case of Liberia the EEPCT was funded from 2007-2011 to address both emergency programming and government capacity building (UNICEF Liberia 2012). Conflict drivers identified by UNICEF include widespread extreme poverty, geographical access due to poor roads and extended wet season, and ethnic diversity. The conflict had reduced the net primary enrollment ratio to 45% and destroyed over 805 of the schools. Most existing schools lacked teachers, drinkable water, latrines, and textbooks (UNICEF Liberia 2012). Government capacity building was supported by the Liberian Primary Education Recovery Programme for 2007-2010, through national policy framework interventions, Education Management Information System implementation, Child Friendly Schools, Accelerated Learning Programmes, water and sanitation infrastructure, and school supplies (UNICEF Liberia 2012). Results were documented for each of four EEPCT designated goals, differentiated by service delivery, capacity development, and impact. Liberian EEPCT measures for the 2010-2011 reporting period are indicative of country level results measurement for Myanmar and South Sudan: •

Improved quality of education: o Service delivery: water and sanitation, school books, first aid kits, internet connectivity, life skills and HIV education.

6

McCandless 2011 provides three suggestions: providing peace dividends, strengthening sector governance, and delivering peacebuilding results.

30

•

•

•

o Capacity development: local implementation of school census, textbook process improvement, parent-teacher association training, national-level gap analysis by stakeholders, LAB4LAB school building, and training for trainers for a pilot programme for girls’ achievement. o Impact: learning outcomes improved through quality of teaching and learning efforts, learning environment improvements through health and sanitation facilities and education, international student connections through the Connecting Classrooms Initiative, and an international visit to the LAB4Lab model school. Increased resilience of education service delivery: o Service delivery: training for and salary payment to accelerated learning teachers, national school census support, and county level education office construction. o Impact: enabling accelerated learning teachers to focus on teaching instead of other income generation activities, reduction of warehouse and training space costs, improved evidenced-based educational planning through EMIS and school census data. Better prediction, prevention and preparedness for education: o Service delivery: improved national and local emergency education planning with key partners, emergency supply delivery, parental education priorities identified through rapid assessments, education cluster training, bi-national refugee education planning between Liberia and Ivory Coast, and an external evaluation done in January 2011. o Impact: joint leadership of education and child protection at national and county levels, education normalcy supported via emergency supplies for Ivorian refugee children, use of assessment data for funding appeals and emergency planning, improved planning and coordination due to cluster training, improved likelihood of Ivorian refugee students to meet Ivorian graduation requirements, and utilization of evaluation findings for future emergency planning. Evidenced-based policies, efficient operations and fit for purpose financing: o Service delivery: national capacity building for school mapping, determination of the need for decentralized education decision making based on a national consultation process, education reform through passage of a New Education Act, evaluation of the pooled funding mechanism and strengthening of financial management. o Impact: benefits to students through funding and better resource distribution, school provision in underserved areas, new education legislation, reduced barriers to school entry, and increased responsibilities for local education units.

This extensive list of measures shows the diversity of impacts of the Liberia EEPCT programme for the full range of stakeholders, from national education ministries to local counties and districts, culminating with students and parents. However, the end impact of these activities for peacebuilding is difficult to measure. As the UNICEF Liberia 2012 report notes, no significant change in student enrollment occurred, the Ministry of Education still has limited management capacity, and poverty and transportation remain major barriers, along with a lack of in-country construction capacity. The report argues that the EEPCT intervention does have a positive impact, through “renewed faith in the potential of good quality education to transform lives,… 31

increasing access to more attractive, safe and hygienic schools,… an effective decentralized education system, [and] evidence-based policy development, programming, planning and management.” (UNICEF Liberia 2012: 18). However, there is little discussion of how these impacts directly influence the larger peacebuilding process. Furthermore, at least in the annual country reports, there is little mention of issues of evaluation, results framework, or theories of change towards long term peace. The results and gaps identified above are insightful for developing possible evaluation methods and indicators for the PEACC. The Myanmar EEPCT programme also ran from 2007 to 2011. Myanmar reports document such conflict drivers as “inappropriate economic policies, longstanding ethnic conflicts and international isolation” (UNICEF Myanmar 2012:4), leading to significant problems of poverty, health and education. Myanmar government spending on education had been historically low, causing parents to cover additional school expenses. Since poor parents are less able to pay such costs, education access is inequitable (UNICEF Myanmar 2012). Due to rapidly changing conditions, it was classified as both emergency and development operations. A summary of results from the Myanmar EEPCT programme for 2010-2011 documents expanded NGO partnerships, insertion of early childhood education programmes, psychosocial support to children and families, learning materials for students, school construction, curriculum development, Education Information Management System development, teacher training, and life skills activities for students (UNICEF Myanmar 2012). Analysis of the Myanmar EEPCT 2011 annual report shows little discussion of evaluation issues or theories of change. Some mention is made of possible political changes taking place, which may impact social and economic conditions. Such changes include policy support for vulnerable populations and improved service delivery (UNICEF Myanmar 2012). The South Sudan EEPCT experience provides a third case study of evaluation. Like Myanmar, one major conflict driver was renewed armed conflict with Sudan and local militias, following long term violence and large scale internal displacement. As of 2011 other conflict drivers include large numbers of displaced persons and returning refugees, and a fragile, newly formed government (UNICEF South Sudan 2012). In this context education problems are severe, include over 1 million primary aged children out of school, extreme gender inequality in schools, a shortage of teachers, and lack of school supplies and potable water in schools. The 2011 Southern Sudan EEPCT results are similar to the Liberia and Myanmar experience, with improvements in gender parity; construction of schools and temporary learning spaces that also support health and child protection; education sector management coordination through education cluster efforts and strategic planning at both national and state levels; and evidencedbased planning through development of the Education Management Information System (UNICEF South Sudan 2012). It is worthwhile to note that UNICEF supported a major country level evaluation of the Southern Sudan Go to School (GTS) Initiative which addressed school access, quality of education, partnerships and systems-level coordination, and accountability during the period 2006-2010 (Culver et al 2010). GTS inputs and expected outcomes are identified in the Culver et al (2010) evaluation, with similar activities as in the EEPCT: 32

Major component activities include increasing access for orphans and vulnerable children (OVC), particularly those out of school; teacher recruitment and training; infrastructure development for child‐friendly school environments; development of curricula and distribution of learning materials; construction of water and sanitation facilities; establishment of strategic planning and coordination mechanisms; pupil recruitment; literacy and numeracy outcomes; development of pupils’ life skills; strengthened capacity at the MoE; the inception of the Girls’ Education Movement (GEM); and the initiation of Parent‐Teacher Associations (PTAs). (Culver et al 2010: 6). The Southern Sudan study used the five OECD/DAC evaluation criteria as evaluation measures, and documents data collection methods and measures satisfactorily (Culver et al 2010). However, the GTS evaluation did not address any of the broader issues such as conflict drivers and impact for peacebuilding. More specifically, the GTS programme and the Culver et al evaluation seems to ignore how the GTS programme impacts the larger social, political and economic environment, and such conflict drivers as marginalization as mentioned in the Joint Assessment Mission Sudan evaluation (Bennett 2010). In sum, there is little mention of integration of the GTS into the larger peacebuilding effort. Several trends can be seen across these three EEPCT case studies which are relevant for evaluation concerns. In each country UNICEF and EEPCT partners worked extensively to develop the capacity of national, regional and local education offices. This is parallel to the broader United Nations peacebuilding efforts. Also, while some evaluation measures are similar across all or most countries, others measures are unique to each country. This hampers global monitoring efforts, and reinforces the need for a critical conceptualization of how each intervention promotes peacebuilding. In the case of Liberia, consideration of a return to normal conditions, the need for sustainable efforts, and role of school dropouts in conflict escalation (UNICEF Liberia 2012) reflects such consideration of peacebuilding goals. Third, the EEPCT country programmes make no reference to peacebuilding per se, in part due to its omission from the EEPCT goals. No clear explanation is provided as to how school construction, increased school enrollment, or accelerated learning impact national level conflicts or social relationships in local communities. Examples of exceptions are limited, such as the UNICEF Liberia’s 2012 report noting how school mapping enables resolution of disparities that may lead to conflict. Fourth, while each of these three case studies documents programme results, there is limited discussion of the coherence and integration of the results framework at the country level, nor issues with how the global results framework is implemented at the country and local levels. Such focused analysis of the results framework from multiple perspectives is necessary for programmes such as EEPCT and PEACC that combine global, regional, country and local design, implementation and evaluation dynamics. 4.2 PEACC Programme The Proposal on Peacebuilding, Education and Advocacy prepared in October 2011 by the UNICEF Education Section is the basis for the programme and the funding agreement with the Government of the Netherlands. A fundamental assumption of the programme is that education is an important and positive contributor to peacebuilding. The proposal problem analysis 33

identifies the problems underlying existing education in post-conflict settings, and justification for expanding education for peacebuilding, including the lack of integration of education into larger peacebuilding efforts, insufficient institutional capacity, the untapped transformative potential of education, insufficient access to quality and relevant education, and weak evidence base (UNICEF 2011 Proposal on Peacebuilding). The PEACC programme’s strategic vision is to “strengthen resilience, social cohesion and human security in conflict-affected contexts,” culminating in the strategic result to “strengthen policies and practices in education for peacebuilding,” with the strategic result indicator being “the number of improved policies, strategies and approaches adopted and implemented in education for peacebuilding in conflict-affected contexts.” (UNICEF 2011 Proposal on Peacebuilding, p. 32). The five outcomes and corresponding result indicators are identified below (UNICEF 2011 Proposal on Peacebuilding, p. 32). 1

2

Outcome Increase inclusion of education into peacebuilding and conflict reduction policies, analyses and implementation; Increase institutional capacities to supply conflict sensitive and peace education;

3

Increase capacity of children, parents, teachers and other duty bearers to prevent reduce and cope with conflict and promote peace;

4

Increase access for children to quality, relevant, conflict-sensitive education that contributes to peace; Adequate generation and use of evidence and knowledge in policies and programming on linkages between education, conflict and peacebuilding.

5

Result Indicator Number of peacebuilding and conflict reduction policies, analyses and implementation with education integrated Number of countries with institutions having measurably increased capacity to supply conflict sensitive and peace education Percent change in sample of children and adult community member perceptions of their own ability to prevent, reduce and cope with conflict and promote peace Number of children reached by quality, relevant education that contributes to peace (Male/Female) Number of policies at country level proposed and/or passed that are informed by programme supported evidence base.

The full results section (section 5.0) and monitoring, evaluation, reporting and research (section 6.0) of the PEACC proposal are attached in Appendix B. The PEACC proposal identifies UNICEF’s approach, key principles, and UNICEF’s capacity in post-conflict countries and on peacebuilding. The implementation strategy identifies standard proposal issues, including building on lessons learned and best practices in a range of related areas; the target population of education recipients under age 18, especially girls; the selection criteria for the 14 priority countries; the methodology and use of conflict analysis in early planning; and coordination mechanisms for the diverse knowledge and implementation partners (UNICEF 2011 Proposal on Peacebuilding). The proposal further defines the initial results framework and the process for refinement of the results framework at a programme design workshop in year one (which was the workshop held in Sept. 2012 at UNICEF headquarters), as well as later modifications based on baseline studies and the evaluability assessment. Regarding the connections between and need for monitoring, evaluation, reporting and research, the UNICEF PEACC proposal recognizes that the linkage between education and peacebuilding 34

is an emergent field of study with expanding hypotheses, but limited concrete evidence. The proposal identifies the need to establish data-based linkages between education and other peacebuilding sectors, such as security, political, economic and social fields, and between education and the root causes of protracted social conflict (UNICEF 2011 Proposal on Peacebuilding). The proposed monitoring mechanism emphasizes process more that a specific model, identifying seven process steps leading to the development of the PEACC monitoring system. The PEACC evaluation system seeks a “deeper analysis of programme relevance, efficiency and effectiveness” (UNICEF 2011 Proposal on Peacebuilding, p. 37) through a midterm evaluation in the last quarter of 2013 and a final, independent evaluation in mid 2015. UNICEF’s evaluation section will provide significant support, especially with the baseline study and independent evaluation. The PEACC proposal also identifies thematic studies for developing the evidence base linking education and peacebuilding, and mechanisms for progress reporting (UNICEF 2011 Proposal on Peacebuilding). From the UNICEF evaluation section perspective, the PEACC evaluation is important to ensure accountability (results, attribution, contribution, and value) and learning and improvement. They identify the aims of the M&E process a clear Global Results Framework and results orientation at all stages and all levels; good baseline data and data collection and utilization processes; and a lucid M&E strategy (Kirk, 2012 presentation). UNICEF's studies in previous conflicts show that education may have both positive and negative effects in conflict situations.7 The Two Faces of Education in Ethnic Conflict study (2000) points out that education is often used to create or preserve economic, social and political privilege; enable cultural repression; manipulate history for political purposes; and as a weapon of war by denying access, segregating and humiliating opposing groups. On the other hand, education that is conflict sensitive can be very positive by nurturing ethnic, tribal and linguistic tolerance; cultivating inclusive concepts of citizenship and community participation; and to help heal the psychological wounds of conflict and restore social cohesion. Although the PEACC proposal document acknowledges the two faces of education in conflict situations, it focuses on promoting the positive aspects and relies on conflict analysis as the explicit strategy to avoid or prevent the potential misuse of education. There is a presumption that governments and other providers of education in conflict situations will use education only in a positive way that supports peacebuilding. A clearly articulated and well supported monitoring and evaluation system is needed to assure that education is conflict sensitive and supportive of peacebuilding. Interviews, presentations and materials made available at the September 2012 Workshop on Peacebuilding, Education and Advocacy coordinated by UNICEF headquarters education section8, address various evaluation issues. Also, at the time of this report, few countries had completed their conflict analysis and none had developed a programme document with a log frame or results framework. There was some criticism among workshop participants that this programme was prepared by headquarters and the donor with little consultation of country and 7

UNICEF Innocenti Centre, The Two Faces of Education in Ethnic Conflict: Toward a Peacebuilding Education for Children, Florence, Italy, August 2000 8 Participants included UNICEF headquarters education staff, plus representatives from other UNICEF headquarters sections, representatives from the regional and 14 participating country offices, other UN staff, and several external consultants.

35

regional offices or strategic partners. The top down planning and scope of the project raised questions about whether this is an alternative way to fund activities already planned under the country programme or whether it involves new initiatives and significant modification of existing programmes. However, there was a general perception that the broad scope of the five key global outcomes would enable country offices to find ways to integrate the global objectives and local country education needs. The use of conflict analysis as a basis for programming had apparently slowed the process of PEACC programming and there seemed to be different interpretations by workshop participants of how conflict analysis is to be done. In Congo it began at village/school level and was aggregated upward, but in other cases it was started at national and district levels. It was also not clear how conflict analysis is incorporated into or used to supplement the normal situation analysis done in the country programming process. Our experience working with one country office on the PEACC results framework design identified some confusion by staff about how to implement the programme design and conflict analysis process, especially of how to link known local conflict drivers with education sector interventions that fit within the five PEACC global objectives. There seemed to be little systematic analysis of the interrelationships among the conflict drivers identified in the early drafts of the PEACC country analyses. For example, how do disputes over land and other resources relate to and affect such conflict drivers as human rights violations or access to school? Also, many of the main conflict drivers, such as disputes over land or other resources, did not appear to be closely linked to or amenable to resolution through education. As a result, the conflict analysis did not necessarily provide a clear rationale for education for peacebuilding nor clearly indicate how it should be approached. It may be necessary to conduct a two tier conflict analysis, a high level analysis of overall conflict and a local, educationspecific conflict analysis to guide the approach to education for peacebuilding. In any case, it seemed as if the programming was going forward to meet the donor timetable and budgeting requirements ahead of the conflict analysis. The baseline of relevant information available at the country level was not clear for the PEACC. For example, it was not clear if education participation rates can be disaggregated by ethnicity, tribal or language group, which would be critically important to assess such peace consolidation issues as inclusion, equity, and coverage in education. The education management information systems were still not well developed, but most countries seem to have initiated them. The EMIS tracks routine information, but not unusual things like truancy, violence or attacks on schools. The multiple indicator cluster surveys (MICS) capture more detail and may enable more disaggregation of data, but it is not clear how many of the 14 participating countries have recent MICS or could do them soon to create baseline data. There have not been routine or widespread knowledge, attitude and practice (KAP) surveys which would be needed to establish baselines and evaluate changes in knowledge and attitudes, which is one of the key outcomes of the PEACC programme. On a positive note, if the conflict analysis is education-specific, it may create a baseline of information that can be used in monitoring and evaluation. Coordination with other key agencies and organizations is mixed, although from an evaluation perspective such coordination is important for the three global objectives on policy action, institutional capacity building, and knowledge generation. Other technical groups within 36

UNICEF, such as Emergency Operations and Child Protection, were involved with the development of this programme proposal and feel there is good continuing cooperation. Both were represented in the September 2012 planning workshop, along with UNICEF units working on gender, disability, communication, adolescents, early child development and sports. There was clearly an effort to involve most relevant parts of UNICEF at the headquarters level. UNICEF regional offices have been involved in training and support for conflict analysis at the country level, but have not done regional level conflict analysis. There are a number of cross border conflicts and refugee movements (i.e. Congo, Uganda, South Sudan, etc) that lend themselves to regional analysis and action. The programme does not specify the role of regional offices. They were not involved in the formulation of the global programme. The budgeting process discussed at the workshop does provide funding for regional office support for country offices and may be available to address cross border issues. The strategic partners outside UNICEF were apparently not fully consulted. For example, there has apparently been only one meeting of the Strategic Partners Advisory Group. The UNDPBCPR was not involved in the programme preparation and often competes with UNICEF for donor funding in this area. UN Peacebuilding Support Office (UNPSO) and Fund (UNPF) made a presentation at the workshop and seemed to be well informed about the programme. They noted that the Fund has designated very little for education, yet recognizes its important role in peacebuilding, and supported donor contributions directly to UNICEF for education programmes. The involvement of the IASC Education Cluster also seemed limited. It brings together over 25 organizations to do joint capacity building and knowledge management through monthly teleconferences and annual meetings. The rapid development of the PEACC programme may explain the limited coordination with strategic partners. Regarding whether a coherent theory of change and feasible M&E strategy exists, the underlying assumption of the PEACC programme's theory of change is that if conflict sensitive and peacebuilding education is widely available and accessible, then children, their parents and communities will be better able to avoid, resolve and eventually transform conflicts leading toward a more peaceful society. The other outcomes support or are necessary inputs to make quality, conflict-sensitive education for peacebuilding widely available. The actual role of education in the larger peacebuilding programmes of the UN is currently very small and simply included as one of many government services to be restored.9 There seems to be minimal recognition outside UNICEF of the unique role of education in peacebuilding as put forth in this programme proposal. This highlights the need for increasing advocacy for education among peacebuilding organizations at the global level. The minimal role and proportion of resources going into education in peacebuilding programmes will also make it very difficult to specifically attribute changes in “resilience, social cohesion and human security” to the education for peacebuilding programme. Given the above described theory of change, the Peacebuilding and Education Programme must operate at every level from local schools and communities up to and including advocacy on peacebuilding policy at the global level. Inducing changes such as conflict sensitive education 9

Sixty fourth Session of the Security Council, “Report of the Secretary General on Peacebuilding in the Immediate Aftermath of Conflict”, 11 June 2009.

37

and education for peacebuilding at the local level requires support and guidance from the higher levels. Programme actions at the local level such as supplying educational materials, Accelerated Learning Programme, Child Friendly Schools are well established but also require support from above and substantial and continued external assistance. The methods for conflict analysis in education planning and programming are being increasingly used and improved, but this also requires external support and guidance. Institutional support through training and curriculum development and evidence-based policy at the national level is also well understood and explained in the document. The need for evidence-based advocacy and improved coordination at the global level is also well articulated. What seems less well explained is the actions and support to be provided at the sub-national district level, and, in conflict affected areas, this may be the most critical level where ethnic or tribal conflicts may arise. Similarly, the role of regional offices and organizations in the programme is not clearly explained. In conflicts between countries or where conflicts result in cross border refugee movements the regional level may be one of the most important levels to address issues of peacebuilding including education. Even where there are no cross border problems, the regional level may be the most suitable level for training and exchanges of experience among national education officials. If the programme is to address all these levels, then the results framework will also need to include baseline information, objectives and indicators for monitoring progress at each of these levels. Peacebuilding is a complex and lengthy process. Because of the limitations of budget and programme cycles, this programme and other supporting programmes for peacebuilding are far too short to fully implement the peacebuilding process. Therefore it is important that the results framework has objectives and targets that are limited and feasible within the timeframe of the programme. It is unlikely that the full impact of education for peacebuilding can be achieved within the programme time frame. However, it is reasonable to expect to be able to monitor and measure progress toward achieving the desired outcomes at the various levels. In order to develop feasible and time-limited objectives within each of the five key outcomes noted above, it is helpful to envision what the ultimate situation would be if education for peacebuilding was fully achieved. This may require thinking in terms of a very long process of education induced psycho-socio-cultural evolution that may take decades. Once such a vision is generally agreed, we can then identify what changes in the situation would be needed at shorter intervals to eventually bring about these larger and longer term changes. This process of long term visioning and identifying is not only helpful in establishing feasible and time-limited objectives, it also enriches the conceptual development of the programme and places it within the larger and longer term peacebuilding process. 4.3 Non-UNICEF Programmes and Evaluations on Education for Peacebuilding A UNICEF sponsored literature review of education in peacebuilding by Smith et al (2011) reviewed 526 documents on education in peacebuilding, with 325 related to programme issues and the balance addressing academic and organizational issues, finding that few education programmes in post-conflict environments specifically addressed peacebuilding. They found that when such programs do exist, they are often managed by development agencies that conduct limited evaluation studies, and are complicated by data collection problems. In this section we summarize evaluation measures and issues for several programmes related to education for peacebuilding. 38

Sudan Peacebuilding Study The 2010 multi-donor country evaluation of peacebuilding activities in Southern Sudan identified significant problems with the theory of change linking the provision of basic services such as education to conflict prevention, peace dividend or positive peace conditions (Bennett 2010:47). Of particular note, the Joint Assessment Mission Sudan (JAMS) implementation framework in 2005 identified “basic education: provide equitable access to education” (Bennett 2010:47) as one of five peacebuilding priorities. Bennett and colleagues note the problem with basic concepts in the theory of change. At a macro level, conflict, resources and perceptions of equity seem to drive change in this case study. Some of the conflict assessments had noted a historical marginalization of the Southern Sudanese people while other appraisals noted historical underdevelopment: the first was assumed to mean “deliberate political and economic marginalization” (Bennett 2010:48) and systematic exclusion and attendant public resentment, while underdevelopment refers to a lack of basic services. For political reasons, the JAMS language referred to the weaker term, underdevelopment. Bennett and colleagues (2010) clarify the implications for this confusion over language and theory of change quite well: The issue of ‘marginalisation’ needs to be reviewed. In the Sudan context this does not equate to ‘lack of services’ but to political and military domination, including violence. It includes elements of deliberate exclusion. In the case of Southern Sudan there is the added resentment that some areas produce enormous wealth, especially from oil but also from cattle, which may then be appropriated by the dominant political entity. The role of the State in regulating such disparities has been minimal or even negative. It operates on the basis of exclusive circles of patronage and marginalisation. Marginalisation is the ‘stick’ and patronage is the ‘carrot’. The confusion between ‘marginalisation’ and ‘lack of development’ led to an assumption that lack of development was not simply a matter of concern but a factor causing conflict. Yet local conflict arises primarily from disputes over access to resources. These might escalate either because of historical factors or because of political manipulation. Lack of development might, at most, be a cause of disaffection that might have a minor contributory effect in such cases but could not be cited as a significant cause. The JAM analysis implies that lack of development is in itself a cause of conflict. This opened the way for what became the dominant ‘theory of change’: that all forms of development contribute to CPPB [conflict prevention and peacebuilding]. This theory may have held good in other situations, but in Southern Sudan more precise targeting of conflict causes was needed. The theory that ‘all development contributes to CPPB’ became transposed into the phrase ‘peace dividend’. The logic seemed to be that development was not only a reward for peace (the CPA) but that failure to deliver a ‘peace dividend’ would lead to conflict. The evidence for such a claim appears to come from global studies103, but in Southern Sudan there was no evidence that conflicts might arise from (or even be seriously affected by) lack of services. And yet this became the 39

dominant paradigm of the aid operations. (Bennett 2010: 49)10 This example and discussion raises two important issues. The first is the difficulty in establishing the relationship between basic service provision, like education, and both conflict drivers and peace promotion. As Bennett and colleagues argue, the linkage is weak. As they elaborate: Our own findings (Chapters 4 and 6) challenge the inherent assumption of there being a direct causal link between the provision of basic services and the prevention or mitigation of violent conflict. The perception of unequal access to resources and services may contribute to general discontent, but is unlikely to be a reason in itself for violent conflict. If it is not a prime reason, then providing such resources cannot be a prime solution. If international aid is a ‘peace dividend’ (meaning that it provides resources as a benefit deriving from the peace), this does not mean that it contributes either to conflict prevention or building future peace. This finding is, of course, controversial, for it challenges the premise upon which a great deal of assistance to Southern Sudan rests – that the provision of socioeconomic services addresses needs, leading ultimately to the enhancement of state legitimacy and stabilisation. (Bennett 2010: 49) This analysis of a fundamental flaw in the basic theory of change regarding conflict drivers and peace factors should highlight the importance of clear conceptual models of change when planning the PEACC programme. A second concern from this case study is the lack of clarity in terms and meanings. As UNICEF advances its education for peacebuilding work, time spent on building clear definitions would be beneficial. Another finding from the Southern Sudan study is also illuminating for our evaluation work. Bennett et al (2010) note that a sub national public survey in Eastern Equatoria state found that local people were most concerned about education and hospital care, even more than clean water and security. The authors also note that this public perception of the need for education was not supported by local data on education, marginalization and violence. In effect, this example raises questions about a fundamental assumption of conflict assessments – that local participation should guide much of the peacebuilding intervention. Conflict assessments must critically compare public perceptions of conflict with local data on quality of life, and make the appropriate inferences to higher level drivers or outcomes. Furthermore, since these assumptions are critical for long term effectiveness of peacebuilding, continual assessment of such assumptions are necessary for the proper prioritization of services. UNICEF’s Timor-Leste Education Programme Evaluation The evaluation of UNICEF’s Timor-Leste education program in 2010 is insightful. The evaluation notes that Timor-Leste is in the process of peacebuilding and post-conflict transitioning (UNICEF Timor-Leste Evaluation 2010:iv) and the evaluation TOR refer to analysis of the impact of education on peacebuilding. However, none of the report recommendations discuss peace-related issues nor does the education sector analysis in 2002-3 10

Footnote 103 stated “Such studies are too numerous to list here, but the CPPB Guidelines and the bibliography consulted for this evaluation (Annex 12) contain examples.

40

address any conflict-related issues, such as displaced children, child or teacher trauma, or need for peace education. There is only a brief mention of problems on children’s rights policies and space for youth participation (UNICEF Timor-Leste Evaluation 2010:13). The current life skills program is one education activity that could be said to be focused towards peace efforts. In general, little consideration of direct education for peacebuilding programming was made in the Timor-Leste education programme, notwithstanding the legacy of conflict that exists. Sri Lanka SEDEC Residential Schools Another case study of the use of education for peacebuilding exists with the work of SEDEC in Sri Lanka. The church provided support for teachers, school supplies, uniforms, and financial support for poor and displaced children, and sent such children to residential schools. In the conflict, both conflicting parties targeted children. The pertinent evaluation finding is that by simply sending the children to residential schools, “a slightly higher degree of security” was provided to the children, than if they were in their own homes (Goydor 2004:21). As the authors note, similar education interventions may provide protection for at risk children in other conflicts. It may also increase the likelihood of finishing primary or secondary education, and improve school quality compared to schools in more violent conflict zones. Nigeria Adult Literacy as Peace Education In Nigeria adult literacy education as conceptualized as peace education (Adelore and MajaroMajesty 2008). The authors identified such conflict drivers as lack of cohesion, poverty, illiteracy, and inequitable distribution of education resources. They also claim a direct link between high levels of illiteracy, and ethnic conflict and peacebuilding, given that literacy is necessary for basic education, and basic education is necessary for economic development, civic participation and political involvement. Adelore and Majaro-Majesty (2008) also hypothesize that adult education using functional literacy and problem solving methodologies is linked to positive social attitudes, communal solidarity, human capital and peacebuilding capacity. While making this argument for adult literacy education to support peacebuilding, they criticize formal peace education models. These assumptions are not supported by data, although the authors work directly in adult education. While the authors lack a clear interaction model for these assumptions, this modeling of the relationship between such basic education functions as (adult) literacy and peacebuilding is valuable, and supported by conflict transformation models such as Azar’s Protracted Social Conflict model (summarized in Appendix A). School Based Peacebuilding in Sierra Leone Through their work in Sierra Leone, Bretherton, Weston, and Zbar’s (2005) argue that educationbased interventions are critical for generating a culture of peace in which civil society works collaboratively with state institutions. The primary method for building the culture of peace is through peace education that generates shared values among citizens. Using this formulation, they developed a culturally appropriate peace education programme with World Bank funding. Bretherton et al define peace education, using Fountain’s 1999 model, as (2005:356-7): The process of promoting the knowledge, skills, attitudes and values needed to bring about behavior changes that will enable children, youth, and adults to prevent conflict and violence, both overt and structural; to resolve conflict peacefully; and to create the conditions conducive to peace whether at an 41

intrapersonal, interpersonal, inter-group, national or international level. To implement the model, the authors developed culturally appropriate materials with local participation that can be used with or separately from the Sierra Leone curriculum by various educators and community workers. The peace education kit has four sections addressing a) the philosophy of peace education, b) broad peace issues c) specific units for English, social studies, health, physical education, and the arts, d) school and community level activities (Bretherton et al 2005). The research team used action research methods to continuously collect information on the programme’s implementation and emphasized the importance of participant feedback. A baseline survey was conducted in participating and control group schools in late 2003 with a post-test in early 2005. Baseline evaluation indicators included (Bretherton et al 2005): • • • • • • • •

Teachers’ sense of efficacy (capacity to make a difference to students’ well-being and learning in relation to peace education themes), Degree of implementation by individual teachers of the peace education curriculum and pedagogy (attitudes and actions), Perceived leadership and school-level support for peace education initiatives Perceived impact of peace education teaching and learning on Teachers (e.g., empathic perspective taking, conflict resolution, tolerance, pedagogy), Students (e.g., sense of well being, conflict resolution skills, active learning, attitudes about the future), School culture (e.g., staff modeling of values and skills; staff and student involvement in decision making; perceptions of trust, safety, and respect). Teachers’ opinions on the usefulness of the Peace Education Kit training for their teaching.

Post-test data collection also included surveys of head teachers from both participating and control group schools, and qualitative data from workshops and focus groups. The data had not been analyzed as of the article’s publication to assess the programme’s effectiveness. The authors suggest a theory of change using an ecological model with nested levels from intrapersonal to interpersonal, intergroup, national and international. They assume that change occurs through capacity building and emphasis on human agency and responsibility. This latter priority is due to strong religious cultural beliefs of God’s role in (social) events, with many Sierra Leoneans cognitively separating consequences from human behavior. Peaceful change thus supposedly occurs when people recognize their ability to influence social relationship, take accountability for their actions, hold others responsible, and then resolve conflicts peacefully (Bretherton et al 2005). 4.4 Academic Studies Related to Education for Peacebuilding 7. One of the most valuable studies for the evaluation of education for peacebuilding has been the work of Bush and Saltarelli (2000) on the role of education for peacebuilding, conducted for the UNICEF Innocenti Research Center. The key finding questions one of the most important assumptions of education interventions, namely that education always is for the good. Bush and Saltarelli (2000) start by describing their conceptual model of armed conflict, which emphasizes the ethnic nature of current conflicts. 42

The structures and processes that appear to turn ethnic intolerance into unbridled violence are highly complex. A list of causal factors might include ‘historical forces’, economic tensions, ‘bad’ governance, perceived threats to cultural identity and (in ways that are not adequately understood) formal, non-formal and informal educational processes. Ethnicity itself is often asserted to be a key contributor to ‘ethnic conflict’. However, it is increasingly evident that “ethnicity neither causes conflict, nor in many cases does it accurately describe it. Rather ethnicity/identity is increasingly mobilized and politicized in contemporary violent conflicts” (Bush, 1997). There are many theories attempting to explain the formation of ethnic identity and some are introduced below. However, this study focuses on the educational structures and processes that politicize identities in ways that allow diversity and cultural difference to become the basis for violent, protracted, conflict. (Bush and Saltarelli 2000:vii). 8. This identification of causal factors may help conflict mapping efforts, while the connection between ethnic identity and education can inform logical models. The conceptualization of the relationship between education and peacebuilding is directly addressed, which also provides a theory of change addressing various institutions, governance structures and conflict management: While the study has a particular interest in education systems, it attempts to understand these within the broader political, economic, and social institutional context. Some have argued that violent conflict is the ultimate expression of the breakdown of a society’s governance system, and that reconstruction therefore rests primarily upon the renegotiation and refashioning of new systems of governance at the community, sub-national and national level. Thus, ‘reconstruction’ requires strategies and interventions to promote institutional arrangements that can facilitate and sustain the transition from violent conflict to sustainable development, and build ‘fire walls’ that prevent societal conflict spiraling into societal violence. An appealing feature of the study’s broad approach is the way analysis of the problems is tied to an understanding of solutions. (Bush and Saltarelli 2000:x). 9. Bush and Saltarelli (2000) then identify both constructive and destructive consequences of education. Negative or destructive consequences of education can include irregular distribution of education, education used for cultural repression, denial of education for war purposes, manipulating history and textbooks for political reasons, self-esteem tied to hatred of out-groups, and segregated education to maintain status quo of inequality and domination. Possible positive or constructive consequences of education include educational opportunities reducing conflict, promotion of ethnically tolerant societies, psychological de-segregation, linguistic acceptance, inclusive citizenship, neutralizing historical accounts, Education for Peace programmes, and direct responses to state controlled oppression (Bush and Saltarelli 2000). In sum, the work of Bush and Saltarelli (2000) provides guidance on theoretical models for evaluation of peacebuilding models, theories of change, and evaluation indicators. 43

Some authors argue that peace education is one of the principal historical foundations of peacebuilding (Ramsbotham, et al 2011). A number of leading peace theorists such as Johann Galtung, Adam Curle, Paolo Freire and John Paul Lederach have worked in the areas of pacifism, non-violence, oppression, and non-violent resistance. A major part of this theoretical work is changing global war systems to global peace systems. Ramsbotham et al (2011:239) argue that peace education provides the opportunity to return to the original philosophies of full emancipation and non-violent social relations. They note that a global consensus supports full education for children, yet major gaps exists for the realization for full education. Positively, a growing number, 57 out of 103, of peace agreements include education provisions, addressing such issues as the right to education, continuing education services, countering conflict dynamics within education, and education reform to address the root causes of conflict between conflicting parties (Dupuy 2008). In peace agreements, education is often addressed in one of four ways: from a security point of view, emphasizing reintegration of combatants; as a protection issue addressing concerns such as child recruitment and mine protection; economically, to support broad development, reduce poverty and improve self-reliance; and socio-politically, to promote educational reform and change the continual reproduction of power structures within societies (Dupuy 2008). The last category, socio-political change, can be very important in deep-rooted conflicts, given that “mandates for education system reform in peace agreements are also mandates for social and political reform, as the education reform plans outlined in the agreements require a restructuring of social and political hierarchies and structures as they are reflected in, and reproduced by, the education system.” (Dupuy 2008:158). Dupuy (2008) makes a strong argument for critical planning of education programmes for peacebuilding, given the diverse conflict drivers that may exist and long term social and political changes that may be needed for sustainable peace. A number of other academics and researchers document and reinforce additional key education for peacebuilding issues. Marc Sommers argues that education has the potential to “provide psychosocial recovery, stability, normalcy, hope and the inculcation of values and skills for building and maintaining a peaceful future” (2002:21). Sommers’ recommendations identify key conceptual issues for the linkage between armed conflict, education and peacebuilding, such as working with state governments, the critical role of teachers, inclusion of psychosocial interventions as part of education programmes for all children living in post-conflict situations, and the many ways that education can help protect children (2002). Several studies address peace education in intractable conflicts. Kupermintz and Salomon (2005) identify three key unique characteristics of peace education programmers in deep rooted conflicts. First, such peace education programmes seek to promote conflict resolution between collective groups of people, not individuals. Second, such “conflicts are deeply rooted in each side’s collective narrative, the story each side tells about itself, its identity, aspirations, perceived role in the conflict, and, mainly, its past and current history.” (Kupermintz and Salomon 2005:294). And third, in such conflicts people have deeply rooted beliefs about their in-group and the out-groups that justify their positions. These beliefs are difficult to change. Also, issues of emotions, power and status are important for long term peace (Kupermintz and Salomon 2005). 44

Bar-Tal and Rosen (2009) note that peace education can promote reconciliation, but must address the needs of the local societies involved in deep rooted conflicts. When societies are in intractable conflicts, the goal of peace education should be reconciliation, so as to change worldviews and promote conflict resolution. This work must be done at the societal level, including schools, and political, social and economic institutions, the media, etc. It is important to build on local peace drivers – local cultural practices that promote peaceful conflict resolution (Bar-Tal and Rosen 2009). No specific indicators or evaluation methods suggested for peace education. Bar-Tal and Rosen (2009) note that effective school-based peace education must change educational objectives, curriculum, textbooks, instructional materials and methods, and school climates. It must address the full range of ways that children and youth are socialized so as to promote reconciliation. A range of political, social, and educational conditions are needed for this change. The authors identify two methods along a continuum for developing peace education programs. Direct peace education methods may be used when political and societal conditions support peace, and indirect methods used when they don’t support peace. Indirect methods could address such issues as “identity, ecological security, violence, empathy, human rights and conflict resolution skills.” (Bar-Tal and Rosen 2009:563). Five major themes for indirect peace education include reflective thinking, tolerance, ethno-empathy, human rights, and conflict resolution. Since direct peace education operates under more favorable conditions, it can directly respond to the causes of the intractable conflict and the culture of conflict. Five possible themes for direct peace education include conflict and peace, peace processes, presentation of rival groups, history of the conflict and new affect and emotions. In conclusion, the education system plays a major role in developing worldviews, ideologies, values, beliefs, etc., and can move those towards reconciliation in divided societies. Also, critical conflict analysis is needed to ensure peace education interventions address local political and social conditions (Bar-Tal and Rosen 2009). One peace education programme in the United States is that of Johnson and Johnson (2005) based on a cooperative learning model, which they expand to cover peacebuilding contexts. The authors note that peacebuilding utilizes economic, political and education institutions to create long-term peace, and peace education is one method for building peace. Their peace education model differentiates between imposed peace and consensual peace. A consensual approach to peace is based on agreements that end violence and hostilities, and establish new relationships with peaceful interaction, mutual goals, just distribution of benefits, mutual dependence on other’s resources, and a mutual identity (Johnson and Johnson 2005). They note five steps to institutionalizing peace through peace education: a) make education mandatory and integrate schools, b) establish mutuality and positive interdependence, c) teach students how to make difficult decisions, d) teach students to resolve conflicts constructively (win-win), and e) promote civic values (Johnson and Johnson 2005). The principal aspects of Johnson and Johnson’s consensual peace model include positive interdependence, which exists when there is: mutual goal attainment (everyone must reach their goals), mutual benefits for all people involved, mutual dependence on each other’s resources, and mutual identity. Positive interdependence leads to what they call promotive interaction, where each person sees the other as equal and tries to work together. Possible outcomes and indicators of promotive interaction include mutual success, positive, supportive relationships, physical and psychological wellbeing which leads to joint agency and joint efficacy (ability to do 45

things), and joint self-esteem (Johnson and Johnson 2005). This model holds promise for conceptualizing education for peacebuilding given its comprehensive nature and range of actions needed to develop the knowledge, skills and abilities for interpersonal peace. One growing global trend leans in the direction of security over peace, especially in developed countries. Bickmore (2011) summarizes research on anti-violence programmes and the growing trend in education to expand security measures to control violence and bullying, at the expense of educational programmes that address the root causes of interpersonal conflict and violence. Bickmore found that multifaceted, comprehensive, long-term educational programmes can be effective (2011:651-2): Meta-analyses of evaluations substantiate the effectiveness of explicit (separate or curriculum-infused) programs of instruction and practice that facilitate young people’s development of social and cognitive competence, respect and tolerance across differences, inclusion of marginalized students, and opportunities to be positively involved and to build strong relationships—in clear contrast to punishment-heavy approaches (Catalano, Berglund, Ryan, Lonczak, & Hawkins, 2002; Erickson & McGuire, 2004; Hazler & Carney, 2002; Osterman, 2002; Scheckner, Rollin, Kaiser-Ulrey, & Wagner, 2002; Schwartz, 1999). Among other findings, improvements in academic achievement are negatively correlated to levels of school violence (Bickmore 2011). Changes in school participants’ conflict resolution skills, mutual respect and interpersonal social skills, combined with participatory processes, reduces conflict and violence. However, Bickmore (2011) notes that many violence prevention programmes are poorly implemented or not sustained for long enough periods, limiting their impact on peaceful relationships. A meta-analysis of 32 classroom-based conflict resolution interventions found a reduction in bullying, hate language and fights of 66 to 75% (Bickmore 2011). However, such curriculum or whole-class interventions must be “part of a comprehensive, long-range approach to explicit instruction and regular practice in conflict management, equity, and restorative practices” (Bickmore 2011:653), along with effective training and support for teachers. 4.5 Lessons Learned for M&E of Peacebuilding, Education and Advocacy This section summarizes lessons learned for the monitoring and evaluation of the Peacebuilding, Education and Advocacy programme, including a draft conceptual model for understanding the relationships between education sector development and peacebuilding. Below are a number of lessons from this study. Education for peacebuilding is a relatively new field and the concepts and linkages between education and peacebuilding are still not well defined. Some studies have noted that education has two faces in conflict situations and may be drivers of conflict or may help to reduce conflicts, and others have found little connection between education and conflict. UNICEF and other actors are conducting many programmes that could be considered education for peacebuilding (e.g. Timor-Leste, Nigeria). Evaluations should be systematically conducted and made readily available to the public to expand knowledge generation for all stakeholders. The EEPCT experience provides a number of 46

lessons in this area, showing large scale results with millions of children benefiting from the program yet an uncertain impact on existing armed conflicts or future peace. Similarly, planning documents such as theories of change, results frameworks, and political/funding issues that constrain planning must be adequately documented to understand how such important decisions impact programme implementation and results. Both the EEPCT and PEACC programmes are insightful in this area, for example, with PEACC goals being determined by the donor without significant field participation, circumventing critical planning steps of conflict analysis which was found to be an important lesson learned from the EEPCT. A broad conflict analysis is needed to assess the extent to which the drivers of conflict are education related and hence the size and role of education in the overall peacebuilding programme. But the conflict analysis also needs to be education-specific to provide a base line of information and to develop a strategic approach to education for peacebuilding. Further research and theoretical conceptualization is needed of the relationship between basic social service delivery (e.g. education), peace dividends and peacebuilding to confirm fundamental assumptions underlying the PEACC programme. As a minimum first step education for peacebuilding should be conflict-sensitive to avoid or eliminate aspects of education that may contribute to or drive the conflict. Some of the characteristics of conflict-sensitive education are that it is inclusive and accessible and the curriculum and teaching methods do not promote violence or discrimination that may drive or further enflame the conflict. As noted by some researchers, conflict-sensitive education requires analysis of and challenges to social, political and economic institutions which maintain structures that are dominating and oppressive. Education for peacebuilding goes further in building toward a positive peace with the following characteristics: • • • • •

Child protection (prevention of child recruitment, responding to attacks on schools, mine protection, prevention of gender or ethnic discrimination and violence in schools); Psycho-social support (coping with post traumatic stress and disabilities, and building self-esteem, tolerance and compassion) Reintegration of combatants, refugees and displaced children and communities; Economic recovery and self reliance (job skills training and apprenticeships); Socio-political reform to reduce conflict drivers (increase access and participation among conflicting groups, reduce discrimination, learn conflict resolution skills and to value compromise with opposing views).

Education for peacebuilding need not be limited to the formal school system and may include community-based education for reconciliation using theater, music and art or peace camps that bring together mothers and children from conflicting groups. Further research is needed to systematically identify forms of education that address the range of child development issues that can be conceived of as relevant for sustainable peacebuilding, including psychosocial development, personal and ethnic identity, interdependence, and tolerance. Evaluations of the PEACC can start to fill this knowledge gap. 47

Given the newness and uncertainties in the concepts and practice of peacebuilding generally and education for peacebuilding in particular, the programming approach is necessarily broad allowing for great variation and experimentation. In fact, it is essentially a large-scale action research programme. Therefore the monitoring, evaluation, research and knowledge management functions should be built in as core components of the programme. There are many implications for an appropriate approach for monitoring and evaluation. • • •

First, it is difficult to apply the results based programme approach and to develop a results framework that has clearly defined outcomes that are broadly applicable. Second, it is also difficult to define the programme components or activities and the processes and linkages that lead to the desired outcomes. Third, as noted above, the differences between general education, conflict-sensitive education and education for peacebuilding are largely qualitative and therefore difficult to define as indictors and to quantify for purposes of monitoring and evaluation.

Despite these difficulties, there are a number of examples of evaluation of education for peacebuilding as noted above. These evaluations have been in specific country contexts and refer to specific project activities which enable a more limited focus on the linkage between project actions and specific results related to one aspect of peacebuilding. However, for the current programme, which is global and multidimensional in scope and implemented at national, district and community levels, the M&E system must be much more complex. It has to be evaluated at the: • • •

global level in terms of effectiveness of policy advocacy, interagency coordination, and funding mobilization and allocation; national level in terms of policy advocacy, institutional capacity building, budget and fund allocation, research and information management; district, school and community levels in terms of effectiveness and efficiency of project actions; capacity development of teachers, students and others; and increased inclusiveness, access, and participation.

4.6 Conceptual Model for Education Interventions and Peacebuilding In Appendix D we provide a conceptual model that attempts to explain the relationships between key conflict issues, education for peacebuilding objectives and peacebuilding outcomes. This is done at three levels - macro, meso and micro – to identify factors and issues to consider at each level. This conceptual model is not meant to be authoritative or prescriptive. Instead, it is meant to help planners and practitioners develop a coherent theory of change and results framework for education for peacebuilding. Conceptually, it is important to note that difference between promoting peacebuilding per se and promoting peaceful interpersonal relationships. As discussed previously in the peacebuilding section, the first concept has more to do with state building and the second concept more to do with psychological and social aspects of peaceful, tolerant, cooperative interpersonal and community relationships where conflict is resolved equitably. 48

4.7 Ethical issues for Peacebuilding Evaluations The last section on education for peacebuilding identifies potential ethical issues that may arise with this work in the fields of education and peacebuilding. Several different fields provide guidance on ethical issues for evaluation of education for peacebuilding interventions, including peacebuilding, Do No Harm, and children associated with armed forces and armed groups. At the broadest level debate exists over just (good or right) interventions for peacebuilding. Some practitioners argue that non-violence must always be upheld, while others argue that in certain cases, such as genocide, force should be used to control the more aggressive party (Ramsbotham, et al 2011). A number of experts have proposed guidelines for intervention in extreme cases, including UN Secretary General Annan’s six principles in 1999, Blair’s six principles in 1999, Chesterman’s 2001 criteria, and the International Commission on Intervention and State Sovereignty in 2001 (Ramsbotham, et al 2011). Ramsbotham and colleagues (2011:278) identify a number of principles to guide interventions: • • • •

• • • •

Impartiality – consider all parties’ interests and seeking win-win outcomes Mutuality – focus on the parties’ concerns, not imposing the interveners concerns. Requires considering local initiatives, economies and social and cultural factors Sustainability – continue the intervention until the problem is completely resolved Complementarity – cooperation between intervening sectors and organizations, with the appropriate level of involvement for the specific context and moment (noting situational changes) Reflexivity – Assess interveners motives, goals, power, etc., and their compatibility with the overall intervention goal Consistency – have consistent responses across different conflicts Accountability – direct parties and sponsoring organizations (governments) must be accountable for their actions and commitments Universality – ensure compliance with international norms and values and meeting needs of all parties regardless of culture, etc.

This debate may play out with UNICEF’s education in peacebuilding work, where armed groups or armed forces may deliberately target schools or recruit children at schools. How UNICEF responds to such situations has ethical consequences. In the peacebuilding field, Lederach (1997) identifies a number of ethical dilemmas or issues. Most importantly, since peacebuilding involves building relationships and trust, trust can be manipulated, creating a potential ethical issue. •

•

Since peacebuilding is primarily focused on building relationships and trust, which involve “creating space, developing relationships, persevering in spite of overwhelming pessimism, and being flexible to respond to emerging opportunities” (Lederach 1997:131), project-related evaluations that use discrete indicators may not be appropriate. Indicators that measure relationships, trust and “adaptive and dynamic processes (Lederach 1997:131) are needed. Similarly, project and results-oriented evaluations often have narrow time and space horizons, which constrict evaluations. Peacebuilding infrastructure needs to be 49

•

•

responsive to changing conflict conditions and require more focus on initiatives and outcomes. Reporting mechanisms that emphasize transparency, comprehensiveness and uniformity may conflict with the need for sensitivity, confidentiality and trust building that is crucial for building social relationships. While some peacebuilding efforts emphasize work with key individuals, more concern is needed with infrastructure development and institutional capacity building.

Lederach, Neufeldt and Culbertson identify additional ethical concerns for peacebuilding implementation and evaluation, primarily around the protection of participants and maintenance of the integrity of the evaluation process (2007:14-15). Possible participant protection concerns recognize that peacebuilding is inherently about building relationships, and should consider limiting unnecessary stress (such as during community member interviews about war experiences and trauma), potential for perceptions of coercion by community members by the donor, maintaining confidentiality of participants when ramifications may arise, ensuring participant protection during travel or when crossing into territory of opposing groups, maintaining realistic expectations of participants who offer suggestions (since not all recommendations may be acted upon), sharing findings with local communities to promote information dissemination and validation, and ensuring respect for all participants, work performed and organizations (Lederach, Neufeldt and Culbertson 2007:14-15). A number of ethical issues arise with researching vulnerable and marginalized populations such as those involved in the PEACC. These include recognition that certain populations, such as those living in armed conflict or post-conflict environments, rural areas, and with limited education are more vulnerable (Culver et al 2010). Even getting informed consent may be challenging. Evaluation data collection methods should require that data collectors are experienced, speak local languages, and are trained in the specific data collection tools. Rigorous data security protocols must be maintained. Participants must be informed of the potential harm and limited benefits arising from participation (Culver et al 2010). In the particular context of the PEACC programme, there have been concerns regarding the use of conflict analysis. Some governments are reluctant to do conflict analysis because they fear it may reignite conflicts. This is a legitimate concern because effective and participatory conflict analysis necessarily brings together representatives of conflicting groups to help identify the conflict drivers. Furthermore, conflict analysis processes may help identify strengths and weaknesses of the various parties in conflict, which can be manipulated by groups in power or by armed groups. Therefore the process of conflict analysis must be carefully organized and controlled to assure that it resolves rather than reignites conflicts. One approach has been to incorporate conflict analysis within the broader situation analysis which is a well established part of the normal UNICEF programming process. Education for peacebuilding necessarily addresses fundamental social, cultural, economic and political issues and structures insofar as they may be underlying drivers of conflict. What is the appropriate role and to what extent should UNICEF be involved in advocating changes in these fundamental aspects of society? The Convention on the Rights of the Child and UNICEF's commitments to protect children and achieve education for all provide a solid justification for 50

involvement, but there are ethical considerations as to how far UNICEF should be involved in advocating political changes or changing cultural traditions. It follows that defining programme outcomes and definitions of indicators for monitoring and evaluation should also be mindful of these ethical dilemmas.

51

5.0 Approach to Evaluability Assessment for PEACC Programme 5.1 Why an Evaluability Assessment is needed for PEACC Programme The PEACC is an ambitious and complex programme breaking new ground in the multiple roles of education in post conflict situations. It is intended not merely to restore education services, but to reform education to be conflict-sensitive and to instill values that will contribute to social, cultural, economic and political transformation leading to longer-term peace. This is a new field of work with many conceptual, policy and programmatic issues that still need to be worked out. As illustrated in the literature reviewed above, the fundamental interrelationships between education and peacebuilding are complex and may be negative as well as positive. Conflict analysis with a particular focus on education provides a basis for developing conflict-sensitive education to limit the potentially negative aspects of education. However, going further to develop peace education and other types of education that ultimately lead to long term peace requires continuing research and experimentation. The quality of participation in the conflict analysis and qualities of the programme interventions are critically important. Much of this work is experimental and needs to be carefully monitored and evaluated. There are still conceptual issues in defining the role of education in peacebuilding. In most of the peacebuilding literature, restoring education services is seen as mainly being helpful in building confidence in local and national government. The much larger role of education in peacebuilding as envisioned in the PEACC is not yet widely understood or appreciated. Although the mandate and experience of UNICEF in restoring education in post conflict situations is well established, this expansion in the role of UNICEF in education for peacebuilding is still being defined. Because of the ground breaking and experimental nature of the PEACC programme the normal results-based approach to programming may not be fully suitable. There is a need to be open for experimentation with new methods or activities and adaptations of existing programme approaches. Finding the right balance between flexibility for experimentation and adherence to a results-based programme approach is critical. Also, because this is an ambitious and experimental programme funded largely by a single donor, UNICEF's obligation for careful monitoring, evaluation and regular reporting to the donor is critically important for continued funding. For all these reasons, the knowledge management function (monitoring, evaluation and research) is a centrepiece of the programme. Conducting a comprehensive evaluability assessment at an early stage of the programme will help to develop and define the monitoring and evaluation system and identify areas where more research is needed. 5.2 Purpose of the Evaluability Assessment Evaluability assessment is being undertaken at an early stage of this programme to determine its readiness to be evaluated, but also to help lay the foundation for an effective knowledge management system. Because of the many conceptual, policy and programmatic issues that need to be explored and clarified, it is critically important to establish an effective monitoring and evaluation system and also to identify areas that will require more intensive studies. 52

Evaluability Assessment (EA) is a systematic process that helps identify whether program evaluation is justified, feasible, and likely to provide useful information. EA explores the objectives, expectations and information needs of program managers and policy makers; explores program reality; assesses the likelihood that program activities will achieve measurable progress toward program objectives; and assesses the extent to which evaluation information is likely to be used by program management. The products of evaluability assessment are: (1) a set of agreed-on program objectives, side effects, and performance indicators on which the program can realistically be held accountable; and (2) a set of evaluation/management options which represent ways in which management can change program activities, objectives, or uses of information in ways likely to improve program performance.11 The EA is expected to help sharpen the programme's logic as embodied in the results framework, sharpen the definitions of indicators, identify the information and data requirements to track changes in the indicators, identify the potential sources of information, identify likely gaps in information and suggest ways in which those gaps may be filled. The EA is also expected to review the management aspects of the programme assessing the participation of key stakeholders, and assessing the management structure and capacities, with particular attention to the approach, methods and capacities for monitoring and evaluation. In this way the EA will help set the programme on the right path by building shared understanding of the programme among key stakeholders. The EA's purpose is to facilitate reflection and learning among the programme managers and key stakeholders leading to increased programme coherence and improved management, particularly in the knowledge management functions in the programme. By working in an open and participatory way with programme managers at all levels and through their interactions with donors, strategic partners and programme implementation partners, the evaluability assessment team will be able to provide a knowledgeable but independent perspective on all aspects of the programme. Through workshops at both the country and global level the EA team will be able to present their findings and pose questions to help stimulate reflection on ways to improve monitoring and evaluation and programme implementation and identify areas for more in depth research. This will enable the programme to meet the accountability requirements of UNICEF and the donors but also to help verify the important contribution of education in peacebuilding. 5.3 Scope and focus of the Evaluability Assessment As a global programme, the PEACC evaluability assessment should examine it at all levels including country case studies and regional consultations as well as in depth consultations and interviews with managers and partners at the global level. However, due to resource constraints and the need to focus intensively on each country case, only a few country programmes will be visited and assessed, with the selection based both on those at a more advanced stage of programming and those which have been significantly delayed. The role of the regional office and regional conflict analysis and cross border issues will be addressed through consultations

11

Quotation from Joseph Wholey in Trevisan, Michael and Yi Min Huang, “Evaluability assessment: a primer”, http://pareonline.net/getvn.asp?v=8&n=20

53

with the relevant regional office enroute to or from the selected countries and their participation in country level workshops. At the global level the EA will focus on the programme formulation process with particular attention to how the donor and strategic partners have been involved, and how the results framework and monitoring and evaluation system is being developed. Fundamental questions will be raised about the suitability and limitations of the results-based programming approach and the assumptions underlying the selection and definition of strategic and key outcomes. The proposed PEACC monitoring and evaluation system is currently described in a generic way, but it is recognized that this programme and its M&E system should not be “business as usual”. A full time knowledge management officer and monitoring and evaluation officer have been brought into the Education Section specifically to work on this programme. The approach and content of the monitoring and evaluation system are likely to be unsettled at the time the evaluability analysis is done which will enable the EA team to have considerable influence over how monitoring and evaluation will evolve. Ideally the EA team will be selected in time to participate in any upcoming workshop on the monitoring and evaluation system. At the country level the EA team will examine how the conflict and/or situation analysis was carried out, with particular attention to the extent and nature of participation at the various levels. The situation analysis in Sierra Leone noted that “the process is just as important as the final product”. One concern is to assure that the conflict analysis is broad enough to identify overall conflict drivers while at the same time focused on how education may be related to conflict drivers. In Sierra Leone, for example, the conflict analysis identified and discussed a wide range of conflict drivers and strategic issues related to peacebuilding before focusing on education. The analysis specifically examined how education can support social transformation and is linked to broader social, economic, and political dynamics. This is a critical step for the EA to assess. The EA will examine how the programming process has been able to build on the local conflict analysis but still respond to the globally developed key outcomes and indicators. The country teams in Liberia and Sierra Leone have incorporated conflict analysis findings into local plans. This reflects local realities, but it makes global level aggregation difficult. An issue to consider is does the PEACC global results framework place too many limitations on local programmes or does flexible local programming stray too far from the intent of the PEACC programme? The EA should also examine the global objective and local conflict analysis process, especially in those countries that have delayed implementation, so as to assess the causes of the delays and how they impact programme evaluation. The Sierra Leone programme identifies core functions of education – socialization and identity formation, acquisition of key knowledge, skills and abilities, and employment skills – as well as the need for equitable education access and gender policies. It linked them to identified conflict drivers and resulted in recommendations for “wholesale curriculum reform” that would make education both more conflict sensitive, but also more broadly support peacebuilding. Developing measurable outcome indicators for such broad changes is difficult. Both Liberia and Sierra Leone use activity output targets – trained teachers, number of schools, number of policies, etc – instead of changes in behavior or attitudes which are the peacebuilding outcomes 54

that need to be measured. The EA should assess how such outcomes could be measured and eventually their impacts in terms of reduced conflict, violence, exclusion and marginalization. The EA will look at data availability, both quantitative and qualitative, to assess what primary and secondary data is available to measure both output and outcome indicators. It will also assess what additional studies or surveys are needed, particularly to gather data on attitude and behavioral changes, and the availability of technical and financial resources necessary to collect it. One issue that came up in Sierra Leone, Liberia and nearly all other countries reviewed is the cross border nature of the past conflicts and the flow of arms and refugees among countries. Therefore, consultations with the concerned regional office will also be needed. 5.4 Issues to be reviewed and questions to be addressed in the EA Although the main focus of the EA is on the evaluation system, an overall review of the programme is needed including both management as well as technical aspects. Management aspects include the process of planning and programme development as well as the structures and procedures for implementation including management, coordination, budgeting and fund allocation and systems for accountability including monitoring and evaluation. Technical aspects would include reviewing the theory of change, assuring that the results framework is coherent with the theory of change, reviewing and refining outcome indicators, identifying information needed for tracking indicators and identifying sources of information and suggesting means to fill any gaps in information that may exist. The PEACC is a large scale programme in a new area of programming where the linkages among the programme activities, outputs and the final outcomes affecting peacebuilding have not been firmly established. However, the standard results-based programming approach has been used. The following fundamental questions about the origin and design of the programme should be addressed to the programme designers and managers. • • •

• • • •

Why did the programme start at a large scale rather than with pilot projects? How were the countries selected and what criteria were used for the selection? If the PEACC is a direct follow up of the earlier EEPCT, what are the main differences between the programmes and how have these been taken into account in the way the PEACC is designed and implemented? What is the basis for the PEACC programme design given that no theory of change is presented and no global conflict analysis has been done? Why was a results-based programming approach used? Were other alternatives such as a process-based approach or an experimental research and development process considered? How can the flexibility for research and experimentation on new methods or alternative processes be accommodated within the results-based approach?

A weakness identified in the EEPCT was the inconsistency in the process of allocation of funds. In view of the fact that country level programming for the PEACC has been slower than anticipated: 55

• • •

What has been the basis for allocation of funds to country offices? How can fund allocation be adjusted among countries to support the varied levels and pace of programming? How will country differences in capacities to utilize funds be reflected in their allocations?

It is also important to identify key stakeholders and assess their knowledge, attitudes and roles in planning as well as implementation of the programme. In this programme there are many stakeholders at the global level, most important is the Government of Netherlands – the primary donor. One of the early interviews should be with the GON to find out their expectations for the PEACC and any specific conditions or constraints on the use of their funds. Some of the questions to be addressed to the GON are: • • • • •

How did the GON view the results of the earlier EEPCT programme which it funded? What were the main concerns regarding how the EEPCT was designed and implemented? Why was there a shift to focus on post conflict countries but not emergencies? What was the GON role in the design of the current PEACC programme? What pre-conditions were placed on the programme design and country selection?

UNICEF technical sections (Child Protection, Early Child Development, Adolescents, Disabled), the Emergency Operations Division (EMOPS), Department of Communications, Evaluation Office, Programme Department Operations and Supply Division are all involved in support of the PEACC programme. EMOPS, Child Protection and the Department of Communications have critical roles to play and should be interviewed regarding how they have been involved in the design and implementation of the programme. Some indicative questions are: • • • • • •

How was your section or division involved in the early design stages of the programme? For example, were you involved in the negotiations with the donor? Does your section or division have an ongoing technical role in the programme? Are you involved in the conflict analysis or programme design at the country level? What role does your section or division have in the monitoring and evaluation of the programme? Have you been involved in design of the results framework and selection or review of indicators? What theories of change has your section identified for UNICEF’s peacebuilding work?

There are a number of strategic multilateral partners: UN Peacebuilding Support Office (PBSO), UNDP Bureau for Crisis Prevention and Recovery (BCPR), the Inter-Agency Network for Education in Emergencies (INEE), the Inter-Agency Standing Committee (IASC) Education Cluster and the Education For All Fast Track Initiative (EFA-FTI). There is likely to be some overlapping in the mandates and activities of PBSO and BCPR in particular so a high level of coordination is desirable. Interviews with some of these strategic partners are needed to assess both the degree of coordination but also the prospects for policy advocacy on the role of education in peacebuilding. •

How much has your organization been involved in the design of the PEACC? 56

• • • •

In your view, what is the role of education in peacebuilding? How does the role of education change at different stages of peacebuilding? In your view, what percentage of peacebuilding funds should be devoted to education? What mechanisms are there to assure good coordination with UNICEF and particularly this programme in the overall peacebuilding process?

There are a number of mechanisms and structures for stakeholder coordination noted in the programme document such as the Programme Advisory Group (Composed of UNICEF offices and divisions noted above plus key external implementing agency partners such as PBSO and UNDP-BCPR), Strategic Partners Advisory Group (SPAG) and the Global Non-thematic Allocation Committee that reviews and approves allocations within the programme. The participation in and effectiveness of these structures for coordination should also be assessed through the structured interviews with strategic partners. 5.5 Monitoring and Evaluation System One of the initial weaknesses of the earlier EEPCT programme was inadequate management, monitoring and evaluation systems and capacities. There have been substantial improvements in the PEACC programme with the appointment of a full time programme manager, knowledge management officer and a monitoring evaluation officer within the Education Section. Nevertheless the monitoring and evaluation system should be carefully assessed. The proposed monitoring and evaluation system is described in a general way awaiting the development of a revised and finalized results framework. The global results framework and national results frameworks should be reviewed to determine if they are consistent with a theory of change. The results framework should be analyzed to determine if the causal links among the key outcomes and between the programme activities and outputs are explicitly identified. Raising questions with the Monitoring and Evaluation Officer and the Knowledge Management Officer along the following lines will clarify the M&E system. • • • • • • • •

Given that no theory of change is stated in the programme documents, what is the basis for a coherent model for monitoring and evaluation? What is the implied theory of change used? Is the current results framework consistent with that theory of change? Are the strategic outcome and five key outcomes in the results framework now fixed? If so, are the linkages among the outcomes clearly defined and measurable? Are the proposed key outcome indicators SMART (specific, measurable, achievable, relevant and time-bound)? If not, how can they be redefined or can they only be defined more specifically at the national level? If different national level indicators are used or the definitions of global indicators vary among countries, how can they be aggregated at the global level?

One important feature of PEACC is Outcome 5 – Knowledge Evidence and Advocacy -“Adequate generation and use of evidence and knowledge in policies and programming on linkages between education, conflict and peacebuilding.” This gives the results-based approach 57

a research and development feed back into the programme and broader peacebuilding policies. To achieve this outcome extra resources will be needed for the knowledge management function. • • • •

What personnel, technical and financial resources have been provided to support the knowledge management function at each level to achieve outcome 5? How will the M&E system be able to measure and evaluate the strength and viability of the links between education inputs, outputs and outcomes and peacebuilding outcomes? What indicators will be used to judge the relative effectiveness of different education programme activities in achieving behavioral outcomes that support peacebuilding? How will these links be documented to support the national and global level advocacy for an increased role for education in peacebuilding?

As has been noted in Sierra Leone, the process is as important as the results in conflict analysis and programming for the PEACC. If this is true, consideration should be given to monitoring and evaluation of processes as well as outcomes. • • • • • • •

What process monitoring and evaluation methodologies are being considered? What process indicators will be used to assess the extent and quality of participation in conflict analysis, programming and M&E? Are process indicators qualitative and subjective or can they also be SMART? How can process indicators be made measurable and able to track changes over time?12 How important is it to have the cooperation and participation of all parties to the previous conflicts in the monitoring and evaluation process? How will local offices get the cooperation of conflicting parties in the M&E process? How will the participation and role of women and youth be monitored?

Once the outcome and process indicators have been reviewed and refined, the M&E officers need to identify the information and data required for tracking each of the indicators. The main problem with this is the severe limitations on existing data sources and the difficulty of conducting surveys and interviews in conflict affected areas. Country level and field visits will be required in order to accurately identify existing information sources and gaps in information. This will vary considerably from country to country. Some questions to be addressed at the country level include: • • • • • •

What was the extent and quality of participation in the conflict analysis and programming process, particularly the participation of conflicting groups as well as women and youth? What education-related or education-specific conflict drivers were identified? What programme actions were taken to address the identified conflict drivers? What structural or policy issues related to education were identified in the conflict analysis? What policy advocacy or support for policy and structural changes is being proposed? What projects that directly support education for peacebuilding are being proposed?

12

The attached “Participation Empowerment Index” may be useful to consider as one tool for measuring and tracking participation. 58

•

What criteria are used to assess the likely effectiveness of the selected programme actions in contributing to peacebuilding?13

Both Liberia and Sierra Leone have results frameworks for the programme. However, both use activity output targets – training teachers, number of schools, number of policies, etc – instead of changes in behavior or attitudes. As noted in PEACC programme Key Outcome 3, establishing a link between education activities and peacebuilding depends on measurable changes in attitudes and behavior. • • • • • • • •

What indicators have been identified to assess and measure the effectiveness of programme activities to bring about attitudinal or behavioral changes? If there are no indicators for behavioral change, how can their link to peacebuilding be established? How can attitude and behavior changes be monitored and evaluated? Did the conflict analysis process provide or enable the extrapolation of a base line on relevant attitudes and behavior? Have relevant knowledge, attitude and practice (KAP) surveys been conducted to establish a base line? Are technical and financial capacities available locally to conduct periodic KAP surveys to enable monitoring and evaluation of attitudinal and behavioral outcomes? What external support or assistance may be needed to conduct KAP surveys? What other methods may be used to track behavioral changes related to peacebuilding?

Evaluating the long term impact of programme particularly related to Key Outcome 4 will also depend on establishing a base line of relevant quantitative data and being able to track changes in that data. Among others this is would include data on education participation rates disaggregated by gender, ethnicity and possibly religion and language, depending on the relevant factors in the context of the conflict. It may also include data on access to schools, quality of school buildings, availability of educational materials and textbooks, and the qualifications, ethnicity and gender of teachers. Much of this data may be available through the Education Management Information System (EMIS). • • • • • • •

What data is being collected and analyzed by the EMIS? What other sources of data are available such as household surveys or census statistics? When was the last Multiple Indicator Cluster Survey (MICS) and what education related information was collected? To what extent is the EMIS, MICS or other survey data available for conflict-affected areas? Is there local technical and financial capacity to periodically conduct follow up surveys? What external support may be needed to conduct surveys or special studies to establish a solid baseline of quantitative data that is appropriately disaggregated? What external support may be needed to conduct follow up studies to support midterm or final evaluations of the programme?

13

The attached tool noting five criteria or qualities of peacebuilding projects may be helpful in considering this issue.

59

As noted above, nearly all country cases reviewed, (South Sudan, Liberia, Sierra Leone, Congo, etc.) have substantial cross border issues related to their conflicts. Therefore it is desirable to visit the regional office concerned to assess cross border issues related to the countries visited. Some of the questions to be addressed at the regional level are as follows. • • • • • • • • •

What regional conflict analyses have been done by the various peace keeping and peacebuilding bodies? How did UNICEF participate in the regional conflict analyses? What education-related conflict drivers, if any, were identified which affect more than one country in the region? What regional level response to the identified conflict drivers is planned or needed? How have the Regional Education or M&E Advisors been involved in country level conflict analysis or programming related to PEACC programme? What joint programme actions, training, research or M&E activities have been conducted or are planned for the region that support the PEACC country programmes? What regional programme activities are planned under the PEACC? How does the regional office support monitoring and evaluation of the PEACC programme? What studies to support the PEACC programme should be done at the regional level?

5.6 Methods to be used in the Evaluability Assessment The EA will require a thorough desk review of programme documents for the current Peacebuilding, Education and Advocacy programme and the previous EEPCT programme as well as a review of the literature on education for peacebuilding and UN policy documents on peacebuilding, education and evaluation. The desk review will focus intensively on the programme proposal's logic as presented in the results framework and identify issues that need to be clarified with the programme management team. Some key documents to be reviewed are: • • • •

• •

•

UNICEF. 2011. Proposal on Peacebuilding and Education presented by the Education Section to Government of Netherlands. UNICEF. 2012. Education in Emergencies and Post-Crisis Transition: Final Synthesis Report to the Government of the Netherlands and the European Commission. UNICEF. 2010. Progress Evaluation of the UNICEF Education in Emergencies and PostCrisis Transition Programme (EEPCT). (Also known as the PREV). UNICEF. 2010. “Programme Review and Evaluability Study (PRES) UNICEF’s Education in Emergencies and Post-Crisis Transition (EEPCT) Programme.” University of York. UN Peacebuilding Support Office. 2010. “UN Peacebuilding: An Orientation. UN Peacebuilding Support Office. No date. “United Nations Peacebuilding Support Office (PBSO) Strategy 2012-2013, available at http://www.un.org/en/peacebuilding/pbso/pdf/pbso-strategy-2012-2013.pdf UNESCO. 2011. EFA Global Monitoring Report: The hidden crisis: Armed conflict and education. 60

Structured interviews covering the issues and questions noted above should be conducted with the following: • • • • • • •

The Programme Manager for PEACC The Knowledge Management and Monitoring and Evaluation Officers, Key staff of EMOPS, Child Protection Section and other related offices in UNICEF Key strategic partner organizations, particularly UNPSO Country programme managers and monitoring and evaluation officers Key national government officials and NGO implementing partners Regional Education and M&E Advisors

Field visits to communities supported through the project should include focus group discussions with local government officials, NGOs, community leaders, school administrators and teachers, and also women and youth groups. Participatory workshops to present and discuss findings and recommendations of the evaluability assessment should be organized with the country offices at the end of the country visits. A final stakeholder workshop to present and discuss the overall findings and recommendations would be organized jointly with the Education Section and Evaluation Office. 5.7 Process and schedule for Evaluability Assessment Because of the complexity and scope of the PEACC programme and the importance of the knowledge management function, a longer and more thorough evaluability assessment will be required. It will take several months with extensive consultations in New York, visits to several country programmes and consultations with a regional office. The EA will begin with document reviews and briefings of the EA team by the Evaluation Office and preliminary consultations with the Education Section's PEACC Programme Manager, Knowledge Management Officer and Monitoring and Evaluation Officer. (One or two days in NY) After further in depth document review and analysis the EA team will prepare an inception report and interview frames. (One week) The team will then conduct structured in depth interviews with the programme management team to review the technical issues including results framework, outcome and output objectives and indicators. This would be followed by interviews with key strategic partners in New York to clarify their perceptions of the programme and their roles within the programme, particularly in the programme design and planning process. (One week in NY) Results of interviews and document review will be documented in an interim report on findings and observations. (One week) Country visits will review the country programming process including conflict analysis, programme documents, results framework, output and outcome objectives and targets. Particular attention will be given to defining indicators and methods to measure both processes and results and what are the information sources and baseline information available. Country visits will conclude with a workshop to present findings and recommendations of the EA team. (10 days per country) 61

Enroute to or from the country visit the EA team would visit the regional office concerned to interview the Regional Education and Monitoring and Evaluation Advisors. (2 days) A draft report will be prepared which extends and elaborates the earlier interim report based on the results of country and regional level interviews, field visits and focus group discussions. (2 weeks) After the draft report has been reviewed by the Evaluation Office, it will be circulated to the Education Section PEACC Programme Manager and management team. (2 weeks). The EA team would then be called in for consultations on their findings and suggested changes to improve the evaluability of the programme. (2 days in NY) A final stakeholder workshop at global level would be organized jointly by the Evaluation Office and Education Section to review the findings and recommendations with other UNICEF divisions and sections and key strategic partners. (one day workshop with 2 days preparation) A final report documenting the entire process with attached reports on country visits and workshops as well as the results of the final stakeholder workshop will be submitted within one month. 5.8 Qualifications of Team and Firm to Conduct EA The evaluability assessment team will be comprised of two members, one expert familiar with UNICEF's programming and management systems who will lead on the overall management and programme design aspects and one expert on evaluability assessment and monitoring and evaluation systems who will lead on the technical aspects of monitoring and evaluation. One or both of the team members and the firm they represent should be experienced in working with vulnerable populations in conflict affected countries and be experienced in facilitating stakeholder workshops. 5.9 Tools to Support the Evaluability Assessment The attached Evaluability Assessment Checklist is simply a listing of the many steps and activities for easy reference and as a reminder of what needs to be covered at each stage of the assessment. The Participation Empowerment Index is intended to facilitate a more analytical assessment of the extent and quality of participation in the conflict analysis, programming, project design and implementation as well as in monitoring and evaluation. Criteria for screening and assessing education activities in relation to peacebuilding lays out five elements to be addressed in education for peacebuilding: child protection, psycho-social support, reintegration, economic recovery and skills development, and socio-political reform.

62

References Adams, Maurianne, Lee Anne Bell, and Pat Griffin. 2007. Teaching for Diversity and Social Justice. New York: Routledge. Adelore, Omobola, and Henry Majaro-Majesty. 2008. “Literacy teaching method and peace building in multi-ethnic communities of Nigeria.” Australian Journal of Adult Learning 48(1):162–187. Azar, Edward. 1990 The Management of Protracted Social Conflict: Theory and Cases. Aldershot: Dartmouth. Bar-Tal, Daniel, and Yigal Rosen. 2009. “Peace Education in Societies Involved in Intractable Conflicts: Direct and Indirect Models.” Review of Educational Research 79(2):557–575. Bennett, J. et al. 2010. Aiding the Peace: A Multi-donor Evaluation of Support to Conflict Prevention and Peacebuilding Activities in Southern Sudan 2005-2010. United Kingdom: ITAD, ltd. Retrieved (http://www.alnap.org/pool/files/1416.pdf). Bickmore, Kathy. 2011. “Policies and Programming for Safer Schools: Are ‘Anti-bullying’ Approaches Impeding Education for Peacebuilding?” Educational Policy 25(4):648–687. Bretherton, D., J. Weston, and V. Zbar. 2005. “School-based peace building in Sierra Leone.” Theory into Practice 44(4):355–+. Bush, Kenneth and Diana Saltarelli, eds. 2000. The Two Faces of Education in Ethnic Conflict; Towards a Peacebuilding Education for Children. UNICEF Innocenti Research Center. Choi, Rainbow. 2010. “Literacy Learning as Peacebuilding in Post-Conflict Societies: The Case of Sierra Leone.” UNIVERSITY FOR PEACE. Collaborative Learning Projects. 2009. Reflecting on Peace Practice. Accessed Oct. 1, 2012 from Back on Track website (limited access). Culver, Keri, Kathryn Brand, Felipe Tejeda, and Thomas Alvares de Azedo. 2010. Evaluation of the Go to School Initiative in Southern Sudan. American Institute of Research. Accessed Oct. 1, 2012 from http://www.unicef.org/evaldatabase/files/Sudan_2010001_Final_Report_GtS_Evaluation.pdf. Danesh, B. H. 2006. “Towards an Integrative Theory of Peace Education.” Journal of Peace Education 3(1):55–78. Retrieved September 19, 2012. Davies, Lynn. 2005. “Evaluating the Link between Conflict and Education.” Journal of Peacebuliding and Development 2(2):42–58. Denskus, Tobias. 2012. “Challenging the international peacebuilding evaluation discourse with qualitative methodologies.” Evaluation and Program Planning 35(1):148–153. Dessel, Adrienne B, and Noor Ali. 2012. “The Minds of Peace and intergroup dialogue: two complementary approaches to peace.” Israel Affairs 18(1):123–139. Dupuy, Kendra E. 2008. “Education in peace agreements, 1989–2005.” Conflict Resolution Quarterly 26(2):149–166. Retrieved September 19, 2012. Fast, Larissa A., and Reina C. Neufeldt. 2005. “Envisioning Success: Building Blocks for Strategic and Comprehensive Peacebuilding Impact Evaluation.” Journal of Peacebuliding and Development 2(2):24–41. Freire, Paolo. 1970. Pedagogy of the Oppressed. New York: Continuum. Harris, Ian M. 2004. “Peace education theory.” Journal of Peace Education 1(1):5–20. Retrieved September 19, 2012. Harris, Ian M. 2007. “Peace Education in a Violent Culture.” Harvard Educational Review 77(3):350–354,391. Retrieved September 19, 2012. 63

Goydor, Hugh, Alistair Dutton and Sus Abhayaratna. 2004. SEDEC Humanitarian Programme Review 1994-2004. Retrieved Sept. 22, 2012 from http://www.alnap.org/pool/files/erd-3237full.pdf. Johnson, David, and Roger. Johnson. 2005. “Essential components of peace education.” Theory into Practice 44(4):280-292. Kaldor, Mary. 2006. New and Old Wars: Organized Violence in a Global Era. 2nd ed. Cambridge: Polity Press. Ketel, Hermen. 2008. Youth Education Pack External Evaluation. Norwegian Refugee Council. Kirk, Colin. 2012. “Evaluating the Peacebuilding, Education and Advocacy Programme.” Presentation at Sept. 2012 PEACC workshop. Kupermintz, H., and G. Salomon. 2005. “Lessons to be learned from research on peace education in the context of intractable conflict.” Theory into Practice 44(4):293–302. Lederach, John Paul. 1997. Building Peace: Sustainable Reconciliation in Divided Societies. Washington, DC: United States Institute of Peace. Lederach, John Paul, Reina Neufeldt, and Culbertson, Hal. 2007. “Reflective Peacebuilding: A planning, Monitoring and Learning Toolkit.” Retrieved September 14, 2012 (http://kroc.nd.edu/sites/default/files/reflective_peacebuilding.pdf). Lowrey, Bill, Allen Harder, and Vachel Miller, eds. 2005. And the Children Shall Lead Them. Monrovia, CA: World Vision Ramsbotham, Oliver, Tom Woodhouse, Hugh Miall. 2011. Contemporary Conflict Resolution. 3rd ed. Cambridge: Polity Press. Mac Ginty, Roger. 2013. “Indicators +: A proposal for everyday peace indicators.” Evaluation and Program Planning 36(1):56–63. Retrieved September 19, 2012. Miller, Derek B., and Lisa Rudnick. 2010. “The Case for Situated Theory in Modern Peacebuilding Practice.” Journal of Peacebuilding and Development 5(2):62–74. Novelli, Mario, and Alan Smith. 2011. “The Role of Education in Peacebuilding: a synthesis report of findings from Lebanon, Nepal and Sierra Leone.” New York: UNICEF. Olberding, Julie Cencula, and Douglas J Olberding. 2010. “‘Ripple Effects’ in Youth Peacebuilding and Exchange Programs: Measuring Impacts Beyond Direct Participants.” International Studies Perspectives 11(1):75–91. Paffenholz, Thania, Mohammed Abu-Nimer, and Erin McCandless. 2005. “Peacebuilding and Development: Integrated Approaches to Evaluation.” Journal of Peacebuilding and Development 2(2):1–5. Poppema, Margriet. 2009. “Guatemala, the Peace Accords and education: a post‐conflict struggle for equal opportunities, cultural recognition and participation in education.” Globalisation, Societies and Education 7(4):383–408. Retrieved September 19, 2012. Ramsbotham, Oliver, Tom Woodhouse and Hugh Miall. 2011. 3rd ed. Contemporary Conflict Resolution. Cambridge, UK: Polity Press. Richmond, Oliver P. 2007. “Emancipatory forms of human security and liberal peacebuilding.” International Journal 62(3):458–477. Retrieved September 19, 2012. Rogan, James. “Conflict Sensitivity and Peacebuilding in UNICEF” Meeting presentation Sept. 25, 2012. New York: UNICEF. Roholt, Ross VeLure, and Michael Baizerman. 2012. “Being practical, being safe: Doing evaluations in contested spaces.” Evaluation and Program Planning 35(1):206–217. Retrieved September 19, 2012. 64

Rosen, Yigal, and Gavriel Salomon. 2011. “Durability of peace education effects in the shadow of conflict.” Social Psychology of Education 14(1):135–147. Salm, Randall. 2005. “From Conflict Resolution Curriculum to Peace Camps,” in And the Children Shall Lead Them: An NGO Journey Into Peace Education, Federal Way, WA: World Vision. Smith, Alan, Erin McCandless, Julia Paulson, and Wendy Wheaton. 2011. The Role of Education in Peacebuilding: a literature review. New York: UNICEF. Smith, Dan. 2004. “Towards a Strategic Framework for Peacebuilding: Getting Their Act Together.” Norway: Royal Norwegian Ministry of Foreign Affairs. Sommers, Marc. 2002. Children, Education and War: Reaching Education for All (EFA) Objectives in Countries Affected by Conflict. Conflict Prevention and Reconstruction Unit Working Paper. Washington, DC: The World Bank. Retrieved September 19, 2012 (http://www.eric.ed.gov/ERICWebPortal/detail?accno=ED474426). Tomlinson, Kathryn, and Pauline Benefield. 2005. Education and Conflict: research and Research Possibilities. Berkshire, England: National Foundation of Educational Research. Retrieved Sept. 21, 2012 from http://www.nfer.ac.uk/nfer/index.cfm. Tschirgi, Necla. 2010. “Strategic Frameworks and Peacebuilding: Current Trends and Critical Directions.” Journal of Peacebuilding and Development 5(2):1–5. United Nations. 2006. “Integrated Missions Planning Process (IMPP).” Retrieved September 21, 2012 (http://www.regjeringen.no/upload/UD/Vedlegg/FN/Multidimensional%20and%20Integrate d/06_DPKO_IMPP_final_.pdf). United Nations. 2010. “Monitoring Peace Consolidation: United Nations Practitioners’ Guide to Benchmarking.” Retrieved Sept. 29, 2012 from http://www.un.org/en/peacebuilding/pbso/pdf/monitoring_peace_consolidation.pdf. UN General Assembly, Report of the Secretary-General on peacebuilding in the immediate aftermath of conflict, 11 June 2009, A/63/881–S/2009/304, available at: http://www.unhcr.org/refworld/docid/4a4c6c3b2.html [accessed 19 September 2012] UN Peacebuilding Commission. 2012. “Working Group on Lessons Learned: Resource Mobilization and Mapping of Relevant Actors.” Retrieved Sept. 20, 2012 from (http://www.un.org/en/peacebuilding/wgll/PBC%20WGLL%20Draft%20Background%2 0Paper%20Final.pdf). UN Peacebuilding Commission. “Roadmap of Actions in 2012.” Retrieved September 19, 2012 (http://www.un.org/en/peacebuilding/pbso/pdf/pbc_roadmap_2012.pdf). UN Peacebuilding Fund. 2012. “The Peacebuilding Fund: Report of the Secretary General.” 16 January 2012. A/66/659. UN Peacebuilding Support Office. 2010. “UN Peacebuilding: An Orientation. UN Peacebuilding Support Office. No date. “United Nations Peacebuilding Support Office (PBSO) Strategy 2012-2013, available at http://www.un.org/en/peacebuilding/pbso/pdf/pbso-strategy-2012-2013.pdf UN Peacebuilding Support Office. No date. “UN Peacebuilding Support Office Policy Issues.” Retrieved September 22, 2012 (http://www.un.org/en/peacebuilding/pbso/policy.shtml). UN System Staff College. No date. Causal Analysis: Course on Conflict Sensitivity and Peacebuilding in UNICEF. Accessed Oct. 1, 2012 from Back on Track website (limited access). UNESCO. 2011. EFA Global Monitoring Report: The hidden crisis: Armed conflict and education. 65

UNICEF. 2010. Evaluation of UNICEF’s Education Programme in Timor Leste. Retrieved Sept. 22, 2012 from http://www.alnap.org/pool/files/1442.pdf. UNICEF. 2010. Progress Evaluation of the UNICEF Education in Emergencies and Post-Crisis Transition Programme (EEPCT). (Also known as the PREV). UNICEF. 2010. “Programme Review and Evaluability Study (PRES) UNICEF’s Education in Emergencies and Post-Crisis Transition (EEPCT) Programme.” University of York. UNICEF. 2011. Proposal on Peacebuilding and Education presented to Government of Netherlands. UNICEF. 2012. Education in Emergencies and Post-Crisis Transition: Final Synthesis Report to the Government of the Netherlands and the European Commission. UNICEF. 2012. Terms of Reference for the review of lessons learned and approach paper for the evaluability assessment of the Education and Peacebuilding Programme. UNICEF Liberia. 2012. EEPCT 2011 Programme Report. Accessed Oct. 1, 2012 from Back on Track website (limited access). UNICEF Myanmar. 2012. EEPCT 2011 Programme Report. Accessed Oct. 1, 2012 from Back on Track website (limited access). UNICEF South Sudan. 2012. EEPCT 2011 Programme Report. Accessed Oct. 1, 2012 from Back on Track website (limited access). U.S. Department of Labor. “Child Labor Programs.” http://www.dol.gov/ilab/programs/ocft/icltc.htm. van Leeuwen, Mathijs, Willemijn Verkoren, and Freerk Boedeltje. 2012. “Thinking beyond the liberal peace: From utopia to heterotopias.” Acta Politica 47(3):292–316. Walsh, Denis, and Soo Downe. 2005. “Meta-synthesis method for qualitative research: a literature review.” Journal of Advanced Nursing 50(2):204–211. Retrieved September 3, 2012. Zartman, I. William. 2007. Peacemaking in International Conflict: Methods & Techniques. US Institute of Peace Press.

66

Appendix A: ToR for Literature Review and Approach Paper Title

Terms of Reference for the review of lessons learned and approach paper for the evaluability assessment of the Education and Peacebuilding Programme

Purpose

The purpose of this consultancy is to develop, for consideration by the Evaluation Office, an approach for the evaluability assessment of the Education and Peacebuilding Programme (as envisaged in the proposal). Consultant’s home base with possible travel to the UNICEF Evaluation Office in New York, NY

Location Duration

40 days in the period August 2012 through November, 2012

Start Date

August 13, 2012

Reporting to

Evaluation Specialist, Evaluation Office

Budget Code/PBA No

tbd

1. Background: One in four people (about 1.5 billion of the world’s population) live in fragile and conflict affected states, or in countries with very high levels of criminal violencei. Countries affected by conflict also experience higher levels of poverty. For example, poverty rates in countries that experienced major violence over the period from 1981 to 2005 are, on average, 21 percentage points higher than in countries that did not experience violent conflict.ii According to the World Bank, fragile states account for 36 percent of all people living in extreme poverty, 33 percent of children with no primary education, and 39 percent of all infant deaths. With these factors contributing to making insecurity a primary development challenge of our time, it is understandable that peacebuilding has emerged as a central strategy to address conflict and accelerate progress towards achieving the MDGs. Peacebuilding practice is a complex undertaking that requires a holistic understanding of needs for security, justice, political stability, and socio-economic recoveryiii. Most societies do recognize the relationship between education and conflict and readily accept education as a peace dividend. For instance, there is consensus that restoring normal functioning of schools and the reconstruction education system can provide an early ‘peace dividend’ in countries recovering from conflict. However, education’s potential in peacebuilding remains largely unharnessed and/or untapped. Education can contribute to dimensions of peacebuilding, such as conflict prevention, social transformation, civic engagement and economic progress.iv It can also contribute to improved governance by addressing underlying inequities that fuel conflict, providing learning and employment opportunities to disenfranchised youth, empowering adolescent girls and women as actors in the peace-building process, imparting civic and political education, and modelling 67

democratic participation and decision-making. Where there is a need to address inequalities that exacerbate grievances between groups within society, education can serve as an important preventative strategy to transform accepted norms around violence, gender, and power. So, while “provision of basic education services” is explicitly recognised as a critical area in delivering initial peace dividendsv in conflict and post conflict situations, reorienting education for reflective peacebuilding practice is equally important. This approach calls for education practice to be inherently reflective, to be able to confront the reality that education can be a driver of conflict when it is found to be complicit in creating the conditions for armed conflict as does happen when inequitable provision of opportunities for schooling, biased curriculum content, or biased teaching methods reinforce existing exclusion and/or stereotypes. In response to issues identified above, the Government of Netherlands and UNICEF are partnering in a programme, unique in both scale and scope, to strengthen resilience, social cohesion and human security in fourteen (14) countries at risk of conflict, experiencing conflict, or recovering from conflict. With the strategic result of strengthening policies and practices in education for peacebuilding, the Education and Peacebuilding Programme14 focuses on five key outcomes: (i) integrating education into peacebuilding and conflict reduction policies, analyses and planning; (ii) increasing institutional capacities to supply conflict sensitive and peace education; (iii) increasing capacity of children, parents, teachers and other duty-bearers (such as guardians and local leaders) to prevent, reduce, and cope with conflict and promote peace; (iv) increasing access to quality, relevant, conflict-sensitive education that contributes to peace; and, (v) contributing to the generation and use of evidence and knowledge in policies and programming related to education, conflict and peacebuilding. Within UNICEF Headquarters, a number of sections and/or divisions are collaborating to ensure that participating countries can benefit from in-house expertise. The Education Section has the primary responsibility for the programme, providing overall guidance and management, as well as putting in place the necessary arrangement for programme monitoring. Emergency Operations (EMOPS) and the rest of the Programme Division (PD) provide important programmatic guidance to the participating countries, while the Evaluation Office is responsible for providing technical advice on the monitoring and evaluation arrangements, as well as providing oversight for the evaluative aspects of the programme. 2. Purpose and objectives of consultancy Peacebuilding is a complex, multifaceted process of change with the ultimate goal of ‘perpetuating peace’, or preventing the eruption of violent conflict. This means that evaluating any programme which stipulates ‘building peace’ as an outcome often includes, among others, measuring a conflict that never erupted. Hence monitoring and evaluating the Education and Peacebuilding programme cannot be a simple matter of measuring a final result – it requires a robust strategy that builds on field-based experiential knowledge (such as lessons from the Education in Emergencies and Post Crisis Transition programme, otherwise known as EEPCT), and which proffers well thought out outcome and process indicators, as well as realistic timeframes. It also requires a variety of tools and avenues of inquiry into how change processes 14

The full name of the programme is Peacebuilding, Education and Advocacy in Conflict-affected Countries

68

operate, while creating evidence that will assist programme implementers, stakeholders, and evaluators in assessing the extent to which programme inputs/outputs have been successful in ‘building peace’. Any arrangements to monitor and evaluate peacebuilding outcomes should therefore be informed by a thorough understanding of the knowledge in the peacebuilding literature, different theories of change and how they may apply to programme interventions, existing evaluative frameworks, and practice. Hence, the purpose of this consultancy is to develop a thought piece to support the Evaluation Office (EO) in preparation for the oversight/advisory role. The work will be used by EO to assess the monitoring and evaluation arrangements of the Education and Peacebuilding programme, and to develop an approach for an ‘evaluability assessment’ The following tasks should be undertaken in the consultancy; 1. Summarize literature on evaluative approaches for peacebuilding programmes that use education as a central strategy for achieving intended outcomes, including a review of the EEPCT programme results framework put in place after the PRES and how it was implemented in a number of EEPCT participating countries. 2. From 1 above, distil key lessons to take into consideration when developing the M & E strategy for the Education and Peacebuilding Programme; 3. Study the Education and Peacebuilding Programme proposal document, and programme documents from two participating countries, determine whether there is a coherent theory of change (from both the peacebuilding and education practices) that influenced programme development, and whether a reasonable M&E strategy has been laid out in view of the proposed theory of change. 4. From 3 above, proffer advice on ethical issues/considerations likely to arise in monitoring and evaluation of the programme, and actions to mitigate them. 5. Proffer an approach for the evaluability assessment planned for the first year of the Education and Peacebuilding programme, including draft terms of reference (TORs) for the evaluability study. It is anticipated that the above-mentioned tasks will be completed in period of three months (August through October, 2012), and that products from the consultancy (the literature review and the evaluability approach) will be taken into consideration by the programming counterparts when finalizing the results framework for the programme. 3. Key skills, academic/technical background, and experience required •

•

Two evaluation professionals/practitioners with extensive experience in planning, monitoring an evaluating international development programmes are required. The preferred approach would be for consultants to bid together as a team. However, individual bids will be accepted (for half the time allotted), as long as bidders are aware that they will be required to work as a team with another qualifying bidder. The successful consultant team should to offer the following range of skills and experience: Programming experience in peacebuilding programmes and the use of education for building peace and social cohesion; 69

• • • • •

Extensive practice in evaluating programmes with education and peacebuilding content; Demonstrate expertise/experience in conducting ‘lessons learned’ exercises; Demonstrate expertise/experience in developing results frameworks, tools or guides for monitoring and evaluation; Be fully acquainted with results-based management orientation and practices and able to prepare products in the UN style; and, Have excellent language and communication and report writing skills, in English.

4. Management arrangements and accountabilities This work will be supervised by the Evaluation Specialist in UNICEF Evaluation Office, to whom the consultant team will report. (a)

(b)

The consultant team will be responsible for the following: • development of a workplan for executing the consultancy; • regular progress reporting of progress and results to the Evaluation Specialist in the Evaluation Office; and, • production of deliverables as shown in the table below, and in accordance with contractual requirements. The Evaluation Specialist in the Evaluation Office will have overall responsibility to:
co-ordinate, direct and supervise all activities of the consultancy.
guide all phases of execution;
consult with Education Section and the Technical Working Group of the Education and Peacebuilding Programme, as well as other internal review processes; and
approve all deliverables;

5. Timeline, budget and deliverables Output/deliverables 1. An inception note (refined approach from Section 6.3 below) and the workplan 2. Brief literature review of education and peacebuilding program evaluation practice and lesson from the EEPCT programme results framework, as well as an outline for the approach paper (3 below) 3 An approach paper for the evaluability assessment of the Education and Peacebuilding Programme with necessary tools, including an annex of draft terms of reference for conducting the evaluability assessment. TOTAL

Length of Report/workplan 5-6 pages

Person days 02

25 - 30 pages including executive summary, and excluding annexes

24

20 - 25 pages including executive summary, and excluding annexes

14

Deadline 20 August 2012 01 October 2012

09 November 2012

40 70

The consultancy will be executed in a total of 40 person days15 in the period 13 August, 2012 to 15 November 2012. Consultancy fees will be payable at P5-D1 level, depending on the qualifications and experience of the consultants. Consultants will be paid as per schedule above, and upon submission of satisfactory products. Within the contract period above, the consultants may be requested to undertake a trip to UNICEF New York for the purpose of consultations – the expenses for which will be covered by UNICEF. (Consultants will be responsible to submit copies of travel health coverage prior to any travel for this contract). Additional expenses up to and not exceeding US $500 will be reimbursed to the consultant upon presentation of breakdown and receipts for miscellaneous expenses (telephone calls, fax, photocopy, postal charges, etc.). 6. Expression of Interest All interested consultants or consultant teams should provide following by 07 August, 2012, at midnight, New York City time. 1. A completed ‘expression of interest’ form (see Appendix 1). Only one form is required per consulting team 2. A copy of CV/resume (or resumes for each member of the consulting team) 3. A 3-4 page outline, detailing the proposed approach for the literature review and evaluability assessment 4. Two writing samples, preferably evaluation reports with a description of the consultant’s role in each report. A design an evaluation or evaluability study, or a report of an evaluability assessment for which the consultant was responsible would be a distinct and added advantage. Correspondence should be sent to the email address below, with the stipulated subject: Email: [email protected] Subject: Education and Peacebuilding Programme: evaluability assessment approach

15

The number of days covers both consultants, or the consulting team

71

Appendix B: PEACC Proposal Results and M&E, Reporting and Research 5. RESULTS 5.1 Results Framework

The programme functions at multiple levels – global, regional, national, sub-national and community levels – to strengthen education systems and policies that support peacebuilding. It works on two fronts – one, to support the education sector in ways that are conflict sensitive and build the capacity of institutions to strengthen the foundations of peace; and two, to increase recognition and support for education within broader peacebuilding processes, plans and policies. As an emerging area of study and practice and as an innovative programme of significant scope and duration, evidence building and knowledge sharing is essential. How education contributes to peace and conflict resolution needs to be shared at all levels and amongst all stakeholders, both in peacebuilding and education fields. The following results framework was developed through a collaborative process within UNICEF and with the support of other UN agencies. It includes both outcome and output level results contributing towards the overall goal and strategic result of the programme. The vision, strategic result and outcomes, will be similar across all countries, whereas the outputs may vary according to contextual relevancy and programme learning and course corrections over time. This is an important point, as it will be important for the relevance and eventual sustainability of the programme that it is well grounded in the local social, cultural, institutional, and economic context. For this reason, a programme design workshop will be arranged in the first year of the project to develop the programme design and the results framework, as well as the associated monitoring and evaluation plan. Further refinements may be required in the light of the baseline study and evaluability assessment, also scheduled for the first year. The overall aim is to develop a coherent results framework, allowing results to be tracked over time and across countries and regions, while responding to local needs and circumstances. Please find the results framework on the next page. One of the key lessons learned from the EEPCT programme is the importance of developing a manageable number of relevant, measurable indicators, given the challenges of accessing data in fragile, conflict-affected and post-conflict contexts. The other lesson learned is the importance of ensuring buy in and commitment to the results framework at the Country Office level. Based on these findings, the programme has identified six programme level indicators that will be adopted by all countries during the programme design workshop. The strategic result indicator and the five outcome level indicators will be tracked by all Country Offices over the life of the programme and will be rolled up at the programme level to demonstrate overall programme progress. Where feasible, indicators will be disaggregated by gender, and outcome and strategic result indicators will be aggregated at global level. (See monitoring section for greater detail). The six indicators are as follows: 72

Strategic Result indicator: •

# of improved policies, strategies and approaches adopted and implemented in education for peacebuilding in conflict-affected contexts.

Outcome level indicators: • • • • •

# of peacebuilding and conflict reduction policies, analyses and implementation with education integrated % change in sample of children and adult community member perceptions of their own ability to prevent, reduce and cope with conflict and promote peace # of countries with institutions having measurably increased capacity to supply conflict sensitive and peace education # of children reached by quality, relevant education that contributes to peace (Male/Female) # of policies at country level proposed and/or passed that are informed by programme supported evidence base

73

Vision Strengthened resilience, social cohesion and human security in conflict-affected contexts achieved. Strategic Result Strengthened policies and practices in education for peacebuilding in conflict-affected contexts. # of improved policies, strategies and approaches adopted and implemented in education for peacebuilding in conflict-affected contexts Outcome 1 Outcome 2 - Institution Outcome 3 - Individual Outcome 4 - Access to Outcome 5 – Knowledge, Peacebuilding and Building Capacity Development Conflict Sensitive Education Evidence and Advocacy Education Integrated Increased institutional Increased capacity of Increased access for children Adequate generation and Increased inclusion of (including UNICEF) children, parents, teachers to quality, relevant education use of evidence and education into capacities to supply and community members to that contributes to peace, knowledge in policies and peacebuilding and conflict sensitive and prevent, reduce and cope including education delivered programming on linkages conflict reduction peace education with conflict and promote as a peace dividend between education, conflict policies, analyses and peace. and peacebuilding. Indicator: # of countries Indicator: # of children (M/F) implementation and vice with institutions having Indicator: % change in a reached by quality, relevant Indicator: # of relevant versa. measurably increased perceptions among education that contributes to policies and programmes # of peacebuilding and capacity to supply conflict representative samples of peace adopted at country level conflict reduction sensitive and peace children and adult that are informed by policies, analyses and education community members of programme supported implementation with their own ability to prevent, evidence base education integrated reduce and cope with conflict and promote peace Output 1.1 Output 2.1 Output 3.1 Output 4.1 Output 5.1 Increased inclusion of Increased number of dutyIncreased number of Increased number of Increased advocacy for conflict analyses and bearers in trained in children, parents, teachers programme supported learning education funding in conflict sensitive delivery of conflict and community members environments with quality and conflict-affected contexts. Indicator: % of national planning and delivery in sensitive education at trained on themes of relevant education. Indicator: # and % of budget allocated to education plans and community, district and peaceful conflict resolution, programme supported learning education annually policies national levels. tolerance and social Indicator: # of education Indicator: # of dutyenv’ts that meet UNICEF Child cohesion. policies that are based bearers trained (M/F) in Indicator: # of duty-bearers Friendly School criteria (locally on conflict analysis delivery of conflict trained (M/F) on themes of adapted to context) out of total sensitive education peaceful conflict mitigation, learning environments reached services tolerance and social by programme cohesion. Output 1.2 Output 2.2 Output 3.2 Output 4.2 Output 5.2 Increased fora for Increased number of Increased number of dutyIncreased number of children Increased evidence base dialogue between Ministry of Education uses bearers trained on formerly recruited or used by for education’s role in peacebuilders and of EMIS data to inform psychosocial support. armed groups and other peacebuilding Indicator: # of duty-bearers Indicator: # of research and educationists at national education sector policies, children affected by conflict into (M/F) trained on case studies completed level plans and programmes education Indicator: # of round Indicator: # of education psychosocial support Indicator: # of children formerly table events including sector policies, plans, recruited or used by armed the topic of programmes that cite groups and other children peacebuilding and EMIS data affected by conflict (M/F) education reached by reintegration programming that includes school integration Output 1.3 Output 2.3 Output 3.3 Output 4.3 Output 5.3 Increased education Increased number of MoE Increased number of Increased number of countries Increased dissemination of representation in staff trained how to do teachers trained on with enhanced monitoring, evidence and knowledge peacebuilding dialogue conflict analysis for equitable education and reporting and response on the relationship between and planning at the informed education peaceful classroom mechanisms to address attacks education and conflict national level. planning and delivery. management, and disaster on schools, including attacks or prevention and reduction Indicator: # and % of Indicator: # of MoE staff risk reduction threats to personnel and and peace. meetings that included (M/F) that report using Indicator: # of teachers Indicator: # of students education representation conflict analysis to inform trained (M/F) in equitable Indicator: # of countries with presentations on out of total # of meetings education planning and education and peaceful enhanced monitoring, reporting programme lessons held annually delivery classroom management and responses to attacks on learned and shared and disaster risk reduction school Output 1.4 Output 2.4 Output 3.4 Output 4.4 Output 5.4 Increased number of Increased number of Increased number of Increased support to learning Increased engagement of national education UNICEF staff trained how children participating in environment construction and national and regional policies that support to do conflict analysis for peace education rehabilitation research institutes in global Indicator: # of learning equitable and conflict informed education programming networks addressing Indicator: # of children environments supported or sensitive education for planning and delivery. peacebuilding and/or Indicator: # of UNICEF (M/F) reached by peace provided marginalized children education staff (M/F) trained on education programming Indicator: # of networks and youth Indicator: # of conflict analysis for with active engagement of institutional systems and education planning and nat’l/reg’l research structures that added delivery institutes and researchers education

74

5.2 Indicative Activities by Output Outcome 1: Increased inclusion of education into peacebuilding and conflict reduction policies, analyses and implementation. Outputs Indicative Activities 1.1 Increased inclusion of conflict • Hosting and attending advocacy events, briefs and meetings for the inclusion of conflict analyses and conflict sensitive analysis into education plans planning and delivery in education • Capacity building of MoE personnel and UNICEF education staff on conflict analysis plans (Gender analysis be included • Provision of technical support to Ministry staff and other personnel engaged in conflict in all priority plans and budgets, sensitive education planning and delivery specifying intended beneficiaries, • Cross-cluster coordination and participation in needs assessments and analyses indicators and benchmarks for including Education Cluster Joint Needs Assessment achieving gender equality aims) • Piloting ways to measure conflict risk reduction and peace in the education sector • Ensure that the priorities established in the peacebuilding frameworks and strategies reflect best practices for achieving gender equity and supporting women’s participation in peacebuilding; engage appropriate partners 1.2 Increased fora for dialogue • Developing communication for development strategies on the benefits of cooperation between peacebuilders and of educationalists and peacebuilders educationalists at national level • Partnership with UN Peacebuilding Support Office, other UN agencies, and external partners and institutes • Organize global, regional and national events to promote dialogue and engagement • Convene civil society dialogues focused on gender and peacebuilding issues and help bridge the disconnect between indigenous women’s organizations and the operational activities of international organizations 1.3 Increased education • Participation of education sector staff in national level peacebuilding dialogue and representation in peacebuilding planning dialogue and planning at the national • Sharing education sector and programme level conflict analyses with key stakeholders. level 1.4 Increased number of national • Provision of evidence based advocacy messages education policies that support • Support of Ministry of Education advocacy strategy for education in conflict-affected equitable and conflict sensitive contexts. education for marginalized children • Develop system wide policy coherence on girls, education and peacebuilding. and youth. Outcome 2: Increased institutional capacities to supply conflict sensitive and peace education. Outputs Indicative Activities 2.1 Increased number of duty• Training of teachers and school administrators in conflict analysis and school bearers trained in delivery of conflict management (attention to women teachers) sensitive education at community, • Support for development teacher training curriculum and learning modules on conflict district and national levels. sensitive education (will take account of gender bias) • Working with families to ensure that they do not prioritize boys education at the expense of girl children 2.2 Increased number of Ministry of • Training of school and Ministry personnel in education management information Education uses of EMIS data to systems, data use and analysis inform education sector policies, • Technical support to Ministry of Education on data use and analysis plans and programmes. • Promoting the use of gender disaggregated data and equitable gender ratios 2.3 Increased number of Ministry of • Capacity building of Ministry of Education staff on conflict analysis Education staff trained how to do • Technical support to Ministry of Education staff to use conflict analysis in education conflict analysis for informed planning and delivery education planning and delivery. 2.4 Increased number of UNICEF • Development of UNICEF guidelines on conflict analysis for education including gender staff trained how to do conflict analysis analysis for informed education • Development of Education module for UNICEF guidelines on conflict analysis planning and delivery. • Training and follow up at all levels of UNICEF Outcome 3: Increased capacity of children, parents, teachers and community members to prevent, reduce and cope with conflict and promote peace. Outputs Indicative Activities 3.1 Increased number of children, • Formation of and support to child and youth clubs that include a focus on peace and parents, teachers and community tolerance members trained on themes of • Training of parents, teachers and community members on themes on, peaceful conflict peaceful conflict resolution, resolution, tolerance and social cohesion tolerance and social cohesion. • Support for the formation and strengthening of social networks 3.2 Increased number of children • Training of teachers parents and community members on themes of psychosocial accessing psychosocial support support • Establishment of referral networks for traumatized children • Initiation of psychosocial support programs at school and community levels • Measures to address sexual and gender based violence

75

3.3 Increased number of teachers trained on equitable education and peaceful classroom management and disaster risk reduction

Training of teachers on themes of gender responsive equitable education and peaceful classroom management • As feasible, initiatives for schools to have equitable male and female staff ratios • Development of complementary teaching and learning modules to promote social cohesion and gender equality. 3.4 Increased number of children • Development of peace education curriculum participating in peace education • Training of teachers to deliver peace education programmes programming • Development of teaching and learning materials to support peace education • Use of media and ICT to support peace education and tolerance Outcome 4: Increased access for children to quality, relevant education that contributes to peace. Outputs Indicative Activities 4.1 Increased number of programme • Formation, support and capacity building of PTAs/SMCs and community supported learning environments with protection mechanisms quality and relevant education. • School reporting of child protection concerns • education delivered as a peace dividend • Schools as Zones of Peace • Schools are made safe and accessible for girls • Provision of resources for students and/or teachers in terms of food for students; temporary allocation of land to schools or teachers; • Addressing issues of teacher remuneration • Promoting rights based, democratic values and deliberations 4.2 Increased number of children • Community based, gender sensitive, reintegration programming formerly recruited or used by armed • Gender responsive skills, livelihoods and vocational training groups, and other conflict affected • Accelerated learning programmes children, into formal/non-formal, • Catch-up activities primary/secondary education • 4.3 Increased number of countries with • Capacity building on monitoring and reporting of attacks on schools enhanced monitoring, reporting and • Capacity building on prevention of and response to attacks on schools response mechanisms to address • Development of action plans with parties of the conflict attacks on schools, including attacks or • Advocacy for engagement in the Monitoring and Reporting Mechanism, threats on personnel and students, in established by UN Security Council resolutions 1612, 1882 and recently 1998 to order to increase the security and address attacks on schools and its personnel protection of schools 4.4 Increased support to learning • Advocacy for safer learning environment construction environment construction and • Construction of learning environments rehabilitation • Repair and reconstruction of learning environments • Provision of child and gender sensitive WASH facilities Outcome 5: Adequate generation and use of evidence and knowledge in policies and programming on linkages between education, conflict and peacebuilding. Outputs Indicative Activities 5.1 Increased advocacy for • Provision of evidence based advocacy messages education funding in conflict-affected • Support of Ministry of Education advocacy strategy for education in conflict-affected contexts contexts • Developing country level advocacy strategies • Participation in conferences of communities of practice 5.2 Increased evidence base for • Conducting a mid-term progress review and end of programme evaluation education’s role in peacebuilding • Organizing and undertaking thematic studies on innovative aspects of programming • Developing a research strategy and undertaking research studies • Collaborating with knowledge partners and institutes • Including research and interdisciplinary collaboration on critical gaps and strategic priorities relevant to women and peacebuilding 5.3 Increased dissemination of • Provision of information to or participation in external studies e.g. UNESCO Global evidence and knowledge on the Monitoring Report or case studies by INEE, FTI or other relationship between education and • Lessons learned seminars with communities of practice conflict prevention and reduction • Participation in professional research e.g. UKFIET, CIES or other conference and peace. 5.4 Increased engagement of • Partnership and support for regional education and/or peace institutes national and regional research • Organizing events that include regional and national education and/or peace institutes institutes in global networks and • Information sharing with regional/national level research institutes institutions addressing peacebuilding and/or education. •

76

6. MONITORING, EVALUATION, REPORTING & RESEARCH

The potential for education to address the root causes of conflict and to support increased human, economic, political and social security is a growing area of study. There are a number of hypotheses on the link between education and peace but little documented evidence. Building an evidence-base for the contribution of education to peacebuilding (social, economic, political, security) as well as addressing the root causes of conflict is an essential element of the programme. Monitoring, evaluation and research each contribute to building the evidence base in different ways. The following is a summary of plans in each area. Section 6.5 sets out the proposed activity schedule. 6.1

Monitoring

To track progress towards programme outcomes and strategic result, a strong monitoring system will be established in the first year. Monitoring activities will be led by Country Office education staff and supported by the M&E position in HQ Programme Management Team and regional staff with expertise in peacebuilding and education. This process will include the following steps: 1. Conflict analysis implemented. The country level conflict analyses will provide important information for developing the programme, including the design of the baseline. 2. Programme design workshop. Country office will engage in a programme design workshop framed by the results framework above. The suggested indicators will be verified and reviewed for feasibility in the respective country context. A programme operational plan will be developed. The programme operational plan will provide further details on definitions, sources and means of verification of indicators. It will also support Country and Regional offices to align programme results with MTSP results and organizational goals. 3. Evaluability Study. Once the conflict analysis and programme design have verified the indicator selection as relevant and feasible, a light touch evaluability study will review the selected indicators for technical soundness. The evaluability assessment will check that the results framework is coherent, allows for comparisons between countries as well as global aggregation, is well-grounded, appropriate to the local context, and draws on reliable information sources. It will also assess the baseline design. 4. Baseline Study. The baseline study will develop the programme indicators and identify appropriately contextualized targets and milestones for the achievements of results. The study will also identify data sources and means of verification, and it will gather the initial data set required for monitoring and reporting purposes. An external agency will be contracted to support the baseline study. The baseline study may be done in two phases: 1. design, which allows for input from the evaluability study; and 2. the implementation phase. The final elaborated results framework will be shared with the Government of Netherlands. 5. Monitoring results will be analyzed and compiled into annual reports by the Country Offices for submission to the Programme Management Team to guide programme management and for external reporting, using standardized reporting formats. 77

6. The routine annual Regional Education Conferences will incorporate a session on the programme to bring together the specific offices to share challenges, results and achieved and lessons learned. The Regional Office annual report will capture this shared progress and learning. 7. Annual training sessions on the results framework and indicator tracking will be led by the M&E Manager. This is to ensure continued awareness of and capacity for the annual monitoring system, particularly important given turnover of staff and programme counterparts. These sessions will also help maintain commitment to monitoring over the life of project. 6.2 Evaluation

Evaluation activities will build on the monitoring system described above, but will undertake deeper analysis of programme relevance, efficiency and effectiveness. Evaluation activities will include a mid-term progress review and a final evaluation. The Education Office will lead a comprehensive mid-term progress review to be undertaken in the final quarter of 2013, the second year of the programme, with a view to informing any midcourse corrections required. The mid-term review will consist of document reviews, interviews and surveys as well as country level case studies. The mid-term review report and management response will be shared with GON. Generally, UNICEF’s Evaluation Office will provide advice on M&E issues throughout the life of the programme, particularly in relation to the baseline study. Specifically, the Evaluation Office will also commission and manage an independent final evaluation. The independent evaluation will be a comprehensive study undertaken in the final year of the programme with a view to assessing not only the relevance, efficiency and effectiveness of the programme, but also the likely sustainability and impact of programme achievements. As well as reviewing the relevant documentation and conducting stakeholder interviews, the evaluation will undertake both country-level reviews and thematic studies to build a comprehensive assessment of the programme’s contribution to human security and peacebuilding. The independent evaluation will therefore have a wider scope than the mid-term review, and it is expected that the evaluation results would be disseminated widely among individuals and institutions concerned with human security and peacebuilding, as well as across UNICEF. A formal management response would be prepared and published along with the report. 6.3 Thematic Studies

Strengthening the evidence base for education’s contribution to peacebuilding is necessary in order to build greater recognition and support for education’s contribution to broader peacebuilding processes. The research strategy will be informed by the Peacebuilding Literature Review and the agendas of key actors, such as INEE, FTI partnership and other UN partners. A number of thematic studies will be conducted to document pilot programmes to deliver conflict sensitive education, skill training programmes for youth or innovative financing mechanisms, as well as processes such as the integration of education into peacebuilding processes, conflict analysis approach, or gender, education, and peacebuilding. Thematic studies will be managed 78

by the Programme Management Team with support from the Education Section and decided in consultation with the Advisory Group. 6.4 Progress and Results Reporting

The annual reporting cycle to the GON has three key parts – country and partner reports, a consolidated annual report and an annual programme review meeting. The annual reporting over the life of project will allow tracking of interim milestones and progress towards the outcomes and strategic result. The country and consolidated reports will include summary statements on the number of countries that have achieved key milestones. Efforts will be made to shorten the reporting cycle to ensure the Consolidated Report and Programme Review meeting occur within the first six months of the subsequent reporting year. A detailed reporting format based on the results framework will be developed. 6.4.1

Country and Partner Reports will be provided to the GON. Country Office reports will follow a standardized reporting format. A common reporting format will also be developed for key strategic partners contributing to programme objectives and supported under the programme. Partner reports will be submitted upon request.

6.4.2

Consolidated Annual Reports will be provided to the donor each year. These reports will be drawn from annual Country Office, Regional Office, Supporting Division and Strategic Partner reports. To move beyond activity based reporting and to support greater analytical content in reporting progress review and lessons learned sessions will be held during regional and global education section meetings.

6.4.3

Annual Programme Review meetings will be held with the donor to share key findings of the Consolidated Report, to receive donor feedback on programme coming year.

6.4.4

Financial reports Consistent with UNICEF’s overall financial reporting process under IPSAS and the new corporate VISION statement, UNICEF will report annually on expenditure levels.

6.5 Calendar The first year will be critical and challenging; critical because, as was learned in EEPCT, crucial to a programme identity and strong M&E is a programme design that builds on a situation analysis (in this programme a conflict analysis), and challenging because--as with all programmes--initial start-up and staffing take time to do well, especially in the unpredictable context of conflict affected contexts. In early 2012, consideration will be given to preliminary allocations to support the peace dividends through education in conflict-emerging countries. For example, allocation to Liberia for continued support of ministry, text-book provision, rehabilitation, and accelerated learning programmes. However, UNICEF has set an ambitious schedule for the first year. This includes: conflict analyses and programme design workshops in the first 2 quarters, followed by roll out of the 79

programme and monitoring system and Evaluability study of the selected indicators. The final quarter of the first year will be dedicated to conducting the baseline and designing the programme research strategy. Separate from programme monitoring system, performance monitoring of programme roll out will be given particular attention by the global programme management team and regional offices to ensure progress towards objectives is achieved, and risks are addressed pro-actively. Strengthening the performance monitoring of roll out in the first year will be the linkage of allocation disbursement to high level performance.

Task Conflict Analyses Programme Design

2012 Q1 Q2 X

Q3

Q4

Q3

Q4

2014 Q1 Q2

X

Baseline Study

X

Evaluability Assessment

X

Research Strategy

X X

Mid-term review

X

X X

Q4

2015 Q1 Q2

Q3

Q4

X

X

X

X

X

X

X

Independent Evaluation Annual Report

Q3

X

Roll out

Thematic Studies

2013 Q1 Q2

X

X

80

Appendix C: Tools for Evaluability Assessment Checklist for Programme Evaluability Aspect of Evaluability

Key Questions

Programme Design

Does the programme clearly define the problem it is intended to resolve?

Y/N

Have the stakeholders and beneficiaries of the programme been clearly identified and have they been involved in planning? Does the programme have a clear theory of change? Is the results framework consistent with the theory of change? Are the outcomes logically interrelated and do they flow logically from the outputs? Are output objectives measurable and realistic within the time frame of the programme? Do proposed activities logically lead to the intended outputs? Technical Aspects

Are the outcome and output indicators SMART (Specific, Measurable, Achievable, Relevant, Time bound)? Are the indicators universally applicable and can they be aggregated at the global level? Have the information needs and sources been identified for each of the indicators? Is there a baseline of information for each indicator? Are there gaps in baseline information or in sources? How can the gaps in sources and baselines be filled? Are there areas that require more detailed studies? Can the information be disaggregated by gender, ethnicity or other relevant factors?

Management Aspects

Does the programme have a adequate knowledge management and monitoring evaluation system? Are there adequately trained monitoring and evaluation staff at global and national level? What are the information requirements of donors and other key stakeholders and are these included in the monitoring system? What are the costs of monitoring, evaluation and related studies and surveys, and is there an adequate budget for this? Are there structures and procedures for coordination among stakeholders and how effective are they? Are governments and other stakeholders supportive of the programme and open to evaluation? What security, access, or ethical considerations may limit the evaluability of the programme?

81

Participation – Empowerment Index It is important to have broad and representative participation in evaluations and evaluability analysis. The Participation and Empowerment Index may be a useful tool to help assure a systematic analysis of the level and quality of participation in the programme. The index combines measures of three aspects of participation: extent (who participates), function (in what do they participate), and intensity (how do they participate). It can be used in several ways, but most frequently it starts with selecting who participates. To give a quantitative result, the ranking number for the item in each column is multiplied by those in the other columns. For example, if community leaders are consulted in the planning of the programme the score would be: 1x4x2=8 whereas if women initiated and planned the programme it would be 4x4x4=32. Whether or not it is used to quantify participation, it will help identify what aspect of participation may be lacking or weak.

Participation - Empowerment Index16

Extent (Who)

Function (In What)

Intensity (How)

5 Children/Youth

5 Management

5 Control

4 Women

4 Planning

4 Initiate

3 All households

3 Implementation

3 Decision making

2 Interest groups

2 Maintenance

2 Consultation

1 Leaders/Officials

1 Distribution/Use

1 Informed

In the case of participants in conflict analysis, the interest groups may be representatives of the parties to the previous conflicts or different ethnic, language or religious groups. However, in the education-specific conflict analysis the interest groups may be teachers and other education professionals.

16

Clarence Shubert, “Effects of Community Participation on the Use and Mobilization of Resources in Development Projects and Programmes”, published in Khalid Shams, Community Participation in Asia, APDC, Kuala Lumpur, 1989

82

Appendix D: Conceptual Model for Education Interventions and Peacebuilding Figure 1: Macro level Conceptual Model of the Relationship Between Education Interventions and Peacebuilding At the macro level, education is just one of many interventions for peacebuilding (PB). Both conflict and peace drivers operate through a nested conflict systems model to influence issues, relationships, subsystems and systems. PB interventions attempt to change dynamics within the nested conflict model. Successful PB interventions lead to strategic peace goals, and ineffective interventions may lead to conflict escalation. Various models exist for drivers, PB interventions and strategic peace goals, some of which are identified. Linkages between concepts require refinement.

Conflict Drivers

Strategic Peace Goals

Poverty, Oppression Lack of resources Unmet human needs Human rights violations Social structures

• Resilience, social cohesion and human security (UNICEF PEACC 2012) • World community with social justice and cosmopolitan governing structures (Ramsbotham et al 2011) • Sustainable, peaceful relationships (Lederach et al 1997)

Peace Drivers Tolerance, Equity Conflict resolution Justice systems Participatory systems

Nested Conflict Model (Lederach 1997)

Conflict Escalation Usually unintended, sometimes education outcomes escalate violent conflict or solidify oppression by powerful groups.

Peacebuilding Interventions • • • • • • •

Education capacity building (UNICEF PEACC 2012) Security – disarmament, demobilization and reintegration, etc. Government – constitutions, elections, representative governance, taxes, etc. Economy – building official economy, employment, service provision, etc. Social and civil society – managing conflict, supporting minorities, etc. Law and order – policing, justice, human rights. (Ramsbotham et al 2011) Strategic capacity and relationship building (Lederach 1997)

700 12th Street, NW | Suite 700, Washington, DC 20005 | t. +1.202.351.6826 | f. +1.888.755.8925 | [email protected] www.konterragroup.net

Figure 2: Meso level Conceptual Model of the Relationship Between Education Interventions and Peacebuilding At the meso level, key concepts and relationships of the UNICEF Peacebuilding, Education and Advocacy (PEA) Programme education intervention are conceptualized. Conflict and peace drivers are impacted by PEA outcomes 1, 2 and 5, which influence outcomes 3 and 4 that more directly impact children. Outcomes 3 and 4 lead to child protection, individual conflict resolution skill building, and educational system changes. These changes can be explained by human development models, such as psychosocial development, human capital and agency, which should lead to long term goals, such as positive peace or liberation. Linkages or relationships between concepts require further refinement.

Conflict Drivers Communal/ethnic conflict Resource conflict Ideological conflict Power imbalances Inequality

UNICEF education capacity building (PEA outcome 4) O4. Increase access to quality, relevant, conflict sensitive education that contributes to peace

Positive psychosocial and cognitive development of children (Erikson, Piaget, Vygotsky, etc.) Child protection and community/ social stability

UNICEF education capacity building (PEA outcomes 1, 2, 5) O1. Increase inclusion of education into PB and conflict reduction policies, analyses and implementation O2. Increase institutional capacities to supply conflict sensitive and peace education O5. Contribute to the generation and use of evidence and knowledge in policies and programming related to education, conflict and peacebuilding

Traditional peace mechanisms Cultures of tolerance/ diversity Conflict resolution mechanisms

Psychosocial recovery, stability, normalcy, hope. Values and skills for peaceful relationships (Sommers 2002)

Freedom from want and fear. Life of dignity (Annan 2005)

Human capital and social capital models

Liberation, emancipation and empowerment (Freire 1970; Adams, Bell and Griffin 2007)

Children and adults able to manage conflict peacefully

UNICEF education capacity building (PEA outcome 3)

Peace Drivers

Positive peace, social cohesion, conflict transformation (UNICEF PEA 2011; Galtung)

O3. Increase capacity of children parents, teachers, etc. to prevent, reduce and cope with conflict and promote peace

Educational systems are equitable, inclusive, and empowering

Self-efficacy, human agency, empowerment models (Bandura 1982, Emirbayer & Mische 1998)

Elimination or mitigation of forms of oppression (Freire 1970; Adams, Bell and Griffin 2007)

84

Figure 3: Micro level Conceptual Model – Continuum of Education Interventions for Interpersonal and Intergroup Peace Some education interventions have the direct goal of improving peaceful interpersonal and/or intergroup relationships, which is the principal characteristic of peaceful societies. This directly responds to growing literature that most conflicts are driven by ethnic and social conflict. These interventions often promote peaceful children’s and adults’ attitudes and behavior for cooperative interpersonal relationships and responsible civic actions within society. Education interventions which seek to change attitudes and behaviors must be supported by improved child protection and fundamental education system changes that address systemic exclusion, inequality and oppression. It is necessary to measure children’s knowledge, skills and attitudes for peace, both pre- and post-intervention. While some research has measured individual and social changes from these types of interventions, further research is needed to determine the effectiveness of individual interventions, and to determine which conflict drivers are most impacted by each type of intervention.

Teacher training on peace

Reflective or active learning models

Peace camps

Life skills curriculum

Civic education

Least direct impact on peaceful coexistence

School based curriculum reform

Most direct impact on peaceful coexistence

Child Friendly Schools, Go to School

Art, music and literature camps

Tolerance and diversity

Conflict resolution education

i

The World Bank (2011). World Development Report 2011: Conflict, Security and Development, Washington: The World Bank, p 2 Ibid, p 5 iii WDR 2011 iv Smith, A. (2011) The Influence of Education on Conflict and Peace Building. Background paper prepared for the Education for All Global Monitoring Report 2011: The hidden crisis: Armed conflict and education. UNESCO: Paris v UN General Assembly, Report of the Secretary-General on peacebuilding in the immediate aftermath of conflict. 11 June 2009, A/63/881-S/2009/304. ii

85