Outcome mapping: a tool for monitoring and evaluation

E.A. Ouma, e.a.ouma@cgiar.org and G.A. Neba, george.akwahneba@iucn.org

IITAroutinely measures impact resulting from its R4D projects and programs. Photo by IITA.
IITA routinely measures impact resulting from its R4D projects and programs. Photo by IITA.

Many development practitioners are preoccupied with the identification and measurement of impact resulting from their research-for-development projects or programs. In many high-level meetings, the importance of results-based management that is goal-oriented and that emphasizes cause and effect of inputs, outputs, and impacts, has been emphasized and a large number of methodological guidelines have been developed.

One such guideline is the Logical Framework Approach (LFA). It is a hierarchical linear causal-effect chain presented at four levels (activities, outputs, outcomes, and impact). It is concrete and encourages the clear formulation of outcomes and goals/impact and the precise definition of quantifiable targets. Its major weakness is the attribution of cause and effect between the levels of outcome and impact (Jones 2006). In reality, this cannot be conclusively determined. Most impacts occur a long way downstream and may not be directly influenced by a single actor. In addition, the linear cause–effect thinking in LFA is a rather strong assumption and has been criticized by many practitioners.

The weaknesses in the existing tools, particularly in the monitoring and evaluation of developmental impacts, motivated the International Development Research Centre to develop a different approach, referred to as outcome mapping.

Figure 1. Boundary partner's link to the program and the real world.
Figure 1. Boundary partner's link to the program and the real world.

Outcome mapping
Outcome mapping is a method for planning and assessing the social effects and internal performance of projects, programs, and organizations (Earl et al. 2001). It helps a project or program team to be specific about its targets, the changes it expects to see, and the strategies it employs, and as a result, to be more effective in terms of the results it achieves. Results are measured in terms of changes in the behaviors of people, groups, and organizations, also known as “boundary partners” (Fig. 1) with which a project/program works directly. The project/program works with the boundary partners to effect a change but it does not control them.

The changes are referred to as outcomes. In so doing, outcome mapping clears away many of the myths about measuring impact and focuses more on social changes within complex and dynamic partnerships. Once boundary partners have been identified, outcome mapping differentiates the levels of behavioral change which may be seen among the partner organizations—known as progress markers. These are grouped according to expected behaviors (early positive responses), desired behaviors (active engagement), and hoped-for behaviors (deep transformation in behavior) (Shaxson and Clench 2011). In the vocabulary of outcome mapping, these are behaviors we would ‘expect to see”, “like to see”, and “love to see” and they may be priorities for change or a time sequence of activities, or a mixture of both (Fig. 2).

Figure 2. Progress markers of a boundary partner. Source: Jones 2006.
Figure 2. Progress markers of a boundary partner. Source: Jones 2006.

Attribution and measurement of downstream results are dealt with through a more direct focus on transformations in the actions of the main actors. The outcomes enhance the possibilities of developmental impacts but the relationship is not necessarily a direct one of cause and effect. The outcomes can be logically linked to a project’s activities although they are not necessarily directly caused by them. Outcome mapping, therefore, focuses on the contribution of a project in the achievement of outcomes rather than claiming the achievement of developmental impacts.

The development of M&E tools (both qualitative and quantitative) for assessing outcomes and impact on commodity systems, including outcome mapping and participatory impact pathway, was identified as an output target for IITA’s Opportunities and Threats Program in 2011 (IITA 2009). This highlights the importance of developing tools not only for documenting technology adoption trends and impact but also those that monitor outcomes, providing stakeholders with timely information about their progress and achievements for systematic and collective learning, reflection, and corrective action.

A few R4D projects at IITA have employed outcome mapping or some of its elements in their M&E framework. For instance, the Consortium for Improving Agriculture-based Livelihood in Central Africa project, largely operating in the East and Central African highlands, follows the spirit of outcome mapping in its arrangements to scale out technology. The boundary partners, comprising international and national NGOs and farmers’ associations, articulate their goals, expectations, and contributions through informal or formal memoranda of agreement with the project. The project endeavors to meet the partners’ expectations through jointly planned activities to achieve the expected outcomes, which have prospects of producing sustainable impacts.
Opportunities for interactions between a boundary partner and the project and among the boundary partners are made available for collective learning, to evaluate progress towards the achievement of goals over time, and to identify corrective measures.

Other CGIAR centers, particularly the International Center for Tropical Agriculture (CIAT), International Livestock Research Institute (ILRI), and the World Agroforestry Centre, apply outcome mapping in their natural resource management and livestock projects.

Stages of outcome mapping and monitoring tools
The process is divided into three stages. The first, referred to as the intentional design phase, is largely a planning stage. This helps a project to establish a consensus on the macro-level changes it will help to bring about and to plan the strategies it will use. It is based on the principle of participation and purposefully includes those implementing the project in the design and data collection so as to encourage ownership and use of the findings. It involves articulation of the vision and mission of the project, the identification of the boundary partners, the outcome challenges, progress markers, and strategies to be employed for changing the behavior of boundary partners to better deliver the progress markers. Supportive strategies facilitate change, possibly by one partner providing information, capacity, or skills to others.

The second stage is outcome and performance monitoring. It provides a framework for an ongoing monitoring of the projects’ actions and the boundary partners’ progress toward the achievement of outcomes. It is largely based on a systematized self-assessment and uses monitoring tools for elements identified in the design stage. The tools include an outcome journal (for monitoring progress markers), a strategy journal (for monitoring the strategy maps) and a performance journal (for monitoring the organizational practices).

The third stage is evaluation planning. It helps the project to identify evaluation priorities and develop an evaluation plan (this targets priority areas for detailed evaluation studies).

Strengths and weaknesses
Outcome mapping provides a focus on institutional transformation that is often lacking in techniques which emphasize the delivery of outputs as an indicator of achievement. It aligns itself with the realities of development, often occurring in complex and open systems with multiple actors. The methodology ensures the clear formulation of responsibilities, roles, and progress markers for each project partner in addition to providing a framework and the tools for continuous monitoring. Measurable outcomes and clear milestones enhance ownership by the local actors and beneficiaries as well as the management of multiple accountabilities (project, beneficiaries, partners, and donors).

Outcome mapping’s one-dimensional focus on “changes in behavior”, although important to sustainable development, cannot be an end in itself for development. The behavioral changes should support improvements in situations at a higher level. There is a need to have clear hypotheses about the framework, tools, and indicators for impact at the level of development results (such as the MDGs). Roduner et al. (2008) have proposed a synthesized model combining the strengths of outcome mapping focusing on capacity building and the logical framework with its focus on development results. The synthesized model has, however, not yet been tested.

References
Earl S, T Smutylo, and F Carden. 2001. Outcome mapping: Building learning and reflection into development programs. IDRC, Ottawa, Canada.

Jones H. 2006. Making outcome mapping work. Evolving Experiences from Around the World. IDRC, Ottawa, Canada. http://bit.ly/19zzsH.

Roduner D, W Schläppi, and W Egli. 2008. Logical Framework Approach and Outcome Mapping: A Constructive Attempt of Synthesis. A Discussion Paper, ETH, Zurich, Switzerland.

Shaxson L, and B Clench. 2011. Outcome mapping and social frameworks: tools for designing, delivering and monitoring policy via distributed partnerships. Delta Partnership working paper No 1, www.deltapartnership.com.