Evaluating the impact of multi-intervention development projects: The case of Ethiopia’s community-based integrated natural resources management project

Abstract

This paper provides a quantitative impact assessment of the community-based integrated natural resources management project (CBINReMP) in the Lake Tana region in Ethiopia during 2011-2019. By promoting greater community participation, the CBINReMP provided support to watershed communities for the restoration of degraded soils and water sources, rehabilitation of forests, as well as in obtaining access to secure land titles and practices for climate change adaptation. The project further provided support towards diversification of incomes in off-farm activities and incentives for women’s empowerment and youth employment. This way the project aimed to support rural livelihoods through improvements in household incomes, dietary diversity, agricultural productivity, and resilience to climatic shocks, among other livelihood objectives.
To assess the project’s impacts, the study had to deal with numerous methodological complications owing to as the project’s nature and design. The lack of a proper baseline survey, incomplete information about targeted watershed communities and often lack of clear distinction lines between the project’s interventions and support provided to communities through other mechanisms made it hard to identify the true impact of the CBINReMP. Four additional challenges had to be faced: possible selection biases because of non-random placement (targeting) of the project; self-selection of beneficiaries into receiving the project; possible spatial spill-over effects of project benefits to non-treatment communities, and the project’s phased rollout. A propensity-score matching procedure was adopted to assess the CBINReMP’s impacts by comparing treatment (beneficiary) and control groups outcomes related to the livelihood indicators listed above. This paper discusses how the mentioned complications were addressed to provide a sound assessments of the project’s true impacts. While certain limitations remain, the key finding that can be drawn with confidence is that the CBINReMP had only very limited, quantitatively verifiable impact on rural livelihoods. It seems to have contributed to higher household incomes and some greater dietary diversity, but only where the project managed greater community participation. However, even for those beneficiaries, livelihood conditions had not become significantly more productive, diversified, resilient, or sustainable than those of the comparison group. The paper ends with recommendations on how to avoid methodological obstacles through better design of the M&E framework for multi-intervention, community-based projects.