Assessing rural transformations: piloting a qualitative impact protocol in Malawi and Ethiopia

Abstract

This paper is intended to contribute to research into how qualitative approaches can contribute to credible impact evaluation, particularly in complex contexts. It reports on substantive and methodological findings from four pilot studies of a protocol for qualitative impact evaluation of NGO sponsored rural development projects in Malawi and Ethiopia. Two targeted the value chains of specific cash crops, while two promoted climate change adaptation through support for a spectrum of livelihood diversification activities. The protocol was designed and tested through participatory action research with the aim of generating evidence in a credible, timely and cost-effective way both to confirm the causal theories underpinning project actions as well as to explore incidental sources of change and unanticipated effects. The paper describes the methodology, provides an overview of findings and critically reflects on implications for addressing problems of attribution, confirmation bias and generalizability. It suggests scope for further development and use of approaches to these based on self-reported attribution, partial blinding of respondents and opportunistic sampling nested within quantitative monitoring