Page 50 complementary suggestions
Impact is conceptually distinct from effectiveness (Hulme, 2000). While effectiveness deals with short and medium term, immediate and intermediate direct outputs, impacts designate broader outcomes, ultimate results, including indirect and/or long-term, intended or unexpected. Those ultimate results are more affected by exogenous factors. In the context of improving or financing FSN, effectiveness could be measured, for instance, by the quantity of food provided through humanitarian assistance in a context of emergency, with regards to the initial target. Impact analysis would assess the improvement in FSN status of the vulnerable groups targeted by this humanitarian assistance.
One of the main debates in the literature on impact analysis and development projects focuses on attribution, meaning how to attribute specific impacts to specific actions and decisions. How to identify the specific direct and indirect contributions of a MSP to FSN and differentiate it from the effects of other exogenous factors, beyond its control? Some authors have developed techniques to identify and measure attribution (see Box 12). Others have questioned the feasibility and even the relevance of measuring attribution (Pawson, 2013), preferring to use impact analysis to understand the complex pathways involving many actors and factors leading to impacts, and to improve development practice instead (Roche, 1999).
Box 12
It would be interesting either to complete the box with the methods developed to understand impact pathways and to document contribution or to make another box on contribution analysis
Different methods to analyze impact: attribution or contribution ?
Randomized control trials (RCTs):
(Banerjee and Duflo, 2011).
Participatory Impact pathways analysis (PIPA, which is more a method to document contribution than attribution)
(first used in January 2006 in Ghana with seven projects funded by the Challenge Program on Water and Food)
ImpresS method (Hainzelin 2017 ; http://www.cirad.fr/en/content/download/12046/141801/version/2/file/Persp42+Hainzelin+ENG.pdf,
Barret et al. 2017 ; https://doi.org/10.19182/agritrop/00005)
Impact pathway (Douthwaite B., Kuby T., van der Fliert E., Schulz S., 2003. Impact pathway evaluation: an approach for achieving and attributing impact in complex systems. Agricultural Systems, 78, 243–265.)
When evaluating the impact of MSPs for FSN, it might be important to clearly communicate which perspective is adopted: one more concerned with measuring the impact of MSP’s interventions or one more concerned with understanding the processes of change and how to improve MSP actions and activities. Put differently, there are both external changes to look for (more money raised; standards with more buy-in established) and also internal changes (such as trust among MSP participants rises; informal interaction among previously distant actors increases; etc.)
Jacques Pages