Trigger and System’s Change

 1. Background and context

The Disaster Relief Emergency Fund (DREF) of the International Federation of Red Cross and Red Crescent Societies (IFRC) provides start-up funding for the IFRC and National Societies (NS) to respond to large-scale disasters (the “loan facility”) or funds NS responses to small- and medium-scale disasters and health emergencies for which no international appeal will be launched or when support from other actors is not foreseen (the “grant facility”). In 2018, the DREF has been expanded to include a new funding window: ‘Forecast-based Action (FbA) by the DREF’ finances Early Action Protocols (EAP) developed by National Societies whose EAP has been approved by the validation committee. Financial allocations are made automatically by FbA by the DREF when a pre-agreed forecast trigger is met that indicates the potential for severe negative impacts on the most vulnerable population.

The Food and Agriculture Organization of the United Nations (FAO) hosts a similar platform. The agency has the Special Fund for Emergency and Rehabilitation Activities (SFERA), which can release money from its Anticipation Action window. At the country level, Early Warning Anticipatory Action (EWAA) systems are developed tailored to the local context. These systems enable FAO to monitor major risks and to act early to mitigate its effects on the agriculture sector and livelihoods through an operational tool, the EWAA plan.

In 2017, the Mongolian Red Cross Society (MRCS) developed an EAP for dzud as part of its Forecast-based Financing (FbF) project to help herder households cope with the anticipated impacts of Dzud, based on forecast information and the Dzud risk map. The EAP was activated in January 2020 and has delivered emergency cash and animal care kits to 1,000 of the most vulnerable herders across 8 provinces and 61 soums. Similarly FAO triggered their Anticipatory Action window in 2020 to mitigate the impact of dzud on vulnerable herder households. The actions matched MCRS, with unconditional cash transfers and livestock care kits to 450 households across 4 provinces and 22 soums.

FAO and the MRCS began working closely together in October 2017, when they started investigating what information would give the best detailed picture of the likelihood and severity of a dzud. Together, the Government, FAO and the MRCS are using the same dzud risk map as the basis for implementing their anticipatory actions.

 2. Objectives, scope and criteria of the evaluation and main questions

The purpose of this TOR is to engage an institution experienced in qualitative data analysis to carry out the assessment of two aspects of the EAP activation: the trigger mechanism and the effectiveness of ex-ante funding, as described under 2.a and 2.b. This TOR is also linked to a separate quantitative FbF beneficiary impact assessment survey that is being carried out under an overarching evaluation process. Findings from this parallel survey, should be worked into this study where appropriate.

a)   Trigger model

Triggers are based on historical impact data, FbA fund requirements for magnitude and return period, as well as uncertainty within the forecast and analysis of historical skill of the forecast to predict impact.. Because of their multivariate nature (and also because of the trigger’s inclusion of both geophysical and socio-economic variables) they present unique challenges in evaluation.  Criteria and questions to be covered by the evaluation:

1)     Appropriateness

• To what degree was the trigger designed as such due to national mandates? What were these factors?
• What were the temporal aspects of the trigger? This includes lead time and target period. How were these temporal aspects justified?
• Were the probabilities and risk levels defined in the trigger model appropriate?

2)     Effectiveness

• Did the activation (or other analyses) reveal any problems with the accuracy of the trigger model?
• Is there evidence of new or better products or data (impact data, new forecast or hydrological models, longer lead times, i.e. drought over a summer period making fodder less available for livestock and winter stores low?) in existence that could be used to improve the trigger model?

3)     Sustainability

• How did the collaboration with the University of Nagoya (?) help the NHMS develop forecasting products? What role could they or other nationally relevant institutions have in supporting the NHMS in trigger development?
• Did FbF contribute to an institutionalised relationship between MRCS and NHMS?
• What did the government do with the information they receive from the 2019/2020 dzud risk map?
• What was the governance structure of the collaboration between the non-government partners that initiated early action? Between government partners and non-government partners? If there can be a map or chart here it would be useful.
• What policy changes relevant to early warning early action or disaster risk reduction are on the horizon (e.g. a disaster law revision in the near future)? What are the opportunities to influence government policy relevant to early warning early action and DRR (e.g. co-writing a government endorsed policy brief?)

 b)   Change of the humanitarian system

 1)     Coherence

• What was the role of the government during the 2020 dzud? How did this interplay with MCRS and FAO’s interventions?
• Were there any other organizations that took early action (e.g. UNICEF) and how did these interventions differ from that of MRCS and FAO? Were these actions coordinated?
• To what extent did the intervention work with or contribute to existing disaster management procedures in the country?
• During the 2020 dzud was the targeting correct from FAO and MCRS? Did it align with government registries? What could be improved?
• What gender and social inclusion elements need to be considered for future activations?
• What worked well in terms of collaboration between the agencies? What can be improved?

2)     Appropriateness

• To what extent was the communication and collaboration between the ex-ante funds and the receiving party effective and appropriate to facilitate the rapid implementation of the EAP and FAO EWAA system?

3)     Effectiveness

• What were the enabling or hindering factors that facilitated or inhibited the efficient and effective performance of the fund in discharging its role following the activation?
• To what extent does the FbF/EWAA mechanism allow MRCS/FAO to act earlier? How does this affect government programming? facilitate anticipatory action earlier response?
• What have been some challenges of Anticipatory Action in Mongolia? Looking back on the experiences in 2017/2018 and 2020.
• How did COVID-19 affect the implementation of the system and what can be learned from this?

4)     Sustainability

• To what extent does the FbF/EWAA mechanism including the ex-ante funding allow MRCS and FAO to promote a shift from reaction to anticipation that goes beyond the FbF project?
• To what extent does the FbF/EWAA mechanism position MRCS and FAO to influence government policy towards a more anticipatory legislation? (i.e. is there a space to integrate this work into social protection models and programming);
• What other anticipatory actions could be looked into for vulnerable herders (or other vulnerable populations) in Mongolia? How can MRCS and FAO support this and the uptake with the government?

3. Deliverables and indicative work schedule

The following schedule of work and deliverables gives an indication of the suggested approach and level of effort though the evaluators may propose an alternative approach with justification:

The principal will provide comments on the inception report and the draft final report within 10 working days. The evaluator(s) will revise the inception and final report within 10 working days.

The final report will contain a short executive summary (no more than 1,000 words) and a main body of the report (no more than 6,000 words) covering a description of the evaluation methods and limitations, findings, conclusions, lessons learned and clear recommendations. Recommendations should be specific and feasible. The report should also contain appropriate appendices, including a copy of the ToR, cited resources or bibliography, a list of those interviewed, an overview/analysis of responses of members on the questions asked in the interviews and any other relevant materials. Details of the final report are outlined in the table below.

Suggested final report outline

4. Reporting, application and contractual arrangements

Reporting:

The main client of this evaluation is FAO. The evaluator(s) will report Resilience Programme Technical Advisor of the FAO country office for the day-to-day management of the evaluation and work under the overall supervision of of FAO Representative, technical guidance of Early Warning Early Action Specialist of the Agricultural Development Economics Division of the FAO HQ in Rome. S/he will also facilitate contact with the country and assist in scheduling meetings and interviews with the MRCS, IFRC, FAO EWAA teams and relevant stakeholders.

Contractual arrangements:

The Evaluator should take all reasonable steps to ensure that the evaluation is designed and conducted to respect and protect the rights and welfare of people and the communities of which they are members, and to ensure that the evaluation is technically accurate, reliable, and legitimate, conducted in a transparent and impartial manner, and contributes to organizational learning and accountability. Therefore, the evaluation team should adhere to the evaluation standards and specific, applicable practices outlined in the IFRC Evaluation Framework accompanying this TOR.

The IFRC Evaluation Standards are:

Utility: Evaluations must be useful and used.

Feasibility: Evaluations must be realistic, diplomatic, and managed in a sensible, cost effective manner.

Ethics & Legality: Evaluations must be conducted in an ethical and legal manner, with particular regard for the welfare of those involved in and affected by the evaluation.

Impartiality & Independence: Evaluations should be impartial, providing a comprehensive and unbiased assessment that takes into account the views of all stakeholders.

Transparency: Evaluation activities should reflect an attitude of openness and transparency.

Accuracy: Evaluations should be technical accurate, providing sufficient information about the data collection, analysis, and interpretation methods so that its worth or merit can be determined.

Participation: Stakeholders should be consulted and meaningfully involved in the evaluation process when feasible and appropriate.

Collaboration: Collaboration between key operating partners in the evaluation process improves the legitimacy and utility of the evaluation.

It is also expected that the review/consultation will respect the seven Fundamental Principles of the Red Cross and Red Crescent: 1) humanity, 2) impartiality, 3) neutrality, 4) independence, 5) voluntary service, 6) unity, and 7) universality. Further information can be obtained about these principles at: www.ifrc.org/what/values/principles/index.asp

All products arising from this review/consultation will be jointly owned by the IFRC and FAO. The consultant will not be allowed, without prior authorization in writing, to present any of the analytical results as his/her own work or to make use of the review/consultation results for private publication purposes.

5. Relevant literature, references

• Forecast-based Financing Manual: https://manual.forecast-based-financing.org/
• FbF M&E Guide (Climate Centre, 2018): https://goo.gl/eG7kw1
• Mongolia Early Action Protocol.
• Initial impact assessment - MRCS
• FAO | Mongolia: Impact of Early Warning Early Action http://www.fao.org/3/ca2181en/CA2181EN.pdf
• ODI | Anticipatory Action for livelihood protection | https://www.odi.org/sites/odi.org.uk/files/resource-documents/202006_odi_anticipatory_action_for_livelihood_protection_wp_final.pdf
• FAO, IFRC, MRCS | Mongolia: Anticipating the 2020 dzud | http://www.fao.org/3/ca9392en/ca9392en.pdf
• FAO | Acting early in Mongolia | https://www.youtube.com/watch?v=bLIwwswd7mA&t=109s
• FAO | Early action against dzud safeguards herders’ livelihoods in Mongolia | http://www.fao.org/3/ca2595en/CA2595EN.pdf
• How to assess the impact of a Drought Risk Financing facility: A guide (Start Network, 2017): https://startnetwork.org/resource/how-assess-impact-drought-risk-financing-facility-guide

Vacancy Announcement: http://www.fao.org/mongolia/news/detail-events/en/c/1305431/