We leverage the multi-stressor nature of the COVID-19 generalized disruption as an opportunity to test the out-of-sample forecasting accuracy of both theory-based and data-driven vulnerability prediction models for the ex ante targeting of preventive interventions. Taking advantage of the World Bank multitopic surveys for Ethiopia and Nigeria, the two most populous African countries, our retrospective evaluation assesses the models’ ability to anticipate households and agrifood system actors experiencing food insecurity and income losses during the COVID-19 pandemic. The results are disappointing: we document that, despite considerable heterogeneity across data and methods, both models do not achieve satisfactory out-of-sample forecasting performances. Our findings are robust to the use of different data, estimation methods, and several heterogeneity analyses and sensitivity checks. This evidence calls for a refinement of current profiling methodologies and for interoperability efforts to close existing microdata gaps. Such efforts would enable policymakers to implement more effective early-warning systems of vulnerability hotspots and improve the cost-effectiveness of development interventions aimed at targeting groups vulnerable to future food crises.