Integrity Matters
We work on complex and challenging issues around the globe.
Integrity Matters
Beyond BASIC Episode 3: Evaluation findings – The future of Social Protection programming
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
It’s time for a second behind the scenes episode on the insights generated by evaluating a complex global research and technical assistance programme.
Beyond BASIC is an Integrity Matters podcast mini-series sharing learning and innovation from Integrity’s endline evaluation of the UK Foreign, Commonwealth and Development Office’s £23.5m Better Assistance in Crises (BASIC) programme.
Hosted by Niki Wood, Integrity’s Head of Monitoring, Evaluation and Learning, Episode 3: Evaluation findings: The future of Social Protection programming focuses on effectiveness, impact, efficiency and sustainability, and what this means for future programmes supporting social protection in crisis settings.
Three guests join Niki: Nick Maunder who led the evaluation; Jen Gordon, Policy and Programmes Advisor in the Social Protection team at the FCDO; and Consulting and Research Team Leader Cathy Shaw from DAI, one of the organisations involved in implementing BASIC.
Look out for the final instalment in the series, launching here next Thursday:
- 16 July – Behind the evaluation: Relationships, transparency and trust
Hello and welcome to Beyond Basic, a podcast series showcasing learning and innovation from evaluating the UK Foreign, Commonwealth and Development Office's £23.5 million Better Assistance in Crisis Programme, also known as BASIC. My name is Nikki Wood, and I am the head of Monitoring, Evaluation and Learning, or Mel for Short, at Integrity. I will be your host for this podcast mini-series where we will be talking about Integrity's endline evaluation of the BASIC program, exploring the methods used, diving into some of the key findings, and reflecting on the relationship between the evaluators and the program team. Today's episode is part two of our conversation about key findings, where we'll be discussing effectiveness, impact, and efficiency, and what this means for future programs supporting social protection in crisis settings. As a reminder, BASIC was a centrally managed FCDO program running from 2018 to date, with the goal of strengthening the use of social protection approaches in crisis settings, particularly in fragile and conflict-affected contexts. I'm joined again by three guests. Could I please ask you to introduce yourselves?
SPEAKER_01Hi, I'm Jen Gordon and I'm a policy and programs advisor in the social protection team at the SCDO. I was part of the team responsible for managing the BASIC program, including the incorporation of findings from the baseline and the midline evaluations into ongoing program delivery.
SPEAKER_00Hi, I'm Kathy Shaw from DAI, one of the organizations involved in implementing BASIC. I head up DAI's consulting and research team, and I was the programme director for STAR, the technical assistance component of BASIC.
SPEAKER_03And I'm Nick Maunder. I was the team leader of the independent evaluation for BASIC for the baseline, midline, and endline phases.
SPEAKER_02Thank you all so much. Let's dive straight in. In the previous episode, we heard that BASIC delivered a large volume of high-quality outputs. These responded to country and global needs across a wide range of areas, from improved policy and program design to conducting primary research, capacity building, strength and coordination, and advocacy to build political commitment. The obvious next question is, did they make a difference? Nick, what did the evaluation find on effectiveness?
SPEAKER_03Well, overall, we found strong evidence that BASIC contributed to meaningful outcomes, particularly at program, policy, and coordination levels. The program strengthened policies and programs in several countries. It improved coordination between donors and agencies and also enhanced practitioner capability. Collectively, this helped build global momentum around using social protection approaches in crises.
SPEAKER_02Great. Could you give us some specific examples?
SPEAKER_03Sure. Well, at country level, BASIC contributed to national policy processes in Nigeria and Ethiopia, and it established an active donor coordination platform in Yemen. The research work stream mentored Southern researchers in Ethiopia, Sudan, and Yemen, and the high-level panel on social protection in fragile and conflict-affected settings was a landmark achievement in building support for using social protection approaches. However, effectiveness varied by context. Outcomes were strongest where there was political commitment, institutional anchoring and financing. In some highly fragile settings where systems were weaker and political barriers higher, progress was understandably more constrained.
SPEAKER_02So effectiveness depended heavily on the enabling conditions.
SPEAKER_03Exactly. Basically catalyzed change, but it couldn't substitute for political will or longer-term financing. Starting a fire needs a spark, but without fuel and careful tending, you won't get a fire going.
SPEAKER_02Kathy, from an implementation perspective, what stands out to you about effectiveness?
SPEAKER_00One key lesson is that relatively small, targeted technical inputs could have outsized influence. For example, a well-timed piece of policy advice or coordination support could shape a national strategy or influence a major donor program. But sustained change required more than a single assignment. Where there was follow-up engagement, strong local partners, and clear ownership, we saw more durable progress. I see.
SPEAKER_01And what about from your perspective, Jen? For us, the Basic Programme gave us leverage. It really helped shape thinking, both internally and among our partners, about the role of social protection in crises. It also enhanced our ability to engage in global policy processes. For example, G20 discussions last year on development and disaster risk reduction. But we recognize that deeper reforms do depend on national politics and long-term partnerships. I think what the evaluation really reinforces is that technical assistance is powerful, but it works best when it's embedded within broader strategies. Absolutely makes sense.
SPEAKER_02Nick, can we dig a little deeper into what really made the difference between outputs that were simply delivered and those that actually influence decisions or programs?
SPEAKER_03Well, the evaluation highlighted the importance of sustained engagement. Where advisors or researchers worked with partners over time, building trust, navigating political contexts, and revisiting issues as conditions evolved, outcomes were much more likely. We also saw that timing mattered. When analysis or advice arrived at the moment decisions were being made, for example, during policy revisions or in the midst of a crisis response, the chances of influence increased significantly.
SPEAKER_01Yeah, and from our side, I think that relationships and credibility were absolutely central. Where star consultants or basic research partners had really strong local networks and were seen as trusted actors, we saw that their evidence was far more likely to enter policy discussions. But the evaluation also highlights a structural change around these outputs. Once delivered, they didn't always translate into policy or program change. At times, the ownership for taking them forward just wasn't as clear or as strong as perhaps it could have been.
SPEAKER_00I'd agree that delivering a high quality output is only part of the process. Starr tried to manage that risk by shaping assignments with uptake in mind, rather than treating them as standalone pieces of work to be delivered and then disseminated. A lot of the effort went into relationships, sequencing and layering support over time. An assignment might introduce evidence into the conversation, and then we'd come back later with more targeted advice when the moment was right. That also meant that some outputs hadn't been taken up by the end of the programme. Influence isn't linear and it's not an exact science. Sometimes it simply takes time. And with the best will in the world, contextual shifts can sometimes mean that strong advice or evidence goes unused. Our aim through STAR was to keep evidence circulating, stay close to partners, and be ready to step in with short, well-timed bursts of support when decisions were being made. The key lesson is that thinking about uptake and sustainability needs to be built in from the outset of all assignments.
SPEAKER_02Let's turn to impact, the broader, longer-term system-level changes that BASIC aspired to contribute to. Nick, what are your thoughts?
SPEAKER_03Yes, these changes are long-term, and we recognize that it was still early for the evaluation to find evidence of impacts. However, we did find plausible evidence that BASIC was contributing to more efficient, effective, and inclusive crisis assistance. Such as For example, BASIC promoted discussions on interoperability and data exchange in Palestine, Yemen, and Ukraine, and that had the potential to lead to significant efficiency gains. Early work in Jordan by BASIC contributed to improving the effectiveness of the government's flagship social assistance fund through improved targeting and improving accountability to beneficiaries. But at the same time, a recurring theme was that the political economy factors, not technical gaps, were often the critical barrier to expanding social protection in crisis. That's an important lesson for future programming. So while BASIC made a good case for the wider consideration of using social protection approaches, putting this into practice requires changes in incentives, financing, and political attitudes.
SPEAKER_02A key finding from the evaluation is that BASIC, particularly through its research and technical assistance, really helped advance debates around gender-responsive and inclusive social protection crises. Could you tell us a bit more about the findings on this, Nick?
SPEAKER_03Sure. From the evaluation perspective, the strongest contributions were in shaping analysis and debate. For example, BASIC research produced evidence on how social protection systems affect marginalized groups, highlighting legislative gaps, risks of digital exclusion, or the role of frontline workers and community organizations. And in several cases, that evidence strengthened the advocacy capacity of civil society organizations and influenced policy discussions. But translating these insights into formal policy reforms or program redesign often takes time, particularly in fragile or politically complex contexts.
SPEAKER_01Yes, and I think from my perspective, that finding isn't wholly surprising. We know that issues around gender equality and inclusion can often take time to be adopted. Evidence and analysis help shape the conversation first, which can then in turn be used to influence the way policymakers and partners think about the issues. Only after this would we expect concrete changes to appear in policy and programme design. And we believe that what the basic program provided was credible, context-specific evidence that could inform those debates and help keep gender and inclusion on the agenda in crisis settings.
SPEAKER_00I agree that this kind of influence is often incremental. In some cases, there were more direct operational changes, more directly attributable to basic, like strengthening disability inclusion in humanitarian cash programs in Ethiopia. But more often the impact was about shifting understanding and framing, which then creates space for future reforms.
SPEAKER_02Before we close this episode, let's talk about value for money. Cost effectiveness can be methodologically difficult to evaluate. But even so, BASIC demonstrated good overall value for money. Nick, could you speak a bit to that point?
SPEAKER_03Well at the assignment level, some relatively small investments influence substantial FCDO programs. For example, support to the design of an FCDO humanitarian program in Jordan costs slightly less than £38,000, yet it influenced a £95 million program. And from a program level perspective, it's plausible to conclude that the program investment was justifiable. Set against the billions spent on social protection in crises each year, BASIC would only need to influence a small number of changes to justify itself from a cost-effectiveness standpoint.
SPEAKER_00Kathy, how did that look from your side? Well, the flexibility of the model was important for efficiency. Being able to mobilize quickly and tailor support helped ensure resources were used effectively. At the same time, uncertainty around budgets and timelines can make planning more complex, particularly when working in fragile environments. And what about from your perspective, Jen?
SPEAKER_01We saw a strong value in how BASIC leveraged relatively modest resources to influence these larger programs and policy debates, as Nick mentioned earlier, using the example in Jordan. That catalytic effect is central to the rationale for programs like this.
SPEAKER_02Thank you all so much for such a rich discussion. The BASIC evaluation shows that technical assistance, research, and learning can catalyse important shifts in how crises are addressed. But also that context, politics, and long-term commitment are critical. That's all for today's episode of Beyond Basic. If you haven't listened to part one of our two findings episodes, you can go back and listen to that and hear all about what the program delivered in practice. They'll be available on the integrity website as well as our LinkedIn. Thanks for listening.