The Indicator podcast
A podcast for people who produce and use evaluations and performance information. Not nearly as dull as it sounds!
The Indicator podcast
Evaluation offices and the Brazilian experience
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
Lycia's Book: https://fgvclear.org/website/wp-content/uploads/guia-clear-monitoramento-e-avaliacao.pdf
3ie evidence portal: https://developmentevidence.3ieimpact.org
Welcome to the Indicator podcast, where we talk to people working to make evaluation systems stronger. This season we're looking at evaluation officers themselves. Not what evaluations find so much, but how the offices that produce them are built, resourced, and run. My guest today is Lysia Lima, a founding partner of Indicate Impact, and someone who spent her career at exactly the intersection of evaluation methodology and institutional design. At the government of Minas Gerais, the Secretariat of Strategic Affairs of the Presidency of Brazil, at the World Bank, the UN, leading the evaluation function at FGB Clima, and now as director of FGB's CLIA Center for Brazil and Lucifer in Africa.
SPEAKER_03Welcome, Lysia. Thank you so much. I'm so happy to be here.
SPEAKER_02We spend a lot of time in the evaluation world debating methods, but not that much time talking about the organizations that do that work. We want to open up that black box, but before we do that, tell us a bit about you. You're a professor, but you do a lot of policy work, especially around linking evaluation to policy design and policy improvement. You work a lot with methods, but also with institutional design. You started your career focusing mainly on impact evaluations and later took on a more broad perspective, a kind of let's call it program evaluation approach.
SPEAKER_03Why did you change?
SPEAKER_00I think my my original drive to work with evaluation was the potential we had to actually impact people's lives by having a positive impact on policy. So this is what this is what drove me to this area. So I realized that by doing impact evaluation, we had the potential of improving policy, finding out what works, what doesn't work, and you know, spending better the public money to have a better impact in people's lives, especially, especially relevant in countries, in lower lower income countries. But then with time, I realized that other things, not only the final impact of a policy, actually mattered for you know designing and understanding and improving management of policies in a way that it increased the likelihood of that policy having an impact. So I realized that evaluation was not really about finding out whether it worked or not, but just really evaluating every way, every step of the way of the policy cycle. You starting with evaluating the idea, evaluating the problem we're trying to mitigate. So, you know, evaluating the design of the policy that you're trying to establish, evaluating the implementation. So I realized that, you know, to actually reach its full potential in terms of transforming people's lives and impacting a society in the most ways, evaluation had to be taught throughout all the evaluation. To ask a bigger question, really. Yeah, it's really like, you know, what do I have to do to fight this problem? You know, how do I do it? How do I choose between options? What's the best way to implement the best option that I found by design? And then at the very end, okay, I did all of this, did it have an impact? So I realized that by focusing on impact evaluation, I was only focusing on the very final bit, which was already like all final. So I was like, you know, we need to shift this idea and just kind of implement what we call the evaluative thinking throughout all the way in the policy cycle. So this is, I mean, it's much more interesting and it's much more challenging. So yes, okay, but let's not get into the method side of things.
SPEAKER_02So tell me. Brazil has grown into one of the strongest monitoring and evaluation countries in the global south. And looking back, this history actually doesn't go all that far back, right? It's relatively recent, really. What's the current state of M ⁇ E systems in Brazil and how institutionalized is it in the public sector and the private sector, in fact, and in civil society? And more broadly, how have M ⁇ E practices in Brazil evolved in recent years? And what role have evaluation officers played in driving that evolution?
SPEAKER_01It's very true.
SPEAKER_00Uh Brazil actually, I I it has uh had an impressive evolution on this theme. Quickly, right? Yes, very impressive. Yeah. Um when I I I mean, I worked uh abroad um with several countries for many years. And then at one point I decided to come back to Brazil to work on on the government working on institutionalization of evaluation in one government. And there was like the very around 2010, so let's say like 15 years ago, was the beginning of that idea in Brazil. So it's not very long ago. And then a couple of years later, we launched CLEAR. And Clear in the beginning, people really didn't like it. CLEAR was the Centers for Learning and Evaluation Results, which is this initiative that works on fostering evaluation capacities in.
SPEAKER_02Which we both know very well, right?
SPEAKER_00Yeah. So uh originally, people we we tried to talk to to governments and and units of even private civil society organizations, NGOs, on the importance of evaluation. People did not like this word. They linked it with oil in Brazil. They did not, they weren't into it. They thought it was like a threat to them. You know, it was like these external people coming to check whether I'm doing my job right, and people hated it. Because they thought it was just like, you know, people coming to see whether I'm working well. And it was, and it was, it was really difficult. It was very, very hard to get through. And then actually, this is this is maybe 2014. So it's very recent, like 12 years ago. Uh, with time, we started doing a lot of work through Clear and also through the Brazilian Monetary Evaluation Network, which we launched um around that that time as well. And on basically, uh basically on culture building and awareness, awareness raising on on the role evaluation had uh as a management tool, and not so much as a as an auditing, you know, a tool to check whether you're doing your work right. This is not what it's about.
SPEAKER_02The weapon of managerialism.
SPEAKER_00Yeah, but you know, it's not where where the where where I mean the main the main richness of it is. So then people realized that by actually embedding evaluation in their in their units, in the governments, in the in the institutions, they became much more powerful to have much more control over how the program, the policy, they worked evolved throughout time. And when people actually understood that, they started to understand that, oh, this is actually very cool. And actually, it's when you when you really embed evaluation into your practice, it kind of shifts the way you think about policy in general. So then it was a beautiful movement here in Brazil because then we got a big project with the federal government where we helped them institutionalize the unit of evaluation and write the guidelines and all of this. And the federal government has a very big impact on subnational government in Brazil, and then that followed by subnational government's wave for also trying to institutionalize. So Brazil actually moved from a very low context of institutionalization around 2014 to a very strong context of institutionalization, including including even like a constitutional mandate, it was really a shift in the culture. Like people really like, I think we, you know, right people at the right time, the managerial, we had we did a lot of work through CLIR on the government, but also there were very highly but listen.
SPEAKER_02I mean, I think be a bit less modest. Didn't the CLIAR sort of cohort of people that had been brewing their sort of move into federal government and get deployed in some ways? I mean, not intentionally in a sort of coordinated way, but essentially people who had learned how to do the stuff at CLIAR landed up getting jobs in government and and promoting a particular approach.
SPEAKER_00Well have the first wave here was uh this this office linked to the federal government, linked to the presidency office. They um invited us clear to come help them design you know the guidelines and the institutional mandate. And the the the beauty about CLEAR is that it's a global initiative, you know.
SPEAKER_02Right.
SPEAKER_00And there were other, there was one uh another center in Mexico at the time, and there were other centers elsewhere, and Mexico was much more advanced than Brazil then. So they already had the guidelines, also Chile. So it was nice because we managed to bring other experience from the global south that were already more advanced than us and share with the Brazilian government.
SPEAKER_02In the South African or two, right?
SPEAKER_00Yeah, yeah, certainly. And then and then actually there was a lot of a lot of learning from the global south, and people just got inspired. And I mean, I have to say that the federal government had some really highly qualified staff who were super engaged to this, to making this happen. And I think it was a joint effort between us in CLEAR, uh, and then all of these people who were very engaged at the federal government. So this move really fast. I I uh I remember there's a newspaper page, one of the main newspapers in Brazil from 2017 that says Brazil starts, first page, Brazil starts to evaluate its policies, but it's it's it's behind Chile, Mexico, and you know, but then it was this big thing, 2017. And then now, like this is a very strong, now actually the constitutional mandate came in 2021. You know, throughout that time also we had many states doing this, several subnational states, so Sao Paulo, Minas Gerais, Spiritu Santo, Maranhão, many others. Um, so like it's strengthening, expanding their structures, approving laws. So it is impressive what he has done. But it's not quite there yet. Because it's massive.
SPEAKER_02Okay, wait, before we get to that, let's just talk a little bit about these entities in these organizations, like so the units, let's call them, offices, that have kind of driven this work. Um tell me a bit about how they've functioned and operated and what do you think have been important factors in their success? Because when I've spent time in Brazil, I have been struck by the extent to which evaluation isn't just like a kind of standalone unit, it's often integrated into planning and that it's something people seem to approach as a kind of key element of management of doing your job.
SPEAKER_01Yeah. Um for sure.
SPEAKER_00I think the key success, I mean um there are many, but I'll have to say that I couldn't rank it, but I I think that is maybe my top three. One is the the cultural shift on why this matters. You know, because if you if you do like a top-down mandate, like, you know, the governor right is there and it's like now we have to evaluate and just tell people to do it, and people don't feel ownership to the process, they're gonna not gonna be very happy about it. They're not gonna use the evaluation findings, they're just gonna be very upset with like extra load of work. But then if you do like what we did in several states here, if you do like a big campaigning and really explaining and training, you need to train not even only the people who are gonna work in the evaluation office, but everyone. People even working only in management, on the day-to-day operations, because those people are gonna be related to the evaluation, to the data collection and everything. They need to understand why that's being done. And once everyone is on board and everyone's like, oh, this is really cool, actually. This is this is helpful, this is a good idea. So people are engaged. Once everyone is engaged, even if the governor changes, you can't really undo that because this is it's once there's a shift in the culture and a shift in the way people behave and think about an issue, it's hard to go back.
SPEAKER_02So I think in terms of like So would you would you say that one of the key things a successful evaluation office needs to be able to do is to promote that shift, that cultural It's like a cultural change, which which means engagement at all levels, not only at the executive level of people who are making the decisions, but you know, at the at the field level, at the program level, everyone needs to understand and understand what this is about.
SPEAKER_00Because it comes even down to like when then you can say, oh, it's a people in the field, it don't matter so much. I was like, well, yeah, they matter because the data that we collect that feeds into the evaluation, usually these people are linked to the quality of the data that we're collecting. If they don't understand what that's for, it just becomes uh admin burden, you know, like this show of this form in whatever way, and then and then it's useful. You know, so so and if everyone engaged, then which is it is what's happening in Brazil in many, many, many places. I think this is number one. Number two is obviously political will, because I mean, you need to be willing to actually do this because it's it's it's tricky, you know, because it's it can be a burden, it is expensive, it's usually a function that you need to add on top of a billion other things that are already going on. Often you need you don't have that much money to like get additional staff. So it is tricky. You really you really want to do it. And also, there's also tricky the issue of what if you have a policy that is very linked to your beliefs, but then when you evaluate that policy, it doesn't do so well. And then I mean what do you do with that? No, seriously. What do you do then? Oh, sometimes it's like a uh a uh like a campaign promise. Like, you know, it's the it's a political party commitment, and then and then you go, you evaluate like okay, this is not so great. I mean, what do you do? You know, so it's it's politically tricky. Yeah, yeah, yeah. And it's it kind of like you can it can put the politicians and the people in the executive positions in a tricky position. But you need to you need to you okay, keep going now. That's enough on that one.
SPEAKER_01Number three now. Number three is uh evaluation is expensive. Money.
SPEAKER_00Yeah, we need resources.
SPEAKER_02Resources, yeah.
SPEAKER_00Human capital resources, but also resources for other things, for collecting data, for doing everything. I mean, the thing is it looks expensive when you think about you know, I mean it is expensive, but then obviously when you think about you know the the the the costs related to the benefit, because if you don't do it, you're risking wasting a lot of money on effective policies. So it's better that you just, you know, you you do invest the money for evaluation.
SPEAKER_02You know, I think the problem is that we often we're often asking these evaluations the wrong questions, right? Because oftentimes the programs are, as you say, a values-based commitment. It's something somebody politically committed to. But I want to move, I want to move on with my question. So you talked about three things. You talked about political will, you talked about money, and what was the first thing you mentioned? Change in culture, engagement at all levels. Oh, culture, like a mindset shift, yeah. Anything you want to add to that or can I go on?
SPEAKER_00Another very key element for having a successful evaluation unit is the capacity of that government to actually use the findings. And I think today, in the space of evaluation and FDCs, this is the main challenge. Because we've gone past, like, we don't know how to do evaluation, even in lower income settings, like we have moved to a situation where we can do it. And where even in lower income settings, they don't they do simpler stuff, but they still do it. But we still have a huge gap on on using evaluation. There's a lot of places that have evaluation functions, but there's no linkage to the budget or planning or anything.
SPEAKER_02Planning.
SPEAKER_00Yeah, so basically there, but I mean, and then what what do we do with the results? I mean, you know, there's no clear path to what what's gonna happen. And then, I mean, it it then it's tricky because maybe just serves no one if it is not used. So this is something we need to look for.
SPEAKER_02And in fact, it's it it it has a weird effect when you're producing all of the stuff that you then ignore. I think it has a very corrosive effect on an institution because what is real, it all becomes fake, right?
SPEAKER_01Yeah, certainly.
SPEAKER_02Okay, let's go on. You were one of the founders of the Brazilian ME network, and you also served as one of its directors directors for quite a long time. How important was the network for the evaluation community and for advancing evaluation practices in Brazil in the way that they have advanced?
SPEAKER_00The Brazilian Monetary Evaluation Network, we launched it uh after the Latin America Network had already been launched for a while. It was a very strong network and it was super relevant. And then, I mean, and Brazil is a little separate from Latin America because we speak a different language. So it's not really like we have a something lack that doesn't usually include Brazil because they speak Spanish and we speak Portuguese. So and then we launched it. In the beginning, it was small, and this is like a nonprofit, and it was a little tricky with the staffing, but then there's some really amazing people that joined the effort over time, and honestly, it's so important. It was so relevant to shaping the ecosystem in Brazil, mainly because people like institutions like CLEAR, uh, I think CLIAR had a very important role in shaping several, you know, several um situations in Brazil. But I mean, Clear has is small and it worked with specific governments to, you know, making very advanced, very big advancements in in the area of evaluation. But what the Brazilian M ⁇ E uh network did was to really democratize. Because Brazil is huge, it has 27 states, almost 6,000 municipalities. So by doing this, and they did this and they launched a platform, and then all of a sudden I would go into our ME platform, I would look like, oh, there's people from like, you know, this city in the like the far north that I, you know, I was like, how did they join? You know what I mean? So it was very relevant to really make knowledge that uh obviously it would be wonderful if that now they're much more professional in a sense that they have more people and they manage to fundraise more, so they have more stuff going on. But honestly, they they are amazing and they were amazing shaping up you know the Brazil ecosystem and then just you know, and just lowering the distances because in Brazil, I mean, we're here based in São Paulo, the super, you know, rich capital, and then we work a lot with the richer states, which is just easier, you know, and they are the people who have the money to make this happen, but knowledge doesn't really get everywhere, and the Brazilian M ⁇ A network does that very well to just linking people. And and also they do a very relevant job working with the private sector and civil society as well. Uh, because a lot of the other, like for example, CLIAR has always been very much focused on governments. That's changing a little bit, but has always just been, you know, main government focus. So the Brazilian also brought people together who were very concerned and civil society organizations and all of that. So we yeah, they were very, very relevant, key to to where we where we are today.
SPEAKER_02Now, in terms of evaluation offices and units, um, do you think every government institution or big civil society organization should have their own? Or do you think it's better to commission evaluations externally? And when should institutions choose one approach over the other?
SPEAKER_01That's a good question. I think yes, everyone should have an evaluation unit, but not with the same design.
SPEAKER_00I think an evaluation unit is more than an evaluation unit. It's it doesn't even need to be called evaluation unit. But there needs to be some sort of evaluation function. I mean, people they're working on management of programs and policies, they need to have evaluation training. Training, because evaluation, I mean, the the the any policy or program should start with an evaluation. They should start with uh needs assessment, with a diagnostics. This is already an evaluation. So you don't want to be designing any intervention without their the initial exercise of doing a diagnostic.
SPEAKER_02Some kind of formative work of some sort, whatever you want to call it. Exactly. And associated with the same thing.
SPEAKER_00I mean, if you don't, you know, you don't design anything without without having some sort of evaluation training in that sense. And there's all these methods on evaluation design that help you choose wiser what you're gonna do. So any institution needs to choose a line of action. Am I going on to track A, B, or C? So how do you decide? And then when you when you get exposed to evaluation training, you learn how to decide between, you know, three different and potentially attractive ways of moving forward to achieve a goal. So I think everyone should have some sort of evaluation function.
SPEAKER_02Except, I mean, what you're really talking about isn't necessarily evaluation. I mean, in some ways it's kind of strategy or it's foresight. Yeah, but there's an overlap.
SPEAKER_00No, because it could when a lot of a lot of this a lot of these tools can be called ex ante tools for evaluation, or they can be be like, you know, strategic planning. And I mean, some of these tools overlap, right? Because, you know, I don't I don't call I don't I don't call evaluation just exposed evaluations. You know, all these exercises who evaluate the design for me are also evaluations, and this is how we see it. Um in terms of design and unit, I think you need to think what that is for. Is that what what does the institution need? And then once you have did these diagnostics, which is you know the problem you're trying to solve, then you decide what you're gonna do. Because No, I think that's exactly right. If it's a small institution, maybe it doesn't make sense for you to set up a team running impact evaluations because you know it's gonna be expensive. There's no need, there's no, but maybe you wanna have training for the people who are there, or maybe you want to have one person to do like all the theories of change and design indicators and do like some exante analysis or or or just to to follow up on implementation and then and then maybe you commission whatever exposed, you know, cost benefit and impact work. Maybe you commission that. So I guess it depends on the.
SPEAKER_02But I think your point holds, which is it kind of depends on the institution and what their setup is, and then designing the and what do they want?
SPEAKER_00You know, what do they want with evaluation function? Because I think at least helping them choose directions, like at least the the all the tools that help with the So it's almost an advisor, at the very least, to have an evaluation an advisory function, a capacity.
SPEAKER_02Yeah. So now listen, you've worked on evaluation capacity development across Brazil and Lucifer and Africa, which means helping governments build the infrastructure for evaluation, not just conducting evaluations themselves. When an evaluation office exists, but it's not really happening, there's nothing really going on. What do you think are the main things that are missing? I want to suggest one, but I'll wait until you've spoken.
SPEAKER_01Um I think the main thing is human capital that people are available, like to, you know. You mean like skills that people say.
SPEAKER_00With the right training, yeah, with the right skills, but also with the enough time. Because sometimes you need you have you have you you have people working on that, but they have a billion other issues and and things on their job list. So I mean, you know, this is a full-time job. I mean, people, you know, this takes a lot of time, so you can't really be like, you know, doing a million things in this. So enough, you know, human capital with the right skills and the right, you know, amount of time available to do this. But also other resources as well, not only human resources, but also like, you know, actually financial resources and other resources. And and also like political will with that that that set up the right incentives for evaluation use. Because if people are not using it, why is an evaluation office gonna, you know, bother to just, you know, get so much done if that's not gonna go anywhere. So I think those three, maybe. Do you have anything else in mind?
SPEAKER_02Uh leadership. I really think that it's a massive question in in the functioning of these offices, is how the leadership functions and kind of where they want to position themselves, right? If they're wanting to be in uh very much close to the operations and are kind of really seeking affirmation from uh from the projects they're evaluating, or if they're willing to be unpopular, which is difficult. So there's a persistent tension between evaluation officers that specialize in what gets called rigorous impact evaluation, randomized trials with counterfactuals, the methods on which you as an economist have built your academic career, and offices that are expected to produce and use a much broader range of evaluative work, often called program evaluation. How should an evaluation office think about that trade-off when designing its mandate? You've recently written a comprehensive guide on ME of public policies in Brazil. Tell us a bit about it as a useful resource for people working on the design of evaluation units, offices, and mandates.
SPEAKER_00Great, thanks. Um yes, I guess.
SPEAKER_01Start by telling us about the book.
SPEAKER_00Yeah? Start by telling about the book. Start by Yeah. Okay, so this book, this book is actually like a compilation of all the work that I've done, you know, helping governments. It's only in Portuguese now, right? It is in Portuguese, but it's coming out in English sometime very soon. Okay. So keep keep an eye for it. It's gonna be on the better evaluation website, hopefully, and also on the Clear Lab website. So it's it's free of charge. And I mean, I've we've been getting a lot of positive feedback on it because uh this this this is obviously not a comprehensive book that goes uh that reviews all the evaluation methods. This is not not what this is about, but this was actually designed to help people who are actually trying to set up an evaluation function and tell them, you know, what can what what what are the main things that you could do? But and the way the book is organized is around the evaluation, the policy cycle. So it's like, you know, what can you do when you're thinking about a policy or a program? What can you do when you're implementing a policy? So, you know, thinking of the different stages of the policy um life cycle and and which tools can you use in each time. So I think uh you asked about the trade-off that evaluation office needs to think about when choosing one method or another. You you first I don't think you choose a you need to choose a method. You need to choose a question, an evaluation question. You need to see my evaluation office has been set up to see whether uh the policies are well designed. So this is something, so this is if this is what you want to do, so then you go after the method that helps you answer that question. Or you can say, Oh, my evaluation office wants to see whether things are being done the way they're supposed to be done. So you have a process implementation question, and then you're gonna go after what what what you need to answer that question. So I think you need first to actually do like a theory of change of your evaluation office and see what is it trying to do, you know? Where is it trying to do?
SPEAKER_02And then Oh, oh, I mean, you know, I think one of the big questions is whether or not you need to do that ex ante, or whether you can just kind of roll with it as it comes up. And I mean, I think one of the questions that we really need to also maybe put some energy into is how evaluation officers decide on their work programs. Yeah. Because the composition of their work program is really where that power play comes into operation, right?
SPEAKER_00That's how these questions are framed and what gets scrutinized, and that's really when you're why actually I was actually talking to someone in the government here in Brazil yesterday, because Brazil has it has institutionalized at the federal level, so it has an evaluation um secretary under the Ministry of Planning that is in charge of all the main evaluation agenda for Brazil. But I mean, obviously, they can't evaluate everything because it's too huge. And then different ministries. So I was talking to this person from another ministry, and she was like, oh, they already have the central evaluation function in the planning thing. Should I have it here in my ministry as well? I was like, Yeah, absolutely, because then you then you then you own the agenda. Because then you then you because those people are doing the evaluation for the planning, you know, looking at the whole country as a whole. If you want to do inside your ministry, maybe you want to have a different agenda with a with the different questions, because you're in charge of the management of this. So, so you know what I mean. Like also in this agenda setting also depends on which level the evaluation office is. And yeah, so I think you know, you need to start designing your evaluation office thinking first on why you're doing that. What kind of questions do you want to answer? And yes, please um check my book because it helps.
SPEAKER_02So you've worked extensively on early childhood education and labor market programs, areas where the evidence base is actually quite strong and the data is often very, very reliable. But the translation of those evaluation findings into policy and implementation is often very weak. Why does that happen? Why do we get these essentially unhelpful evaluation reports?
SPEAKER_00I think there are two main reasons. I mean um first one is one issue I've mentioned before. I think the main challenge in the area of evaluation that we have today is the um limitation of use, you know, and the lack of structure, how findings are gonna be used. Because this is if this is not well structured, it becomes a political problem. Like, okay, I have this problem here with telling me to change all these things, but then I'm gonna have all these problems with these other groups of people. Like, why did you ask that question if you can't handle the answer?
SPEAKER_03Like, ask a different question.
SPEAKER_00Yeah, but I mean, I guess I guess people wanna know, but then when they see the problem, they realize that, okay, maybe this is trickier than I thought to address. So the thing is there needs to be governance of how this is gonna go. Like, you know, so if there is a clear path on how these things are gonna be used, the findings are gonna be used, then it becomes easier and less politically loaded to do something about it.
SPEAKER_02Are you saying like what one of the things maybe we should think about is how evaluation officers agree before the evaluations are undertaken, exactly what use is gonna be made of the findings. And so that the recommendations are very tailored.
SPEAKER_00Maybe not only in evaluation, like this is maybe when you're talking about government, it's like a government-wide thing.
SPEAKER_02Like if if it is linked to the governance structure of the whole system, you know, you're talking about institutionalization, which governance generally aren't single entities that govern these systems, they're like kind of like wild waste chaotic.
SPEAKER_00Yeah, but sometimes they're like legislation, which don't always work, but you know what I mean? These things should be done. I mean, one thing that you should not do is do an evaluation office, then think about it later how you you're gonna do deal with the results. Don't do that.
SPEAKER_02Yeah, yeah, yeah, yeah.
SPEAKER_00At the same time, because you know, later is a little tricky. Um, so one thing is um the importance of use. And something else, and I think I think this is actually a newspaper article I wrote recently here in Brazil, which is the lack, we need we need better bridges. Because what happens as well is a lot of academic production of very relevant materials about policy uh effectiveness that don't get where it was supposed to get. There's just a lot of people like working on the management of the policies, they never see that academic paper. And plus, oftentimes the academic paper is not even translated. And I mean, there's there has been over time some very beautiful efforts to translate that work, like you know, the work of 3IE, the work of JPAL, the work of Innovation of Poverty Action, and So let's just also take a moment to, and I'll link again in the chat uh the 3IE evidence portal, so so people can see that kind of thing. Yeah, 3IE, and then it's also the other these other uh initiatives who work on on evidence synthesis and and and systematic reviews, like you know, the Campbell Collaboration, Cochrane, also 3IE has some amazing stuff. So, I mean, there are some initiatives, but we need more. We need more. We need to make sure we close that that that gap.
SPEAKER_02I mean, I don't know that we can get more, but I think we need better use of what we have, right? I think we need a clearer understanding of that.
SPEAKER_00Yeah, maybe we don't need more. We need we need a better, like, you know, um linkage, better like networks making this information go around. And I don't think it goes around enough. We do a lot of this everywhere I am, we do a lot of this at FGV Clima. We also working on a platform on evidence on climate as well, and clear also does. I mean, we we need to do that more. We need people who work at the intersection of policy and academic, because maybe these people are also not many, because you're either in an academic or uh uh or you're working on policy, but we need to make sure that link is strengthened. This is very, very, very crucial.
SPEAKER_03This is my last question.
SPEAKER_02If a minister or a senior official listening to this podcast, someone who approved the creation of an evaluation office, but who's never really questioned whether or not it has or it needs to function well, changed just one thing as a result of our conversation, what would you hope that would be?
SPEAKER_01What would you want that change to be? Just one. Just one.
SPEAKER_00Um I think it's the importance of doing something with that. Just don't just come back with the idea of just let me have an evaluation after you got a good idea. Like, why is it a good idea? How are you gonna use it? You know, link it to planning to budget, make sure there's a governance to do something with it, not just like don't just have it on paper because then it seems less.
SPEAKER_02Yeah, I mean, I think the thing for me is for people to get really intentional about how they understand these institutions to offer value. Like, what do they really get out of them that's gonna make the money spent on them worth it? That's I think the big question.
SPEAKER_03But thank you for the conversation and very lovely to see you. Thank you so much. Always lovely to see you. Have a good afternoon. Bye bye.