Assessments for effective social change

William MacAskill’s book “Effective Altruism” questions how, despite multiple attempts to improve the world’s social problems through donations or tax-financed social programs, taxpayers have little idea of ​​how much benefit is being made. in relation to the investment made or which of all the programs should be prioritized. Based on the experience in sub-Saharan Africa of the Nobel Prize winner for economics, Michael Kremer, MacAskill argues that public policies and social programs must be evaluated based on empirical evidence and not based on an ideological value judgment. While the first presents observable data in a strictly technical framework (impartial and irrefutable), the second is based on subjective perceptions and the conclusions will depend on the interest of the person who observes them.

So one could think as a Colombian, Eureka! over there it is. In fact, the last two governments have made an important effort to evaluate with empirical evidence some of the programs in which the taxes you and I pay are invested. However, there are two recommendations that the new government could take into account if we really want to achieve social transformations in the country.
Improve data availability and quality

Impact evaluations are the best known methods for providing empirical evidence. Usually, it is believed that an impact evaluation is just an econometric exercise that yields numerical data that translates into how well a policy served, but in reality this exercise involves much more in the before and after.

In the first place, before carrying out an impact evaluation it is necessary to have sufficient data to be able to guarantee the quality of the results that are going to be obtained. Unfortunately, today it is decided that you want to evaluate the impact of a program when it has been running for a considerable time and there are no appropriate data. And what would be appropriate?

As an impact evaluation, it estimates the effect of an intervention from a comparison between those who did receive the treatment and those who did not (controls), since the minimum necessary is the information of these two groups. If you want to better understand the mechanism through which the intervention is producing the changes, it is advisable to monitor the two groups before, during and after the intervention. At least on the aspects to be evaluated (health, education, income, for example). The latter is key to avoid losing contact with the beneficiaries and not having the information when carrying out the impact evaluation. In addition, it is more cost efficient for the Government to have these data because it reduces study collection costs.

For those policies whose monitoring budget is very low, secondary information can be used from other government entities that may have collected it (Dian, Dane, etc.). Today, accessing this data is cumbersome. However, guaranteeing access to these would save public spending and improve the quality of the studies contracted by the State. It is just a matter of achieving greater articulation between agencies. For example, in the evaluation of the Solidarity Income program, the harmonization of several databases was very useful for its better implementation and evaluation.

Now, the data should always be at the same level as the intervention was performed. One difficulty of the recent study carried out by Econometrics Consultants on the impact of the Science, Technology and Innovation Fund is that, although the resources (intervention) were delivered by projects, the evaluation was carried out at the department level. This was because the impact on indicators of economic growth, science and technology was sought, and these data were only measured at the department level, not projects. When the impact is measured at a different level of observation than the level of the intervention, the true impact is blurred and it is more difficult to find an effect, when there is one. Hence the importance of projects or programs designing an impact evaluation strategy before implementation and collecting data at the same level as the intervention.

Awareness: stages of learning

Second, it is important that the human talent of government institutions internalize that impact evaluations do not necessarily find an impact. It is difficult for a policy or program to be effective the first time. Like any other invention (airplanes, cell phones, etc.) it requires testing, evaluation and feedback so that one day it becomes as functional as desired.

For this reason, when the impact of a program is evaluated and the result shows that no impacts were found or they are very marginal, public opinion should not focus on politically discrediting the program, but on positively assessing that an evaluation is being carried out technically. and responsible and trying to understand what works and what doesn’t. This poses a communication and education challenge for citizens, who must understand that, beyond populist discourses, it is necessary to make informed decisions based on empirical evidence. Only if this awareness of the stages of learning in public policy is achieved, it will be possible to use impact evaluations as an instrument for true social change.

We would like to give thanks to the writer of this write-up for this amazing content

Assessments for effective social change