Case study

Contributing to the reduction of the psychological trauma of migrant and displacement children in Colombia

Technical assistance to Universidad de Los Andes

General Info

Supported entity and its role in implementing the program:

Universidad de Los Andes – outcome payer of the Semillas de Apego program

Sector:

Early Childhood Development

The entity’s point of contact:

Maria Alejandra Palacio – ma.palacio43@uniandes.edu.co

Instiglio’s point of contact:

Santiago Ospina – santiago.ospina@instiglio.org

Type of support provided by Instiglio:

RBF mechanism design

Executive Summary

By the end of 2020, one out of five children globally under age five was at risk of not fulfilling their development potential because of their exposure to armed conflict and displacement. To address this challenge, Universidad de Los Andes, a top-ranked university in Colombia,1 adopted and adapted the Child-Parent Psychotherapy program, which had been designed by the University of California, to the most vulnerable municipalities of Colombia to protect children from the negative psychological effects of stressful situations. This adaptation, the Semillas de Apego program, works with primary caregivers and focuses on 1) assisting mothers and primary caregivers who have been victims in processing their own trauma; 2) understanding children’s development trajectories and how they are affected by the exposure to adverse events; and 3) fostering good child-rearing practices.

The program was first piloted in 2015 and has been run yearly, albeit at a limited scale. Initially, the program was run directly by Universidad de Los Andes and was subject to a rigorous impact evaluation, where significant improvements in maternal mental health, child-parent relationships, and early childhood mental health were found, highlighting the program’s potential to help children thrive despite adverse circumstances. Moreover, the program effectively reduced anxiety symptoms and parental stress during the Covid-19 pandemic (Moya, et al. 2023).

Building on the positive findings of the impact evaluation, Universidad de Los Andes sought to scale the program, but recognized that doing so would require bringing on an implementing partner with national reach and expertise in running large-scale programs. In 2022, Universidad de Los Andes hired Heartland Alliance International to implement the first pilot for scaling to five municipalities, and, during 2023, to maintain fidelity to the original program design, incorporated a results-based financing mechanism in the contract.

Results-based financing can effectively help programs scale with maintained program quality by setting incentives for the service provider to focus on the critical program activities that have proven to deliver the desired program impact (Instiglio, et al. 2022). Instiglio, with support from the Conrad N. Hilton Foundation, supported both Universidad de Los Andes and Heartland Alliance International in incorporating a results-based financing approach into their contractual relationship. Through collaborative design workshops, Instiglio supported both parties in identifying the pain points in implementation and defining the payment metrics (i.e., results to be paid for when successfully achieved) to be used as incentives to achieve improved performance and fidelity to the original model.

As the project is being implemented in 2023, this case study will be updated once the project is completed. The project stakeholders are documenting the design and implementation process, expecting that this case study will be useful to others working to scale up evidence-backed interventions with fidelity.

Project
Partners

What was the opportunity?

Since the 1950s, Colombia has experienced internal conflicts that have displaced and disturbed the lives of millions of people. As of April 2022, over 8 million people had been displaced. Over 1 million children between the ages of 0 and5 have been exposed to armed conflict. In addition to this, more recently, nearly 2.5 million Venezuelan nationals have migrated to Colombia due to violence, persecution, and economic necessity (R4V 2022). This group has included more than 300,000 children (Semillas de Apego n.d.). Venezuelans are now more than 5 percent of the population living in Colombia (R4V 2022). Violence and displacement cause both physical and psychological trauma. Displaced people are prone to develop mental health problems. For instance, 1 in 4 displaced people suffer from depression, due to resettlement in new environments, traumatic events and forced migration (Bedaso and Duko 2022). Displacement affects children in multiple ways, ranging from immediate stress responses, increased risk of mental disorders, distress from forced separation from parents, and fear for personal and family safety (Burgin, et al. 2022)

This violent history motivated Professors Andres Moya and Arturo Harker at the Universidad de Los Andes to develop the program Semillas de Apego in 2014. Based on the Child-Parent Psychotherapy program designed by the University of California, Semillas de Apego aims to promote the mental health of victims by building safe and healthy affective bonds between caregivers and children. This intervention is implemented during 15 weeks, during which participants meet in 2.5 hour weekly group sessions of 12-16 participants led by community facilitators to discuss practices to promote emotional self-regulation, resilience capacity, and reflection capacity among caregivers. In addition, these aim to reframe the impact of violent environments and support networks (Moya, et al. 2023).

Semillas de Apego was first piloted in 2015, and in 2018 it was implemented in Tumaco, one of Colombia’s most vulnerable municipalities, withi 714 participants. The program in Tumaco was evaluated with a rigorous impact evaluation that found that the program delivered positive results, demonstrating the program’s effectiveness to improve mental health outcomes for vulnerable children and caregivers, including migrants. For instance, at the 1-month follow-up, the intervention produces significant effects on the child-caregiver relationship, and at the 8-month follow-up, the intervention has significant effects on the maternal mental health index, the child-caregiver interactions, early childhood mental health index, and early childhood development index. In addition, preliminary results indicate that the program allowed caregivers to better cope with the shock of the Covid-19 pandemic and exhibit lower anxiety and parental stress symptoms 9 months into the pandemic (Moya, et al. 2023).

To reach more families, the Universidad de Los Andes realized it would need to hire an implementing partner with experience implementing programs at scale. In 2022, it hired Heartland Alliance International to deliver the program in five more municipalities affected by violence. Heartland Alliance has been working in Latin America and the Caribbean since 2010 and has experience working in Córdoba, Chocó, Valle del Cauca, Cauca and Nariño, regions in Colombia that have been severely affected by the conflict During 2022, the program had challenges in keeping the same high participation and quality indicators during scale-up as the program had experienced when implemented at a smaller scale. This is a common challenge when scaling up programs, and one that can be addressed by using results-based financing.

What was our response and what was it about?

The challenge faced by Semillas de Apego is a common one among evidence backed interventions. Evidence is often developed at smaller scale and in highly controlled experiments, and it is a challenge to scale up an intervention while maintaining fidelity to the original model that was subject to rigorous evaluation. One way to maintain high fidelity is to align incentives around achieving the outcomes that the impact evaluation found to have a causal relationship with improved outcomes.

Instiglio, with support from the Conrad N. Hilton Foundation, supported both Universidad de Los Andes and Heartland Alliance International in identifying the pain points in scaling up the program and including a results-based approach in the contract between the two parties to achieve enhanced results. Instiglio initially worked with both stakeholders to map out the theory of change, the implementation plan and identify pain points. The parties identified two main pain points: low attendance rates for caregiver training sessions and difficulties enrolling participants with the appropriate profile (victims of armed conflict or migrant populations who are primary caregivers of children aged 0 to 5 years). Based on this, Instiglio conducted a series of 12 workshops to plan, design and validate the results-based agreement within both parties. The teams agreed to incentivize the following payment metrics:

Participants selected that fit the defined profile, (e.g., victims of armed conflict or migrant populations who are primary caregivers of children aged 0 to 5 years),
Participants attend one of the first two sessions (of 15 planned), and
Participants complete at least 12 of 15 sessions.

The payment metrics selected reflect the need for program fidelity and respond to the pain points identified. First, selecting participants (e.g., the primary caregivers of children) that fit the established criteria ensures the program reaches those that benefit most from the program. Second, the program has proven more effective for the people attending one of the first two sessions of the program, so it is vital to work towards that objective. Third, the evidence shows that caregivers that complete 12 sessions (of 15 total) obtain better program results. Taking this into consideration, the teams agreed on verifying the following indicators:

Number of registered participants who meet the appropriate profile, taking as input an initial survey participants take under the initiation of the program.
Number of participants attending at least one of the first two sessions, taking as input platform reports detailing attendance information for each session.
From the participants that met the condition of the second indicator, the number of participants completing at least 12 of 15 sessions, taking as an input platform reports detailing attendance information for each session.

The total amount of resources destined to the intervention is around USD 700K, of which nearly 5% of this total is tied to the achievement of the agreed results. The 2023 contract started its implementation in April 2023.

Implementation Phase

The contract was signed in February 2023; implementation started in April and will conclude in December 2023. The stakeholders seek to understand if including incentives that encourage the service provider to better select and retain the program participants improve the results, and if so, would like to refine these incentives moving forward. Following the 2023 implementation, the stakeholders will work to re-design the results-based financing mechanism incorporating lessons from the first experience.

Lessons Learned

Below are key considerations learned from this experience so far, which may be especially useful for those seeking to use results-based financing to ensure high fidelity to programming during a scale-up phase, and for those working with migrants as a target population.

A results-based financing mechanism can also be used to incentivize the achievement of outputs

When faced with a situation where a program lacks evidence backing and teams actively seek an effective intervention, it is accurate to explore incentivizing results, encouraging implementers to think outside the box, experiment, and discover interventions that yield positive outcomes. However, when working with evidence-backed interventions, the focus should shift towards ensuring fidelity and scaling up the intervention successfully. In such cases, the right outputs will lead to the desired outcomes; therefore, setting the incentives for the service provider at the output level of the theory change can ensure a successful implementation. By aligning incentives with the specific goals and challenges at hand, we foster an environment that promotes both creativity and adherence to proven methodologies.

The results-based financing mechanism can be leveraged to build capacity across all stakeholders

The outcomes payer and service provider may not necessarily have the same knowledge or experience regarding this innovative financing tool. Accordingly, adjustments to the design approach may be necessary to speak to stakeholders at different levels and bring them along at different speeds.

For this design, Instiglio conducted separate workshops with each stakeholder to ensure the two parties understood all the implications of the design process according to their different levels of understanding and experience with results-based financing. This approach can be useful to alleviate frustrations or time constraints during the design process, particularly when stakeholders individually need more time to cover design topics (e.g., the service provider wanting to understand the risks of non-performance) that is opposite the need of the other stakeholder who may have less concern with a given topic.