India has made significant progress in achieving universal primary school enrolment, with a gross enrolment ratio (GER) of 100 percent. However, most students still lack basic learning skills. Only 43 percent school students in grade 5 can read a grade 2 level text, and only 26 percent of these students can do division.
Foundational literacy and numeracy (FLN) forms the basis of all future learning, and affects life outcomes. High rates of FLN are directly correlated to increased workforce participation and higher per capita gross domestic product (GDP). Both the public and private sectors have invested in improving the quality of foundational learning in schools, and India has emerged as a sandbox for innovations in financing models for education with grants, debt-based models, impact bonds, and other instruments. Three impact bonds have been trialled: the Educate Girls Development Impact Bond (DIB) launched in 2015, the Quality Education India (QEI) DIB launched in 2018, and the Haryana Early Literacy DIB launched in 2019.
With all three impact bonds now completed, it is an opportune time to reflect on them and distil early lessons for the primary education sector in India. We at the British Asian Trust draw on our experience with various education initiatives, including the QEI DIB, to share five learnings with funders and donors within the education sector. These learnings offer an insight into how programmes can be designed and delivered more effectively to improve the quality of learning for India’s children.
Learning #1: Donors can link funding to improvements in learning outcomes, rather than only to inputs or activities
The focus of the three education impact bonds was on improving learning among children. Accordingly, they measured and tied funding to learning outcomes as key success metrics. All three surpassed their learning outcome targets, even though two of these—the QEI DIB and Haryana Early Literacy DIB—were being run during prolonged COVID-19 school closures.
To illustrate, the Educate Girls DIB achieved 116 percent of the enrolment target and 160 percent of the learning target by its final year. The QEI DIB improved literacy and numeracy skills among its students by 2.5x in comparison to their peers in other schools. And, students in the Haryana Early Literacy DIB could fluently read 42.4 words per minute while children in non-intervention schools could read only 30.3 words per minute.
We are often asked if outcomes under impact bonds are better or higher than grant-funded programmes. There is very little robust research on this in India or globally, partly because it is still an evolving area of practice and partly due to the difficulties in conducting such research—it requires running a parallel model that is similar to the impact bond model in all respects, except that it is funded via grants. However, some implementation partners in the QEI DIB told us that learning outcomes under the DIB exceeded those in their grant programmes by 50 percent or more.
Factors that make impact bonds work can also be applied to other types of financing mechanisms, including grants. These are:
- Clearly defined outcomes, along with rigorous and independent evaluation and verification of these outcomes.
- Payments and incentives that are linked to the achievement of outcomes.
- Appropriate transfer of risk from implementation partners to investors when needed. This provides freedom and flexibility to nonprofits to innovate with interventions and delivery processes.
- Robust performance management and data-driven decision-making.
- Strong governance, transparency, and accountability.
Learning #2: Both direct and indirect interventions can improve learning among children, with varying implications on depth and scale of impact
Education interventions can be broadly categorised as direct or indirect. In direct interventions, implementation partners work directly with children to deliver their inputs. In indirect interventions, they build the capacities of key stakeholders such as teachers, principals, and government officials to enable them to perform well, and therefore improve quality of learning.
The QEI DIB included both direct and indirect interventions, with a common end outcome—improvements in literacy and numeracy skills. At the end of the programme, students under both direct and indirect interventions achieved learnings gains in comparison to their peers. Students under direct interventions showed greater improvement in learning when compared to indirect interventions. However, the indirect model was found to be 1.5x more cost-effective (in terms of producing a unit of outcome).
The selection of an intervention should therefore be based on programme objectives and learning contexts, such as the level at which students are. Where students lag more or where rapid gains are the objective, direct interventions have a higher impact and show better results. Indirect interventions are well suited where large-scale change or systems strengthening is the goal, as they are easier to mainstream and are cost-effective but may need more time to yield measurable learning improvements.
Learning #3: Targeted and structured collaboration between implementation partners can improve learning outcomes
The QEI DIB brought together the complementary expertise of two vastly different social organisations—an EdTech partner and a nonprofit—to deliver better learning outcomes for a common group of children. The EdTech partner provided the digital solution, closely monitored the data from the solution, and adapted it to the needs of the students. The nonprofit partner led engagement with students, teachers, parents, and community leaders to improve uptake and usage of this application, especially in rural areas where digital learning was limited or absent.
The collaboration between the two organisations was enabled by the clear set of roles and responsibilities laid out to achieve a shared goal and dedicated performance management support to aid integration. Both organisations agreed to joint duties to deliver combined targets, which were measured annually to assess their performance. They devised strategies together to leverage each other’s strengths and navigate challenges.
Taking joint accountability and responsibility can create a blueprint for achieving better learning outcomes.
For instance, EdTech labs generated data on students each day; this data was collected and stored in a central database. The EdTech partner conducted workshops to train the nonprofit team on understanding and reading the system-generated data. Based on the data, the five best and worst performing schools were identified, and an executive committee at the programme level deliberated on solutions to mitigate challenges encountered by the five worst performing schools. At the end of the QEI DIB programme, students under the intervention improved their literacy and numeracy skills by 5x compared to other students.
While collaboration between two social organisations for delivering an intervention is not uncommon in the social sector, taking joint accountability and responsibility for targets and outcomes in a structured manner can create a blueprint for achieving better learning outcomes.
Learning #4: Providing flexibility to nonprofits to adapt, adopt, and innovate can yield high outcomes
A common learning for all three impact bonds has been around providing flexibility to implementation partners to adapt their interventions based on on-ground needs and realities. Along with the implementation partner’s expertise, intervention adaptations are guided by strong performance management systems and data-based decision-making.
For example, Educate Girls pointed out that the DIB gave them financial as well as operational flexibility to achieve outcomes, which spurred innovative and creative classroom solutions. At the community level, their field workers were given the freedom to test different approaches and adapt strategies for enrolment and learning outcomes specific to a child’s unique challenges. Educate Girls was also able to hire and move resources freely through the course of the DIB.
Implementation partners from the QEI DIB and Haryana Early Literacy DIB also highlighted how the flexibility provided during COVID-19 allowed them to pivot their interventions to include components such as home and community classes and/or e-learning tools via WhatsApp and more. These pivots were made with a continued focus on achieving learning outcome targets at the end of the programme.
Learning #5: Building and sharing evidence on learning can lead to improved programme and evaluation design
Assessing learning outcomes remains a complex and nuanced process. It starts with defining the right metrics to capture learning skills and competencies across different age and grade groups, determining the period needed to realise these changes, and developing appropriate methodologies that allow for this testing.
All three impact bonds developed customised evaluation frameworks in line with programmatic needs and shared resources explaining the final evaluation results and approach in detail. Most education programmes commission their own assessments, as national databases typically do not offer benchmarks at the right level of geography or do not address the needs of programmes.
While this is unlikely to change in the short term, donors are highly encouraged to share their evaluation methodologies, data, and results on programme completion with the sector as it helps set a standard for similar interventions, reduces the need for duplication, and informs evaluation design for future programmes.
These lessons from impact bonds in education stand to offer invaluable insights to donors who aim to improve the quality of learning among children. While it may not be possible, or even desirable, to replicate impact bonds to achieve these goals again, the learnings can be incorporated into grant-based or other financing mechanisms to ensure that development funds are able to achieve the outcomes they seek in an efficient and effective manner.
Note: The authors would like to acknowledge Dalberg Advisors’ contribution in culling out some of the insights mentioned in this article.
—
