What we have achieved
Evaluating Our Impact through Early Grade Reading and Writing Assessments
Recent evidence shows that learning to read both early (in Primary 1 and 2) and at a sufficient rate (about 60 words per minute by the end of P2) with comprehension is critical for academic success. Children who do not learn to read in the first few grades fall further and further behind academically; many eventually drop out of school. In 2006, Research Triangle Institute (RTI) developed the internationally accepted Early Grade Reading Assessment (EGRA) model. Mango Tree has developed its own EGRAs in Leblango and English to assess the literacy gains of pupils in our project schools. Mango Tree has also developed its own Early Grade Writing Assessment (EGWA). The EGWA measures children's creative writing skills using a 6-point rubric for assessing early writing development along a continuum from emerging to experienced. To self-assess our impact, Mango Tree's first quantitative evaluation in 2010 involved comparing the project's first eight program schools with two similar, although not randomly selected, non-program schools. Each subsequent year (2011 and 2012), Mango Tree selected non-program schools to test pupils in similar learning environments who met the project's school selection criteria, but did not receive the model. While this self-assessment provided a good benchmark for Mango Tree's process evaluation, non-randomized studies that simply compare test scores across schools before and after an educational intervention are likely to be biased because schools selected for a program are often different from those not selected in important ways.
In order to prove that our intervention alone was creating substantial learning gains among pupils, Mango Tree conducted a pilot randomized evaluation of the program in partnership with University of Michigan researchers in 2013. Thirty-eight schools were randomly assigned, via a public lottery, to either receive the full program implemented by Mango Tree's field officers, a partial implementation of the program conducted through government teacher tutors, or to serve as controls.
Early results on NULP's impact on literacy learning show that the program is effective and has the potential to be more effective than many other programs that have been implemented. J-PAL evaluated 27 education interventions with randomized control trials (RCTs) from across the world based on learner's reading test scores. None of the interventions in the J-PAL study showed as significant an effect on learner's performance as initial findings from Mango Tree's NULP. Preliminary results suggest learning effects that are three to four times higher than those found in other randomized trials in low-income countries. By the end of Primary 1, pupils in Mango Tree schools were one year ahead of their peers in non-Mango Tree schools in regard to key early literacy skills. In schools where the program was implemented by government teacher tutors, pupils were slightly less than one half of a year ahead of their peers in non-program schools.
Compared to the studies, Mango Tree's intervention is placed among the most effective education programs to have been studied in the developing world.
To access the 2013 result summary report on our literacy project go to About us and click on Reports, Links and Downloads.