Evaluation of Let’s Read Fluently!

‘Let’s Read Fluently!’ (LRF) is an intervention in Jordan that aims to support pupils in grades 1-3 to become more fluent and confident readers.

Funded by the Queen Rania Foundation (QRF) and supported by the Education Endowment Foundation (EEF) in partnership with the BHP Foundation, the National Centre for Social Research (NatCen) worked with Integrated, Oxford MeasureEd, and School-to-School International on this pilot randomised controlled trial.

About the study

A strong foundation in literacy is crucial for children to develop. Learners of Arabic face unique challenges: the script is comparatively complex, and pupils use a variety of Arabic at home which differs from the formal Modern Standard Arabic taught in schools (Abadzi, 2017). Many children in Jordan finish their first three years of school without the foundational reading skills they need to reach their potential. Results using the Early Grade Reading Assessment (EGRA) in 2018 suggest that only 19% of Grade 2 and 3 pupils meet or exceed the lowest levels of reading fluency benchmarks (RTI International, 2018).

LRF! was developed in the UAE by cognitive psychologist Dr Helen Abadzi and the Al Qasimi Foundation, and is centred around a method and a student practice book designed to encourage repeated practice. 

The purpose of the pilot evaluation was to test the intervention’s feasibility, evidence of promise and its readiness for an efficacy trial. The pilot was also intended to provide preliminary evidence on the impact of LRF!, the mechanisms of change, and any lessons. 

Methodology

In the pilot evaluation of LRF!, two different approaches of the intervention were delivered: a Whole Class (W/C) model, delivered to Grade 1 pupils in their home classroom by their classroom teacher, and a Literacy Catch-Up (C/U) model, which targeted and was delivered in a resource room by a support teacher to the lowest-achieving 20% of pupils in grades 1, 2, and 3.

To assess LRF!’s potential impact and provide insights that can inform a future trial design, the evaluation was designed to: 

• be an experimental study, with schools randomly allocated to one of the pilot arms or a control arm (W/C, C/U, or control). 
• include a comprehensive implementation and process evaluation (IPE) using focus groups discussions with teachers, parents and pupils, interviews with school stakeholders, classroom observations, a teacher survey, and a pupil survey

Findings 

Evidence of promise

In the case of the W/C model, the pilot evaluation showed evidence of promise. The impact evaluation suggests that delivery of the W/C model could improve pupils’ literacy attainment. In the case of the C/U model, there is no evidence of improvement. However, the pilot RCT had a small sample size and was not designed to measure impact robustly. 

Feasibility of Intervention (in terms of implementation)

LRF! was feasible to deliver, and most schools delivered LRF! as intended with minor adaptations. The pilot identified a number of potential improvements needed to the interventions, including modifications to the training for C/U resource room teachers, the format for LRF! delivery, the content of the practice book, and the selection process for C/U pupils.

Overall, the results of the LRF! pilot indicate the intervention in its W/C form is likely feasible at the level of a full-scale efficacy trial. 

Some of the key evaluation procedures such as recruitment and randomization were executed well during the pilot study, suggesting that a clustered RCT design with allocation at the school level will be feasible in the future. However, it was easier to recruit schools in the near south region in comparison to (1) schools in Amman, (2) the middle region excluding Amman, and (3) the near north region. Recruitment for any future evaluation should consider these regional differences. Finally, retention rates of schools were high for both the W/C and C/U approaches, and the outcome measures were seen as appropriate to identify progress in literacy.