NFER publishes National Tutoring Programme 2020/21 evaluation findings

NFER and its partners have completed their independent evaluation of the first year (2020/21) of the National Tutoring Programme (NTP) Tuition Partners (TP) pillar and their impact evaluation of the Academic Mentoring (AM) pillar.

The delivery and evaluation of both pillars took place under extreme and unprecedented circumstances due to the disruption caused by the Covid-19 pandemic. There were different assessment arrangements for Year 11s (teacher assessed grades) and Key Stage 2 assessments (for Year 6 pupils) were cancelled. Findings should be interpreted in this context and with caution.

The evaluation of the Tuition Partners programme found that higher amounts of tutoring were related to better scores in English in primary schools, and with better grades for Year 11s in maths and English (teacher assessed grades/TAGs). In schools with higher proportions of pupil premium pupils taking part, the Tuition Partners programme had a positive impact on Year 11 grades (TAGs) for those pupils. 

NFER’s Head of Classroom Practice and Workforce, Ben Styles, said:

“This evaluation was carefully designed to provide schools with evidence to guide decisions and support disadvantaged pupils, but a rapidly changing situation at the height of the pandemic presented numerous methodological challenges, meaning findings should be interpreted with caution.

“The limited reach of the programme across pupil premium pupils in participating schools as a whole meant benefits were difficult to detect across pupil premium pupils, but tuition was associated with improvements in English performance for primary schools and both English and maths among Year 11 pupils.

“As schools now receive funding to spend on whichever model of tutoring they choose, it is vital that the research community provides them with the evidence they need to guide decisions. The best way to generate such evidence is a programme of randomised controlled trials designed to optimise what is, at present, a hugely diverse market.”

In addition to changes to assessment arrangements, the first year of NTP’s delivery and evaluation needed to take account of:

• Partial school closures in January 2021 which affected the intended delivery models. Many schools paused Tuition Partners tuition or chose to wait to start until after schools re-opened, shifting delivery to later in the academic year. More tuition partners sessions took place online than originally expected. For Academic Mentoring, Covid-related absences and partial school closures meant some re-deployment of mentors by some schools. 
• A relatively small proportion of pupil premium (PP) pupils in participating schools were selected for tutoring, meaning a large number of pupil premium eligible pupils were included in the analysis that did not receive tutoring. The evaluation aimed to explore the effect of the programme on pupils eligible for pupil premium funding in participating schools compared with similar pupils in schools that did not participate. But this development made comparison (and therefore analysis and interpretation) challenging and less reliable. 

Independent evaluation findings:  

Tuition Partners 

While it was challenging to detect an impact across all pupil premium pupils in schools that accessed tuition through Tuition Partners, the evaluation found that:

• Higher amounts of tutoring were related to better assessment scores in English in primary schools.   
• Higher amounts of tutoring were also associated with better teacher assessed grades for Year 11s in maths and English. 
• In schools with a higher proportion of pupil premium-eligible pupils taking part in tutoring through Tuition Partners , the programme had a positive impact on Year 11 Teacher Assessed Grades (TAGs) in English and in maths. This analysis was based on a smaller sample of schools with some different characteristics to all tuition partner schools, and so may not be generalisable to all tuition partner schools.
• Most schools were satisfied with the quality of tuition (80 per cent of school leads were either ‘very’ or ‘somewhat’ satisfied).  
• 232,892 unique pupils were enrolled onto the tuition partner programme with just under half (46%) eligible for pupil premium funding (of those for whom data was provided).
• 6,082 schools signed up to receive tutoring through the tuition partners pillar, and over half of these signed up schools (59%) had 24% or more pupil premium pupils (the reporting category for tuition partners).

Academic Mentors (AM) 

In our analysis looking at Year 11 PP-eligible pupils in AM participating schools, the majority of PP-eligible pupils did not receive academic mentoring. Whilst the results indicate a positive effect of +1 additional months progress in maths in Year 11 compared to pupil premium-eligible pupils in Year 11 in comparison schools, there is a degree of uncertainty around this result. The large proportion of analysed pupils who did not receive academic mentoring, means it is unlikely that any difference seen in this analysis model was due to academic mentoring. This means that this evaluation is unable to conclude, with any certainty, whether or not academic mentoring had an impact on the English and maths attainment outcomes of those pupils who received it.  

NFER recommends: 

• Future programmes should either clearly define for whom they are designed, or acknowledge that schools may have different views about which of their pupils most need and would benefit from tuition.  
• More should be done to target support at disadvantaged pupils. This recommendation is based on a) this evaluation’s findings when restricting to the higher pupil premium subsample, b) previous research from NFER and other organisations which shows the disadvantage gap remains wide and c) evidence in the EEF toolkit about targeted tuition.
• Schools need greater clarity about the expectations of their role in managing and delivering (different kinds of) tuition and, where necessary, additional support for doing so. 
• Schools and tutors need to work together on how best to ensure tutoring is aligned with (and additional to) classroom teaching, tailored to pupils’ needs, and that pupils complete their tuition.
• A programme of evaluation should be developed to help explore which models of tutoring are most effective for which pupils and in what circumstances. This could evaluate different types of tutoring through a range of Randomised Controlled Trials (RCTs).

The evaluation was designed and carried out by a consortium, led by NFER, with Kantar Public and the University of Westminster, and commissioned by the Education Endowment Foundation (EEF).

---

Sector Response

Julie McCulloch, Director of Policy at the Association of School and College Leaders, said:

“We welcome this evaluation. As the National Tutoring Programme ended up being pretty much the only way in which the government provided additional funding for education recovery, it’s crucial that it is effective.

“The evaluation suggests that there were benefits to pupils receiving support under the tuition partners strand of the programme, but the evidence for the academic mentors strand appears less certain.

“We agree that ongoing evaluation should take place to explore which models of tutoring are most effective for which pupils and in what circumstances. If tutoring is to become an ongoing part of the education landscape, as the government hopes, it is vital we understand as much as possible about how to use it effectively.

“But the biggest barrier to an effective tutoring system is the fact it is only partially being funded by the government. At a time when schools and colleges already face a funding crisis, more and more leaders are telling us they simply cannot afford to continue to provide tutoring.

“This report provides even more evidence that a government which was serious about helping children and young people to recover from the pandemic would properly fund a long-term programme of education recovery, with tutoring a key part of this. Instead, they now appear to be planning to cut spending on education even further. Children and young people deserve better.”

Nick Brook, Deputy General Secretary of school leaders’ union NAHT, said:

“As these reports note, there is a large body of evidence that shows 1:1 tutoring and small-group tuition are effective, particularly for disadvantaged pupils.

“Given that the National Tutoring Programme was specifically designed to support the education recovery of pupils from disadvantaged communities, it is both a surprise and a concern to note that less than half of pupils that accessed programmes were in receipt of pupil premium. The achievement gap between poorer pupils and their more affluent peers is at a ten-year high. If the National Tutoring Programme is to help narrow the gap going forward, it is essential that it is precisely targeted at those that need it most.

“The National Tutoring Programme has gone through multiple changes since the start of the pandemic. Understanding what worked in 2020/21 is different to understanding what works now. And right now, very little is known about how well school-led tutoring is bedding-in, what is proving effective, and in what contexts. Without this insight, the ‘tutoring revolution’ risks grinding to a shuddering halt the moment financial incentives are removed.”