Reflections on the Summer Symposium

On Tuesday @Ofqual held their annual Summer Symposium:

Given the circumstances this summer, it was a webinar, meaning we could speak to many more people than usual.

In relation to GCSEs, AS and A levels in England, we focused on three things – a first look at the overall distribution of centre assessment grades (CAGs), the detail of the statistical model, and some early analysis of the impact of our approach this summer on students with particular protected characteristics and those from different socioeconomic backgrounds.

Distribution of centre assessment grades (CAGs)

What we’ve seen is that CAGs tend to be higher than we would have expected and higher than results in recent years, although not to the same degree in all centres or at all grades, and not in every centre – some centres have been a little severe in their judgements.

Our initial analysis suggest that if all students were given final grades based solely on CAGs, overall A level results for England would be up, compared to 2019, by 6 percentage points at A*, 12 percentage points at A and above and 13 percentage points at B and above. And at GCSE we’d see overall results up by 6 percentage points at grade 7 and above, and up by 9 percentage points at grade 4 and above. There have never been year-on-year increases in overall results on that scale and those outcomes at a national level would undermine confidence in the currency of those grades.

Maintaining standards, and therefore public confidence in the grades, is one of the reasons for standardising between centres. The other reason – arguably the more important reason for this year’s students – is because we now know that the standard of the CAGs is not consistent across schools and colleges. Not all centres have been optimistic in their CAGs and those that have been were not uniformly optimistic. Some centres have been a little optimistic; others much more so. Simply using the CAGs to determine final grades would have been unfair.

For a great majority of students, particularly those taking A levels for university entry, they will be competing for places against students from other centres across the country, so it’s important that, as far as possible, they are competing on a level playing field. That’s why it is essential that we standardise the final grades.

The standardisation model

We have set out more detail on how the model works but it’s worth bearing in mind that it will work at individual subject level, so a centre might see more adjustments in some subjects than in others, or adjustments in some subjects but not in others.

As we’ve designed the model, we’ve had to make a number of decisions about the detail – how to manage small cohorts where statistics are less reliable, generating subject-level predictions, selecting historical data, for example. Given the exceptional circumstances this year, we have prioritised fairness for students over the precise alignment of standards with previous years. For this reason, wherever possible, we’ve taken decisions which will act in students’ favour.

As we set out at the Symposium, the combined effect of those decisions is likely to mean some leniency overall and we might expect to see more leniency in those subjects where the majority of centres have only a small number of students. As a result, we’re likely to see slightly more higher grades than in previous years, by around 2 percentage points at A level grade A and above, and by up to 1 percentage point at grade 4 and above at GCSE. But it’s important to be clear that the figures are not yet final. Exam boards are still doing their quality assurance checks and the final results issued in August could look a little different.

We’ve seen some accounts that suggest, for example, that because the CAGs appear to be optimistic by 12 per cent, and because we’ve suggested there will be some generosity of up to 2 per cent, that means all A level grades will be brought down by 10 per cent. That won’t be the case. There won’t be a ‘blanket adjustment’ to all.

The scale of adjustments will vary by centre, and by subject. Adjustments will be determined for each subject in each centre, and they will be based on the centre’s historical results and the prior attainment of this year’s students compared to those in previous years.

Some centres’ CAGs will be adjusted downwards to bring them into line with the centre’s predicted grade distribution in that subject – most by just one grade – and some will need to be adjusted upwards. However, the majority of grades won’t be adjusted.

In those centres where the number of students taking a subject is simply too small for us to standardise statistically, or where there is little or no historical data, CAGs have been the primary source of evidence to determine the final grades. For other centres, they are an important part of the overall quality assurance.

Equalities

We, like many others, have been concerned that the approach this year might exacerbate some of the gaps in attainment between different groups of students, particularly affecting students from certain ethnic minorities, those with special educational needs or from more disadvantaged backgrounds.

We produced guidance to help teachers identify and minimise any unconscious bias in their judgements, and we have been carefully monitoring the data. We presented early findings at the Symposium and it is reassuring to note that the gaps in attainment do not appear to have systematically increased.

Finally, a word about optimism. We do understand why teachers have tended to be optimistic in generating CAGs. In a normal year, they will have seen some students ‘pull it out of the bag’ on the day and they’ll have seen others do less well than expected.

In many of those borderline decisions about individual students this year, teachers have tended to act in the student’s favour. This optimism is entirely natural and should not, in any sense, be seen as teachers getting it wrong. In fact, since we expect that the majority of CAGs will not be adjusted, we believe the vast majority of those judgements have been made carefully and professionally.