Is the calculated grades approach to summer examinations fair on students? Episode 1: #SkillsWorldLIVE

Presented by the Chief Executive of the Federation of @AwardingBodies, Tom Bewick, #SkillsWorldLIVE is a new radio show that builds on the popular #SkillsWorld podcast series, where Tom interviews leading figures shaping the post-compulsory education and skills systems, including apprenticeships in the UK, and across the world.

Episode 1: Is the calculated grades approach to summer examinations fair on students? – 23rd April 7-8pm (BST)

Guests include:

• Dr Gill Wyness, Associate Professor in Economics of Education, and Deputy Director of the Centre for Education Policy and Equalising Opportunities (CEPEO) at University College London
• Dr Fiona Summers, Head of Quality and Assessment at YMCA Awards
• Dr Rebecca Conway, Head of Policy and Strategy at Federation of Awarding Bodies (FAB)
• Mardy Leathers, Director, Office of Workforce Development at State of Missouri
• Lucy Dunleavy, CEO & Founder at LearnBox
• Lucy Hunte, National Programme Manager – Apprenticeships at NHS Health Education England

Songs include: Walking a tightrope – Mindme with Mia Pfirrman, Tired of Waiting – Hallman, Celebration of Life – Tomas Skyldeberg, Help you let it go – Deanz feat. Andy Delos Santos, and Makes me feel – Alex Dovo

Predicting A-level grades accurately ‘near-impossible task’

Predicting A-level grades is a ‘near-impossible task’, and the system needs to be overhauled to reduce inaccuracies that can lead to unfair disadvantages for some students, says Aug 2020 research from the UCL Institute of Education.

Listen to Dr Gill Wyness discuss the findings of her research here:

For the working paper published on Tuesday 11 August 2020, academics from UCL Centre for Education Policy & Equalising Opportunities (CEPEO) and Oxford Brookes Business School studied data from 238,898 pupils’ GCSE performance to see whether they could accurately predict their subsequent A-level results.

For the first time, they found that even by removing any opportunity for bias – and running additional checks on pupils’ gender, ethnicity and socioeconomic status – they could only predict one in four pupils’ best three A-levels correctly.

The researchers say the disruption COVID-19 has brought on formal examinations this year – with pupils instead assigned calculated grades by their teachers, which are then moderated by exam boards – highlights a wider problem with the general UK system.

The paper also shows that high-achieving comprehensive school pupils are more likely to be under-predicted compared to their grammar and private school counterparts.

Among high achievers, where under-prediction is most common, the team found 23% of comprehensive school pupils were underpredicted by two or more grades compared to just 11% of grammar and private school pupils.

One of the paper’s authors, and CEPEO Director, Professor Lindsey Macmillan (UCL Institute of Education), said:

“This research raises the question of why we use predicted grades at such a crucial part of our education system.

“This isn’t teachers’ fault – it’s a near-impossible task. Most worryingly there are implications for equity, as pupils in comprehensives are harder to predict.

“Our work shows that these pupils have more noisy trajectories from GCSE to A-level. If you’re a straight-A student at a grammar or private school, you’re more likely to continue that to A-levels. But this research is telling us there’s a lot more movement around the grades between the two exam levels for comprehensive students.”

Teacher predictions of pupil grades are a fundamental feature of the English education system, forming the basis of students’ university applications and determining the wider life chances of pupils in post-secondary education.

Research by one of the authors, CEPEO Deputy Director Dr Gill Wyness (UCL Institute of Education), earlier this year showed that only 16% of university applicants were correctly predicted across their best three A-levels, when comparing teachers’ predictions to students’ actual grades. Of the rest, 75% were overpredicted and just 8% underpredicted.

The same study found that the grades of high-achieving students from low socio-economic backgrounds were more likely to be underpredicted.

For this working paper, the researchers used statistical and machine learning approaches to analyse detailed administrative data on prior achievement as well as demographic and school-level information.

They found their modelling only made modest improvements to the accuracy of teacher predictions, increasing the success rate from one in five to one in four pupils’ grades correctly predicted.

Predictions were improved by including data on ‘related’ GCSEs – those A-level subjects that have an equivalent GCSE – showing that exam subjects themselves need to be taken into account alongside student achievement and school type.

Maths was easier to predict among high achievers than other subjects such as history and chemistry, but for average and low achievers, the opposite was true.

English Literature was most accurately predicted across all achievement levels, while Law predictions were the least accurate.

For subjects without related GCSEs, the task was even more challenging, with lower prediction rates across the board.

The researchers say only moving towards a post-qualification applications and admissions (PQA) system would help remove potential inequalities.

Co-author Dr Wyness said: “We definitely don’t think teacher predictions should be replaced by computer predictions – this research serves to highlight the difficulty faced by teachers, and provides further evidence that the UK’s predicted grades system should be re-examined.”

The research was based on National Pupil Database records for a cohort of state and privately educated pupils who took their A-levels in 2008.

Don’t miss Episode 2: Should the skills sector get its own bailout? – 24th Apil 7-8pm (BST)

Please let us know if there is an important topic you would like to debate that we haven’t included yet!

Tune in to #SkillsWorldLIVE 7-8pm Thursdays during term time and subscribe here for notifications!