Reviewing inter-subject comparability

Considering inter-subject comparability for science and language A levels.

Subject experts have a shared concern that A level entries in French, German and Spanish and physics, chemistry and biology could be suppressed if students and teachers believe it is harder to achieve top grades in these subjects compared to others.

Ofqual has examined these concerns, as well as a wide-range of other evidence, as part of a comprehensive programme of research it has undertaken into inter-subject comparability spanning the past 3 years. This has involved engaging the broadest possible range of interested parties, including Ofqual’s Board and Standards Advisory Group, higher education, subject experts, teachers and students.

We are today (Wednesday 21 November) publishing our decision not to adjust grading standards in these A level subjects, based on all the evidence. However, we recognise that perceived grading severity undermines confidence. Therefore, we have committed to working with the exam boards to ensure that these subjects do not become statistically more severely graded in the future.

After analysing an extensive base of statistical evidence and contextual data, and having considered a wide range of other evidence, including detailed representations from the subject communities, we have concluded that there is not a uniformly compelling case to adjust grading standards in these subjects.

However, we recognise stakeholders have concerns about the impact that the perception of grading severity may be having on take-up of these subjects; and in particular, acknowledge their concerns over the falling numbers studying modern foreign languages. Although we did not conclude that changing grading standards for the qualifications is justified, we will consider with exam boards how we should act to avoid the potential for these subjects to become statistically more difficult in the future. We will implement and review this as part of our normal approach to awarding.

Commenting on the decision, Dr Michelle Meadows, Executive Director Strategy, Risk and Research, said:

Public perceptions of the difficulty of these subjects is longstanding. However, adjusting grading standards is something we would only consider if there were a uniformly compelling case to do so. We have carefully considered the arguments of stakeholders and closely examined a broad range of evidence against our criteria. We have concluded there is no such uniformly compelling case to adjust grading standards in these subjects.

Tom Bramley, Director, Research Division, Cambridge Assessment, said:

Ofqual have thoroughly investigated this question from a range of angles, recognising that statistical evidence about the difficulty of different subjects is rarely conclusive and can contradict other legitimate sources of evidence.  Much of the concern about subject difficulty appears to be about the declining take-up of certain subjects. In my view lowering the grade boundaries is not the right way to make subjects more attractive to potential students.

Barnaby Lenon, chair of the Independent Schools Council, said:

It is to their credit that Ofqual has committed so much time to researching the thorny question of ‘are some subjects harder than others and if so what can be done about it?’ They conclude that adjusting grades is not the best option. Dumbing-down A level science grades, for example, could lead to students being taken onto degree courses for which they are not equipped.

Work undertaken

Following considerable discussion with the teaching community, higher education and subject associations about the perceived difficulty and potential grading severity of certain subjects, we considered the relative difficulty of 3 science (physics, chemistry and biology) and 3 modern foreign language (French, German and Spanish) subjects.

In looking at these A level subjects, we were building on a significant body of research first begun in 2015 to consider this complex issue. This included looking internationally at how other educational jurisdictions seek to achieve inter-subject comparability, and studies which examined some of the statistical models which attempt to measure the relative difficulty of different subjects. This work also took place in the context of a small adjustment we made to grading standards in French, German and Spanish A levels in 2017 to reflect research into the impact of native speakers taking these qualifications.

We considered an extensive evidence base including statistical measures of subject difficulty and stakeholder concerns (including responses from subject associations, exam boards and their senior examiners for these subjects, and representatives from HE institutions, including those directly responsible for teaching first year undergraduates). We also looked at contextual data (such as data on teacher supply, trends in A level entry and research into the motivations behind students’ subject choices).

We judged the evidence against 4 criteria, which we applied to determine whether there was a ‘compelling case’ to adjust the established grade standards in a subject. The criteria were:

• statistical measures of subject difficulty show evidence of persistent grading severity over several years
• persuasive evidence of the potential detrimental impact caused by severe grading on those who use the qualification, and on society at large, over several years
• evidence which shows that those who use the qualification and those responsible for maintaining the grading standard judge an adjustment to be acceptable
• likely benefit to users of the qualification and society as a whole from a change to grading standards must outweigh any potential negative effects

Our research included asking students about the things that influence their decisions about what to study. The evidence shows that perceptions of difficulty were not the most important factor for students, who considered enjoyment and perceived usefulness of the subject to be more so.

Our decisions document and reports provide further detail on the 6 A level subjects against these 4 criteria. We did not find sufficiently strong evidence under these criteria to support an adjustment to grading standards.

We will now look at the evidence for a potential adjustment to grading standards in GCSE French, German and Spanish. We committed to expand our work on inter-subject comparability to include GCSE languages earlier this year, in light of concerns from some stakeholders that the legacy qualifications were more severely graded than other subjects.