How can we measure apprenticeship success and quality in a meaningful way?

Later this month AELP hosts its annual National Conference which gives people from across the skills and employability sector an opportunity to discuss and debate the hot topics affecting learners, employers and providers.

At last year’s conference, we saw the then Skills Minister, Alex Burghart, outline his ambition for achievement rates on apprenticeship standards to reach 67% overall by 2025. A year on, it’s time to revisit the topic of qualification achievement rates (QAR) and reopen the conversation about how we can accurately measure quality and success – while not losing sight of the government ambition to see higher completion rates.

Are we measuring the right thing?

As the existing methodology goes back to historic qualification-based apprenticeship frameworks, before the shift towards an employer-based system, the first question to ask is whether the current success rate methodology which informs the QAR is fit for purpose? Given qualification-based frameworks have, for the most part, now been phased out and replaced by longer apprenticeship standards, that answer is, I’m afraid, a resolute ‘no’.

Apprenticeship standards are longer and more technical and are based on knowledge, skills, and behaviours as well as including further external rigour through end point assessment. That alone brings about a need to look again at success rate methodology.

In addition to this, we currently face extremely tight labour market conditions which affects the way both learners and employers react. Any updated measure of success needs to acknowledge the reasons for non-completion that are out of the control of the provider, and those non-completions which are actually a positive outcome, for example an apprentice going on to a better paid job with more responsibilities.

AELP has consistently outlined its concerns on a renewed focus on QAR while the methodology remained so outdated. Until success is measured properly, focusing on targets which won’t be achieved risks undermining confidence in apprenticeships.

That’s why we’ve seen a dubious attempt by some to compare apprenticeship retention and achievement rates with those for A-Levels. They are, of course, in no way comparable at all! One is a full-time academic programme, the other, a job with training. How many students on an A-Level drop out because they get another job, get promoted, get let go or are refused the time they need to train?

Improving apprenticeship achievement measurements

We need to see a much wider set of measures assessed, including tracking learner progression and earnings following an apprenticeship. Focusing on these types of outcomes, rather than outputs, including for those who don’t complete their apprenticeship, should be a priority. I think we’re beginning to see some acknowledgement from the Department for Education that this is an important factor in determining success and quality within apprenticeships.

This time last year we saw some welcome changes to apprenticeship rules and measures. For example, we’ve had an increase to the period of change of employer, which now sits at up to 12 weeks through a break in learning. These haven’t gone far enough though and that means there are still remaining issues with the measures, with the next obvious step to take being to disaggregate achievements. The sheer range of sectors and occupations means there are big variations that need to be recognised and accounted for more easily.

As well as this, there are five other changes to the framework which we believe would deliver a more accurate framework for measuring apprenticeship success. These are:

1. Removing the QAR Pass Rate measure and replacing it with a pass rate that relates to end point assessment (EPA). End point assessment is a key facet of the reformed system and needs greater transparency and oversight.

• Extend data capture via the Individual Learner Record (ILR) to include a range of reasons for withdrawal (non-completion). The current data capture fields on the ILR are too narrow to fully collect enough data on the underlying reason for non-completion.

• Make the methodology more reflective of quality by removing withdrawals that are out of the providers control from the QAR measures, such as if the apprentice has been dismissed from their job.

• Expand the methodology to include wider success measures. In further education though longitudinal outcomes remains underdeveloped as a tool, measuring earnings and other indicators of success such as promotion and progression, would give a fuller picture.

• Develop a more inclusive set of accountability measures that better reflect the role of employers and their specific behaviours in the wider apprenticeship system. This should include capturing feedback from providers and apprentices on their experiences. 

Valuing apprenticeship achievement

All of this is important to measure achievement and quality fairly and accurately. That will lay the foundations for working out where support can be given to learners, employers and providers to help increase success rates. However, this alone won’t achieve greater completion rates. To do that we urgently need to raise the currency and value of apprenticeships themselves with learners and employers. That’s something our recent report, Raising the Standard, heavily lent in to.

The perception of the overall value of apprenticeships in the labour market in particular needs improving. Even amongst training providers, Raising the Standard found that apprenticeships are often not specified as a qualification level needed when recruiting for staff. That’s a missed opportunity to raise the value of apprenticeships with jobseekers, and in turn give them more incentive to complete their qualification.

More widely, Raising the Standard raises some really important themes and as a result has received a significant amount of traction beyond the usual sector and governmental interest – including from some prestigious academic institutions. Next month, AELP’s Dr Chihiro Kobayashi will present the report’s findings to a conference at the University of Oxford and the report is set to be presented at the 2023 European Association of Test Publishers Conference in Vienna this autumn.

The current system for measuring success is weighted far too much towards training providers and a more inclusive approach should be implemented which recognises the breadth and responsibilities of all stakeholders. At the same time the right balance is required to not unduly disincentive employers from engaging with the apprenticeship system. The earmarked changes above would modernise the way we measure success and quality within apprenticeships, giving both employers and learners transparency and a wide range of information as they make their choices on their training needs.

FE News on the go…

Welcome to FE News on the go, the podcast that delivers exclusive articles from the world of further education straight to your ears.

We are experimenting with Artificial Intelligence to make our exclusive articles even more accessible while also automating the process for our team of project managers.

In each episode, our thought leaders and sector influencers will delve into the most pressing issues facing the FE sector, offering their insights and analysis on the latest news, trends, and developments.