Ofqual’s approach to EQA and driving up quality in assessment

Simon Lebus spoke about @Ofqual’s role in the apprenticeship system.

Good morning and thank you for inviting me to speak to you at this year’s apprenticeship conference. It is a great honour to be the first speaker at this marathon five-day event and I am especially glad as it also gives me the opportunity to thank all of you for the huge effort that is being put in to ensuring that high quality apprenticeship assessments continue to be delivered to apprentices during this exceptionally difficult period.

Apprenticeships clearly have an important role as we embark on the process of building back better and I welcome the role that Ofqual plays in ensuring that the end-point assessments (EPAs) that are an important element of the overall apprenticeship programme are high quality and deliver the outcomes that employers expect. I say that as an organisation that is not only a provider of external quality assurance (EQA), but also as an organisation that has recruited apprentices who have gone on to become permanent members of staff and which has current apprentices who we hope will do the same.

The goal of our apprenticeship system must be for apprentices to qualify and progress with their careers. We recognise that delivering apprenticeships during 2020 and 2021 has been challenging for many sectors – and that the impact has been felt differently across different sectors. The challenges faced in the health care sector, hospitality and hair and beauty sectors will be very different from those industries that have been able to continue working or have otherwise been able to introduce flexibilities to assess remotely. Ofqual has worked with the Institute for Apprenticeships and Technical Education and with the Education and Skills Funding Agency to be as flexible as possible in these circumstances whilst maintaining high-quality assessments.

Today I plan to focus on our approach to EQA and what we do to drive up quality in assessment to achieve this goal, as well as how we are progressing with the transition of EQA to Ofqual from other providers.

And although this is a conference on apprenticeships and I’m here to talk about our role in EQA, I also want to take the opportunity to mention Functional Skills qualifications as I know there has been a lot of attention given to apprentices not being able to progress due to a backlog in these assessments due to the pandemic.

I am therefore glad that all awarding organisations have now been able to resume delivery of Functional Skills assessments with flexibilities in place to allow apprentices to progress when they are ready. These include remote assessment and invigilation options and the availability of teacher assessed grades where it is not possible for a learner to access an assessment. This should allow learners quickly to progress to the next stage in life whether that be completing their apprenticeship and moving into employment or further study.

Now moving on to apprenticeships and the transition of EQA to Ofqual, I am glad to report that we are well on the way to ensuring that the vast majority of apprentices are benefiting from regulated end-point assessments. This is a goal that has been discussed for a long time and which will be realised by the end of 2022.

At the time of last year’s Apprenticeship Conference, the Institute’s consultation on strengthening the EQA system was still live. Now, one year later, the outcomes of that consultation are well on the way to becoming a reality with the consequence that EQA will be delivered primarily by Ofqual (along with the Office for Students for integrated degree Apprenticeships) as the relevant regulatory bodies.

I am pleased therefore to be able to confirm that we are now the EQA provider for over 200 standards, with 112 of these added since the announcement was made. By the end of this year that will increase to over 300 standards and by the end of the transition period we will be responsible for the EQA of more than 500 standards, offered by potentially over 120 end-point assessment organisations (EPAOs). We currently have 27 applications for recognition from EPAOs under review and over 60 working on their applications for recognition, which we expect to be submitted soon.

EPAOs are engaging actively with this process, though I recognise that applications and recognition have been rather slower than we would have wished. This is perhaps no surprise given the other challenges many EPAOs have faced in dealing with COVID-19. Given this, we are glad that the Institute extended the deadline for EPAOs in phase 1 to apply for recognition. EPAOs who are delivering standards that are EQA’d by the Institute now have until 1 July 2021 to apply for recognition and until 16 December to complete the process. This gives EPAOs flexibility and certainty on the timescales for becoming recognised. If you are an EPAO listening to this, we recommend that you submit your application to us as soon as possible, and that you make use of your contact at Ofqual and the resources we have available to support you.

So far, we have focused on transferring standards where the Institute is the EQA provider. But we are now also working with the Institute and other EQA providers to plan the transfer of most other standards to Ofqual by the end of 2022. We are determined to make this transition as smooth as possible for the benefit of employers, apprentices and existing EQA providers.

We have therefore scaled up our resources to deal with the challenge that this transition and the EQA of many more standards will bring. We are confident we have the capacity and capability to deliver despite the challenges we have faced as an organisation over the last year and a half.

The guiding principle behind our approach to EQA is to ensure that there is a consistent quality approach to assessment across an apprenticeship standard, regardless of which EPAO is delivering the assessment and where and when it’s carried out. And that the end-point assessment delivers the knowledge, skills and behaviours that employers set out in the apprenticeship standard. All our EQA activity has this purpose in mind.

So, how do we go about this? When I first started at Ofqual in January I had the challenge of trying to understand what is, of course, quite a complex landscape. The basic principle, however, is straightforward. The Institute has the overall responsibility for the quality of Apprenticeships and owns the EQA Framework which sets out how EQA should be delivered and Ofqual exercises its regulatory powers within that framework.

The ability to use our regulatory powers is one of the key reasons that Ofqual EQA will strengthen and increase confidence in apprenticeship end-point assessments. We can, for example, direct an EPAO to do something or stop doing something, and we can fine organisations if they are found to be in breach of our rules. Our regulatory powers are therefore available to support quality and compliance by EPAOs though we would always look to work with them before deploying these powers to try to resolve problems. But it is always useful to have regulatory powers available to take further action if needed and the Institute’s framework supports this.

What this translates to in practice is a range of activities to support compliance and quality but with the power to act to put things right if they do go wrong.

The EQA Framework states that delivery of EPA should be carried out by organisations that are proven to have the right level of sector and assessment expertise. This means that all EPAOs who deliver EPAs for which Ofqual is the EQA provider need to be recognised by us. The recognition process is the means through which we assure employers and apprentices that EPAOs have the right resources, capacity and capability to design, develop and deliver quality EPAs. Our recognition process sets a consistent bar for all organisations that want to deliver end-point assessments in the market. Recognition is a crucial element of our process, as once recognised EPAOs become subject to our regulation.

In line with our overall approach, we provide EPAOs with appropriate guidance on our requirements and expectations when applying for recognition, but we don’t tell them how to do it as this could put us at risk of regulatory capture – essentially making judgements on our own advice if things go wrong and not acting in the best interests of the public.

One famous example of regulatory capture would be the financial regulators taking a ‘light touch’ approach in the years leading up to the global financial crisis of 2007 to 2009 and their failure to understand the trades that were taking place and the risks that the banks were taking.

I am glad to say any failure at Ofqual wouldn’t lead to a crisis on that scale, but the damage that would be done if we were unable to act effectively would have a serious impact on apprentices and on their opportunities, which is what we exist to protect. That’s why the recognition process for an EPAO is as stringent and thorough as that used for any awarding organisation that is seeking to be regulated by Ofqual.

Acknowledging the risks associated with regulatory capture doesn’t mean, however, that EPAOs aren’t given support. We have put a lot of resource into providing additional guidance to EPAOs this year, to help them understand our requirements, as we know that for many EPAOs regulation is new to them. From providing webinars and videos, to publishing updated guidance and introducing a single point of contact at Ofqual, we have tried to tailor our resources to areas where we know applicants struggle. And I would just like to take this opportunity to thank FAB, AELP and others for their support in providing the opportunity for us to engage with their EPAO members and to Autoexel, EMPI and EUIAS who, as recognised EPAOs, worked with us to create webinars to support the EPAO community.

As part of our ongoing monitoring we can undertake a review of end-point assessment materials – what we describe as technical evaluation. This is an important check to ensure that the assessment design proposed by the EPAO meets the requirements of the assessment plan, and that it is compliant with our rules. Following the technical evaluation of our first 50 sets of assessment materials, we published our findings on the common themes and issues we found in materials, to help support EPAOs avoid the pitfalls we identified. This work continues and we have now reviewed over 100 sets of materials, it is pleasing to note that we are seeing improvements in the materials we are reviewing following the feedback we have given.

But we don’t undertake technical evaluation in isolation as we recognise that industry expertise is essential in reviewing the occupational requirements outlined in the assessment plan. Experts and practitioners understand the subject context and requirements of their industry much more clearly than we can ever hope to. Industry experts work with our assessment experts and share their expertise on subject content, coverage and level and ensure that the employer voice in apprenticeships is maintained. We have looked at standards from poultry worker to high speed rail and the range of standards we review and experts we need will only grow as more standards transition to us.

As well as working with specialist practitioners, we also engage with a number of professional bodies and employer representative groups to understand how EPA is being delivered in different sectors and whether it meets the needs of employers.

For example, in the health and fitness sector, we are working closely with the Chartered Institute for the Management of Sport and Physical Activity and the Institute for Outdoor Learning, to understand the implications of COVID-19 on delivering assessments as the sector begins to re-open. This helps us to be aware of any potential risks to delivery, and makes us a better informed regulator. Building and maintaining these relationships between employer groups and EPAOs is fundamental to ensuring the right assessment outcomes in the relevant industry.

The Institute’s EQA Framework requires us to monitor delivery of EPA to check it provides a relevant, reliable and independent assessment of occupational competence. Monitoring enables us to identify good practice and where things are going well, pinpoint where there are areas that need to be addressed or improved, and to refine our understanding of risk. This is important as we take a risk-based approach to regulation, targeting our activities where we have greatest concern and where our work will have the greatest impact in preventing things from going wrong.

To support this monitoring activity, Ofqual is in the process of establishing a Field Team, which will provide direct engagement with EPAOs nationally. The purpose of the team is to gather intelligence across all aspects of an EPAO’s activity, including observation of live assessments. This will support a targeted and efficient use of our regulatory tools, and with the aim of helping us support continuous improvement in the apprenticeship sector. The initial work of the team has shown that the EPAOs engaged with have welcomed the direct and explorative approach we are taking, and have understood and appreciated the investment Ofqual is making in growing this function. In this spirit, EPAOs we have engaged with have been ready to provide access to their business and personnel, and have been willing to explain their approaches, systems and processes in detail, while being open about any risks they have identified that might threaten the validity and consistency of their EPAs.

Our rules require EPAOs to notify us of any issues in the delivery of their EPAs that might negatively affect their ability to deliver their end-point assessments. They also need to tell us what they will do to deal with the problem and prevent it from happening in the future. However, we aren’t just here when things go wrong. During COVID-19 we have worked with EPAOs on the flexibilities that they have had to put in place and provided advice to ensure consistent approaches as they moved towards more remote assessment. For example, when discretions were agreed for the Learning and Development standards to allow pre-gateway projects and witness testimonies in the end-point assessment, we worked with all the EPAOs delivering that standard to agree the parameters of the discretions applied so as to ensure consistent implementation and secure fair outcomes for apprentices.

We are also very aware in delivering our regulatory responsibilities of the need to make sure that we do not inadvertently stifle creativity or innovation, especially given some of the rapid changes in how we all operate arising out of COVID-19. This has impacted virtually every element of how we live our lives and has also had an impact on how qualifications are delivered and awarded. Among the changes has been the wider adoption of remote assessment and remote invigilation, as many assessments are unable to take place face-to-face. From the outset, we have sought to encourage awarding organisations to think creatively and to innovate. And they did most directly in considering the ways in which their assessments can be validly delivered remotely and how best to ensure that the risks of malpractice are mitigated through their chosen approach. When done well, remote assessment and remote invigilation can also bring convenience for providers and for apprentices, enabling assessments to take place at a time that suits an individual, as well as permitting the timely award of results that might otherwise have been delayed.

We will continue to work with EPAOs to encourage innovation in delivery where possible and make sure that this is at the heart of our regulatory approach.

I want to talk finally about how we work with the Institute, as it is perhaps one of the most important ways in which we can help secure quality in apprenticeship end-point assessments. Our approach is based on collaboration and includes sharing information and findings, particularly if we have concerns that the validity of end-point assessments is in jeopardy or if the interests of apprentices are being put at risk.

We are also active members the Institute’s Quality Alliance with the ESFA, Ofsted, the Quality Assurance Agency and the Office for Students. Collaboration between us all is essential to ensure quality remains at the heart of the apprenticeship system.

And finally, one of the key areas where we are working with the Institute at the moment is the development of an Employer Directory. The Employer Directory is owned by the Institute and will include professional and employer-led bodies that have a role in standards and professional competency for their industry. In addition to using the Employer Directory to help us to find the experts we need to carry out EQA activities we are also hoping also to use it to source employer expertise which we can deploy in our monitoring approach as part of our field work in the future. We are also, right now, running a pilot with the Institute on using the Employer Directory to help us find experts for our next round of technical evaluation work, which includes standards such as advertising and media executive and veterinary nurses. Having the Employer Directory to draw on directly to support and inform our EQA work will ensure we can access experts from the relevant industry when we need them. It is crucial that we get this important piece of the EQA puzzle right to ensure that employers continue to play a pivotal role in the quality of apprenticeships.

That has been a brief – and I hope useful – survey of the different ways in which Ofqual is approaching its responsibility for EQA to support the Institute in its delivery of apprenticeship reform. Thank you very much for listening and I hope you enjoy the rest of the conference.