Lifting the lid on external quality assurance of end-point assessment
An insight into external quality assurance of end-point assessment
This article is the first in a series of articles on the External Quality Assurance of End-point Assessment of apprenticeship standards.
The series will aim to enable the reader to understand the background and context of EQA, and then, through a series of interviews with EQA providers, understand the approaches and perspectives of EQA providers.
Firstly, let’s start with a little background
The April 2017 DfE Strategic Guidance to the Institute for Apprenticeships (referred to as “the Institute”) stated that one of the core functions of the Institute was to ensure all end-point assessments are quality assured, including quality assuring some itself (known as External Quality Assurance).
The document then set out the options for quality assurance that sat alongside the Institute quality assuring some themselves. The chosen quality assurance option sits with the employer group designing the apprenticeship standard and is named in the assessment plans.
These options were, and remain, as follows:
- an employer led model,
- a professional body,
- Ofqual, or
- the Institute
This means that the Institute has both an overarching role on EQA, as well as in the delivery of EQA itself for some standards. Where the institute provides EQA, the strategic guidance was clear that there must be separation between their overarching role and that of delivery of its own EQA. That separation has been made through the contracting of its EQA delivery to Open Awards.
The 2017 strategic guidance stated that it was up to the Institute to decide what form its EQA took, but that they should ensure that effective and high-quality EQA processes are available and are applied to all end-point assessments so as to ensure consistent, valid assessment and require a high standard from all apprentices.
In the March 2019 DfE Strategic Guidance to the Institute for Apprenticeships and Technical Education, the Institute, was asked to “review and strengthen its framework for EQA of apprenticeship assessment, setting out requirements for all bodies delivering EQA.”
Furthermore, the House of Commons Committee of Public Accounts in May 2019 recommended that the Institute write to them within 6 months to “set out what they will do to streamline and strengthen quality assurance arrangements in order to give greater confidence that end-point assessments are robust, fair and consistent.”
I understand that the EQA framework from the Institute is due out soon.
What is the role of the EQA Provider?
The Institute website summarises the 3 areas that should be explored by EQA providers:
- Standards and assessment plans (for example, checking they are fit-for-purpose, that the assessment methods are valid and align to KSBs, and independence)
- End-point Assessment (for example, assessment operating effectively, grading is accurate and consistent, fair access, assessor competency and sufficiency, gateway)
- End-point Assessment Organisations (for example, robust IQA and data collection, storage/sharing processes; effective resits, retakes, appeals, complaints handling; timeliness)
The latest quality statement from the Institute confirms that EQA is there to ensure that end-point assessment is valid, reliable, manageable and independent, providing a high-quality experience that allows apprentices to showcase their competence and gives confidence that they have achieved the right standard.
What can an EQA provider charge?
The Institute is clear that EQA should be a not-for-profit service. The charging structures vary considerably across EQA providers, and it depends on what the EQA provider quoted within their original proposal (EQA methodology). Part of this calculation will have been based around anticipated apprentice volumes. I will be looking into this in greater detail in my follow up articles/interviews with EQA providers.
A review of the published letters of recognition to EQA providers shows that the majority charge a per apprentice fee, ranging from £40 to £179 per apprentice (average of £70 per apprentice for those that charge a ‘per apprentice’ fee). There are some that do not make any charges and others that apply an annual fee to the End-point assessment organisation, instead of, or in addition to, a per apprentice fee.
EQA providers are be required to submit an annual financial report to the Institute for EQA activities evidencing income and expenditure.
Reporting and Action – Ofqual
Ofqual is the only statutory regulator for non-degree apprenticeships. Ofqual treat End-point assessments as qualifications and all EpAOs regulated by them must be recognised by them. Ofqual have the power to
intervene where there are issues with the quality of assessments, and in extreme cases issue fines or withdraw recognition.
Reporting and Action, all other EQA providers
All recognised EQA providers, in accordance with their recognition letters, are required to produce an annual report for each end-point assessment organisation on the register for the standard they are responsible for externally quality assuring.
This report must provide an overall judgment on compliance, data, strengths, recommendations, assessment validity, reliability, comparability and independence, internal quality assurance and costs. This report will be shared with the end-point assessment organisation and the Institute.
In addition, the EQA provider must produce a report (at least once a year) on the quality of delivery and fitness-for purpose of the Standard and Assessment Plan including a summary across all relevant End-point Assessment Organisations. Templates for these reports can be found on the Institute website. If an EQA provider identifies a serious issue, such as fraud or malpractice, it must be reported to the Institute immediately.
The reports allow the Institute’s Quality Assurance Committee to take decisions on what improvements might be required either by individual End-point Assessment Organisations, or across the standard as a whole. The Institute may make reports or elements of them publicly available. If the EQA provider notified the Institute of a serious issue or concern, the Institute Board or Quality Assurance Committee may decide that a review is needed.
I will be looking in more detail about how each EQA operates in my follow up articles and interviews with EQA providers.
The current picture
Based on the 20th May 2019 spreadsheet of apprenticeship standards published on the Institute for Apprenticeships and Technical Education website, there are 35 named EQA providers, across the 4 options listed previously. Of these, 17 are providers are marked as “TBC” (to be confirmed).
Using this same spreadsheet there are 449 standards approved for delivery, of which 353 (79%) have a named EQA provider. The remaining 96 of the standards have the EQA provider marked as “TBC”.
Of those with a named EQA provider, 202 (57%) are the Institute for Apprenticeships and Technical Education, 64 (18%) are Ofqual, and 87 (25%) are either an employer led or a professional body EQA provider.
The proportion may seem high for the Institute given the statement in the April 2017 DfE Strategic Guidance paragraph 29 “We would expect the Institute to be named as the EQA organisation only in instances where alternatives are not viable”. However, this may be explained by the preceding paragraphs in the same guidance which stated that “an EQA option must be included in each assessment plan, selected by the employer group, before it can be approved”, and that “the Institute was added as an option to all to avoid assessment plans, and therefore starts, being held up”.
The percentage with the Institute named as their EQA provider is falling, from a high of high of 56% of standards approved in 2015, to 40% of standards approved in 2019.
Of those marked as “TBC”, 55% have an EQA provider name written next to the “TBC”. The largest proportion of those is QAA (31 of the 96 standards). This probably explains why 52% of the standards with the EQA provider marked as “TBC” are at level 6 and 7. However, please note that, as at the time of writing, I understand that an announcement with regard to the quality assurance of level 6 and 7 is due out soon. The sectors with the highest volume of standards with the EQA provider marked as TBC are the Engineering and Manufacturing, Construction and Health and Science sectors.
Jacqui Molkenthin, JEML Consulting
I hope you have found this background article useful. Over the next few weeks I will be writing articles and adding videos and podcasts on EQA following interviews with a range of EQA providers. If you are an EQA provider, who is not already booked in for an interview, and you are interested in being interviewed, please do get in touch.
Responses