EPA and Ofqual, Learning lessons from 2023 – Part 2

Continuing my theme of learning lessons from 2023, here are my next set of hints and tips that may help the EpAO sector. Whilst some suggestions may sound simple, getting the basics right provides the building blocks for future growth and ongoing compliance:

1, Check your staffing and governance structures remain compliant

I have worked with many EpAOs that have expanded since first becoming recognised. As part of the growth some have recruited more staff or made changes to roles and responsibilities. These changes are fabulous for ensuring that the EpAO has sufficient resources (Condition A5), however, I would recommend making sure that you check that any changes remain compliant with the conditions around governance and conflicts of interest. For example, making sure that you have the appropriate balance of responsibility and accountability, and that if adjustments are made to roles, you can still ensure you have persons with no personal interest involved with investigations.

2, Remember to update your polices to reflect the ESFA / DfE latest conditions

In 2023 the register of EpAOs was replaced with the APAR, and along with this the conditions were refreshed.  If you have sections within your policies which refer to the ESFA (DfE) conditions, make sure you update them to the latest version and check for ongoing compliance.  I wrote an article comparing the old and the new conditions which readers may find helpful.  Don’t forget to record any changes and updates to your policies in a version log, and make sure your staff are kept up to speed with / trained in any changes. It may also be worth considering adding your UKPRN number to some of your documentation, purely because the new APAR does not use EPAO numbers (a mystery to me as training providers need the EPAO number in order to complete the ILR).

3, Explore rationalisation and efficiencies

Many EpAOs, as they transitioned to Ofqual, created new policies / procedures / strategies to meet the Ofqual Conditions.  As a result, I have seen examples where an EpAO has policies that were written to meet the ESFA application requirements and policies that have been written to meet the Ofqual requirements, resulting in sometimes 2, or even 3 policies, that could be combined and refreshed.

Another observation, call it semantics, relates to the title – is it a policy, strategy, process, or procedure?:

The ESFA application required:

(a) Policies – internal audit, safeguarding, prevent, conflict of interest, complaints and appeals, and fair access;

(b) Procedures / process / strategy – monitoring procedures; moderation process, and consistency assurance strategy.

The Ofqual Conditions require:

(a) Policies: – conflict of interest, prior learning, invoicing,  and for those with Centres a sanctions policy.

(b) Strategy: – Centre Assessment Standards Scrutiny strategy (for those with Centres),

(c) Procedures for: – the investigation of suspected or alleged malpractice or maladministration, and for complaints;

(d) Processes for: –  appeals, and endorsement;

(e) Arrangements: (i) for making Reasonable Adjustments and special considerations, (ii) for the retention of data; (iii) which will ensure that sufficient technical equipment and support is available to it at all times; (iv) which will ensure that it retains at all times a Workforce of appropriate size and competence; (v) with third parties and Centres; (vi) to ensure that, as far as possible, the criteria against which Learners’ performance will be differentiated are understood by Assessors and accurately applied, and applied consistently by Assessors, regardless of the identity of the Assessor, Learner or Centre.

4, Be clear on your approach to moderation

In my previous article I talked about providing clear definitions so as not to get confused between Ofqual requirements and assessment plan requirements; well here is another example.  The Ofqual conditions only refer to moderation in the context of when ‘Centres’ are used (Condition C and H). However, many assessment plans refer to the moderation of end-point assessment. Ofqual must audit / technically evaluate EpAOs compliance against the assessment plan, and as such if an EpAO has assessment plans that refer to moderation, an EpAO must explain what that approach is and what it means, even if they do not have ‘Centres’ (Ofqual definition). So I would recommend checking your definitions and approaches so that it is clear what is meant by moderation and how it is applied.

5, Make sure you have robust processes for the management of third parties

Many EpAOs use third parties to support the operation of their service, such as employers or training providers providing the venue / equipment for assessment. When an EpAO does use third parties they must make sure those arrangements are clear because ultimately it is the EpAO that remains accountable to the regulator. This also means that the EpAO should have clear processes in place to monitor the third parties to ensure that they are operating as per the arrangements, and to take action where the actions / provisions of the third party do not comply with those arrangements.

6, Professional Discussions and Interviews

This has been a huge area of challenge during 2023. Many EpAOs are reporting that the assessment plan requirements are not fit for purpose. Examples include:

(a) timeframes that cause adverse effects on the learner, such as specifying a minimum time even if the apprentice has covered all the required KSBs in less than the time specified; or specifying a maximum time and a large number of KSBs to be covered that are all but impossible to cover in the time permitted;

(b) assessment plans requiring question banks for a professional discussion that is then described as dialogue and discussion;

(c) IfATE descriptors of professional discussions and interviews not matching the assessment plan descriptors.

Whilst I can’t solve the issues with the assessment plans, you must remember that the assessment plan is sacrosanct (refer to my previous article). If it proves impossible to design, or deliver, to the assessment plan, you must report it to Ofqual (and the trailblazer group if they still exist) and go from there, which includes capturing it as part of your risk management process.

7, Bundling

Whilst bundling is an ESFA condition, it is still very relevant to Ofqual because of condition B5.2 which states “An awarding organisation must not (and must take all reasonable steps to ensure that any person connected with it does not) advertise or promote its qualifications in a manner that is likely to be misleading to Users of qualifications”.  Regardless of whether you agree with the ESFA condition, it is there, so EpAOs must adhere to it.  It is a major issue which EpAOs have been raising for years.

8, Mapping and grammatical accuracy

Make sure all your assessment question banks, tasks and activities are clearly mapped to the KSBs of the standard and assessment plan, and that you have a clearly defined methods of question and/or task selection that: (a) ensure full KSB coverage; (b) provide different assessments for resits, and (c) minimise the risk of predictability. Linked to questions and tasks, don’t forget that grammatical accuracy and vocabulary is as important as technical accuracy. 

I have reviewed some questions banks, and have been surprised about how much value I have been able to add when focussed purely on the grammar and vocabulary.  So if you are reviewing your questions banks, have a check to make sure you are checking for accuracy across both technical details and language. Another aspect linked to this is to make sure you monitor trends across your questions and apprentice answers to enable you to spot potential issues. For example, are some questions being answered incorrectly more often than others, and if yes, could it be down to the language used within the questions (and answers if multiple choice)?

9, Don’t ignore Generative AI

The possibilities and opportunities associated with generative AI are endless, but as with all new things and innovations, you must be mindful of the risk as well as opportunities.  As an EpAO make sure you have a whole organisation approach to its use from design and development through delivery and onto quality assurance.  I wrote an article on this last year, and Ofqual have been carrying out research, with many questions about it on their annual statement of compliance, so expect more on this from the regulator in the future.

10, Remember that the security of assessment materials is as important as the security of personal data

We are often so carried away (rightly so) with the protection of personal data under GDPR, that we can forget that the protection of assessment materials is equally as important.  As an EpAO, it’s worth double checking that your GDPR / security / data protection / data storage / confidentiality policies (all EpAOs have different names for the documents) cover the protection of assessment materials as well as the protection of personal data. Within that, don’t forget to be clear on who can and who cannot have access to your confidential materials within the organisation (condition A1.2, A5.2b, G4, and EPA3.2b).

On a final note, remember that Ofqual are human, their views will be based on the perspectives of people. This means that when they come back with their observations/views/perspectives following review/evaluation/portal queries you have the right to push back, if you can demonstrate that those perspectives are incorrect or unfounded. I have worked with EpAOs that have successfully pushed back when Ofqual have claimed that a particular assessment question is not valid, or where an approach does not appear compliant.  However, if you do this it must be fact based and auditable; challenging for the sake of it is not productive or helpful.