Curriculum intent, implementation and impact: Refocusing Ofsted inspections on the quality of education

Amanda Spielman provides a commentary on phase 3 of Ofsted’s research into the school curriculum:

In January, I will consult on our new education inspection framework (EIF). As I have already announced, the heart of our proposals will be to refocus inspections on the quality of education, including curriculum intent, implementation and impact.

To ensure that inspection of the quality of education is valid and reliable, I commissioned a major, 2-year research study into the curriculum. I would like to thank the school leaders and teachers who have contributed to this work. We visited 40 schools in phase 1, 23 schools in phase 2 and now 64 schools in phase 3. When you add the focus groups, reviews of inspection reports and other methods, it’s clear that this is a significant study and we can be confident in its conclusions.

Read Amanda Spielman’s commentaries on Phase 1 and Phase 2 of Ofsted’s curriculum research.

To recap, in phase 1 of the research, we attempted to understand more about the current state of curricular thinking in schools. We found that many schools were teaching to the test and teaching a narrowed curriculum in pursuit of league table outcomes, rather than thinking about the careful sequencing of a broad range of knowledge and skills. This was disappointing but unsurprising. We have accepted that inspection itself is in part to blame. It has played too great a role in intensifying performance data rather than complementing it.

Having found that some schools lacked strong curricular thinking, phase 2 sought to look at the opposite – those schools that had invested in curriculum design and aimed to raise standards through the curriculum. Although we went to schools that had very different approaches to the curriculum, we found some common factors that appear related to curriculum quality:

• the importance of subjects as individual disciplines
• using the curriculum to address disadvantage and provide equality of opportunity
• regular curriculum review
• using the curriculum as the progression model
• intelligent use of assessment to inform curriculum design
• retrieval of core knowledge baked into the curriculum
• distributed curriculum leadership

In phase 3, which is the subject of this commentary, we wanted to find out how we might inspect aspects of curriculum quality, including whether the factors above can apply across a much broader range of schools.

Read the phase 3 report ‘Curriculum research: assessing intent, implementation and impact’.

We also wanted to move beyond just looking at curriculum intent to looking at how schools implemented that thinking and what outcomes it led to. There has been some debate since we published my commentary on phase 2 about whether this would lead to an Ofsted-approved curriculum model. However, to reiterate there will be no ‘Ofsted curriculum’. We will recognise a range of different approaches.

Phase 3 of our curriculum research shows that inspectors, school leaders and teachers from across a broad range of schools can indeed have professional, in-depth conversations about curriculum intent and implementation. Crucially, the evidence also shows that inspectors were able to make valid assessments of the quality of curriculum that a school is providing. Both parties could see the distinction between intent and implementation, and inspectors could see differences in curriculum quality between schools and also between subject departments within schools.

Importantly, what we also found was that schools can produce equally strong curricula regardless of the level of deprivation in their communities, which suggests that our new approach could be fairer to schools in disadvantaged areas. This is distinctly encouraging as we move towards the new inspection framework. You can read the full findings of this research study. I have summarised the research design and main findings below.

Curriculum study – phase 3

In phase 3, we wanted to design a model of curriculum assessment that could be used across all schools and test it to see whether it produced valid and reliable results. Based on the phase 2 findings, discussions with expert HMI and our review of the academic literature, we came up with several hypotheses (detailed in the full report) and 25 indicators of curriculum quality to test (detailed at the end of this commentary). These indicators will not be directly translated into the new inspection framework. First, they were only tested in schools, not early years provision or further education and skills providers. Second, 25 indicators is too many for inspectors to use on an inspection, especially given the short timescales of modern inspection practice. What we were aiming to do was first to prove the concept (i.e. that it is possible to make valid and reliable assessments of quality) and second, to find out which types of indicators did that most clearly.

The 25 indicators were underpinned by a structured and systematic set of instructions for inspectors about how to use them for the research. Using conversations with senior leaders and subject leaders and collecting first-hand evidence of implementation, inspectors were able to make focused assessments of schools against each of the indicators. Inspectors used a 5-point scale, where 5 was the highest, to help distance inspectors’ thinking from the usual Ofsted grades. The full descriptors are at the end of this commentary, but by way of illustration:

• a score of 5 means ‘this aspect of curriculum underpins/is central to the school’s work/embedded practice/may include examples of exceptional curriculum’
• a score of 1 means ‘this aspect is absent in curriculum design’

Within each school, inspectors looked at 4 different subjects: 1 core and 3 foundation. This allowed us to look at the level of consistency within each school, but also to find out more broadly which subjects, if any, had more advanced curricular thinking behind them. Inspectors also gave each school an overall banding, again from 5 to 1.

This gave us 71 data points for each school, based on all the evidence gathered. While this approach would not be suitable for an inspection, what it allowed us to do was to carry out statistical analyses to look at the validity of our research model and to refine and narrow the indicators to those that more clearly explained curriculum quality.

We visited 33 primary schools, 29 secondaries and 2 special schools. The sample was balanced in order to test the validity of our curriculum research model across a range of differing school contexts. The main selection criteria were: previous inspection judgements (outstanding, good and requires improvement (RI) only), geographical location (Ofsted regions) and school type (local authority (LA) maintained/academies), although we over-sampled for secondary schools and schools that were judged outstanding or RI at their last routine inspection. We ensured a wide spread in terms of performance data. Importantly, we also took care to select a range of institutions across an area-based index of deprivation. This meant that we had roughly equal numbers of schools in more and less deprived areas.

Primary/secondary

Figure 1: Curriculum overall banding by school phase

^{The 2 special schools are included in the secondary school data.}

Figure 1 shows a clear difference in the distribution of the overall bandings between primary and secondary schools. Only 8 out of 33 (around a quarter) primary schools scored highly, i.e. a 4 or a 5 overall, whereas 16 out of 29 secondaries (over half) did.

Only 18 out of 64 schools scored poorly, i.e. a 1 or a 2, which is more encouraging than our phase 1 research might have suggested.

When we dig down into the subject-level data, we can begin to see why this might be.

Figure 2: Indicator 6a by subject departments assessed during the 33 primary schools visits

^{Indicator 6a: The curriculum has sufficient depth and coverage of knowledge in subjects.   Includes 2 subject reviews conducted in the primary phase of an all through school.
Technology includes computer studies.}

Figure 2 shows that when we look at subject depth and coverage in primary curricula, there are few low scores for the core subjects – just 8 scores of 1 or 2 out of 46 assessments. English and mathematics scores are particularly good. This would appear to be a result of two factors.

First, especially in key stage 1, literacy and numeracy are extremely important. Children cannot access other subjects if they do not have those basic reading, writing and calculation skills. This was recognised, albeit imperfectly, in the national strategies, which have given rise to the current modus operandi of many primary schools: English and mathematics in the morning and everything else in the afternoon.

Second though, it is a truism that what gets measured gets done. English and mathematics are what are measured in primary schools. It is hardly surprising, then, that they get the most lesson time and most curricular attention from leaders. It is clearly possible to do this badly, as we found in phase 1 where some schools were practising SATs as early as Christmas in Year 6 and focusing on reading comprehension papers rather than actually encouraging children to read. However, our results here appear to suggest that many more primary schools are doing it well.

Unfortunately, the same cannot be said for the foundation subjects. It is disappointing to see so few higher scores in technology subjects, humanities and arts.

In phase 2 of our research, we saw that almost all of the primaries used topics or themes as their way of teaching the foundation subjects. However, the ones that were most invested in curriculum design had a clear focus on the subject knowledge to be learned in each subject and designed their topics around that. What appears to happen more often, though, is a selection of topics being taught that do not particularly link together or allow good coverage of and progression through the subjects. Figure 2 shows that 7 schools had a complete absence of curriculum design in humanities for example.

The picture appeared much stronger in secondary schools than in the primary schools we saw. There was considerably less difference between how well foundation subjects were being implemented compared with the core subjects.

Figure 3: Indicator 6a by subject departments assessed during the 29 secondary schools visits

^{Indicator 6a: The curriculum has sufficient depth and coverage of knowledge in subjects.  One of the secondary schools visited was an all through school. Only 2 subject reviews were carried out in the secondary phase for this visit.
Technology includes computer studies.}

Arts subjects (art, music and drama) appeared particularly strong, with 10 out of 13 arts departments scoring a 4 or a 5. However, some subjects were still being implemented weakly compared with English and mathematics. In modern foreign languages, many of the features of successful curriculum design and implementation were absent or limited due to the lack of subject specialists. History was also less well organised and implemented in a number of schools, often to the detriment of a clear progression model through the curriculum. A lack of subject expertise, especially in leadership roles, contributed to these weaknesses.

The research visits to the 2 special schools in the sample showed that the curriculum indicators worked equally well in this context. We are unable to give more detail about the curriculum quality in these schools as it would be possible to identify which schools we are referring to. This would violate our research protocol and our agreement with the schools.

Ofsted grade, disadvantage and progress

Figure 4: Curriculum overall banding by the overall effectiveness judgement of the schools visited at their last routine inspection

^{Overall effectiveness judgements are based on data at time of sampling.}

Figure 4 shows that there is a positive correlation between the banding schools received on their curriculum and their current Ofsted grade, albeit a weak one. This was being driven in part by relative curriculum weakness of the primary schools in the sample. The 3 schools that achieved the highest curriculum score all do have a current outstanding grade. However, we also assessed 9 outstanding schools as band 2 or 3 and 9 good schools as band 1 or 2.

It is worth remembering that some of the outstanding schools have not been inspected for over a decade, which means that the validity of that ’outstanding’ grade is not certain.

A quarter of the RI schools visited were assessed as band 4, a high score. We know that under the current system it is harder to get a good or outstanding grade if your test scores are low, even if this is primarily a result of a challenging or deprived intake. This research suggests that some RI schools may in fact have strong curricula and should be rewarded for that

Figure 5: Curriculum overall banding by the income deprivation affecting children index quintile of each school

^{Deprivation is based on the income deprivation affecting children index. The deprivation of a provider is based on the mean of the deprivation indices associated with the home post codes of the pupils attending the school rather than the location of the school itself. The schools are divided into 5 equal groups (quintiles), from ‘most deprived’ (quintile 5) to ‘least deprived’ (quintile 1).}

Figure 5 shows that there is no clear link between the deprivation levels of a school’s community and a school’s curriculum quality. In fact, there are more schools in the top 3 bands that are situated in the most deprived communities (69%) than there are in the least deprived (62%), although the numbers of schools in each quintile are small. This is encouraging as we move towards the new inspection framework. It suggests that a move away from using performance data as such a large part of the basis for judgement and towards using overall quality of education will allow us to reward schools in challenging circumstances that are raising standards through strong curricula, much more equitably.

Figure 6: Curriculum overall banding by key stage 2 and key stage 4 progress measure bandings

^{Key stage 4 progress 8 data (2018) and key stage 2 mathematics data (2017) have been merged for statistical disclosure control purposes. Proportion of schools in each banding differ between the various progress measures. Bandings and the key stage 4 progress measure encompasses more subjects than the single-subject measures used at key stage 2.Data for the 2 special schools is not included.
Schools with no data are newly opened schools or infant schools.
The progress bandings shown are based on the 5 progress measure bandings calculated by the Department for Education. ‘Above average’ combines ‘above average’ and ‘well above average’; ‘Below average’ combines ‘below average’ and ‘well below average’.}

Figure 6 shows the link between published progress data and the overall curriculum banding given to each school. The vast majority of those scoring a band 4 or 5 have above average or average progress scores (20 out of 24). But there are clearly some schools in our sample where despite having strong progress scores, we found their curriculum to be lacking. It would be wrong to speculate on the reasons for this, but it is clearly something that inspections under the new framework would look to explore.

Intent and implementation

We carried out further statistical analyses of our research model. Although we had 25 indicators in the model, our analysis showed that the relationship between the scores for each one and the relationship to the overall banding really boiled them down to 2 main factors: intent and implementation.

Figure 7 visualises this statistical model of curriculum quality and the relationship between intent and implementation. There is more detail on how we calculated this in the main report.

Figure 7: Scores for the intent and implementation indicators at the individual school-level, grouped by phase

Figure 8 expands on the model and shows that the scores for intent and implementation factors for most schools were well linked. However, there are some schools in the top-left and bottom-right sections in which inspectors were able to see a difference in quality between the intent and the implementation. This reinforces the conclusion above that intent and implementation can indeed be distinct. It should also dispel the suggestion by some commentators that our inspectors will be won over by schools that ‘talk a good game’ but do not put their intent into practice.

Most of the schools that scored well for intent but not so well for implementation (top left) were primaries. It is not hard to see primaries, particularly small ones, being less able to put their plans into action. It is difficult in many areas to recruit the right teachers. In small primaries, it is asking a lot of teachers to think about and teach the curriculum right across the range of subjects and even across year groups. Inspectors will of course consider these challenges when making their judgements.

Figure 8: List of curriculum indicators in the research model

In contrast, those schools that scored much better for implementation than for intent were all secondaries (bottom right). Again, it is not hard to imagine why that might be. Weak central leadership and lack of whole-school curriculum vision were more easily made up for in some of the secondary schools, particularly large ones, by strong heads of departments and strong teaching. This could also be a consequence of GCSE exam syllabuses playing the role of curricular thinking in the absence of a school’s own vision. This is another nuance that inspectors will deal with under the new framework.

The variation between intent and implementation scores suggests that our research model is valid. It appears to be assessing the right things, in the right way, to produce an accurate and useful assessment of curriculum intent and implementation.

Despite the fact that relatively few schools scored highly across the board on these measures, particularly at primary, it is worth reiterating our commitment to keeping the overall proportions of schools achieving each grade roughly the same between the old framework and the new framework. We are not ‘raising the bar’. That means explicitly that we will not be ‘downgrading’ vast numbers of primary or secondary schools. Instead, we recognise that curriculum thinking has been deprioritised in the system for too long, including by Ofsted. We do not expect to see this change overnight. The new framework represents a process of evolution rather than revolution. To set the benchmark too high would serve neither the sector nor pupils well. Instead, we will better recognise those schools in challenging circumstances that focus on delivering a rich and ambitious curriculum. At the same time, when we see schools excessively narrowing and gaming performance data, we will reflect that in their judgements.

Looking towards the EIF 2019

Through the autumn term, we have been piloting inspections under the proposed new framework. These pilots have drawn heavily from our curriculum research, including the indicators listed below. When we consult on our proposals in January, we will have behind us:

• the pilots
• our curriculum research
• research on lesson observation and work scrutiny
• an interrogation of the academic literature on educational effectiveness

The proposals will be detailed and firmly grounded in evidence.

It will be a full consultation and we genuinely welcome proposals for refinement. As the curriculum research has shown, there are still some challenges for us, particularly how we calibrate our judgement profile. Our aim over the spring term will be to listen to as many of you as possible, to address your concerns and hopefully hear your positivity about this new direction too. This research has given us a lot of confidence that our plans to look beyond data and assess the broader quality of education are achievable and necessary.

Figure 9: List of curriculum indicators in the research model

^{Indicators 1 to 2 are indicators framed around curriculum intent; 3 to 7 are implementation indicators and 8 to 9 relate to impact.}

Figure 10: Categories applied in the rubric for scoring the curriculum indicators

List of schools visited