Subject Improvement

Over the years I think we’ve overlooked the role that subject improvement plays in school improvement. Perhaps in the days of retakes, early entry, coursework, ECDL etc we could use generic management approaches to raise outcomes in schools. No longer!

improvement

With increasing numbers of pupils following a general academic curriculum, and with reforms to assessment leading to longer, terminal exams, school improvement now requires pupils to demonstrate a good understanding of their subjects, an understanding that can only really be gained through 5 years of a coherent curriculum and good teaching in each subject.

It’s for this reason that subject improvement is at the heart of our approach to school improvement at United Learning. I explain our school improvement strategy as follows:

Our Regional Directors ensure that the basic foundations are in place in all schools: leadership, curriculum, behaviour and teacher development. With these foundations in place, our subject advisors ensure that subject specialism thrives in each school.

I manage our team of subject advisors who all have the same objectives:

  • Raise standards in your subject: We believe that the most important thing we can do for our pupils is to send them out into the world with a good set of grades. We therefore provide sharply focused guidance to heads of department and teachers so that pupils make good progress over the 5 years and end up with good outcomes
  • Build the capacity of teachers and leaders in your subject: For our subject advisors their team is their Heads of Department – they meet with them regularly and visit them in their schools to support their subject leadership.
  • Develop curriculum and assessment resources in your subject: As a large group of schools we are able to develop a central bank of teaching resources that ease teachers’ planning and ensure that all of our pupils benefit from a consistently challenging classroom experience.

Underpinning these is a fourth point about communication: the advisor is the advocate for their subject and communicates clearly and confidently so that headteachers and line managers know what they can do to support the subject in their school.

We’re currently recruiting a third advisor in each of the core subjects: English, maths and science. Details HERE. If successful, you’ll see parts of the country you never knew existed. More importantly, you will be immersed in school improvement while retaining your subject specialism, restoring subject improvement to its rightful place at the heart of school improvement.

Advertisements

Still Fighting the Last War?

They say that we over-estimate the impact of big changes in the short term, but under-estimate their impact in the long run. So in the 90s those who thought that mobile phones were going to change the world might have initially doubted themselves. And those who now scoff at the impact of driver-less cars might want to wait twenty years before getting too smug.

We’ve had our own ‘big change’ in education in recent years, as reforms to curriculum and assessment have increased the amount that pupils need to know in each subject. Back in August 2015, a few days after a volatile set of GCSE results in schools across the country, I wrote the post below. The post argued that some schools and heads were still seeking improvement by squeezing kids over last year’s grade boundaries, rather than gradually gaining a proper understanding of their subjects over five years. The low hanging fruit within reach of tactical approaches to school improvement (early entry, ECDL, iGCSE etc) gradually ran out, and schools that relied on them suffered.

tank-war-armour-heavy-64239

The 2019 Ofsted framework opens up a new front in the war against tactical school improvement. With the outcomes judgement now subsumed within a broader ‘quality of education’ judgment, Ofsted has placed more weight on what pupils learn over 5 years, rather than the grades they walk away with. Last month at the Wonder Years conference The Chief Inspector suggested that ‘getting a grade 3 in history GCSE may ultimately prove more beneficial than a Merit in a BTEC’. She might have completed this sentence with ‘even if the BTEC makes the school look better in the league tables’.

A few years ago if you went to a headteachers’ conference you would hear talk of the latest quick win that could be shoe-horned into the curriculum late in Y11 to score a few league table points. Times have changed. Last week in our headteachers’ meetings we discussed how to give more time for departments to meet together to share subject knowledge, and how to organise our curriculum to ensure that pupils revisit prior content. Earlier in the year we heard from educational psychologist Paul Kirschner on how our knowledge of how we learn should affect how we teach.

Longer, tougher exams that can’t be retaken didn’t change our education system overnight. But like mobile phones and driver-less cars, their impact in the long run is proving to be profound.

 

Fighting the Last War – Reflections on 2015 GCSE Exam Results

August 27, 2015 steveadcock81

National measures might have remained stable in last week’s GCSE results, but this stability hides significant volatility amongst schools serving lower attaining intakes.  Many schools that have become accustomed to strong outcomes based on intense intervention struggled to get students over the line this year.

As I reflected on these results the line that stuck in my head was that too many schools are fighting the last war.  In too many classrooms, teaching Year 11 involves helping students to pass the previous year’s exam.  In too many revision sessions, students are coached to creep over thresholds based on the previous year’s grade boundaries.  This approach might have worked in the past but it’s ill-suited to the new landscape in which we find ourselves.

This new landscape includes longer exams with tougher questions; questions which require students to have a solid foundational understanding of their subjects.  It’s easy to say, but we need to nurture mathematicians in our schools, rather than spend our time helping students hunt around for the easier marks to scrape a C grade on a GCSE Maths paper.

In this new landscape there’s no place for the props on which so much GCSE ‘success’ was built: early and repeated entry and a strong reliance on vocational equivalences.  Even iGCSE English is no longer the safe bet that it was.  It’s a shame that so many students’ apparent success in English was actually based on meticulous preparation for speaking & listening and controlled assessments, followed by a few easy marks in the final exam to take them over the line.  This had become a trusted method for securing C grades with challenging students, but this year it failed to deliver.

Meanwhile in Maths, a rise in the grade boundary for a C grade at one popular exam board was met with consternation by many teachers who felt aggrieved that they had done their bit to get students over the line, only for the exam board to change the rules at the last minute.  Laura McInerney was right to tweet “a higher grade boundary does not mean it was harder to get a certain grade; that’s not how it works” but the sad reality is that many students had been taught how to score a C grade against last year’s grade boundaries.  The sense of entitlement felt by many teachers that a score which gained a C last year should gain a C this year shows the paucity of assessment awareness in our profession.

The last war was fought by directing a huge proportion of a school’s energy on Year 11.  The best teachers, the best classrooms, the best heads of year were allocated to Year 11.  Schools opened their doors after school, at weekends and in the holidays to deliver intervention for Year 11.  Some schools paid for the whole year group to attend residential boot camps, while others brought in external tutors and swat teams for holiday revision programmes.  One company guarantees to raise every student’s maths grade in a week, or you get your money back.

The law of diminishing returns is now dampening the impact of Year 11 intervention.  These tactics, tips and techniques to get students over the line have become a victim of their own success in the (almost) zero sum game of national exams.  While schools will always pay close attention to the fine tuning that gives students the best chance to do well in exams, we need, at last, to play the long game and invest in high quality teaching in every subject and every year group.

No longer should a forensic knowledge of exam tactics be a prerequisite for school leadership.  In turn, school leaders need to be given time to turn their schools around, rather than feeling pressured to make superficial year-on-year gains based on intensive coaching of borderline students.  The introduction of Progress 8 is an opportunity to re-focus on every student and every grade.

Year 11 intervention is becoming our Trident nuclear defence system – a costly relic of a war that is no longer relevant, and a distraction from the threats and opportunities in the here and now.

The Rosenshine Papers

Why Rosenshine?

In 2018 we (United Learning) adopted Rosenshine’s principles of instruction as the basis for our approach to teaching and learning across our schools. It’s the first time that we’ve taken a collective position on teaching and learning, rather than leaving this critical issue to each school. Our focus previously was on supporting each school in having an internally coherent and effective T&L strategy. With the adoption of the Rosenshine principles we were attempting to go a step further by ensuring that each school’s approach was anchored in a shared understanding of the characteristics of effective teaching.

We did this for a few reasons. Firstly, we wanted to support schools in challenging approaches to teaching that are not supported by good evidence, such as teaching which is overly driven by the exam specification, teaching that is founded on the belief that pupils learn better by discovering things for themselves, teaching that takes differentiation too far by placing different groups of pupils on different ‘tracks’ in the same lesson, and teaching that is overly focused on securing evidence of progress in each lesson, rather than gradually building a secure long-term understanding of each subject.

As a growing Trust, and a Trust that comprises primary and secondary schools in the state and independent sector, as well as an initial teacher training programme, we could see benefits in building a shared understanding of the characteristics of effective teaching. A trainee teacher could leave their summer institute and arrive at their school in September safe in the knowledge that the philosophy towards teaching and learning would be consistent; a deputy head leading on teaching and learning could share resources with counterparts in our other schools; subject advisors could produce curriculum materials confident that they would be applied in the classroom in similar ways. We would move from each school having an internally coherent approach to teaching and learning, towards a coherent approach across the whole group which would serve as a foundation for great teaching in each school and each subject.

Over time we are using the principles to develop a shared and precise language for the way we talk about teaching and learning. In my experience, the language commonly used to describe teaching and learning is anything but precise. Obvious examples would be phrases such as ‘the lesson lacked a bit of oomph’ or ‘pupils weren’t fully engaged’ or, more positively, the lesson featured ‘awe and wonder’. But even terms that seem more clear such as ‘pace’ and ‘challenge’ can lack the precision required to develop teaching practice. Take ‘pace’ – do we mean that the teacher went through things too slowly or that pupils didn’t work quickly enough, or perhaps the teacher wasn’t clear on timings, or maybe the start of the lesson drifted and time was squeezed for the challenging stuff at the end? That leads us to ‘challenge’ – was the content itself too easy, or was it the task, or are we simply saying that not enough pupils produced work at the standard required?

We chose the Rosenshine principles because they’re sensible, evidence-informed and provide the shared foundation we were seeking rather than a rigid checklist to be applied to every lesson. As an established set of principles we were able to avoid a long process of navel-gazing which would inevitably have been required if we had attempted to write our own. The fact they’ve been around for a while also enabled us to reassure our schools that we would commit to these principles for several years ahead, rather than replace them with a passing fad in twelve months’ time.

We’ve got a long way to go, but we’re seeing some early fruits of our labour.  I write this while returning from an inset day in Shoreham where all teachers from four of our schools started 2019 by gathering together to explore the principles in the context of their own subject. Meanwhile our subject advisors have written case studies on how to apply these principles in their subject. The curriculum resources we are producing contain the modelling, the question prompts and the scaffolds that Rosenshine promotes in his work.

So what might Rosenshine look like in the classroom?

As we’ve worked with schools in exploring Rosenshine’s work we’ve confronted the question of what his principles look like in the classroom. I’m in two minds here as to how usefully Rosenshine presented his research. On the one hand, I’m grateful that his principles are contained in short, concise pamphlets such as this 2012 one and this 2010 one. One of the simplest things we’ve done is simply ask schools to ensure that all teachers read all 9 pages of the 2012 paper.

But I do have a few gripes with the way Rosenshine presented his work. Firstly, the 2012 paper contains a list of 17 principles alongside the main list of 10. Rosenshine explains this decision (the list of 17 provides slightly more detail and overlaps with the list of 10) but given Rosenshine’s knowledge of the limits of working memory and cognitive load, it seems slightly curious to share two separate lists alongside each other.

We can take this overlap as a reminder that the principles do not seek to provide a checklist to be followed in order in every lesson. This becomes clear when we note his sub-heading for point 6 (check for student understanding): “checking for student understanding at each point can help students learn the material with fewer errors” (my emphasis). So – to be clear – we don’t check for understanding between point 5 (guide student practice) and point 7 (obtain a high success rate), we check for understanding throughout the whole process. Tom Sherrington has noted that this becomes clear when we read Rosenshine’s 1986 and 1982 papers which emphasise the importance of checking for understanding.

The 1982 paper also helps us understand Rosenshine’s intentions in proposing the principles:

dogma

There’s another gem lurking in his earlier papers that I think gets lost in the latter versions. In his 1986 teaching functions paper Rosenshine writes:

“Three of these functions form the instructional core: demonstration, guided practice, and independent practice. The first step is the demonstration of what is to be learned. This is followed by guided student practice in which the teacher leads the students in practice, provides prompts, checks for understanding, and provides corrections and repetition. When students are firm in their initial learning, the teacher moves them to independent practice where the students work with less guidance. The objective of the independent practice is to provide sufficient practice so that students achieve overlearning (Brophy, 1982) and demonstrate quickness and competence. A simple version of this core is used frequently in the elementary grades when a teacher says: “I’ll say it first, then you’ll say it with me, and then you’ll say it by yourself”.”

This seems like critical guidance, and helps us to understand the intention behind Rosenshine’s principles, which I think we can now summarise as:

  • Prior review
  • Instructional core (I>we>you):
    • Presentation and modelling of new material in small steps
    • Guided practice with prompts and scaffolds
    • Independent practice with monitoring and feedback from teacher
  • Future review

At each of these points – every single one of them – we check the understanding of all pupils by asking lots of questions and providing correction and feedback.

This model – the instructional core sandwiched between prior review and future review, with checking for understanding at each point – captures the essence of Rosenshine’s principles of instruction and provides an answer to that question of what Rosenshine looks like in the classroom.

Rosenshine’s back catalogue also helps us understand his 7th principle ‘Obtain a high success rate’.  In his 1986 Teaching Functions paper he writes: “Although there are no scientific guidelines as to exactly what the percentage of correct answers should be, a reasonable recommendation at the present time (suggested by Brophy, 1980) is an 80% success rate when practicing new material. When reviewing, the success rate should be very high, perhaps 95% and student responses should be rapid, smooth and confident.” So this idea of success rate supports teachers in deciding when to move through the instructional core, particularly when to move from guided practice (when around 80% of student responses are correct) to independent practice (when around 95% of student responses are correct).  This 7th principle seems a bit obvious and not overly helpful in the 2012 pamphlet, but it gains practical use thanks to the 1986 paper.

These principles now serve as a foundation for our support for teaching and learning across our schools. There’s a couple of things about foundations – in the sense of a building’s foundations – that I think are useful here. One is that we don’t tinker with foundations once they’re in place. They’re built to last. The second is that foundations are designed to be built on. We hope that throughout United Learning our teachers will explore these principles and bring them to life in the context of their school, their subject and their pupils. Rosenshine closes his 1982 paper with this very point:

in sum

 

Periodisation: Learning from the Flying Finn

This post has been co-written with United Learning’s Head of Sport Shaun Dowling (@ShaunD10)

“Some do well in other races, some run fast times, but they cannot do well in the ultimate, the Olympics … The question is not why I run this way, but why so many cannot.”

These are the words of Lasse Viren, also known as the ‘Flying Finn’ as he tried to explain his knack of peaking at the right time – a knack that landed him four gold medals in long distance events at the 1972 and 1976 Olympics.

lasse_viren_1975_paraguay_stamp

Like our pupils, athletes have a long build-up to the events that matter, and like our teachers, it’s the job of coaches to break this build-up into smaller units to ensure that their athletes peak at the right time. It’s a process athletes call ‘periodisation’.

Amidst growing concern about the impact of high stakes tests on pupils’ mental health, perhaps we can learn from periodisation to support our pupils in playing the long game and peaking just in time for their public exams.

A definition of stress which will be familiar to many PE and psychology students and teachers is ‘the difference between the demands placed upon us and our perceived ability to cope with them’.  Public exams will certainly be demanding, but if the specification has been covered and the content learned thoroughly, then students can be in a position to approach the exams with confidence and optimism.

Attribution Theory describes the Locus of Causality: an individual’s perception of whether their success is within or outside of their control.  The timing, importance and difficulty of the public exam season clearly falls into the latter category. However, many of the stresses being placed upon KS4 students are ones which schools do control: extra lessons; compulsory revision sessions; regular high-stakes assessments; all in the pursuit of target grades which might be based on flimsy evidence … all, of course, with the very best intentions in mind.

However, an unintended consequence of all of this (as well as the additional workload for teachers) is what appears to the students to be a constant and unrelenting pressure.

Is there a solution?

“The starting place for your planning is adopting the belief that training must be a steady and gradual building process.” (Joe Friel, 2016. The Triathlete’s Training Bible. 4th edition. Velo Press)

Periodisation in sport involves athletes identifying the races/tournaments which they want to be at their best for.  If they trained and raced with the same intensity all year round they would risk:

  • fatigue
  • injury/illness and
  • stagnation/boredom.

Sound familiar?

So what would the 5 years of secondary education look like if we approached them as an athlete/coach would?

The literature on periodisation varies in the number and names of the periods which they break down the training plan into. In a linear periodization model they can be grouped into three broad headings:

  1. Base
  2. Build
  3. Pre-competition (leading in to the A race itself)

To support this, an athlete’s season is likely to be broken down into macro, meso and micro periods: four-week meso periods within the yearly macro period, with four one-week micro periods within each meso one.  A twelve week build phase, for example, may have three meso periods of four weeks, the fourth micro period of each being a recovery week.

Applying this principle of periodization to secondary education could look something like the simplified model below, with the exam period of Year 11 classified as the students’ ‘A Race’, their mocks exams as their ‘B Races’ and end of term tests as the ‘C Races’.

Within this model, the micro period idea of a weekly plan of what and when takes place is particularly helpful for revision timetables and avoiding clashes with scheduled revision sessions and other ‘life’ priorities.

Base period

KS3 = mastering the basics, focusing on core skills and preparing for the harder work to come. It should be a relatively stress-free and enjoyable period of time with opportunities to learn new things and explore wider opportunities, but it is also an opportunity to baseline and set goals. Target-setting using both quantitative data and qualitative information can be adjusted throughout the period, but there are no high-stakes assessments. This phase is designed to provide both a strong base for the build period to follow and to enhance enjoyment of the subject.

Build period

  • Build 1 – Autumn term Year 10: increasing demands from KS3 but managed in a way to protect health and avoid burn-out
  • C Test – low stakes indicator of progress towards goals
  • Build 2 – Spring term Year 10: potential for re-setting based on previous test (i.e. adjust training zones). Gradually increasing demands.
  • C Test – low stakes indicator of progress towards goals, including aspects of Build 1
  • Build 3 – Summer term Year 10: potential for re-setting based on previous test (i.e. adjust training zones). Gradually increasing demands.
  • C Test– low stakes indicator of progress towards goals, including aspects of Build 1 and 2
  • August = recovery period (re-charge in order to come back stronger)
  • Build 4 – Autumn term Year 11: final build stage leading into opportunities to practice preparations such as exam technique, revision techniques and nutrition prior to the ‘B’ exams
  • ‘B’ exams (mocks) – all work covered thus far and, on occasion, in conditions similar to the ‘A race’ exam season. This includes timings, density, environment, rules/expectations etc.

Pre-competition

Spring Year 11: Increase in specificity and intensity as every effort is made to ensure that all knowledge has been learned thoroughly. Final preparations and a tighter focus on the micro periods to space out revision effectively and manage the other pressures on 16 year olds’ lives. As the exams draw nearer, prioritise time and manage ‘essential’ sessions so that students are fresh for them.

‘A’ race – exams!

May/June Year 11: Tapering for exams – shorter periods of high intensity revision sessions, the exams themselves, brief recovery, preparation for the next one. Make every effort to psychologically prepare too, getting the exams into perspective, teaching processes of positive self-talk, how to manage the “Chimp” and how to arrive in the exam hall full of confidence and looking forward to the challenge.

 This periodisation approach relies on honest and clear communication with pupils, with frequent reminders of which point they are at in the 5-year journey. We can’t expect pupils to step up in Key Stage 4 if we’ve pretended to them for three years that their KS3 exams are cliff-edge assessments. By sharing the 5-year journey with our pupils we are trusting them to respond appropriately to the demands of each phase. In doing so, we provide a sense of ownership and control, perhaps reducing the pressure on staff and leaders to throw everything at Y11 in the hope that some of it might stick.

Concerns that playing this long game would lead to a lack of urgency at KS3 should consider the current situation we see in many schools where there is a stark difference in the intensity of Y7 compared with Y11. Periodisation seeks to harness the urgency of Y11 and use this to provide purpose and focus throughout the secondary years, rather than unleash it in a sudden wave when students return from their summer break at the end of Y10.

There is an argument, of course, that just by adopting this periodization model nothing will change in terms of outcomes. There are so many variables that influence exam performance that this is just one more idea that the impact of which would be impossible to measure.

However, if schools adopting the periodisation mindset means that the pressure felt by students is indeed “steady and gradual”, then isn’t that worth a try? Is it not worth trying to alleviate the increasing mental health concerns by re-thinking how we approach the secondary phase of education, KS4 and the lead-in to the public exams? Not all athletes who periodise their training go on to become Olympic champions. But athletes who do tend to become better athletes than they would have been had they not adopted this approach to their training. And along the way they pick up fewer injuries, less fatigue and a reduced risk of burnout.

That has to be worth considering.

For articles on periodization see numerous online posts by Joe Friel and others or here: https://www.trainingpeaks.com/blog/macrocycles-mesocycles-and-microcycles-understanding-the-3-cycles-of-periodization/

League Tables Part 3: The Best Model of a Cat is a Cat

In the first post in this short series we outlined the dizzying array of changes to school league tables since they were introduced in 1992. In the second post we looked at some of the problems associated with Progress 8 – the measure that currently dominates our league tables. What both of these posts have in common is that we shouldn’t expect any single measure to accurately capture the complex picture of school performance.

Cat

Harry Fletcher-Wood once suggested to me a way around this problem with the idea that schools are given a variety of performance measures but each year the government chooses to rank schools on just one of these, such as A*-B grades in modern foreign languages. The twist in the tale of Harry’s plan is that the DFE would only announce which measure it was going to use after pupils had sat their exams. Not knowing which measure to focus on, schools would simply get on with securing the best outcomes for all pupils across all subjects.

Harry’s cunning solution addresses the problem with performance tables known as Goodhart’s Law: “when a measure becomes a target, it ceases to be a good measure”. We see this with Progress 8. The intention might be that P8 gives credit for the performance of every pupil across a wide range of subjects, but the reality is that the quickest way for schools to boost their Progress 8 is to cram the open bucket full of vocational qualifications, particularly if you have a lower attaining cohort (because you then get an even bigger boost if these low prior attainers secure top grades in vocational qualifications).

If any single measure dominates, or is perceived to dominate, the attention of schools is diverted towards this measure. Just as the old 5A*-C with English and Maths measure was flawed because it led to an obsession with pupils working at the C/D borderline at the expense of other pupils, so too P8 is flawed because of the easier pickings in the open bucket which benefit the school more than the pupil. The solution is not to dispose of Progress 8, but to see it as one performance measure amongst many.

We could wait for the DFE to do this, or we could do this ourselves. Imagine if a school, local authority, or academy trust said that it was going to strive towards the following indicators, as well as the DFE’s official measures:

  • Ensure that all pupils can read, write and multiply with fluency by the end of Year 7 (useful for a school with weak attainment on entry so that pupils can access the curriculum in the rest of their time at school)
  • Ensure that no pupils leave school at 18 without a degree or an apprenticeship (useful for a school with high youth unemployment in the local area)
  • Increase the number of pupils gaining more than five top grades by 50% over three years (useful for a school that hasn’t previously enabled pupils to reach our very best universities)
  • Transform the performance of boys in languages or girls in triple science (useful for schools with gender gaps in these areas)
  • Double the number of pupils taking a course that is particularly valued in the local employment market e.g. engineering (useful for schools in regions with a distinctive local economy).

None of these metrics are official government measures, but they all seem like reasonable aims, and I can’t imagine the DFE or Ofsted taking umbrage with schools that choose to prioritise outcomes that they feel will make a particular difference to their pupils. We saw in the first post that one school in Salisbury secured a good Ofsted rating despite a very low Progress 8 score because it had chosen to enter its pupils for iGCSE English even after this qualification had stopped counting in league tables. The earth did not stop spinning for this school or its pupils. So maybe the bark of the league table is worse than its bite, and we have more freedom than we think to treat performance measures as an accountability framework that sits alongside our own aims for our pupils, rather than a singular goal that we must obsessively strive towards.

At their best, league tables work by honouring the actual outcomes that actual pupils walk out with when they leave our schools. It’s another reason why the predominance of Progress 8 is not helpful: it’s an irrelevant metric for individual pupils. At one school I worked at it was my job to write the press release on results day for GCSEs and A Levels. In addition to the usual published figures we also compiled a list of the top 20 pupils, showing their full range of grades across all subjects. For the 18 year olds, we added where they were going for university (GDPR wasn’t even a glint in a lawyer’s eye at this point). It struck me that this was powerful ‘data’ and would allow prospective parents to see the actual outcomes that pupils walk away with. As a parent I would be more keen to send my child to a school where performance is consistent across subjects, rather than a school with a spiky attainment profile, particularly if the best grades were in lower tariff subjects. Revealing the real outcomes of pupils, whether it’s the top twenty or a typical pupil at each decile, might be too messy to be an official government measure, but it could be a useful internal metric for schools or trusts to use. It’s a reminder that any performance measure is a model of pupil achievement, but sometimes it’s best to just look at the thing itself: ‘the best model of a cat, is a cat’ (paraphrasing the mathematician Norbert Wiener).

While we should continue to refine our performance measures, we shouldn’t expect to arrive at a perfect measure. Progress 8 was heralded as a saviour for schools with lower attaining intakes, as indicated by a BBC article in January 2016: “Head teachers have long complained measuring success on the basis of GCSE results alone is unfair as it does not take into account the intake of the school”. Three years on, many of these same heads bemoan the fact that Progress 8 – like all other measures – doesn’t tend to favour schools with lower attaining intakes, especially if these pupils are white British and disadvantaged.

I’m not sure we can blame league tables for exposing the under-performance of schools, regions or groups of pupils. This under-performance would exist whether or not it was reported in performance measures, and this reporting at least gives us a chance to confront some ugly truths about educational disadvantage and perhaps do something about it. But maybe it’s time for a more nuanced treatment of league tables, moving away from judging schools on one measure in a single year and towards looking at a range of measures over a series of years, as well as giving schools the opportunity to focus on the priorities that are particularly relevant to them in their local context.

League Tables Part 2: Progress 8

Search ‘never skip leg day’ and you’ll find a barrage of graphic warnings on the risk of working out your arms and upper body while ignoring your lower half. I wonder if Progress 8 has led to some schools ignoring ‘leg day’ – a solid academic core – in favour of superficial gains in the open bucket.

leg-day.jpg

When Progress 8 was launched I remember thinking that this was the performance measure to end all performance measures – at last schools would be credited for the achievement of every pupil in every subject at every grade. No longer would schools limit their attention to the ‘key marginal’ pupils at the C/D borderline, writing off in the process those pupils deemed incapable of securing a pass and ignoring higher attaining pupils who might be able to gain exceptional grades with a bit of a push.

Such naivety!

I still think that Progress 8 has its place in the performance measure mix, but I’m not sure that the predominance it has gained is justified, primarily because it’s too easily skewed by tactical behaviour in the open bucket and too far removed from the actual grades that pupils walk out with when they leave our schools.

First, is it right to say that Progress 8 has become the predominant measure of school improvement? I think so:

  • The DFE’s floor standard for secondary schools is based on Progress 8 (lower than -0.5)
  • The DFE’s Compare School Performance service gives a higher profile to Progress 8 than other measures e.g. if you enter the name of a school the first data you’ll see is that school’s Progress 8 score alongside a comment such as ‘well above average’ and if you create a comparison list of several schools, or compare all schools, the list is presented against Progress 8 scores by default
  • The DFE’s Multi Academy Trust performance tables begin by comparing Progress 8 between trusts.
  • Since its inception Progress 8 has topped the list of performance measures for secondary schools in DFE communication*

Separately, Ofsted’s Section 5 handbook explicitly states a preference for progress over attainment: ‘In judging achievement, inspectors will give most weight to pupils’ progress.’ It’s very clear from the rest of this paragraph that Ofsted is referring here to progress with a small ‘p’ rather than ‘Progress 8’ (e.g. the paragraph goes on to say that ‘inspectors will consider the progress of pupils in all year groups, not just those who have taken or are about to take examinations or national tests’) but with the DFE pushing its progress measure and Ofsted reaffirming its interest in progress, we can see why schools have reached the conclusion that Progress 8 is the performance measure that matters most.

When Progress 8 was introduced in 2015 it was – from my memory at least – schools with lower attaining intakes which welcomed its arrival most enthusiastically. At last, the key measure on which they would be judged would recognise pupils’ varying starting points and credit schools only for the progress made under their care.

It hasn’t quite turned out like that. In 2017 the average Progress 8 score for selective schools was 0.45 – well above average – while those schools languishing at the lower end of the P8 tables tend to be schools that also struggled under the previous attainment measures. If my Twitter feed is anything to go by, it’s now heads of schools with lower attaining white British intakes who feel most aggrieved by Progress 8, arguing that the playing field remains uneven.

So why have some people lost faith in Progress 8 so soon? Let’s take a few key issues:

  • The Open Bucket – Despite the removal of ECDL (see previous post) there are still easier pickings to be found in the open bucket which can artificially inflate a school’s P8 score if they enter large number of pupils (thereby deflating the P8 score of schools that don’t). Whether it’s the LIBF Certificate in Personal Finance or the TLM Level 2 Certificate for IT User Skills in Open Systems and Enterprise, these qualifications benefit schools more than pupils, undermining our faith in performance tables in the process.
  • The EAL effect – One striking feature of schools with exceptionally high P8 figures is that several of them have a high proportion of EAL pupils. If we go through the top 12 schools by P8 in 2018 then check the number of EAL pupils in their 2017 Y11 cohort (2018 EAL information is not yet available) we see EAL figures of 117 from a cohort of 119; 169 from 209; 17 from 111; 0 from 25; 105 from 118; 3 from 35; 68 from 75; 71 from 75; and 7 from 147 (data unavailable for three of the 12 schools because they didn’t have Y11 cohorts in 2017). The average here is 61% EAL. Switch the digits over and we get the national average of 16%. Perhaps the key point here is less about EAL and more that KS2 outcomes are not a great indicator of KS4 potential, particularly if performance at KS2 has been held back because pupils haven’t yet gained a secure grasp of English. Schools with lots of EAL pupils are potentially on to a winner: once their grasp of English is secure they motor ahead of first language English speakers. This is brilliant news, but it’s unclear why the schools they happen to go to should be credited for this.
  • KS2 results – Just a hunch, but I wonder if disadvantaged pupils are more likely to have inflated SATs results compared to non-disadvantaged pupils. My thinking here is that disadvantaged pupils are more likely to attend primary schools at risk of poor KS2 performance, so these schools are likely to devote more time preparing for SATs rather than just teaching the normal curriculum. If this hypothesis holds water, then secondary schools that recruit a high proportion of disadvantaged pupils are recruiting pupils whose real attainment is weaker than their SATs results would suggest, making it difficult for these secondaries to secure higher rates of progress. Even if this is a bit far-fetched, we can probably agree with Dr Becky Allen that ‘Key stage 2 test score is quite a noisy measure of a child’s educational attainment at age 11‘ and that KS2 results depend on a wide range of factors such as the quality of teaching; the amount of preparation for KS2 tests; the way the tests are administered etc – so are fairly limited as indicators of a child’s ability at the age of 11, let alone their potential ability 5 years later. Given that one side of the Progress 8 ledger is based on KS2 outcomes we should probably avoid using P8 to make sweeping judgements about schools.

It’s the first point above – the problem with the open bucket – that troubles me the most. The DFE’s Compare School Performance service reveals the P8 score of each element (for 2017 at least, not yet for 2018), and it’s not unusual to find schools with modest P8 figures for English, Maths and EBacc, and then stunning P8 figures for the open bucket. We can assume with some confidence that this is because schools will be entering the whole cohort for two or even three vocational subjects, and then securing excellent outcomes in these subjects (of course the open bucket also includes subjects like RE, the English GCSE not already counted, as well as Art, Music and Drama etc, but it doesn’t really stand to reason that a school performs massively better in these subjects than in English, Maths, Science, languages and humanities, hence my confidence in suggesting that a disproportionately high Open Bucket P8 figure is likely to be based on large numbers of pupils doing vocational subjects).

If you’ve got a lower attaining cohort who all secure top grades in three vocational qualifications (in 2017 this would often have been ECDL, Business and Sport) then your P8 score would go through the roof. We’ve then got a strange situation where lower attaining pupils might only be making average progress in English or Maths – therefore leaving school without good grades in these subjects (because they started with low attainment and only made average progress) – yet the school’s P8 figure would indicate that the school is excelling. The school is rewarded for skipping leg day.

The first rule of league tables should surely be that they credit schools for the things that matter for pupils, thereby aligning the interests of the school and the interests of the child. The fact that Progress 8 rewards tactical behaviour in the open bucket is therefore hugely damaging for its integrity as a measure of school performance, and represents a dangerous drift away from the things that matter to pupils.

So despite the best of intentions in creating a measure that values every student, every subject and every grade, and recognising schools that add value from lower starting points, Progress 8 has not been the ‘measure to end all measures’ that I naively welcomed back in 2015. I think it still deserves its place in the mix, but I’m not sure it’s worthy of the first among equals status that it seems to hold at the moment. It would surely be better for school leaders to focus their energy on outcomes that matter to pupils – such as decent grades in English and Maths – rather than the school’s Progress 8 figure.

A simple solution to this is to remove the open bucket to create a ‘Progress 5’. We don’t need the DFE to do this for us – it’s something we do at United Learning to shine a light on the ‘core stability’ of schools, with the view that schools with strong performance in the academic core have a more secure basis for future success than schools relying on their open bucket.

The point of this post isn’t to attack schools that skip leg day, schools where the academic core lags behind the open bucket. It’s difficult for outsiders to know the challenges that a school might face in recruiting a stable team of English, Maths and Science teachers, for example, and I understand the drive to gain momentum at the beginning of a school improvement process. The open bucket offers quick wins to restore league table pride while longer term gains are made at the core of the curriculum.

Instead, the point of this post is to reflect on the way we rank and judge schools, and to think twice before heaping praise on schools that have secured a stunning Progress 8 without gaining decent grades in the academic core. These pupils might have starred distinctions in ECDL, Business BTEC and Sport BTEC, but unless they’ve also got 5s in English and Maths, they’ll struggle to get on to A Levels and degrees.

Where does this leave Progress 8? I’ve tried to argue that the two bits of data on which P8 is based – KS2 performance and performance across 8 subjects at KS4 – are both flawed, the former because KS2 performance depends a lot on how seriously primary schools take SATs and the latter because the P8 figure is easily skewed by the open bucket. As such, I don’t think Progress 8 is worthy of the status it seems to hold as the first among equals of our performance measures.

One of the criticisms of P8 that I considered but didn’t include here was the argument that a school’s P8 figure is easily distorted by a small number of ‘outliers’ – often pupils who have left school with no qualifications. Such pupils didn’t particularly affect a school’s figures under the old threshold attainment measures, such as 5A*-C with English and Maths, because although these pupils would be marked as failing on these measures, their impact on the school was no greater than ‘near miss’ pupils. Under Progress 8, pupils leaving school with nothing have a significant impact on a school’s league table performance.

I actually think this is a huge strength of the P8 measure. By incorporating the outcomes of all pupils in all subjects we’ve finally got a measure which encourages schools and the system as a whole to grapple with the ugly truth that thousands of pupils leave our schools each year with nothing to show for their education. The current and long-overdue discussions about off-rolling and exclusions are perhaps an unintended consequence of Progress 8, and might just represent Progress 8’s most important contribution to our collection of measures.

*KS4 performance measures, as stated by DFE in 2018

  • Progress 8
  • Attainment 8
  • EBacc Average Point Score
  • the percentage of pupils entering the EBacc
  • the percentage of pupils achieving a grade 5 or above in English and maths
  • the percentage of students staying in education or employment after key stage 4 (destinations).

League Tables Part 1: Cat & Mouse

League tables divide opinion. For some they support our core purpose of securing the best outcomes for our pupils, providing in the process the transparency, accountability and feedback that all organisations need to sustain improvement – sunlight is the best disinfectant, and all that. But for their critics, league tables say little about what really matters in schools; not only do they fail to capture the complexity of life in our classrooms, they distort our behaviour and encourage teachers and leaders to make decisions based on what looks best in league tables rather than what’s best for our pupils.

Over the following posts I want to see how league tables have evolved since they were introduced in 1992, before taking a detailed look at Progress 8, which has become the headline figure for secondary schools. We’ll finish by suggesting how schools might respond to the ongoing flux of league tables.

League tables have long been one of two ‘eyes’ of accountability, with the other being Ofsted. With signs that Ofsted want to focus more on the inputs of education than the outputs, it’s all the more important that the gaze of league tables is fixed on the things that matter.

First, a quick point on language. ‘League tables’ is a bit of a misnomer. What we’re really referring to is the DfE’s Compare School Performance service which provides open access to a wide range of measures from any state school in the country. When we search for a particular school we see key measures (see example of Paddington Academy below) and then more specific measures for that school, and we can compare a group of schools, or all schools, by any measure we want.

League Table

Part 1: Cat and Mouse

Given that league tables were introduced to provide transparency and to invite public scrutiny, it’s fitting that all league tables published since their inception in 1992 are available online. Naturally I started my search at the school I attended – Cantell School in Southampton – where in 1992, the year before I joined, 32% of pupils gained 5 A*-C grades in their GCSEs (not necessarily including English and Maths). The national average that year was 38.3, and once again this was 5 A*-C grades in any subject – it wasn’t until 2006 that English and Maths had to be included. By the time this headline measure was phased out in 2015, 64.9% of pupils left school with this basic benchmark. Our profession is good at self-flagellation, and a week rarely goes by without another article bemoaning the state of our schools, but things were a lot worse just a generation ago. We can of course debate whether league tables have caused this improvement, or simply revealed it.*

The most striking finding from a year-by-year check of league tables is how much they’ve changed over time. I’ve identified some key changes here (all quotations below are from DFE’s guidance document accompanying each year’s league tables):

  • 1992-1996: 4 simple measures captured in league tables %5A-C, %1A-C, %5A-G and %1A-G
  • 1997: GNVQs combined with GCSEs for the first time this year on the basis of ‘broad equivalencies’ e.g. intermediate GNVQ equivalent to 4 GCSEs at A*-C.
  • 1998: the GCSE/GNVQ average point score per 15 year old is introduced. “This provides a fuller picture of the GCSE and GNVQ achievements of pupils of all abilities. The average point score is calculated by dividing the total GCSE/GNVQ points achieved by all 15 year olds by the number of 15 year olds”.
  • 2002:
    • Introduction of KS2-KS3 value added measure and KS3 to KS4 value added measure
    • Average capped point score based on pupils’ best eight results introduced to deter school from entering pupils for an excessive amount of qualifications
  • 2004: Introduction of KS2-KS4 value-added measure.
  • 2006:
    • Contextual Value Added (CVA) introduced: “CVA takes into account the varying starting points of each pupil’s KS2 test results, and also adjusts for factors which are outside a school’s control (such as gender, mobility and levels of deprivation) that have been observed to impact on pupils results.”
    • 5A*-C including English and Maths included for the first time. This had a significant impact on the league table position of some schools, as reported by this BBC article “The effects on some schools have been dramatic. One school went from a score of 82% passing the equivalent of five A*-Cs to just 16% when maths and English were included.”
  • 2009: Progress figure for English & Maths included for first time
  • 2010: “The percentage of pupils who have met the new English Baccalaureate requirements reported for the first time this year”
  • 2011:
    • CVA measure scrapped, replaced by VA measure (best 8 with bonus for E&M), including separate VA measure for each EBacc subject
    • Figures for high, middle and low attainers introduced for first time
    • Headline figures published with and without equivalences
    • In-school gaps published for first time, revealing the gap between the GCSE outcomes of each school’s FSM pupils and its non-FSM pupils
    • iGCSEs included
  • 2012:
    • Pupil premium reported (in place of the gap analysis above)
    • Destination measures introduced
    • Gender breakdown introduced
  • 2013:
    • Progress gaps now revealed alongside attainment gaps
    • ‘Similar schools’ table introduced
  • 2014:
    • No early entry – only first entry counts for EBacc subjects
    • Wolf reforms lead to the “removal of around 3,000 qualifications from performance measures; adjustment of the point scores of non-GSCEs and the restriction on the number of non-GSCE qualifications that count to two per pupil”.
  • 2015:
    • Last year of 5A*-C with English and Maths
    • Early entry policy now counts for all subject areas
    • Progress 8 score published for schools who opted in
  • 2016: Progress 8 introduced for all schools, so the headline measures are now:
    • Progress 8
    • Attainment 8
    • English and Maths at C+
    • EBacc (entering and achieving)
    • Staying in education
  • 2017:
    • iGCSEs no longer count
    • 1-9 grades replace A*-G in English and Maths
  • 2018:
    • ECDL no longer counts
    • New measure for EBacc (average points score)
    • 1-9 grades replace A*-G in most subjects

This dizzying array of changes (and there are plenty more I didn’t include here) reveals one of the challenges of league tables: they don’t tend to remain stable for long enough to guide school improvement in a meaningful and sustainable way. Given the fact that league tables are a retrospective check on the outcome of 5 years of schooling, you would think that stability would be built into their design so that schools can gradually work towards known metrics. Instead, the frequency of reform has left some schools lurching from one change to another.

The extent of this lurching is not insignificant. In 2016 there were more than 300,000 entries for iGCSE, the final year that they were included in league tables. In 2017 the figure had fallen to 110,000, creating one of the more unusual examples of our divided school system as independent schools retained the iGCSE while state schools took flight. Independent schools, of course, do not feature in league tables.

The rise and fall of the BCS Level 2 European Computer Driving Licence Certificate in IT Application Skills (ECDL) is another example of the influence of league tables on schools’ behaviour. ECDL entries increased from 26,000 in June 2015 to 117,000 the following year, and rose again in 2017 as the ECDL was promoted as an accessible qualification that can be quickly delivered to boost the open bucket of Progress 8. Figures for 2018 are not yet released but between January and March 2018 there were just 2,800 entries for ECDL compared to 37,650 in the same quarter in 2017. Did schools suddenly decide that their pupils no longer needed this certificate in computer skills? Of course not – this qualification fell out of favour the moment it ceased to count in league tables.

As I was searching for the entry figures above I found these questions on web forums for parents and students:

  • Mumsnet, March 2016: “Is ECDL worth having? I’m sure it is in its own right. But is it equivalent to taking Physics, History, Art or French GCSE?”
  • Student Room, February 2017: “I have asked many people on whether or not ECDL counts as a GCSE and they all give different answers. Does anybody with the ECDL qualification know? Say you get 5 GCSE’s and ECDL, would a job/uni/sixthform count that as 6 GCSEs? Thank you”

“My school calls it a GCSE” is the depressing reply of two respondents to the question posed by a student.

Since the ECDL has been discounted, some school leaders have searched for the next quick win, which no doubt will itself be discounted in time. And thus continues the bizarre dance of cat and mouse that has played out between the DFE and schools since league tables were introduced. Take this example from an IPPR report in 2003:

“The Government’s decision to give intermediate GNVQs an equivalency rating of four A*-C GCSEs has led to a surge of schools taking advantage of what is seen as easy league table success. Thomas Telford School in Shropshire, for example, has embraced GNVQs from its inception. Yet now, all of their pupils take at least one GNVQ and some leave with a total equivalent of nineteen GCSEs contributing in large part to their outstanding league table performance.”

One final example of the responsiveness of schools to changes in performance measures: in 2014 when the government decided that only first entries would count for EBacc subjects, the number of early entries for GCSE Maths fell from 169,000 in 2013 to 31,000 in 2014 – a profound overnight change to the delivery of Maths at thousands of schools.

It’s difficult to defend a system which appears to result in such obviously tactical behaviour by so many schools, and it is pupils, particularly poorer pupils at schools vulnerable to weak league table performance, who are caught up in this not-so-merry dance between schools and the DFE. We see this in the fact that so many independent schools continue to enter pupils for the iGCSE, while state schools abandoned the iGCSE when it ceased to count. Pupils in independent schools therefore benefit from teachers who have taught the same qualification for several years, while state schools gradually familiarise themselves with new specifications. In my experience it is disadvantaged pupils, even the high attainers among them, who are more likely than their more affluent peers to take vocational qualifications such as BTEC Sport or BTEC Business, which again might be because they tend to find themselves in schools desperately seeking any possible league table advantage.

Does it have to be like this? One school in York gained an outstanding Ofsted judgement in 2017 with a report which praised leaders for putting the needs of pupils ‘above all else’. The report continues: “The curriculum reflects leaders’ integrity because it is designed to match pupils’ needs and aspirations regardless of performance table measures. Leaders have made provision for almost all pupils to study a modern foreign language because research tells them that pupils will develop valuable skills for their future.” Sure enough, the league tables show that 83% of this school’s Year 11 pupils took a language GCSE in 2017, with a progress score for language of -0.43 – sharply at odds with the school’s progress score in other subjects (e.g. science 0.48, humanities 0.63).

250 miles south, in another cathedral city, one school gained a good Ofsted judgement in 2018 despite a Progress 8 score of -0.71. The reason, as I understand it, is that the school continued to enter pupils for iGCSE in English as they believed this was in the pupils’ best interests, even though the league table performance of the school would suffer as a result (the P8 score for English here is -2.91).

Perhaps these two schools, which have flourished despite taking decisions that have hampered their league table standing, prove that the bark of the league table is worse than its bite? What would happen if more schools did their own thing rather than dance to the DFE’s tune?

More on this in our next post

*After checking my own school I went on to check the schools that I work with at United Learning. Just look at these improvements between 1992 and now:

league tables ul

Obviously this is just a handful of schools among the 3000 or so secondaries across the country, but let’s be grateful that the woeful outcomes evident in the 1992 column hardly exist anymore.