My favourite passage in Daisy Christodoulou’s Making Good Progress is this one, in which she reveals the flaws in a ‘catch-all’ assessment model which uses a single scale to measure progress in each subject:
“The best analogy for this kind of measurement is measuring height. You can use the same tape measure and the same scale to measure a 4 year old as you can a 16 year old. The method stays the same even as the child grows. You can also use the same scale to measure progress over time and, if you have a tape measure with small enough divisions, you can measure progress over quite short periods of time. This was ultimately why people started using grades in every lesson and why the desire for subgrades and sublevels came about: people saw them as the millimetres and centimetres that made up the metres of progress. And, if you subscribe to this model, then ever finer subdivisions of grades are a perfectly good idea: they will allow you to measure the precise amounts of progress that are added in each lesson. It would be possible to say that after a week of lessons a pupil had added x% of a GCSE grade, or z fraction of a sublevel. As we have seen, learning is not actually like this. Complex skills are made up of many different elements and those distinct elements all look very different and cannot be measured with the same scale.”
This distinction – between performance in a given subject and the incremental practice required to enable this performance – is at the heart of Daisy’s book. It’s a distinction that results in lessons that might “look very different from the final skill they are hoping to instil”. Our assessment of the process of learning therefore serves a different purpose to our assessment of the product of learning, since when we’re assessing the process we should be seeking out gaps and errors to enable the student to improve, whereas in assessing the product of learning we are seeking to describe an individual student’s performance in relation to her peers.
The protagonists of Making Good Progress are formative and summative assessment – star-crossed lovers who both have their place in a coherent assessment system, but rarely together.
Daisy accurately notes the “tension within schools between senior managers, who are often more concerned with accurate summative data, and teachers who are often more concerned with accurate formative data.” Yet rather than wishing a plague on both these houses, Christodoulou offers practical guidance on disentangling formative and summative assessment in our schools, liberating our teachers to address the specific elements of their subject. The exam hall, not the classroom, will generate the hard-nosed data our senior leaders seek (but only once or twice a year).
Christodoulou’s focus is the assessment of students, but her book got me thinking about the way we assess teachers and schools, and whether we need to disentangle formative and summative assessment in these areas too.
Take the assessment of teachers, better known as performance management. Too often we combine summative and formative assessment – “we judge you to be a Good teacher … to improve try to model the appropriate use of challenging vocabulary”. There are two very different intentions at work here. The first – providing a holistic judgement of a teacher’s effectiveness – can only be done, if at all, with reference to student outcomes across several of the teacher’s classes. The second – providing feedback to help the teacher improve – can only be done meaningfully if the lesson(s) that we see the teacher deliver are a fair reflection of what usually happens in that classroom: if the sample reflects the domain, to use Daisy’s language. Effective formative feedback also requires trust between the teacher and the coach. This trust evaporates if the coach is also grading the teacher. In trying to formatively and summatively assess teachers at the same time we end up doing neither to any useful standard.
There’s a similar risk when judging schools. Ofsted is intended to support school improvement – “We focus on how standards can be raised and outcomes improved” (Ofsted Strategic Plan 2014-2016) but in providing a summative judgement on the effectiveness of individual schools, based on a 1 or 2 day visit, our inspectorate has perhaps failed to create the conditions that would allow for genuine school improvement to emerge, such as trust and transparency. Making Good Progress argues convincingly that summative descriptors of the end product struggle to capture the small steps that constitute progress towards this end goal. Such feedback amounts to telling a comedian to ‘be more funny’, just as some Ofsted reports can make recommendations such as “Further raise achievement across the school, particularly that of the most able pupils and boys” without capturing the steps required to enable this.
Christodoulou calls for teachers to devise a ‘model of progression’ for their subject – a coherent framework which links the granular activities taking place in the classroom to the end product: mastery of the subject. In simple terms, this model of progression means a clear sense of what getting better looks like in any given subject.
It strikes me that Ofsted, and we as a profession, lack a model of progression for school improvement. It’s difficult to argue with the Ofsted criteria for Outstanding (e.g. “The school’s thoughtful and wide-ranging promotion of pupils’ spiritual, moral, social and cultural development and their physical well-being enables pupils to thrive”) but these post hoc descriptors fail to provide a route towards this end goal. This is why school improvement plans which borrow the language of Ofsted are often just bland statements of intent. Just as National Curriculum Levels distracted teachers from the incremental practice required to make genuine gains in learning, so Ofsted descriptors might distract us from the incremental growth required for genuine school improvement.
A further implication for Ofsted in Making Good Progress is the problems caused by Ofsted’s interest in in-year data. It’s commendable that Ofsted seek to give credit to the current performance of students, but Christodoulou makes a compelling case that the primary purpose of in-year data should be to reveal gaps in students’ understanding, which is clearly at odds with the pressure to provide a positive in-year picture for Ofsted. The presence of new HMCI Amanda Spielman in the acknowledgements page offers hope.
This book can be a challenging read, as Christodoulou gently reveals the flaws in practice that are commonplace in our schools, even after NC levels have disappeared. Perhaps those hours that we spent trying to define what a Grade 5 looks like in Geography in Year 7 could have been used more productively.
Yet there is plenty of hope in Christodoulou’s work; hope for a future in which we genuinely seek out – rather than cover up – gaps, flaws, errors, and misconceptions; not just in our students, but also in our development of teachers and our improvement of schools. The proper work of student learning, teacher development and school improvement can then begin.