The Art of Prediction

“Prediction is very difficult, especially if it’s about the future”

It’s also difficult if it’s about exams that have not yet been sat; exams which will be graded against standards which are only fully clear after the event.

So how can we navigate this world of known unknowns?

We should start by recognising that predicting student outcomes is a useful and legitimate endeavour if it’s done properly.   It provides students with an indication of where their current efforts might get them, and it enables teachers and schools to direct additional support to students who need it most.

While some schools choose to talk only in terms of working-at grades rather than predicted grades, it’s only when teachers are asked to consider a predicted grade that they are prompted to consider all information – of the student, the student’s work, and the course – in the round.

Predicting grades will never be an exact science, but the approaches below might help avoid nasty surprises in August:

Start with why: Predicting accurately begins with a shared understanding of why we are predicting.  Teachers might need reassurance that optimistic predictions are no good to anyone, and neither are pessimistic predictions intended to give students a wake-up call.   Trainee teachers in particular might not realise that predictions can be the trigger for targeted support and tough conversations.  Make it clear how predictions are used to inform the support that students receive.

What are we asking teachers to do?  Be crystal clear about the judgement that teachers are being asked to make.  One source of confusion arises when people use the phrase “if students sat the final exam now”, which often leads teachers to penalise students for parts of the course that haven’t even been covered, which surely isn’t useful for anyone.

We can’t just wish people to be more accurate: I remember working with a colleague who asked teachers to predict with 90% accuracy.  The next year he asked for 95% accuracy.  The following year it was 100%.   But of course we can’t just wish teachers to be more accurate.  Encourage honesty over accuracy.

Be clear about what teachers should do if they simply don’t know: If I was a school leader, a teacher saying to me “I simply don’t know what this student will get in my subject” would be an incredibly helpful thing to know.  Make it ok for teachers to admit this.   If there are teachers in this position, school leaders should link them up with other schools for moderation and quality assurance.

Gather your evidence: It’s easier to make accurate predictions if you have a broad sample of students’ work throughout the year.  Think of ways to ensure that teachers collate students’ assessed work and keep it to hand when making predictions.  Compare this with the work they’ve produced in timed exam conditions on unseen past papers.  Analyse trends from previous years to help make predictions – look at the trajectory of students who were at a similar stage at the same time.

Start at the beginning:  Before the start of the year ensure that subject leaders have access to recent past papers, exemplary work at selected grades, and all recent mark schemes and examiners’ reports, with a summary of key points from each.   Assembling these resources before September will enable more accurate predictions throughout the year.

Avoid the sugar coating: It’s ok to predict a U grade if that’s what you think a student is going to get.  There’s no hiding place on results day, so it’s better to ring the alarm bells in good time.

Greater precision doesn’t mean greater accuracy: Predicting in fine grades (C1, C2, C3) might seem to be more accurate than whole grades, but we should be careful not to take comfort in illusory precision.  Given that swings in grade boundaries can wipe out whole grades, such precision can – in hindsight – seem misplaced.  In my experience, predicting in fine grades makes sense in maths and English, where teachers can draw on a bottomless pit of past papers (and schools tend to make time for students to sit these papers).  In other subjects, I’m not so sure.

Allow for shifting grade boundaries:  Look back at past papers and base your own grade boundaries on the toughest you can find.

Safety in numbers:  Be wary of grades being predicted in isolation – build quality-assurance into the process by ensuring that Heads of Department and other colleagues check the predictions before they’re submitted.  Be especially wary of standalone courses (e.g. economics, psychology, sociology) which don’t always benefit from the scrutiny of a head of department or a line manager.  We could even take this a step further by holding a CPD session in which all teachers of examined classes gather together, armed with their course materials, to make their predictions.  This would build quality assurance into the process and allow less experienced teachers to learn from colleagues.  Moderate these predictions with other schools wherever possible.

We can’t predict the unpredictable:  Perhaps the whole business of making predictions is based on a false assumption that if students meet certain criteria, they will be awarded a certain grade.  In reality, comparable outcomes dictate that the standards required for each grade are not absolute, but will depend to an extent on the performance of the national cohort.  There are no easy answers to this, except to be honest that making predictions is not an exact science.

Reformed GCSEs and A Levels, with tougher, longer and unknown terminal exams, will make the art of prediction even trickier in the coming years.  Let’s hope that the emergence of Progress 8 will ease the obsession with the C/D borderline – an obsession which often drives the desperation for accurate predictions, as ‘C3’ and ‘D1’ students are funnelled into intervention sessions.

As I reflect on what I’ve written I’m drawn to consider again if the time and effort required to make accurate predictions is justified.  Perhaps we would all be better off if teachers invested that effort in supporting all students to do the very best they can?  On balance I think that it is still worth taking the time to make predictions, not least because the information required to reach a judgement involves teachers thinking deeply about their students’ work, and matching this against the specific requirements of their course.  This can only be a good thing for all involved.  That said, given the amount of information required to predict accurately, we should refrain from asking for predictions too frequently.

Thanks to Aidan, Ben, Dave, Janina, and Matt for your predictably helpful support with this.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s