When evidence doesn’t work
There is a particular type with whom it is very difficult to discuss education. I will call him “rational man”. Often a science teacher, rational man recoils on hearing any strong opinion and instantly demands, “What evidence do you have to back that up?” When challenged on his beliefs, rational man presents himself as a superior form – the unideological pragmatist – and declares, “I believe in evidence. I believe in what works”. To rational man, I would like to say, evidence does not always work.
On Saturday, I attended ResearchED Conference at Dulwich College. Organised in six months and inspired by a Twitter conversation, it was a remarkable event and by far the most energetic education conference I have attended. However, throughout the day I detected a worrying tendency encapsulated in the conference tag-line “working out what works”. Some education research enthusiasts seem to believe that evidence can overcome opinion. It cannot. It can inform opinion. It can promote or discredit opinion. But evidence will never be opinion.
I would like to qualify that I am not an anti-enlightenment dinosaur. I like evidence. I respect evidence. I sometimes even use evidence in my articles. Tom Bennett, who put together ResearchED, is entirely right that better use of evidence is needed to show up bogus educational fads such as VAK, learning styles and Brain Gym©. However, evidence is limited. Perhaps unfashionably, I believe that some of the most important debates in education will never be solved by evidence.
Firstly, the measurable outcomes in education are complex. The physician Ben Goldacre, one of the ResearchED organisers, is evangelical about applying Randomised Controlled Trials (RCTs) to educational research. For RCTs in medicine, normally used to trial drugs, there is a clear outcome: either the patient gets better or they do not. Such clear outcomes can exist in education. Debates over how to teach literacy were greatly aided by the 2005 research of Joyce and Watson, which demonstrated the superiority of Systematised Synthetic Phonics over other primary reading programmes. This worked because reading and spelling levels are very easy outcomes to measure. But what about deciding on the outcome in the first place? Can an RTC tell us, for example, whether secondary school pupils benefit from studying Shakespeare? For us in education, the outcome is rarely clear. In fact, the outcome is the subject of our most impassioned debates.
School improvement can be broadly measured by exam performance, but this can lead to a bleakly instrumentalist view of schooling. Exam results are important, but they are not the sole purpose of school. At a talk delivered by Professor Coe of Durham University, I was shown the slide below. The graph takes the Education Endowment Foundation (EEF)’s meta-analysis of 33 educational ‘tools’, and plots the average cost of each ‘tool’ against evidence that it aids academic attainment. Such evidence classifies ‘Behaviour interventions’ as ‘may be worth it’, and ‘after school programmes’ as ‘not worth it’.
Has it occurred to the EEF that some aspects of school life may be ‘worth it’, even if they do not improve academic outcome? After school activities can enrich a pupil’s life, uncover talents, and build character – a principle to which schools such as Dulwich College, our venue, hold dear. These outcomes are no less important because they are hard to measure. Or take behaviour. I believe that schools should inculcate in pupils the importance of habits such as: not swearing at adults; not bullying your peers; putting litter in bins; and holding open doors. I do not think that pupils should acquire such habits because they will help them in their GCSEs. They should acquire such habits because they are normative goods.
Rational man believes that they can make their way in the world without recourse to the murky business of ideology and morality, or to use a more contemporary term, ‘values’. However, we can never dispense with values. If you do not agree, consider this hypothetical case. The Institute of Education runs a RTC across 200 schools into the use of caning, and conclusively proved that caning in schools improves exam performance. Would you be convinced of its merits? My hope is that you would not.
The caning example may seem hysterical, but it is not unusual. In order to resolve many core debates in education, we first have to decide upon what we value. Should schools predominantly teach British history? Should pupils memorise poems? Should the school day begin with an act of collective worship? Should pupils wear blazers and ties? This is why it was such a breath of fresh air to hear Frank Furedi introduce himself as the “antichrist at the last supper”. He declared that ‘scientism’ is endemic in education, and rightly observed that it is almost unheard of in education debates for people to appeal to the “evaluative, normative language of ethics or morality”. I would not go so far as Furedi to say that I am an ‘educational pluralist’, but I would agree with his claim that much education research is too often an unnecessary distraction.
It was terrific to hear former Gove adviser, Teach First employee and all round brainbox Sam Freedman refer to the psychologists Jonathan Haidt and Daniel Kahneman in his talk on “Evidence-based Policy Making”. Both psychologists have written about the dominance of the subconscious, emotional part of our minds, over the logical, conscious part. In his book The Righteous Mind, Haidt writes of the “rationalist delusion” that human beings can be swayed by logic alone, concluding that the first principle of moral psychology is that “intuitions come first, strategic reasoning second”. Similarly, Nobel Peace prize winner Kahneman has spent a large part of his career building evidence to show that evidence, paradoxically, rarely influences personal judgement. One of the most sensible things that I heard all day was from an audience member during Freedman’s talk. The lady said, “I don’t want evidence based policy. I want evidence informed policy. Basic values are important.”
ResearchED was a fantastic, thought provoking event and the calibre of discussion and presentations was remarkable. However, I was occasionally alarmed by cases of evidence-zealotry displayed by those present. Prior to working out what works, we need to work out what we want. To solve those questions, we cannot abandon our human faculties of judgement for the false Gods of educational research. As David Hume wrote in 1739, “reason is, and ought only to be the slave of the passions, and can never pretend to any other office than to serve and obey them.” Hume had no evidence to make such a statement. He just happened to be right.