Obsessed with Assessment

Highlights from The Exam Man podcast, Series 2 Episode 14

Recently, we teamed up with Sam Denno, Educational Assessment Leader at Twinkl, for an episode of his brilliant Obsessed with Assessment podcast. Sam interrogated John about the English exam system, how it works and the purposes and appropriateness of high stakes exams for both students and schools.

Sam: John, can you tell me what it means to be an Exam Officer?

John: The first thing it means is that you're tired quite a lot of the time, particularly in the summer months, though Exams Officers get very annoyed if you only talk about the summer exams, as if there aren't other exams that take place throughout the year! Your job is to organise all the exams that take place in a school. That includes obviously all the public exams, but mock exams as well. It differs across schools. We have a small sixth form, but other schools might have very large sixth forms, doing a whole range of different kinds of courses. Then there’s all the qualifications that you've got in Year 11 and running exams for younger years as well. It's a full on job throughout the year. It has its obvious pinch points. But basically, what you're trying to do is to make that experience as good as it can be for students. It's a stressful time, particularly the high stakes exams are stressful. You're trying to organise it well, so that it's a smooth process for them. And also to help them to get the most out of that experience.

Sam: Do you have interactions with the students themselves?

John: Our podcast is called The Exam Man, and the joke behind that is that none of the students know who you are until Year 11. And then they really, really know who you are. You're like a malevolent presence in their life throughout the year! So you do have interactions with students, but principally in those exam years. In Year 11, Year 13, they will often come to you with questions about exams and how they work. And if they've got any worries or concerns, you'll speak to their parents often as well along the same lines. I do assemblies, go and talk to tutor groups and things like that, to try and explain the process to them. I think one of the things that sometimes gets overlooked is that obviously you're very focused on the content of what we're assessing, but the process itself is something that can have its own barriers and its own issues. Helping them to navigate that as well is also quite an important thing to do. We definitely try and get out there as much as possible and interact with the students- you still have to do duties and stuff. Toilet duties, break duties.

Sam: Are you also a classroom teacher then?

John: No, I'm not. For a while our school had a small sixth form and I did actually teach A-level for a few years. I'm not doing that anymore because our sixth form's massively reduced and is closing. But I did do that and so I had some view of it from that side as well. But I have to say teaching and being an Exams Officer, though there are people who do it and do it very effectively, it's really difficult. I'm the Data Manager as well, so I do those two jobs. It's changed a lot actually. When I started in 2011, there were a lot more exams because students did modular exams back then. I started as the Exams Officer, and I also had an exams assistant. Plus there was a separate Data Manager in the school as well. Now I do all three of those jobs effectively. I don't know whether Exams Officers will thank me for saying this, but the role of Exams Officer has slightly reduced in some respects since 2015/16, as there are fewer high stakes exams. That doesn't mean there are fewer exams because they often get replaced with mocks etc. But there are fewer high stakes exams now than there were, because you don't have as many exams in November and January as you used to.

Sam: Does every school have an exam officer?

John: Yes. You have to. If you're running exams, you have to have a designated Exams Officer. But that person varies from school to school. In some places, you'll find that that person might be a member of the SLT. That was quite an old school model, I think, so if you went back to maybe 20 years ago, then I think you'd find a lot of the time that Exams Officers were members of SLT who had been given that responsibility. As the role grew and it became more demanding, they then had a specialist person in schools to do it. The person in your school who is designated as an Exams Officer is the person who the exam boards will interact with principally. You also have a Head of Centre, who is usually the Headteacher. It got described by someone on our podcast that the Head of Centre is like a ceremonial monarch. The Exams Officer does all the stuff, but the Head of Centre is there to wave and sign things off.

Sophie: Unless something goes really wrong and then they really have to step in and then it’s less ceremonial, but hopefully that never happens. And they'll generally just blame you for it!

Sam: So I suppose what's mad is, I've worked in a number of schools, and I would not have known who the Exams Officer was.

John: Sophie said that on a podcast as well, which is nice because had I been working in the same school as her, she would have noticed who I was......

Sam: What led you to this, John?

John: It wasn't planned. It wasn't intentional at all, which again is another theme that comes up. No one sets out to become an Exams Officer.

Sophie: I really want to talk to someone who does. I really want to find someone who, when they had their careers chat at 16 in school, said I really want to do that job in a school.

John: I've seen this person running around the school, really stressed out, and I'd like to do that job.

Sophie: If anyone has listened to this, who wants to come on and talk to us about always wanting to be an Exams Officer, please get in touch? Most people have fallen into the role, in quite interesting ways.

John: Sophie was a teacher. We were living in London and she got a job at a school in Brighton that she really, really wanted to do. So Sophie started commuting down to Brighton from London every day. But obviously in schools, you often have to get there really, really early. And it just wasn't feasible, was it? So we moved to Brighton. At the time working I was working at London Bridge in government. So I was doing the commute from Brighton to London every day, which is not a particularly long train journey, but it is a particularly terrible train line. Every day it seemed I was getting stuck at Haywards Heath station for a couple of hours. I'd be leaving the house before 6, getting home after 8 and I just thought this is rubbish. I'm not going to continue. But we enjoyed living in Brighton, didn't we? So I just started looking for jobs locally and it wasn't that easy to find a job. In the end, I hit upon this one at this school, which was looking for an Exams Officer. And I thought, well, I've worked in a school as a TA before, I know the environment, but I don't know anything about the job. I think they just had a real dearth of people applying for this job, because I didn't know what I was talking about at all. But they offered me the job and 13 years later, I'm still doing it.

Sam: That's interesting. So even though you taught A level at that school, you're not a qualified teacher?

John: I'm not a qualified teacher, no. They didn't have someone to teach a subject, which I had a degree in. So they asked me if I would do it, and I did for a while. I really enjoyed it, but it was a lot, it was hard work, it was stressful, and it created all sorts of conflicts of interest that you have to deal with as well, because you're running the exams, but you're also teaching an exam subject. It makes life very, very complicated too.

Sam: Interesting. Well, can I get your thoughts on this then? And perhaps, then, as the teaching specialist, Sophie, you might be able to chip on this as well. Do you think that high-stakes exams are effective? You're not going to say no because they give you your income, but are they effective?

John: I think it depends on what you're trying to achieve. So, like with any assessment, I think it's all about what the purpose of that assessment is, and then trying to work out what is the best way to achieve that purpose. I think that there are considerable sort of trade-offs as well. One of the things about GCSE exams, for example, is that they really act as a sifting tool now. No one now should be leaving school at 16, but continuing in some kind of education or training until 18. They're basically like a sifting tool, which gets you to the next stage of where you need to be. So, I think there is a strong argument that because that's what they're doing, that they are a bit too high stakes. It feels like too much pressure for something which is just basically sifting you onto the next stage. But on the other hand, I think creating high stakes is at times important too. If you really want to understand what people can do, and you really want to push people to achieve their best, then creating high stakes around something is one way to achieve that. So I'm not against high stakes exams per se, but I do think that maybe they're overused at times. GCSEs is a good example of where I think they might be kind of slightly overused. That's basically my very fudged answer to that question. What do you think, Sophie?

Sophie: Over the years there's been lots of attempts to do different things, to assess children differently. It just keeps coming back to exams for a reason. I think this is because it is the kind of best case that we have for assessing children at the moment. And I think also it's about the wider purpose of going to school. It's not just about what you learn in lessons, but also what does sitting high stakes exams give you overall as a child at school. I think that's not as often discussed, but teaching resilience, high pressure situations, getting used to that before you enter the workplace is really important as well. I think there's other benefits to it, which probably factors into why they endure as well.

Sam: If I may play devil's advocate for a moment then. John, one thing you said is creating high stakes can be important. You did say can be important. I've been looking at three Nordic countries recently, Denmark, Finland, Norway, in which there are very, very few high stakes exams and these are held up as being among the most prestigious education systems in the world. They don't think that creating high stakes is important. You probably don't have a comeback for that.

John: Well I do. I think what I was basically saying is that in some circumstances creating high stakes is important. So for example, I would want any doctor that I was treated by to have sat some high stakes assessments. If the purpose for which you're assessing people is really important, then I think it's important that they are assessed in a fairly high stakes way, because otherwise I don't think that you can guarantee that what you're getting out the other end is a really, really true picture of what the person can do.

I'd say another thing as well, which is there's a kind of accountability thing to high stakes as well. I'm a type 1 diabetic and I have to assess myself every day. I measure my blood sugars every day. And that, I guess, is a kind of formative assessment. So I'm gaining feedback every day thinking, okay, what do I need to do differently? If I eat this, how much insulin should I take? If I'm exercising, what should I do differently? So that's the sort of model of formative assessment. However, often I don't do those things well and I lose track of myself. And one thing that's really good for me is every six months I have a high stakes assessment with a medical practitioner who takes my entire health check. This looks at my blood sugar over three months, tells me off if I'm doing it badly, sets me straight, puts me right, and says, you're not doing this properly, you need to do it properly.

I think injecting a little bit of high stakes into things just focuses the mind and ensures that you can get the best out of what you're trying to do. I think there's a nice view, which is that we can all be intrinsically motivated all the time to do things, but I think that's idealistic in that, at times, people need some extrinsic motivation as well, because you do lose your way with whatever you're doing. Particularly, kids who are studying at school can lose their way. And I think having that focus, that you need to get to this point is important. I think it has its place. But I do agree that it shouldn’t be the central focus of any kind of system. And I don't think it's necessarily the best way to help students improve, but I think it can work in some circumstances and in others it's not so appropriate.

Sam: That was actually on our formative and summative episode, which you can find in our back catalogue. One of my conclusions was that it's a blunt instrument of a form of assessment. In the system that we've got in this country at the moment, actually sometimes blunt instruments do motivate people, not motivate, is that the right word? But certainly push pupils towards achieving an end goal. But my feeling is it's much more about accountability, school accountability, than it is about pupil achievement and outcome, because it doesn't have to be assessed in that way.

John: So do you think it's good for school accountability? Do you think it works for measuring schools and how they perform?

Sam: Again, within the system that we're working with in England, yes. Can you imagine a world in which next year we just removed Ofsted and said there is no accountability. Schools, you're answerable to your local authority. What would happen? I don't know the answer to that.

Sophie: It would be great to trial something really, really different. But the problem is that when you're dealing with children, they get one chance at being this age, and everything is so dependent on the next stage, that doing really wide scale experiments of the system is going to impact large cohorts of children, possibly really positively, but potentially also catastrophically negatively as well. So I think that's why in education, things are deliberately cautious. I think you see this across education, any reforms need to be deliberately cautious and careful for a reason, because of who you're dealing with. We've been talking to lots of people about the Curriculum and Assessment Review. Obviously we're particularly focused on the assessment aspect of that. I think everyone we've spoken to has that real sense of how important it is, and that if anything comes out that people want to introduce, it needs to consider who is this ultimately affecting over these next few years?

John: Can I ask you a quick question, Sam? So when you talk about other countries having, potentially better systems, what is it that those systems are better at doing?

Sam: That does lead to my general thesis on how ideally the exam system would look. They do formative assessment much better, or much more effectively, than we do in this country. Not only do they do the formative assessment that we do fairly well in the classroom, just like the checking in, the hinge questions, the one-to-ones, the guiding the student onward to the next piece of work, that stuff, which we do OK here, but they also treat their summative assessment as formative assessment. So my example - I did three A-levels. I did English, Art and History A-level. And I was predicted a B in each of those A-levels. I got a B in English, I got a B in art, and I sat my history exam and I thought, I've done really well in my history exam. I absolutely smashed it. I'd got an N. Do you remember N grades?

Sophie: Oh, vaguely. Like a U, wasn't it? Sam: Well, there was a U as well. It was like a U plus. I don't know why. #

John: U plus or U minus? Was it better than U?!

Sam: I think it was better than U. But it was like no grade. I think nowadays, that would be challenged straight up. The school would be saying, okay, well, let's go back to the exam board and find out why. But back then, in the mid-90s, that wasn't the case. It was fine, because I got onto a degree and I got the course I wanted. But I do always look back at that and think, I don't know why I got an N. Maybe I struggled a bit during my degree because maybe I wasn't very good at writing essays and that sort of thing. Now, had I had somebody sat down with me and said, okay, here's the assessment.

John: I completely agree with that. In that situation, you are left with a feeling that you're not very good at this, but no one's explained to you why. There have been improvements in that you are able to obtain some of that feedback now. It's still not that easy to do, though, and often there's a cost attached to it as well. I do agree, if you take any form of assessment, you should be able to find out afterwards what you did well, what you did badly, how you can improve. Even if it's a summative assessment, I don't think there's any good reason for not giving or attempting to give people that feedback. Obviously in the UK you're talking about vast numbers of students, vast numbers of exams. So the system is huge. I think often we do run into problems from a scale point of view. What is it possible to do at that level of scale? And I think you see that with digital exams as well, trying to roll that out. For most people, that would seem just like an absolute no brainer, surely the whole system would be more efficient, easier to manage and for students would probably, to a large degree, now represent their normal way of working. But it will take time, I think, to roll that out just because you're trying to create this new system at scale and you’ve got to try and take everybody with you and make sure that it works because it is high-stakes. If it fails, then it's a disaster. Maybe for countries like the UK, France, big kind of Western countries the problem is how do you examine, assess at scale for a national system?

Sam: To go back to those Nordic countries, what I suppose they do very well is at those transition points, they mop it up much better. So at the transition point, you've got much more communication between one school and the next. The middle school is 10 years, and you've got communication all the way along, and you might often carry the same teacher over. Like you say, scale is an issue, but these are things that we could embed better. We all know how awful transition is, kind of from primary to secondary, and secondary to sixth form, in certain institutions. It's not seamless.

John: I often see as well people who have got what they need, so they move on. But again they're not obtaining any feedback about it. The exam was just a means to an end. They don't want to know. They're not interested. It doesn't matter.

Sophie: But that feedback is actually really important for teachers as well, for the next cohort.

John: And with the transition, moving on to the next place, you'd want to understand a bit about what that student's particular strengths and weaknesses were rather than just a set of results. It's interesting when you do things like CATS assessments with year 7s which I run. You actually get quite detailed information back from those which you can then use throughout your school to inform practice. You can understand a lot about students' abilities across a range of different areas, and you can highlight strengths and weaknesses, and then everybody's got access to that information and they can use it to inform their teaching. You're absolutely right, I think, with any kind of assessment, getting decent feedback from it is an absolute must.

Sam: And Sophie, was it you that just said about the benefit to teachers as well, that it's a learning partnership, between the teacher and the student, so actually everybody should benefit from assessment, not just the pupil or whoever is benefiting from it? It should be the teacher saying, okay, I understand where you're struggling, and therefore I can adapt my teaching further, and it should be the pupil being able to say, okay, I understand where I'm up to, and therefore I can adapt my performance. So I think what I found interesting, and I said this in my introduction to my podcast, were some of those really straightforward assessment facts or bits of information hacks, ways of doing assessment that as a classroom teacher I wasn't even aware of. And I was always a classroom teacher, not in leadership at all. I knew I had to run assessment, I knew I had to get my pupils to that next big test, I knew I had to do formative assessment in the classroom, but there is so much more to it than I even had any comprehension of. So I suppose for me with the Obsessed with Assessment podcast, what I'm thinking is, if I didn't know that stuff, then who does? And maybe it's because I was particularly head in the clouds, had no idea what was going on, but I suspect there are a lot of teachers out there who think, I have to just roll it out.

John: So are you talking about techniques here, Sam?

Sam: Let me tell you some of my favourite things that we’ve spoken about. So the value of formative and summative assessment respectively, which obviously we've touched on quite a lot here. Terms that you bandy around, and I was quite comfortable with the distinction between the two, but never gave any thought to it. Like you said, John, what is the purpose? Why am I doing this? Except beyond the obvious, to get a result from my pupil. The use of formative as that more sensitive, developmental learning partnership tool for students and teachers, I would never have thought of it in quite that way. Then the use of summative as that more blunt instrument for hammering out outcomes, kind of carrot and stick for pupils, but also for school accountability. Again, I would never have thought of those things in that way. So it's just been quite eye-opening. What else have I enjoyed doing? In terms of technique, I had never given metacognition any thought before. I'm in a very fortunate position that I'm working for a business that gives me time to be able to just sit and read, and to find out about how this all works. But when I was teaching, I didn't have time to do that. So metacognition, techniques like distributed practice…. Are you familiar with distributed practice?

John: No, I'm not.

Sam: Distributed practice is the idea of spacing out learning. So instead of saying I've got two hours to revise this test that I've got on Friday, it is much more effective not to revise for two hours on Thursday night, but it's much more effective to do half an hour on Monday, half an hour on Tuesday, half an hour on Wednesday, Thursday, because that embeds it into your permanent memory much more effectively, and therefore it is much easier to retain and to retrieve at the point of the exam. So something like that is a really simple tool to be aware of, both in terms of the way you deliver in your class, but also in terms of how you advise your pupils. I had no idea that that was even a concept. And then things like retrieval practice, retrieval track practice being where you test yourself on the piece of work that you're revising. I did a podcast on this, but that distributed practice model, if you do half an hour on Monday where you do absorption of the material, then half an hour on Tuesday where you test yourself against it, half an hour on Wednesday you do a little bit more reading and assimilation, half an hour on Thursday where you test yourself against it, then you've got a far better chance of getting better scores on Friday when you go into the test. It sounds like a no-brainer, but to me it was just a moment of revelation.

John: And what about the importance of assessment? How do you think, as a classroom teacher, you would now rank the importance of assessment in terms of the development of students.

Sam: I would say very important, but with that understanding that to me it's the formative assessment side of things, which is the important side for the pupil. Like we say, I think the summative assessment is good for school accountability, but, simply put, I think all assessment should be formative, even when it's summative. So, as we've discussed already, if you've got an exam, that needs to be used formatively. If you've got an end of unit test, that needs to be used formatively. If you're doing formative assessment, it obviously is being used in that way, so it's very beneficial.

John: One thing that's really interesting is that we always used to run all our mock exams right at the end of term. So the kids would do their mock exams and then they’d go off on holiday, and then come back and get their results. Thankfully after quite a lot of pushing, we've moved the mock exams now, several weeks forward. So when the kids do the exam, there's actually a process, so that they're not just treated like a pure summative assessment.

Sam: Well, Sophie and John, that has been really, really fascinating. I suspect we've probably gone long I'm going to have to trim you down in the edit, because you're just so interesting.

John: Thanks. It's been a pleasure!

To listen to and read more interviews like this, subscribe to The Exam Man - the #1 podcast about exams and assessment

To listen to Obsessed with Assessment, go to: Obsessed with Assessment | Podcast on Spotify

To find out more about Examscreen please contact us: Examscreen