Highlights from The Exam Man podcast Season 1, Episode 12
Michael Turner, Executive Director of Customer and Product at AQA, talked to us about the lessons he has learned and the wisdom he has gained from his extensive experience working in the world of exams and education. We covered topics such as the role and processes of exam boards, innovations in digitising exams, and ensuring fairness and security in the exam system.
Michael, could you just start by telling us a little bit about yourself, how you've ended up working for AQA, and what your journey in the area of exams has been?
Yep, sure. So I've been at AQA for nearly six years now. I'm currently the Executive Director of Customer and Product. A big focus of my work is about our customers, our schools and our colleges who take our qualifications, but also the products- our specifications. our examinations and all the resources as well that we have that support teachers. I've been in education for about 20 odd years in different roles. I've worked for another exam board back early on in my career in the press office. I know I won't mention who they are! So yeah, so I worked there for a few years in the press office, which was exciting and interesting. I've also worked for the Specialist Schools and Academies Trust, also in a media/ public affairs role.
Then I went to the Joint Council for Qualifications (JCQ)- Initially as Head of Policy there and then became the rather sort of grand title of Director General which is basically CEO. I ran JCQ I think for about eight years and I knew the exam boards quite well because obviously they were our members. I knew AQA very well and the opportunity came up to join them in a corporate affairs role. The rest is history as they say! I've been here nearly six years. I must say, my time at JCQ is interesting, because obviously we went through a lot of the reforms when I was there. So yeah, it's been an interesting few years.
What stood out from your time at JCQ that you were responsible for introducing, that people might like to be aware of?
Oh, I don't know about sort of me introducing. The thing about JCQ actually is that it's got a small membership and it works so closely with them. So, it's about bringing them together to act as much as they can with one voice. And I think that the effort is made to make things as simple as possible. It's not always possible. It is a complex, complex area, but by having one set of rules or instructions for conducting examinations- that yellow booklet that I'm sure people are aware of!- is key, rather than having lots of different processes for people to follow. The time of reform was very interesting because as well as coming together, the awarding bodies also compete. Trying to navigate that was always challenging, but successful, I think- what was implemented in a very short period of time was quite amazing.
Could you tell us a little bit about the role and aim of exam boards?
So, I think it's about assessing and awarding for people's skills and knowledge. The primary aim really is that, but it is also about progress. It's about enabling the student to be able to progress onto the next stage of learning, whether that's onto GCSEs, onto A levels, onto a degree, onto vocational courses, or even into the workplace. The government sets the content for the subjects and what the exam boards then do is take that content and some of the parameters in which we have to work and set the specifications. That's where the awarding bodies can differ- they can differ in terms of the specification, perhaps the topics that are chosen or the books that are chosen that meet certain government criteria.
Then it's about making sure that there's support for the schools and colleges, the teachers and the lecturers, and to be able to provide training and steer people through that course. And then to assess them at the end and to make sure that there's a standardisation and a level playing field across the exam boards as well.
Actually, it's a very collegiate sort of way of working across the exam boards. You know, one thing we really don't compete on is standards. It's the same standard.
So how do you work with Ofqual?
Very, very closely as you can imagine. They're the regulator- they regulate the qualifications. They will make sure that our specifications are approved and meet certain criteria. They also make sure that when we come to setting exam questions, exam papers, that they're fair, that we follow certain rules and guidance, that we have certain things in place for modified papers and for those students with disabilities or special educational needs. They look at how we're marking and awarding, checking that we do so fairly, and they have maintenance of standards meetings where the awarding bodies come together and we look at data. We look at prior achievement of certain cohorts. We also look at examiner judgment as well. We look at all of this together and say, well this is where we're going to set the grade boundaries this year.
Could you talk to us a little bit about some of the practicalities of being an exam board? From an exams officer point of view, I always think it must be the most enormous operation to pull off.
I think the exam boards look at the schools and colleges as well as the exams officers and say that's a hell of an administrative process to work to, to follow. But you're right, it's a really busy time for the exam boards. So, once the exam's been sat and the exams officers have done their end of it and sent the scripts back to us, they go to our centre in Milton Keynes, where we have a large number of scanners. We separate the pages out and we scan, I think it's over three million bits of paper a day. It's an amazing thing to see- you've got these scanners lined up and they're whizzing through. You get these big trucks coming into the back of the warehouse, offloading it- they're in these crates and then they get sorted.
It's the meticulous way that it's organised that is absolutely phenomenal. It really, really is. Obviously then they're captured electronically. For some subjects, they are then electronically cut by question- so an examiner will get a batch of, for example, question 4 on this paper. The idea behind that is they get used to teaching it, they can standardise much more easily, and not get distracted about what else is on the paper- as if they're marking the whole paper, they might start to form a judgment about this student. They will mark these questions on screen. They're then captured and then put back together into a paper, giving an overall paper mark. We have in the region of nearly 40,000 examiners that we use each year- the vast majority of those are teachers, people who are in the classroom. Exams officers are absolutely critical to the process. Examiners are absolutely critical to the process.
Once the marking's done, it's on to awarding. We need to decide, how did that paper compare with last year? Was it slightly easier? Was it a bit more difficult? And there's lots of statistics and evaluation from examiners that we can use to do that. And based on that, we can set grade boundaries. So, the grade boundaries are set by the exam board each year for each subject and they're put in place. During this process we work with the regulator and we work with other exam boards to make sure that the standards are maintained across the exam boards. So that's a huge, huge undertaking.
When I first started working in this sector, we didn't have capturing by scanners. It was just starting to come in. You would have bundles of scripts being sent across the country. It now means that once the papers are back in the system, we've got them, they're tracked, they're there, they're captured electronically which is a huge, a huge, huge, huge part of it. And then, of course, once we've got that, we've got results day. Which is what everyone's sort of building up towards and, you know, you can almost feel the anticipation across the country. I'm not sure other countries have it like we do, but you can sort of feel the anticipation, can't you?
It's about a week or so before results day that we start sending the results through to UCAS. UCAS has to then process them for those who are going on to university, so we need to ensure the accuracy with it and that all the checks and balances are in place. And then we send them into the schools and colleges the day before, so they can have a look at it and make sure that everything's correct.
Our job then is really about being available. We've got people at the end of the phone and we've also got live chat as well on our website. We take several thousand calls a day around results time. It's a large number ranging from the simple to answer through to perhaps a bit more complicated, perhaps a result hasn't come through or there's a query about a result that we then investigate. For parents and students, the first port of call is your school, as the exam centre, who will then largely do the communication with the exam board- to make sure that when students are getting their results, they've got somebody they can talk to who understands it, who understands the process, who understands what's next available to them. Perhaps they didn't get the grades that they wanted or they expected. Perhaps they haven't got what's needed to go on to university. At school there's someone there to support them. It's interesting. It's a topic that's been discussed for many years about well actually, could you send results straight to the student? My personal view is that the school or college or examination centre is best placed to provide that support when it's most needed. I'm a big advocate of results days.
Now there's a lot of talk at the moment and a lot of action as well around the question of digitalisation. I've noticed that AQA is taking about what I would describe as a kind of forward looking but careful approach. Would that be fair?
Yeah, I think that's fair. I mean it's a sort of evolution not revolution. The digital move to exams has been spoken about for decades- it's been a really long period of time where people have spoken about it. I think now we're at a different point in time where we're really talking about when not if. I think there's a real desire to make this happen. It's the way students are learning increasingly. It makes sense for something on the scale of the system that we've got, it creates more opportunities.
If you think about an exam in- let's say geography- what you could do onscreen in showing a volcano or tectonic plates or something like that, and how engaging that might be in an examination- there are some real opportunities. But what we're not saying is it's about everything moving digital, and it's not about everything moving digital in a short period of time.
We'll always have handwriting for a long period- many, many decades. I can't see that disappearing. Some exams perhaps might not lend themselves to having it on screen. Other exams really lend themselves to it, and we just need to look at that, we need to understand what do schools and colleges need, and how are they set up? How can they do that? What's the technology? How does that work? But I think it is very much when's it going to happen. We've spoken about venturing into Italian and Polish, for example, last year. Languages is one of those areas that lends itself to on screen exams.
But we need to move together. We need to move with the regulator. We need to move together with government. So, as I say, it's something we're progressing. But actually, also, it's not just about the summative exams. It's not just about the GCSEs and the A Levels and the end exams. It's about the formative stuff as well. And how do we create exams the way that schools and colleges are teaching their learners? How can we use technology for that? We've got a couple of things. We've got a new diagnostic tool. We've got one for maths and we're looking at other assessments that will be on screen- what you can do from that is you can capture so much data.
One phrase that's familiar to exams officers everywhere is the idea of the normal way of working. So, the idea that assessments should reflect the student's normal way of working. When you're changing a system, is that quite a difficult thing to manage?
It is a challenge and it's not one that can't be overcome. And I think that's where we need to work with government. We need to work with schools and colleges to look at what is in place. We can't have a system where we say, well, we're going digital and actually large parts of the sector can't deliver that for whatever reason. Interestingly as well, we need to make sure- whereas at the moment technology can be used for those students with special needs as their normal way of working, we may have a small number for whom paper actually continues to be the right way of doing it.
But that would be the exception. What we're saying is, quite strongly and believe quite strongly, is that you can't have a system that's got, I don't know, 30 percent doing it on screen and 70 percent doing it on paper.
We run a student advisory group with about a dozen students who- and they are proper advisors- advise our board, our board of trustees. They are really, really key. And when we asked them the question about digital examinations, the first thing that they came back with and the strongest thing they came back with was fairness. How do you ensure that there is fairness across the system? And that's got to be the fundamental.
What else do you see in the future of exams?
It's a really, really good question, and with an election soon upon us and the change in government. As you'd expect, we're not a political organization in that respect.
I do think there's an understanding about progression- of what does that actually look like? Are we making sure that we are equipping all students with the skills and the knowledge that they actually need to progress? We're looking at, for example, another qualification which would be a digital one around maths, that is more about real-life maths. About what you use on a day-to-day basis- percentages, working out if there's 25 percent off in a sale or what's your APR on a loan or something like that.
Could you talk to us about what some of the challenges for you as an exam board are around maintaining the security of the exam system, and therefore its integrity?
There are various parts to that, various stages of the exam process where security is paramount. It's really, really important. And that starts, obviously, with the exam board and making sure that when we're writing questions and when we're putting papers together that there's security around that.
The regulator, Ofqual, is very clear about the processes and who can see papers, who can't see papers, and what they can then do as part of their job. So that's quite tightly secured. The other part is the actual sending of papers into schools and colleges and making sure that we're tracking them, that they're processed, that any problems, the exams officer contacts us as an exam board, and we can try and help if there are any issues or any problems.
I've been to quite a large number of schools and colleges and there are interesting variations around the high security they have for storing papers. I've been to one place, I think it was an old bank, that has this big old vault! We need to make sure that the processes we have is that we send out papers on the morning of exams, as your exams officers know. Because a leaked paper causes phenomenal chaos and difficulty for the school. And we've had to put extra security in place to stop this from happening.
With social media it is so much quicker that this sort of stuff gets out. It gets around and it means that we often have to then recall the paper, which means exams officers have to then send those back and make sure that the new ones are sent out. We then have to monitor and make sure they've all come back in and they've all gone back out. It's a really big, really big thing. It means putting extra things in place such extra checks and balances, which can sometimes be a bit, you know, not always welcomed. It's a bit like when you go to your bank, isn't it? And you're asked all these questions, but you're pretty glad that it works and it means that, you know, no one's taking money from your account.
In broad terms, what do the threats look like? Particularly on the digital side, with downloadable papers?
Well without going into detail-I feel like I'm going to give this answer sort of like an MI5 operative!- but I can't tell you exactly because that would be telling people how to go about it. I'll have to kill you! In broad terms, the threat is about systems.
It's about parties getting into systems, not necessarily to the source. They can find other ways other ways around it. And in such a large system and where it's time critical, you know, the exam starts at nine o'clock. How do we make sure that we're doing the verification process correctly and accurately?
So, that can be done, in various ways. As is often the way with criminals and cybercriminals, they can often be one step ahead, so how do we try and keep ahead of them? But, you know, touch wood- it's going well. And a huge thank you to exams officers who, especially on the morning of exams, it must be a hugely stressful time for them.
The interesting thing is, though, that we get cases of leaked papers every year on social media. And nearly always they are fake. Some of them are quite good, some are absolutely appalling! But you know, and I think that's the thing is, you know, they're going to be fake if you see one really. It's targeting people who are at that moment in a vulnerable state because they're revising and they're under that stress.
The process is now tighter partly because exam boards have tightened up their processes and understand where the vulnerabilities were. We've also got a regulator who's very focused on that as well.
it's been nice talking to you both. Good luck with your podcast as well. It's great.
To listen to and read more interviews like this, subscribe to The Exam Man - the #1 podcast about exams and assessment on your favourite podcast platform.
To find out more about Examscreen, please contact us: Examscreen