All a-Board! (with AQA)

[00:00:02] I've worked for another exam board back early on in my career in the press office, I know,

[00:00:29] I won't mention who they are.

[00:00:37] Sorry what?

[00:00:38] So it's one of these people Michael who boasts about not being very good at maths in a way

[00:00:45] that nobody would ever boast about not being able to read.

[00:00:48] It's interesting isn't it that?

[00:00:50] Yeah.

[00:00:51] Hello and welcome to The Exam Man podcast, I'm your host John Gaston and I am here with

[00:01:15] my highly numerate co-host Sophie Gaston.

[00:01:19] Thanks John, how are you?

[00:01:22] Yeah I'm alright, I'm not too bad, I had my JCQ inspection today.

[00:01:27] Yeah you can tell that just from looking at your face, you've got a bit of like colour

[00:01:31] that's come back.

[00:01:32] The sense of relief is very palpable I can tell you, so we got through that which

[00:01:37] is good news.

[00:01:38] It would be a bit embarrassing wouldn't it doing this podcast, they're not going

[00:01:43] through the JCQ inspection.

[00:01:44] Yeah our centre hasn't been closed down and I haven't been struck off so this

[00:01:48] is all good news.

[00:01:50] It's been a tiring week again and yeah it has started to feel a bit sort of emotional

[00:01:57] as well towards the end of this week which wasn't helped by I think one of the most

[00:02:02] lovely messages I've seen about this podcast.

[00:02:05] Oh yeah you had a little tear John.

[00:02:06] I did which we had from Lesley Haldane who's the exams officer at Sir William

[00:02:11] Borlay's grammar school.

[00:02:13] She basically just said she'd been having a pretty rubbish week as well but she'd

[00:02:17] been using the podcast to kind of help her get through it and was enjoying all the help

[00:02:23] and support and stories as well from other exams officers which were helping her through

[00:02:30] this tough week in the cycle which is great.

[00:02:33] That's lovely.

[00:02:34] We also had a message off someone called Denise who said that she'd been listening

[00:02:39] to the podcast while she was running, do you remember that one John?

[00:02:44] That was lovely to hear that she was listening to it that way but I'm mainly just so impressed

[00:02:49] that one of you is doing Couch to 5k at this point in the academic year as an exams

[00:02:55] officer.

[00:02:56] I'm so exhausted every single day.

[00:02:57] I'm doing like 20,000 steps and…

[00:02:59] Just all hell Denise basically.

[00:03:01] Yeah, yeah, Denise Thornley.

[00:03:03] Top work, top work Denise.

[00:03:11] So something very exciting has happened to us over the last week hasn't it?

[00:03:17] Absolutely massive, yeah.

[00:03:19] We have been invited to Milton Keynes.

[00:03:23] Which is something I wasn't expecting, so I'll wait to be honest.

[00:03:26] No, no me neither, me neither.

[00:03:27] It's been a big week.

[00:03:28] Do you want to explain why this has happened?

[00:03:30] Yeah, yeah I would like to.

[00:03:32] So we are very happy to be joined on the podcast this week by Michael Turner who

[00:03:37] is AQA Executive Director of Customer and Product and we had a wonderful far-reaching

[00:03:44] chat with Michael.

[00:03:47] All about everything exams really covered a lot of ground didn't we?

[00:03:52] Yeah we did, we talked about the role of exam boards, we talked about the processes

[00:03:59] that exam boards go through to get students to their qualifications, we talked about

[00:04:03] all the new developments in exams around…

[00:04:06] Yeah what he thought about the future of exams.

[00:04:08] Yeah the future of exams, around technology but also a lot broader than that as well

[00:04:14] and we also touched on the area of security which is a kind of hot button issue for the

[00:04:20] exam boards at the moment.

[00:04:21] For avid listeners of the podcast you'll know that John is particularly fixated on

[00:04:25] the journey of exam papers and what happens and we've spoken to people in Taiwan

[00:04:32] and the UAE about what happens to their exam papers afterwards so it was really

[00:04:37] quite exciting to have someone from the awarded bodies there to ask about literally

[00:04:42] what happens the moment a student puts their pen down, what happens to the papers.

[00:04:47] Yeah yeah so the Milton Keynes thing is that AQA have this enormous processing

[00:04:53] centre at Milton Keynes and you'll hear from the interview just like how

[00:04:57] impressive it sounds and also the fact that we get a cheeky little invite to

[00:05:03] go and have a look which I'm really excited about.

[00:05:06] Also genuinely that is our summer holiday.

[00:05:09] Yeah that will be the highlight of mine.

[00:05:11] Genuinely that is our only summer holiday so we really do have to make it count.

[00:05:16] Yeah yeah a little thing before we start this interview with Michael is around

[00:05:22] terminology so I keep talking about exam boards I noticed you just said

[00:05:26] awarding body right which I think is the politically correct term.

[00:05:31] Well that's me all over.

[00:05:32] But having said that Michael in this interview refers to exam boards quite

[00:05:36] a lot so you know it's anybody's guess who's right.

[00:05:39] I don't think anyone's offended.

[00:05:40] I hope not, I hope not. Anyway enjoy the interview.

[00:05:51] Michael could you just start by telling us a little bit about yourself actually

[00:05:55] and how you've ended up working for AQA and what your journey sort of in

[00:06:00] this area of exams has been.

[00:06:02] Yeah sure so gosh I've been at AQA for nearly six years now.

[00:06:09] I'm currently the executive director of customer and product so a big focus of

[00:06:14] my work is about our customers, our schools, our colleges who take our

[00:06:20] qualifications but also the products.

[00:06:22] So our specifications, our examinations and all the resources as well that we

[00:06:27] have that support teachers in teaching those.

[00:06:32] I've been in education for about 20 odd years in different roles.

[00:06:39] I've worked for another exam board back early on in my career in the press office.

[00:06:45] I won't mention who they are.

[00:06:49] So yes I worked there for a few years in the press office which was exciting

[00:06:52] and interesting.

[00:06:54] I've also worked for specialist schools in Academy Trust also in a

[00:06:58] sort of media public affairs role and then I went to the joint council

[00:07:03] for qualifications initially as a policy head of policy there and then became

[00:07:08] the rather sort of grand title of director general which is basically CEO.

[00:07:14] It sounds quite military doesn't it?

[00:07:17] It's funny some epaulettes or something but yeah so I was there so I ran

[00:07:22] joint council for qualifications about eight years.

[00:07:25] So I knew the exam boards quite well because obviously they were

[00:07:29] members and I knew AQA very well and the opportunity came up to join them as

[00:07:36] more in the corporate affairs world.

[00:07:39] So I joined there and then the rest is history as they say.

[00:07:43] I've been here nearly six years.

[00:07:45] So really, I just think Michael's the exam man rather than you John

[00:07:49] because you've got quite a niche experience of exams compared to

[00:07:52] those haven't you?

[00:07:53] That's a pretty broad experience you've had.

[00:07:56] I'm feeling like defenestrated right now.

[00:07:59] No not at all.

[00:08:01] I must say my time at JCQ is interesting because obviously we went

[00:08:04] through a lot of the reforms.

[00:08:06] The going through the reforms when I was there.

[00:08:09] So yeah it's been an interesting sort of year.

[00:08:13] Were there any particular things that stand out from you for your time at

[00:08:17] JCQ that you were sort of responsible for introducing that people

[00:08:22] might like to be aware of?

[00:08:23] I don't know about sort of me introducing.

[00:08:25] The thing about JCQ actually is that it works, you know it's got a small

[00:08:29] membership and it works so closely with them so it's about bringing

[00:08:33] them together to act as much as they can with one voice.

[00:08:37] And I think that for hopefully the people listening to this podcast

[00:08:41] and you know the effort is made to make things as simple as possible.

[00:08:46] It's not always possible.

[00:08:48] It is a complex area but by having sort of one set of rules or

[00:08:52] instructions for conducting examinations, that yellow booklet that

[00:08:56] I'm sure people are aware of.

[00:08:58] You know having that is key rather than having lots of different

[00:09:02] processes for people to follow.

[00:09:04] But you know during times of, you know the time of reform was very

[00:09:08] interesting because as well as coming together the ordering bodies also

[00:09:12] compete and trying to navigate that was always challenging.

[00:09:17] But successful I think, you know I think it was, you know what was

[00:09:21] implemented in a very short period of time was quite amazing.

[00:09:26] Could you tell us a little bit about the role and aim of exam boards

[00:09:37] because we talk about exam boards all the time but actually it would

[00:09:39] be really good to know your definition of that really, what the

[00:09:42] role and aims are?

[00:09:44] So I think it's about assessing and awarding for people's skills

[00:09:53] and knowledge.

[00:09:54] The primary aim really is that but also about progress.

[00:09:58] It's about enabling the student to be able to progress onto the next

[00:10:03] stage of learning, whether that's you know onto GCSEs, onto A

[00:10:08] levels, onto a degree, onto vocational courses or even into the

[00:10:12] workplace as well.

[00:10:15] So it's about making sure that a course of study that has actually

[00:10:20] initially been set by the government.

[00:10:22] So the government sets the content for the subject and what the

[00:10:26] examples then do is take that content and some of the parameters

[00:10:31] in which we have to work and set the specifications.

[00:10:35] And that's where the awarding bodies can differ in terms of

[00:10:39] the specification, perhaps the topics that are chosen or the

[00:10:44] books that are chosen that meet certain sort of government criteria.

[00:10:48] And then it's about making sure that there's support for the

[00:10:51] schools and colleges, the teachers and the lecturers to be able to

[00:10:54] provide and train and steer people through that course.

[00:10:59] And then to assess them at the end and to make sure that there's

[00:11:02] a standardisation, that there's a level playing field across the

[00:11:07] exam boards as well actually.

[00:11:08] It's a very collegiate sort of way of working across the exam

[00:11:12] boards.

[00:11:13] You know one thing we really don't compete on is standards.

[00:11:15] It's the same standards.

[00:11:16] Yeah, of course which sets it doesn't it across them.

[00:11:19] But in terms of how does it work with Ofqual?

[00:11:22] Very, very closely as you can imagine.

[00:11:26] So they're the regulator.

[00:11:27] They regulate the qualifications.

[00:11:30] So they will make sure that our specifications are approved and

[00:11:35] meet certain criteria.

[00:11:37] They also make sure that when we come to setting exam questions,

[00:11:42] exam papers, that they're fair, that we follow certain rules and

[00:11:45] guidance, that we have certain things in place for modified papers.

[00:11:50] So for those with perhaps disabilities or special educational

[00:11:55] needs, that we consider those when we're setting the papers.

[00:11:59] And making sure as well that when we're marking and awarding the

[00:12:03] scripts, the papers, that we do so fairly.

[00:12:06] So they have maintenance of standards meetings where the

[00:12:09] awarding bodies come together.

[00:12:10] And we look at data.

[00:12:11] We look at prior achievement of certain cohorts.

[00:12:15] We also look at examiner judgment as well.

[00:12:18] And we sort of look at all of that together and say, well actually,

[00:12:22] you know, this is where we're going to set the grade boundaries this year.

[00:12:25] And that's across all of the awarding boards.

[00:12:27] Could you talk to us a little bit about some of the practicalities,

[00:12:38] Michael, of being an exam board?

[00:12:40] So I always look at it from an exams office point of view.

[00:12:43] From an exams office point of view, I always think, God,

[00:12:45] it must be the most enormous operation to pull off.

[00:12:50] So we're obviously, as schools, involved with it up to the point

[00:12:55] that the students take their exams.

[00:12:58] And then you breathe a sigh of relief and go to sleep for a month.

[00:13:02] Yeah.

[00:13:03] And then we kick back.

[00:13:04] But obviously at that point, at the end of June,

[00:13:07] I presume you guys are not kicking back at all.

[00:13:10] So could you take us through the process,

[00:13:12] once the papers have gone to you guys,

[00:13:15] what happens then to bring about the students' results in August?

[00:13:20] It's interesting you say that about looking at exam boards

[00:13:22] and saying, wow, what a massive operation.

[00:13:24] I think the exam boards look at the schools and colleges as well

[00:13:28] and the exams offices and say, God,

[00:13:30] that's a hell of a lot of sort of administrative process work to follow.

[00:13:36] But you're right.

[00:13:37] It's a really busy time for the exam boards.

[00:13:42] So once the exams been sat,

[00:13:44] the exams officer has done their sort of end of it

[00:13:47] and sent the scripts back to us.

[00:13:50] They go to our centre in Milton Keynes where we have scanners,

[00:13:57] a large number of scanners.

[00:13:59] So we separate the pages out and we scan,

[00:14:02] I think it's over three million bits of paper a day.

[00:14:07] I know it's an amazing thing to see.

[00:14:10] You've got these scanners lined up and they're whizzing through.

[00:14:13] I really want a photo of this, just can't get it in my head.

[00:14:17] I tell you what, why don't we invite you up there?

[00:14:20] You can come and have a visit.

[00:14:22] This is like John's dream.

[00:14:24] Do you know what, I would really like to do that.

[00:14:29] And we'll try and do it when it's a busy time as well

[00:14:31] because you get these big trucks coming into the back of the warehouse,

[00:14:34] sort of offloading it, they're in these crates and then they get sorted.

[00:14:38] The meticulous way that it's organised is absolutely phenomenal.

[00:14:42] It really, really is.

[00:14:44] But obviously then, so then they're captured electronically.

[00:14:48] For some subjects they are then sort of electronically cut by subject.

[00:14:53] Sorry, by question.

[00:14:55] So an examiner will get a batch of question four on this paper.

[00:15:01] And the idea behind that is they get used to teaching it,

[00:15:04] they can standardise much more easily

[00:15:07] and not get distracted about perhaps what else is on the paper.

[00:15:11] OK, so that's almost like a slight protection against bias there, is it?

[00:15:15] That's right, absolutely.

[00:15:17] Because if they're marking the whole paper

[00:15:19] they might start to form a judgement about the student.

[00:15:21] Exactly.

[00:15:22] Oh, I saw something back there that all sorts of...

[00:15:25] But actually that's not in the question.

[00:15:27] Yeah, that's good, yeah.

[00:15:29] And we have, I think it's in the region of nearly 40,000 examiners

[00:15:33] that we use each year.

[00:15:36] And the vast majority of those are teachers,

[00:15:40] people who are in the classroom.

[00:15:42] So, you know, exams officers are absolutely critical to the process,

[00:15:45] examiners absolutely critical to the process.

[00:15:49] So they'll be sent those questions, they will mark those on screen.

[00:15:52] They're then captured and then put back together into a paper

[00:15:56] so you get an overall paper mark.

[00:16:00] And then what we have is, so once the marking's done, awarding.

[00:16:04] So once the papers have been marked

[00:16:07] we need to decide how did that paper compare with last year?

[00:16:11] Was it slight easier, was it a bit more difficult?

[00:16:14] And there's lots of statistics and evaluation from examiners

[00:16:18] that we can use to do that.

[00:16:20] And based on that, we can set grade boundaries.

[00:16:23] So the grade boundaries are set by the exam board each year

[00:16:27] for each subject and they're put in place.

[00:16:31] And through that process is where we work with the regulator,

[00:16:34] we work with other exam boards to make sure that the standards

[00:16:37] are maintained across the exam boards.

[00:16:39] Yeah, yeah.

[00:16:40] So that's a huge undertaking.

[00:16:42] It's interesting, when I first started working in this sort of sector

[00:16:46] we didn't have this really capturing by scanners.

[00:16:50] It was just starting to come in.

[00:16:52] So you would have bundles of scripts being sent across the country.

[00:16:57] That's what I thought still happened.

[00:17:00] So that's a big change then going back to our point about bias

[00:17:04] because previously someone would have marked a whole exam paper, right?

[00:17:08] They would have marked the whole thing.

[00:17:10] That's right.

[00:17:11] OK, so that's a major innovation actually.

[00:17:14] Huge.

[00:17:15] Not just a technical one but in terms of the quality presumably of marking as well.

[00:17:19] Yeah, and it means that once they're back in the system

[00:17:22] we've got them, they're tracked, they're there, they're captured electronically

[00:17:26] which is a huge, huge, huge part of it.

[00:17:29] And then of course once we've got that we've got results day

[00:17:32] which is what everyone's sort of building up towards

[00:17:35] and you can almost feel the anticipation across.

[00:17:39] I'm not sure other countries have it like we do

[00:17:41] but you can sort of feel the anticipation, can't you?

[00:17:43] Yeah, yeah, definitely.

[00:17:46] It's the day before for you, isn't it?

[00:17:48] So how stressful is that couple of days leading up to?

[00:17:52] Well it's actually about a week or so before that we start sending through

[00:17:56] the results through to UCAS.

[00:17:59] So UCAS has to then process them for those who are going on to university

[00:18:04] and to start looking at that.

[00:18:06] So we need to make sure that several days before that that's starting to go through

[00:18:11] making sure that obviously the accuracy with it

[00:18:14] and all the checks and balances are in place.

[00:18:17] And then we send them, you're absolutely right, into the schools and colleges

[00:18:19] the day before to allow everyone to have a look at it

[00:18:22] and make sure that everything is correct.

[00:18:25] That's when you always feel like the most important person in the school don't you Charlie?

[00:18:28] You've said before because literally all the senior leadership team are like hanging round.

[00:18:34] For the first time ever Michael this year I organised to be able to remote onto my desktop at work

[00:18:41] so that at midnight I could download the results and be ready for everybody in the morning.

[00:18:47] Because otherwise you've just got people standing over you while you're trying to download.

[00:18:52] Nothing worse than that, is there?

[00:18:54] It's just dreadful.

[00:18:56] It's like looking at someone trying to...

[00:18:57] Preserve a sleepless night.

[00:18:58] Yeah, I can imagine.

[00:19:00] And you've got all of the emotion as well, haven't you, from the teachers, the heads of department, the head,

[00:19:04] students as well when they come in sort of the following day as well.

[00:19:08] But our job then is really about being available.

[00:19:13] We've got people at the end of the phone,

[00:19:16] we've also got something we've introduced this year actually is live chat as well on our website

[00:19:20] which is an opportunity, if you've got a question you don't necessarily need to wait to go through

[00:19:25] to speak to somebody, hopefully we can answer your question doing that.

[00:19:30] But we take several thousand calls a day around results time.

[00:19:36] It's a large number.

[00:19:38] Ranging from the quite simple to answer through to perhaps a bit more complicated perhaps.

[00:19:44] I don't know, a result hasn't come through or there's a query about that result,

[00:19:49] perhaps there's an issue with it that we then investigate and find that out.

[00:19:55] Can I just, for sort of parents and any parents or students listening as well,

[00:20:00] is it right that the first port of call is your school as the exam centre

[00:20:07] who will then largely do the communication with the exam board?

[00:20:10] That's the route that works best, isn't it?

[00:20:13] Absolutely.

[00:20:14] It's making sure that when students are getting their results,

[00:20:19] they've got somebody they can talk to who understands it,

[00:20:22] who understands the process, who understands what's available next to them.

[00:20:27] Perhaps they didn't get the grades that they wanted or they expected,

[00:20:31] what's available to them, perhaps they haven't got what's needed to go on to university,

[00:20:35] there's someone there to support them.

[00:20:37] It's actually interesting, it's a topic that's been discussed for many years about

[00:20:41] could you send it straight to the student or could you do this, could you do that?

[00:20:46] My personal view is that the school or college or examination centre

[00:20:51] is best placed to provide that support when it's most needed.

[00:20:55] Yeah, definitely. I'm a big advocate of results days,

[00:21:00] so we've done quite a lot actually because obviously now we all have the technology

[00:21:04] where we could issue results to students digitally if we wanted to

[00:21:07] and say don't come to school and of course like over Covid and things like that.

[00:21:10] We were in that position.

[00:21:12] But I'm a big advocate of having in-person results days because

[00:21:16] like you say, people don't feel isolated, there's somebody to talk to,

[00:21:20] there's somebody to get advice from and also it's a nice community moment as well

[00:21:24] because everyone's been working at this thing for,

[00:21:27] well, a secondary school for five years if it's GCSE, seven years if it's A level

[00:21:33] so it feels right to come together at that point

[00:21:36] and to help people who are having a hard time but also celebrate.

[00:21:39] But also celebrate.

[00:21:40] Yeah, who have done well as well.

[00:21:43] Yeah, that's right.

[00:21:44] That's absolutely right.

[00:21:45] And write on your shirts and stuff.

[00:21:47] Do they still do that? Was that in the 90s?

[00:21:49] No, that's at the end of the...

[00:21:51] Oh, is that where they leave?

[00:21:52] That's not a result day.

[00:21:53] That's better in a couple of weeks' time.

[00:21:55] And all the eggs and flour and all that.

[00:21:57] Eggs and flour, that's it.

[00:22:06] So Michael, really interesting what you said about the scanning

[00:22:10] and that kind of innovation.

[00:22:12] Now obviously there's a lot of talk at the moment

[00:22:14] and a lot of action as well around the question of digitalisation.

[00:22:21] I've been observing this

[00:22:23] and I've noticed that AQA is taking what I would describe

[00:22:28] as a kind of forward-looking but careful approach.

[00:22:32] Would that be fair?

[00:22:34] Yeah, I think that's...

[00:22:35] Description?

[00:22:36] That's sort of evolution not revolution.

[00:22:39] Right, yeah.

[00:22:40] To sort of coin a phrase.

[00:22:43] The digital move to exams has been spoken about for decades,

[00:22:48] could I say?

[00:22:49] A really long period of time where people have spoken about it

[00:22:53] and can be moved towards it.

[00:22:54] I think now we're at a different point in time

[00:22:58] where we're really talking about when not if.

[00:23:01] I think there's a real desire to make this happen.

[00:23:05] It's the way students are learning increasingly.

[00:23:08] It makes sense for something on the scale of the system that we've got.

[00:23:13] It creates more opportunities.

[00:23:15] If you think about an exam in, let's say, geography,

[00:23:20] what you could do on screen in showing a volcano

[00:23:23] or tectonic plates or something like that

[00:23:25] and how engaging that might be in an examination.

[00:23:28] There's some real opportunities.

[00:23:30] But what we're not saying is

[00:23:34] it's not about everything moving digital

[00:23:36] and it's not about everything moving digital in a short period of time.

[00:23:39] We'll always have handwriting for a long period, many, many decades.

[00:23:43] I can't see that disappearing.

[00:23:45] Some exams perhaps might not lend themselves to having it on screen.

[00:23:50] Other exams really lend themselves to it.

[00:23:52] And we just need to look at that.

[00:23:54] We need to understand what the schools and colleges,

[00:23:56] how are they set up?

[00:23:57] How can they do that?

[00:23:59] What's the technology?

[00:24:01] How does that work?

[00:24:03] But I think it is very much a sort of when is it going to happen?

[00:24:07] And we've spoken about venturing into Italian and Polish, for example, last year.

[00:24:13] Yeah, I know what you're saying.

[00:24:15] Yeah, and I think language is one of those that lends itself to on screen exams.

[00:24:21] But we need to move together.

[00:24:23] We need to move with the regulator.

[00:24:24] We need to move together with government.

[00:24:26] So it's something we're progressing.

[00:24:29] But actually also, it's not just about the summative exams.

[00:24:32] It's not just about the GCSEs and the A-levels and the end exams.

[00:24:35] It's about the formative stuff as well.

[00:24:37] And how do we create the way that schools and colleges are teaching their learners?

[00:24:44] How can we use technology for that?

[00:24:46] So we've got a couple of things.

[00:24:47] We've got a new diagnostic tool.

[00:24:49] We've got for maths.

[00:24:52] And we're looking at other assessments that will be on screen.

[00:24:56] Because what you can do from there is you can capture so much data.

[00:24:59] Yeah.

[00:25:00] And data is very powerful.

[00:25:02] One phrase that's familiar to, I'm sure, exams officers everywhere would be the idea of the normal way of working.

[00:25:11] So the idea that assessments should reflect the students' normal way of working.

[00:25:16] When you're changing a system, is that quite a difficult thing to manage?

[00:25:21] Because obviously when you look at schools across the country,

[00:25:25] students might be at different stages in terms of, you know, some schools might have more technology, other schools less, that sort of thing.

[00:25:33] So being able to have a system that does cater to students' normal way of work, is that a particular challenge, do you think?

[00:25:40] It is a challenge and it's not one that can't be overcome.

[00:25:46] And I think that's where we need to work with government.

[00:25:48] We need to work with schools and colleges to look at what is in place.

[00:25:52] We can't have a system where we say, well, we're going digital and actually large parts of the sector can't deliver that for whatever reason.

[00:26:00] So we need to be able to do that.

[00:26:02] Interestingly as well, we need to make sure, whereas at the moment technology can be used for those with special needs or as their normal way of working,

[00:26:13] we may have a small number for whom paper continues to be the right way of doing it.

[00:26:20] But that would be the exception.

[00:26:22] What we're saying is, what you can't, we should say quite strongly and believe quite strongly,

[00:26:26] you can't have a system that's got, I don't know, 30% doing it on screen and 70% doing it on paper.

[00:26:32] Definitely. Yeah, yeah, yeah.

[00:26:34] And that's why I think the fact that the considered approach that you take into it is so important.

[00:26:39] Absolutely.

[00:26:40] I mean, if we're just giving it up with exam screen, the numbers of schools that you wouldn't expect who come back and say that they can't use it because they can't get a projector is,

[00:26:53] was surprising to me.

[00:26:56] I mean, and we're talking big numbers here.

[00:26:58] Yeah, who haven't been able to, for whatever reason, have a projector or they can't get a screen in their hall.

[00:27:06] Yeah, really quite things that even us working in schools for decades, really shocked by.

[00:27:12] So I think that the way that is considered looking at certain subjects that fit that I think is a really good way of moving forward.

[00:27:21] And fairness.

[00:27:22] Absolutely, because that's the most important thing with exams, isn't it?

[00:27:26] Yeah, we've got a really good group.

[00:27:28] We run a student advisory group where we have, it's about sort of a dozen students who, and they are proper advisory group.

[00:27:36] They advise our board, our board of trustees in front of them.

[00:27:40] Really, really key.

[00:27:41] And when we ask them the question about digital examinations, the first thing that they came back with and the strongest thing they came back with was fairness.

[00:27:49] How do you ensure that there is fairness across the system?

[00:27:53] And that's got to be the fundamental.

[00:27:55] Yeah, yeah, yeah.

[00:27:56] Absolutely.

[00:27:57] What else do you see in the kind of future of exams?

[00:28:01] Big question, but, you know, aside from digitalisation of exams, what else do you see as a kind of future?

[00:28:08] It's a really, it's a really good question.

[00:28:11] And with an election, you know, soon upon us and the changing government, and we're not, you know, you'd expect me to say we're not a political organisation in that respect.

[00:28:22] I do think there's an understanding about progression of what does that actually look like?

[00:28:30] Are we making sure that we are equipping all students with the skills and the knowledge that they actually need to progress?

[00:28:40] And perhaps, you know, we're looking at, for example, another qualification, which would be a digital one around maths, that is more about real life maths.

[00:28:51] It's about...

[00:28:52] Can I do that? I need that.

[00:28:54] And it's what we use on a day to day basis.

[00:28:57] You know, it's about, it would be about understanding percentages working out, you know, if there's 25% off in a sale or what's your APR on a loan or something like that.

[00:29:07] Sorry, what?

[00:29:10] So it's one of these people, Michael, who boasts about not being very good at maths in a way that nobody would ever boast about not being able to read.

[00:29:18] It's interesting, isn't it?

[00:29:20] Yeah, yeah. I always find that interesting.

[00:29:23] Could we talk to you about one more thing, Michael, before we let you go?

[00:29:35] Which is, I think is a subject that's become increasingly sort of pertinent relevant over recent years, is to do with security and exams.

[00:29:46] So, just leaving aside digital for one second and just talking about the current system.

[00:29:53] Could you talk to us about what some of the challenges for you as an exam board are around maintaining the security of the exam system and therefore its integrity?

[00:30:02] Yeah, sure. Absolutely. And there's various parts to that, various stages of the exam process where security is paramount. It's really, really important.

[00:30:14] And that starts, you know, obviously with the exam board and making sure that when we're writing questions, we're putting papers together, there's security around that.

[00:30:23] And actually the regulator of QAL is very clear about the processes and who can see papers, who can't see papers and what they can then do as part of their job.

[00:30:32] So that's quite tightly secured.

[00:30:34] The other part is the actual sending of papers into schools and colleges and making sure that we're tracking them, that they're processed, that any problems the exams officer contacts us as an exam board.

[00:30:48] And we can try and, you know, if there's any issues or any problems.

[00:30:51] And I know I've been to sort of quite a large number of schools and colleges and the variations but the high security they have for putting these papers.

[00:31:01] I mean, I've been to one place, I think it was an old bank even, like this big old vault, you know, that sort of smut open.

[00:31:07] We talked about that on an earlier episode didn't we? About the keys and the filing cabinets and the, yeah, I mean, it's a whole other world.

[00:31:17] I was interested about whether or not it would be possible to steal a metal filing cabinet, like a big metal cabinet.

[00:31:23] Bit suspicious maybe walking down the corridor with one of those on your back.

[00:31:28] But we do have and we have also, we need to make sure that the processes we have, we send out papers as well on the morning of exams as your exams officers know.

[00:31:38] We've had to put extra security in place to stop this from happening because a leaked paper causes phenomenal chaos and difficulty for schools.

[00:31:51] Well, particularly now with social media isn't it? I mean, it's just so much quicker that this sort of stuff gets out.

[00:31:56] It gets around and it means that we often have to then recall the paper, which means exams officers have to then sort of send those back and make sure that the new ones are sent out.

[00:32:05] We then have to monitor and make sure they've all come back in and they've all gone back out.

[00:32:09] So you haven't got too lots of papers being sent. It's a really big, really big thing.

[00:32:13] So by putting extra things in place, extra checks and balances, which can sometimes be a bit not always welcomed.

[00:32:21] It's a bit when you go to your bank isn't it?

[00:32:23] And you're sort of asked for all these questions, but you're pretty glad that it works and it means that no one's taking money from your account.

[00:32:31] So I think that's really important.

[00:32:33] So, Michael, just on that a bit because you touched on obviously that you have introduced a high level of security.

[00:32:42] What do the threats look like?

[00:32:45] What are we talking about in terms of particularly on the digital side of the downloadable papers without going into too much detail?

[00:32:55] I was going to say I feel like I'm going to get sort of like an MI5.

[00:32:59] I can't tell you that because that would be something people would have to go about.

[00:33:02] I'll have to kill you.

[00:33:04] In broad terms, what are the types of threats?

[00:33:10] It's about systems.

[00:33:12] It's about sort of parties getting into systems not necessarily to the source.

[00:33:19] They can find other ways around it.

[00:33:22] And in such a large system and where it's time critical, the exam starts at nine o'clock.

[00:33:29] How do we make sure that we're doing the verification process correctly and accurately so that can be done?

[00:33:37] So it's various ways and as is often the way isn't it with criminals and sub-criminals.

[00:33:43] They're sort of one step ahead.

[00:33:45] So how do we try to keep ahead of them?

[00:33:48] Yeah, sure.

[00:33:50] Touch wood, it's going well.

[00:33:55] As I say, a huge thanks to exams officers who, especially on the morning of exams, it must be a hugely stressful time for them.

[00:34:05] And to deal with things like that as well is we're very thankful for it.

[00:34:09] Yeah, I have noticed, I've been an exams officer for 12 years.

[00:34:14] If I go back 12 years ago and think about how tight the rules and regulations were about things,

[00:34:23] I think there's been a definite, particularly over the last five or six years, if you discount COVID maybe,

[00:34:30] but over the last five years, there's been a definite feeling of tightening.

[00:34:36] And I guess that's because some of the threats that have been posed didn't exist so much before and the technology is advancing.

[00:34:44] So it's funny because you work in a system where there's more pressure because of that, like you say, on the morning of the exam and stuff.

[00:34:52] But you also work in a system where you feel like, yeah, this is a robust system now.

[00:34:56] Whereas maybe before, not to say it wasn't, but it feels tighter now than it did in the past.

[00:35:03] It really is. And I think that's partly because exam boards have tightened up their processes and understand where perhaps the vulnerabilities were.

[00:35:16] We've also got a regulator who's very focused on that as well.

[00:35:21] The fairness point comes in quite strongly as well.

[00:35:24] I think that underpins a lot of the assessment system, and it's not fair if people have sighted the exam in advance.

[00:35:31] So I think there's, and I think actually as well, the currency of exams has perhaps become a bit more critical as well.

[00:35:40] Perhaps going into university and getting the grades is, people are needing to get those grades

[00:35:47] and therefore it becomes a currency for people to commit a criminal act because there's a payoff for it.

[00:35:57] The thing is though, the interesting thing is we get social media and things, we get cases of leaked papers every year.

[00:36:05] Nearly always they are fake. Some of them are quite good. Some are quite good.

[00:36:12] Some are absolutely appalling. And I think that's the thing is they're going to be fake if you see one really.

[00:36:20] We were talking about this on a past episode and it's really horrible really when you think about it, isn't it?

[00:36:27] Because the poor kids as well who are stressed out their minds, kind of want to do their best.

[00:36:34] You can see how someone could get suckered into something like that. It's so deeply cynical, isn't it?

[00:36:40] It's targeting people who are at that moment in a vulnerable state because they're revising and they're under that stress.

[00:36:47] Thanks so much. I learnt loads there.

[00:36:51] Yeah, that was brilliant. Thank you Michael.

[00:36:53] Like I said, we'll bring you up to Milton Keynes.

[00:36:55] I was going to ask you that at the end so I'm really glad you brought it up because it sounded a bit desperate if I'd been like,

[00:37:01] sorry Michael that was nice but remember Milton Keynes, you're not prepared.

[00:37:05] We'll do that. I'm well up for it.

[00:37:07] Do a podcast from there.

[00:37:09] That would be brilliant. Live podcast. I see that postcode on those yellow labels every day.

[00:37:15] So I do wonder like what goes on, what's going on in Milton Keynes.

[00:37:19] Yeah, it's the place, it's where it's all happening.

[00:37:21] It's been so nice to meet you.

[00:37:23] It's been nice talking to you both.

[00:37:25] Yeah, hear about everything and your busy summer ahead.

[00:37:29] Well good luck with your podcast as well, it's great.

[00:37:31] Thanks.

[00:37:32] I've listened to a few of the past episodes.

[00:37:34] Oh good, I love that.

[00:37:38] Do you know what I wasn't really expecting from that interview?

[00:37:47] What?

[00:37:48] How much experience Michael has in the area of exams over such a long time?

[00:37:52] I mean so much, he really knows his stuff.

[00:37:56] I think we could use him as a bit of a consultant, couldn't we just send him loads of messages?

[00:38:01] Yeah, he could be like our fact checker, couldn't they? Stop us spreading misinformation.

[00:38:05] He's going to love that.

[00:38:07] Yeah and it was so fascinating to hear obviously about the enormous scope and scale of what the awarding bodies do.

[00:38:14] Definitely.

[00:38:16] It just shows you doesn't it that wherever you are in this system of delivering exams,

[00:38:23] the job is massive and it's stressful and very pressurised.

[00:38:28] And very important, everyone's doing, playing an incredibly important role.

[00:38:33] Exactly, yeah it was really really enjoyable to find out what they do

[00:38:38] and obviously can't wait to get to Milton Keynes.

[00:38:45] Thank you so much for listening to the Exam Man podcast, we really really appreciate your support.

[00:38:49] Remember that you can access it on all the major podcast platforms.

[00:38:53] Give us a rating, give us a follow and we will catch you next time.