Highlights from The Exam Man podcast- Season 1, episode 16
After the exams had finished in the summer we wanted to know more about what happens to the exam papers once pens are put down. We were invited by AQA to visit their vast processing centre in Milton Keynes, where we had a tour and also did a live podcast interview with Mark Collins, Head of Operations at AQA.
We're really interested in what the transferable skills are between your previous industries and working in education. It must be quite a different beast in some ways to other operations roles?
It is very much a different beast. So I've been in operations for most of my adult life, with 12 years as part of the AQA group. The biggest challenge that we have is the product that we deal with. I worked for Amazon for a while, I've worked for Tesco, I've worked for Iceland, backfilling stores, that type of thing, so I knew all about having to move boxes in, through and out. And you've got deadlines to hit, and transport offices to keep happy, and a number of issues that you face on a day to day basis.
The difference between what we do here and what I've done before is the product that we deal with. We're dealing here with exam papers and those exam papers represent the journey of a student that may have taken two, three years to get to that point. And we have to be so careful that we treat them with the respect they deserve. We never let a learner down and we have to account for every single page. It takes some time to get your head around the difference and the scale of what we do. But I would say that's probably the biggest difference between my past and my present.
You mentioned to us a couple of times on the way round, that what you're really optimising for is quality rather than efficiency. And obviously, you've got deadlines to hit. Maybe in other industries, is there more of a focus on efficiency?
In my experience there is, and I think the difference here is for a number of reasons. One is that we're in the exams industry and that's very different. I think the other part is that being a charity, as AQA is, we reinvest the money that we make into education, both the physical process of education and research for education. So we're able to really look at how can we best serve our customers and ultimately the learners in the education system. It's a slightly different mindset, but certainly we have more of a focus on quality than where I've been before. The first thing that we do when the deliveries arrive is sub-sort, and we're always sub-sorting it to component level. So for its onward journey, for the life of those scripts that are with us, they're all together in relevant components. So when we ultimately scan stuff, we'll do multiple centres, but the same component at the same time.
At any given point in the series, we'll get between five and six deliveries a day, which could be up to 50,000 packets of scripts. On average in every packet, there's 21 scripts. That will vary between centres, but that's the average that we get. At any given point in the peak of the series, we will have a work in progress of around 3 million scripts. So the first thing we're doing then is sorting to component level. We have a team of people, flow managers, that will determine what it is we need to process and in which order. We have our internal SLAs, but ultimately we're looking to keep the marking teams as busy as possible and have the right level of work available for them. So they'll go through and they'll select the next shift’s work that they've got to process. That's a rolling programme. So night shift will look after day shift and vice versa.
You were saying earlier, if the markers in a particular subject are getting low, then you need to make sure that they've got more stuff to mark. So you might prioritise that component?
Correct. So, although we have a plan that we work to, there are times where we'll coordinate with our colleagues in the marketing department. They may say there's a particular component where they're running a bit short. Could we push them to the front of the queue? And we do that in a number of ways. The first thing we do is investigate the system to see where that product is. And it may be that there's some sitting waiting at scanning, and it will naturally flow through in the next couple of hours anyway. But it may be that it's further back, and we will push some of that to the front, because it's all about that collaboration.
Before we scan, we need to determine the different journeys for the different types of product that we get inside the envelopes. That's done in an area called logging. We are logging the receipt of that envelope onto our system and we are separating the different types of material that are in it. And so the main types of material you'll get are batches of scripts that can essentially just be scanned. You've got scripts with additional materials. You've got additional materials on their own. You've got attendance registers which need scanning and filing. And then you'll have what we call white mail. So that's non-candidate specific stuff like seating plans that may have been put in and included. The team of people are sub-sorting all of that and logging on the system at the relevant point.
Now, one of the key points that they do is they use the data on the outside of the envelope to log the number of scripts inside the envelope. And that's really important for reconciliation. We count the number of scripts in that envelope at the point we log it. And then we count them again when it's scanned or processed in another way.
So what is really helpful for schools and centres to do with their envelopes?
On the front of the envelope, there is a grid that asks for multiple pieces of information. We're currently redesigning the envelope, and we'll only be asking for the number of scripts, because some of the information currently is duplicated on the label. The piece of the puzzle that was missing, that we'd really like all centres to help us with, is the number of scripts contained in that envelope.
I haven't been doing that. I'll give myself a slap on the wrist. I will, I promise from now on. I now understand the value of doing that....
Thank you! So once everything's logged and we understand where it's going to go and how it's going to be processed, it moves through its journey and at all points through the journey, we're tracking it either in cages of work or trolleys of work. Attendance registers will go off on their own journey and they'll be scanned and then uploaded into our system so that we can deal with them later on. But the main bulk of the scripts go on to what we call the guillotine. That's essentially some machinery we've got that takes the spine from the booklets and they convert exam booklets into sheets of paper because we need them to be individual sheets so that we can scan them.
Each of those guillotines can cut around 11,000 scripts an hour and we've got three of them, so it's 33,000 scripts an hour. On average, a script will have 13.1 sheets in it. So we generate enough work for us to scan between 3.5 and 4 million sheets of paper in a 24-hour period. We do that consistently over the summer, which means ultimately we'll scan about 146 million sheets of paper for an average summer series at the moment. For summer 2024, we have about a thousand people, the vast majority of which are temporary workers in the building.
We work six days a week, Monday to Saturday, and we run three shifts depending on which function you're fulfilling. We have day shift and night shift, which are the main processes, right from the back door through to the scanning. And then we have day shift and twilight shift for the more electronic processes that we saw upstairs. We tend not to do electronic processes at night, because we know that quality is a challenge for that type of thing when people are tired at 2, 3 am. It varies slightly depending on exam dates, when they exactly start and finish. But about six to eight weeks is probably about the right period of time.
What are your main challenges apart from getting the number of employees that you need to do the work?
I think there are a number of challenges that come with scale. When you've got that number of people in a building at the same time, keeping them all busy and focused is always going to be a challenge. And I am really lucky that the team that surround me, that work here permanently, have over the years become experts at that. They really know how to manage and motivate these teams of people.
Could you talk a bit about how you prepare people to do these roles?
You were saying that you don't necessarily expect people to be productive immediately, that you have a process of getting them to that point. That will vary depending on the exact task that they're asked to fulfil. But in broad terms, we will bring supervisors in first and early, and train them to the relevant level. Quite often, they're the returners. So they may have been an operator last year, and they've come back as a supervisor. They've already got that basic understanding, which is great for us. On day one for an operator, we wouldn't really assume they're productive at all, we’re doing the induction.
We instill in them on that first day the importance of what they're being asked to do. So, everybody's asked to look upon these exam scripts as if they've done them, a family member’s done them, a friend has done them. These are irreplaceable documents. They're also told on day one that quality is the absolute focus of what we do. We want them to slow down, take their time. We don't expect people to be fully productive until about day eight. We will tell people we would much rather they stopped, put their hand up if they weren't sure, wait for their supervisor. Nobody is going to get in trouble for asking the right questions.
Do you find that you need to refresh people as well because there could be quite long gaps? And changes?
So we are really conscious that people come back and they remember last year's processes. And whilst we don't necessarily fundamentally change the process, we do drive quality and efficiency through a continuous improvement programme. As you saw downstairs, we look at absolutely everything. Even to the point that the team had highlighted a challenge with the way the papers drop into the guillotine. And there's a small hole for treasury tags in the top of the paper which by chance lined up with the way we need to add compressed air to process it. By moving that, we got a 27% increase in productivity, which is just huge.
We bring people back in and the reason we start slowly is because everybody, regardless of their history with us, is retrained and refreshed. We make sure that they're all following the processes as closely as they can.
That's really empowering for the workers, isn't it, as well, to know that if they can see a way of making things a little bit better, that they'll be listened to and things taken on board?
I think it's a really valuable point. And the whole lessons learned post-summer review, constant continuous improvement reviews that we do, it's really important to us here in Milton Keynes, but it's also important across the whole business. This isn't just a Milton Keynes thing. This is an AQA thing. We talked about the impact on exams officers, and I was lucky to spend some time yesterday with Gemma Miller, our new Exams Officer engagement manager, at a centre in Leicestershire. We want to understand how our processes work side by side with Exams Officers and what we can do to improve. It's really worthwhile listening to what others have to say in that regard.
As an Exams Officer, I was thinking what a valuable experience it is coming here, I think a lot of Exams Officers are running their own mini logistics operation. That's kind of what they do. To come here and see such a large logistical operation, there are things that obviously we can take from each other.
And logistics really is about refining processes. Yesterday's conversation in Leicestershire primarily was on logistics, and the themes were time and space, essentially they were the two big ones. How do I get candidates in and out of the right rooms, and how do I spread my invigilators far enough, and how far in advance should I be getting papers from the awarding organisations versus where am I going to store them? That's why we've recently recruited Gemma. She's got a team of regional engagement officers that are going to spend much more time out in the centres and liaising with Exams Officers. They should provide support, guidance, training, those types of things, but also be the voice of the customer for AQA. What's really interesting about that team is they've got a wealth of knowledge because all of them were previously Exams Officers, so they know some of that pain already, and they're already starting to make some changes. The company will share more information about what Gemma's team will do with schools in the new academic year. We're all part of the same process and the same system. Rather than feeling like you've got the schools over here and the exam boards over here. We've all got the same aim.
One thing that I'm particularly impressed by here is the security element, the security of those exam papers, but also in terms of the building.
Let’s start with security, you're both wearing badges. You'll see that across the site, you can't get through doors unless you've got the right access. So we are really careful with security. Everybody that works here permanently has ID badges that do the same.
You showed us the fire safety, flood defences, etc here, which are just astonishing. I think it's so important for Exams Officers, schools, and students in exams to know the care that is taken over each piece of paper that they write anything on. So much money is invested back in to make sure that they are secure. Could you just talk us through the resilience that you've built within the building to ensure that you keep the exam papers as safe as possible?
We’re really conscious around fire, flood, anything that will affect them. As with all businesses, the people within this building are our primary concern. So safety first, absolutely. But we can't just go and reorder exam papers. We’ve put in a fire suppression system, which works by adding a naturally occurring gas -which doesn't harm humans, paper, equipment, or the environment - into a particular zone or space, should there be a fire. And all it does is reduce the oxygen level from about 15% to about 11%. At that point, combustion can't happen, so it would put any fire out, but humans can still breathe. So we've got that and we know it works as it's been fully tried and tested. We've also got two completely separate server rooms in this building, two completely separate internet lines coming into the building. We've got a generator. So at the front of the building, there's a box which is a plug-and-play really for a huge generator which we have on contract. So within four hours of us losing power, we would be back up and running because we can't move deadlines. We've got to get this work done.
When trying to design this building, we tried to look for every possible place where we would have a problem. We’re on the first floor at the moment where we've got kitchens, toilet facilities, those types of things. Underneath there are flood detection points. If there is a leak from a sink, from a toilet, then the flood detection system will stop all the water on the first floor, so that it won't flood the ground floor where all the scripts are. And so we've gone to that level to make sure that we never let anybody down.
What are some of the biggest sort of anomalies that you'll experience here, and how do you deal with some of those anomalies?
We are proud to be the biggest awarding organisation in the UK, and we've talked about the millions and millions of scripts that we deal with. But we do have anomalies in things like the completion of the name and centre date and candidate data on the front of the scripts. So we have an automated process that will match that script to the official enrolment data that we upload into our system that's come from the entries data of the centres. If the candidates don't fill them in properly, we have to manually intervene in that. And that can be for a number of things - it can be preferred name, not legal name. It can be candidate number written incorrectly, or really poor handwriting. There's a number of reasons that would cause that. So help and support in getting that done correctly would make a huge difference here. Word process scripts are treated, not as an anomaly, but they have their own particular route through here.
They take up, I guess, compared to a normal exam paper, a disproportionate amount of time. And as you showed us, the exam papers that might have been vomited on during the process. Hopefully you don't see too many of those!
We do get some odd things occasionally. You'll sometimes get damaged scripts come through. You'll sometimes get contaminated scripts come through. You'll occasionally get odd things come in an envelope. The oddest thing I ever saw in an envelope, and this is probably going back five or six years, was a £10 note. Absolutely no reason. We have a process, so the centre was contacted.
But you'll be surprised, it's those little odd things that really slow the process down. On attendance registers, when they're returned, they're two-part. And the idea of being two-part is that one part is retained by the centre and one part is returned. But if the centre returns both parts, we have to have people separate them.
I've definitely done that quite a few times!
You're just working quickly. Well, now you know the impact.
Mark, I just want to say a huge thank you for showing us around today. As someone who can't keep their kids' socks together, I feel quite staggered by the scale and just how impressive it is! The attention to detail and everything that goes into an operation of this size is incredibly impressive, so thank you for that.
That's all right. You're welcome!
To listen to and read more interviews like this, subscribe to The Exam Man - the #1 podcast about exams and assessment
To find out more about Examscreen please contact us: Examscreen