Ms Elaine Sheridan:

We have referred previously to reviewing candidate responses. Basically, we prepare the examination paper and in doing so, we keep a number of things in mind. Obviously, the specification is significant. That is why we are coming back to this. As was mentioned earlier, it can lead to a frustrating conversation in terms of the learning outcomes. We must make sure that what we ask in the exam is in the specification. We also develop a draft marking scheme. When we are preparing a question, we have an eye all to the time to what we expect the candidates to write and what the weighting is. We look - perhaps in the winter - at what we expect the students to present on the day. That is just a draft outline and not a final marking scheme. The next key step is when the examination has been sat and the material comes back in. We then spend a number of days with our senior advising team sampling candidate responses across the different ranges. This means taking out candidate responses and reviewing and analysing them in the context of the kinds of responses students gave, what the standard was, what kind of material was coming up and what kinds of mistakes they were making.

It is then brought to a pre-conference where the senior team sits down to work through and review as much material as possible. The team has the initial draft outline marking scheme, which then gets refined. It is then brought to the main marking conference where the bigger and full examining team has a voice and feeds into the final version of the marking scheme that will be applied to the sample 20 scripts. All our examiners then mark a random sample of their scripts. That has been hugely helpful since we moved online; it expedites the process. A random sample of every examiner's scripts is marked and we review the outcomes. We review the outcomes generally, in terms of what the overall grade profile is. We also look at the average mark by question to see if there is any particular question that is not performing as we would have expected. We establish if the question is flat-lining or if are we seeing, after looking at the particular sample, that the question is not scoring well. In such an instance, we revisit the question to consider if there was a flaw or ambiguity in it that misled the students and meant they did not present as they should. If so, we might change the marks and tweak the marking to the benefit of the candidate. We do not set a standard. We do not decide a set of outcomes without reviewing candidate responses. The point is that when people ask for more papers and more marking schemes, we do not finalise a marking scheme until we have an opportunity to review what the candidates are presenting.

At the end of the day, we want to be able to reward them for whatever they have presented. Particularly in the early days of an examination when it is new to everybody, it is extremely important we review and make huge efforts in terms of the new exams. This was particularly the case in the summer just gone. Ms Feeney referred to the big bang in 2022 when we saw a full suite of new exams across a range of new junior cycle subjects. We were very conscious that they were the first exams for a full cohort in many subjects. We reviewed a lot of material. We do not turn around to a particular examiner and say they are going to have a certain number of As, Bs, Cs, distinctions or merits. We set the standard at the sample 20-script stage. At that point, when we review the outcomes on the whole in terms of the distribution of grades, or in a particular question or even part of a question, if we see something that looks like it is malfunctioning, we address it. We continue to monitor the outcomes as we mark on through. If something becomes apparent, we revisit the marking scheme but after the sample 20, it is unusual that we would do so. One of the benefits of having a no-choice examination is that we are preparing a marking scheme for a question that everybody has answered. There are other benefits, beyond even those for the candidate in terms of having less reading in the examination.

That is how we work through our standard-setting process. It is very much candidate-centred. We do not have a predetermined set of outcomes heading into it. We have a standard and we know what the expectation would be in terms of what the specification was setting out. The NCCA has published exemplars of standards for teachers to help them in marking the classroom-based assessments. The Chair asked earlier how the teachers know how to proceed when they are marking the classroom-based assessments. There is material published on the NCCA website to assist them in that. We have worked as critical friends with the NCCA in the development of that material, and also with teachers because obviously they are the key people. While I do not want to go on too much about standard-setting, I will mention that our examiners are teachers. That is a really key thing. The committee is aware that we are looking and recruiting at the moment. We keep saying we need teachers because at the end of the day, they are the people who are in the classroom and who know what these junior cycle students are experiencing and the challenges they are having with these new specifications. All of those pieces feed into our standard-setting process.