Colette Kelleher (Independent)
Link to this: Individually | In context | Oireachtas source

The next item is engagement with officials from the State Examinations Commission, SEC, and the Department of Education and Skills.

Before we begin, I remind members, witnesses and persons in the Public Gallery to turn off their mobile phones. I welcome from the State Examinations Commission Mr. Aidan Farrell, chief executive officer, Dr. Tim Desmond, acting head of the examinations division and Ms Andrea Feeney, director of operations and IT. I welcome from the Department of Education and Skills Mr. Gary Ó Donnchadha, assistant secretary, and Mr. Eamonn Moran, principal officer. I thank them all for their attendance. They have been invited here today to discuss petition number P0013/2017 from Ms Tara O'Sullivan, entitled "Make the new English junior certificate exam fairer by adding 30 minutes". In her petition Ms O'Sullivan requests that the Department consider the possibility of extending the length of time for the junior certificate English paper by 30 minutes to allow the students enough time to complete the paper.

Before we commence, in accordance with procedure I am required to read the following: by virtue of section 17(2)(l) of the Defamation Act 2009, witnesses are protected by absolute privilege in respect of the evidence they are to give to this committee. However, if they are directed by the committee to cease giving evidence on a particular matter and they continue to do so, they are entitled thereafter only to qualified privilege in respect of their evidence. They are directed that only evidence connected with the subject matter of these proceedings is to be given and are asked to respect the parliamentary practice to the effect that, where possible, they should not criticise or make charges against any person, persons or entity by name or in such a way as to make him, her or it identifiable. Members are reminded of the long-standing parliamentary practice to the effect that members should not comment on, criticise or make charges against either a person outside of the Houses or an official either by name or in such a way as to make him or her identifiable. I remind our guests that their presentations should be no more than five to ten minutes in duration. The presentations submitted have been circulated to members in advance of the meeting.

Mr. Aidan Farrell:

The State Examinations Commission, SEC, and our colleagues in the Department of Education and Skills very much welcome the opportunity to attend today’s committee meeting to assist the committee in fully considering the petition to make the new English junior certificate exam fairer by adding 30 extra minutes. I note that the committee would particularly like to discuss how the mock examination papers provided by commercial suppliers might be scrutinised in advance of their sale to schools. Before dealing with the issue at hand I might briefly set out to the committee the role and functions of the SEC within the State examinations system.

The SEC is a statutory body established in 2003 under section 54 of the Education Act 1998. Our role is to provide a high-quality State examinations and assessment system incorporating the highest standards of openness, fairness and accountability in operating the Irish State examinations, principally the junior and leaving certificate and the leaving certificate applied examinations. Examinations and assessment policy is set by the Minister for Education and Skills and the SEC is then responsible for the operation and delivery of the State examinations system.

The commission is headed by a board of commissioners, appointed by Government on the nomination of the Minister for Education and Skills. The role of the commissioners is to act as the custodians of the State examinations in terms of affirming and upholding the principles and integrity underpinning the system and the need to maintain public confidence in the examinations themselves. The functions of the SEC are precisely defined in the State Examinations Commission (Establishment) Order 2003. These encompass the preparation of State examination materials to their marking, resulting and ultimately appeal processes, along with arrangements for the conduct of the examinations. The SEC prepares over 500 test items each year, which are taken by almost 120,000 candidates, leading to the marking of 1.9 million items of candidate work and the award of 1 million grades.

The purpose of the State examinations is to measure the extent to which candidates demonstrate achievement in terms of meeting the aims and objectives of each particular subject specification, or syllabus as they are often known.

Examination papers do not exist in a vacuum. Central to the SEC’s role in maintaining public confidence in the State examinations is the setting and maintenance of examination standards from year to year. We do this through the relationship between the preparation of examination papers and their marking schemes, how we standardise these in order to maintain standards year on year and then how we quality assure these through our marking and appeal processes. The examination paper preparation process itself involves the key personnel of chief examiners, drafters, setters, translators, and in the case of the leaving certificate, university observers and scrutineers. The process takes over 15 months to conclude and contains a number of overlapping steps to provide examination papers of the highest possible standard to test, and to give candidates the opportunity to answer on, the appropriate aspects of the subject specification, or syllabus.

The SEC provides a range of supports and services to teachers and candidates in preparing for the examinations. These are an important element of the work of the SEC as candidates are understandably apprehensive in the lead-up to the State examinations, particularly when a new programme is being introduced, as happened this year with the introduction of a new style of examination in junior cycle English. To provide appropriate support for future candidates and their teachers, the SEC provides full, open, and free access to its entire electronic archive of past examination materials. This includes past examination papers, past marking schemes and chief examiners’ reports on previous examinations. Such access to past papers and marking schemes allows all stakeholders to have equal and fair access to information about what to expect as regards the structure, format, and style of the examinations, so as to be appropriately prepared. Chief examiners’ reports are issued periodically in each subject and provide additional information and insights into how candidates have performed with respect to the various assessment objectives of the syllabus, as well as offering advice to future candidates.

When a new subject specification or syllabus is introduced, such as the new junior cycle English examination which candidates sat for the first time earlier this month, past examination papers in that subject are of more limited use for the purposes described above.

For this reason, sample examination papers are produced by the SEC, in consultation with the Department of Education and Skills and the National Council for Curriculum and Assessment, in advance of the first State examinations for any new subject or revised subject specification. These sample papers are produced as a guide to students, teachers and the public as to the structure and format of the new examination and, like real examinations, are produced with a high level of oversight and quality assurance to ensure their fitness for purpose.

Mock examinations take place in many schools in spring each year. These examinations, as the name suggests, are intended to mimic the forthcoming State examinations in June. Despite the fact they have no official status and their use is not approved or endorsed by the Department of Education and Skills or any other State body, they have been used by schools for some time as a means to help prepare students for State examinations. The teachers and schools that choose to use them believe their value rests in providing students with an opportunity to sit a full-length examination that attempts to replicate the demands of the State examinations when they come around and thereby enhance candidate preparedness. While in the past many teachers prepared their own papers for this purpose, commercially available alternatives have existed for a long time and have become increasingly prevalent. Decisions to hold mock examinations are taken by schools themselves. The SEC and Department play no role in this activity.

Mock examinations are not the business of the SEC. We have no role or involvement in the preparation or delivery of examinations provided by mock examination providers that are private commercial entities. The companies involved go to lengths to make their products look similar to the State examinations. They mimic the layout and fonts of the State examinations, including in some instances logos and subject codes akin to those used by the SEC. These companies usually provide solutions and marking schemes. In some instances they also provide, on a commercial basis, a marking service and return marked scripts and results to schools that engage them to provide this additional service.

Due to the efforts made by such providers to mimic the look and feel of the SEC's examination papers, many students and parents may incorrectly believe that these mock examinations have some official status or are subject to some form of State endorsement or oversight, but this is not the case. The SEC does not engage with the mock examination providers or the public in respect of mock examinations. As with any other commercially produced educational service or product, such as school textbooks, both the SEC and Department consider that it is inappropriate for a State agency to comment on the quality of either the papers or marking service provided. Indeed, to do so would certainly constitute inappropriate interference by a State agency in a private commercial market.

The proposal that the SEC would provide commentary or advice on mock papers to the providers of those papers or any degree of oversight on the papers or the process of their preparation is a cause for concern. The reality is that mock examinations are produced by private companies within a competitive commercial market. In our view, it would have a number of inevitable inappropriate implications. It would imply an endorsement by the State of the very process of conducting mock examinations in schools, enhancing their perceived value despite the fact the Department does not endorse their use or consider them to be an appropriate and effective use of school time. It would effectively provide endorsement or criticism of the products and services of private commercial companies. The legislative and administrative architecture that would be required to ensure such oversight and-or involvement with a commercial market by a State body was in full compliance with Irish and EU competition law would be onerous in the extreme. The resources required to provide an adequate level of oversight to allow any credible endorsement by a State body of the quality of these commercial products and services, including the large numbers of additional staff required, would not be tenable, particularly in pursuit of a goal that is at variance with stated educational policy. The SEC, in giving its observations or guidance, would as a result inevitably provide insights into examination paper development in such a way as to increase the predictability of examinations and-or provide inappropriate guidance as to future live examination paper content. The SEC's endorsement would result in inappropriate expectations as to the relationship between material that appears on mock papers and material that might appear on the real State examinations provided by the SEC. It would expose the SEC to legitimate avenues of criticism if, by sheer coincidence, the content of a mock paper from one company turned out to be a closer match than the content of a paper from another company as to the content that later appeared on the real examination, both mock papers having received the endorsement of the SEC.

In short, unless mock examinations could be subjected to the same level of preparative rigour as the real State examinations, with suitable oversight mechanisms placed on a sound legislative footing, it would not be credible for the SEC to provide an endorsement of these products. Even if the substantial resources required to implement this oversight role were made available, the negative consequences of carrying out this role would substantially outweigh conceivable positive effects. Consequently, the SEC considers that it would be inappropriate for us to have any involvement in the setting or oversight of mock examination papers.

Today's meeting came about as a result of an online petition organised by junior cycle candidates arising from their concerns based on the adverse experience of some candidates who sat mock examinations last spring in advance of the new 2017 junior cycle English examinations. The petition sought the addition of an extra 30 minutes for the completion of these examinations as many candidates had experienced real-time pressures in the mock examination. Candidates, who face the first examination in this new programme this year, were understandably concerned for their prospects in the real English examination in June based on this experience in their mock examination. Following receipt of the online petition last March, the SEC moved to reassure candidates that the examination papers in preparation for the June 2017 sitting of junior cycle English would be suitable for completion in a two-hour examination and that they would be fair to candidates.

The English examination sat by candidates earlier this month was fully in accordance with the structure, format and likely time demands of the sample papers at higher and ordinary level set by the SEC and issued in November 2015 for the assistance of teachers, candidates and, indeed, for the guidance of the producers of the mock examinations. I am pleased to report to the committee that the reports from teachers and candidates on the 2017 junior cycle English examinations set by the SEC show that they were executed without any issues arising and no difficulties were experienced by candidates in respect of completion time.

The SEC and the Department do not believe it is appropriate for the State to make any intervention aimed at improving the quality of commercial products and services in a market, especially since those products and services do not advance the Government's educational policy. The reality is that commercial producers have identified an opportunity in a market, which is their right. The SEC and the Department cannot get involved in endorsing any such commercial opportunities or activities.

My colleagues and I would welcome observations and will answer any questions that members wish to raise.

Colette Kelleher (Independent)
Link to this: Individually | In context | Oireachtas source

I thank Mr. Farrell for his presentation, overview of the role of his organisation and response to the matter at hand. I ask the officials from the Department of Education and Skills to comment on what Mr. Farrell has said and then I will ask members to put some questions about the petition to the Department officials.

Mr. Gary Ó Donnchadha:

I have a few remarks and I will provide the secretariat with a copy of my remarks after the meeting. A key innovation in the junior cycle reform process was the provision for junior cycle examinations to be marked by State Examinations Commission and for new classroom-based assessments to be conducted by classroom teachers. We consider that this dual approach will deliver the very best context to ensure a learner experience appropriate to the needs of the 21st century. Teachers will support student learning through conducting and grading classroom-based assessments. This is a major element in the assessment arrangements for the junior cycle. This kind of classroom assessment complements the end of cycle examination. It is very real and immediate for the learner because the teacher scaffolds and gives constructive feedback through the journey that is the three years of junior cycle. Importantly, the focus on the final examination is significantly reduced, which was a key objective of the reforms. The dual assessment approach is being developed to align better with the learning goals of a modern curriculum programme. The new junior cycle profile of achievement is the record that the learner will receive at the end of the junior cycle. The record will give students, parents and guardians a much broader picture of the young person's learning experiences and achievements throughout the junior cycle. Parents and students asked for a rich picture of engagement, involvement and celebration of the learning.

We recognise that embedding formative classroom assessment represents a significant culture change for teachers, for students and for parents. Thus, the junior cycle for teacher support teams will be engaging with schools during the next five years of implementation.

The rebalancing of assessment modes to introduce classroom-based assessments and complement final summative examinations changes the conversation fundamentally. The call to directly supervise external commercial providers is misplaced in a context where our junior cycle strategy and focus are moving away from reliance on a single external examination and promoting a wider range of learning opportunities for all. What we can do is to positively draw the attention of teachers to a wide range of materials on the NCCA, SEC and junior cycle for teachers support service websites that will assist in developing appropriate, valid and reliable assessment instruments and approaches. We can commit to providing this additional advice, and links to materials, on our websites in response to the very matters raised through this committee.

Part of this issue relates to the strong commitment of Government and the Department to reduce the reliance on the junior cycle summative examination. This is reflected in the framework, which reduced the time to two hours but balanced it with the time allocated to ongoing classroom-based assessment modes throughout the three years of the junior cycle. Adding time to an examination would be, in our view, the wrong thing to do, bearing in mind the many new ways teachers and schools have of capturing and celebrating the learning in junior cycle. It may be that the provider of the mock examination can already see that the test design was, on balance, demanding too much of students on this occasion. The solution may be for providers to ensure that the right kind of concepts and skills are assessed and rewarded while checking, through standard quality control procedures, that the assessment can be completed within a reasonable time. I have already said what we propose to do, which is to provide a mix of materials on our websites. Our commitment to package the currently available resources in one place should be a significant help to all, especially the learners.

Shane Cassells (Meath West, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

I welcome Mr. Farrell and officials from the Department. It is 24 years since I sat a mock English junior certificate paper but I have an appreciation of that period in my life and the importance these examinations play. This is especially the case in the junior certificate cycle because it is one's first big examination in secondary school. English is the big opener in the process and it is a huge step for the young people who take it. Today's meeting has arisen from the real-time pressure of a mock paper, as opposed to the State papers. I pay tribute to the petitioner, Ms Tara O'Sullivan, for showing the experience and foresight for this at such a young age. She has come to the Houses of the Oireachtas to seek positive change in the State examinations and there has been a reaction from several thousands of people, not to mention considerable media coverage. We talk about positive engagement and this is a clear demonstration of that.

The responses seem to be that the mock papers are not what we are concerned with. They seem to say that we are only here to deal with the State side but that is a disengagement from the reality, which is that the mocks play a vital role in the preparation for the State exams. We cannot dissociate ourselves from that reality as they are part of our educational system. I accept the points made by Mr. Farrell and Mr. Ó Donnchadha that there needs to be oversight. In the absence of engagement, however, we leave ourselves open to the scenario we see today. Mr. Ó Donnchadha spoke about positively drawing attention to guidelines but that is to say it is up to individual schools to decide if they want to positively engage. I feel we need a cohesive approach among all secondary schools. How does the Department believe we can achieve that if we just ask people to draw attention to guidelines?

The fact that Ms O'Sullivan drew attention to this allowed the State Examinations Commission to look at the addition of time but how did the commission reassure candidates that the examination papers would be suitable for completion in a two-hour period? Reference was made to everybody being happy. How did the witnesses from the Department draw those conclusions? When one walks out of an examination hall one is happy that it is over, whether one has completed all the questions or not. What sample did the witnesses get of people across the country being happy? English is a different paper from all others. In economics, accountancy or maths there is a set time to complete questions but in English, for which I had a huge grá, one tries to elicit creativity from students. The witnesses said adding time was going in the wrong direction but, of all papers, English is the one in respect of which we should allow creativity to flourish and restricting time is not the way to go. I acknowledge that continuous assessment is positive in this regard.

Can Mr. Farrell answer my question about reassurance? How was the assessment done which found that people were happy?

Colette Kelleher (Independent)
Link to this: Individually | In context | Oireachtas source

I invite the State Examinations Commission and the Department to respond. Can Mr. Ó Donnchadha leave us with a copy of his statement?

Mr. Gary Ó Donnchadha:

Yes, I will. I agree wholeheartedly on points made about the confidence and experience of the young person who has come forward to engage with the democratic process, which is music to the ears of the Department in the context of the new politics and society programme it is running.

This is not a disengagement from the mocks. We are trying to find an appropriate balance among the free right of providers to exist and to provide all manner of supports to schools in the way of resources and materials, the solemn responsibilities of the State Examinations Commission and the care the Department needs to show in responding to needs.

The issue of the change in junior cycle is fundamental here. We have brought the assessment and feedback conversation right back into the three years of the junior cycle. In the future it will not be a case of waiting for the big bang at the end of the three year cycle. I accept that it is an important experience and that going into an exam hall is a daunting experience, but there will be a huge amount of conversation. Unlike when the Deputy and I did our junior certificate examinations, there will be ongoing conversation and dialogue with the teacher as well as these structured tasks, assessments and pieces of work and positive and supportive, scaffolding feedback from the teacher pointing out how the work can be improved and developed. There is a much better ongoing conversation with learners during the course of the junior cycle at relevant key points and the arrival of the examinations is not a falling over the cliff situation or a date with destiny. It is a more natural continuation of the assessment conversation they have had.

On the issue of adding additional time, the State Examinations Commission did provide reassurance. I will hand over to Mr. Aidan Farrell on that issue.

Colette Kelleher (Independent)
Link to this: Individually | In context | Oireachtas source

We would like to know how it was done. That is the question.

Mr. Aidan Farrell:

I thank the Deputy for his questions. In terms of the framing of examination papers, the State Examinations Commission has a detailed protocol and procedures in place. We provide our drafters and setters with detailed manuals in terms of how to go about setting examination papers that would be free from bias or any form of discrimination and that will accord with the syllabus specification for the particular subject. These expert people, who are either serving or retired teachers, work under the direction of the chief examiner. Within the team, we will have a drafter, a setter, assistant setters, on occasion, and translators in the case of subjects that need to be translated, all of whom are expert in the subject area and in the devising of test instruments. Part of our responsibility is to ensure that the examination paper can be completed within the time set down in the specification. We also need to ensure that the examination itself in terms of its content and questions is fully testing and sampling the full breadth of the particular syllabus.

In terms of junior cycle English this year, having received the online petition and recognising that there was a very real issue of concern among a large group of candidates, we took a number of steps. The chief examiner for junior cycle English would have liaised closely with his counterpart, the chief examiner for leaving certificate English, in terms of subject expertise and as another expert pair of eyes. We also asked a scrutineer to review the test paper. This is a subject expert who has had no involvement in the setting of the paper, that is, who has no ownership of the paper and would have no understandings or misunderstandings around the paper. This person sits down and sits the paper. Scrutineers have a number of important roles. One is to determine whether the paper can be conducted within the time allowed. Second, do the questions themselves make sense? Is it a sensible paper? Third, is the paper in accordance with the subject specification as set out in policy by the Department? That was the final reassurance that we brought to the process this year. We had someone independent of the process sit down and actually sit the paper to reassure and guarantee us that the paper could be completed in time. That allowed us to offer reassurance to candidates throughout the country.

The Deputy also raised the issue of how the State Examinations Commission knows that candidates are happy with the outcome of the examinations. There are a number of ways in which we get that feedback immediately after examinations. Generally, in the national media each day, each of the teachers unions will appoint a subject specialist who will draw feedback from among the membership in terms of how the paper has been received by candidates and the teacher's own expert view on how the paper read, how it sampled the curriculum, whether it was fair to candidates and whether it was sufficiently challenging but also sufficiently accessible for candidates. That whole range of issues will be addressed by the subject expert. That feedback comes to us.

Shane Cassells (Meath West, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

Bar reading the Irish Independent'sassessment of it, was particular attention paid to this paper given the fact that it was raised and highlighted?

Mr. Aidan Farrell:

Absolutely. I have to say that all of the spokespersons at the time, on behalf of the various unions, would have commented on the time issue as well as the content of the paper. The second avenue through which we gain feedback on the papers is through the subject associations. These are professional associations be they of English teachers or Geography teachers. Shortly after the examination, they will canvass their teaching members on how the paper was received, how it was for their candidates and so on. That is a valuable part of the quality assurance for us in terms of whether it was a particularly challenging paper - perhaps too challenging - compared to previous years and whether there were any issues around the paper that need to feed into our framing of our marking scheme so that we will have a fair and balanced approach to the marking of the papers.

Colette Kelleher (Independent)
Link to this: Individually | In context | Oireachtas source

Before I call Deputy Brendan Ryan, I have a question on the numbers. How many young people sat the junior cycle English paper this year?

Mr. Aidan Farrell:

It was of the order of 55,000 between higher level and ordinary level.

Colette Kelleher (Independent)
Link to this: Individually | In context | Oireachtas source

It is a big statement to make to say that there was no difficulties experienced given that 55,000 people sat it. I understand that the Department went to the bodies, but 55,000 people sat it. Mr. Farrell spoke about unions, teachers and others but I would also be interested in, as in the case of this petition, the people who actually sat it themselves. I call Deputy Brendan Ryan.

Brendan Ryan (Dublin Fingal, Labour)
Link to this: Individually | In context | Oireachtas source

First, I congratulate Tara O'Sullivan for taking this to this point today and organising the petition and I thank the officials from the State Examinations Commission and the Department for coming in to talk to us about the petition today.

A couple of things come to mind. Mr. Ó Donnchadha, or perhaps it was Mr. Farrell, seemed to suggest that the Department would not consider mocks an effective use of school time. If that is the case, has the Department made recommendations to schools to desist from them? To the Department's knowledge, are there any schools not doing mocks on a regular basis? It would seem to me to be a feature of school life. To suggest that it ought not happen is perhaps something that we should not be suggesting. I would suggest that it is something that will continue and that the school community likes doing them. Parents like them as well because they test their children.

It was touched on somewhat but in terms of capturing feedback about time in the exams in a systematic way, do the delegates believe that is being dealt with adequately on an ongoing basis? Presumably it is factored into decisions in the following year and so on. How is the time to be allowed decided in the first instance? Is it the case that a decision is made that it will be a two hour exam and that, therefore, the questions are to fit into a two hour scheme? Should time be a factor in exams? I know that we have to manage exams and the timetable but it would seem to me that, if he or she has a certain amount of knowledge, a person should never come out of an exam feeling that he or she had more to say and to do and that time was a factor. I would be interested in the witnesses' comments in that regard.

In terms of perhaps not endorsing commercial entities and all of that kind of stuff, is there a case for regulation of these commercial entities given that they are likely to be an ongoing feature of school life?

I note the two actions suggested, including sending a message to providers. The question is who would do that. Would it be the Department or this committee? The Department committed to drawing attention to website materials and so on.

Colette Kelleher (Independent)
Link to this: Individually | In context | Oireachtas source

There were some questions about mocks and their value, the Department's position on them, their regulation at the end and time. How is time decided and is time that important?

Mr. Gary Ó Donnchadha:

The context in which the mocks are currently serving a purpose is about to change over the lifetime of the introduction of the junior cycle. That is certain. There are arguments that mock examinations play a role in the current dispensation, but as the junior cycle moves on through the implementation of more and more subjects, the assessment conversations teachers have with students will begin to populate the calendar right through the three years of junior cycle. From a Department point of view, we have to point out the opportunity cost in the loss of teaching time, especially if we can demonstrate and promote the kind of structured, gainful student learning experiences which can be facilitated while also serving an assessment purpose so that teachers capture the learning much earlier in the process. They are certainly capturing the learning formally at key points during the second and third years. Those learnings factor into the junior cycle profile of achievement and actually will be a very attractive report on the many dimensions of learning, not only the State examinations piece but the recording of their performance and achievements on classroom-based assessments and what we call "other areas of learning". These are areas where the teachers will for the first time have an opportunity to input directly on the student profile the valuable learning encounters in which the student participated, whether it is debating, tackling a challenging sporting encounter or participating in a musical event. These are all important aspects of the learning which will be pulled together. We are not talking about banning mock examinations or anything like that at this point, but I have no doubt that as the junior cycle adds in more subjects over the years - we are running right up to 2022 in phasing in this - the context will change for mock examination providers. Perhaps their business will evolve into more strategically supporting the other kinds of learning schools are trying to promote for which materials and assessment tasks are needed throughout the period of the junior cycle rather than just focusing on the terminal examination, the significance of which has been reduced. That would be a logical place for those providers to be considering. That could indeed provide a valuable service if they provided the right kinds of supports and materials to serving teachers.

Colette Kelleher (Independent)
Link to this: Individually | In context | Oireachtas source

Is there any commentary on the time question from Deputy Ryan?

Dr. Tim Desmond:

Observations on that are twofold. The first thing is to stay with English. In the context of junior cycle reform, we are satisfied that within a two-hour allocation, there is adequate time for us to set tasks which give candidates the opportunity to display the skills they have achieved and the things they know. Within that period, we can elicit adequate information to make a judgment on a candidate's level of achievement at the end of it. As far as exam duration is concerned generally, our position is that there is adequate time. Exams are not a race against the clock albeit there must be some time constraints on them in so far as there are problems with leaving candidates there indefinitely. Without time constraints, one will get the approach which is to write everything one knows. That is not educationally or assessment sound because what one needs people to do is to answer the question that was asked. The response to that is judged against the time available. To do that, the higher order skills of being able to analyse a question, draw conclusions and synthesise an answer are important.

In our view, the most important people to look at the clock in the first place are us and our setters to determine what questions we can ask which can be meaningfully answered within the allocated time and whether the array of those is such that we can elicit the information we need to say a candidate has achieved to a certain level. We work within that. Within junior cycle reform, the spirit of it and the change of philosophy there, we are certainly comfortable that a two-hour examination gives candidates adequate time to show what they have achieved in the three years without it being a race. If one makes examinations very long, they become marathons for other reasons. There are other issues at that stage and they become endurance tests and marathons. Within the context of it, two hours is fine.

Denise Mitchell (Dublin Bay North, Sinn Fein)
Link to this: Individually | In context | Oireachtas source

We acknowledge Tara O'Sullivan, congratulate her and wish her the best with her results. The fact that more than 12,000 students signed the petition shows us how worried they are about this exam. We should bear in mind that it is a difficult time for students and their families.

Was there much feedback on the sample paper produced by State Examinations Commission? Was anything flagged when it was sent to schools? What is the average cost of producing State exam papers? Have we ever looked at the commission producing the mock papers for schools? On the feedback issue, the commission went to the membership and teachers. I know students run out the door when the exam is over and it is a few days later that they digest what has happened, but is there any way for their feedback to be taken in? Can the commission provide us with the number of complaints it received, if any, around this particular English exam paper?

Colette Kelleher (Independent)
Link to this: Individually | In context | Oireachtas source

There are questions around the 12,000 people, which we need to note, the cost and whether the State Examinations Commission could do the work itself instead of commercial operators and, lastly, what plans it has in light of the petition to get feedback from students as well as all the other bodies which were mentioned.

Mr. Aidan Farrell:

I mentioned earlier two mechanisms by which we receive feedback after the examinations. It is either the expert subject commentators as reported through the media or through the actual subject associations, namely the teachers dotted across the country feeding back through their representative structures and as subject experts feeding into our marking processes through an engagement with the chief examiner to set and frame the marking scheme for that examination. A third element of feedback we get is from parents and candidates after the examinations.

Having worked for 15 years in the SEC, my experience is that if the candidates or their parents have concerns about the content of a paper, either that it was too challenging or that the candidates had a particular difficulty with a question that really fazed and threw them for the purpose of completing the examination, they contact us immediately after the examination. On occasion we have had 2,000 to 3,000 calls or emails from candidates or their parents around a particular question if candidates were somewhat thrown in the course of the examination. I believe we are an open and accessible organisation in terms of getting that feedback. The teachers are engaging with the candidates at school level, because quite often the teacher will go into school on the day of the examination to reassure and show support to the candidates and to see how things went for them. There are well-honed paths of feedback from that point of view.

Colette Kelleher (Independent)
Link to this: Individually | In context | Oireachtas source

Deputy Mitchell asked if there were any specific complaints about the paper this year.

Mr. Aidan Farrell:

No.

Denise Mitchell (Dublin Bay North, Sinn Fein)
Link to this: Individually | In context | Oireachtas source

No parent rang or contacted the State Examinations Commission?

Mr. Aidan Farrell:

To my knowledge no, but I will check with my colleagues.

Dr. Tim Desmond:

As far as I remember there was one email about the leaving certificate and one about the junior certificate. I have been manning the emergency desk for the past six or seven years and it was the quietest afternoon I have put in ever.

Colette Kelleher (Independent)
Link to this: Individually | In context | Oireachtas source

That is good. Does Deputy Murphy wish to ask a question?

Denise Mitchell (Dublin Bay North, Sinn Fein)
Link to this: Individually | In context | Oireachtas source

I am delighted the witnesses are here. I am enjoying the answers to the questions.

Colette Kelleher (Independent)
Link to this: Individually | In context | Oireachtas source

I am sure the Deputy is glad he is not to be doing the junior certificate.

Eugene Murphy (Roscommon-Galway, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

That was a long time ago.

Colette Kelleher (Independent)
Link to this: Individually | In context | Oireachtas source

It was the intermediate certificate in those dark days.

Eugene Murphy (Roscommon-Galway, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

Having read through the presentation, I have a brief question. Nowadays there is more of a buildup of anxiety, and there seems to be more media coverage and articles on the dos and don'ts of examinations. What is Mr. Farrell's opinion of that? I have a view that examinations are hyped up too much for the students, the teachers and parents as well.

Mr. Aidan Farrell:

As the person charged with responsibility for the operation and delivery of the State examinations, we recognise the importance of the State examinations in an Irish context, particularly the high stakes leaving certificate examination. The nature of the examination systems means the leaving certificate examination is a pivotal access point, be it into the world of work or to the world of further studies. We recognise the significance of that examination. We are very conscious as an organisation of the need to keep things as stable as possible for the candidates and their parents in the lead-up to the examinations and to deliver a credible examination system without in any way setting out to catch candidates out in any way. That is not the purpose of the examination.

I think Deputy Murphy is correct that for a period there was a particular hype around the State examinations, especially leading up to the written examinations in June. Undoubtedly that would increase the focus, the stress levels and the tension levels for candidates and their families. I have noticed a lessening of that in the past two to three years. Undoubtedly, there is, and rightly so, commentary on the examinations. They are significant in Irish family life and in the education environment. There is a focus on them, but the focus has been balanced with information and feedback on how students have perceived the examinations. They then move on to the next examination.

A number of years ago, there might have been a greater focus on tips for tomorrow, which only feeds into people's stress level, because if the tip is on the poet one has not revised, it will stress people to hear this expert extolling a particular approach on radio or television on the night before the examination. That has lessened significantly in recent years. As the body responsible for both maintaining public confidence in the examinations as well as maintaining a stable environment for candidates taking their exams, that is certainly something we would welcome.

Colette Kelleher (Independent)
Link to this: Individually | In context | Oireachtas source

I know that some aspects of Deputy Mitchell's question were not answered.

Denise Mitchell (Dublin Bay North, Sinn Fein)
Link to this: Individually | In context | Oireachtas source

How much does it cost to produce the State examination papers and did anyone ever look at the possibility that schools could purchase quality assured mock papers from the State Examinations Commission?

Mr. Aidan Farrell:

I do not have the information to hand on the average cost of producing an examination paper, but I will get the information to the committee in a matter of days.

In terms of whether the State Examinations Commission has ever considered producing mock papers, to be honest, we have not. Let me explain the reason. There is probably a mixed practice in schools in that quite a number of schools provide mock examinations, but not all the mock examination papers would come from commercial suppliers. In many cases, teachers will set their own mock questions in a mock examination paper. I suspect one of the reasons is that mock examinations are taken in spring each year. The leaving certificate, or for that matter the junior certificate cycle, are a two or three-year programme, respectively. Teachers have not finished teaching those programmes. A mock paper could be put in front of candidates covering a range of areas which the students would not have yet covered in the classroom. Any value the mock examinations would have is very much diminished at that point because the students have not covered the material. They are learning very little from that experience. Another area where we would have to be very careful is the significance of the examinations in an Irish context. Naturally students want to prepare as well as they can for the examination. It is only human nature for people to try to predict what they think will come up. If a question on the poet Sylvia Plath came up last year, it will not come up this year, or whatever the case might be. That is human nature and that is fully understandable. If it were the case that the SEC were either to produce mock papers or to provide oversight or endorsement of the mock examination papers provided by one or a number of the commercial suppliers, we would then be entering into the area of commenting or people believing they know what may come up in the live examination. Clearly, if we were to design a mock paper and ask a question framed in a particular way, are we going to ask that question again in the June examination? Even in terms of proper preparation from an education or assessment point of view, there would be real negative implications for teaching and learning if we were to engage in that sort of activity.

Mr. Gary Ó Donnchadha:

The Department of Education and Skills and schools now place a major emphasis on the student voice. It is in the Education Act. Schools are encouraged to tap into and access the student voice, and they can do this through the school self-evaluation process and the external inspections of schools through the whole-school evaluation, WSE. Every whole-school evaluation that is conducted involves a student questionnaire to survey students directly on the issues of concern to them. Those responses are processed. As I reflect on it, it might be a wholesome thing to check the issues that are coming up in terms of the assessment experience and the examination experience of students during the course of the junior cycle. These might be a very appropriate set of questions to ask. The students are on a learning journey during the three years and, at various points, really valuable information surfaces. The outcomes of the WSE questionnaires are published for the school system. Indeed, they form a relevant part of each school's own reflection on its services and the quality of its provision, and the data from the evaluations is published along with the school report. That is a very important point.

The Department of Children and Youth Affairs is actively involved in supporting the student voice as well.

Dáil na nÓg frequently highlights issues of concern. Our Department goes to the Dáil na nÓg events where we are often called to account and called on to respond. Where concerns are summarised in themes, we happily take them on board and feed them in wherever the change is needed. It gives the young people agency in their own lives. We also encourage that at school level where they can comment on their learning experience.

Colette Kelleher (Independent)
Link to this: Individually | In context | Oireachtas source

It is very encouraging to have the voice of the student. We also have the voice of the student in this petition and we will get back to that.

Would the State Examinations Commission, SEC, be prepared to consider its feedback processes in light of this petition? I am concerned because 55,000 people sat the junior cycle examination. There were no complaints, the line was not hopping but the SEC heard from the media, the teaching bodies and the professional associations. Could it review its feedback process on the students' experience and build that into its processes?

The Department and the SEC are ambivalent about the use of mocks, particularly in respect of the new junior cycle. Given that and the difficulties of quality control, of the SEC getting more directly involved and regulation, is there an opportunity for the Department to review their use by schools and to consider issuing guidelines? The Department says it has not endorsed their use or considered them to be an appropriate and effective use of school time, which is a strong statement, yet they happen all over the country and caused concern to this individual. Given the ambivalence, would the SEC consider abandoning the practice?

Mr. Aidan Farrell:

We have several feedback processes and we engage with the education partners, which include management bodies, school management, representatives and of course parents, and the National Parents Council Post-Primary. We are finalising our strategic statement and policy direction for the next few years. One area we will consider is the engagement with stakeholders and their feedback on the service, as well as input into its design and operation. We can consider how we might trap the candidate voice in a way that has not been considered before. The examination system operates through the schools. We are dealing with students who are 15 years of age at junior cycle or 17 or 18 years at leaving certificate level. Schools enter their candidates for examination and act as advocates on their behalf, rightly so. In that way we get a feedback from schools on the impact of the delivery and operation of the examinations system.

Colette Kelleher (Independent)
Link to this: Individually | In context | Oireachtas source

I thank Mr. Farrell for being willing to consider having the student voice as part of that process.

Mr. Aidan Farrell:

We have the ultimate feedback on how candidates perceived an examination in their scripts. If candidates ran out of time, for example, that would be absolutely apparent to us when marking them. If they found a particular question unduly challenging or impossible to answer that would also be apparent. It is a mute voice at one level but it is a very telling one.

Mr. Gary Ó Donnchadha:

From the Department's point of view it is the opportunity cost that we lose the opportunity to engage with learners in a formative way. These are very young learners, aged 13 or 14, and we already have decided that setting them high-stakes examination tests is the wrong way to support their learning. That is why the reforms are driven by the Government to shift the emphasis back into the classrooms and close to the teaching-learning loop where the really valuable good work happens. We do expect the goalposts to be seen to have changed over time. We expect that the rebalanced dispensation that will exist in a short few years will make it very clear that there is no point in trying to service a high-stakes terminal examination at the end of junior cycle. We are more interested in seeing how resources and materials can be developed and facilitated for the formative classroom-based assessment processes. I will take away the suggestion that we consider some guidance for schools in the context of what they do around the new junior cycle and how the legacy practice of mock examinations fits into that, if at all. I will be happy to take that back to the Department and give it some thought.

Colette Kelleher (Independent)
Link to this: Individually | In context | Oireachtas source

Thank you. Democracy will have been well served if Tara O’Sullivan's petition leads to new guidelines and perhaps a wider feedback process from the student voice.

Denise Mitchell (Dublin Bay North, Sinn Fein)
Link to this: Individually | In context | Oireachtas source

A total of 12,000 people signed the petition, one contacted the hotline. Were any amendments made to the 2017 junior cycle paper following this petition?

Dr. Tim Desmond:

We look at papers continuously until they go to press. They are under review and the scrutiny process occurred subsequent to that. The paper was organic at that point. When the petition was made, the paper had not been completed and therefore was a work in progress.

Eugene Murphy (Roscommon-Galway, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

What has happened today, following a student's petition, shows that great engagement can take place between young people and officials. I hope this opens up the process.

I hope it gets some media coverage in terms of the avenues that are open for people to send in petitions and have issues like this discussed. It is a breath of fresh air and is so important for our democracy. I very much welcome it. It is a great experience. Sometimes there is criticism that young people cannot get involved. We see what has happened in this process, however. In fairness, the witnesses have performed very well and taken on board our comments and the content of the petition. It is good for the whole system.

Colette Kelleher (Independent)
Link to this: Individually | In context | Oireachtas source

I thank Deputy Murphy. I will not try to improve on that.

Eugene Murphy (Roscommon-Galway, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

Will the Vice Chairman not ask me about my junior certificate again?

Colette Kelleher (Independent)
Link to this: Individually | In context | Oireachtas source

We will compare our results later. I thank all the officials for coming in and presenting to the committee today. I thank them for being open to our suggestions. We may well see them again to ask if they have acted upon those suggestions on behalf of the 12,000 people. Perhaps they will encounter some of the younger people directly themselves.

I propose that we suspend for a moment to allow the witnesses to exit. We will then resume in private session for the remainder of the meeting. Is that agreed? Agreed.