Aodhán Ó Ríordáin (Dublin Bay North, Labour)
Link to this: Individually | In context | Oireachtas source

111. To ask the Minister for Education and Skills if the junior certificate core subjects of Irish, English and maths were given equally weighting during the process of calculating the estimated grades for the leaving certificate; and if she will make a statement on the matter. [28959/20]

Aodhán Ó Ríordáin (Dublin Bay North, Labour)
Link to this: Individually | In context | Oireachtas source

112. To ask the Minister for Education and Skills if the specific percentage result or overall grade of students were used from their junior certificate to calculate their leaving certificate 2020 results with regards to the leaving certificate 2020 and the calculated grades process; and if she will make a statement on the matter. [28960/20]

Norma Foley (Kerry, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

I propose to take Questions Nos. 111 and 112 together.

The decision to adopt a model of Calculated Grades by my Department was a direct result of COVID-19, which prevented the state from running the conventional Leaving Certificate Examinations.

The design of the Calculated Grades model was informed by advice from a Technical Working Group comprising experts drawn from the State Examinations Commission, the Inspectorate of the Department of Education and Skills, the Educational Research Centre and international external expertise.

Schools provided an estimated percentage mark and a rank order for each student’s subjects. The process of national standardisation was applied to the school information in order to ensure comparability between the standards applied by individual schools and the national standard. We know from research that teachers are very good at making judgements about their students in the local context of the school. Schools approached this task in a very professional manner, in line with detailed guidelines about the process, but inevitably some schools were overly harsh in their estimations while others were overly generous. This is to be expected given that there is no national standard on which to base an estimated mark. But in order to be fair to the class of 2020, the teacher judgements made at the level of the school had to be adjusted so that a common national standard was applied.It was inherent to the system of calculated grades that school estimates would be subject to adjustment through this standardisation process.

These adjustments resulted in the school estimates staying the same or being revised upwards or downwards. The standardisation process operated on the premise that the school estimates should only be adjusted through the standardisation process where there was credible statistical evidence to justify changing them.

A standardising process happens every year and would have happened in 2020 had the Leaving Certificate examinations been run as normal.In the system of calculated grades, the standardisation process applied uniformly across all subject and levels and school types. The degree to which mark changes occurred related to the degree of over or underestimation in the school estimates for each subject and each level. This means that some students experienced mark changes from the school estimates but no changes to the grades based on the school estimates; while others will have experienced marks changes from the teacher estimates leading to grade changes in one or more of their subjects.

Following standardisation, the estimated percentage mark was converted to a calculated mark and subsequently, a calculated grade, which was provided to students on 7 September. It is only at this point that students were awarded a grade.

The use which has been made of Junior Cycle data in the model is set out in detail in the Report of the National Standardisation Group. Individual Junior Cycle results were not used to determine any individual’s Calculated Grades. Rather, the Calculated Grades process took account of the overall Junior Cycle performance of the Leaving Certificate class of 2020 in each school and used this data to help in predicting the likely range of Leaving Certificate performance of that group using related information about the relationship between performance at Junior Cycle and Leaving Certificate based on national data over time for that subject at that level.

On 30 September, I announced that two errors had been found in the Leaving Certificate 2020 Calculated Grades process. These errors related to the way in which the coding for the process utilised Junior Cycle data.

On 03 October, following a further review of the coding by an independent third party, I announced that a further error had been identified.

These three errors were rectified, and following this, a total of 6,100 students have received higher grades. These students were able to access their improved results 03 October.

Technical details of the Calculated Grades model and standardisation process were published on the date of issue of the results, 7 September, and are available here:

.

Aodhán Ó Ríordáin (Dublin Bay North, Labour)
Link to this: Individually | In context | Oireachtas source

113. To ask the Minister for Education and Skills when students will be notified if they are impacted by the two errors in the algorithm; and if students can now appeal same and request to see the way in which their grade was calculated. [28961/20]

Norma Foley (Kerry, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

The decision to adopt a model of Calculated Grades by my Department was a result of COVID-19, which prevented the State from running the conventional Leaving Certificate Examinations. The purpose of this process was to allow as many students as possible to progress to employment, further education and training, or higher education.

On 30 September, I made a statement that advising that two errors had been identified in the Calculated Grades process. These meant that incorrect grades were issued to some students when they received their results on 7 September.

All Leaving Certificate 2020 students received a text message on 30 September advising them to check for information regarding the announcement that was made that day.

On discovering the errors, my Department engaged a specialist third party, Educational Testing Service (ETS), to review the coding

On 03 October, I announced that the independent review had been completed by US-based company Educational Testing Services (ETS). All students received another text message. This message advised them either:

that they were receiving at least one higher grade and needed to check the portal to view their results or,

that they were not impacted and their grades were unaffected.

All students who were receiving a higher grade also received an email advising them to log onto the Calculated Grades Student Portal to access their amended results.

Students who are impacted were able to log on to the Calculated Grades Student Portal from 6.00pm on 03 October to view their improved grades and print a new statement of provisional results which will replace their first statement.

Students whose grades were not affected would see no change to their Calculated Grades if they log onto the Calculated Grades Student Portal. These students may, however, have seen that they achieved a higher calculated mark in a particular subject(s), but that this increase did not result in an increased grade.

The appeals process will continue as normal following the announcements on 30 September and 03 October regarding errors found in the Calculated Grades model. This applies whether or not a student received an improved grade following the rectification of the errors. This is because the appeals process is a technical appeal, focussed on looking for errors in the transmission and processing of student data through the process.

The appeals process is a process review focussed on looking for errors in the transmission and processing of student data through the process. It is not possible to appeal the information (estimated percentage mark or rank order) provided by the school. Due to the nature of the Calculated Grades system the professional judgement of the school is outside of the appeals process. It should further be noted that the design of the statistical model and the application of the national standardisation process is also outside of the scope of the appeals process.

At the time of closing the appeal application process on Wednesday 16September, some 12,300 students had appealed almost 33,700 grades.

Every effort will be made to process appeals as quickly as possible but it is not possible at this time to commit to a date for the issue of the appeal results. Students will be notified of this date as soon as possible.

Aodhán Ó Ríordáin (Dublin Bay North, Labour)
Link to this: Individually | In context | Oireachtas source

114. To ask the Minister for Education and Skills when students will be made aware of the results of the appeals process regarding any potential administrative errors and what impact this will have on the review happening with the errors in the algorithm (details supplied). [28962/20]

Norma Foley (Kerry, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

The Calculated Grades Student Portal reopened on 14 September to allow students to view their estimated percentage marks and calculated marks to allow students to consider this data if they were considering an appeal of their Calculated Grade.

The appeals process is a process review focussed on looking for errors in the transmission and processing of student data through the process. It is not possible to appeal the information (estimated percentage mark or rank order) provided by the school. Due to the nature of the Calculated Grades system the professional judgement of the school is outside of the appeals process. It should further be noted that the design of the statistical model and the application of the national standardisation process is also outside of the scope of the appeals process.

At the time of closing the appeal application process on Wednesday 16September, some 12,300 students had appealed almost 33,700 grades.

The appeals process will continue as normal following the announcements on 30 September and 03 October regarding errors found in the Calculated Grades model. This applies whether or not a student received an improved grade following the rectification of the errors. This is because the appeals process is a technical appeal, focussed on looking for errors in the transmission and processing of student data through the process.

Every effort will be made to process appeals as quickly as possible but it is not possible at this time to commit to a date for the issue of the appeal results. Students will be notified of this date as soon as possible.

It is not possible to provide an expedited appeal for any student or group of students. All appeals will be processed and results released simultaneously to ensure fairness and equity to all.

The CAO will be automatically notified by the Department of the outcomes of the appeals process.

If, as a result of an upgrade, a student becomes eligible for a college place at a higher CAO preference than the one they were already offered, they will be eligible for that course.

It is not possible to guarantee at this stage that, following a successful appeal, a student will be able to take up such a place offered in the academic year 20/21. This depends on factors such as the number of appeals received, the nature of the appeal, and the higher education course being applied for. However, where possible, higher education institutions will attempt to facilitate students who are upgraded and receive a new offer.

Aodhán Ó Ríordáin (Dublin Bay North, Labour)
Link to this: Individually | In context | Oireachtas source

115. To ask the Minister for Education and Skills her plans to undertake a full review of the entire calculated grades process and publish the results. [28963/20]

Norma Foley (Kerry, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

The decision to adopt a model of Calculated Grades by my Department was a result of COVID-19, which prevented the State from running the conventional Leaving Certificate Examinations. The purpose of this process was to allow as many students as possible to progress to employment, further education and training, or higher education.

On 30 September, I made a statement advising that two errors had been identified in the Calculated Grades process. These meant that incorrect grades were issued to some students when they received their results on 7 September.

The first error, in one out of 50,000 lines of code, was in relation to the processing of Junior Cycle data. The system was meant to draw on the core Junior Cycle subjects of English, Irish and Maths, and combine them with the students’ 2 best non-core subjects for all of the students in each class in each school. The coding error instead combined the core subjects with the students’ 2 weakest non-core subjects.

The error meant that, in some subjects, some students received Calculated Grades that were lower than they should have been, while some students received grades that were higher than they should have been.

In the course of a review which the Department then undertook, Departmental staff found a further error in the section of the code dealing with Junior Cycle results. The Junior Cycle subject Civil, Social and Political Education (CPSE) was meant to be disregarded as part of the model but was not. This second error, however, had a negligible impact on results.

I provided a further update on 03 October, announcing that improved Calculated Grades would issue to impacted students that day. This announcement was made following a review of essential aspects of the coding by Educational Testing Services, ETS, a non-profit organisation based in the United States.

ETS completed their review and provided it to my Department on 3 October. The review identified one further error, relating to how the code handled cases where students did not sit all three core subjects (Irish, English and Maths) at Junior Cycle level.

As a result of rectification of the three errors, a total of 6,100 students have received higher grades. This breaks down as follows:

5,408 students have received a higher grade, by one grade band, in one subject,

621 students have received a higher grade, by one grade band, in two subjects,

71 students have received a higher grade, by one grade band, in three or more subjects.

In addition to the students who received a lower grade than they should have in this year’s Leaving Certificate, some students received a higher grade than they should have. However, these students grades are not being amended and the students will not be affected.

ETS also identified an issue concerning how the algorithm used in the standardisation process treated students’ marks at the extreme ends of the marks scale (99% and over, 1% and below). However, ETS also stated that this had no material impact on the results and no student could receive a lower grade as a result of it.

Following correction of the errors the Department sent a corrected file of student results to the CAO. The CAO have identified that a little under 450 of the 6,100 students will be eligible to receive a new CAO offer. The Department of Further and Higher Education, Research, Innovation and Science will work with the CAO and the higher education institutions to see how these students can be facilitated to commence the course that they would, in other circumstances, have been offered in an earlier round. Any student who would have been entitled to a different offer in previous CAO rounds if they received the correct grade on 7 September will receive this offer or a deferred offer as part of the CAO Round 4 offers which are being released on 8 October 2020.

A copy of the report submitted by ETS is available on.

On 03 October I also stated that I have asked that an independent comprehensive review of the operation of the Calculated Grades process take place when the process is complete. This review will include independent international expertise.

Catherine Connolly (Galway West, Independent)
Link to this: Individually | In context | Oireachtas source

116. To ask the Minister for Education and Skills the details of the tendering process that resulted in Educational Testing Service getting the contract for the review of essential aspects of the coding used to implement the leaving certificate standardisation process; the value of the contract awarded to same; the number of companies applied for the tender; the way the tender was advertised; and if she will make a statement on the matter. [28971/20]

Catherine Connolly (Galway West, Independent)
Link to this: Individually | In context | Oireachtas source

117. To ask the Minister for Education and Skills the details of the tendering process for the contract, which was awarded to a company (details supplied) for the leaving certificate standardisation process in 2020; the number of companies applied for the tender; the details of the companies which applied for the tender; the reason given for the decision to give the contract to that company; and if she will make a statement on the matter. [28972/20]

Norma Foley (Kerry, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

I propose to take Questions Nos. 116 and 117 together.

The decision to adopt a model of Calculated Grades by my Department was a direct result of COVID-19, which prevented the state from running the conventional Leaving Certificate Examinations.

As part of initial contingency planning for the Leaving Certificate 2020, the State Examinations Commission procured international expertise through Polymetrika International Inc.

Polymetrika was contracted to provide statistical and psychometric expertise, initially on the basis of contingency planning in the event of the Leaving Certificate not taking place as planned.

The value of that contract was €71,500 to cover 65 days. Any additional days are to be charged at €1,100 per day.

Polymetrika played an integral part of the development and running of the Calculated Grades model, and as such the contract for Polymetrika was extended to implement the Calculated Grades model under the pre-agreed terms of the contract. This was done following the decision of Government on 8 May 2020 to postpone the Leaving Certificate Examinations.

Due to the time constraints the Department availed of the procurement process known as the Negotiated Procedure without Prior Publication. This is used in circumstances where it is a case of extreme urgency.

To date, the total gross cost of services provided by Polymetrika under contract has been approximately €193,000 which reflects services provided to end September.

This expenditure reflects their work in both the contingency planning and the development and statistical work around the operation of the model.

As was announced on 30 September, the Department has also engaged the services of ETS, Educational Testing Service, a US-based non-profit organisation which specialises in educational measurement, to review essential aspects of the coding. ETS submitted their report of the review to the Department on 03 October.

The cost of services provided by ETS has not yet been finalised.

Catherine Connolly (Galway West, Independent)
Link to this: Individually | In context | Oireachtas source

118. To ask the Minister for Education and Skills the number of persons who have applied to sit the leaving certificate examinations in November 2020; the details of the closing date for applications for the November 2020 sitting; if, based on the current restriction levels nationally, if she will state the likelihood of holding the exams in November 2020; and if she will make a statement on the matter. [28973/20]

Norma Foley (Kerry, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

The deadline for registration for the postponed Leaving Certificate 2020 examinations is 5.00pm on Wednesday 07 October.

The State Examinations Commission has statutory responsibility for operational matters relating to the certificate examinations.

In view of this I have forwarded your query to the State Examinations Commission for direct reply to you.

Catherine Connolly (Galway West, Independent)
Link to this: Individually | In context | Oireachtas source

119. To ask the Minister for Education and Skills if the coding error in the leaving certificate standardisation process, which resulted in the two worst non-core junior certificate grades of a student being counted, rather than their two best non-core grades, was made by a company (details supplied) or if it was as a result of an error at departmental level prior to instruction being given to the company; and if she will make a statement on the matter. [28974/20]

Catherine Connolly (Galway West, Independent)
Link to this: Individually | In context | Oireachtas source

145. To ask the Minister for Education and Skills if the coding mistake in the leaving certificate standardisation process, which resulted in the inclusion of CSPE in the junior certificate grade data, was made by a company (details supplied) or if it was as a result of an error at departmental level prior to instruction being given to the company; and if she will make a statement on the matter. [29232/20]

Norma Foley (Kerry, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

I propose to take Questions Nos. 119 and 145 together.

The decision to adopt a model of Calculated Grades by my Department was a result of COVID-19, which prevented the State from running the conventional Leaving Certificate Examinations. The purpose of this process was to allow as many students as possible to progress to employment, further education and training, or higher education.

On 30 September, I made a statement advising that two errors had been identified in the Calculated Grades process. These meant that incorrect grades were issued to some students when they received their results on 7 September.

The first error was identified by Polymetrika International Inc as having taken place when processing students’ data. The error, in one out of 50,000 lines of code, was in relation to the processing of Junior Cycle data. The system was meant to draw on the core Junior Cycle subjects of English, Irish and Maths, and combine them with the students’ 2 best non-core subjects for all of the students in each class in each school. The coding error instead combined the core subjects with the students’ 2 weakest non-core subjects.

The error meant that, in some subjects, some students received Calculated Grades that were lower than they should have been, while some students received grades that were higher than they should have been.

In the course of a review which the Department then undertook, Departmental staff found a further error in the section of the code dealing with Junior Cycle results. The Junior Cycle subject Civil, Social and Political Education (CPSE) was meant to be disregarded as part of the model but was not. This second error, however, had a negligible impact on results.

I provided a further update on 03 October, announcing that improved Calculated Grades would issue to impacted students that day. This announcement was made following a review of essential aspects of the coding by Educational Testing Services, ETS, a non-profit organisation based in the United States.

ETS completed their review and provided it to my Department on 3 October. The review identified one further error, relating to how the code handled cases where students did not sit all three core subjects (Irish, English and Maths) at Junior Cycle level.

As a result of rectification of the errors, a total of 6,100 students have received higher grades. This breaks down as follows:

5,408 students have received a higher grade, by one grade band, in one subject,

621 students have received a higher grade, by one grade band, in two subjects,

71 students have received a higher grade, by one grade band, in three or more subjects.

In addition to the students who received a lower grade than they should have in this year’s Leaving Certificate, some students received a higher grade than they should have. However, these students’ grades are not being amended and the students will not be affected.

ETS also identified an issue concerning how the algorithm used in the standardisation process treated students’ marks at the extreme ends of the marks scale (99% and over, 1% and below). However, ETS also stated that this had no material impact on the results and no student could receive a lower grade as a result of it.

A copy of the report submitted by ETS is available on .