Donnchadh Ó Laoghaire (Cork South Central, Sinn Fein)
Link to this: Individually | In context | Oireachtas source

236. To ask the Minister for Education and Skills when a person (details supplied) retired from an organisation or associated third party institute or foundation; and her views on whether it is proper in view of the person's close relationship with the organisation and that the organisation was chosen to complete the independent review of the calculated grades model and work, including that of the person. [29832/20]

Norma Foley (Kerry, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

My Department appointed Dr. Kentaro Yamamoto to the Leaving Certificate Calculated Grades National Standardisation Group in an expert role which included a level of contingency cover for the company Polymetrika; as technical advisor on the standardisation process; and to provide expert advice and oversight regarding data integrity.

Prior to his retirement in May 2020, Dr. Yamamoto was Deputy Director of the Centre for Global Assessment in Educational Testing Service’s (ETS) Research & Development Division in Princeton, NJ. He had retired from ETS when contracted to work on the Calculated Grades National Standardisation Group.

Dr. Yamamoto had no role in the review conducted by ETS regarding the error in the coding for Calculated Grades.

Donnchadh Ó Laoghaire (Cork South Central, Sinn Fein)
Link to this: Individually | In context | Oireachtas source

237. To ask the Minister for Education and Skills the positive factors which led to the choosing of a company (details supplied) to supply the leaving certificate calculated grading model; and if there was work of a similar purpose, or with a similar level and skill required, undertaken by the company previously. [29833/20]

Norma Foley (Kerry, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

The decision to adopt a model of Calculated Grades by my Department was a direct result of COVID-19, which prevented the state from running the conventional Leaving Certificate Examinations.

As part of initial contingency planning for the Leaving Certificate 2020, the State Examinations Commission (SEC) procured international expertise through Polymetrika International Inc. (PII). PII’s principal, Fernando Cartwright, is a highly skilled expert in the areas of psychometrics, large-scale educational assessments, and computer programming related to these areas. He has had significant involvement with the Irish educational system since 2010 and, unlike most experts of his calibre, has a strong knowledge and familiarity with the Irish educational system. He worked with the Educational Research Centre in providing a full technical review of Ireland’s PISA results in 2009, producing innovative research in an Irish context that has had a global impact

Mr Cartwright’s extensive experience, expertise and unique set of skills, coupled with his knowledge of the Irish education system, were the reasons he was originally approached by the SEC. Other possible sources of expertise/agencies that the SEC or my Department might have turned to (such as OfQual in England, or the Scottish Qualifications Authority) were fully consumed with the development of their own alternative models, all of which were designed around the requirements of the A-level system rather than the broader-based Leaving Certificate approach.

Polymetrika played an integral part of the development and running of the Calculated Grades model, and as such the contract for Polymetrika was extended to implement the Calculated Grades model. This was done following the decision of Government on 8 May 2020 to postpone the Leaving Certificate Examinations.

Donnchadh Ó Laoghaire (Cork South Central, Sinn Fein)
Link to this: Individually | In context | Oireachtas source

238. To ask the Minister for Education and Skills the level of oversight applied over the calculated grading formula; if all previous testing was replicated after it was amended to remove any aspect of school profiling in the days before the results were released. [29834/20]

Norma Foley (Kerry, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

The decision to adopt a model of Calculated Grades by my Department was a direct result of COVID-19, which prevented the state from running the conventional Leaving Certificate Examinations. A range of steps were taken to ensure adequate oversight of the Calculated Grades process.

Technical details of the Calculated Grades model and standardisation process were published on the date of issue of the Calculated Grades results on 7 September and are available at .

The design of the Calculated Grades model was informed by advice from a Technical Working Groupcomprising experts drawn from the State Examinations Commission, the Inspectorate of the Department of Education and Skills, the Educational Research Centre and international external expertise.

The National Standardisation Group is the decision-making group responsible for the implementation of the iterative design and development cycles required to produce and refine the standardisation process and the application, review, and adjustment of the data in line with the commitments, principles, parameters and constraints associated with the calculated grades process to arrive at fair and just representations of student performance. The integrity, validity and reliability of the process of national standardisation was overseen by the National Standardisation Group.

Details regarding this group, including its terms of reference, membership and its decision making framework are available in the Report from the National Standardisation Group to the Independent Steering Committee and the Programme Board which is available at /. Minutes of the meetings of this group are available at

A non-statutory Independent Steering Committee to oversee the quality and independence of the process being undertaken by the Calculated Grades Executive Office was established by me. The Steering Committee comprises six independent members appointed by me and is chaired by Dr Áine Lawlor, former Director of the Teaching Council. The Steering Committee was established to provide assurance as to the quality and integrity of the outcomes of the Calculated Grades system. The 1st Letter of Opinion regarding the Calculated Grades System of the Independent Steering Committee can be found at: Details of the Independent Steering Committee including its membership are also available at the above link.

A Calculated Grades Programme Board provided governance and decision-making for the overall programme associated with the Calculated Grades Model and process. The Board was chaired by an Assistant Secretary General from the Department of Education and Skills and membership comprises officials from the Calculated Grades Executive Office, the wider Department of Education and Skills and the Department of Further and Higher Education, Research, Innovation and Science.

In addition, Dr Janet Brown, former Chief Executive of the Scottish Qualifications Authority, was appointed to the role of External Reviewer. The External Reviewer is an independent expert, unconnected with the design of the Calculated Grades model, who provides further independent oversight of the system and overall validation on the model, including the operation of the appeals system. Dr Brown’s Statement in Advance of the Issue of Results can be found at .

The system of Calculated Grades is a complex and sophisticated system which had to be developed from scratch within an extremely tight timeframe in order for students to get their Calculated Grades results in time.The statistical model itself was developed specifically for the Irish Leaving Certificate. The Department had checks at a number of stages to ensure that data was collected correctly from schools and transferred correctly between the various stages of the standardisation process, the validation process and the grading processes.

Donnchadh Ó Laoghaire (Cork South Central, Sinn Fein)
Link to this: Individually | In context | Oireachtas source

239. To ask the Minister for Education and Skills if the bonus for completing leaving certificate exams through Irish was taken into consideration when the algorithm was being applied to give calculated grades to students from Gaelscoileanna; and if these students received the bonus for 2020. [29838/20]

Norma Foley (Kerry, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

Circular letter 0037/2020, ‘Calculated Grades for Leaving Certificate 2020 - Guide for Schools on Providing Estimated Percentage Marks and Class Rank Orderings’, which was published on 21 May, set out the detailed guidance for schools on the operation of the Leaving Certificate calculated grades process. The document provided detailed information on the process of estimating marks for students to receive calculated grades.

This included guidance to schools that, in arriving at an estimated percentage mark in the case of students who were entered to take their examinations through Irish, the teacher should base her/his estimate of the student’s likely mark on the assumption that the bonus that is applicable for answering the written paper through Irish is included in the estimate. Therefore, as the bonus marks for sitting examinations through the medium of Irish had already been provided in the teacher estimates, this would also have been reflected in the final Calculated Grades awarded to the students.