Maria Byrne (Fine Gael)
Link to this: Individually | In context | Oireachtas source

I welcome the Minister of State, Deputy Smyth, and her officials. When we adjourned the debate the previous day we were on section 32, amendment No. 35, in the names of Senators Stephenson, Higgins and Ruane. I understand Senator Stephenson was in possession on the last day.

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

I thank the Minister of State for coming back to the House. I know when we left off we were talking about the question that Senator Ruane talked about, which is the term "gagging" and how it can be evocative and uncomfortable because no one wants to feel they are doing that. However, we strongly believe the provisions laid out in this Bill will make it very difficult for the CEO to express opinions on Government policy and to be scrutinised as a witness in committees. In order for the CEO to properly regulate and support Government policy, they need to be able to say what is good but also what is bad, and the provisions laid out in this Bill do not allow for that.That is why we have amendments Nos. 35 and 36. They are to allow the CEO to have opinions on Government policy and to appear before committees. Without that, we are effectively gagging them, even if that is not the intention.

Niamh Smyth (Cavan-Monaghan, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

I thank Senators Stephenson, Higgins and Ruane, who I believe all spoke on the same matter the last day. The provisions in sections 32 and 33 are not new and do not create any precedents or special restriction on the AI office. They reflect an established model of accountability whereby senior officials and chief executives account for administration, governance and performance, while Ministers account for Government policy. The CEO is not prevented from appearing before Oireachtas committees, providing evidence, answering questions, identifying operational challenges or explaining the impact of policy decisions on the work of the office. The provision recognises that the assessment of Government policy is a matter for Ministers, who are democratically accountable to the Oireachtas. The CEO remains entirely free to explain how a policy is being implemented, the resources required to implement it, any operational difficulties arising in the implementation and the practical consequences for the work of the office.

The distinction has been blurred between commenting on the effect of policy and commenting on the merits of policy. Nothing in these sections prevents the CEO from informing an Oireachtas committee that a policy has operational consequences, that its implementation has challenges or that there are resource implications or unforeseen effects. The sections do not affect the ability of the office or the CEO to provide advice to the Minister under section 9(1)(f) or through other official channels. The Government will continue to benefit from the expertise of the office in the deployment and implementation of AI-related policy and regulation. These sections do not diminish the independence of the office; they preserve the constitutional and administrative distinction that has long underpinned accountability across the public service. The chief executive officer remains fully independent in carrying out the functions of the office and remains fully accountable to the Oireachtas for the discharge of those functions. However, Government policy is properly a matter for Ministers, who are elected representatives and are directly accountable to the Houses of the Oireachtas for policies they adopt. For that reason, and in keeping with established governance arrangements across the public service, I cannot accept amendments Nos. 35 or 36.

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 36:

In page 27, to delete lines 35 to 38.

Maria Byrne (Fine Gael)
Link to this: Individually | In context | Oireachtas source

Senator Higgins has proposed that amendments Nos. 37, 37a and 37b be grouped together. Is that agreed? Agreed.

Alice-Mary Higgins (Independent)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 37:

In page 28, after line 36, to insert the following: "(3) Engagements under this section shall be subject to the Public Service Management (Recruitment and Appointments) Act 2004 and, where relevant, the Civil Service Regulation Acts 1956 to 2005.".

Amendment No. 37 looks to the issue of the engagement of consultants and advisers. While there are careful measures in relation to those who are hired as staff of the proposed office, consultants and advisers may be taken on under such terms and conditions as are considered necessary or expedient. "Expedient" literally means that it can be done quickly or simply. There is nothing in relation to what measures will exist, including to ensure no conflicts of interest and to ensure best practice. All the careful checks and balances that are proposed for the staff have no equivalent for consultants and advisers. The legislation simply refers to expediency and how their fees are to be paid. There is a real concern here because there is quite a likelihood that the consultants and advisers taken on will have been consultants and advisers to, or even direct employees of, those bodies that are to be regulated. There is a concern that there is not an appropriate check and balance in relation to issues such as conflicts of interest and ensuring proper safeguards. I will come to that more specifically.

Again, amendment No. 37 relates to provisions in section 35 allowing for the engagement of consultants and advisers where it is considered necessary or expedient. The previous section, section 34, outlines the arrangements for the appointment of staff to the office and requires that appointments be subject to the Public Service Management (Recruitment and Appointments) Act 2004 and the Civil Service Regulation Acts 1956 to 2005. The safeguards in those Acts – for example, sections 54 and 55 of the Act of 2004 – include that it would be an offence for someone to recklessly make an application for a position that was false or misleading in a material respect. They also deal with issues such as conflicts of interest.

Currently, there are no clear safeguards in place around the engagement of the office with consultants. As set out, the decision to engage consultants and advisers is purely at the discretion of the office. There are no protections against those who might commit the offence of being misleading in their dealings with the office. There are no protections in relation to conflicts of interest. There is not even clarity on which functions of the office consultants may be hired in relation to, including, potentially, consultancy in relation to the crucial areas of regulation, for example.

Amendment No. 37 simply seeks to apply the same kinds of safeguards and standards that apply to the staff of the office under the Public Service Management (Recruitment and Appointments) Act 2004 and, where relevant, the Civil Service Regulation Acts, to the hiring of consultants and advisers. Again, this is our endeavour to ensure that we do not end up in a situation where consultants become a risk in terms of unexamined or undisclosed conflicts of interest that could permeate or affect the operation of the office.

The related amendments, Nos. 37a and 37b, point to why this matter is so important. They relate to further concerns as to the role consultants and advisers are going to play within the AI office, particularly when we consider the "move fast and break things" mentality and the conflict between the AI-promotion function and the regulatory role the office is meant to play in ensuring no damage is caused by AI, including the very high-risk AI we have been discussing. There is a tension between the promotion role and the regulatory role. We are concerned about how this plays out, particularly for consultants.

The prohibition on the unauthorised disclosure of confidential information includes a reference to consultants who may disclose confidential information. This is what amendment No. 37a tackles. A couple of points arise in this regard. Section 36(1) states:

Subject to subsection (2), a person shall not disclose confidential information while performing functions as— (a) a member of the Board or a committee of the Board,

(b) a member of the staff of the Office,

(c) the Chief Executive Officer,

(d) a consultant engaged by the Office or other person providing services ...

There is an assumption here that consultants are being given access to confidential information. That goes back to the issue of there being no safeguards in terms of conflicts of interest. It is clearly envisaged by this section that consultants will be given access to confidential information that relates to the functions of the board.The other really important point is the fact that there are exceptions to the disclosure of confidential information set out in this section. The exceptions include where the disclosure of the confidential information by a person referred to in that subsection, which may be just a consultant hired for a particular purpose, is allowed when it is made in the performance of the functions of the office. However, when we go back to the functions of the office, we see that they include promotion of AI innovation. Are we creating a scenario whereby a consultant can disclose confidential information for the purpose of promoting AI innovation? We are in a very dangerous space there. What are the limits? Is a consultant allowed to divulge confidential information to the CEO of a multinational tech company if he or she believes the disclosure of that information will foster or promote AI innovation?

The AI regulation unit has stated that retaining these companies in the country is an overarching national imperative. Are we looking at situations where confidential information will be shared by a consultant, without any trace of accountability, with those companies because that is deemed to be in the interests of AI innovation? Presumably, as part of its function to promote AI innovation, the AI office also wants to retain these companies. Can we have clarity around how confidential information will or will not be shared with companies to incentivise them to stay in Ireland or to encourage them to expand their interests in Ireland? Can it be shared by consultants who have been employed without checks as regards conflicts of interest?

I have serious concerns around the discretion the Bill gives to consultants to disclose confidential information and to have as a cover this function of promoting AI. The same discretion is given to the CEO and staff of the office. That is problematic in itself but it is particularly concerning when persons who are acting without a proper screening for conflict of interest may be in a position to disclose that information.

Amendment No. 37aseeks to prohibit entirely consultants from being able to disclose confidential information. It would mean they are not allowed to disclose information and there are none of the exemptions that are attached to the office disclosing that information. A consultant should not have leeway to say he or she is disclosing confidential information for the purposes of any of these functions that are in the office. Amendment No. 37bseeks to prohibit the disclosure of confidential information where it is done in the performance of a function to promote AI innovation.

I hope the Minister of State will accept these amendments, or at least give them very serious thought, because there are gaps in terms of a regulatory system and where one creates ambiguities, gaps or areas with lack of oversight, it opens the system up to potential abuse or even perceived abuse. I ask the Minister of State to address these concerns and look to the disclosures measures. If she accepts either of these approaches to try to close that gap and indicates what checks and balances will be in place with regard to the operation and function of consultants by the office, I might accept that.

Niamh Smyth (Cavan-Monaghan, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

Section 36(1) establishes the restriction on the disclosure of confidential information by the office's board members, CEO and staff and any external consultants and advisers engaged in the office. Section 36(2)(d) ensures that a disclosure that qualifies as a protected disclosure under the Protected Disclosures Act 2014 is protected against criminal prosecution. The amendment, which would exclude persons referred to in paragraph (d) from this protection, would mean a consultant or adviser engaged by the office who becomes aware of suspicious wrongdoing could not make a protected disclosure without risking prosecution under section 36(3).

In addition, section 36(2)(f) allows disclosure to An Garda Síochána where the person believes the information may relate to an offence. The office will need to engage extensively with external consultants and advisers for expertise because the field of AI is so broad and is rapidly growing and evolving. Removing the protection for consultants and advisers would mean that someone in that category who discovers evidence of wrongdoing or a crime while working for the office would commit an offence by reporting it. While I understand the concern behind the amendment, which is that confidential commercial information should not seep out through consultants, I believe it is important that they are not stripped of the safeguards that allow them to report suspicious wrongdoing. For that reason, I cannot accept that amendment.

Section 36(2)(a) permits disclosure only where it is made in the performance of the functions of the office. This is not an unrestricted permission to disclose information. Any disclosure must be necessary and connected to the exercise of a statutory function. The function in section 9(1)(c) is to promote and foster AI innovation and literacy. That function is intended to support awareness, understanding, capability building and innovation in AI. It is not a vehicle for sharing confidential information obtained by the office in the course of regulatory activities. The amendment appears to assume that the innovation and literacy functions create a unique risk of disclosure. However, many of the office's other functions may require extensive engagement with stakeholders, including facilitating co-operation between authorities, supporting information sharing and enhancing public awareness. There is no clear policy basis for singling out one function. The office will need to engage with businesses, researchers, start-ups, public bodies and civil society organisations in carrying out its innovation and literacy role. Restricting disclosures in connection with that function could make it more difficult for the office to provide guidance, education resources and practical support.

Confidential information remains protected in any event. Section 36(4) narrowly defines confidential information as commercially sensitive information or information expressly designated as confidential by the office. The existence of the innovation and literacy functions does not weaken those protections. If there are concerns regarding the handling of commercially sensitive information, those concerns are better addressed to the governance arrangements, confidentiality obligations and authorisation procedures, rather than disapplying one particular statutory function.

The amendment may create legal uncertainty. A disclosure may simultaneously support several statutory functions, for example, promoting AI literacy, enhancing public awareness and facilitating regulatory co-operation. It may become difficult to determine whether a disclosure falls within or outside the exception.

The Bill is carefully drafted so that all of the office's statutory functions can be carried out effectively while maintaining appropriate confidential protections. Removing one function from the framework risks upsetting that balance without delivering any clear additional safeguard. For that reason, I oppose the amendment.

Maria Byrne (Fine Gael)
Link to this: Individually | In context | Oireachtas source

Before I call Senator Higgins, I acknowledge Deputy Michael Cahill in the Gallery. He is accompanied by Grace Kelly, John Kelly, Tara Kelly and Oran Kelly from Tipperary and Kerry. They are very welcome. I hope they enjoy their visit.

Alice-Mary Higgins (Independent)
Link to this: Individually | In context | Oireachtas source

This is a particular concern. The promotion function is very different. The core tension in this body is that, on the one hand, it is meant to be a regulatory body and, on the other hand, it is literally being given a promotional role, which we know has been described by the predecessor body as involving the financial interests of this sector as an overriding national imperative. That is a direct tension. On the one hand, it is the body responsible for ensuring that standards that have been set out collectively at European level are met and, on the other hand, it is pursuing the national imperative in terms of business promotion in this area. That is a real tension and that is why it is appropriate that we pick out that particular function of promoting AI innovation and literacy.

I do not agree that our proposal would interfere with the educational pieces because in the Minister of State's response, she described how there is a narrow definition of what is confidential information that explicitly includes commercially sensitive information.Somebody who is hired to do an educational programme is not going to accidentally come into the commercially sensitive information and I do not see them using commercially sensitive information. They are not coming under the definitions of the kind of information. There is not a question that they are dealing with these kinds of pieces. The other purposes the Minister of State described do not map onto it. Where there is a potential clash is between commercially sensitive information and AI innovation and the idea that you can disclose commercially sensitive information for the purposes of promotion of innovation in AI. Put those two pieces together. That is the problem. The problem is the function - we have talked about the functions of the body - and the information.

Regarding the idea that this could inadvertently affect school programmes, it will not because they do not involve commercially sensitive information. It is the CEO and the staff of the office who have commercially sensitive information. Those who benefit from that potentially commercially sensitive information are commercial operations. There is an issue if that information is being disclosed potentially to commercial operators because the desire is to promote and foster AI innovation. If they are given commercially sensitive information to support their innovation, that is a problem in itself. However, it is a problem if a consultant who has not had to go through a process in relation to conflict of interest, and who is not under this Bill properly bound by conflict-of-interest measures, is in a position to disclose that information. That is the problem. As the Minister of State described it, it remains an issue. It is an example of some missteps being made at the earliest stage of this new body, which will affect both its ability to deliver proper regulation and public confidence in its operation. That is why we were looking to have safeguards put in here. It is unfortunate that the Minister of State was not able to accept those ones. What safeguards is the Government going to put in place tackle these issues?

Maria Byrne (Fine Gael)
Link to this: Individually | In context | Oireachtas source

Before I call on the Minister of State, I acknowledge our colleague Senator Manus Boyle and his guest Kieran Harvey from Killybegs. I hope he enjoys his visit.

Niamh Smyth (Cavan-Monaghan, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

I do not have anything further to add. I appreciate where the Senator is coming from. As I said to her before, this is phase 1. We will be coming back to this. My door is always open. We can discuss it further at a later time. On this occasion, I cannot accept the amendments.

Alice-Mary Higgins (Independent)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 37a:

In page 29, line 11, after “referred to” to insert “, other than a person referred to in paragraph (d),”.

Alice-Mary Higgins (Independent)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 37b:

In page 29, line 12, after “Office” to insert “, other than the function listed in section 9(1)(c)”.

Maria Byrne (Fine Gael)
Link to this: Individually | In context | Oireachtas source

Amendments Nos. 38 to 40, inclusive, are related and may be discussed together by agreement. Is that agreed? Agreed.

Lynn Ruane (Independent)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 38:

In page 30, line 4, to delete “and submit to the Minister”.

Maria Byrne (Fine Gael)
Link to this: Individually | In context | Oireachtas source

Does the Minister of State wish to comment?

Niamh Smyth (Cavan-Monaghan, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

I cannot accept the amendment.

Alice-Mary Higgins (Independent)
Link to this: Individually | In context | Oireachtas source

The issues here are the same issues that have been highlighted before and concern the excessive control by the Minister in relation to the functioning of a body that should be independent. We previously highlighted that the Minister will get to determine certain elements. The Bill as drafted provides that the strategy statement would be submitted to the Minister. These amendments provide instead that it should be prepared in the form and manner that the office sees fit. They are trying to strengthen the independence of the office and the commission would, as soon possible after the publication of the strategy statement, make sure that it is laid before the Oireachtas. They would ensure that there is transparency in relation to the strategy statement and the Houses of the Oireachtas would have sight of that strategy statement as well.

Niamh Smyth (Cavan-Monaghan, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

The amendment would remove the requirement that the office submit its strategy statement to the Minister after it has been prepared. The strategy statement is an important governance and accountability document. It sets out the strategic ambitions and priorities of the office and provides a basis for assessing its performance over time. It is appropriate that the Minister, who is accountable to the Oireachtas for the legislative framework, receives the strategy statement. The requirement to submit the strategy statement to the Minister does not undermine the independence of the office. Rather, it forms part of the normal accountability arrangements that apply to statutory bodies throughout the public sector. The Bill also contains a range of provisions designed to ensure transparency and public accountability, including requirements related to publication and reporting. The removing of the requirement to submit the strategy statement to the Minister would weaken an important accountability mechanism without providing any corresponding governance benefit. Accordingly, I am satisfied that the existing provision is appropriate. I do not propose to accept the amendment.

Lynn Ruane (Independent)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 39:

In page 30, to delete lines 13 and 14 and substitute the following: “(3) A strategy statement shall be prepared in the form and manner that the Office sees fit.”.

Lynn Ruane (Independent)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 40:

In page 30, to delete lines 17 to 22 and substitute the following: “(5) The Commission shall as soon as may be after the preparation of a strategy statement under subsection (1) cause copies of it to be laid before each House of the Oireachtas.

(6) The Commission shall ensure that, as soon as practicable after copies of a strategy statement are laid before both Houses of the Oireachtas in accordance with subsection (4), the strategy statement is published in such manner as the Commission considers appropriate.”.

Lynn Ruane (Independent)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 41:

In page 30, to delete lines 38 to 40, and in page 31, to delete lines 1 and 2.

Maria Byrne (Fine Gael)
Link to this: Individually | In context | Oireachtas source

Does the Minister of State wish to comment on this amendment?

Niamh Smyth (Cavan-Monaghan, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

The provision secures an important accountability and oversight function in the financial management of the office. As a publicly funded statutory body, the office will be entrusted with significant public resources. It is appropriate that mechanisms are available to provide assurance regarding the proper management and use of those resources. The power provided for in this section is supplementary to and does not replace the normal audit and accounting requirements that apply to the office. Its purpose is to ensure that, where necessary, an examination of the office's books and records may be undertaken in the interest of transparency, accountability and sound financial governance. The existence of the powers does not imply any routine or ongoing interference in the management of the office. Rather, it provides a mechanism that may be used where circumstances warrant additional examination or assurance. Similar oversight and accountability provisions are found in the legislation governing a range of statutory bodies. Therefore, I am satisfied the provision represents an appropriate safeguard and forms part of the overall governance and accountability framework established by the Bill. Accordingly, I will not accept the amendment.

Lynn Ruane (Independent)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 42:

In page 31, between lines 2 and 3, to insert the following: “Annual report

40. (1) The Office shall not later than 6 months after the end of each financial year of the Office prepare a report on its activities during that year, and the Office shall as soon as may be after preparing the report, cause copies to be laid before each House of the Oireachtas.

(2) Notwithstanding subsection (1), if, but for this subsection, the first annual report would relate to a period of less than 6 months, the report shall relate to that period and to the year immediately following that period and shall be prepared, and the Office shall cause copies to be laid before each House of the Oireachtas as soon as may be, but not later than 6 months after the end of that year.

(3) A report under subsection (1) shall include information on the performance of the functions of the Office during the period to which the report relates and without prejudice to the generality of the foregoing, shall include such other information in such form as the Office thinks fit or the Minister may request.

(4) The Office shall ensure that, as soon as practicable after copies of an annual report are laid before both Houses of the Oireachtas in accordance with subsection (1), the annual report is published in such manner as the Office considers appropriate.”.

Niamh Smyth (Cavan-Monaghan, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

Amendment No. 42 would replace the existing annual reporting framework contained in the Bill and would remove the Minister from the process by which annual reports are laid before the Houses of the Oireachtas. The Bill adopts a standard governance model used across many statutory bodies, whereby annual reports are submitted to the responsible Minister, who then causes them to be laid before the Houses of the Oireachtas. This reflects the Minister's accountability to the Oireachtas for the legislative framework and governance arrangements applying to the office.

The existing provisions already ensure the activities of the office will be reported publicly and will be made available to both Houses of the Oireachtas. The Minister's role in this process is administrative and accountability based. It does not give the Minister any role in drafting, approving or altering the content of the annual report. The annual report remains a report of the office and is an important mechanism through which the office demonstrates transparency and accountability in the performance of its statutory functions.

The Bill already contains comprehensive requirements regarding reporting, publication and accountability, and I am satisfied these provisions provide an appropriate balance between operational independence and democratic oversight. Retaining a consistent reporting framework throughout the Bill also provides clarity and coherence in the governance arrangements applying to the office. Accordingly, I do not accept the amendment.

Maria Byrne (Fine Gael)
Link to this: Individually | In context | Oireachtas source

Amendments Nos. 43 to 45, inclusive, are related and may be taken together by agreement. Is that agreed? Agreed.

Lynn Ruane (Independent)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 43:

In page 32, line 3, after “a” to insert “publicly accessible”.

Amendment No. 43 relates to the AI register and seeks to make that register publicly accessible. The Minister of State has stated the AI register is not currently intended to operate as a public-facing register. She also stated the AI office may in time consider making elements publicly accessible in a way that protects confidentiality and commercial and data protection interests.

It is important to note what the AI register will contain. It will contain all incidents of prohibited AI practices under Article 5 of the AI Act; all serious incidents of high-risk AI systems in the State, reported under Article 73; high-risk AI systems that are employed in critical infrastructure; and any other AI-related incidents or notifications in the State required to be reported under the AI Act. We have subsequent amendments adding to what is contained in the register, including copies of fundamental rights impact assessments for every high-risk AI system deployed in a public body.

It is strongly in the public interest that the information contained in the register is publicly accessible. Commercial interests should not prevent the public knowing information about the AI register, such as incidents of prohibited AI practices and other serious incidents involving high-risk AI systems. Prohibited practices listed in Article 5 include facial, emotional and social recognition systems, as well as other biometric identification and categorisation systems, and other AI systems that deploy subliminal techniques as purposefully manipulating individuals or trying to otherwise exploit the vulnerabilities of certain groups.

The AI register will contain the details of incidents of those prohibited practices. What commercial interests should prevent people having sight of this prohibited use? It is the commercial interest of the companies involved in the development or deployment of these prohibited technologies. This certainly should not be put ahead of the need to give the public access to information, particularly in instances where these technologies are affecting people's lives and, in some instances, their liberties and freedoms. The Minister of State has said the AI register should be seen as complementing that wider transparency framework. It is difficult to understand how this could be true if the information contained is not accessible to the public.

Amendment No. 44 provides that the register shall contain copies of fundamental rights impact assessments for high-risk AI systems deployed by public bodies. This aims to give a statutory mechanism to Article 27, which states that prior to deploying a high-risk AI system, public body deployers should perform an assessment of the impact of fundamental rights the use of such systems may produce. The assessment should contain: the deployer's process in which the high-risk AI system should be used in line with its intended purpose; a description of the period and frequency with which each high-risk AI system will be used; the categories of people likely to be affected by its specific use; descriptions of implementation of human oversight measures; and the specific risks of harm and the measures that will be taken if those risks materialise.

The AI register seems like a sensible place to create a repository of the information and, again, it is highly in the public interest for this information to be publicly available. These high-risk systems include biometric identification systems that will be deployed for the purpose of law enforcement. This is incredibly dangerous. This is technology that has profound impacts on our freedoms and civil liberties, and we should have sight of the impact assessments completed before these systems are deployed.

It is also important to highlight the dangers of creating our own definitions for the deployment of high-risk AI systems, particularly relating to the Government's recent use of the novel term, "biometric analysis", as previously discussed in another piece of legislation. Despite the fact biometric analysis requires the use of biometric identification systems, it is unclear if the deployment of biometric analysis will have to undergo the same type of fundamental rights impact assessments that are required for defined biometric practices under the AI Act.

Niamh Smyth (Cavan-Monaghan, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

I thank Senators Ruane and Higgins for these amendments, as well as Senator Stephenson. I will take amendments Nos. 43, 44 and 45 together.

While the amendments are framed differently, they seek to achieve the same objective, namely, to require the AI register maintained by the office to be publicly accessible. Transparency is an important objective and one that is strongly reflected throughout the AI Act and this Bill. However, in considering these amendments, it is important to view the national arrangements in the context of the broader transparency framework already established at European level. In particular, Article 71 of the AI Act requires the establishment of a European database of certain high-risk AI systems. That database is intended to provide a significant level of public transparency, and will contain information that is publicly accessible, reachable and user-friendly.

Public authorities deploying certain categories of high-risk AI systems are also required under the AI Act to register those systems in the EU database, ensuring a substantial degree of transparency regarding the use of such systems. The AI register established under section 43 serves a different purpose. It is intended primarily as a regulatory and supervisory tool to assist the office in carrying out its oversight functions. It may contain information relating to prohibited AI practices, serious incidents, ongoing investigations, enforcement activities and other notifications received under the AI Act.

For that reason, information held on the register may include material that is commercially sensitive, security related, subject to confidentiality obligations, related to ongoing supervisory or enforcement activity, or may involve data protection considerations. While I support the objective of transparency, I do not consider it appropriate at this stage to place a statutory obligation on the office to make the entire register publicly accessible. Such a requirement could create difficulties for information required for careful assessment before publication, where disclosure could prejudice investigations, undermine supervisory activity, compromise security interests or affect legitimate commercial interests.

As the office becomes established and experienced in gaining and operating the regime, it may be appropriate to consider whether certain information contained in the register could be published in a manner that promotes transparency, while also protecting confidentiality, information, security considerations, commercial interests and personal data. I also do not consider it necessary to prescribe in primary legislation the specific means by which information would be published. The office should retain flexibility regarding how information is made available to the public, as technologies and publication practices evolve. Accordingly, I am satisfied the Bill strikes the appropriate balance between transparency, accountability and effective regulation.I do not propose to accept either amendment.

It is important to distinguish between transparency and publication. The Government supports transparency. However, a regulatory register may contain information relating to, as I said, investigations, commercially sensitive information and so on. The Bill seeks to strike a balance between public transparency and effective regulatory oversight.

On section 44, I thank the Senator for the amendment. Having transparency and accountability around deployment of high-risk AI systems by public bodies is a legitimate and shared objective. The Bill already pursues that directly. The AI Act establishes obligations in relation to fundamental rights impact assessments for specific categories of high-risk AI systems. Those obligations are directed towards ensuring that the deployer assesses and mitigates risks before deployment and that competent authorities can exercise appropriate oversight, where required. I refer to the requirement in Article 27 for a fundamental rights impact assessment for every high-risk AI system deployed by a public body prior to its first use. This obligation applies primarily to public bodies and private organisations providing public services and users of certain high-risk AI systems, identified in annexe III of the Act. The purpose of the assessment is to identify and evaluate any potential negative effects that the AI system could have on individuals' fundamental rights and freedoms before it is deployed.

The assessment must describe how the organisation intends to use the AI system and the processes in which it can be involved. It must also explain how often the system will be used and over what period. The organisation must identify the individuals or groups who may be affected by the system and consider any risks of harm that could arise from it. These risks should be assessed in light of information provided by the AI system's provider, including any known limitations, risks or conditions of use.

In addition, the organisation must explain what human oversight measures will be put in place to supervise AI systems and ensure that the human decision-makers can intervene, where necessary. It must also document the actions that will be taken if risks materialise, including governance arrangements, procedures for managing incidents and the mechanisms through which affected individuals can make complaints and seek redress.

Where an organisation has already completed a data protection impact assessment under GDPR or the data protection directive for law enforcement functions, the fundamental rights impact assessment does not replace that work. Instead, it should build upon and complement the existing data protection impact assessment, DPIA, ensuring that the broader fundamental rights considerations are also addressed. In practical terms, the provision of Article 27 of the AI Act requires organisations to think systematically about how high-risk AI systems may affect people, identify and mitigate potential risks, establish appropriate oversight and complaints mechanisms, document their findings and demonstrate compliance before the system is put into operation.

Last year, the Department of Public Expenditure, Infrastructure, Public Service Reform and Digitalisation published its guidelines for the responsible use of AI in the public service to provide practical information and resources for all public servants and Government officials on how to design, develop, deploy and maintain AI solutions responsibly. Additionally, under Article 71 of the AI Act, the European Commission, in collaboration with member states, is required to set up and maintain an accessible and public available EU database containing information concerning high-risk AI systems and the market surveillance authorities, MSAs, are obliged to report on serious incidents to the EU AI office and our own AI office.

As I mentioned, the AI Act and this Bill already contain a broader framework for the fundamental rights oversight through the designation of relevant fundamental rights bodies, the powers of market surveillance authorities, complaints mechanisms, incident reporting obligations and enforcement powers. In particular, section 43 establishes the AI register, maintained by the AI office, and subsection (2) sets out an exhaustive list of what it must contain. These are: prohibited practices under Article 5; serious incidents reported under Article 73; high-risk AI systems referred to in annexe III in accordance with Article 49(5); and any other AI-related incidents or notifications required to be reported under the regulation.

While I support the objective of ensuring that the office is visibly overseeing fundamental rights impact assessments carried out by public bodies deploying high-risk systems, I am satisfied that this objective is already met in the Bill. Section 60(2)(c), read with Article 27(3), already ensures that every such notification made by a public body deployer is sent to the office, through the relevant market surveillance authorities, and the existing paragraph (d) of section 43(2) already captures that material within the register. On this occasion, I oppose the amendment.

Alice-Mary Higgins (Independent)
Link to this: Individually | In context | Oireachtas source

I do not really see how section 62 satisfies our requirement in respect of fundamental rights impact assessments for high-risk AI systems because the section deals with the reporting of infringements and the protection of reporting persons. I am not sure if I perhaps misheard what the Minister of State was implying. The complaints mechanisms are set out at that point.

The fact is that Ireland's record on enforcement applied to large technology companies is relatively poor. This has been criticised and called out by others in Europe. It has led to situations where European data protection authorities have had to step in when they felt it was not being addressed properly in Ireland.

I am passionate about GDPR. There are key parallels and predecessors here that need to be looked at. What has led to actual rights and the actual enforcement are the really important citizens' rights, public rights and rights of the individual that were set out under GDPR. These were largely championed by individuals. It has been civil society groups and groups like Digital Rights Ireland. It has been those who have taken class action cases against big tech companies. It has been those who have highlighted and publicly scandalised their own experiences of the inadequacy and sometimes abusive practices they have encountered in their engagements with big tech companies. What has driven action in this area has been civil society and public champions.

Sadly, it is not the case that Ireland has, through its governance mechanisms, led the way in zealously delivering the highest possible standards and interpretations in respect of those regulations. Now we have the new AI regulations. Ireland is in a core position, yet again, because so many of the largest tech companies are located here. The public got on board. It was a glorious moment in terms of GDPR because there was a new recognition of rights. It was like this incredible push-back against the idea that people are just the subjects, the serfs or chattel, to be pushed through a commercial model. It said that people own their own data. The public embraced that and said what they were not happy with. They led in this regard. That is what gave meaning to GDPR. That is what led the position where, instead of having to opt out, people need to opt in to cookies. All of these were steps take by those who championed the issues. If we go into this new AI era, which is seeking to embed itself in everything and everywhere, where there are countless new factors even recognised as high risk and if we do not have transparency about the problems and issues and we do not have transparency as to how fundamental rights are being implemented and enforced and checked, it will not only be very worrying in terms of the outcomes for the public, but it will also be deeply disempowering for the public, who have been the actual watchdogs in terms of rights and standards.It is the public who started shouting about Grok. It is the public who led the way on opt-outs and opt-ins. It is the public who said certain kinds of data are personal, that they can see that it is personal and how it is being used. This has happened before. Now we have this new system, this register, and on the idea of it not being public-facing, the Minister of State said it might happen in time, but we are at a crunch point now. This is when the massive scale-up is happening and the gazillion new data centres, which, absurdly, are being allowed to use up all of our electricity and water to such a crazy degree, are happening. It is a time when Ireland is embedding itself in this with a huge vulnerability. This is a time when AI is at a make-or-break, with many people pointing to the emperor's new clothes factor in that it is not very financially secure, but we are looking at embedding it, when wild risks are being taken and when we have seen from some of the companies that are championing and leading on AI intensely anti-democratic language and direct statements of their intentions to interfere in democracy. The very minimum we can do is ensure the public gets to see what is happening.

I do not think it is okay to leave it on the basis of we will work with the industry for a while and, down the line, we will see if there are certain elements we might make accessible to the public in a way that protects confidentiality and commercial and data interests. Why are commercial interests being given such a high rating in this? There are protections on certain commercial interests but we can choose what we are going to prioritise here, and I say to the Minister, who is someone I really respect and who I wish was in a different brief, frankly, because I know she comes with a strong community and cultural background that is about society, that we need to not make this be a black box. We cannot have it that the AI register is just something that happens as a little conversation between the regulator-promoter and the companies. It is dangerous and anti-democratic. It is going to lead to regulation that does not work and it is going to lead to corporate capture to an even greater degree. When the scandals break, if it turns out that the red flags for those scandals were sitting inside this register and not being looked at or examined, that is going to be a reckoning as well.

I urge the Minister of State to say that this is public information and, even if she does not accept the full wording of our amendment that any of such information that there is would be publicly accessible, which is what I want to see, then at a minimum, that any information that is in the public interest should be made accessible. As we are having Committee and Report Stages rammed together, we cannot go into that nuance, but that would allow you to balance your commercial or other data protection interests while any such information that is going into this register that is considered to be in the public interest would be made publicly available. We have looked for full public accessibility. I urge the Minister of State, in her role, to think about even a compromise in terms of information that is in the public interest, because otherwise we are in a dangerous space.

The Minister of State said it is complementing a wider transparency framework, but this is where the core information is going to be. That piece about fundamental rights impact assessments at a minimum needs to be looked at. People's digital rights are one thing, as is the general data protection regulation, GDPR, but there are also those core fundamental rights and the human rights we have. Article 27 of the AI regulation refers to "deploying a high-risk system" and that public body deployers "shall perform an assessment of the impact on fundamental rights that the use of such system may produce". That is a process that has to happen in respect of fundamental rights. That is required under the regulation. Why would we not share that information? In what realm would we not share the fundamental rights impact assessment? If I said the information generally on the register that is in the public interest should be shared in the public interest, I cannot see any argument why the assessments done in relation to fundamental rights and their impact on them would not be in the public interest to share.

If we are going to say commercial sensitivity trumps clarity on the impact on fundamental rights, done in an objective way in respect of this assessment, then I do not accept that and I do not think the public will accept it. I am saying to the Minister of State to look to the register, look to making transparent and available not just the bits it wants to let her know about but anything that is in the public interest, and look to that question of the impact assessments on fundamental rights. These are public bodies that are meant to serve the public. They are meant to be doing this assessment for the public in relation to these systems and that information regarding the assessment of the impact on the fundamental rights of the public should be public. If companies have something to hide in that, then they need to change their practices. One of the first solid measures we can do to ensure companies start delivering on fundamental rights is that, if they know if they come out poorly from an assessment, everyone is going to see. Maybe that will get taken up legally or politically in Ireland but maybe it will get taken up in one of the many other countries where the companies headquartered here operate. It would allow individuals in France, Luxembourg and Sweden to point to a really problematic practice that one of these big companies has which was found in a fundamental rights impact assessment to be a problem for fundamental rights and maybe take it up locally. That is the duty of Ireland.

I have listened so many times to the people who want us to spend a bazillion on arms and all the rest of that hype and playing our part when it comes to cyberattacks, etc. This is it; this is the cybersecurity bit; it is here. The fundamental role of Ireland stepping up is not us buying a couple of tanks or joining in any wars of interest. Where we step up for our colleagues in Europe is by taking our role as the centrepoint for digital and AI regulation really seriously and dealing with it transparently.

I will leave that with the Minister of State. She said these Bills will be back. It is before the summer now. I genuinely appeal to her to show leadership on this not just for Ireland but for all of our colleagues across Europe.

Niamh Smyth (Cavan-Monaghan, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

I thank the Senator. I feel, as I am sure everyone in the room feels, the Senator's passion about this. I am of the same mindset and I know we have a real opportunity here. Unfortunately, I cannot accept the amendment because I have a deadline. I know the Senator will say deadlines are not good enough but I am concerned about the prohibitive practices piece and how AI is impacting all of society. I am equally concerned that the fundamental rights of people trump everything here. While I cannot accept the amendment, I will give the Senator a commitment that, following this, even over the summer duration or into the autumn when we are coming back to this, my door is always open. I am happy to sit down with the Senator and my very smart, bright and intelligent officials here to see what more we can do to copperfasten and give her and the public absolute confidence. The Senator is absolutely right. Our 27 member states need to be able to look to Ireland and see that these companies have reached the benchmark for empowering people and are not infringing their fundamental rights. While I agree with the Senator and on this occasion I cannot accept her amendment, my door will always be open.

Alice-Mary Higgins (Independent)
Link to this: Individually | In context | Oireachtas source

I thank the Minister of State.

Niamh Smyth (Cavan-Monaghan, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

I want to say one other thing. It is a correction. Maybe it was me; I am not sure. It relates to section 60(2)(c) rather than section 62(c). I may have said it wrong, or maybe the Senator did. I just wanted to clarify that.

Alice-Mary Higgins (Independent)
Link to this: Individually | In context | Oireachtas source

I thank the Minister of State for clarifying that and for her commitment to engagement.

Alice-Mary Higgins (Independent)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 44:

In page 32, between lines 11 and 12, to insert the following: “(d) a copy of the fundamental rights impact assessment for every high-risk AI system deployed by a public body;”.

Alice-Mary Higgins (Independent)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 45:

In page 32, between lines 18 and 19, to insert the following: “(5) The AI register shall be made publicly available via the Office’s website.”.

Alice-Mary Higgins (Independent)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 46:

In page 32, line 28, to delete “Minister” and substitute “Government”.

Alice-Mary Higgins (Independent)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 47:

In page 32, between lines 31 and 32, to insert the following: “Co-operation forum

45. (1) The Office shall establish a co-operation forum which shall include representatives of Oifig IS na hÉireann, Market Surveillance Authorities and Fundamental Human Rights Bodies, to meet a minimum of once per quarter and to discuss issues such as but not limited to—

(a) assignment of complex adjudications,

(b) co-operation strategies,

(c) means of information exchange, and

(d) other issues relevant to the enforcement of this Act, at the discretion of Oifig IS na hÉireann as the case may be experts may be summoned to give evidence regarding a matter before the Co-operation Forum.

(2) Where two or more Market Surveillance Authorities receive a complaint relating to the same potential breach of this Act they shall determine between themselves the lead Market Surveillance Authority who shall then appoint an authorised officer and shall be responsible for liaising with the other relevant Market Surveillance Authorities regarding the case.

(3) Should under subsection (2) the relevant Market Surveillance Authorities fail to agree the lead Market Surveillance Authority within 14 days the matter shall be referred to Oifig IS na hÉireann which shall appoint a lead Market Surveillance Authority.”.

This is an amendment put forward by Senator Stephenson. It is looking to establish a co-operation forum that would include representatives of the market surveillance authorities, the fundamental human rights bodies, and oifig IS na hÉireann, to meet a minimum of once per quarter to discuss issues including, but not limited to, the assignment of complex adjudications, co-operation strategies, means of information exchange and other issues relevant to the enforcement of this Act and, implicitly so, also to the enforcement of the regulation. It is at the discretion of oifig IS na hÉireann as the case may be in terms of experts who may be summoned to give evidence regarding a matter before the co-operation forum. Where two or more market surveillance authorities receive a complaint relating to the same potential breach of the Act, they will determine between themselves who the lead market surveillance authority will be. They will then appoint an authorised officer who will be responsible for liaising with the other relevant market surveillance authorities regarding the case. Under subsection (2), should the relevant market surveillance authorities fail to agree the lead market surveillance authority within 14 days, the matter will be referred to oifig IS na hÉireann, which will appoint a lead market surveillance authority.

The fundamental rights bodies have been inserted again into this. Crucially, the amendment seeks to ensure that adjudication of decisions, processing and enforcement are happening and that we do not end up with people being sent from pillar to post or, in the worst scenario, something akin to forum shopping from those who may be the subject of investigation, surveillance or adjudication, where they would be able to choose to go with this market surveillance authority rather than that one. It is crucial that there is a clear pathway that ensures that action is being taken, and that we do not have a number of different potential authorities wondering who will take the lead and how it will be led forward, ending up in a limbo. It is also important that those who are being adjudicated on do not get to choose who they want to do the adjudication because of a lack of a clear process. There must be a proper process whereby the regulatory system itself will determine who is going to lead on the adjudication, having the oifig as a fail-safe if that decision becomes difficult, along with the useful insight of fundamental human rights bodies such as IHREC and others, who may be there to have a perspective.

This is a very good-faith attempt by Senator Stephenson to ensure that we do not end up with breaches falling between stools and not being properly acted upon or ambiguity as to how they are being brought forward. Is the Minister of State interested in the co-operation forum the Senator has proposed in good faith, or an alternative mechanism to ensure that we do not end up with gaps or a situation where those making the breach can almost choose where they are going to be adjudicated from?

Niamh Smyth (Cavan-Monaghan, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

I thank Senator Higgins for speaking to this amendment and Senator Stephenson for putting it forward. I fully agree on the importance of co-operation and co-ordination between competent authorities in the implementation and enforcement of the AI Act. Indeed, one of the central purposes of the office is to facilitate co-operation, co-ordination and information sharing among the various authorities with responsibilities under the AI regulatory framework. However, I do not consider it necessary or appropriate to prescribe in primary legislation the detailed structures or operational arrangements proposed in the amendment.

The AI Act is a substantial and evolving regulatory framework and it is important that the office retains sufficient flexibility to develop co-operation mechanisms that respond to operational experiences and emerging needs. The amendment would establish a highly prescriptive statutory forum, including requirements relating to membership, meeting frequency, agenda items and dispute resolution processes between authorities. While these matters may be useful operational arrangements, I do not consider that they require specific provision within primary legislation. The Bill already provides the office with co-ordination and co-operation functions and enables it to support engagement between competent authorities and other relevant bodies. In practice, co-operation arrangements may need to evolve over time as the AI regulatory system develops and it is preferable the office has the flexibility to determine the most effective structures for achieving that objective. I am also conscious that aspects of the amendment overlap with co-operation obligations and institutional arrangements already provided for under the AI Act itself. Accordingly, I am satisfied that the existing provisions within the Bill provide an appropriate framework for co-operation and co-ordination and I do not propose to accept the amendment.

Pat Casey (Fianna Fail)
Link to this: Individually | In context | Oireachtas source

Amendments Nos. 48 to 54, inclusive, are related and may be discussed together by agreement. Is that agreed? Agreed. Is Senator Higgins moving Senator Stephenson's amendment No. 48?

Alice-Mary Higgins (Independent)
Link to this: Individually | In context | Oireachtas source

I am not moving it.

Alice-Mary Higgins (Independent)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 49:

In page 33, line 29, after “purposes” to insert “other than the personal data of a child,”.

Amendments Nos. 49 to 53, inclusive, seek to ensure that the personal data of a child shall not be collected for the purposes of developing, training and testing AI systems in a sandbox. Article 59 does allow for the training of AI systems in a sandbox, however, paragraph 3 of Article 59 also allows for member states to develop national law which excludes the processing of certain kinds of personal data. Those are choices that can be made at a national level. Our amendments seek to utilise this discretionary power to prevent the personal data of children from being used to develop AI systems in a sandbox. That is a very minimal piece.There has been a strong focus on the question of access to the Internet for children and all of that. We had that debate in 2017 and 2018 when the Data Protection Act was going through. This is an important point. At the time the Data Protection Act was going through, Senator Ruane and I put forward a new section, which, rather than focusing on the child and the child's access to the Internet, focused on the practices of the companies that operate on the Internet. We put forward an amendment at that time explicitly stating that the personal data of children should not be available to those companies for use for commercial purposes. That is basic. Here is the thing: that amendment passed. It went into the Bill. It was a part of the Bill but that section was never commenced. That is why I sometimes take it with a grain of salt when I hear people saying it is terrible where children and companies are concerned and the fact, for example, that Grok is available to children. It is not just the availability. We made this point back then, almost a decade ago. It is not solely about children accessing these services. It is the idea that children's data is used for commercial purposes. The specific requirement was that children's data could not be used commercially for micro-targeting and the development of profiles. We were told at the time there was not a definition of "micro-targeting", or some other excuse was given for non-commencement. I urge the Minister to revisit that issue. There is a cross-party mandate from the Oireachtas. The Seanad agreed and voted with us on that amendment. Government Members voted with us on it. There is not a monopoly on caring about that issue. It went into the Act but was not commenced for technical reasons, etc. Nothing happened then for years and years. There is now a high-profile thing about under 16-year-olds going on the Internet but the real problem is that companies are commercially using children's information. They create products. You can buy advertising that is targeted at children. You can sandbox products that are designed to commercially target children or use children's data in a commercial way. That is the problem. The business models of tech companies are based on the exploitation of children's data. That is the bigger problem. It is not whether children can access the hellish room where all the horrible stuff is happening. It is that there is a hellish room where horrible stuff is happening. That is the other fundamental problem. Part of the things happening in that room are based on the commercial exploitation of children's data, the building of profiles and the kinds of algorithms that target it. They should have been part of Commissioner McGrath's measures but were not sufficiently included in terms of tackling the issues of algorithms.

We now have another shot. We have another area. We have another sandbox. This is why I was concerned about the transparency in the register. When you create these boxes, there is concern around what is happening in them. The Minister has the opportunity and the right under the regulation to explicitly exclude children's data from what is going to be happening in the sandbox. This is to be strongly considered.

I will speak the rest of our amendments in this section. They seek to use the discretion to prevent the data of children being used to develop AI systems, including commercial AI systems, in a sandbox. It is the same thing we have been looking for since 2018 but now the dangers are even more acute.

Amendment No. 51 relates to the processing of repurposed personal data for the development, training and testing of certain AI systems in a sandbox environment. On personal data being given for a particular purpose, going back to that issue under the GDPR, you own your data. This new dilution, which we know is in the AI omnibus Act, makes a carve-out for AI training as a reason that personal data can be used. That should be hugely opposed. Personal data that was given for one reason can just be grabbed up and sucked up and used for the testing of certain AI systems in the sandbox environment. The AI regulatory sandboxes are controlled environments where developers and researchers test new AI systems under the direct supervision of the regulatory authorities. The Bill empowers the new AI office to establish the regulatory sandboxes in which new technologies can be tested in real-world scenarios. They are subject to oversight. The idea is that we are going to ensure the technologies comply with legal fundamental rights and ethical standards. The GDPR generally prohibits the use of personal data for a new, secondary purpose, unless that purpose is compatible with the original reason for collection. However, the AI Act is allowing, and we know the AI omnibus Act continues this, the repurposing of personal data in controlled regulatory requirements, subject to certain safeguards. In this context, the Bill proposes that personal data gathered for other purposes may be processed to support the development, training or testing of AI systems.

Our amendment seeks to introduce an additional privacy safeguard that will require that any repurposed personal data being processed in sandbox conditions must first be anonymised so it can no longer be linked to an identifiable individual. We know that data is considered anonymised when the data subject cannot be identified or is not identifiable having regard to all methods reasonably likely to be used by the data controller or other persons to identify the data subject directly or indirectly. This applies to direct or indirect identification. Carried out effectively, such anonymisation can assist in the privacy rights of individual data subjects. We must bear in mind that we also have our constitutional imperative on privacy here in Ireland. The anonymisation can allow the data controllers to achieve the balance between the right to privacy and other interests. This is an additional safeguard, but to be frank, it is a minimum safeguard because the core of the GDPR is not just your right to privacy. It is not a privacy piece. It is about your right to ownership of your data and your right to decide what it gets used for and does not get used for. There is often a misnomer when people talk about data protection. They say that the data is anonymised. The anonymity is an essential, and should be a safeguard in these sandboxes. That is a core consideration. You should not be able to recognise anyone from their data. As I say, there is a constitutional imperative, leaving aside anything else.

The other piece is that we should be with the spirit of the GDPR, which was a high watermark for Europe. We hear about innovation and competitiveness. The GDPR was one of the great fosterings of innovation. It is what forces good innovation so that the products and technologies fit with society's needs and rights. That is good innovation. It is not just about money maximising. That is not the only form of innovation. It is also about making things better. That is why strong regulations, such as the GDPR, fostered areas where if we were genuinely competitive and Europe was genuinely looking to be competitive - I am going to sidetrack for one moment on the competitiveness we hear about - we would be looking to have those high standards to set the benchmark for all the technologies that are operating within Europe, which is a huge market, so that then companies within Europe that meet those high standards will have a competitive advantage. I sometimes hear about the competitiveness agenda and then I see the Commissioners going into rooms with Exxon Mobil, Microsoft and all the rest. If it is about European competitiveness, why are they letting giant US tech companies and commercial entities set the agenda and get you to lower the bar so they have more access to European markets and European companies, which invest and do the work of trying to do good technology , are at a disadvantage? That is a side point on competitiveness.

There is a question of privacy. That is the basic that we are looking for here. When people's data is being repurposed, they should be informed of that and should be given a right to object. That is a stronger reading of the GDPR for which we have not even pushed in this amendment. I am just pointing out that is my view.Amendment No. 52 imposes an obligation on the AI office to consult with the Data Protection Commissioner before an AI regulatory sandbox is established, insofar as the sandbox involves issues related to personal data. This encourages early engagement on privacy and data protection matters, as opposed to addressing these issues only after the sandbox is operational and data has been repurposed and processed. We need to consult beforehand, not just after we have scraped the data and are already using it.

As drafted, the Bill empowers the Data Protection Commissioner in a supervisory capacity after the sandbox is operational. While I welcome the supervisory role for the commissioner, it should be there before the fact. The data impact assessment and all of that piece should be beforehand. The role of the Data Protection Commissioner should come in before the data is taken and processing has begun. The amendment would ensure privacy and data protection safeguards would be built into regulatory sandboxes at the development process, preventing privacy risks and rights risks being overlooked in the design stages. That would help build public confidence in what the Government is trying to do with sandboxes, that AI innovation and so forth.

Amendment No. 54 seeks to ensure any personal data lawfully collected for other purposes that is used in the training of an AI system would be subject to a data protection impact assessment, as I mentioned. The Minister of State has stated that the purpose of the Bill is to put in place governance structures necessary to implementing the AI Act. Amendment No. 54 seeks to ensure there is an appropriate governance structure. It is in line with what the AI Act has called for, namely effective mechanisms to identify if there are high risks to the rights and freedoms of data subjects. Doing the data impact assessment beforehand is what allows us to comply with the AI Act and the requirement to identify risks. We do not want to identify risks after the fact. The data protection impact assessment is the mechanism we have for that. These are good faith amendments to ensure the sandbox delivers what it is hoped it will deliver.

Niamh Smyth (Cavan-Monaghan, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

I thank the Senator for her comprehensive arguments on each of those amendments. As regards the use of personal data, including children's personal data, within the sandbox, I understand and support the underlying concerns motivating the amendment. Nobody in the House disputes that children's personal data warrants the highest standard of protection, including in the context of AI.

Article 57 of the AI Act requires member states to establish an AI regulatory sandbox. The purpose of the sandbox is to promote AI innovation, support the development of compliant AI systems and facilitate regulatory learning across the AI ecosystem. Article 57 also mandates that the national competent authorities must involve national data protection authorities in the establishment and operation of AI regulatory sandboxes.

Article 59 of the AI Act expands on the use of personal data in the sandbox framework. Section 47 of the Bill gives effect to Article 59, which permits personal data, lawfully collected for one purpose, to be further processed solely within an AI regulatory sandbox and for the purpose of developing, training and testing certain AI systems. Article 59 provides a robust safeguard in terms of processing sensitive personal data in the sandbox. Those conditions include, among others, that the AI system is being developed to safeguard a substantial public interest such as public health, safety or the environment, and that the processing of personal data would require consent from the data subject in certain circumstances.

Article 59 does not disapply the general data protection regulation; it operates within it. The GDPR provides that children merit specific protections because they may be less aware of the risks involved. That principle runs through the regulation. Where consent is the basis relied upon for an information society service, Article 8 of the GDPR requires parental consent below the relevant age threshold, which section 31 of the Data Protection Act 2018 sets at 16 in this jurisdiction. The Data Protection Commission's published guidance on children's data goes further, requiring controllers to treat the best interests of the child as a primary consideration, and takes the view that profiling or automated decision-making in respect of children will rarely be justifiable.

None of this is disturbed by section 47. A sandbox participant processing any personal data must identify a lawful basis, satisfy data minimisation and purpose limitation, and conduct a data protection impact assessment, where required, and is directly answerable to the Data Protection Commission, as provided in section 47(2) of the Bill.

The amendment, therefore, does not add a protection that is missing; it duplicates protections that already bind every controller in the State. While I share the underlying objective of ensuring children's data is never processed carelessly in an AI sandbox, I do not accept that the amendment is necessary to achieving that. Article 59 of the AI regulation, faithfully transposed by section 47, confines this process to a narrow, closed set of public interest purposes subject to isolation, no adverse decision, deletion and continuous monitoring conditions, with the Data Protection Commission statutorily embedded in oversight under section 47(2). The GDPR, including child-specific protections under Article 8 and Recital 38, and the enhanced domestic threshold in section 31 of the Data Protection Act 2018, continues to apply over and above all of that, undiminished.

The Department is required to consult with the Data Protection Commission pursuant to Article 36(4) of the GDPR and section 84(12) of the Data Protection Act 2018 with respect to the personal data provisions in the Bill. The Department has concluded that consultation and I am satisfied the personal data provisions in the Bill are well developed and in line with requirements set out under the GDPR.

Alice-Mary Higgins (Independent)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 50:

In page 33, between lines 31 and 32, to insert the following: “(2) No personal data for the purposes outlined in subsection (1) shall be used without the permission of the data subject.”.

Alice-Mary Higgins (Independent)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 51:

In page 33, between lines 31 and 32, to insert the following: “(2) Any personal data intended to be processed subject to subsection (1) shall first be effectively and sufficiently anonymised so as not to relate to an identifiable person.”.

Alice-Mary Higgins (Independent)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 52:

In page 33, between lines 34 and 35, to insert the following: “(a) consulted with prior to the establishment of the sandbox in relation to those aspects of the sandbox relevant to data protection,”.

Alice-Mary Higgins (Independent)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 53:

In page 33, after line 37, to insert the following: “(3) The personal data of a child shall not be collected or processed for the purpose of developing, training and testing certain AI systems in a sandbox.”.

Alice-Mary Higgins (Independent)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 54:

In page 33, after line 37, to insert the following: “(3) Personal data shall not be processed under this section without a Data Protection Impact Assessment having first been undertaken.”.

Pat Casey (Fianna Fail)
Link to this: Individually | In context | Oireachtas source

Amendments Nos. 55 to 60, inclusive, are related and may be discussed together by agreement. Is that agreed? Agreed.

Alice-Mary Higgins (Independent)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 55:

In page 34, line 18, after “60(4)” to insert the following: “and has clearly established how a risk management system referred to in Article 9 will be implemented”.

Section 48 outlines the provisions for the testing of high-risk AI systems in real-world conditions. We are out of the sandbox now and into real-world conditions. We oppose this section. We have also offered a number of significant amendments to it. As I will outline, the provisions in the section are highly concerning.

High-risk systems that might be permitted for testing in real-world conditions under this section include AI systems used in law enforcement, the administration of justice, biometrics and access to education and essential services like healthcare. Paragraph 7(a) of Annex III outlines high-risk AI systems that may be tested in relation to asylum, including polygraphs.What the section provides for is the approval of the testing of AI lie detectors in real-world conditions. This is one of the most discredited technologies, yet we are going to have AI operate it. It is an experimental mechanism that gets everything wrong all the time. AI hallucinates and has sycophantic tendencies. These are all things about AI that make it one of the most problematic and discredited mechanisms to operate in real-world conditions on vulnerable people, including asylum seekers. It is dystopian.

What this section provides for is the testing of these systems. This could have a massive impact on freedoms and civil liberties. "Real-world conditions" is a weird way of describing reality. It is not "real-world conditions". It is the real world, with testing on real people. It is important to be very clear that this is about testing not just any AI systems, but those identified as high-risk AI systems, on real people. We are back to the "move fast and break things" piece. They are moving fast, trying stuff out and breaking things, but in these cases, the things that may be broken are people, their lives and their life chances. That is the experimental space we are in. That is the space being facilitated by this section, and it is why we will be opposing it.

As well as opposing it, we have also proposed a few amendments to try to safeguard against the mass of potential damage of this kind of reckless provision. Again, this is where the tension comes in. Are we experimenting in these real-world conditions? Will that be done for the sake of AI innovation and to promote it? A lot of what we are doing that affects society seems not to be about AI helping us to do what we want to do as a society, but how society can help AI as an incredibly overleveraged new economic experiment, with the money flowing to a very few individuals, predominantly. It is about how society can help to make that work. That is the fear I have. It is real-life experimenting with high-risk AI systems, not just AI helping people to find their way or provide maps or food services. These are identified high-risk areas.

Amendments Nos. 55 and 56 seek to ameliorate the damage by requiring that the approval of any application for the testing of high-risk AI systems in real-world conditions would be subject to a clearly established risk management system, as referred to in Article 9 of the EU AI Act, being presented to the relevant market surveillance authority making the approval decision. It is the minimum that a risk management system would be presented and be part of the process, and that is what is envisaged in Article 9 of the EU law. Clearly establishing these risk management systems should facilitate the relevant market surveillance authorities in making better, safer and more informed decisions with regard to approving any high-risk AI system for training in real-world conditions, in the real world, on real people. The risk management measures outlined under Article 9 of the EU AI Act include detailing how identified risks can be eliminated or reduced, including detailing the risks that cannot be eliminated. This information will be extremely useful for the relevant market surveillance authorities when making an informed decision on permitting the testing of high-risk AI systems in real-world conditions. That is what we are trying to ensure as part of the process.

Amendment No. 58 is an important amendment that seeks to ensure that fundamental rights considerations are an explicit factor in decisions on the granting of permissions for the testing of high-risk AI systems in the real world, with particular reference to the rights of vulnerable persons. Under the Bill, providers must apply to relevant market surveillance authorities for permission to conduct testing of their high-risk AI systems. The Bill does not provide sufficient information as to what are the aspects of a system, or the potential impacts that the market surveillance authority will be considering when making the decisions. We know that they have to apply, but we do not really know what is getting properly considered or how it is getting considered.

The real-world use of experimental systems, or systems that are being tested, can expose individuals to very serious risks and harms before those systems have been demonstrated to be safe or, crucially, non-discriminatory. We know there is a very real situation of discrimination and prejudice with AI systems. The excellent research done by Abeba Birhane during her time in UCD, and in her work at MIT, has shown this. It is a known fact that there are huge issues of prejudice and discriminatory applications. This is an opportunity to try to comply with fundamental rights and privacy protections.

The amendment seeks to ensure that any decision to permit or facilitate testing is informed by an assessment of the potential impacts, shifting the focus beyond purely technical or commercial considerations, and requiring regulators to assess the human consequences of high-risk systems. The amendment specifically highlights the fundamental rights of vulnerable persons who are likely to be disproportionately affected by AI system errors, bias, discrimination or misuse, and, therefore, require special consideration. We use the term "vulnerable persons" because there is precedent for that term in Irish law. The intention is to capture within it the wide range that we know of. I have sat in on the AI committee, and I substitute regularly for Senator Ruane. We have heard from older people, younger people, people with disabilities, people with mental health difficulties, people with limited literacy or digital literacy, and people from marginalised communities and backgrounds. A lot of different vulnerable communities are being captured here. I point to the obligation to look at special measures for vulnerable persons that is in the original EU AI Act. We want to know how that is being applied to the crucial decision to let an AI model test itself on real people in the real world. I should say that within that "vulnerable persons" category are those from ethnic minority backgrounds who have been shown to have been mistreated and impacted.

With regard to testing in real-world conditions, there is talk of testing it on law enforcement, on the administration of justice, which could be on an asylum application, which is life or death for some people, on access to education, which is the absolute core foundation of people's life chances, and on essential services like healthcare, where there is vulnerability upon vulnerability. These are serious points. Everybody is vulnerable when they are seeking to engage with education, the justice system or healthcare. Then there are people who are even more vulnerable.

On amendment No. 59, witnesses and stakeholders who have spoken to the Joint Committee on Artificial Intelligence have expressed real concern as to how their data will be processed in the development of high-risk AI systems and whether their fundamental rights will be imperilled. This is a genuine concern on the ground from representative bodies of all kinds. They want the primary legislation to give effect to the EU AI Act, which acknowledges the need for fundamental rights and privacy, and speaks about vulnerable persons. Amendment No. 29 provides that in making a determination to permit the testing of a system in the real world, the market surveillance authorities shall have due regard to the potential impact of such testing on climate, biodiversity, environmental protection and our obligations under the Paris Agreement.

There is an intrinsic link between the use of AI and energy consumption. Ireland is an absolutely absurd outlier, with a quarter of our electricity going to this one industry. It is wild. We are far away from anything like what the norm is anywhere else in relation to this matter. There is the energy issue, but there is also the water issue.We have seen in the United States huge issues in relation to the environmental impact on water of these AI systems. When you test or train a system that is all energy usage and it is also water usage. I am sure the Minister of State is hearing from people who get frustrated when they are told to turn off their taps when they are brushing their teeth and then they see data centres down the road guzzling water. We know Uisce Éireann is not currently even documenting properly what is happening with water consumption.

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

Yes.

Alice-Mary Higgins (Independent)
Link to this: Individually | In context | Oireachtas source

The Dublin Inquirer has done incredibly important reporting on that. The national usage Uisce Éireann was suggesting is less than how much one individual data centre reported for itself so the figures are all up on this. These are factors. This relates to AI systems, high risk systems that could result in massive energy consumption. There is also the question of AI systems related to critical infrastructure in paragraph 2, which might involve the supply of water, gas, heating, electricity or the management or operation of what is called critical digital infrastructure. Paragraph 5 could include AI systems being given access to essential public and private services and infrastructure in their testing phase. It could include things like eligibility for healthcare services and categorising responding to emergency calls. These areas are really not areas for testing or experimenting.

The Paris Agreement and an environmental assessment in terms of doing things, is it worth doing it and is the cost worth what we get, needs to be part of the calculation, in general, in regard to AI, but particularly to the testing and experimenting related to AI in the real world.

I am coming to near the end. Amendment No. 60 seeks to provide for an ethical review of the testing of high-risk AI systems on an annual basis. There are obvious ethical concerns arising from this section. It is important we have a clear sight of the concerns and our amendment provides to undertake an annual ethical review of the operation of this section of the Bill. I do not think annual is too much in the context of an area that is so fact moving and when the risks are so clearly delineated. It reflects Article 60, paragraph 3 of the EU AI Act, which states:

The testing of high-risk AI systems in real world conditions under this Article shall be without prejudice to any ethical review that is required by Union or national law.

When the Minister of State states we are just implementing what is there, there are a lot of choices. There are choices around excluding vulnerable groups, like I mentioned earlier. There are choices around particular categories of data. There are choices around the ethical framework and I ask the Minister of State to choose to have an ethical review on an annual basis.

One of the listed functions of the office in the Bill is AI innovation. I am moving to a slightly different note. My last amendment, No. 63, is in this grouping. This amendment, No. 63, seeks to shorten the maximum timeframe within which a provider must comply with a request by a market surveillance authority to suspend, terminate or modify the testing of a high-risk AI system in the real world. As the Bill is drafted, providers are provided with a 14-day period within which the suspension, termination or modification of the testing should take effect. Bear in mind, that is 14 days after alarm bells have been raised, in which they can continue to experiment in the real world, on real people, on real infrastructure within real public service systems. When the market surveillance authority says "stop", they need to stop. They should not get two more weeks.

Article 76 of the AI Act sets out the powers of the market surveillance authority to ensure compliance and to say they can suspend or terminate testing if serious incidents occur or if the conditions of the testing permissions are not met by the provider. When those two things happen, the off-switch has to happen right away. This is a time when the State needs to move fast. It is positive that there is a mechanism to allow the suspension, termination and modification but we have put seven days in the amendment. I personally think it should be, not later than seven days. Two weeks is simply too long. Not only does it allow the damage to continue but it also creates the space for a spin to be generated around the damage, for the minimising of it and the potential removal of relevant evidence. It is not okay.

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

Yes.

Alice-Mary Higgins (Independent)
Link to this: Individually | In context | Oireachtas source

If somebody is already breaking the conditions, they should not get two weeks to cover their tracks. There should be a stop, and where necessary an investigation by the market surveillance authority. That concludes my amendments in this grouping and I will pass over to my colleague to speak to the remaining amendments.

Pat Casey (Fianna Fail)
Link to this: Individually | In context | Oireachtas source

Senator Stephenson has four amendments in this grouping.

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

Yes. I will speak to amendment No. 61, which is linked to what Senator Higgins was just referring to on the 14-day notice period for revocation or amendment of a real world testing permit for high risk systems. I will add to what Senator Higgins discussed. The idea there would be a 14-day window through which a harmful, high-risk system is allowed to continue implementing or using the practice that has been found to break some sort of regulation is nuts. The Grok AI scandal is a perfect, recent example of how the provider could maintain that system for a 14-day period even though we know it had harmful impacts on society. That is why my amendment No. 61 talks about how it "shall come into provisional effect immediately unless otherwise specified in the notice provided to the provider", so the provider will take the system off immediately, as soon as they possibly can, due to the severe risk they pose. A fair appeals process can still be included within that. The provider can appeal and if the appeal finds the testing is not damaging then the provider can reinstate that. If there are any concerns around any high-risk systems being implemented that have the potential to endanger society they should be immediately removed for the period during which that appeal takes place. The outcome of the appeal might that in fact it is a high-risk system and it should be taken down and therefore, we would have been correct in doing so immediately.

Amendment No. 57 provides that:

In making a determination on permitting or refusing to permit testing, the market surveillance authority shall have due regard for the potential impact of this testing on the fundamental rights of those exposed to the high risk system.

The "fundamental rights of those exposed to the high risk system" is really the main point here. We are legally and statutorily obligating companies that are engaging in high-risk systems while performing real world testing - us, existing in the real world and not in a safe sandbox, as I know has been discussed earlier - to consider fundamental rights. I suspect the Minister of State might say the fundamental rights of any individual will be protected through other instruments or statutes, perhaps through the EU AI Act itself, but given the potential risk to individuals - again I will use Grok since it was such a recent and major example - the potential for people's fundamental rights to be undermined with these high risk systems is endless. Therefore, putting that on a statutory footing really does matter. We can point to other directives or constitutional rights and all of these things but this is such new legislation that, by not including that we are inadvertently, or advertently, sending the message that fundamental rights are not as important as the testing of these products.

Niamh Smyth (Cavan-Monaghan, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

I thank both Senators for speaking to these amendments. I will cover amendments Nos. 55 to 65, inclusive, with the indulgence of the Acting Chairperson. Before addressing individual amendments, it may be helpful to briefly outline the framework that already applies in real world testing of high-risk AI systems. Under Article 60 of the AI Act, real world testing is not unrestricted. It is one of the most heavily regulated aspects of the AI Act. Testing may only take place where permission has been granted under the relevant market surveillance authority and where the detailed conditions set out in Article 60 are satisfied.

Those conditions already require, among other things: a documented testing plan; appropriate risk management measures; human oversight arrangements; safeguards for health, safety and fundamental rights; compliance with applicable union law; regulatory supervision and oversight; record keeping and documentation requirements; and powers for authorities to suspend, modify or terminate testing where concerns arise.

In addition, Articles 61 and 62 provide specific protections for natural persons participating in or affected by real world testing, including informed consent requirements and safeguards for participants. The AI Act therefore embeds risk management, participant protection and fundamental rights safeguards directly into the authorisation and operation of real world testing.Against that backdrop, I will address the individual amendments.

Amendment No. 55 would require an applicant seeking authorisation for real-world testing to clearly establish how the risk management system referred to in Article 9 will be implemented. I fully agree that effective risk management is a critical element of the AI Act framework. However, Article 60 already establishes a detailed and harmonised set of conditions governing real-world testing of high-risk AI systems. Those conditions must be assessed by relevant market surveillance authorities before authorisation may be granted. The amendment would introduce an additional national authorisation criterion beyond those contained in Article 60. Authorisation decisions must continue to be assessed against the harmonised criteria set out in the AI Act itself. Introducing additional national requirements risks creating uncertainty regarding the relationship between national law and the harmonised framework established by EU law. Accordingly, I cannot accept amendment No. 55.

Amendment No. 56 would require an application for real-world testing to be refused when the applicant has not clearly established how Article 9 risk management systems will be implemented. Again, I fully recognise the importance of risk management. However, Article 60 already sets out the conditions that must be met before testing may be authorised, including safeguards relating to risk management, oversight and participant protection. This amendment would, in effect, create a new mandatory ground for refusal beyond those provided for in the AI Act. Decisions to authorise or refuse testing should continue to be determined by reference to the criteria specified in Article 60. Accordingly, I cannot accept amendment No. 56.

Amendments Nos. 57 and 58 concern fundamental rights considerations. The amendments would require a market surveillance authority to have explicit regard to the impact of testing on fundamental rights and, in the case of amendment No. 58, the rights of vulnerable persons. I fully agree with the protection of fundamental rights as a central objective of the AI Act. However, these considerations are already embedded throughout that Act and specifically within the framework governing real-world testing. Article 60 requires testing to take place under a controlled and supervised framework, with appropriate safeguards in place. Article 60 forms part of a wider framework of safeguards within the AI Act. In particular, the regulation contains extensive provisions relating to risk management, human oversight, informed consent, participant protection and fundamental rights. Real-world testing under Article 60 must operate within the broader regulatory framework. Accordingly, market surveillance authorities are not assessing real-world testing in a legal vacuum. Article 60 already contains detailed safeguards that must be applied in the context of the wider protections contained throughout the AI Act, including those related to participant protection and fundamental rights. Introducing an additional national statutory requirement risks creating overlap and uncertainty as to whether a different threshold is intended to apply in Ireland from that provided for in the AI Act. I am satisfied that the protection of fundamental rights, including the protection of vulnerable persons, is already fully embedded within Article 60 of the framework. Accordingly, I cannot accept amendments Nos. 57 and 58.

Amendment No. 59 would require market surveillance authorities, when deciding whether to authorise real-world testing, to have regard to climate, environmental protection, biodiversity and obligations under the Paris Agreement. I fully recognise the importance of environmental protection, biodiversity and climate action. However, as I outlined earlier, section 48 gives effect to Article 60 of the AI Act, which establishes a harmonised framework for authorising real-world testing. The role of the market surveillance authority is to assess whether the conditions specified within Article 60 have been met. While the AI Act recognises environmental protection as a broader objective, Article 60 does not require separate assessments of climate and biodiversity impacts or compliance with the Paris Agreement when determining whether testing may proceed. This amendment, therefore, would introduce additional national authorisation criteria that do not form part of the harmonised EU framework. That could create uncertainty regarding the basis upon which authorisations should be granted or refused, the scope of the authority's assessment, how such impacts can be measured and how those considerations relate to the conditions already set out in Article 60. For those reasons, I cannot accept amendment No. 59.

Amendment No. 60 would require the Minister to undertake and publish an annual ethical review of the operation of section 48. Real-world testing should be undertaken responsibly and ethically. However, a separate statutory annual ethical review is not necessary. The ethical and fundamental rights considerations identified by the amendment are already integrated directly into the AI Act framework. Article 60 itself contains extensive safeguards, while Articles 61 and 62 provide additional protections relating to informed consent and participant protection. In addition, the AI Act contains formal review and elevation provisions at EU level. Under Article 112, the European Commission is required to assess the operation and effectiveness of the regulation and to report on its implementation. This provides an existing mechanism through which the functioning of the AI Act, including its real-world testing framework, can be reviewed and evaluated over time. The amendment also introduces the concept of ethical review without defining its scope, the standards that should be applied, who should conduct it or how it should interact with the existing regulatory framework. I consider that the existing real-world safeguards and oversight mechanisms provide meaningful protection. Accordingly, I cannot accept amendment No. 60.

Amendments Nos. 61, 62, 64 and 65 would introduce the concept of a proposal taking provisional effect immediately and then no longer coming into full effect. The existing provision sets out a clear statutory process and provides certainty as to when decisions take legal effect. Introducing concepts such as provisional effect and full effect risks creating uncertainty regarding the legal status of testing and the applicable safeguards at various stages of the process. The current framework already provides clarity, certainty and appropriate regulatory oversight. Accordingly, I cannot accept amendments Nos. 61, 62, 64 and 65.

Amendment No. 63 would reduce the current period of 14 working days to a period of no more than seven days. The purpose of section 49 is to balance regulatory intervention with fair procedures. The existing framework allows authorities to suspend, modify or terminate testing where concerns arise, while ensuring affected parties have an opportunity to make representations. The current 14-day period forms part of a coherent procedural framework and provides legal certainty for all parties involved. Replacing it with a shorter and less certain timeframe could reduce the opportunity for affected parties to respond, create uncertainty regarding implementation dates and be particularly challenging where complex, high-risk AI systems are involved. I consider that the 14-day period strikes an appropriate balance between responsiveness and procedural fairness. Therefore, I cannot accept amendment No. 63.

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 56:

In page 34, line 21, after "60(4)" to insert the following: "or has not clearly established how a risk management system referred to in Article 9 will be implemented".

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 57:

In page 34, between lines 23 and 24, to insert the following: "(5) In making a determination on permitting or refusing to permit testing, the market surveillance authority shall have due regard for the potential impact of this testing on the fundamental rights of those exposed to the high risk system.".

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 58:

In page 34, between lines 23 and 24, to insert the following: "(5) In making a determination under subsection (4), the market surveillance authority shall have due regard for the potential impact of this testing on the fundamental rights of those exposed to the high risk system, with particular regard to the fundamental rights of vulnerable persons.".

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 59:

In page 34, between lines 23 and 24, to insert the following: "(5) In making a determination under subsection (4), the market surveillance authority shall have due regard for the potential impact of this testing on—

(a) climate;

(b) environmental protection;

(c) biodiversity, and;

(d) our obligations under the Paris agreement.".

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 60:

In page 34, between lines 28 and 29, to insert the following: "(6) The Minister, in consultation with the Office, shall, within 12 months of the passing of this Act and annually thereafter, undertake and publish an ethical review of the operation of this section and, following such a review, may make regulations in relation to the ethical operation of this section.".

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 61:

In page 35, to delete lines 1 to 3 and substitute the following: "(c) inform the provider or prospective provider that the proposal shall come into provisional effect immediately unless otherwise specified in the notice provided to the provider or prospective provider, and".

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 62:

In page 35, line 12, after "into" to insert "full".

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 63:

In page 35, line 12, to delete "14 working days" and substitute "as soon as is practicable and no later than 7 days".

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 64:

In page 35, line 23, after “into” to insert “full”.

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 65:

In page 35, line 25, after “into” to insert “full”.

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 65a:

In page 37, between lines 19 and 20, to insert the following:

“CHAPTER 6

Policy coherence

Report on policy coherence

53. The Minister shall publish, within 12 months of the passing of this Act, a report on policy coherence by Government departments, on the implementation of the Artificial Intelligence Regulation.”.

This is around policy coherence and reporting of same. Amendment No.65a calls for a report that looks at the coherence in the implementation of the AI regulation across various Departments. As was discussed in relation to amendment No.1, the Minister for justice recently passed legislation permitting the use of biometric analysis by An Garda Síochána. The AI Act contains many safeguards regarding the use of biometric technology by law enforcement. However, the term “biometric analysis” does not feature in the AI Act. We have concerns that it may avoid the safeguards provided for in the regulation by not having that language within it. Using novel definitions of AI technology that are considered high risk in the AI Act is not consistent or coherent with the implementation of the AI regulation.

The implementation of the AI regulation will involve co-ordination across government. AI will affect many, if not all, parts of our lives and it will touch on the responsibilities of a wide range of Departments. There should be a reflection on how effective that co-ordination is to ensure that proper enforcement of safeguards contained in the AI Act. The idea that we have coherent policy across different Departments is fundamental.

When the Minster for justice, Deputy O’Callaghan, was in the Chamber passing legislation with definitions that diverge from the AI Act, he said:

When it comes to domestic Irish legislation we are perfectly entitled to use our own definitions and our own interpretations or interpretation and that is what has been done in this legislation.

The idea that we can choose to define words in any given way in our legislation is quite incoherent and inconsistent with the implementation of the AI regulation. There needs to be a concerted effort to promote that cohesion across different Departments. I hope the Minister of State will consider this amendment. It is simply about making the role she holds easier when coherence is put on a statutory footing.

Niamh Smyth (Cavan-Monaghan, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

I thank the Senator for her amendment. I recognise the objective behind the amendment but I cannot accept it. Under sections 38 and 40 of the Bill, the office is already required to prepare a strategy statement every three years setting out its key objectives as well as a review of its work, annual report and performance of its functions. Moreover, these sections are well developed and clearly specify who is consulted, the form and the manner in which the Minister may direct and what the statement or report may contain. The Bill also provides that both reports must be laid before Houses of the Oireachtas and published.

In addition, section 43 establishes a co-operation forum specifically to enable co-operation, co-ordination and information exchange among the competent authorities in order that there is consistent implementation of the AI act in the State by the competent authorities. Many of the regulators are long-standing market surveillance authorities, MSAs, in their sector and are accustomed to working together on matters that need consideration and co-operation between the relevant authorities. The AI office unit is working with those regulators to leverage their experience and existing practice to inform an approach in the context of the regulation of AI.

A new, one-off ministerial report on policy coherence, as proposed, alongside the well-developed provisions related to the offices accountability and functions in the Bill, which are already designed to support the coherent enforcement across the competent authorities, would not add any substantial value to what is already provided for in the Bill.

This amendment is also rather vague and lacks legal clarity on the meaning of policy coherence, the Departments it would cover, the criteria the report would assess and the words that would count as compliance. On this occasion, I have to oppose the amendment.

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

Can I confirm that we are speaking to amendments Nos. 65a, 68 and 69 as a grouping?

Maria Byrne (Fine Gael)
Link to this: Individually | In context | Oireachtas source

We are discussing amendment No. 65a.

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

That is fine. The screen is wrong. I was just checking against the screen, which states that amendment Nos. 68 and 69 are included.

Maria Byrne (Fine Gael)
Link to this: Individually | In context | Oireachtas source

Amendments Nos. 67 and 72 are related and may be discussed together by agreement. Is that agreed? Agreed.

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 67:

In page 38, between lines 13 and 14, to insert the following: “(3) A relevant market surveillance authority shall ensure that any affected person subject to a decision which is taken by the deployer on the basis of the output from a high-risk AI system, and which produces legal effects or similarly significantly affects that person in a way that they consider to have an adverse impact on their health, safety or fundamental rights shall be able to obtain from the deployer clear and meaningful explanations of the role of the AI system in the decision-making procedure and the main elements of the decision taken, as referred to in Article 86.”.

Amendment No. 67 is seeking to create a mechanism by which a person can secure their right to a meaningful explanation where they have been significantly affected by a decision taken on the basis of output from a high-risk AI system. We discussed this on Second Stage. I received the Minister of State’s letter in this regard. I would like to note that she wrote to me outlining some of the issues I raised on Second Stage, particularly with regard to a person’s right to an explanation as to why an AI system has made a decision about them. This is important because, as I mentioned in the previous debate, in the Netherlands, social payment decisions made by an AI system discriminated against certain groups in society. They did not receive their social welfare payment as a consequence of that decision taken by a non-human.

Incidentally, that is a great example of AI putting people out of work. If we automate decisions on social welfare payments - which is probably coming down the line for most Departments - we are putting those potential civil servants out of work. Often, when it comes to job losses as a result of AI, I hear the Government talk about job losses in the technology sector, but there will be job losses in every sector of society. For example, such losses will be seen among engineers because they will be able to do their drawings online. They will also relate to civil servants reviewing applications for social welfare payments. While that may not necessarily happen in Ireland, we have seen it happen elsewhere. It might be coming for us as well, depending on how Departments choose to hire and use automated systems. We are not taking the issue of job losses seriously.

To return to the amendment, in the Minister of State’s correspondence, she said that:

The purpose of this Bill, therefore, is not to recreate those substantive rights but to establish the national supervisory and enforcement framework through which compliance with those obligations can be monitored and enforced.

This is exactly what amendment No. 72 is seeking to achieve. Currently, there are no statutory provisions in the Bill that provide for the enforcement of a person’s right under Article 86. We have to allow people the right to get a meaningful explanation as to why an AI system has made a certain decision about them on an application or in any walk of life. Otherwise, there is absolutely no transparency. We know AI systems have in-built bias, depending on who originally wrote the code.

Similarly, amendment No. 72 seeks to provide for the right of a person affected by an automated decision-making system developed by a public body to human intervention. It would allow a person to receive a meaningful explanation and a formal review of the decision conducted by a natural person, that is, a human. When we are going to start seeing automated decision-making systems rolled out en masse – it will happen – it is crucial that there is the option for people to have human intervention. If anyone has ever operated within a system where they are speaking to a chatbot, whether it is for customer service or finding out why a decision has been made, they will personally understand how difficult it can be to speak to a human. If public bodies are using automated systems to make decisions on individuals, people should be entitled to human intervention in order to hear about the decision from a human, that is, a natural person.

Amendment No. 72 inserts a subsection in section 61 of the Bill, which relates to complaints concerning infringement of AI regulation. It also provides for the enforcement of a person’s right under Article 86.

Niamh Smyth (Cavan-Monaghan, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

I will respond to amendment Nos. 67 and 72. The rights set out in these amendments are already provided for directly by Article 86 of the AI Act. Article 86 is one of the principal individual rights created by the AI Act. It was introduced to address concerns that individuals may be significantly affected by decisions based on high-risk AI systems without understanding how those systems contribute to the outcome.The provision applies where a decision is taken on the basis of an output of a high-risk AI system listed in Annex III; the decision produces legal effects and similarly significant effects; or the person considers that the decision adversely affects their health, safety or their fundamental rights. The obligation falls on the deployer of the system and it is intended to provide meaningful transparency regarding the role played by AI in the decision-making process. Article 86 grants any affected person the right to obtain from a deployer clear and meaningful explanations on the role of a high-risk AI system in the decision-making process and maintain elements of the decision taken where the decision produces legal effects and similarly significant effects on that person.

As the right arises directly under the AI Act, it is not necessary to recreate or restate that right in national legislation. The Government has generally sought to avoid unnecessary duplication of directly applicable provisions of EU law throughout the Bill. Replicating Article 86 in domestic legislation could create uncertainty if the national texts were interpreted differently from the regulation or if future amendments to the regulation were not reflected in domestic provision.

In addition, the amendment would replace an obligation on the relevant market surveillance authorities to ensure that a person can obtain such explanations. However, Article 86 places the underlying obligation on the deployer, not on the market surveillance authority. The deployer is the entity responsible for providing explanations to affected persons, while market surveillance authorities have a distinct supervisory and enforcement role under the AI Act. The amendment therefore risks blurring the allocation of responsibilities established by the AI Act. It could be interpreted as imposing a new statutory duty on the market surveillance authorities beyond those contemplated by the regulation. I am aware that Article 86 was included in the AI Act specifically to strengthen transparency and accountability in relation to high-risk AI systems. It provides a direct right for affected persons and applies independently of the enforcement arrangements established by member states. The Bill does not remove or limit the right in any way.

Similarly, the objective underlying amendment No. 72 is already substantially addressed by the existing framework of the general data protection regulation, GDPR. The GDPR already contains protections in relation to certain forms of automated decision-making involving personal data, including safeguards related to human involvement and the ability to contest decisions in circumstances falling under Article 22 of the GDPR. These protections continue to apply alongside the AI Act where relevant.

Maria Byrne (Fine Gael)
Link to this: Individually | In context | Oireachtas source

Amendments Nos. 68 and 69 are related and may be discussed together by agreement. Is that agreed? Agreed.

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 68:

In page 38, between lines 18 and 19, to insert the following: “(2) Should a request be authorised under subsection (1), the completion of necessary conformity assessment procedures shall be undertaken without undue delay.”.

This section provides for derogations for conformity assessment procedures to take place. The article in the EU AI Act states that this is to provide time for the assessment to happen, so it should then happen as soon as possible. We need to include the wording "undue delay" because we do not want companies to wait for longer than is necessary. They could be operating systems that have not gone through conformity procedures and that could potentially be putting out damaging tech into our societies and communities without any oversight. Derogation should only be allowed for the shortest possible time period. The specific parameters of the AI Act should be transposed, which will not necessarily have happened. This is requesting that the wording of "undue delay" be added when it comes to conformity assessment procedures.

Amendment No. 69 is similar. Before high-risk AI systems can be put onto the market, they will be subject to strict conformity assessments. That is a good thing. They determine whether the system meets certain requirements of the AI Act, including in relation to risk management, data governance, technical documentation, record-keeping transparency and the provision of information, human oversight - which is crucial - and accuracy, robustness and cybersecurity, which is also crucial.

Derogations from the usual approval process for high-risk AI systems can be sought by providers in certain circumstances wherein a market surveillance authority can temporarily allow these high-risk systems to be used for a period while they are being assessed. According to the AI Act, derogations can be sought in certain justified circumstances, including for public safety and environmental protection reasons, but also in urgent situations where law enforcement or civil protection authorities might seek to use these systems without full approval. Given the potential risks posed to the fundamental rights and protections of citizens by high-risk AI systems, it is imperative that derogations are only granted in the most limited of circumstances and, where they are sought, that a full conformity assessment is undertaken without undue delay.

I recognise that there might be instances where high-risk systems have to be used without going through the proper conformity assessment, but with regard to that piece around undue delay, we do not want any legislative misunderstanding about the period in which that derogation can be allowed. Amendments Nos. 68 and 69 provide for conformity assessments to be undertaken without that delay after a derogation is granted. While recognising that there will be circumstances where a derogation may be needed, the assessment has to come as soon as is practically possible. This would ensure that high-risk systems that are operating subject to a derogation can ultimately be terminated or suspended if it is demonstrated that they do not meet the specified criteria, preventing temporary derogations from posing undue harm to members of the public because that is a risk. It is a risk that we might provide a derogation for a particularly high-risk system. If it has not gone through the conformity assessments, we do not know where the risk might be for society.

Maria Byrne (Fine Gael)
Link to this: Individually | In context | Oireachtas source

I acknowledge the students from the National Print Museum. They are guests of Deputy Bacik and are very welcome. I hope they enjoy their visit to Leinster House today.

Niamh Smyth (Cavan-Monaghan, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

I will respond in relation to amendments Nos. 68 and 69. The purpose of amendment No. 68 is to place an expressed statutory obligation on the provider to complete any outstanding conformity assessment procedures as quickly as possible following the granting of an authorisation. I fully agree with the objective that any necessary conformity assessment procedures should be completed as soon as practicable where an authorisation has been granted under this section. However, the provisions of the AI Act already establish detailed requirements regarding conformity assessment procedures and the obligations that apply to providers of AI systems.

Section 55 of the Bill gives effect to Article 46 of the EU AI Act which establishes a carefully calibrated mechanism by which, in limited and exceptional circumstances, a relevant market surveillance authority may authorise the placing on the market or putting into service specific high-risk systems in advance of completion of a conformity assessment. Article 46.1 makes clear that such derogations are only permitted in narrowly defined situations, including for reasons of public security, the protection of life and health, environmental protection or the protection of key industrial and infrastructural assets. It further provides that any such authorisation shall be time-limited and subject to appropriate conditions, including that necessary conformity assessment procedures are to be carried out. Importantly, the regulation already provides that the completion of those conformity assessment procedures must be undertaken without undue delay, taking into account the exceptional circumstances justifying the derogation.

Where an authorisation is granted under this section, the provider remains subject to all relevant obligations under the AI Act, including any requirements relating to conformity assessment and compliance. Introducing an additional requirement in the Bill that such procedures are completed without undue delay would not materially alter those obligations. The wording is also potentially open to interpretation and could create uncertainty as to whether a separate or additional legal standard is intended to apply beyond that already provided for under the AI Act. I am satisfied that the existing framework already ensures that providers are required to proceed with conformity assessment procedures in accordance with their obligations under EU law. Accordingly, I do not consider the amendment necessary and do not propose to accept it.

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 69:

In page 38, between lines 18 and 19, to insert the following: “(2) Should a request be authorised under subsection (1), a conformity assessment, in accordance with Article 43, shall be undertaken without undue delay.”.

Maria Byrne (Fine Gael)
Link to this: Individually | In context | Oireachtas source

Amendments Nos. 70 and 71 are related and may be discussed together by agreement. Is that agreed? Agreed.

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 70:

In page 40, between lines 35 and 36, to insert the following: “(3) Should the Office be the subject of the complaint in question, the Office shall ensure its assessment under subsection (2) is conducted by an independent person appointed by the Office.”.

As a regulator, complaints against the office should be dealt with independently. This section deals with complaints with market surveillance authorities. If the complaint is about the office, it should not investigate itself. That feels like a no-brainer. It aims to address any potential conflict of interest where the office is effectively subject to a complaint concerning an infringement of the AI regulation. While the AI office would ordinarily be responsible for assessing the complaints, if the complaint concerns the office's own actions or decisions, it would be put in a position of reviewing and assessing a complaint about the office itself. The amendment, therefore, proposes that in such circumstances, the assessment is carried out by an independent individual appointed by the board of the AI office as opposed to the office itself so that there is an appropriate gap between who is carrying out the assessment of a complaint and the subject of the complaint. Where the office is itself the subject of a complaint, we are of the view that it is incredibly inappropriate for the office to assess that complaint internally as this risks creating a conflict of interest. Requiring an independent person appointed by the board to conduct the assessment will strengthen confidence in the complaints process, ensure impartial decision-making and help maintain trust in the State's AI regulatory framework. I know that this came through in the Dáil as well and the Minister of State did not accept there. This is a good and sound amendment that will bring about independence, accountability and fairness within the system. We need the public to trust the AI office. The Minister of State would agree with that. There are serious concerns around independence, governance and the role the Minister plays in direction and so on. We spoke about it previously. We are talking about the need for an independent complaints process. The Minister of State could not investigate a complaint against her and I could not investigate a complaint against me so the office should not be able to investigate a complaint against it. It is a sound amendment and I really hope the Minister of State takes it on board.

Niamh Smyth (Cavan-Monaghan, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

I recognise the objective underlying amendment No. 71, namely, to ensure that complaints concerning the office are assessed in an independent and impartial manner. I fully agree with the importance of maintaining public confidence in the integrity of the complaints process. However, I do not consider that this amendment is necessary. The proposed amendment would introduce a specific statutory requirement that in a particular category of cases, an assessment must be conducted by an independent person appointed by the board. This represents a departure from the established approach, which is to ensure independence at an institutional level rather than through the creation of bespoke procedure arrangements for individual cases. As part of the setting up of the office, the complaints procedure will be put in place and aligned with the code of practice for the governance of State bodies and the Ombudsman guidance on complaints handling, as well as relevant legislation. As with other the statutory bodies, if individuals are not satisfied with the management of their complaint, they can make a complaint to the Ombudsman.

The objective underlying amendment No. 72 is already substantially addressed through an existing provision in EU law, including the AI Act and where personal data is involved, the general data protection regulation. In particular, Article 86 of the AI Act provides a right for affected persons to obtain clear and meaningful explanations of decisions taken on the basis of the output of certain high-risk AI systems where those decisions produce legal effects or significant effects that are considered by the individual to adversely affect his or her health, safety or fundamental rights. Furthermore, the GDPR already contains protections in relation to certain forms of automated decision-making involving personal data, including safeguards relating to human involvement and the ability to contest decisions in circumstances falling within Article 22 of the GDPR. These protections continue to apply alongside the AI Act where relevant.

I would be concerned that the amendment would create a separate national rights regime that differs from and potentially overlaps with the rights and obligations already established under directly applicable EU law. This could create legal uncertainty regarding the category of AI systems covered, the relationship between the proposed right and Article 86 of the AI Act, the relationship between the proposed right and existing GDPR protections and finally the scope of the term "essential public service provider", which is not defined in the Bill. The amendment would also apply to any automated decision-making system whereas the AI Act adopts a more targeted and carefully calibrated approach that focuses on specified categories of high-risk AI systems and decisions producing legal or similarly significant effects.

In addition, the Bill is primarily concerned with governance, supervision and enforcement of the AI Act. It is not intended to establish a parallel national framework of substantive individual rights beyond those provided for under the regulation itself. I consider that introducing additional rights of human intervention and review in this manner could create inconsistency between national law and a harmonised EU framework grammar.

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

Under the current process, if there is a complaint against the AI office, it will still investigate itself under the current legislation. Is that correct?

Niamh Smyth (Cavan-Monaghan, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

The answer is "No". The Government supports the principle that individuals could appropriately be protected where AI systems are used in decision-making, however, rights relating to explanation, transparency and human oversight are already addressed through the AI Act and, where applicable, the GDPR. The proposed amendment would create a parallel national regime and uncertain interaction with existing EU laws rights and would extend beyond the approach adopted by the AI Act. For those reasons, the amendment cannot be accepted.

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

For clarity, the AI office normally handles complaints about other organisations such as social media companies but were I to make a complaint about a decision made by the office, who would handle that complaint? What is the process behind that? The Minister of State may have said it and I might have missed it but in simple terms, who would be responsible under this legislation for that complaint?

Niamh Smyth (Cavan-Monaghan, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

It would be the relevant market surveillance authority and it would depend on the AI system or the sector. Therefore, the individual market surveillance authorities would take the Senator's complaint. I will come back to her with further detail but on this occasion, I cannot accept the amendment.

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

I understand that but the Bill states it is the office that will be handling that complaint. I believe the Bill states that this is the case. Certainly, my reading and interpretation of the Bill is that the AI office will be responsible for a complaint concerning its own actions. The Bill says the office will handle these complaints and that is where our concern lies. If that is the case, it is bad governance. If the Minister of State is saying that this is actively not the case, I will take that, go back and read the Bill. If it is the case, we might bring amendments later down the line in a new form of legislation.

Niamh Smyth (Cavan-Monaghan, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

To clarify, my own mumblings may not have been clear but the Bill actually says that the AI office will transmit that complaint to the relevant market surveillance authority.

Maria Byrne (Fine Gael)
Link to this: Individually | In context | Oireachtas source

How stands the amendment?

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

I will press it.

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 71:

In page 40, between lines 35 and 36, to insert the following: “(3) Should the Office be the subject of a complaint under this section, any assessment under subsection (2) shall be conducted by an independent person appointed by the Board.”.

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 72:

In page 41, between lines 16 and 17, to insert the following: “(6) A person affected by an automated decision-making system deployed by a public body or an essential public service provider shall have a right to human intervention, a meaningful explanation of the decision, and a formal review conducted by a natural person.”.

Joe Conway (Independent)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 73:

In page 41, between lines 20 and 21, to insert the following:

“Report on the protection of whistleblowers

63. The Minister shall within 12 months of the commencement of this Act lay a report detailing how whistleblowers who expose information related to artificial intelligence technologies can receive enhanced protections apart from what is listed in the Protected Disclosures (Amendment) Act 2022.”.

The amendment states that the Minister shall within 12 months of the commencement of this Act lay a report detailing how whistleblowers who expose information related to artificial intelligence technologies can receive enhanced protections apart from what is listed in the Protected Disclosures (Amendment) Act 2022.As I said at the outset, the amendment is largely self-explanatory. It centres around three items: duties of care, protection and transparency.

Niamh Smyth (Cavan-Monaghan, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

I thank Senator Conway very much. I appreciate the amendment. I recognise and share the Senator's underlying concern to ensure people who expose wrongdoing in AI systems are properly protected. I do not accept that this amendment is necessary to achieve that, though. Section 62 already applies the full Protected Disclosures Act 2014 framework, as strengthened by the 2022 Act, to the reporting of AI regulation infringements, in direct implementation of Article 87 of the regulation. That framework already offers broad personal scope, tiered reporting channels, a reverse burden of proof, interim relief, confidentiality and criminal sanctions for retaliation. In addition, Article 87 of the AI Act provides that Directive (EU) 2019/1937, the EU whistleblowing directive, applies to the reporting of infringements of the AI regulation and the protection of people who report them. I believe a bespoke AI-specific report on enhanced protections would not improve that. For that reason, I am not in a position to accept the amendment.

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 74:

In page 42, line 1, to delete "The Minister and any other relevant Minister" and substitute "The Government".

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 75:

In page 42, line 7, to delete "Minister and any other relevant Minister" and substitute "Government".

Maria Byrne (Fine Gael)
Link to this: Individually | In context | Oireachtas source

Amendment No. 76 is in the name of Senator Keogan. She is not present to move it.

Maria Byrne (Fine Gael)
Link to this: Individually | In context | Oireachtas source

Amendment No. 77, in the name of Senator Stephenson, has been ruled out of order. Would the Minister of State like to say something or is she okay?

Niamh Smyth (Cavan-Monaghan, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

I am okay.

Maria Byrne (Fine Gael)
Link to this: Individually | In context | Oireachtas source

Amendments Nos. 78 and 79, both in the names of Senators Higgins and Ruane, are related and may be discussed together, by agreement. Is that agreed? Agreed.

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 78:

In page 68, lines 27 to 29, to delete all words from and including "or" in line 27 down to and including line 29.

Part 6 of the Bill provides for a general administrative sanctions regime to be utilised by the applicable market surveillance authorities, allowing administrative fines to be imposed on those in breach of their obligations under the AI Act. Amendments Nos. 78 and 79 relate to the procedure following an adjudication of an infringement or breach of the AI regulation. In circumstances where an independent adjudicator finds a breach of the AI Act, this will include a finding as to the amount of the administrative fine that should be imposed and the date by which it must be paid. When an infringement is deemed by an independent adjudicator to have been committed, the relevant market surveillance authority is empowered under the Bill either to adopt the finding or adjudication, or indeed to decline it. Concern has been expressed by a large swathe of civil society representatives regarding the provision that empowers market surveillance authorities to decline to adopt a finding or adjudication under the section. We are proposing the deletion of the section as a consequence.

Allowing a market surveillance authority to decline to adopt an adjudicator's findings risks undermining the whole purpose of the adjudication process. If the authority can just say it is going to opt out, not going to acknowledge the finding or not going to proceed with or act on it, then what is the purpose of the adjudication process? What is the purpose of it if the authority is going to follow only those findings it accepts?

Public confidence and trust in regulation and oversight of AI systems are fundamental to this whole process. If suspected breaches have taken place and an adjudicator finds that there has been some sort of breach but an AI market surveillance authority can decline to adopt that finding, it is an issue in terms of the process but also in terms of how people perceive the AI regulation system and process and can have faith that they are working.

Niamh Smyth (Cavan-Monaghan, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

I oppose amendments Nos. 78 and 79. Section 102 establishes an important safeguard within the adjudication framework. While the adjudicator is responsible for independently determining the facts of the case and making findings under sections 99 and 100, the applicable market surveillance authority remains the statutory authority responsible for enforcement of the AI Act within the system. Accordingly, section 102(1)(b) permits the applicable market surveillance authority, in limited circumstances, to decline to adopt a finding that an infringement has occurred where it is not satisfied that a breach of the AI Act has been established.

I consider that the provision reflects the fact that the adjudicator's role is to make findings and recommendations, while the final regulatory decision remains with the statutory authority charged with enforcement responsibility. Similar decision-making structures exist in a number of Irish regulatory bodies where investigative, adjudicative and enforcement functions are separated. The provision, therefore, forms part of the overall constitutional and administrative architecture of Part 6. The amendment would remove that safeguard and would effectively make the adoption of an adjudicator's findings mandatory in every case where an infringement is found, irrespective of the views of the responsible market surveillance authority. The Government does not consider that such an approach would be appropriate.

As regards amendment No. 79, which proposes the deletion of section 102(4), I do not accept it. Section 102(4) provides the procedural consequences where the applicable market surveillance authority declines to adopt a finding under section 102(1)(b), or where the adjudicator finds that no infringement has occurred and that finding is adopted under section 102(2). In such circumstances, the authority is required to notify the adjudication subject and take no further action against the person in relation to the matter. This provision is an important element of legal certainty and fair procedures. It ensures that, where enforcement action is not being pursued, the person concerned receives formal notification of that fact and the matter is brought to a conclusion. If subsection (4) were removed, the Bill would no longer specify what happens where an adjudicator finds no infringement or where the authority declines to adopt a positive finding. That would create uncertainty for both the authority and the person concerned and would leave a significant procedural gap within the adjudication process.

Maria Byrne (Fine Gael)
Link to this: Individually | In context | Oireachtas source

Before calling Senator Stephenson, ba mhaith liom fáilte a chur roimh Katie agus Gerry Kelleher chuig an Seanad. Mr. and Mrs. Kelleher recently celebrated their 50th wedding anniversary, so congratulations to them. They are the parents of our colleague Senator Garret Kelleher and are very welcome here today.

I also welcome former Deputy Cathal Berry and his guests. May they enjoy their visit here today.

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

I am wondering about the guardrails that are in place if a market surveillance authority can just refuse to accept a finding. Are there criteria or grounds laid out under which it can refuse to accept that finding? If we are just saying it can refuse to accept a finding and, therefore, adopt that finding and act upon that finding within its own practices, it does not really strike me that we are holding these market surveillance authorities to account in any way. If we are just saying they can refuse to accept a finding by an adjudicator that they just do not like and does not work with what they are doing, I find that incredibly problematic. At the very least, if we had some sense of safeguards or guardrails in place like a series of grounds upon which it can do that, that would be something. I do not think it would be enough, but it would be something.

Niamh Smyth (Cavan-Monaghan, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

I thank the Senator. It may be argued that the provision allows a regulator to disregard the work of an independent adjudicator and creates unnecessary additional steps to the enforcement process. The provision does not permit arbitrary disregard of an adjudicator's work. Rather, it recognises the adjudicator and the market surveillance authority perform distinct functions within the statutory scheme. The adjudicator independently determines the facts and makes the findings while the authority remains responsible for the ultimate enforcement decision. The Bill intentionally separates those functions and section 102 gives effect to that structure.

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 79:

In page 69, to delete lines 9 to 14.

Maria Byrne (Fine Gael)
Link to this: Individually | In context | Oireachtas source

Amendments Nos. 80 to 82, inclusive, are in the name of Senator Keogan but she is not present to move her amendments.

Maria Byrne (Fine Gael)
Link to this: Individually | In context | Oireachtas source

Amendments Nos. 83 to 86, inclusive, are all in the name of Senator Keogan who is not present and, therefore, they cannot be moved.

Maria Byrne (Fine Gael)
Link to this: Individually | In context | Oireachtas source

Amendments Nos. 87 and 88 are in the name of Senator Keogan who is not present to move them.

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 89:

In page 75, line 4, to delete “Minister” and substitute “Government”.

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 90:

In page 75, line 17, to delete “Minister” and substitute “Government”.

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 91:

In page 76, line 36, to delete “Minister” and substitute “Government”.

Maria Byrne (Fine Gael)
Link to this: Individually | In context | Oireachtas source

I welcome the Ambassador of the State of Palestine to the Distinguished Visitors Gallery. I hope she enjoys her visit and business in Leinster House today.

Patricia Stephenson (Social Democrats)
Link to this: Individually | In context | Oireachtas source

I move amendment No. 92:

In page 96, line 26, after “oath” to insert “or affirmation”.

Maria Byrne (Fine Gael)
Link to this: Individually | In context | Oireachtas source

When is it proposed to take the next Stage?

Linda Nelson Murray (Fine Gael)
Link to this: Individually | In context | Oireachtas source

Now.

Maria Byrne (Fine Gael)
Link to this: Individually | In context | Oireachtas source

Is that agreed? Agreed.

Maria Byrne (Fine Gael)
Link to this: Individually | In context | Oireachtas source

When is it proposed to take the next Stage?

Linda Nelson Murray (Fine Gael)
Link to this: Individually | In context | Oireachtas source

Now.

Maria Byrne (Fine Gael)
Link to this: Individually | In context | Oireachtas source

Is that agreed? Agreed.

Mark Daly (Fianna Fail)
Link to this: Individually | In context | Oireachtas source

I welcome the men of Sagres, who are intrepid walkers. Tadhg O'Brien, Harry Canning, Pat McEvoy, Gabriel McHugh, Tom Cunnane and Brendan McNamee are guests of Senator Joe Conway. Apparently they have done the Camino a couple of times, so anyone looking for spiritual guidance will find them in the Visitors Gallery.