Malcolm Byrne (Wicklow-Wexford, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

The joint committee is conducting a number of hearings about how artificial intelligence impacts various aspects of public policy and will make recommendations to Government about the approach that Ireland should take. We are following a formula of going module by module. We are currently looking at issues around AI and defence, security and cybersecurity, and we have had a number of sessions. I welcome the witnesses who are joining us today. Apologies have been received from Senators Harmon, Dee Ryan and Ruane - Senator Alice Mary Higgins will substitute for her.

Members obviously know their constitutional requirements that they have to be physically present on the Leinster House complex to take part in our public meetings. Evidence given within the parliamentary precincts it is protected by absolute privilege, but members are expected not to abuse that privilege.

I am very happy that we joined online by Ms Bonnie Docherty, who is an expert in security and defence, a lecturer law at the international human rights clinic at Harvard Law School and a senior arms advisor with Human Rights Watch; and Ms Rosanna Fanni, a researcher, youth community advocate and expert working on the ethics of emerging technology, who has published extensively on this subject.

I invite Ms Docherty to make a brief opening statement.

Ms Bonnie Docherty:

I am grateful for the invitation to participate in this meeting. I am speaking on behalf of the international human rights clinic at Harvard Law School and on behalf of Human Rights Watch.

As artificial intelligence becomes commonplace on the battlefield, the prospect of machines making life and death determinations in armed conflict and law enforcement operations is rapidly moving towards reality. It is time for states to adopt a new treaty to prohibit and regulate autonomous weapons systems. Ireland has expressed support for such an instrument in multiple international forums, including the Convention on Conventional Weapons, the UN General Assembly’s First Committee on Disarmament and International Security and the UNGA’s informal consultations on autonomous weapons systems, which were held last year. We would welcome additional support at parliamentary level.

Autonomous weapons systems select and engage targets without human intervention. The systems apply force based on sensor processing rather than human inputs. They raise a host of legal, ethical, security and accountability concerns. Autonomous weapons systems would face obstacles complying with fundamental principles of international humanitarian law or the laws of war. They would find it challenging to distinguish between a combatant and a civilian because they could not interpret subtle behavioural cues, such as tone of voice or body language, as easily as humans who can relate to other people’s actions. It would be even more difficult for autonomous weapons systems to weigh the proportionality of an attack, such as whether civilian harm is excessive compared with military advantage, in quickly changing complex situations. Such a subjective assessment requires human judgment.

The use of autonomous weapons systems in peacetime law enforcement operations would contravene the right not to be arbitrarily deprived of life, which is the foundation of international human rights law. It would fail to meet the right’s three-part test that force be necessary, proportionate and a last resort.

From an ethical as well as a legal perspective, allowing machines to determine who lives and dies undermines human dignity, which says that every person has inherent worth that should be respected. Machines cannot understand the true value of a human life because they are not themselves living beings. In addition, they would instrumentalise and dehumanise their targets by relying on algorithms that reduce people to data points.

Autonomous weapons systems would threaten the principle of non-discrimination. As shown by other AI technology, algorithmic bias can disproportionately and negatively affect already marginalised groups and discriminate against people based on such categories as race, sex, or disability

All of these problems would be exacerbated because of a gap in accountability for any harm an autonomous weapons system caused. There are obstacles to holding individual operators criminally liable for the unpredictable actions of a machine they cannot understand. Legal challenges also exist to find programmers and developers responsible under civil law.

Finally, autonomous weapons systems would heighten security risks because they could lower the threshold to war and lead to an arms race. A legally binding instrument is needed to adequately address this plethora of problems. It should prohibit the autonomous weapons systems that inherently operate without “meaningful human control” and those that target people.

The treaty should regulate all other systems to ensure meaningful human control is always maintained over the use of force. The concept of meaningful human control encompasses criteria such as the ability to understand how a system works, to predict the outcome of its actions and to limit the time and space of its operations. More than 70 countries, including Ireland, have stated that it is time to begin negotiations of a new legally binding instrument on autonomous weapons systems.

We encourage Parliament to welcome Ireland’s leadership on this front and to press the Government for details on the concrete steps it will take to ensure negotiations take place. Given the speed of technological development and gravity of the threat these systems pose, moving diplomatic discussions to formal treaty talks is both urgent and essential. I thank the committee and look forward to answering questions as we move along.

Ms Rosanna Fanni:

I thank the distinguished Chair and members of the committee. I am very grateful for the opportunity to contribute to the committee's deliberations today in my personal capacity as a researcher and expert working on the ethics of emerging technology, and as a youth community advocate. My intervention focuses primarily on ethics of AI in the domains of defence and security, as well as its implications for development, deployment and governance.

Let me start with the concept of ethics. Ethics, as we all know, is not only an abstract concept, but also very concretely lived. Ethical norms guide our decisions every day and ask what is good for individuals and society. Therefore, ethics also sets limits. Ethics of AI defines what should not be delegated to machines regardless of what is technically possible. This is also reflected in international frameworks such as UNESCO’s 2021 Recommendation on the Ethics of Artificial Intelligence. Ethics is distinct from concepts such as safe, secure, responsible or trustworthy AI. The integration of AI into defence creates many new ethical questions. Allow me, for the purpose of this committee and its deliberations, to focus on two that may be most relevant to this committee.

First, I wish to focus on the operationalisation of human control. Increasingly, the use of AI delegates part of this control to machines, as we have heard from the previous witness. This fundamentally changes how decisions are made. AI systems are adaptive, probabilistic and, in many cases, not fully explainable. This creates a tension with core principles that have traditionally governed the use of force, such as accountability, proportionality and human judgment. A response to this challenge is the principle of meaningful human control, as articulated by Mariarosaria Taddeo and others. This means that humans must understand, supervise and ultimately remain responsible for AI-supported decisions and its outcomes. Research by Giancotti and myself illustrate this challenge in practice. Practitioners and experts are concerned that human oversight becomes blurry, particularly when AI systems operate at speed or scale beyond meaningful human intervention. If such uses are permitted, ensuring accountability and compliance with ethical and legal standards will be challenging.

The second concept I wish to focus on is the development of national approaches to appropriately govern AI in defence. International and regional processes are essential for building shared norms but they remain under development and uneven in their implementation. This increases the importance of states ensuring on a national level that ethical principles are operationalised within their own governance frameworks, civilian and military alike.

Importantly, as we heard from the previous witness, AI in defence is often in the form of autonomous weapons, colloquially known as “killer robots”. Beyond this well-established and important debate, AI facilitates many non-kinetic purposes and processes, including intelligence analysis, decision support systems and logistics operations. These uses still have a significant ethical impact and raise important governance considerations.

Data also matters. If data is not representative, systems can produce biased or incomplete outputs, particularly gendered imbalances. These issues can affect how threats are identified and how decisions are made. Ethical governance of data is, therefore, directly relevant to operational effectiveness in defence.

A national approach to ethical reflection for the use of AI in defence is critical in light of Ireland’s ongoing transformation of its Defence Forces, including integration of data, cyber and AI capabilities. Likewise, the trend towards cross-border, interoperable digital systems and intergovernmental co-operation may test Ireland’s approach to security, which has long been shaped by a commitment to military neutrality.

In conclusion, the key challenge is to ensure that ethical choices are clearly defined, reflected in governance and applied in practice. I hope these points may assist the committee in its consideration of this important issue. I look forward to answering any questions the members may have.

Malcolm Byrne (Wicklow-Wexford, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

Thank you. Members know there is a speaking order, and they have seven minutes for both questions and answers.

Paul Murphy (Dublin South West, Solidarity)
Link to this: Individually | In context | Oireachtas source

I thank the witnesses for the presentations. I will start with Ms Docherty. I will read a quote from senior officer B in the Israel Defense Forces, taken from an article in +972 Magazine. It states:

We didn’t know who the junior operatives were, because Israel didn’t track them routinely [before the war]. ... They wanted to allow us to attack [the junior operatives] automatically. That’s the Holy Grail. Once you go automatic, target generation goes crazy.

He is obviously speaking about these various AI systems, such as Lavender and Where's Daddy, that allowed automatic identification of alleged, so-called low-level Hamas members or other Palestinian groups, and apparently identified over 30,000 targets. The system allowed these targets to be attacked, even allowing for ten to 15 civilian casualties, and sometimes that was increased even more. Effectively, it seems the process was largely automated. Does that meet the definition of the fully autonomous “killer robots" that Ms Docherty is talking about?

Ms Bonnie Docherty:

It is a great question. It would not fit in the bucket of autonomous weapons systems that I was referring to because those systems would both select and engage a target without human intervention. In other words, in the Lavender and Where’s Daddy cases, theoretically, the system is identifying and choosing the targets, but a human has the final choice to use force. There is a question of whether there is a lot of automation bias, and how much input the human is actually adding to the targeting choices, but theoretically, the human is making the final decision. Therefore, they would not fall into the category of autonomous weapons systems that I was referring to.

Of course, we have a lot of concerns with the digital decision support systems that the Deputy referred to. However, that is for reasons other than those I was referring to, although there is some overlap, such as for reasons of accuracy, privacy in terms of data collection, whether they comply with international humanitarian law and so forth. The specific answer to the question is “No.”

Paul Murphy (Dublin South West, Solidarity)
Link to this: Individually | In context | Oireachtas source

That is helpful. In her opening statement, Ms Docherty makes the point that fully autonomous weapons contravene the core international humanitarian law principles of discrimination and proportionality. These are effectively the same core principles contravened by antipersonnel landmines, and were a driving force in their being banned.

In a sense, once a landmine is laid, it becomes fully autonomous. There is no more human intervention apart from someone walking where it is located. There is no conscious human intervention to trigger one. Is the legal rationale for banning fully autonomous weapons much the same as the argument for banning landmines, which is an argument I support?

Ms Bonnie Docherty:

Very much so. There has long been a concern in disarmament regarding weapons over which there is a lack of human control. Landmines are a perfect example. There is also concern about the lack of control over chemical weapons in that, once they are dispersed, you do not know where they are going to go. The autonomous weapons systems raise a similar concern. They operate a little differently from chemical weapons, for example, but the same principle applies in that once they are launched, there is a lack of human control. The law has been used to prohibit or regulate chemical weapons. The same guidance should be used here.

Paul Murphy (Dublin South West, Solidarity)
Link to this: Individually | In context | Oireachtas source

I thank Ms Docherty for that.

Ms Docherty spoke about the 70 states that have supported, at the UN, the banning of killer robots. That is welcome, but which states they are obviously matters. Do any of the major military powers, such as the US, Russia, China, the UK, France and Israel, support a ban on fully autonomous weapons? A number of them, such as Israel, China, Russia and the US, never signed up to the Ottawa treaty banning landmines. Smaller states that do not have military might are willing to sign up, but those that have the capacity to, or are most likely to, use such weapons do not. Is that currently the situation?

Ms Bonnie Docherty:

There are actually two numbers. I did not want to cause confusion in my opening statements but let me flesh things out. There are 130 states that have said they support a treaty banning prohibition and the regulation of autonomous weapons systems. Those include states such as France, which the Deputy mentioned, and Germany – some of the NATO powers. I would not go so far as to include Russia, Israel and China.

A text is being debated at the Convention on Certain Conventional Weapons forum and those states concerned are saying it should be used to initiate negotiations right now. I refer to a smaller number, but they are taking immediate action. The big powers have not joined in the call for immediate negotiations, which is disappointing. In the past, however, treaties have had influence, though not completely, over some states. This includes the US in respect of the mine ban treaty, even though it did not sign up to it. We still consider it an important step forward if we are to get new standards.

Paul Murphy (Dublin South West, Solidarity)
Link to this: Individually | In context | Oireachtas source

I have a final question. Ireland can potentially play a very important role in driving this internationally. Would it be helpful for Ireland to pass a national ban on killer robots to create an example and generate momentum towards having an international agreement?

Ms Bonnie Docherty:

That would be a critical step forward. National laws have in the past helped drive movements towards international bans through treaties, as happened with landmines and cluster munition. In the latter case, Belgium played a leading role. If Ireland were to show this kind of leadership, it would be a significant step forward. We would welcome that.

Paul Murphy (Dublin South West, Solidarity)
Link to this: Individually | In context | Oireachtas source

I thank Ms Docherty.

Johnny Mythen (Wexford, Sinn Fein)
Link to this: Individually | In context | Oireachtas source

On autonomous lethal weapons, do the guests have any faith in the international community? Do they believe Ireland, as a country that values neutrality, can play an international leadership role as a trusted diplomatic partner in supporting the responsible and ethical governance of artificial intelligence in defence forces internationally?

Malcolm Byrne (Wicklow-Wexford, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

Is the question for a particular witness?

Johnny Mythen (Wexford, Sinn Fein)
Link to this: Individually | In context | Oireachtas source

Both.

Malcolm Byrne (Wicklow-Wexford, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

Does Ms Fanni wish to start? Then Ms Docherty may contribute.

Ms Rosanna Fanni:

I would be happy to start. Would it be possible to hear the question again?

Johnny Mythen (Wexford, Sinn Fein)
Link to this: Individually | In context | Oireachtas source

Does Ms Fanni believe that Ireland, as a strong believer in neutrality, can play an international leadership role as a trusted diplomatic partner? How can it assist in the responsible and ethical governance of AI in international affairs and security?

Ms Rosanna Fanni:

I thank the Deputy for the question. As I mentioned in my opening statement, the international space is still being shaped and developed. I understand that one of the previous witnesses was Professor Barry O’Sullivan, who was on the committee on AI in the military. The committee recently issued a report that is key in looking more at the emerging international governance landscape when it comes to the codification of ethical, responsible AI governance in defence. This report mentions, for example, multi-stakeholder governance, basically entailing not only governance decisions involving relevant members of government but also those in industry, academia, civil society and so on. This is a key issue. I understand that Ireland is already considering these topics from a multi-stakeholder angle.

At the same time, the space in which norms or the governance process is emerging is definitely uneven in respect of implementation. As I also mentioned in my opening statement, national approaches in this regard are especially important, as the previous witness mentioned. National action often leads to a triggering process among certain states, which then advocate for coalitions and to proceeding on that basis. Any action at national level will definitely be very instrumental in driving a more important international process forward in this respect.

As rightly mentioned, Ireland has long maintained the principle of neutrality, which can create many opportunities to inspire other states to go ahead and ensure ethical governance and ethical principles are being maintained and even promoted further. Of course, this may also raise some questions around the interoperability of systems, especially now with regard to the European Union’s recent actions in the defence space. There is certainly weighing up to be done. It is important that any national governance takes these principles into the emerging governance space. Not everything is shaped yet, so governments that feel strongly about neutrality and also, importantly, examine issues of ethics will be important players in shaping the space internationally.

Johnny Mythen (Wexford, Sinn Fein)
Link to this: Individually | In context | Oireachtas source

I thank Ms Fanni.

Last week, we had Professor Barry O’Sullivan, an expert in his field, before us. He was speaking about trustworthy AI. He stated that in the domain of military defence and national security, AI provides limited protection, if any. Does Ms Fanni agree with that statement?

Ms Rosanna Fanni:

Protection in terms of?

Johnny Mythen (Wexford, Sinn Fein)
Link to this: Individually | In context | Oireachtas source

Military AI.

Ms Rosanna Fanni:

There are several elements to the question. What is key when we consider the discourse about accountability and transparency, and also non-discrimination, is meaningful human control, as was already mentioned.

This concept requires that any action, whether AI assisted or fully implemented by AI, has meaningful human control meaning that ultimately there is a person accountable for these actions. This is very important in terms of the oversight processes that may or may not be established through our governance procedures or processes. In this regard, it is also critically important also in terms of transparency and oversight to ensure that these processes are in place through, for example, the putting in place of ethics committees or review processes at the very beginning of the design stage of an algorithm and looking at the entire life cycle of AI systems. In this respect, the design stage is very important to ensure that clear accountability is already integrated at the beginning of this stage.

Another point that is also relevant to consider is the implementation of trustworthy systems that are guided by ethical principles but are also accountable in a certain way and that there are checks and balances in place to ensure that these systems ultimately do what they are supposed to do. There is the principle of non-discrimination when it comes to the integration of data and the verification of the data being used for these systems. This is also very important to ensure that ethical principles are being implemented and maintained throughout the life-cycle process.

Alice-Mary Higgins (Independent)
Link to this: Individually | In context | Oireachtas source

I will begin with Ms Docherty. I thank her for her presentation. Does she agree that we could add cluster munitions to the same category as landmines and other weapons that have an unforeseen effect? This is very relevant because Ireland hosted the negotiations on the ban on cluster munitions. Does Ms Docherty agree that this is one of those same pieces when the consequences of who these weapons will affect moves out of human control?

Ms Bonnie Docherty:

I very much agree. Cluster munitions are also a case where weapons cannot be controlled once they are launched at the time of attack because they spread over a broad area. There are also the after effects; they basically lie around like landmines. I have a soft spot for Ireland's contributions having been in Dublin during the time of the adoption of that treaty. I think the Senator made a contribution to that.

Alice-Mary Higgins (Independent)
Link to this: Individually | In context | Oireachtas source

I thank Ms Docherty. I was in Croke Park for those negotiations as well. It is something we should be proud of. Building on the questions, that is an example of where Ireland could be giving the same kind of leadership. I note, disappointingly, that when the Department appeared before the committee, its representatives did not seem to be ready to speak about Ireland's contribution on the autonomous weapons space. When they did engage previously, for example, around the UN General Assembly resolution, they talked about autonomy as a feature of a weapon's functionality that needed to be addressed rather than a specific weapon type. The International Committee of the Red Cross, ICRC, defines them as weapons systems designed to select or engage one or more targets without need for human intervention after activation. Is that a good definition or is there another definition Ms Docherty would point to?

Ms Bonnie Docherty:

We very much agree with the ICRC definition, as I think most states do. As far as I know, Ireland does as well but I cannot speak exactly to the Irish definition. Unlike cluster munitions, the idea is that we are not trying to pinpoint a specific system as much as a type of system where the concern is to maintain meaningful human control over the use of force. It is not a specific number of a system but a type of system.

Alice-Mary Higgins (Independent)
Link to this: Individually | In context | Oireachtas source

Going to a slightly wider piece and the test Ms Docherty mentioned, the idea of meaningful human control needs to include understanding the predicting of actions and the capacity to limit the time and space of the operations. Does that not go beyond prompts?

There is that idea suggested sometimes that there is a human input at some point saying, "We want you to identify this kind of thing", and that is the human input, but it has to be much wider in that it has to include that understandability and explainability.

Ms Bonnie Docherty:

Sorry, the line broke up. I did not hear what-----

Alice-Mary Higgins (Independent)
Link to this: Individually | In context | Oireachtas source

Ms Fanni might come in on this too. It is the idea of understanding, meaningful human control and predictability of the outcomes rather than simply the idea of a prompt input at some point. I will ask a couple of questions because I will run out of time. Will Ms Docherty comment a little more on that piece?

We discussed Lavender and Where's Daddy, which I think is particularly awful. Is there also need - separate from this specific one - for greater regulation in relation to those other parts of the AI technology that are involved in identification? Of course, this may not incorporate the human rights law considerations. Separate even to the ban, is that something that needs to be more regulated? I will ask Ms Fanni to come in on that. I include the dual-use technologies that are inputting into this piece, which we know may carry biases and inadequate consideration of international law. I ask the witnesses to talk about the need for regulation. As I understand it, there is a bit of a gap in properly targeting security and defence in terms of the EU AI Act.

I have one more question for Ms Fanni. She has done work in the past with the Italian military. It seems the Italian military has very significant concerns about AI and its uses and potential biases that may be reflected in how it may act. It is notable that the Italian military has also recently moved away from Microsoft for similar security reasons. Perhaps she could comment on that.

Malcolm Byrne (Wicklow-Wexford, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

You each have about one minute. Ms Docherty might start.

Ms Bonnie Docherty:

Yes, I can be quick. On the first question about meaningful human control, those criteria are designed to make sure that the operator can comply with legal and ethical rules because they know what the machine will do. That is the quick answer.

On Lavender and Where's Daddy, there is definitely a need for regulation. We see it as separate from the treaty I described because it is a slightly different system. There is a need for regulation to make sure that they also comply with human rights law. I will defer to Ms Fanni to make sure she has time to answer as well.

Ms Rosanna Fanni:

For accuracy and speed, I will focus on the research I have been conducting as part of my project for the Italian defence forces. This also touches on a very important point the Senator mentioned and Professor O'Sullivan also mentioned in his deliberations. The EU AI Act excludes any uses of AI for defence and security purposes, mainly because of the treaties and the non-responsibility of the EU to act upon these things. This also comes to the point I made about the importance of national approaches.

In the interviews we conducted with defence leaders, we found that they were not yet working so much with AI. The research was conducted in 2023, so there may have been other developments in that regard since then. However, there was definitely a very strong sense from the Italian leadership to integrate more ethical approaches and to look more at non-kinetic uses, which are anything that relates to AI use for systems operations, mobility or planning purposes and so on. It is about them being able to manage these systems but also to have a clear governance framework in place that guides these decisions and their defence, and also looks at very high-risk cases that are not covered by the EU AI Act.

Alice-Mary Higgins (Independent)
Link to this: Individually | In context | Oireachtas source

Is national action is needed to address the gap in this area?

Ms Rosanna Fanni:

Yes. That is what became very clear from our research.

James Geoghegan (Dublin Bay South, Fine Gael)
Link to this: Individually | In context | Oireachtas source

I thank the witnesses for coming before the Irish Parliament. It is greatly appreciated. Their contributions have been very valuable and interesting to hear. I will start with Ms Docherty.

In her opening remarks, she complimented Ireland on its role on a multilateral basis to try to progress the regulation of lethal autonomous weapon systems. In reply to a parliamentary question, the Minister mentioned that Ireland is active at the group of governmental experts on lethal autonomous weapons systems and that Ireland has also co-sponsored resolutions on this issue that have been adopted by the UN General Assembly’s First Committee, most recently in October 2025. The Minister also said that Ireland had expressed the view that autonomous weapons systems that operate outside of human control should be prohibited and other autonomous weapons systems should be regulated, given their unique characteristics.

Ms Docherty has an impressive history in the roles she played spearheading conventions on disarmaments, whether nuclear or other treaties. The world is struggling with a fear that the international rules-based order is not what it was, to put it in polite terms. The Prime Minister of Canada, Mark Carney, talked about a mutual interest world and countries clubbing together. Will Ms Docherty give us some optimism, based on her work over the years dealing with disarmament treaties and international law, that even in this world we now live in, her efforts to get an international treaty dealing with autonomous weapons will have some force of law and be taken up by the world, including by those states that might perpetrate and use those weapons? What sort of journey do we have to go on? Is there any hope for optimism?

Ms Bonnie Docherty:

It is definitely a challenging time but there is optimism. I have been doing this for 25 years. Disarmament is a slow process. It ebbs and flows. When I was in Afghanistan in 2001 looking at the use of cluster munitions, I thought we would never get a treaty. Seven years later, we were in Croke Park adopting the treaty banning cluster munitions. That was shortly after 9/11 during a tense security time from all sides. In some ways, the world came together but it was also a time when there were a lot of threats to international law. While they may not have been as severe as those today, there were a lot of threats to the international legal order. It was a moment we weathered and we came through stronger with new international law. Often, the times we weather lead to stronger international law, such as after the Second World War or the Vietnam war. The war in south-east Asia led to the Convention on Certain Conventional Weapons. These are times we can get through together, and that gives me hope.

The Treaty on the Prohibition of Nuclear Weapons came about similarly. After the Convention on Cluster Munitions, it was said we could never apply humanitarian disarmament to nuclear weapons. People laughed at us but ten years later, we were banning nuclear weapons. Again, it is not a panacea. Not every nuclear weapon state is on board but it is the next step forward. That can apply to this case as well. It is a slow process. It has been slowed by the challenges the Deputy mentioned but we can overcome them with patience. The fact that 130 states have expressed support for a new legal instrument is a good sign. We will succeed.

James Geoghegan (Dublin Bay South, Fine Gael)
Link to this: Individually | In context | Oireachtas source

Obviously, the political world and the military world are different. In this country, to be fair, we have a fairly limited knowledge of the defence industry, as some countries call it, because of our long-standing historical stance of neutrality. We are also very proud of our peacekeepers. Will Ms Docherty provide an insight as to what the generals in the United States or in the militaries around the world think? What are their thoughts on autonomous weapons and the risks they hold, as distinct from the leaders who might direct them to use such weapons?

Ms Bonnie Docherty:

Studies and surveys have been carried out that show that, in many cases, those in the active duty military have serious concerns about autonomous weapon systems.

Although I do not remember the exact numbers, in one such study, the majority of military respondents spoke against autonomous weapon systems for many of the reasons we spoke about earlier. They want to be able to control their weapons. They do not like the idea of sending out weapons they cannot control. In some ways, the political leaders are more eager to develop and use them than military leaders. Maybe that is not universal but the majority in these surveys spoke out in this way. These are same reasons that chemical weapons and landmines were banned. This military support will add to the case for restricting autonomous weapon systems in the long run.

James Geoghegan (Dublin Bay South, Fine Gael)
Link to this: Individually | In context | Oireachtas source

Do we have evidence yet of these types of weapons being used in what might be termed - I hate using this world but I hear defence people use it – the theatre of war? Have these weapons been deployed? If so, where have they been deployed and by whom?

Ms Bonnie Docherty:

Autonomy is a spectrum. It is always hard to determine exactly when we cross over the line to autonomous weapon systems. The Kargu-2 drone was used in Libya a few years ago. The UN described it as the first autonomous weapon system that was used. It is unknown or undocumented whether it killed anyone. Since then, there have been a lot of systems, such as loitering munitions and so forth, about which there is a question as to whether they count as autonomous weapon systems. Arguably, they have not been used except for the Kargu-2 drone. Even if they have been used, they are not being widely used in the kind of definition that I have given for this purpose. There is definitely a wide use of AI in weapon systems, however, which demonstrates the need for urgency to deal with this issue. It is hard to tell whether we have actually crossed that line. There is still plenty of time for action.

Sinéad Gibney (Dublin Rathdown, Social Democrats)
Link to this: Individually | In context | Oireachtas source

I thank the witnesses for being here. Their contributions have been interesting so far. I wish to ask for Ms Docherty’s thoughts on the current strikes in Iran, especially reports that Anthropic technology was used in target recommendations for a strike on a school in which 150 girls died. With a lot of my questions, I seek to drill further into the line between human and machine decision-making. We will start with that first question, however.

Ms Bonnie Docherty:

There were a lot of reports that Anthropic technology was used in that strike. Human Rights Watch published a detailed report on that strike. We were unable to determine whether AI was used in making the determination, so I do not have any comments in that regard. I cannot say for sure whether Claude was used in the decision-making. We instead focus on the harm done and the fact that it was an unlawful attack. I cannot say for sure what technology drove the decision-making behind it.

Sinéad Gibney (Dublin Rathdown, Social Democrats)
Link to this: Individually | In context | Oireachtas source

Will that be a symptom going forward? Will the difficulty in determining whether AI is deployed potentially prevent us from finding that accountability? Will that be an issue? Is it an issue beyond this specific attack?

Ms Bonnie Docherty:

It may be difficult for us to determine when AI is being used to decide an attack. There are a couple of challenges to think about in this regard. In the autonomous weapon systems I mentioned earlier, the issue was that the system selected and did the firing, which is not the allegation in the school attack. The allegation in this case is whether it chose the target. That is one issue to deal with, and that was not the allegation there.

The second allegation is whether autonomy was used to choose an attack, which could also be problematic. In those cases, it seems like the question is mainly about whether the attack violates existing international humanitarian law and whether the human is still choosing to rely on the AI. In that case, does the AI make the issue any worse? That is a question to which I do not have the answer. How much automation bias there is in relying on AI, if it is indeed being used, is something for people to consider.

That is something we will have to start thinking about. As I said, I do not have a clear answer on when it is being used, but it is something we will have to start thinking about.

Sinéad Gibney (Dublin Rathdown, Social Democrats)
Link to this: Individually | In context | Oireachtas source

Okay, I thank Ms Docherty. I want to compare the use of AI within a military setting with its use in a medical setting. In Irish law the Dunne principles, which were affirmed in 1985 and reaffirmed in our Supreme Court in 2020, state that if a medical practitioner takes actions outside normal medical practices they can be held responsible for any resulting failure and that applies now in a case where an AI analytic provides false advice or information. It essentially provides a legal backstop and access to justice for patients, who can have recourse to the medical practitioner, or the manufacturer if it involves a tool. Is there any precedent, to Ms Docherty's knowledge, in the law of any jurisdiction that could compare with or mimic that type of accountability with respect to the deployment of AI in a military setting? If not, is it a good idea to seek that?

Ms Bonnie Docherty:

There is none that I am aware of. Maybe Ms Fanni knows of something, though I do not want to put her on the spot. She does not. There is no accountability I know of and the accountability gap is one of the big concerns we have. I am not aware of any comparable programme. Obviously accountability is crucial and that could be a useful tool, especially for the decision support systems. There is also a question of whether they should be allowed in the first place.

Sinéad Gibney (Dublin Rathdown, Social Democrats)
Link to this: Individually | In context | Oireachtas source

I thank Ms Docherty. I have a broader question that is also for her. I hope to get a second round when I can turn to Ms Fanni. In Ms Docherty's work at Human Rights Watch she has reported on large-scale wars like the second Iraq war. In light of that experience, what changes now that AI is in the picture? I do not want to give her too many hypotheticals as I appreciate it is difficult but I am trying to tap into her experience of reporting on war and her observations on the deployment of AI by militaries. How is it going to change Human Rights Watch's role as academics in this space who we rely on for impartial reporting?

Ms Bonnie Docherty:

That is a great question. With the current AI being used, some of it has not changed. I have not been, but I could be doing the same kind of research looking at AI and the use of armed drones I did in Iraq in 2003 when we were on the ground looking at civilian casualties and trying to figure out how casualties are caused. In that case it was precision-guided munitions and in this case it is caused by armed drones using autonomy, but it is still a violation of international humanitarian law. We are just trying to find the cause of those violations in order to show all feasible precautions were not taken, soldiers were not distinguished from civilians and identifying the cause in the hope we could encourage that method of war not be used again. In that sense, the methodology is the same. There is new technology we need to identify, just like in Iraq where we identified new uses of technology - it was not autonomy - and were able to pinpoint that they were problematic. When we get autonomous weapons systems where they start selecting and engaging, that leads to another level of technology that would need to be identified. A fortunate challenge of trying to ban these weapons is there have not been victims yet. I hope it stays that way but it makes it more difficult from an advocacy perspective not to have those on-the-ground effects yet. The short answer is that I do not know how much difference it will make but it will make it more challenging to identify the problems because it is new technology we have not seen before.

Sinéad Gibney (Dublin Rathdown, Social Democrats)
Link to this: Individually | In context | Oireachtas source

I thank Ms Docherty.

Keira Keogh (Mayo, Fine Gael)
Link to this: Individually | In context | Oireachtas source

I am not sure what time zones our witnesses are in but they are very welcome. I am looking at Ms Docherty's opening statement and I 100% agree that we should not allow machines to determine who lives or dies. I fully agree that machines cannot understand the true value of a human life because they are not themselves living beings.

I wanted to start with the caveat that I agree with all of that. As a behavioural psychologist, I want to play devil's advocate, kind of. I will bring a totally different perspective, just to upset the conversation, by looking at what we have seen humans do across the ages in psychological tests but also in genocides and holocausts. According to reports from Vietnam, for example, we have seen that human judgment can be completely biased and fallible and in many ways can have catastrophic effects. I wonder whether there is any hope that if we build in safety by design, autonomous weapons taught to obey international law will in some cases be better than human judgment. It is a very devil's advocate kind of question but history has shown us that humans will follow orders in some desperate circumstances. It is a very out-there question, but why not?

Ms Bonnie Docherty:

It is not an out-there question; it is a very valid one. I have a couple of responses. It is true that autonomous weapon systems would lack emotion and therefore lack the anger and - I will not say evil - the things that drive people to commit genocide or act in anger and kill people unnecessarily. At the same time they would lack the compassion that humans also possess, or that some humans possess, at least, and that studies have shown can serve as a check on killing. I always feel the lack of emotions counterbalances the opposite case. As for them being taught to obey international law, there is the question of whether they can be, given that they lack human judgment and certain factors. I am not a roboticist but from what I hear from roboticists there are some serious challenges to that. Even if they were taught to obey international law, there are still other factors to deal with, such as the lack of accountability if there are gaps in compliance with international law. There are still the security risks in terms of upping the threshold to war. There remain the dignity concerns, which the Deputy is playing devil's advocate about so we can set those aside for the moment. One of my feelings, having dealt with a lot of different weapons systems, is that there is such a breadth of problems with these systems rather than just one main problem. Even if we could address one, there are still many others that will follow it.

Keira Keogh (Mayo, Fine Gael)
Link to this: Individually | In context | Oireachtas source

I thank Ms Docherty for coming back with that different perspective. I wanted to ask it in that other way because sometimes we can focus just on the negative. We sometimes flatter ourselves as humans but really we are behind so much of the devastation.

My next question is on AI in Ireland and our neutrality policy. In her opening statement Ms Fanni mentioned how we are already integrating data and AI here in Ireland. How should we move forward on adopting AI into our Army while taking into consideration our neutrality policy and her concerns about that?

Ms Rosanna Fanni:

I thank the Deputy. It is definitely a very relevant question. If I have a minute left at the end I might also say something on her previous question. Three things are really key in respect of her question. The first is doing something that it is being stated will be done. Principles may be established on a national level. Increasingly, states are moving towards establishing some sort of specific AI policy. For example, the Italian defence forces have just released an AI strategy for defence. This is always a starting point. If we start with governance that defines the different choices, options and permissible uses and under which circumstances they may be used, this is always a useful starting point.

Of course, it is not always easy to do, given the range of actors and different interests that are at stake at national level when it comes to determining such governance principles. It is also very important that this is followed through, for example, if there are ethical principles established. This can be through ethics committees or ethical impact committees, which can be consulted for specific users such as of AI systems. They could be consulted on the different implications, also with various degrees of impact or autonomy about the outcome of these decisions or processes they would assess.

Second, also crucial is capacity-building among staff. This has been already considered in various policy documents I have read in preparation for the meeting. This is absolutely critical so that the ethical consequences of the use of this technology and the potential risks are being outlined clearly to the people who would use them and also to those who are not using them. It is really critical. It is also about looking at more long-term implications and consequences as well as upskilling and reskilling of personnel with regard to the new technologies, and the links not only to AI but also to data systems and to the cyber digital environment that is really important when we look at AI technology.

The third element I would mention is the principle of meaningful human control and what this means in the context of integrating AI in national contexts. This is about ensuring that those people who use the technology are also ultimately being held accountable and that accountability is also being produced. All of this is already being integrated if there is a national strategy or at least principles for the governance of the use of this technology. This is an important base for a national level to move forward.

If I may add one thing in respect of autonomous weapons, the research they have been conducting in the Italian defence forces examines the concept of the moral distancing of something that is delegated to machines. If such a line of argumentation is going to be normal or acceptable - and this is not only related to defence but can relate to any use of AI in any sector - and the principle becomes more accepted that machines are taking decisions, we would be looking at a very different world than we have today. From the outset, it is a very important principle that humans take decisions and are held responsible for that. I consider that to be a very important principle that should be maintained.

Malcolm Byrne (Wicklow-Wexford, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

It now falls to me to thank our witnesses. I am very supportive of the idea of states committing to not using autonomous weapon systems. I want, though, to talk about how we address the challenge of non-state actors. We touched on it earlier. One of the challenges is with attribution, where states use proxies. We know that the technology is out there. In terms of the approach we need to take where non-state actors are going to avail of these weapons, what should we do?

Ms Bonnie Docherty:

That is a very important question to think about. There are a couple of responses. One is that it is all the more reason we need to get a treaty as soon as possible. We feel a treaty should address not just the use of these weapons but also their development and production. Obviously it is more challenging with these kinds of systems than with cluster munitions, for example. All our past treaties on humanitarian disarmament have also banned the development, production and use of whatever the system is. It is more challenging to limit the development of an AI system, because it is a dual-use system, than cluster munitions, for example. There have been past systems like chemical and biological weapons where we have tried to restrict dual-use systems.

Better minds than mine should be working on verification issues and ways to do that. The sooner we get in a prohibition on developing systems that are able to operate without meaningful human control, the better, to keep them out of non-state groups' hands. That is step one. Step two is an issue of transfer, making sure there are controls on transfers of these weapons systems and of the technology. That would be really important. I am not an arms trade expert but it would be about making sure there are end-use agreements and that we keep track of where the technology goes to, and keep it out of non-state armed groups' hands. Those are some measures states could take to prevent the technology getting in the wrong groups' hands. It is not a panacea and more needs to be done on this matter, but these are some things that could be done. Even cluster munitions, which are banned, have got into non-state armed groups' hands. Some things can be made very cheaply and so on, but there are measures that can be taken to reduce this problem.

Malcolm Byrne (Wicklow-Wexford, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

This is the challenge for defence and security forces here in Ireland where we have to make recommendations. With us building our own cyber resilience, what is Ms Docherty's recommendation to our own Defence Forces in terms of how to address these autonomous weapons systems? We can obviously campaign for a global ban, but if those killer robots, as they are referred to, or indeed autonomous cyberattacks are going to be hitting our systems, what do we need to do in terms of building our own defences?

Ms Bonnie Docherty:

I am not a cyber expert so I do not really have an expertise on how to defend against cyberattacks. I do not know if my colleague has any cyber expertise. I do not want to speak out of turn on that front.

Ms Rosanna Fanni:

I am not a cyber expert either. I can maybe comment a little more on the use of non-kinetic AI systems in defence, which, although maybe from a different point of view, also relies heavily on non-state actors, namely private companies. Those private companies may sometimes also have different interests or may not be 100% aligned with the national values, so to speak, especially with regard to Ireland's principle of neutrality. There could be a dependence on those companies to then procure or use AI systems because of the national capacity. Even in a regional setting, some European companies are increasingly developing AI but maybe not the systems that would be needed. That is also an aspect of reliance and capability. It is really important, if there is a shared responsibility, to look at the procurement aspect.

Malcolm Byrne (Wicklow-Wexford, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

Yes, it is companies but not just companies. I am also thinking about terrorist organisations or proxies of states that would be alien to our views. I might suggest that they might not follow normal procurement rules either.

Ms Rosanna Fanni:

Absolutely. I was not commenting on these in this case but also wanted to point out that there is a dependence on private entities that then may make decisions. Establishing procurement rules is really important to ensure the technology is being procured in line with the national values.

On non-state actors that have malicious intent, I am not sufficiently expert on that to comment. We know that AI definitely increases the scale and speed of such operations and of those actors, especially when we look at cyber operations and disinformation campaigns. AI really has become an enabler for those actors to execute their actions at scale and at speed.

I also point to the importance of national regulations and governance principles but also hard law, potentially, to at least ensure there is legal accountability, even though it is sometimes very difficult to trace back these actors and these actions. However, if there is no governance framework in place, it becomes difficult in the first place to ensure due diligence in legal terms when these malicious actors or actions happen.

Malcolm Byrne (Wicklow-Wexford, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

We will now have a second round of questions. There will be five minutes for questions and answers with colleagues.

Johnny Mythen (Wexford, Sinn Fein)
Link to this: Individually | In context | Oireachtas source

I will ask a question following on from the Chairman's analysis. Does Ms Docherty have any faith in the international community to find common ground on the limitations of the use of autonomous AI weapons?

Ms Bonnie Docherty:

I do. The numbers so far indicate that 130 states have indicated support both for a legally binding instrument on autonomous weapons systems and for the nature of the legally binding instrument or for prohibitions and regulations. They largely agree on what the content of that would be. Basically, they would have a working definition of what autonomous weapons should be, they support a prohibition on weapons that lack meaningful human control or whatever equivalent term they wish to use - they have different versions of that - and regulations to ensure meaningful human control, or some similar concept, over the use of force. There is widespread agreement on that. I believe they will reach common ground. The main hurdle right now is they are in a consensus-based forum where any one state can block progress. Right now, Russia seems to do most of that blocking. If they can get a new forum, they will have much more success.

Johnny Mythen (Wexford, Sinn Fein)
Link to this: Individually | In context | Oireachtas source

Ms Fanni's report on the Italian defence system stated that leaders put strong emphasis on human control and responsibilities. Yet, it came up a few times in the report that the lack of awareness and knowledge of ethical implications and ethical consequences among that group were very low. Will Ms Fanni elaborate on that? It is a very worrying situation.

Ms Rosanna Fanni:

I thank the Deputy for the question. There are several points that are relevant to consider. There was a big awareness inside the Italian defence system that ethics are key. This is also very important when you look at the Italian armed forces, and Ireland, being part of the EU and the shared understanding and commitment that there is a sense of responsibility towards the actions. Yet, as rightly stated, there is a lack, especially at leadership level, in how to operationalise these ethical principles or the normative moral standard into specific action steps. This is important to do on an individual level but it becomes even more important on a systems level. This is why we were very pleased to see that the Italian defence forces in February this year adopted an AI strategy that really looks at the system-level governance side to implement these ethical principles into specific action steps. This is very important.

I will flag another point that may be of interest to Ireland. It is this conflict that many leaders sensed with regard to the non-compliance of these ethical norms, especially in the defence context. The argument was made that while ethical principles are always the right thing to do and morally justifiable, sometimes against bad actors it is then very hard to take a defensive stance and actually conduct operations that may not 100% align. That is because of a need to defend and a need to use force against actors that do not comply with these ethical principles or are not really interested in complying with them in the first place. This is something that is always going to be a challenge in this space.

We also see the emergence of companies on the European level, for example, Helsing, which is a company that looks at defence with ethical principles and democratic principles behind it. The landscape is shifting, luckily, towards "How can we defend ourselves as a nation or as an entity?" but also to "How can we do that in a way that is compliant with our ethical principles and democratic values?"

Alice-Mary Higgins (Independent)
Link to this: Individually | In context | Oireachtas source

I will focus on the meaningful human control piece, beginning with Ms Fanni this time, on the idea that they understand, supervise and ultimately remain responsible. I worry that while you may have a prompt that initiates the kind of target to be generated, where the moment of the pressing of the button is still sitting with a person so it might not fall under the wide definition of target and engage, that target piece may end up being something that has not properly been examined.

Ms Fanni talked about moral distancing and Ms Docherty talked about the automation bias. Will they elaborate a little on those two? There is a danger that with the kind of language we see used, such as "reasonable grounds" or "reasonable basis", there would begin to be a kind of a defence of saying, "Well, these were the targets presented to me". There might be a need specifically to look at, for example, limiting the use of automated targeting in situations that can have lethal consequences, even if there is a human intervention coming at the end point. Will they comment on that?

This links into data governance, which was mentioned, and actual ethical data governance being one of our key pieces. Will Ms Fanni comment on that? Under the new AI omnibus, there is a proposal to create exemptions for AI training in relation to data protection. Does she have any concerns in relation to that?

Ms Rosanna Fanni:

I can start but my other witness may have very insightful comments on this. I am not an expert on autonomous weapons systems specifically. What I was referring to when I mentioned moral distancing is really something that we see more and more in the narrative and also in the literature that analyses the narratives. Now that AI is so widely used in defence, there is more and more acceptance of this principle of "We need the machines to do the operations that we want to do at scale", especially in actors that are actively engaged in conflict. This then leads to some sort of justification of these systems being used. Maybe I am repeating myself, but I again say that it is a deliberate decision to integrate AI systems and tools into certain processes. As the Senator rightly said, it is also a choice to allow these integrations into decisions that may determine life and death decisions.

As I mentioned, the UNESCO recommendation on the ethics of AI purely for civilian users also has this provision that AI should not be used for life and death decisions as it is inherently in conflict with ethical principles. What does this mean now for Ireland? Making these decisions deliberately and then including them in some sort of governance for legislation may be a first step towards deciding on a very conscious basis that these are the uses to be allowed while others are not. What is also very important in this discourse about meaningful human control is again the life cycle. It is not just, as the Senator rightly said, at the implementation step but at the design stage where a lot of data already comes in. I again thank the Senator for pointing this out. Data governance, ethical data governance, making sure that there is an inclusive and representative set of data, and that the data is tested, is very important, as is revisiting the dataset once the systems are being used so there is updating on the subject.

As we touched on the omnibus, looking at the AI Act and European Union legislation, which does not include the use of AI in defence and national security, the omnibus would not really have an impact on that. The data governance Act and other European legislation on data are quite strong in national comparison. It did not apply to defence.

Alice-Mary Higgins (Independent)
Link to this: Individually | In context | Oireachtas source

Will Ms Docherty comment on the matter of automation bias? Due to her experience on the international piece and the importance of having this disarmament measure, the bad actor frame can often be used to say that there must be exemptions to it. It is coming out at times of conflict and high tension where there are many actors who may not trust each other and that these kinds of disarmament initiatives are most important.

Malcolm Byrne (Wicklow-Wexford, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

I ask the witnesses to answer as succinctly as possible.

Alice-Mary Higgins (Independent)
Link to this: Individually | In context | Oireachtas source

It is also about the role that neutrality can have in that because-----

Malcolm Byrne (Wicklow-Wexford, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

It has to be answered inside 60 seconds.

Ms Bonnie Docherty:

Regarding automation bias, the key word in terms of meaningful human control is the word "meaningful". It is not just human control but that it is able to have an effect. I think of a control as being both the physical control, perhaps the ability to intervene once it happens, but also the mental control in terms of judgement and application of those kind of features. Once a machine gets the data, the person has the time to think about whether it is a good decision. On the automation bias, if the human is just doing what the machine says, that would not be very meaningful.

In relation to disarmament, we need it most when times are tough and there is an armed conflict. We cannot just make it during times of peace and then ignore it during times of war, as has been happening with the landmine treaty recently. It is important that we start working on this treaty now so that, as the conflicts go on and AI is being used more and more, we are prepared.

James Geoghegan (Dublin Bay South, Fine Gael)
Link to this: Individually | In context | Oireachtas source

I thank the witnesses very much for staying on. My questions are for Ms Docherty. Ireland is a small country. The last time we took up weapons against a state was to get a state out of our country. We do not have a history of military participation other than in peacekeeping missions. We do not have a very big defence industry. We have had historical underspending in defence generally. How can Ireland play a meaningful role in what the witnesses are trying to achieve in this issue of autonomous weapons? What is Ms Docherty's experience of small countries - maybe not just Ireland but other countries - advancing these issues diplomatically that they get copper-fastened into international law that is both respected and adhered to?

Ms Bonnie Docherty:

The whole idea of humanitarian disarmament, which is seeking to prevent and remediate arms-inflicted human suffering, is an idea that began with the landmine treaty in the 1990s and continued with the Convention on Cluster Munitions, which Ireland was deeply involved with, and then onto the nuclear ban treaty and the political declaration on explosive weapons that Ireland also played a lead role in. All these instruments were led by so-called middle- and small-powered states. I do not know if I necessarily like that term, but it is states with less of a military presence. Ireland has been a key player in all those instruments. It is not necessarily the US, China, Russia or the big military powers. In each case, they have proven to be instruments that have had a major effect legally in terms of setting high standards. Even for states that have not joined them, they have stigmatised the weapon and influenced states that have not joined the instruments. The Convention on Cluster Munitions brought together states that were major users, producers and stockpilers. In this case, Ireland can play the same role. It has been a leader in this field. I hope it will continue to be in the autonomous weapons system. I hope it will do so from here on out.

It is crucial that states like Ireland can play a role nationally and at the international level, such as persuading other states, to join to proposing initiatives, to hosting conferences, to pushing things at the General Assembly as well as those conventions on weapons. There are a whole range of activities that Ireland can take part in. I appreciate that the committee is taking the time to address this important issue. Those are some of the ways that Ireland can be involved.

James Geoghegan (Dublin Bay South, Fine Gael)
Link to this: Individually | In context | Oireachtas source

This is maybe a question for both witnesses. Ms Fanni mentioned that the AI Act does not deal in the area of defence for a simple reason that it is not within the competence of the European Union. As an institution, what role does the European Union have in what it is that both the witnesses are supportive of, advocating for and what Ireland is advocating for, which is a ban on autonomous AI weapons? What role does or can the EU as an institution play in this process?

Ms Rosanna Fanni:

I will give this question to my fellow witness.

Ms Bonnie Docherty:

In some ways, it is a similar role. The EU has also spoken out on these issues in the forums. I mentioned the UN General Assembly. On the Convention on Certain Conventional Weapons, I have heard it speak many times. The EU can set ethical and legal standards. I am not a EU expert, but I know it has set standards that can influence conduct just like national standards can. That helps build the case for new international law as well as starting to set norms that can influence behaviour in the meantime until we get new national standards.

James Geoghegan (Dublin Bay South, Fine Gael)
Link to this: Individually | In context | Oireachtas source

For the general public and policymakers, including myself, to properly understand and appreciate, when I was asking about examples of deployment of autonomous AI weapons, it would be fair to say that there is a narrative that seems correct that Ukraine has relied heavily on AI systems in the defence of its country from the war of aggression by Russia and its invasion, and that the use of AI in its defence systems has been critical in, I do not know about preventing additional fatalities because there has been so much senseless death in that war, but in terms of defending its country. Europe collectively and European countries are involved in that in the form of defence capabilities, supporting it and investing in it. Where is the line between the use of AI in defence and the ban that the organisation is pursuing at an international level of autonomous AI? Do the witnesses have observations on the use of AI in defence? Can it be a force for strength in a way that has proven itself successful for countries that are facing an aggressor nation? Alternatively, is there any example of the use of AI serving to reduce casualty deaths in war?

Malcolm Byrne (Wicklow-Wexford, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

I ask the witnesses to answer succinctly.

Ms Bonnie Docherty:

In the Ukraine case, I will distinguish. It is not using autonomous weapons systems. It is short of that in terms of just using armed drones. There is a distinction in the law between - the Deputy said "ban" - whether a state can, for example, in Ukraine's case, use force to defend itself and the conduct of the war in terms of what weapons it can use to defend itself. The issues we have been talking about today have been related to the latter, to how it conducts a war and what weapons it can use. The treaty we are talking about would relate to that, not to whether it has a right to defend itself. They are slightly separate issues. What we are talking about is whether AI should be used as a weapon of war, that is, how it can defend itself, not whether it can defend itself. I hope that helps a little bit.

Sinéad Gibney (Dublin Rathdown, Social Democrats)
Link to this: Individually | In context | Oireachtas source

I thank both witnesses.

My questions around the AI Act have been largely covered. I am not sure who will be best placed to answer this next question so both witnesses should feel free to contribute but it is on the EU migration and asylum pact. This was a very hard-wrestled and wrangled directive from the European Union, which states are in the process of transposing into law. The deadline was last April but there are increasing regulations being developed around it. Some commentators describe it as a direction of travel towards Fortress Europe, much more emphasis on detention centres and locking people out. I would certainly have concerns around the right to claim asylum and how that is being compromised through the pact. Is there any appetite to deploy automated systems in defending Europe's borders, in that kind of terminology, and essentially to further militarise anti-migration policies more broadly, regardless of AI?

Ms Bonnie Docherty:

That one is outside my experience. Hopefully, Ms Fanni can help.

Ms Rosanna Fanni:

I can start on EU-related issues. What the research has definitely shown is that there is an increase in spending by the EU on technologies, including AI technologies, that are being used for border management and migration control. This includes, for example, automatic scanning of faces, facial recognition, biometrics and so on. It also includes document checks that are being increasingly digitalised or automated and processed and in the data processing for these files that then get migrated across certain countries and institutions. Increasingly, the EU also looks at the interoperability of their systems in the management of the data for asylum visa processing. We do not have borders luckily but then the communication between these entities becomes important. What human rights groups have always argued for is that these checks are often not 100% accurate and that there is still, especially in terms of the data that is being used, not a 100% guarantee that everything was handled automatically. You also do not always have the same case with the human processing of such data but there is a lot about data bias, especially when you look at marginalised groups, that has not yet been worked on. I have not worked on these issues very recently but I know there has been a lot of criticism of the EU's approach. Again, as we have discussed here, it is about maybe linking back to the principle of meaningful human controls. AI can and will increasingly be used for these processes but, at the same, it is very important that there is always a human in the loop who reviews these decisions especially when they are critical decisions. The AI Act touches quite heavily on these users of AI as part of its high-risk category. This is also very important to look at.

Sinéad Gibney (Dublin Rathdown, Social Democrats)
Link to this: Individually | In context | Oireachtas source

I thank Ms Fanni. I have one final question. It is a psychology question, very different from Deputy Keogh's. Does the distance that is imposed between one human killing another human in a war situation affect things? We see a level of distance and barriers imposed through AI and the use of automated systems. Does that create a psychological barrier from guilt? Do we have research around that in terms of the deployment in combat situations?

Ms Bonnie Docherty:

Yes, it does. I do not have exact studies at my fingertips - Ms Fanni should chime in if she does but my understanding is that when drone operators have been interviewed they do feel that they have dehumanised their targets. They feel detached from their targets so it is easier for them to kill another human being when they are separated than when they are in more hand-to-hand or closer to the individual contact.

That said, there can still be examples of PTSD for drone operators. That has been an issue as well. My understanding is "Yes". I heard a study about that just the other day. I do not remember the details but that has been an issue and that does arguably make it easier for killing and, some people would say, easier to go to war, which could lead to an increase in violence which is one of the concerns with autonomous weapon systems.

Sinéad Gibney (Dublin Rathdown, Social Democrats)
Link to this: Individually | In context | Oireachtas source

I thank Ms Docherty. If she could share that research with us after the session that would be great.

Malcolm Byrne (Wicklow-Wexford, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

I thank the witnesses again. I want to follow on from Deputy Geoghegan's earlier question on Russia's illegal invasion of Ukraine. I would certainly put the argument, and there is a lot of justification, that it is only because of AI and technology that in many ways some of Russia's aggression has been held back in that Ukraine has been able to use technology in a way to defend itself. This includes the use of autonomous drones. Ground robots have been used in target detection. Again, we have looked at AI-enabled cyber resilience. I stress that there is always a human in the loop but clearly in these circumstances, Ukraine is using the technology in a way to be able to defend itself. The challenge then, which comes back to the EU's approach, is around setting those ground rules around when those technologies can be used. I appreciate we can learn from previous times but it comes back to the question of what a just AI war is. Do either of the witnesses want to take that?

Ms Bonnie Docherty:

I think what I meant by my response, which was a little quick, was that it is not so much whether they can use the tools or not; it is how they use the tools. They have a right to self-defence if they are being invaded, for example, and Human Rights Watch does not take a position on that or not so I cannot get into that comment, whatever my personal opinions. If a country is being invaded unlawfully and the country being invaded has a right to self-defence, it can use AI or whatever tools are legal to defend itself but it must use them in a legal way. For example, if armed drones are lawful and it can use them in a lawful way, that is perfectly fine, but it does not give them a right to use AI tools in an unlawful way. I would say it is not that a country has free rein to use artificial intelligence tools. It does not give it extra right to use AI tools because it is self-defence but it cannot use them in other ways. They have to still comply with the international laws of war in the same way it would comply with the laws of war if it was another type of war.

Ms Rosanna Fanni:

To complement that, it is a very good question that also touches back on the question of ethics. Ethics also looks at legitimate uses and legitimate purposes. It is exactly as Ms Docherty said at the start. There is a country that is being attacked and AI is used for primarily non-kinetic uses so not involving a use of force directly against other humans but for defensive purposes and for intelligence, analysis, surveillance and recognisance. These align very clearly with ethnical principles such as proportionality and protection of human civilians and so on. Using AI for these purposes is not only legitimate but just a natural evolution. Humans have always used technology for their own purposes so there is also the link to be made with ethics and ethical use of AI in this context.

Malcolm Byrne (Wicklow-Wexford, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

I totally agree with Ms Fanni. My concern is that had Russia had the possibility to be able to avail of some of these weapons, I am not sure it would have been too concerned about the ethical considerations given its record.

Finally, we are making recommendations to our Government and our own defence and security forces and Ms Fanni has the experience in Italy. Something that has been raised is co-operation at EU level and sharing information. What would her recommendations be as regards what we can we learn from Italy but equally at EU level in terms of co-operation?

Ms Rosanna Fanni:

This is a very important question. As we have stated in our research, co-operation is absolutely key. We see different national approaches emerging but they are all going in the same direction, that is, towards strengthening capabilities and using this technology for legitimate purposes. It is very important to share national experiences and strategies as well as approaches to oversight, upskilling and reskilling, and defence. I am not aware of any entities or fora for such formal exchanges but, based on our research, it is something that would clearly help us find a coherent European approach or different European approaches that all look to the principles of proportionality, the rule of law and democratic oversight of these systems. This is something that should be strongly encouraged. Notable countries that have spent much more on defence may also have approaches that could be tested or scrutinised. This is something that is definitely very important and would be encouraged from a research point of view.

Malcolm Byrne (Wicklow-Wexford, Fianna Fail)
Link to this: Individually | In context | Oireachtas source

I thank Ms Fanni and Ms Docherty for their testimony. It will certainly help us as regards our thinking here. I also thank members for all of their questions. It was a very useful discussion.