Professor Muiris MacCarthaigh:

I am speaking on behalf of Dr. Deepak Padmanabhan and myself, although I do not think that gives me eight minutes.

My first point concerns the general use of AI in government. With the widespread popularity and rapid penetration of AI into contemporary social and economic life, governments have come under significant public pressure to incorporate AI into their operations. This is on the basis that AI will bring efficiency and productivity to activities and bring new insights to bear on policy problems, while also enhancing the capacity for objective and data-driven decision-making. A report in September by the OECD on the use of AI by governments internationally identified many risks but also noted the risk of not engaging with AI to yield benefits and develop capacities.

Although there is greater recognition of the different types of AI, that is, symbolic, deep learning and generative, AI is primarily a data-driven technology that operates by distilling patterns from large historical datasets into decisions. This implicitly encodes a decision-making paradigm underpinned by historical records, one that seeks to ensure continuity and consistency. These attributes make it more appropriate for some policy sectors than others. In sectors with a volatile policy context, such as policing, trade or immigration, the logic of AI may not be as applicable when compared to sectors such as education, health or housing.

An uncritical or inappropriate adoption of AI across government may precipitate unintended, undesirable or even harmful consequences. The adoption of AI should be informed by a careful consideration of the trade-offs between reliance on historic decision-making patterns and sensitivity to the concrete circumstances around individual decisions.

On AI leading to changing modes of decision-making and accountability in government, major advances in the scale and pace of government operations during the 20th and into the 21st century were, in general, transparent and subject to public accountability mechanisms. AI stands in contrast to this as it offers recommendations based on complex data patterns, with the underpinning logic of the recommendation being more opaque than transparent. This presents obvious challenges for the role of public servants, especially as ever more consequential decisions are subject to AI-informed considerations. Public servants may find themselves having to explain why they do not agree with an AI recommendation and there is the danger that the pathway of least resistance is to uncritically accept AI-generated decisions.

It is important to recall that the design of AI has emerged from the corporate sector. With the market economy being driven by logics of utilitarianism, that is, the greatest good for the greatest number, rather than concerns about democratic inclusion, AI technologies embed the utilitarian calculus in their operation. For example, the training process of deep neural networks seeks to minimise average error, which is a paradigm that does nothing to prevent a small number of cases from being heavily disadvantaged.

The opacity of AI-based decision making also poses some fundamental problems for democratic accountability. The Government, and by extension public services, are accountable to Parliament and the people at large.

This encompasses the ability to ensure that each decision and the process of how the decision was arrived at hold up to public scrutiny. Different forms of AI present different challenges, but, essentially, there are issues concerning how information is generated and used.

As the committee will be well aware, ensuring effective regulation of AI is a challenge faced by most governments, not least because large technology companies have global presence which makes jurisdiction-specific regulation hard to institute and enforce. The EU has pioneered legislation, such as the GDPR and the AI Act, that has significant implications, but these remain only preliminary steps towards ensuring that AI serves the public good. Efforts towards AI regulation ought to include transparency mandates for usage within critical sectors so that key decisions remain amenable to scrutiny, challenge and rectification, as necessary. The influence that AI firms wield presents a lopsided power equation where citizens may feel caught in a binary, finding themselves caught between using AI subject to terms they may not find acceptable or not using it at all. Governments around the world have found it difficult to hold AI firms to account. Such is the pace of change that it has been a case of catching up rather than setting the regulatory frameworks and democratic guardrails within which AI must operate. We therefore really welcome the work of this committee and thank it for the opportunity to speak.