Professor Alan Smeaton:

I can take that one. A lot of the energy consumption in generative AI is divided into two things: the training of those models, which takes months and huge volumes of data, and then the inference, which is the processing of a request. These big tech companies are in a kind of prisoner's dilemma. They want bigger and bigger, and the concentration is on building bigger and bigger models. Since these models got so big, they would not fit on devices like our phones and the companies looked at ways in which the sizes of the models could be reduced. The techniques to do that included pruning - in other words, chopping out parts that are not needed - and a technique called quantisation, which is using just 16 b or 8 b instead of 64 b. Those topics are ongoing in research centres in Ireland as well as in those big tech companies.

On the inference side, the great aspect of the story of DeepSeek was that it introduced a clever algorithm to navigate through the model much faster than was done with the big tech companies in the US. The US big tech companies have an approach of just throwing more at it, building bigger data centres, using up more energy and so on. DeepSeek was a moment that showed that the projection of energy increasing and increasing could inflect downwards. Even more recently than DeepSeek, there is a variation of the algorithm in the transformer model called Mamba, which reduces the training time needed. While a lot of the projections that we all read about and accept because there is no alternative talk about the energy demands going up like that, we are already seeing signs of inflection points not only levelling but possibly even going down because these algorithmic changes and developments that Dr. Seoighe mentioned will change the perception of the energy demands for both the training and the inference costs.