Dr. Dan McQuillan:

The discussion around such an industry rubric would come under small language models. There is a kind of narrative around that. The problem there is that it is not only about the data, but also about the mode of processing. As I alluded to at the beginning, the mode of processing does not have any factuality or causality in it. What you are really looking for in these things is not just a carefully curated dataset but a situation where essentially unreliable information is functional, which applies to a fairly narrow range of things. You are really looking for a situation where only the form of the output is important. It could be imagined that, in some very limited situations of language learning, a language model like that might be useful because it is more about imitation. However, for things that actually require novel output that can be relied on in the real world, there are no uses for it.