We can know, thus we must know
On Pontus Stenetorp’s keynote at the HPC-AI Advisory Council UK 2026
Lastly, society, and this is a frustration on my part. We are being told a lot of things about these models right now: their capabilities, how good they are, and their potential consequences. Yet we are not allowed to see what went into them. […] If a plane crashes, there’s a commission that goes in and asks what went wrong. Sure, it might not reveal all the trade secrets. So why are we suddenly being willfully ignorant about something we are told is of a much larger magnitude? We don’t need to be willfully ignorant here. We can know. We must know, to quote Hilbert.
This is from the keynote that opened the HPC-AI Advisory Council UK conference this year. My live notes of it are a list of points in the order he made them. Watching it again, I noticed he uses two words for what bothers him. “Frustration” only comes up on this one slide, and it is about the companies. The word he keeps coming back to is “annoyed”, which he takes from Feynman, and it is about the science. I think the two together are what the talk is about: what annoys him are questions only pre-training can answer, and what frustrates him is everything that keeps academia from pre-training.
Annoyed
The science in UK-LLM starts from two things that have annoyed him for years. The first is the curse of multilinguality:
The curse of multilinguality says that if I have the option of training a model on, say, English only, or on English and French, the model trained on both will perform worse than the one trained on English. That’s annoying. Feynman said at some point that the natural state of a scientist is to be annoyed, because there are things you do not understand, and this is one of the questions that has annoyed me greatly for many years.
The second is that every model does better when asked in English, and “That includes Chinese, for the Chinese models”:
That’s also annoying, because it shouldn’t be the case. […] We have tried balancing the training data, and it doesn’t work. It’s not as simple as that: something else is causing this. Again, you’re annoyed, in the natural state of a scientist.
Their answer was to translate high-quality English training data into the other languages. He expected translationese to make the model worse, and guessed they would need about 90% real data. Training on translated data alone came within 3 or 4%, and with about 5% real data, their 1B model, TransWebLLM, beat their own 3B model, Gemma and Llama 3 on Welsh (Wang et al. 2025). In a preliminary experiment, adding French and German to English this way didn’t lower English performance either. He calls being wrong “an annoying outcome”, and owns up to it in front of his students.
None of this could have been found by fine-tuning someone else’s model:
If pre-training is off the table and we leave it to the corporate world, there are scientific questions we cannot answer.
We can know, thus we must know
He said it again in the questions, when a chemist asked how a scientist can trust a model whose training data is unknown:
One thing I’ve heard from the national security community, for example, is the fear that people plant false information and the model trains on it. Again: we can know, thus we must know. We simply need to draw a line and say: these are the areas where we require a fully transparent model.
His “must” is not Hilbert’s, which was a conviction that everything can be known.1 His is an obligation, because his “can” is down to earth: what went into these models, the training data, how the safeguards were instilled, even the model sizes, already exists inside the companies, so someone already knows. And we are told these models matter on a much larger magnitude than a plane crash, yet when a plane crashes, a commission goes in and asks what went wrong, and for these models no one does. So I read “can” as the reason for “must”. If the knowledge is there and we decide not to look, we are being willfully ignorant, in his words, and that is the status quo: the companies keep it as trade secrets, and the rest of us accept it.
That matters more the more we let these models do. As he told the chemist, “if a model is allowed to make even small-scale decisions, I think it is irresponsible for us to put these models in that position.”
I think this is the stance of science. Without it, it is no longer (open) science. Generalization is his example of why it matters:
It’s the difference between your friend acing an exam, and your friend acing it having seen the exam, or something very similar to it, beforehand. […] The level of generalisation these models achieve is unclear, and it has massive scientific consequences for our understanding, and, as I’ll come to, societal ones. If it generalises really well, we might have an intelligence explosion on our hands very soon. If it generalises very poorly, that’s still great, but it probably won’t happen.
So whether an intelligence explosion is coming is, in principle, something we can know, if we can see what went into the models.
Frustrated
Pre-training is how academia would find out for itself, and this is what’s in the way.
The companies know, and won’t say
And the companies have now moved into a stage where they are unwilling to share even minute details of their training. There are policies inside them right now: whenever you see a paper come out of, say, Google DeepMind, it has been vetted to make sure that nothing mentioned in it actually goes into production. There is no moat. It’s only trade secrets, and they’re very good at keeping those trade secrets right now.
That goes as far as model sizes: “even inside these organisations, they don’t tell you exactly how large the models are.” The Hugging Face incident in the opening quote is the case he points to: OpenAI’s agents broke into Hugging Face, and he asks why we are not allowed to know what the model was trained on, or how the safeguards were instilled.
Pre-training takes compute
When he started UK-LLM in January 2023, “the largest clusters in the UK available for artificial intelligence research were on the scale of 64 or 128 GPUs, shared between maybe ten groups. Not great. On top of that, those GPUs were about eight years old.” DiRAC and then Isambard-AI made it better, but not enough:
In Sofair we are blessed: we get 2 million GPU hours per year, guaranteed by the UK government. […] To put it in perspective, the whole of Isambard-AI, burning every second of the year, would give 46 million GPU hours. […] I estimate that the largest viable open source AI model with ethical training data requires about 20 million GPU hours, just to train it, with a little fudge factor. […] But you most likely want to train two per year, with a fudge factor, and you probably also want to explore further. That number is awfully close to me saying: I want the whole of Isambard for a whole year if you want me to get to the frontier. That’s not going to happen.
Industry won’t foot the bill (“Their shareholders would think they’re insane, and they probably would be”), so he thinks it has to be government. And the government’s next machine is for inference:
There’s a problem, though: that investment is earmarked as inference only, and I cannot for the life of me understand why. Inference is important, in particular for agentic models. But, as I’ve shown you, my question is: which models are we planning to run inference on? The PRC ones? […] I had a meeting with the government last week where I said: if there is no plan to build something akin to Isambard again within the next couple of years, I can move abroad, and shut down trying to do research in the UK. Isambard is not going to last forever.
This is the £750 million AI supercomputer in the UK AI Hardware Plan of June 2026, “reflecting the growing shift towards heterogeneous, inference-optimised systems”, and training isn’t mentioned anywhere in the plan. When I met him afterwards, he went further: any compute spent on inference on Isambard-AI is wasted, because inference can generally be done very cheaply and efficiently in the cloud.
The unit of innovation is too small
Even with the compute, academic AI isn’t organized to use it. This is how his talk opens:
Yes, it is two circles and a line. This was the unit of innovation in artificial intelligence for the longest time. One circle is the professor, the other a PhD student or perhaps a postdoc.
At one point, if you were interested in astronomy, you could be like Galileo: have your own little workshop, polish your lenses, observe the moons of Jupiter, and draw scientific conclusions from that. But at some point you needed to build an observatory in Chile, and that needed a different unit of innovation, with different levels of support. […] The kinds of questions you want to ask in science dictate your unit of innovation. This is a fundamental insight which I think a lot of the academic AI community still has not realised, but industry has.
He has done it in his own group. In 2023 he gave his students the choice between staying first author on their own papers and joining UK-LLM, where “you get a ton more compute and can answer different questions”, and most of them joined. The cost is in careers:
You will no longer have three ICLR papers accepted by the time you graduate; you might have zero. But maybe you contributed to something greater.
The rules favour training in secret
We have a major disadvantage in open source AI. OpenAI, we know, has trained illegally on copyrighted data since 2019. I cannot do that. So why should it be an advantage to stay closed and hide things, as opposed to operating in the open? That creates perverse incentives, and it makes it impossible for us to create responsible AI in the open.
His side obeys takedown requests and robots.txt, while, in his words, “I’m looking at you, Facebook, because you’ve been DDoS-ing my server for the last eight months.”
This was also his answer when I asked him afterwards why not found a company, since in the last few years AI research seems to have only been possible in private companies. What he wants is open training data, and a company with money would be sued by people who see their work in it (he pointed to the Copyright Term Extension Act, nicknamed the “Mickey Mouse Protection Act”). So the only way to do it is to have no money worth suing for. The people behind the Global Open Source AI Summit have even thought of setting up two companies, one in the most liberal country, and another holding the money and compute.
No one country can do it
The summit, in London on 30 November, is his answer to this one. “Sovereign does not mean isolated”, and in the questions he went further:
I think sovereign AI is ending, and it needs to end. […] there needs to be an international initiative, and the sovereign AI initiatives need to merge. Otherwise there is no future in fielding models.
The status quo
His talk ends with four things to change. The first two answer the unit of innovation, the third compute, and the fourth the rules:
He ended his answer to the chemist like this:
For those areas, I think open source AI is the only way. Sadly; I wish I could accept the status quo.
I think the only question is whether we should accept the status quo. In January 2023, a lot of the NLP community did: “we could no longer ask interesting questions, […] we should just accept that and analyse the models from the big companies, and I refused to accept it.” I think the world lacks people like him who don’t.
References
Footnotes
Hilbert’s line, from his 1930 address to the Society of German Scientists and Physicians in Königsberg, is “Wir müssen wissen. Wir werden wissen.” (“We must know. We will know.”), now his epitaph in Göttingen (Reid 1996, 192). It answered “ignoramus et ignorabimus” (“we do not know and will not know”), the physiologist Emil du Bois-Reymond’s verdict on the limits of science, given to the same society in 1872 (du Bois-Reymond 1872). The day before Hilbert spoke, in the same city, Gödel had first announced his incompleteness theorem (Dawson 1997, 68–71), which showed that in mathematics some truths cannot be proved in a given system. Stenetorp’s version does not depend on that.↩︎