What if the button came with the understanding?
Along the lines of my own posts on the Navier–Stokes incident, and many of the guest posts on Terence Tao’s blog, it seems the main reason for the incident to be outrageous is that it landed without understanding. The understanding is the product, and what came out of the agents had no theory laid out in it.
Is that really true though? Suppose hypothetically we now have a button that, once pushed, lands a theorem with the understanding. In the extreme case, think “solve Fermat’s Last Theorem”, and out comes a beautiful connection between two fields, like the one between modular forms and elliptic curves. Henry Cohn worried about a different button:
[…] it’s a problem if people have a button that creates technical debt without their knowledge. (Cohn 2026)
The button in my thought experiment creates none, so what would people’s reaction be? Like Go after AlphaGo? And would pushing it still be considered a sin? (No one used this word, but hey.)
I think it would still be very disruptive, because transferring the understanding, from the theory built here to the understanding in someone’s brain, is still a bottleneck.1 To reason in the extremes, all professional mathematicians would be overwhelmed by catching up with the understanding alone. And since catching up doesn’t produce any new results, funding will dry up. Timothy Gowers already worries about this, even without the button:
A related risk is that the perception among policy-makers will be that mathematicians are no longer needed and that funding will become much harder to come by […] (Gowers 2026)
We could argue that, very much like in software engineering, the “problem solving” part is solved, but “asking the right question” is still the most important part, and a bottleneck in many cases: what do we really want to build? Mathematicians could still be the ones asking the most insightful questions and directing the field. But how would they have the capacity and time to catch up with all the problems AI has solved, and then start asking the next questions? Tao says the questions are exactly what is running out:
In fact, it is now the identification of a promising problem which is the scarce and precious resource. We have now seen that even the rumor of someone working on a problem can trigger a massive amount of AI-powered effort to flatten it before the original research project has time to reach its full potential. (Tao 2026)
Also, how would AI labs pushing these magic buttons and dropping proofs together with theory work? It is expensive to push this button. Would pushing it remain a demonstration of capability? DeepMind did something like that with Go: it released 50 games AlphaGo played against itself as its contribution to the Go community, retired AlphaGo, and moved on to AlphaFold, never looking back. If so, maybe it is fine (less harm), as the disruption would be one-off, or at least one-off per new model.
That depends on the cost, which is one of the three variables I mentioned in A flood from within the community: how hard a problem is, the probability of solving it, and how expensive that is. If the cost comes down, and mathematics is an infinite game (likely true), then the disruption will be continuous, so long as AI progress hasn’t stagnated. Infinite here doesn’t mean there are plenty of good questions, though. As Tao puts it, “a country or region can suffer a critical shortage of drinking water while simultaneously being surrounded by a massive ocean.” (Tao 2026)
I guess the bottom line of my point is this. Is just dropping a verified proof without a theory, in the sense of theory building, what makes it unethical? The thought experiment seems to point to the uncoordinated part as a core issue, one that understanding alone doesn’t fix. That is where progressive disclosure ended up too: the rush is the harm, and what mathematics could borrow from cybersecurity is coordination.
And coordination is starting to happen. The Advisory Group on Mathematics and Artificial Intelligence was set up after OpenAI approached some of its members, and it says:2
We are currently facing the very specific challenge of advising OpenAI on how to coordinate the release of a large number of significant results in mathematics that they report have been produced by their internal model.
That is exactly OpenAI trying to coordinate with the community. But a large number of significant results sounds more like a flood to me.
References
Footnotes
In Long-horizon agents write dead programs I said how long it takes a human to digest the output is a separate concern. Separate is not the same as no concern, and when results come faster than anyone can digest them, the rate matters.↩︎
Gowers is one of its nine members.↩︎