A Few Personal Thoughts on AI and Science

Some thoughts, from a seat fairly near the front, on watching the Millennium Prize Problems get capped off one by one by AI.

1. Multi-agent collaboration really does help with test-time scaling. It extends the reasoning process and broadens the search. Go already showed us the “gradually, then suddenly” pattern of AI progress. Yet when a breakthrough actually arrives, it can still exceed almost everyone’s expectations.

2. Lean may achieve far more than its creators originally imagined. I think the developers of systems such as Lean and Coq deserve a Turing Award.

3. From allegations of hidden human help in chess, to disputes over ko in Go, to action-speed controversies in StarCraft, our first encounters with superhuman AI always seem to come with some drama. These disputes expose weaknesses that can be addressed. Systems improve, and yesterday’s objections lose their force. That improvement is worth pursuing.

4. Superintelligence will, unfortunately, bring collateral damage. In the years after AlphaGo, many familiar players from our childhood, including Lee Sedol, retired. The disappearance of an old order can be painful. But expecting it to survive unchanged, or trying to resist reality through boycotts, seems futile to me.

5. Human scientists may spend a very long time learning from AI. Go master Hideyuki Fujisawa famously expressed how little of the game’s truth he felt he understood. Mathematics, science and engineering surely contain even more that we have yet to discover.

6. AlphaGo transformed how professional Go players train. Learning from AI has substantially raised the level of play and changed our understanding of the game. I expect scientific research to undergo changes at least as profound.

7. Kasparov put it bluntly: “There’s a brief moment of equality in performance with humans. But that doesn’t last long, and forever after machines will do it better, cheaper, and more safely.”

   The threshold for “AI + Human > AI” seems to be rising fast. At the mathematical frontier, useful human guidance may increasingly require a leading expert working in their strongest field.

8. Millennium Prize Problems matter enormously to mathematicians. Some may feel a sense of loss if AI solves them. For AI companies, solving one through LLMs, multi-agent collaboration and long-horizon reasoning — or producing ten million lines of formal proof — would also be an extraordinary milestone. Their attention, however, may be on the capability demonstrated rather than the mathematics itself.

9. Fields transformed by superhuman AI will probably have to build their own new institutions and practices. AI companies may contribute only a limited part of that work.

   After AlphaGo left competition, projects such as Tencent’s Fine Art, Leela Zero and KataGo carried Go AI forward. Much of the ongoing development now happens through communities, making powerful tools available for people to run and improve themselves.

10. We may need to look inward more often for the value of what we do. Do you enjoy thinking about the problem? Does it still fascinate you?

    Are you having fun?

11. On a more optimistic note, we may live to see some extraordinary things. Could we travel beyond the Solar System? Discover an unknown ancient civilization?

    And if you’re already fed up with grand conjectures falling to counterexamples, imagine someone finding a counterexample showing that quantum mechanics and general relativity can never be reconciled, or decrypt Linear A/B of Crete. 🙄