Superhuman mathematical problem solving before 2030, assuming no AGI yet?
Imagine that any math problem you can write down on a piece of paper that a team of Fields medalists can solve, AI can as well. Until recently, I would've predicted that that was an AGI-complete problem. Of course people used to think grandmaster-level chess would require AGI. Until 2022 I was sure that commonsense reasoning and being able to explain jokes would require AGI. If subhuman general intelligence can be a superhuman mathematical intelligence, that will be another big update for me. FAQ 1. What if AGI happens first? This is a conditional prediction market. If AGI, as defined in my other market, happens first, this resolves N/A. 2. Does the AI need to max out the FrontierMath benchmark for this to resolve YES? Yes, and every math benchmark, plus gold-medal performance on the International Math Olympiad [Update: now achieved, as of summer 2025]. Even acing the Putnam. 3. What if it's essentially true but there are rare exceptions? The spirit of the question is that we'd only consider an AI failure to be an exception if it failed for a reason other than being insufficiently brilliant at math. Like tricksy wording, or any trick question. The posing of the question has to be non-adversarial. 4. What about a book-length question? Tentative answer so far: The problem has to be posed on a single human-readable sheet of paper or equivalent. But a question can cite any peer-reviewed math paper as background. (Dumping an impenetrable tome on the arXiv doesn't count.) If you have an example where this feels limiting, let me know. My suspicion is that all interesting math problems can be posed on a single page and in any case it won't harm the spirit of this question to limit ourselves to such. 4. What about research taste? That's a big part of being a mathematician and isn't required for this market. The AI just has to be superhuman at answering questions, not asking them. 5. What about cost and speed? The AI has to dominate the best humans on all metrics. We'll find an authoritative source for the market value of mathematicians' time if it comes down to that. 6. What about availability to the public? Not required. If there's any doubt about the veracity of claims that this has been achieved, we'll discuss and delay resolution as needed. 7. What if the AI is sometimes super- and sometimes sub-human at math? In some senses that's already the case but there may be ambiguous edge cases. As an extreme example, imagine that the AI is so blatantly superhuman that it cracks a famous open problem, yet it's routinely stumped or wrong on problems amateur human mathematicians can do. For the spirit of the question for this market, we'll try to assess whether we'd consider a human with the AI's math abilities to be the greatest mathematician of all time. (Or the greatest raw math prodigy of all time -- see FAQ 4 on the distinction between problem solving and knowing what questions to ask. The latter is a key part of being a successful mathematician and is explicitly not part of this prediction.) Related markets https://manifold.markets/dreev/in-what-year-will-we-have-agi https://manifold.markets/jack/will-an-ai-outcompete-the-best-huma-cj3ul7a2g2 https://manifold.markets/MatthewBarnett/will-an-ai-achieve-85-performance-o https://manifold.markets/Manifold/what-will-be-the-best-performance-o-nzPCsqZgPc [ignore the subhuman clarifications that keep automatically appearing below this line]
