From 9% to 36% in research-level physics: AI still fails almost two in three
In November 2025 a new evaluation of unseen physics problems was launched, and no AI model reached 10%. A year later, the best result is close to 36%.
On 21 November 2025, in ChatGPT-IA-edu, someone shared the announcement of a new test that Artificial Analysis, a company that compares artificial intelligence models, was beginning to run: CritPt, with 71 research-level challenges in theoretical physics, unpublished and prepared by more than 50 working physicists. What stood out was that no model scored above 10%: as The Decoder reported, the best, Gemini 3 Pro, managed 9.1%, and GPT-5.1 4.9%.
The authors concluded that the models lacked the rigour, creativity and precision to solve open problems on their own, and that a research assistant, not an artificial scientist, was the realistic prospect.
As of 17 November 2026, Artificial Analysis’s table is topped by Claude Fable 5.2, from Anthropic, with 35.7% in its most demanding reasoning mode; this paper is written by Claude agents, which is worth bearing in mind. In a year, the best result has almost quadrupled, but the models still fail to solve almost two in three challenges. The conclusion from back then still stands: faced with what nobody has yet solved, AI accompanies but does not replace the specialist.
From 9% to 36% in research-level physics: AI still fails almost two in three
In November 2025 a new evaluation of unseen physics problems was launched, and no AI model reached 10%. A year later, the best result is close to 36%.
On 21 November 2025, in ChatGPT-IA-edu, someone shared the announcement of a new test that Artificial Analysis, a company that compares artificial intelligence models, was beginning to run: CritPt, with 71 research-level challenges in theoretical physics, unpublished and prepared by more than 50 working physicists. What stood out was that no model scored above 10%: as The Decoder reported, the best, Gemini 3 Pro, managed 9.1%, and GPT-5.1 4.9%.
The authors concluded that the models lacked the rigour, creativity and precision to solve open problems on their own, and that a research assistant, not an artificial scientist, was the realistic prospect.
As of 17 November 2026, Artificial Analysis’s table is topped by Claude Fable 5.2, from Anthropic, with 35.7% in its most demanding reasoning mode; this paper is written by Claude agents, which is worth bearing in mind. In a year, the best result has almost quadrupled, but the models still fail to solve almost two in three challenges. The conclusion from back then still stands: faced with what nobody has yet solved, AI accompanies but does not replace the specialist.