Researchllmmathematicsbenchmarksresearch evaluation
Mathematicians Launch Challenge Testing AI Problem-Solving
8.2
A team of 11 mathematicians led by Harvard and Stanford professors launched First Proof, unveiling 10 encrypted research problems on Feb. 5 and revealing solutions on Feb. 13. The problems span number theory, combinatorics, topology and numerical linear algebra, and were devised to benchmark AI systems; preliminary tests show leading LLMs solved only two problems. The effort aims to define AI's limits on research mathematics.
Key Points
- 1Unveil ten encrypted recent research problems across diverse fields, published Feb. 5 and revealed Feb. 13.
- 2Establish an independent, objective benchmark to evaluate AI systems on research-level mathematical problem solving.
- 3Indicate current LLMs (GPT-5.2 Pro, Gemini 3.0) solved two problems, implying limited creativity and reliability.
Scoring Rationale
High relevance and credible organizer team, but scope is specialized and doesn't yet demonstrate broad AI breakthroughs.
Sources
Public references used for this report.
Practice with real Logistics & Shipping data
90 SQL & Python problems · 15 industry datasets
Used by DS/ML engineers at top companies
High-Value Overnight OrdersEasyDelivered International ShipmentsMediumOn-Time Delivery Rate by CarrierHard
250 free problems · No credit card
See all Logistics & Shipping problems
