Beyond Benchmarks: MathArena as an Evaluation Platform for Mathematics with LLMs

AIPR assessment

This is a difficult, fast-moving benchmark-and-platform paper in a saturated area where results age quickly, so the maintenance model matters as much as the one-time scores. The strongest aspects reinforce each other: continuous updates, public traces, careful protocol repair, and multi-benchmark coverage make the platform more credible and more useful. The main weaknesses compound more mildly: several benchmarks are small, some headline comparisons depend on provider configurations, and the wor

Abstract

Large language models (LLMs) are becoming increasingly capable mathematical collaborators, but static benchmarks are no longer sufficient for evaluating progress: they are often narrow in scope, quickly saturated, and rarely updated. This makes it hard to compare models reliably and track progress over time. Instead, we need evaluation platforms: continuously maintained systems that run, aggregate, and analyze evaluations across many benchmarks to give a comprehensive picture of model performance within a broad domain. In this work, we build on the original MathArena benchmark by substantially broadening its scope from final-answer olympiad problems to a continuously maintained evaluation platform for mathematical reasoning with LLMs. MathArena now covers a much wider range of tasks, including proof-based competitions, research-level arXiv problems, and formal proof generation in Lean. Additionally, we maintain a clear evaluation protocol for all models and regularly design new benchmarks as model capabilities improve to ensure that MathArena remains challenging. Notably, the strongest model, GPT-5.5, now reaches 98% on the 2026 USA Math Olympiad and 74% on research-level questions, showing that frontier models can now comfortably solve extremely challenging mathematical problems. This highlights the importance of continuously maintained evaluation platforms like MathArena to track the rapid progress of LLMs in mathematical reasoning.

Score Breakdown

More from this week

• Beyond Visual Fidelity: Benchmarking Super-Resolution Models for Large-Scale Remote Sensing Imagery via Downstream Task Integration
• Eliminating Hidden Serialization in Multi-Node Megakernel Communication
• Succinct Graph Representations and Algorithmic Applications
• A Faster Deterministic Algorithm for Fully Dynamic Maximal Matching
• Claw-Eval-Live: A Live Agent Benchmark for Evolving Real-World Workflows

More in Machine Learning

• STAR-KV: Low-Rank KV Cache Compression via Soft Thresholding for Adaptive Rank Control
• Verifiable and Confidential DNN Inference on Low-End Edge Devices
• HASTE: Hardware-Aware Dynamic Sparse Training for Large Output Spaces
• LeAP: Learnable Adaptive Permutation for Feature Selection in Heterogeneous and Sparse Recommender Systems
• TriAxialKV: Toward Extreme Low-Precision KV-Cache Quantization for Agentic Inference Tasks