Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Evaluation of LLMs on latest math competitions
| Date | Stars |
|---|---|
| 2026-07-31 | 276 |
| 2026-08-06 | 276 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<div align="center">
<h1><img height="150px" src="./images/matharena_icon.png" alt="MathArena"><br>MathArena</h1>
<a href="https://www.python.org/">
<img alt="Build" src="https://img.shields.io/badge/Python-3.12-1f425f.svg?color=blue">
</a>
<a href="https://opensource.org/licenses/MIT">
<img alt="License: MIT" src="https://img.shields.io/badge/License-MIT-green.svg">
</a>
<a href="https://huggingface.co/MathArena">
<img alt="MathArena Datasets" src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Matharena-ffc107?color=ffc107&logoColor=white">
</a>
</div>
## 👋 Overview
MathArena (NeurIPS D&B '25) is a platform for evaluation of LLMs on latest math competitions and olympiads. It is hosted on [matharena.ai](https://matharena.ai/). This repository contains all code used for model evaluation. This README explains how to run your models. For more details about other aspects of the project, such as adding new competitions, please refer to the specific README files in the `readmes/` folder. You can find logs from our evaluation containing full reasoning traces (if available) and solutions produced by the models on our HuggingFace page: [https://huggingface.co/MathArena](https://huggingface.co/MathArena).
---
## 🚀 Installation
MathArena uses [UV](https://github.com/astral-sh/uv) to manage dependencies. If you want to run local models, uncomment the vllm installation in `pyproject.toml`.
### Install UV
- **macOS and Linux:**
```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
```
- **Windows:**
```powershell
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
```
### Alternative installation
As an alternative to UV, you can also create a conda environment and install the package as follows:
```bash
conda create -n matharena python=3.12
conda activate matharena
python -m pip install -e .
```
If you choose this option, disregard `uv run` in all instructions and use python directly instead.
---
## 🏃 Running an Eval
Execute the following command to evaluate a model on a competition:
```bash
uv run python scripts/run.py --comp path/to/competition --models path/to/model1
```
- `path/to/competition`: Relative path from the `configs/competition` folder to the competition config file (excluding the `.yaml` extension).
- `path/to/model1`: Relative path (or multiple) from the `configs/models` folder to the model config file (excluding the `.yaml` extension). See [Adding a Model/Agent](#adding-a-model) below for model config file structure.
The scripts `run_all_fa.sh` and `run_all_kangaroo.sh` provide convenient shortcuts to run a model on all current non-deprecated competitions. They can be executed as follows:
```bash
bash scripts/run_all_fa.sh path/to/model
```
Importantly, for ArXivLean, make sure to follow [the setup instructions](readmes/README_arxivlean.md) below before running the evaluation.
**Example:**
```bash
uv run python scripts/run.py --comp aime/aime_2025 --models openai/gpt-4o
```
**Additional Flags:**
- `--n`: Number of runs per problem (default: 4).
- `--redo-all`: Ignore existing runs for this model and rerun everything (default: false, continues from existing runs found in `outputs/`).
- `--problems`: One-based indices of problems to run (default: runs all problems).
### Current Website Competitions
The table below maps all non-deprecated competitions currently shown on the website to their competition config. `Requires judging` indicates whether the config requires a separate `scripts/judge/judge.py` pass.
| Website section | Website competition | Competition config | Requires judging |
| --- | --- | --- | --- |
| BrokenArxiv | 02/2026 | `arxiv_false/february.yaml` | Yes |
| BrokenArxiv | 03/2026 | `arxiv_false/march.yaml` | Yes |
| ArXivMath | 01/2026 | `arxiv/january.yaml` | No |
| ArXivMath | 02/2026 | `arxiv/february.yaml` | No |
| ArXivMath | 03/2026 | `arxiv/march.yaml` | No |
| Visual Math | Kangaroo 2025 1-2 | `kangaroo/kanExcerpt of 9,444 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:1c5327a3defcffa9, topic:llm