Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
| Date | Stars |
|---|---|
| 2026-07-31 | 1171 |
| 2026-08-01 | 1171 |
| 2026-08-05 | 1172 |
| 2026-08-06 | 1173 |
Today
+1 stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Math-Verify
A robust mathematical expression evaluation system designed for assessing Large Language Model outputs in mathematical tasks. This evaluator achieves the highest accuracy and most correct scores compared to existing evaluators on MATH dataset:
| Evaluator | Score |
|---------------|---------|
| Harness | 0.0802 |
| Qwen | 0.1288 |
| Math-Verify | 0.1328 |
## Installation
Math-Verify currently supports multiple antlr4 runtimes:
- `antlr4-python3-runtime==4.13.2`
- `antlr4-python3-runtime==4.11.0`
- `antlr4-python3-runtime==4.9.3`
To install Math-Verify with a specific antlr4 runtime, use the following command:
```bash
pip install math-verify[antlr4_13_2]
```
We recommend always specifying the antlr4 runtime to avoid any potential issues.
## Example Usage
```python
from math_verify import parse, verify
# Parse the gold and answer
# If you know that gold will only contain latex or expr (no latex env), use
# parse(gold, extraction_config=[LatexExtractionConfig()]) or parse(gold, extraction_config=[ExprExtractionConfig()])
gold = parse("${1,3} \\cup {2,4}$")
answer = parse("${1,2,3,4}$")
# Order here is important!
verify(gold, answer)
# >>> True
```
## Extraction Targets
The parser supports three main extraction targets:
1. **LatexExtractionConfig** - Extracts LaTeX expressions with configurable options (see docstring) (e.g. '\[ \sqrt{2} \]'). Do note that the latex must be placed in latex environment to be parsable.
2. **ExprExtractionConfig** - Extracts plain mathematical expressions (e.g. '1/2')
3. **StringExtractionConfig** - Extracts literal strings (e.g. 'A')
By default, the parser uses both LatexExtractionConfig and ExprExtractionConfig for maximum flexibility in extracting mathematical expressions from model outputs.
## Why Another Math Evaluator?
Existing math evaluators often fail to correctly assess model outputs due to:
1. Strict format requirements (expecting exact patterns like "Final answer is X")
2. Limited parsing capabilities (especially for complex mathematical notations)
3. Inflexible comparison logic (unable to recognize equivalent expressions)
As result, this can lead to significant underestimation of model performance, in extreme cases, even by 40 points.
## Key Features
### 1. Robust Answer Extraction
- Multiple extraction strategies (LaTeX, Plain Numerical Expressions)
- Answer retrieval is done in format agnostic manner, with best effort to extract the answer.
- Supports all standard latex formats for the best retrieval.
### 2. Advanced Parsing Capabilities
- Complete set theory support (Intervals, FiniteSets, set operations)
- Unicode symbol substituion support (e.g. `β -> beta`)
- Applies Latex fixes for common malformations (e.g. `frac13 -> 1/3`)
- Equation and inequality parsing, with symbol assignment resolution (e.g. `x = 1 -> 1`)
- Percentage best effort conversion (e.g. `10% -> 0.1`)
- Units in text handling (e.g. `10 cm -> 10`)
- Exact representation of the input expressions (e.g. `0.333 -> Float(333, 3)`)
### 3. Intelligent Expression Comparison
- Both numerical and symbolic comparison support
- Precise numerical comparison for numerical types with configurable rounding tolerance
- Matrix expression equivalence validation
- Set and interval comparison
- Relation evaluation with flip support (e.g., `a < 2 == 2 > a`)
## Advanced Usage
If you already have a model outputs, format them into a csv file with `answer`, `gold` columns.
Then run the following command:
```bash
python evaluate_model_outputs.py --input_csv <path_to_csv> (examples/model_outputs.csv) --output_csv <path_to_csv> (output.csv)
```
If you want to evaluate a model from ground up, we have provided a script for end to end evaluation with support for following datasets:
- MATH-Hard
- MATH-500
- GSM8K
- AMC23
- AIME24
This script requires the optional "inference" dependencies to be installed, e.g. as follows:
```bash
pip install 'math-verify[inference]'
```
Run the following comExcerpt of 10,770 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:6bdd4aa7cc507974, llm:Repository: huggingface/Math-Verify. Name suggests verification of math expressions; no topics, no README provided. Likely a tool for verifying math solutions or expressions, related to evaluation of mathematical reasoning in models or datasets from Hugging Face.
matched fp:6bdd4aa7cc507974, llm:Repository: huggingface/Math-Verify. Name suggests verification of math expressions; no topics, no README provided. Likely a tool for verifying math solutions or expressions, related to evaluation of mathematical reasoning in models or datasets from Hugging Face.
