Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
A Heterogeneous Benchmark for Information Retrieval. Easy to use, evaluate your models across 15+ diverse IR datasets.
| Date | Stars |
|---|---|
| 2026-07-24 | 2252 |
| 2026-07-25 | 2252 |
| 2026-07-28 | 2252 |
| 2026-07-30 | 2252 |
| 2026-08-06 | 2252 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<h1 align="center">
<img style="vertical-align:middle" width="450" height="180" src="https://raw.githubusercontent.com/benchmarkir/beir/main/images/color_logo_transparent_cropped.png" />
</h1>
<p align="center">
<a href="https://github.com/beir-cellar/beir/releases">
<img alt="GitHub release" src="https://img.shields.io/github/release/beir-cellar/beir.svg">
</a>
<a href="https://www.python.org/">
<img alt="Build" src="https://img.shields.io/pypi/pyversions/beir?logo=pypi&style=flat&color=blue">
</a>
<a href="https://github.com/beir-cellar/beir/blob/master/LICENSE">
<img alt="License" src="https://img.shields.io/github/license/beir-cellar/beir?logo=github&style=flat&color=green">
</a>
<a href="https://colab.research.google.com/drive/1HfutiEhHMJLXiWGT8pcipxT5L2TpYEdt?usp=sharing">
<img alt="Open In Colab" src="https://colab.research.google.com/assets/colab-badge.svg">
</a>
<a href="https://pepy.tech/project/beir">
<img alt="Downloads" src="https://img.shields.io/pypi/dm/beir?logo=pypi&style=flat&color=orange">
</a>
<a href="https://github.com/beir-cellar/beir/">
<img alt="Open Source" src="https://badges.frapsoft.com/os/v1/open-source.svg?v=103">
</a>
</p>
<h4 align="center">
<p>
<a href="https://openreview.net/forum?id=wCu6T5xFjeJ">Paper</a> |
<a href="#beers-installation">Installation</a> |
<a href="#beers-quick-example">Quick Example</a> |
<a href="#beers-available-datasets">Datasets</a> |
<a href="https://github.com/beir-cellar/beir/wiki">Wiki</a> |
<a href="https://huggingface.co/BeIR">Hugging Face</a>
<p>
</h4>
<!-- > The development of BEIR benchmark is supported by: -->
<h3 align="center">
<a href="http://www.ukp.tu-darmstadt.de"><img style="float: left; padding: 2px 7px 2px 7px;" width="220" height="100" src="./images/ukp.png" /></a>
<a href="https://www.tu-darmstadt.de/"><img style="float: middle; padding: 2px 7px 2px 7px;" width="250" height="90" src="./images/tu-darmstadt.png" /></a>
<a href="https://uwaterloo.ca"><img style="float: right; padding: 2px 7px 2px 7px;" width="320" height="100" src="./images/uwaterloo.png" /></a>
</h3>
<h3 align="center">
<a href="https://huggingface.co/"><img style="float: middle; padding: 2px 7px 2px 7px;" width="400" height="80" src="./images/HF.png" /></a>
</h3>
## :beers: What is it?
**BEIR** is a **heterogeneous benchmark** containing diverse IR tasks. It also provides a **common and easy framework** for evaluation of your NLP-based retrieval models within the benchmark.
For **an overview**, checkout our **new wiki** page: [https://github.com/beir-cellar/beir/wiki](https://github.com/beir-cellar/beir/wiki).
For **models and datasets**, checkout out **Hugging Face (HF)** page: [https://huggingface.co/BeIR](https://huggingface.co/BeIR).
For more information, checkout out our publications:
- [BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models](https://openreview.net/forum?id=wCu6T5xFjeJ) (NeurIPS 2021, Datasets and Benchmarks Track)
- [Resources for Brewing BEIR: Reproducible Reference Models and an Official Leaderboard](https://dl.acm.org/doi/10.1145/3626772.3657862) (SIGIR 2024 Resource Track)
## :beers: Installation
Install via pip:
```python
pip install beir
```
If you want to build from source, use:
```python
$ git clone https://github.com/beir-cellar/beir.git
$ cd beir
$ pip install -e .
```
Tested with python versions 3.9+
## :beers: Features
- Preprocess your own IR dataset or use one of the already-preprocessed 17 benchmark datasets
- Wide settings included, covers diverse benchmarks useful for both academia and industry
- Evaluates well-known retrieval architectures (lexical, dense, sparse and reranking-based)
- Add and evaluate your own model in a easy framework using different state-of-the-art evaluation metrics
## :beers: Quick Example
For other Excerpt of 27,078 characters
Read on GitHub
402
Nouamane Tazi · @huggingface · France
52
4
4
Nils Reimers · Cohere
4
3
2
2
1
Julian Risch · @deepset-ai · Germany
1
1
Kevin Canwen Xu · United States
1
1
1
1
Ikko Eltociear Ashimine · Japan
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:86575c5b2d3ca3a8, topic:deep-learning, topic:pytorch
matched fp:86575c5b2d3ca3a8, topic:sentence-transformers, readme:reranking
matched fp:86575c5b2d3ca3a8, topic:benchmark, readme:leaderboard
matched fp:86575c5b2d3ca3a8, topic:dataset, readme:dataset, desc:datasets