Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Tevatron - Unified Document Retrieval Toolkit across Scale, Language, and Modality. Demo in SIGIR 2023, SIGIR 2025.
| Date | Stars |
|---|---|
| 2026-07-24 | 743 |
| 2026-07-25 | 743 |
| 2026-07-28 | 743 |
| 2026-07-30 | 743 |
| 2026-08-06 | 743 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Tevatron V2.0
<div align="center">
<a href="https://arxiv.org/abs/2203.05765" target="_blank"><img src=https://img.shields.io/badge/arXiv-b5212f.svg?logo=arxiv></a>
<a href="https://huggingface.co/Tevatron" target="_blank"><img src=https://img.shields.io/badge/%F0%9F%A4%97%20HuggingFace%20Datasets-27b3b4.svg></a>
<a href="https://opensource.org/license/apache-2-0"><img src="https://img.shields.io/static/v1?label=License&message=Apache-2.0&color=red"></a>
<a href="https://pepy.tech/projects/tevatron"><img src="https://static.pepy.tech/badge/tevatron" alt="PyPI Downloads"></a>
<a href="https://star-history.com/#texttron/tevatron"> <img src="https://img.shields.io/github/stars/texttron/tevatron?style=social" alt="GitHub stars"> </a>
<!-- -->
</div>
Tevatron: Unified Document Retrieval Toolkit across Scale, Language, and Modality.
> Some of the features in Tevatron v1 is not yet migrated to Tevatron v2.0. We are working on it.
> If you are looking for the Tevatron v1 features, please pull the [v1 branch](https://github.com/texttron/tevatron/tree/tevatron-v1).
## Features
- Training billion-scale LLM neural retriever on GPUs and TPUs.
- Parameter efficient tuning with LoRA.
- Integration with vLLM, DeepSpeed, FlashAttention, gradient accumulation, and other efficient training and inference techniques.
- Self-contained [huggingface datasets](https://huggingface.co/Tevatron) for multi-modal and multilingual neural retrieval and open-domain QA tasks.
- Direct loading and finetuning SoTA pre-trained models (BGE-Embbedding, Instruct-E5) from HuggingFace.
## Installation
<details><summary><b>PyTorch (GPU)</b></summary>
0. Clone the repository.
1. Install PyTorch based on your CUDA version from [PyTorch](https://pytorch.org/get-started/locally/).
2. Install dependencies and Tevatron.
```bash
pip install git+https://github.com/luyug/GradCache.git
pip install transformers datasets peft
pip install deepspeed accelerate
pip install faiss-cpu
pip install pillow qwen-omni-utils
pip install -e .
```
</details>
<details><summary><b>JAX (TPU)</b></summary>
0. Clone the repository.
1. Install JAX by following the [official guide](https://jax.readthedocs.io/en/latest/installation.html#pip-installation-google-cloud-tpu)
2. Install dependencies
```bash
pip install transformers datasets
pip install flax optax
```
3. Install Magix and GradCache
```bash
git clone https://github.com/luyug/magix.git
cd magix && pip install -e . && cd ..
git clone https://github.com/luyug/GradCache.git
cd GradCache && pip install -e . && cd ..
```
4. Install Tevatron
```bash
pip install -e .
```
</details>
<details><summary><b>JAX (GPU)</b></summary>
To run the JAX implementation of Tevatron on GPU, we encourage using the jax-toolbox [jax container](https://github.com/NVIDIA/JAX-Toolbox/pkgs/container/jax) image from NVIDIA.
Below is a Dockerfile example to set up Tevatron on top of the jax container.
```Dockerfile
FROM ghcr.io/nvidia/jax:jax-2024-03-08
RUN apt-get update && \
apt-get install -y --no-install-recommends python3-pip && \
apt-get clean && \
rm -rf /var/lib/apt/lists/* && \
pip install --no-cache-dir transformers sentencepiece simple_parsing datasets orbax==0.4.8 && \
pip install --no-cache-dir torch --index-url https://download.pytorch.org/whl/cpu
RUN git clone https://github.com/luyug/magix.git && \
cd magix && pip install -e . && cd .. && \
git clone https://github.com/luyug/GradCache.git \
cd GradCache && pip install -e . && cd .. \
git clone https://github.com/texttron/tevatron.git && \
cd tevatron && pip install -e .
```
</details>
## Tevatron 101
In this example, we will demonstrate how to use Tevatron to LoRA fine-tune a Mistral-7B model on the MSMARCO passage dataset. The obtained LLM Retriever is expected to have `MRR@10=42.3` on the MS MARCO dev set with straightforward training.
<details><summary><b>Data Preparation</b></summary>
Tevatron takes training or inference data in `jsonl`Excerpt of 12,257 characters
Read on GitHub
206
77
63
14
8
6
5
4
4
2
2
Jasper Xian
2
2
2
Zhuofeng-Li
1
1
1
1
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:69916cdbb3ae2102, topic:pytorch, topic:jax
matched fp:69916cdbb3ae2102, topic:transformer
matched fp:69916cdbb3ae2102, topic:question-answering, desc:document retrieval, readme:document retrieval