Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Generative Representational Instruction Tuning
| Date | Stars |
|---|---|
| 2026-07-31 | 697 |
| 2026-08-05 | 697 |
| 2026-08-06 | 697 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
## Generative Representational Instruction Tuning
This repository provides all materials for the paper [Generative Representational Instruction Tuning](https://arxiv.org/abs/2402.09906). We continue developing the repository and welcome any contributions. If you want to use the code in the exact same way as in the paper, please use the 1.0.0 release (commit hash = `3ac39052ef878371a658a060e69f9c0124bfd59b`). [63min video](https://www.youtube.com/watch?v=JAaua0jNIdA&t=1748s&pp=ygUuZ2VuZXJhdGl2ZSByZXByZXNlbnRhdGlvbmFsIGluc3RydWN0aW9uIHR1bmluZw%3D%3D) & [30min video](https://www.youtube.com/watch?v=wC7CdhtAMnk) on the paper by Niklas Muennighoff; [60min slides link](https://docs.google.com/presentation/d/1aHwDQaGfy2Dg6r68uRH7kFOuTKka3tzZzEIZIf35VNM/edit?usp=sharing&resourcekey=0-q6APQHTzHtB6jqYa-Snk2g).
- [Inference](#inference)
- [Basic](#basic)
- [Caching](#caching)
- [Models](#models)
- [Training](#training)
- [Data](#data)
- [Run](#run)
- [Alignment](#alignment)
- [Evaluation](#evaluation)
- [Embedding](#embedding)
- [Generative](#generative)
- [Known issues](#known-issues)
- [Visuals](#visuals)
- [Acknowledgements](#acknowledgements)
- [Citation](#citation)
### Inference
#### Basic
`pip install gritlm`
```python
from gritlm import GritLM
# Loads the model for both capabilities; If you only need embedding pass `mode="embedding"` to save memory (no lm head)
model = GritLM("GritLM/GritLM-7B", torch_dtype="auto")
# To load the 8x7B you will likely need multiple GPUs.
# All the kwargs are passed to HF from_pretrained so you can just do the below to load on multiple GPUs:
# model = GritLM("GritLM/GritLM-8x7B", torch_dtype="auto", device_map="auto")
# You can also load other models e.g.
# model = GritLM("Muennighoff/SGPT-125M-weightedmean-nli-bitfit", pooling_method="weighted_mean", attn=None)
# model = GritLM("hkunlp/instructor-base", pooling_method="mean", attn=None)
### Embedding/Representation ###
instruction = "Given a scientific paper title, retrieve the paper's abstract"
queries = ['Bitcoin: A Peer-to-Peer Electronic Cash System', 'Generative Representational Instruction Tuning']
documents = [
"A purely peer-to-peer version of electronic cash would allow online payments to be sent directly from one party to another without going through a financial institution. Digital signatures provide part of the solution, but the main benefits are lost if a trusted third party is still required to prevent double-spending. We propose a solution to the double-spending problem using a peer-to-peer network. The network timestamps transactions by hashing them into an ongoing chain of hash-based proof-of-work, forming a record that cannot be changed without redoing the proof-of-work. The longest chain not only serves as proof of the sequence of events witnessed, but proof that it came from the largest pool of CPU power. As long as a majority of CPU power is controlled by nodes that are not cooperating to attack the network, they'll generate the longest chain and outpace attackers. The network itself requires minimal structure. Messages are broadcast on a best effort basis, and nodes can leave and rejoin the network at will, accepting the longest proof-of-work chain as proof of what happened while they were gone.",
"All text-based language problems can be reduced to either generation or embedding. Current models only perform well at one or the other. We introduce generative representational instruction tuning (GRIT) whereby a large language model is trained to handle both generative and embedding tasks by distinguishing between them through instructions. Compared to other open models, our resulting GritLM 7B sets a new state of the art on the Massive Text Embedding Benchmark (MTEB) and outperforms all models up to its size on a range of generative tasks. By scaling up further, GritLM 8X7B outperforms all open generative language models that we tried while still being amonExcerpt of 43,226 characters
Read on GitHub
Niklas
60
2
2
Himanshu Shukla · Siemens digital industry · India
1
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:290c9105481f5e3c, topic:instruction-tuning, desc:instruction tuning
matched fp:290c9105481f5e3c, topic:llm
matched fp:290c9105481f5e3c, topic:embeddings