Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Official Implementation of "KBLaM: Knowledge Base augmented Language Model"
| Date | Stars |
|---|---|
| 2026-07-31 | 1448 |
| 2026-08-02 | 1448 |
| 2026-08-06 | 1448 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# KBLaM - Knowledge Base Augmented Language Models [ICLR 2025]
This repo contains the official implementation of [KBLaM: Knowledge Base Augmented Language Models](https://arxiv.org/abs/2410.10450).
Authors: Xi Wang, Liana Mikaelyan, Taketomo Isazawa, Mathew Salvaris, James Hensman.
KBLaM is a new method for augmentating LLMs with external knowledge.
Unlike Retrieval-Augmented Generation, KBLaM eliminates external
retrieval modules, and unlike in-context learning, its computational overhead scales linearly with KB size rather than quadratically.
## Supported Models
The following models from Hugging Face hub are currently supported:
- [meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct)
- [meta-llama/Llama-3.2-1B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct)
- [Phi-3-mini-4k-instruct](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct)
To add support for new model types, you will need to update the model processing scripts to incorporate an adapter similar to `llama_model.py` in `src/kblam/models`.
## Setting up
Install the kblam package with
```
pip install -e .
```
To use Llama models, you will need to generate a token from Hugging Face and use it to log in:
```
pip install huggingface_hub
huggingface-cli login
```
The experiments in the paper can be replicated by running the scripts in `./experiments`.
## Dataset Construction
To run the synthetic dataset construction, you will need a valid Azure OpenAI endpoint.
To construct a synthetic KB and question-answer pairs use `dataset_generation/gen_synthetic_data.py`
The question-answer pairs are constructed in the form:
```
What is the description of {entity_name}?
The description of {entity_name} is {description}.
```
To generate KB embeddings, use `dataset_generation/generate_kb_embeddings.py`.
The embeddings we current support are [text-embedding-ada-002](https://openai.com/index/new-and-improved-embedding-model/) and [all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2).
## Training
As an example of model training, see the following:
```
python train.py --dataset_dir <Your dataset directory> --train_dataset synthetic --N 120000 --B 20 --total_steps 601 --encoder_spec OAI --use_oai_embd --key_embd_src key --use_data_aug --use_cached_embed
```
Note in particular the `--use_cached_embed` argument. This should be set to prevent recomputation of embeddings, which can take significant time especially when using APIs such as OpenAI's text embeddings.
There are a number of optional arguments in `train.py` that you may want to consult.
## Contributing
This project welcomes contributions and suggestions. Most contributions require you to agree to a
Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us
the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.
When you submit a pull request, a CLA bot will automatically determine whether you need to provide
a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions
provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).
For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or
contact [[email protected]](mailto:[email protected]) with any additional questions or comments.
## Trademarks
This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft
trademarks or logos is subject to and must follow
[Microsoft's Trademark & Brand Guidelines](https://www.microsoft.com/en-us/legal/intellectualproperty/trademarks/usage/general).
Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply MicrosoftExcerpt of 6,155 characters
Read on GitHub
12
9
5
Microsoft Open Source · Microsoft · United States
5
2
1
1
1
1
Ikko Eltociear Ashimine · Japan
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:1f653fc37ad96f9c, desc:knowledge base