Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Hypernetworks that adapt LLMs for specific benchmark tasks using only textual task description as the input
| Date | Stars |
|---|---|
| 2026-07-31 | 1296 |
| 2026-08-04 | 1297 |
| 2026-08-06 | 1297 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Text-to-LoRA (T2L): Instant Transformer Adaption
<div align="center">
:newspaper:<a href="https://x.com/SakanaAILabs">Tweet</a> |
:scroll:<a href="https://openreview.net/forum?id=zWskCdu3QA">Paper</a> |
:hugs:<a href="https://huggingface.co/SakanaAI">Hugging Face</a> |
:octocat:<a href="https://github.com/SakanaAI/text-to-lora">GitHub</a>
<br>A reference implementation of Text-to-LoRA (T2L).<br>
</div>
<div align="center">
<img height="300px" src="assets/overview_animation.gif" />
</div>
---
<h1 align="center">🛠️ Installation</h1>
Install `uv` if you don't have `uv` (see https://docs.astral.sh/uv/getting-started/installation/)
With `uv` installed, run the following to install the dependencies.
```bash
git clone https://github.com/SakanaAI/text-to-lora.git
cd text-to-lora
# make sure you have `uv` installed
# (see https://docs.astral.sh/uv/getting-started/installation/)
uv self update
uv venv --python 3.10 --seed
uv sync
# we use the following wheel for installation
# you might have to change the wheel to be compatible with your hardware
uv pip install https://github.com/Dao-AILab/flash-attention/releases/download/v2.6.3/flash_attn-2.6.3+cu123torch2.3cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
uv pip install src/fishfarm
```
---
<h1 align="center">🚀 Demo</h1>
***Downloading trained T2L***
:warning: **You need to download the checkpoints before running any of the demos.** :warning:
:warning: **You need a >16GB GPU to handle both models simultaneously to run any of these demos.** :warning:
```bash
uv run huggingface-cli login
uv run huggingface-cli download SakanaAI/text-to-lora --local-dir . --include "trained_t2l/*"
```
***Web UI***
This demo runs `Mistral-7B-Instruct-v0.2` locally alongside a T2L model.
```bash
# start webui locally
uv run python webui/app.py
```
<div align="center">
<img height="500px" src="assets/text-to-lora-demo.gif" />
</div>
***LoRA generation from CLI***
This script allows us to generate a LoRA based on a task description. Running for the first time would take longer as the base model will be downloaded and cached.
```bash
# uv run generate_lora.py {T2l_DIRECTORY} {TASK_DESCRIPTION}
# e.g.,
uv run python scripts/generate_lora.py \
trained_t2l/llama_8b_t2l \
"This task challenges your problem-solving abilities through mathematical reasoning. You must carefully read each scenario and systematically work through the data to compute the final outcome."
# You might be able to run T2L w/ `gemma-2-2b-it` with a smaller GPU.
uv run python scripts/generate_lora.py \
trained_t2l/gemma_2b_t2l \
"This task challenges your problem-solving abilities through mathematical reasoning. You must carefully read each scenario and systematically work through the data to compute the final outcome."
```
***Evaluating generated LoRA***
We can evaluate the generated LoRA by using the path printed at the end of the above script.
```bash
# uv run python scripts/run_eval.py --model-dir {base_model_dir} \
# --lora-dirs {lora_dirs} --save-results --tasks {tasks}
# e.g.,
uv run python scripts/run_eval.py \
--model-dir meta-llama/Llama-3.1-8B-Instruct \
--lora-dirs {PATH_TO_GENERATED_LORA} \
--save-results --tasks gsm8k
# You might be able to run T2L w/ `gemma-2-2b-it` with a smaller GPU.
uv run python scripts/run_eval.py \
--model-dir google/gemma-2-2b-it \
--lora-dirs {PATH_TO_GENERATED_LORA} \
--save-results --tasks gsm8k
```
> ***Disclaimer :grey_exclamation:***
>
> Even with random descriptions, SFT-trained T2L still generates reasonable LoRAs.
> This result with SFT-trained T2L differs from Table 5 in the paper that uses reconstruction-trained T2L.
> Nonetheless, comparing the results obtained between aligned and unaligned descriptions, aligned LoRAs perform noticeably better overall. The training descriptions can be found in `tasks/` folder, while the unseen ones used for evaluation can be found in `configs/hyper_lora_decontam_lol_tasks.yaml`
---
<h1 align="center">🏋️ Training</h1>Excerpt of 12,694 characters
Read on GitHub
Rujikorn Charakorn · https://sakana.ai/ · Japan
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:af49524d967ecf4a, topic:fine-tuning, topic:lora, name:lora