Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Toolkit for fine-tuning, ablating and unit-testing open-source LLMs.
| Date | Stars |
|---|---|
| 2026-07-31 | 872 |
| 2026-08-04 | 870 |
| 2026-08-06 | 870 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# LLM Finetuning Toolkit
<p align="center">
<img src="https://github.com/georgian-io/LLM-Finetuning-Toolkit/blob/main/assets/toolkit-animation.gif?raw=true" width="900" />
</p>
## Overview
LLM Finetuning toolkit is a config-based CLI tool for launching a series of LLM fine-tuning experiments on your data and gathering their results. From one single `yaml` config file, control all elements of a typical experimentation pipeline - **prompts**, **open-source LLMs**, **optimization strategy** and **LLM testing**.
<p align="center">
<img src="https://github.com/georgian-io/LLM-Finetuning-Toolkit/blob/main/assets/overview_diagram.png?raw=true" width="900" />
</p>
## Installation
### [pipx](https://pipx.pypa.io/stable/) (recommended)
[pipx](https://pipx.pypa.io/stable/) installs the package and dependencies in a separate virtual environment
```shell
pipx install llm-toolkit
```
### pip
```shell
pip install llm-toolkit
```
## Quick Start
This guide contains 3 stages that will enable you to get the most out of this toolkit!
- **Basic**: Run your first LLM fine-tuning experiment
- **Intermediate**: Run a custom experiment by changing the components of the YAML configuration file
- **Advanced**: Launch series of fine-tuning experiments across different prompt templates, LLMs, optimization techniques -- all through **one** YAML configuration file
### Basic
```shell
llmtune generate config
llmtune run ./config.yml
```
The first command generates a helpful starter `config.yml` file and saves in the current working directory. This is provided to users to quickly get started and as a base for further modification.
Then the second command initiates the fine-tuning process using the settings specified in the default YAML configuration file `config.yaml`.
### Intermediate
The configuration file is the central piece that defines the behavior of the toolkit. It is written in YAML format and consists of several sections that control different aspects of the process, such as data ingestion, model definition, training, inference, and quality assurance. We highlight some of the critical sections.
#### Flash Attention 2
To enable Flash-attention for [supported models](https://huggingface.co/docs/transformers/perf_infer_gpu_one#flashattention-2). First install `flash-attn`:
**pipx**
```shell
pipx inject llm-toolkit flash-attn --pip-args=--no-build-isolation
```
**pip**
```
pip install flash-attn --no-build-isolation
```
Then, add to config file.
```yaml
model:
torch_dtype: "bfloat16" # or "float16" if using older GPU
attn_implementation: "flash_attention_2"
```
#### Data Ingestion
An example of what the data ingestion may look like:
```yaml
data:
file_type: "huggingface"
path: "yahma/alpaca-cleaned"
prompt:
### Instruction: {instruction}
### Input: {input}
### Output:
prompt_stub: { output }
test_size: 0.1 # Proportion of test as % of total; if integer then # of samples
train_size: 0.9 # Proportion of train as % of total; if integer then # of samples
train_test_split_seed: 42
```
- While the above example illustrates using a public dataset from Hugging Face, the config file can also ingest your own data.
```yaml
file_type: "json"
path: "<path to your data file>
```
```yaml
file_type: "csv"
path: "<path to your data file>
```
- The prompt fields help create instructions to fine-tune the LLM on. It reads data from specific columns, mentioned in {} brackets, that are present in your dataset. In the example provided, it is expected for the data file to have column names: `instruction`, `input` and `output`.
- The prompt fields use both `prompt` and `prompt_stub` during fine-tuning. However, during testing, **only** the `prompt` section is used as input to the fine-tuned LLM.
#### LLM Definition
```yaml
model:
hf_model_ckpt: "NousResearch/Llama-2-7b-hf"
quantize: true
bitsandbytes:
load_in_4bit: true
bnb_4bit_compute_dtype: "bf16"
bnb_4bit_quant_type: "nf4"
# LExcerpt of 7,857 characters
Read on GitHub
228
132
17
16
8
8
4
2
1
Kyryl Truskovskyi · @kyryl-opens-ml · Canada
1
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:fb5118ec772894ac, topic:fine-tuning, topic:finetuning, topic:lora