Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Tools for merging pretrained large language models.
| Date | Stars |
|---|---|
| 2026-07-31 | 7267 |
| 2026-08-02 | 7267 |
| 2026-08-04 | 7279 |
| 2026-08-06 | 7279 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# mergekit
[](https://www.gnu.org/licenses/lgpl-3.0)
[](https://github.com/arcee-ai/mergekit/actions/workflows/pre-commit.yml)
[](https://discord.gg/arceeai)
`mergekit` is a toolkit for merging pre-trained language models. `mergekit` uses an out-of-core approach to perform unreasonably elaborate merges in resource-constrained situations. Merges can be run entirely on CPU or accelerated with as little as 8 GB of VRAM. Many merging algorithms are supported, with more coming as they catch my attention.
## Contents
- [Why Merge Models?](#why-merge-models)
- [Features](#features)
- [Installation](#installation)
- [Community & Support](#community--support)
- [Contributing](#contributing)
- [Community Tools](#community-tools)
- [Usage](#usage)
- [Merge Configuration](#merge-configuration)
- [Parameter Specification](#parameter-specification)
- [Tokenizer Configuration](#tokenizer-configuration)
- [Chat Template Configuration](#chat-template-configuration)
- [Examples](#examples)
- [Merge Methods](#merge-methods)
- [LoRA Extraction](#lora-extraction)
- [Mixture of Experts Merging](#mixture-of-experts-merging)
- [Evolutionary Merge Methods](#evolutionary-merge-methods)
- [Multi-Stage Merging (`mergekit-multi`)](#multi-stage-merging-mergekit-multi)
- [Raw PyTorch Model Merging (`mergekit-pytorch`)](#raw-pytorch-model-merging-mergekit-pytorch)
- [Tokenizer Transplantation (`mergekit-tokensurgeon`)](#tokenizer-transplantation-mergekit-tokensurgeon)
- [Citation](#citation)
## Why Merge Models?
Model merging is a powerful technique that allows combining the strengths of different models without the computational overhead of ensembling or the need for additional training. By operating directly in the weight space of models, merging can:
- Combine multiple specialized models into a single versatile model
- Transfer capabilities between models without access to training data
- Find optimal trade-offs between different model behaviors
- Improve performance while maintaining inference costs
- Create new capabilities through creative model combinations
Unlike traditional ensembling which requires running multiple models, merged models maintain the same inference cost as a single model while often achieving comparable or superior performance.
## Features
Key features of `mergekit` include:
- Supports Llama, Mistral, GPT-NeoX, StableLM, and more
- Many [merge methods](#merge-methods)
- GPU or CPU execution
- Lazy loading of tensors for low memory use
- Interpolated gradients for parameter values (inspired by Gryphe's [BlockMerge_Gradient](https://github.com/Gryphe/BlockMerge_Gradient) script)
- Piecewise assembly of language models from layers ("Frankenmerging")
- [Mixture of Experts merging](#mixture-of-experts-merging)
- [LORA extraction](#lora-extraction)
- [Evolutionary merge methods](#evolutionary-merge-methods)
- [Multi-stage merging](#multi-stage-merging-mergekit-multi) for complex workflows.
- [Merging of raw PyTorch models (`mergekit-pytorch`)](#raw-pytorch-model-merging-mergekit-pytorch).
## Installation
```sh
git clone https://github.com/arcee-ai/mergekit.git
cd mergekit
pip install -e . # install the package and make scripts available
```
If the above fails with the error of:
```
ERROR: File "setup.py" or "setup.cfg" not found. Directory cannot be installed in editable mode:
(A "pyproject.toml" file was found, but editable mode currently requires a setuptools-based build.)
```
You may need to upgrade pip to > 21.3 with the command `python3 -m pip install --upgrade pip`.
## Community & Support
- **Issues**: [GitHub Issues](https://github.com/arcee-ai/mergekit/issues)
- **Discussions**:Excerpt of 21,097 characters
Read on GitHub
249
3
Traun Leyden · Founder · Germany
3
3
3
Thomas Gauthier-Caron
2
2
2
Shamane Siri · Australia
2
Zhipeng Huang · China
1
1
1
1
1
1
1
1
1
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:f3b282b84b8d630e, topic:llm, topic:llama