Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
High Accuracy and efficiency multi-task fine-tuning framework for Code LLMs. This work has been accepted by KDD 2024.
| Date | Stars |
|---|---|
| 2026-07-31 | 712 |
| 2026-08-06 | 711 |
| 2026-08-14 | 710 |
| 2026-08-28 | 711 |
| 2026-09-03 | 712 |
| 2026-09-20 | 712 |
Today
— stars today
This week
— stars this week
This month
+2 stars this month
Momentum
0.0
growth rate 0.00%/day
# MFTCoder: High Accuracy and Efficiency Multi-task Fine-Tuning Framework
<p align="center">
<img src="./assets/github-codefuse-logo-update.jpg" width="50%" />
</p>
<div align="center">
<p>
<a href="https://github.com/codefuse-ai/MFTCoder">
<img alt="stars" src="https://img.shields.io/github/stars/codefuse-ai/MFTCoder?style=social" />
</a>
<a href="https://github.com/codefuse-ai/MFTCoder">
<img alt="forks" src="https://img.shields.io/github/forks/codefuse-ai/MFTCoder?style=social" />
</a>
<a href="https://github.com/codefuse-ai/MFTCoder/LICENCE">
<img alt="License: MIT" src="https://badgen.net/badge/license/apache2.0/blue" />
</a>
<a href="https://github.com/codefuse-ai/MFTCoder/issues">
<img alt="Open Issues" src="https://img.shields.io/github/issues-raw/codefuse-ai/MFTCoder" />
</a>
</p>
<p>
🤗 <a href="https://huggingface.co/codefuse-ai" target="_blank">HuggingFace
</a>• 🤖<a href="https://modelscope.cn/organization/codefuse-ai" target="_blank"> ModelScope
</a>
</p>
[[中文]](README_cn.md) [**English**]
</div>
## Contents
- [News](#News)
- [Articles](#Articles)
- [Introduction](#Introduction)
- [Requirements](#Requirements)
- [Training](#Training)
- [Models](#Models)
- [Datasets](#Datasets)
- [Star History](#Star-History)
- [Join Us](#Join-Us)
## News
🔥🔥🔥 [2024/10/31] We released **MFTCoder v0.5** mainly for MFTCoder-accelerate, which is now supporting preference alignment methods like **DPO/RPO/ORPO** in the new **xxpo** module, adding full-parameter continue-training in the additional **mpt** module along with its **offline_tokenization** module, updating selfpaced method to new convergence balance(CoBa) method for MFT in the original **pefts** module.
🔥🔥🔥 [2024/10/31] Our paper [CoBa: Convergence Balancer for Multitask Finetuning of Large Language Models](https://arxiv.org/abs/2410.06741) has been accepted by EMNLP-2024, which achieves balanced convergence across various tasks.
🔥🔥🔥 [2024/05/20] We released **MFTCoder v0.4**, mainly for MFTCoder-accelerate. It supports **QLoRA + DeepSpeed Zero3** and **QLoRA + FSDP** as options allowing you training very large models. It now supports new models like Qwen2, Qwen2-MoE, Starcoder2, Gemma, etc.
🔥🔥🔥 [2024/05/20] Our paper [MFTCoder: Boosting Code LLMs with Multitask Fine-Tuning](https://arxiv.org/abs/2311.02303) has been accepted by KDD2024.
🔥🔥🔥 [2024/05/20] [CodeFuse-StarCoder2-15B](https://huggingface.co/codefuse-ai/CodeFuse-StarCoder2-15B) has been released, achieving a pass@1 (greedy decoding) score of 73.2% on HumanEval.
🔥🔥 [2024/01/30] The model [CodeFuse-DeepSeek-33B](https://huggingface.co/codefuse-ai/CodeFuse-DeepSeek-33B) fine-tuned with MFTCoder ranks first in HuggingFace [Big Code Models LeaderBoard](https://huggingface.co/spaces/bigcode/bigcode-models-leaderboard)
🔥🔥 [2024/01/17] We released MFTCoder v0.3.0, mainly for MFTCoder-accelerate. It now supports new models like Mixtral(MoE), DeepSeek-coder, chatglm3. It supports FSDP as an option. It also supports Self-paced Loss as a solution for convergence balance in Multitask Fine-tuning.
🔥🔥 [2024/01/17] [CodeFuse-DeepSeek-33B](https://huggingface.co/codefuse-ai/CodeFuse-DeepSeek-33B) has been released, achieving a pass@1 (greedy decoding) score of 78.7% on HumanEval. It lists as top-1 LLM on Bigcode Leardboard in terms of win-rate, the official result is going to be published later.
🔥🔥 [2024/01/17] [CodeFuse-Mixtral-8x7B](https://huggingface.co/codefuse-ai/CodeFuse-Mixtral-8X7B) has been released, achieving a pass@1 (greedy decoding) score of 56.1% on HumanEval.
🔥🔥 [2023/11/07] [MFTCoder Paper](https://arxiv.org/abs/2311.02303) has been released on Arxiv, which discloses technique details of multi-task-fine-tuning.
🔥🔥 [2023/10/20] [CodeFuse-QWen-14B](https://huggingface.co/codefuse-ai/CodeFuse-QWen-14B) has been released, achieving a pass@1 (greedy decoding) score of 48.8% on HumanEval, which gains Excerpt of 17,296 characters
Read on GitHub
ss41979310
48
15
10
5
4
3
1
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:ba37615725a6d3ad, desc:fine-tuning, desc:fine tuning