Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
The official codes for "Aurora: Activating chinese chat capability for Mixtral-8x7B sparse Mixture-of-Experts through Instruction-Tuning"
| Date | Stars |
|---|---|
| 2026-07-31 | 261 |
| 2026-08-05 | 260 |
| 2026-08-06 | 260 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<div align="center">
<h2>
Aurora: Activating chinese chat capability for Mixtral-8x7B sparse Mixture-of-Experts through Instruction-Tuning
</h2>
</div>
<!--
> [!NOTE]
> We apologize for the misnaming of the paper due to our mistake: `Mixtral-8x7B-Instruct-v0.1` was incorrectly named `Mistral-8x7B`, and `Mix` and `Mis` do not seem to be the same thing. **We will make a correction in the next release**.
-->
***Rongsheng Wang**, Haoming Chen, Ruizhe Zhou, Yaofei Duan, Kunyan Cai, Han Ma, Jiaxi Cui, Jian Li, Patrick Cheong-Iao Pang, Yapeng Wang, Tao Tan☨*
☨Corresponding author
<h5 align="center">
<a href='https://arxiv.org/abs/2312.14557'><img src='https://img.shields.io/badge/Paper-Arxiv-red'></a> <a href='https://huggingface.co/wangrongsheng/Aurora'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Models-blue'></a>
</h5>
<a href="https://trendshift.io/repositories/6402" target="_blank"><img src="https://trendshift.io/api/badge/repositories/6402" alt="WangRongsheng%2FAurora | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
> [!IMPORTANT]
>
> - **We highly recommend using our Aurora based on DPO! 👉[Here](https://github.com/WangRongsheng/Aurora/tree/dpo)** If you don't have enough GPU or tutorials to run it, we recommend you to run it with one click using the 👉[Xian Gong Cloud Aurora image](https://www.xiangongyun.com/image/detail/bbcdc87e-20de-561c-a298-4323cf2f6fb2). You can also check out our 👉[tutorial videos](https://www.bilibili.com/video/BV1xw411E7To/).
> - **We now support using Aurora locally with [Ollama](https://ollama.com/). 👉[Here](https://ollama.com/wangrongsheng/aurora)**
## Overview
Existing research has demonstrated that refining large language models (LLMs) through the utilization of machine-generated instruction-following data empowers these models to exhibit impressive zero-shot capabilities for novel tasks, without requiring human-authored instructions. In this paper, we systematically investigate, preprocess, and integrate three Chinese instruction-following datasets with the aim of enhancing the Chinese conversational capabilities of Mixtral-8x7B sparse Mixture-of-Experts model. Through instruction fine-tuning on this carefully processed dataset, we successfully construct the Mixtral-8x7B sparse Mixture-of-Experts model named "Aurora." To assess the performance of Aurora, we utilize three widely recognized benchmark tests: C-Eval, MMLU, and CMMLU. Empirical studies validate the effectiveness of instruction fine-tuning applied to Mixtral-8x7B sparse Mixture-of-Experts model. This work is pioneering in the execution of instruction fine-tuning on a sparse expert-mixed model, marking a significant breakthrough in enhancing the capabilities of this model architecture.
## Evaluation
It is known that LLM evaluation remains a significant challenge. We use three public benchmarks in our study.
Scores of different checkpoints on BLEU and ROUGE.
|Model Checkpoints|BLEU-4|ROUGE-1|ROUGE-2|ROUGE-l|
|:-|:-|:-|:-|:-|
|checkpoints-6000|18.4134|38.2669|18.9526|26.572|
|checkpoints-8000|18.3351|38.4327|19.058|26.6573|
|checkpoints-8000|18.5638|38.5497|19.1992|26.8305|
|checkpoints-12000|18.7156|38.7787|19.3347|27.0613|
|checkpoints-14000|18.5194|38.6898|19.2032|26.8863|
Aurora's performance was tested in the medical evaluation benchmark [CMB](https://cmedbenchmark.llmzoo.com/)
|Model|Avg. Scores|
|:-|:-|
|Aurora|29.87|
|Mistral-7B|22.26|
<details>
<summary>More details</summary>
```json
{
"accuracy_per_category": {
"医师考试": 0.305,
"护理考试": 0.33875,
"药师考试": 0.289375,
"医技考试": 0.30666666666666664,
"专业知识考试": 0.27875,
"医学考研": 0.27625
},
"accuracy_per_subcategory": {
"医师考试": {
"规培结业": 0.295,
"执业助理医师": 0.3175,
"执业医师": 0.3375,
"中级职称Excerpt of 15,762 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:c568597a70d89b09, topic:fine-tuning, topic:lora, topic:qlora
matched fp:c568597a70d89b09, topic:large-language-models, topic:llm, topic:gpt