Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Fine-tune mistral-7B on 3090s, a100s, h100s
| Date | Stars |
|---|---|
| 2026-07-31 | 735 |
| 2026-08-05 | 737 |
| 2026-08-06 | 737 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# fine-tune-mistral Code used to fine-tune this model: [abacaj/mistral-7b-sft](https://huggingface.co/abacaj/mistral-7b-sft). Add your data in the data folder as `train.jsonl` and `validation.jsonl`. **Note** this repo is intended for full fine-tuning of mistral not qlora or other methods. # How to run Install dependencies: ``` python -m venv env \ && source env/bin/activate \ && pip install -r requirements.txt ``` [Get a Hugging Face token](https://huggingface.co/settings/tokens) and set the variable: ``` export HF_TOKEN="[insert token here]" ``` Run training code: ``` torchrun --nnodes=1 --nproc-per-node=<REPLACE_WITH_NUMBER_OF_GPUS> train.py ``` # Tips - If running with a small batch size, lower the learning rate - I did not have to adjust grad clip or weight_decay but YMMV - Use enough data, I recommend > 1k samples - I ran this for 3 epochs on 40k samples, will need to experiment more on epochs because the model was still improving. - The better way to tell if your model is improving or just overfitting or even getting worse, you should add evaluation on your task. This is data that is not part of training. For example, on code completion you can evaluate your model on the mbpp validation set or a custom set you have. - Use FSDP option: `backward_prefetch=BackwardPrefetch.BACKWARD_PRE` if you have the GPU memory, or `backward_prefetch=BackwardPrefetch.BACKWARD_POST`. This can cause OOM so it was set to None
Excerpt of 1,449 characters
Read on GitHub
32
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:daeae87c3be96271, llm:description: "Fine-tune mistral-7B on 3090s, a100s, h100s"
matched fp:daeae87c3be96271, llm:description: "Fine-tune mistral-7B on 3090s, a100s, h100s"