Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
An open collection of implementation tips, tricks and resources for training large language models
| Date | Stars |
|---|---|
| 2026-07-31 | 503 |
| 2026-08-01 | 503 |
| 2026-08-02 | 503 |
| 2026-08-06 | 503 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# 📖 The Large Language Model Training Playbook This playbook is a companion to the [LLM Training Handbook](https://github.com/huggingface/llm_training_handbook) which contains a lot more details and scripts. An open collection of implementation tips, tricks and resources for training large language models. The following covers questions related to various topics which are interesting or challenging when training large language models. ## [Deciding on a model architecture](./architecture/) ## Deciding on a model parallelism strategy ## Deciding on the model size #### Scaling laws #### Trade-off of large language model sizes ## Issues and questions related to tensor precision ### What to chose between fp32, fp16, bf16 ### Mixed-precisions for optimizers, weights, specifics modules ### How to finetune and integrate a model trained in a precision in another precision ## [Selecting training hyper-parameters and model initializations](./hparams) ### Learning rate and learning rate schedules ### Questions on batch size ## [Maximizing throughput](./throughput) ## [Avoiding, recovering from and understanding instabilities](./instabilities) ### Detecting instabilities early ### Training tips to reduce instabilities ## Issues with data and data processing ## [Debugging software and hardware failures](./debug/) ## Tips on what metrics to follow during the training ## [Resources](./resources/)
Excerpt of 1,428 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:3b7d42fafe741e5e, topic:large-language-models, topic:llm
matched fp:3b7d42fafe741e5e, topic:cuda
matched fp:3b7d42fafe741e5e, topic:pytorch
matched fp:3b7d42fafe741e5e, topic:nlp
