Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Machine Learning Engineering Open Book
| Date | Stars |
|---|---|
| 2026-07-24 | 18462 |
| 2026-07-25 | 18464 |
| 2026-07-28 | 18464 |
| 2026-07-30 | 18464 |
| 2026-07-31 | 18495 |
| 2026-08-06 | 18523 |
Today
+28 stars today
This week
+59 stars this week
This month
— stars this month
Momentum
171.0
growth rate 0.32%/day
# Machine Learning Engineering Open Book This is an open collection of methodologies, tools and step by step instructions to help with successful training and fine-tuning of large language models and multi-modal models and their inference. This is a technical material suitable for LLM/VLM training engineers and operators. That is the content here contains lots of scripts and copy-n-paste commands to enable you to quickly address your needs. This repo is an ongoing brain dump of my experiences training Large Language Models (LLM) (and VLMs); a lot of the know-how I acquired while training the open-source [BLOOM-176B](https://huggingface.co/bigscience/bloom) model in 2022 and [IDEFICS-80B](https://huggingface.co/HuggingFaceM4/idefics-80b-instruct) multi-modal model in 2023, and RAG models at [Contextual.AI](https://contextual.ai/) in 2024. I've been compiling this information mostly for myself so that I could quickly find solutions I have already researched in the past and which have worked, but as usual I'm happy to share these notes with the wider ML community. ## Table of Contents **Part 1. Insights** 1. **[The AI Battlefield Engineering](./insights/ai-battlefield.md)** - what you need to know in order to succeed. 1. **[How to Choose a Cloud Provider](./insights/how-to-choose-cloud-provider.md)** - these questions will empower you to have a successful compute cloud experience. 1. **[When Is It Worth Upgrading GPUs?](./insights/when-to-upgrade-gpus/README.md)** - a practical framework for deciding whether a GPU generation upgrade is worth its cost, worked through on a real H200 → B200 benchmark. **Part 2. Hardware** 1. **[Compute](compute)** - accelerators, CPUs, CPU memory. 1. **[Storage](storage)** - local, distributed and shared file systems. 1. **[Network](network)** - intra- and inter-node networking. **Part 3. Orchestration** 1. **[Orchestration Systems](orchestration)** - managing containers and resources 1. **[SLURM](orchestration/slurm)** - Simple Linux Utility for Resource Management **Part 4. Training** 1. **[Training](training)** - model training-related guides **Part 5. Inference** 1. **[Inference](inference)** - model inference insights **Part 6. Development** 1. **[Debugging and Troubleshooting](debug)** - how to debug easy and difficult issues 1. **[And more debugging](https://github.com/stas00/the-art-of-debugging)** 1. **[Testing](testing)** - numerous tips and tools to make test writing enjoyable **Part 7. Miscellaneous** 1. **[Resources](resources)** - LLM/VLM chronicles ## Updates I announce any significant updates on my twitter channel [https://twitter.com/StasBekman](https://twitter.com/StasBekman). ## Ebook versions of the book You can download various ebook formats of this book: * [PDF](https://huggingface.co/stas/ml-engineering-book/resolve/main/Stas%20Bekman%20-%20Machine%20Learning%20Engineering.pdf?download=true) * [EPUB](https://huggingface.co/stas/ml-engineering-book/resolve/main/Stas%20Bekman%20-%20Machine%20Learning%20Engineering.epub?download=true) I will try to rebuild these once in a few weeks or so, but if you want the latest ebook versions, the instructions for building are [here](build). Thanks to HuggingFace for giving me permission to host my book's ebook formats at the [HF hub](https://huggingface.co/stas/ml-engineering-book). ## SKILL.md for AI agents I maintain a [SKILL.md](./SKILL.md) file that you can use to teach your AI agent to train and operate large-scale ML models better. See also the companion skills: [The Art of Debugging](https://github.com/stas00/the-art-of-debugging/blob/master/SKILL.md) and [Stas' Python Cookbook](https://github.com/stas00/python-cookbook/blob/master/SKILL.md). ## Courses - **[Lessons Learned from Training LLMs](courses/lesson-learned)** - provides a very different way of reading my open books, by going over the terse learned insights and allowing you to quickly dive deeper when you need to. ## Lectures/Ta
Excerpt of 8,957 characters
Read on GitHub
Stas Bekman · Stasosphere Online Inc. / · Canada
993
Stas Bekman · Snowflake · Canada
9
Quentin Anthony
7
Jordan Nanos · @SemiAnalysis
6
Pietro Monticone · Harmonic · United States
6
Xiaoyu Zhang · RadixArk
5
Bruno Scaglione · University of São Paulo (USP) · Brazil
5
Shubham · @fuzzylabs
4
Sam Foreman · @argonne-lcf · Israel
4
Charles-Francois Natali
4
functionstackx
4
Patrick von Platen
4
Andy Yang · Tencent · China
3
HUIJONG JEONG · @SqueezeBits · South Korea
3
ambivalent · Vietnam
3
Yao Lu · NLP Group, University College London @uclnlp · United Kingdom
3
2
Victor Skvortsov · @dstackai
2
Val · France
2
Guspan Tanadi
2
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:bf675b804ed17c95, topic:pytorch, topic:training
matched fp:bf675b804ed17c95, topic:inference
matched fp:bf675b804ed17c95, topic:mlops
matched fp:bf675b804ed17c95, topic:large-language-models, topic:llm