Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Build a modern LLM from scratch. Every line commented. Explained like we are five.
| Date | Stars |
|---|---|
| 2026-07-24 | 2309 |
| 2026-07-25 | 2311 |
| 2026-07-28 | 2311 |
| 2026-07-30 | 2311 |
| 2026-07-31 | 2314 |
| 2026-08-06 | 2314 |
Today
— stars today
This week
+3 stars this week
This month
— stars this month
Momentum
3.0
growth rate 0.13%/day
# 🧠 How to Train Your GPT
> *A guide to building a world-class language model from absolute scratch. Taught like you're five. Built like you're an engineer.*
>
> *I made this with the goal of learning something I didn't understand completely. Specifically the attention part. I use AI a lot to understand key concepts and verifying them.*
<p align="center">
<img src="https://img.shields.io/badge/chapters-12-blue" alt="12 chapters">
<img src="https://img.shields.io/badge/lines-7%2C500%2B-green" alt="7,500+ lines">
<img src="https://img.shields.io/badge/topics_explained-28-teal" alt="28 topic explainers">
<img src="https://img.shields.io/badge/code%20commented-100%25-brightgreen" alt="100% commented">
<img src="https://img.shields.io/badge/prerequisite-python%20basics-orange" alt="Python basics only">
<img src="https://img.shields.io/badge/architecture-LLaMA%203%20style-purple" alt="LLaMA 3 style">
<img src="https://img.shields.io/badge/purpose-learning%20only-lightgrey" alt="Learning only">
<a href="https://colab.research.google.com/github/raiyanyahya/how-to-train-your-gpt/blob/master/notebooks/colab_train.ipynb">
<img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab" height="25">
</a>
</p>
---
## 📖 What Is This?
This is a **12-chapter, 7,500+ line interactive textbook** that teaches you how to build, train and run a modern language model from absolute scratch. The same family of architecture behind ChatGPT, Claude, LLaMA and Mistral.
Alongside the chapters there are **28 standalone topic explainers** covering every technique in depth. RoPE, attention, RMSNorm, SwiGLU, KV cache, AdamW, mixed precision and more. Plus two narrative walkthroughs that trace a single sentence through the entire model step by step. Each file follows the same style: child language, no jargon, a code example you can run.
You won't just read about Transformers. You'll **write every line yourself**: tokenizer, embeddings, attention, training loop, inference engine. Every single line annotated to explain **what** it does and **why** it's there.
---
## 🤔 Why This Exists
Most ML tutorials fall into one of two traps:
| ❌ Too Shallow | ❌ Too Academic | ✅ This Guide |
|---|---|---|
| `model = GPT().fit(data)` | 40-page papers, dense notation | 5-year-old analogies → full working code |
| You learn to call APIs | Assumes PhD in ML | Zero ML experience required |
| No understanding of internals | No worked examples | Every line annotated with WHAT & WHY |
**The goal:** After finishing, you won't just know that attention "works". You'll understand the variance argument behind `1/√d_k`. How RoPE captures relative position through rotation. Why pre-norm beats post-norm for deep networks. And exactly where every gradient flows during backpropagation.
---
## 👥 Who Is This For?
| 🧑💻 You Are... | 📚 You Need... |
|---|---|
| A Python developer curious about how ChatGPT actually works | Basic Python (functions, classes, lists). No ML experience |
| A student who wants to deeply understand Transformers | Willingness to read ~3,500 lines of commented code |
| An engineer evaluating LLM architectures | Understanding of tradeoffs (RoPE vs learned, RMSNorm vs LayerNorm) |
| Someone who got lost at "attention" in other tutorials | Party analogy + worked numeric example with real numbers |
**🔧 Prerequisites:** Python basics (variables, functions, classes, `pip install`). That's it. No calculus, no linear algebra, no PyTorch experience required. We teach those as we go.
---
## 🗺️ Chapters
| Chapter | What You'll Learn |
|---|---|
| **[0: Overview](chapters/00_overview.md)** | What is a GPT? The big picture |
| **[1: Setup](chapters/01_setup.md)** | Install tools, GPU vs CPU, venv, PyTorch basics |
| **[2: Tokenization](chapters/02_tokenization.md)** | BPE walkthrough: how "unbelievably" becomes tokens |
| **[3: Embeddings](chapters/03_embeddings.md)** | How numbers become meaning. king − man + woman = queenExcerpt of 17,861 characters
Read on GitHub
40
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:67b2cb59663dda22, topic:llm, topic:gpt, topic:llama
matched fp:67b2cb59663dda22, topic:deep-learning, topic:pytorch
matched fp:67b2cb59663dda22, topic:tutorial, topic:educational