English • 中文 (earlier version) • 日本語 (earlier version) • 한국어 (earlier version)
An open textbook series on how machine learning systems are built and why they are built that way, with companion tools for building, measuring, and teaching them.
Build status on dev
📘 Textbook Series •
📗 Vol I •
📘 Vol II (Preview) •
🟣 Vol III (Draft) •
🌲 Vol IV (Draft)
🔥 TinyTorch •
🔬 Labs •
🧰 Kits •
🔮 MLSys·im •
🎓 Instructors •
💼 StaffML •
Taxonomy
📚 Volume I is forthcoming in print from MIT Press.
The world is rushing to build AI systems. It is not yet engineering them.
A model that scores well on a benchmark still has to fit in memory, meet a latency budget, stay within a power limit, and fail safely in front of real people. Those constraints decide the architecture, and they follow laws that outlast any framework or chip. Memory bandwidth, network latency, energy, and failure rates govern a microcontroller with 256 KB of memory and an accelerator in a datacenter alike, even though the two differ in scale by many orders of magnitude. I learned this while working on TinyML at Google, and it became the idea that runs through every volume: constraints drive architecture.
AI engineering is the discipline of building efficient, reliable, and safe intelligent systems that operate in the real world, not just models in isolation. My aim is to help establish it alongside software engineering and computer engineering, and to teach it the way computer architecture has long been taught, from first principles and with numbers a student can recompute.
Everything lives in one repository because each part teaches something the others cannot. Students who only read do not internalize, and students who only code do not generalize. The textbook builds the mental model, TinyTorch makes it concrete by having students build the machinery themselves, and the hardware kits and the MLSys·im simulator show where the model breaks, on devices students can hold and at scales they cannot rent.
The repository is the curriculum.
The work began in a classroom. In 2023, the students in CS249r at Harvard wrote the first class notes together, one topic per group, and we published them on GitHub. Those notes have been rewritten every semester since, and the HarvardX TinyML course that preceded them reached more than 100,000 learners. The goal now is one million learners by 2030.
There is also a personal reason, which the FAQ tells in full. As an undergraduate, I wrote to a professor I had never met and received a VHS tape in the mail that changed what I studied. This curriculum is my way of sending that package back out.
Each chapter connects an idea a reader may already know from machine learning to the systems that make it work in practice.
| You know... | You will learn... | |
|---|---|---|
| How to train a model | → | How training scales across accelerator clusters |
| That quantization shrinks models | → | How low-precision arithmetic maps to silicon |
| What a transformer is | → | Why the KV cache dominates memory at inference |
| Models run on GPUs | → | How schedulers balance latency against throughput |
| Models need good data | → | How data pipelines, selection, and drift decide what a model can learn |
| Models get deployed | → | How to benchmark, monitor, and roll back models in production |
| Hardware fails sometimes | → | How to checkpoint and recover training jobs that span thousands of accelerators |
| Edge devices have limits | → | How to co-design models and hardware |
| AI uses a lot of energy | → | How to measure and reduce the energy and carbon cost of a system over its full lifecycle |
| How to prompt an LLM | → | How to architect agent loops, tool sandboxes, and context memory (Vol III, draft) |
| How vision and robotics models infer | → | How to close real-time sensor-actuator loops under physical constraints (Vol IV, draft) |
Each volume widens the boundary of the system and raises the cost of failure. The distinction between volumes is scope, not depth; every volume is equally quantitative.
| Volume | Unit of work | Core systems question | Status | |
|---|---|---|---|---|
| 📗 | I. Introduction to Machine Learning Systems | The model | How does intelligence execute efficiently on a single machine? | Released; forthcoming in print from MIT Press |
| 📘 | II. Scaling Machine Learning Systems | The fleet | How does intelligence scale across distributed clusters and datacenters? | Preview; in active revision |
| 🟣 | III. Agentic Machine Learning Systems | The trajectory | How is intelligence governed when it acts autonomously over long horizons? | Draft; changes often |
| 🌲 | IV. Physical AI: Machine Learning Systems That Sense and Act | The physical action | How can intelligence act safely on matter and physical systems? | Draft; changes often |
Each volume answers a question the previous one leaves open:
- Vol I → Vol II, when one machine is not enough. Volume I covers single-machine execution, memory walls, and kernel efficiency. Volume II carries those foundations across thousands of accelerators, collective communication, fault-tolerant networks, and datacenter orchestration.
- Vol II → Vol III, when a stateless response is not enough. Volume II serves stateless requests at scale. Volume III adds stateful, multi-step loops that manage context memory, call tools, run in sandboxes, and recover from nondeterministic failures.
- Vol III → Vol IV, when software acts on the physical world. Volume III governs software environments. Volume IV crosses into physical systems, where computational latency becomes distance traveled and a failure cannot be undone.
⚠️ Volumes III and IV are open drafts. Chapters are added, reorganized, and rewritten as I iterate, so please do not cite or teach from them yet. Feedback is welcome through the book feedback issue forms.
The pieces are designed to work together, but none of them requires the others.
| If you are... | Start here | Then go deeper |
|---|---|---|
| A student or self-learner | Read Volume I | Build TinyTorch, model systems with MLSys·im, try the labs (early release), and practice with StaffML |
| An instructor | Open the Instructor Hub | Pick a course and use its syllabus, slides, kits, and labs |
| A contributor | Pick the component you use most | See Contributing |
| Component | What it does | Status | Links | |
|---|---|---|---|---|
| 📖 | Textbook | Four volumes that build the mental models, from one machine to fleets, agents, and physical systems. | See above | Read online |
| 🔥 | Tiny🔥Torch | Build an ML framework from scratch in 20 progressive modules, from tensors to a transformer. | Released | Get started · Repo guide |
| 🔮 | MLSys·im | Model memory, compute, network, and energy limits at scales no student can rent. The textbook's calculations are built on it. | Released | Use simulator · Repo guide |
| 🛠️ | Hardware Kits | Deploy models to Arduino, Seeed, Grove, and Raspberry Pi devices, with real memory limits, power budgets, and latency. | Released | Browse kits · Repo guide |
| 🔬 | Labs | Interactive Marimo notebooks, run in the browser, that pair with textbook chapters. A student predicts an outcome, changes a parameter, and sees what breaks. | Early release; built in the open | Try labs · Repo guide |
| 💼 | StaffML | More than 9,000 systems design questions across cloud, edge, mobile, and TinyML, with practice drills and mock interviews. | Early release | Practice · Repo guide |
| 🎓 | Instructor Hub | A syllabus for each course, with weekly readings and slides: introductory, scaling, TinyML, Edge AI, Physical AI (draft), and Agentic AI (in preparation). | Released | View hub · Repo guide |
| 🎬 | Lecture Slides | Beamer decks for each chapter of all four volumes, in one shared theme that is easy to restyle. | Released | Browse decks · Repo guide |
| 📬 | Newsletter | Updates on new chapters, releases, and what the community is building. | Ongoing | Subscribe |
Two projects are earlier-stage experiments: SocratiQ, an AI tutor that adds guided reading, quizzes, and spaced repetition to static learning sites, and MLPerf EDU, a teaching benchmark suite aligned with MLCommons MLPerf.
No chapter is written once. Each one goes through a cycle that repeats as the field changes.
- Watch the field. I follow where machine learning systems are going through the research and industry venues where that work appears: ISCA, MICRO, HPCA, ASPLOS, ISPASS, MLSys, NeurIPS, ICML, ICLR, Hot Chips, and industry summits.
- Distill. A chapter keeps the principles and leaves out the variations. Quantization has been implemented in countless ways, and the principles that explain all of them are what a student needs. David Patterson, who wrote the foreword to Volume I, gave me the test for what belongs: a technique earns a place only when several companies use it in practice. That test keeps the material tied to systems that exist, so a student can connect what they learn to the real world.
- Review with practitioners. I review chapters one-on-one with the engineers who build these systems inside large companies: hyperscale infrastructure engineers and model developers. Practitioners at Arm, Google, Intel, Meta, Microsoft, Qualcomm, STMicroelectronics, and other organizations have reviewed the material, and academic reviewers in Europe, Asia, and the Americas have tested it against their own research and teaching.
- Teach it. Volume I has been taught at Harvard through CS249r since 2023 and adopted by instructors at UC Berkeley, Columbia, Georgia Tech, IIT Bombay, and universities across Europe, Latin America, Africa, and Asia. The classroom shows where students get stuck.
- Revise. Reader issues, reviewer notes, and classroom experience go back into the next draft.
Reviewers do more than check facts; they change what the book covers. Conversations with industry experts on sustainable AI, who kept asking where the data comes from and what a system costs over its full lifecycle, led to a chapter I had not planned. That chapter was written in 2023, before sustainable AI was a common topic.
The cycle leaves a record. Eleven of Volume I's sixteen chapters began as the 2023 class notes, three were added in 2026 as the field moved, and the chapters have been revised in more than 16,000 commits, all public.
Numbers are computed, not typed. Every derived quantity in the books comes from Python code that ships with the text, built on MLSys·im, whose hardware and model specifications trace to cited sources. A test suite runs on every commit and checks unit consistency, cross-references, notation, and citations.
AI tools are part of the workflow, and the responsibility is mine. I researched, wrote, and rewrote every chapter. I used AI tools throughout: to write code and checks, to survey literature I then read in the original, to draft passages I then rewrote substantially, and to audit the manuscript for consistency. AI-generated material can be plausible and still be wrong, which is why the standard is traceability rather than trust. Volumes III and IV are released earlier, as drafts, before they go through the classroom teaching and expert review that Volumes I and II have had. Each volume includes a full Note on AI Assistance.
Readers are part of the review. Every chapter is open. When something is wrong, unclear, or missing, the book feedback forms send it straight into the next revision.
Why do you do all this?
I have always learned best by building from scratch and teaching others.
When I want to understand an emerging or complex systems area, writing about it and writing code for it forces me to slow down. It forces me to find the core invariants, work through the math, and separate what is fundamentally true about physics and hardware from whatever happens to be trending this week.
This repository is my open workshop and teaching lab. It is how I clarify my own thinking for my students at Harvard, my research lab, and open benchmarking efforts like MLPerf.
There is also a deeper reason. In the late 1990s, as an undergraduate, I wrote to a professor at Stanford because I was fascinated by genetic algorithms. I did not expect an answer. Then a brown package arrived at my house with a VHS tape on genetic algorithms inside. I never learned who sent it, so I never got to say thank you. But I have not forgotten what it felt like to have someone I had never met take the time to share what they knew with a curious student. It sparked my curiosity, and it stayed with me.
Students write to me now the way I once wrote to that professor, far more than I can answer one by one. This curriculum is my way of sending the package back out, to a much bigger classroom than any lecture hall can hold.
Who is this for, and what should I know first?
It is for anyone who wants to understand how intelligent systems actually run, not only how models are trained: students, software engineers moving into ML infrastructure, hardware architects, and educators designing courses.
The only assumptions are that the reader can program in Python and has met basic machine learning ideas, such as a neural network or a loss function. The systems concepts are built from the ground up. No prior background in computer architecture, networking, or distributed systems is needed.
Do I need Volume I before Volume II?
No. The two volumes differ in scope, not in depth or rigor, and both are equally quantitative.
Volume I is the single-machine world: how an ML system works on one machine with a handful of accelerators, from the memory hierarchy and kernel efficiency to quantization and on-device deployment. Volume II is the fleet: thousands of accelerators connected by high-speed networks, parallelism across them, fault tolerance, and serving at datacenter scale.
Volume II does not assume that a reader has finished Volume I, so a reader with strong single-machine foundations can start there. Reading Volume I first gives the cleanest mental model from the silicon up, in the same way that Hennessy and Patterson's Computer Organization and Design precedes Computer Architecture: A Quantitative Approach.
Do I need TinyTorch, the labs, and the kits, or can I just read the book?
Each volume stands on its own as a textbook. The companions are there for readers who want to go further. Reading builds the mental model, and building and measuring make it last. Start with the book and add the hands-on pieces when an idea needs to become practice.
Isn't this just a deep learning book?
Books such as Goodfellow, Bengio, and Courville's Deep Learning, Bishop's texts, d2l.ai, and fast.ai teach the mathematics, architectures, and optimization of models, and they teach them well.
This curriculum starts where those books stop. It treats the model as one component inside a system that must ingest data, run on real silicon under power and thermal limits, answer queries within milliseconds, and keep running across thousands of machines. A reader can finish an excellent deep learning course and still not know why a cluster idles at 30% utilization, why serving memory grows with the KV cache, or how gradients are synchronized without stalling the network. Those questions are the subject here.
Isn't this MLOps, or the same as Designing Machine Learning Systems?
MLOps books, such as Chip Huyen's Designing Machine Learning Systems, focus on operational pipelines: feature stores, workflow orchestration, and deployment with current industry tools. They are valuable practical guides.
This curriculum focuses on what lies underneath: the hardware limits, quantitative trade-offs, and first principles that explain why those tools are designed as they are, and the limits no tool escapes, such as memory bandwidth, network latency, energy, and failure rates.
The difference resembles that between a recipe and an understanding of how heat, salt, fat, and time interact. A recipe works until the oven changes or the batch grows tenfold. Understanding the chemistry lets a cook work in any kitchen, fix a dish that went wrong, and invent new ones.
How is this different from The Datacenter as a Warehouse-Scale Computer?
Barroso, Hölzle, and Clidaras give a firsthand account of how one organization built hyperscale infrastructure, and I admire that work. This curriculum aims to complement it in three ways:
- A progressive curriculum, not a reference. It starts from a single machine and its memory hierarchy (Volume I) and builds up to multi-node clusters and warehouse-scale fleets (Volume II), with quantitative exercises along the way.
- Vendor-neutral foundations. It generalizes across accelerator architectures (GPUs, TPUs, custom ASICs) and deployment targets (cloud clusters to milliwatt microcontrollers), so the principles carry across hardware generations.
- Hands-on and living. The text is surrounded by code readers build themselves (TinyTorch), open hardware kits, cluster modeling (MLSys·im), and community contributions, and it changes as the field does.
Why read a textbook in the age of LLMs?
Language models are very good at retrieval, code generation, and summarizing facts. A paragraph that only delivers a fact a reader could look up in seconds has not earned its place in a textbook.
What a textbook provides is perspective and a coherent mental model: the order in which to learn ideas, how trade-offs constrain one another, and what not to worry about. Deciding what to leave out is one of the most important choices an author makes.
Bruce Davie made this point in his essay “Textbooks in Tokenland” (Systems Approach): a language model generates text without communicative intent, while a textbook is a deliberate path from a teacher to a student toward a shared understanding of how the world works.
The two are not in opposition. The book provides the structure and the quantitative argument, and interactive tools such as the Marimo labs let a reader ask questions and test understanding.
Is it free, and how do I read it?
Yes. Every volume is free to read online at mlsysbook.ai.
The textbook is released under a Creative Commons license (CC BY-NC-ND 4.0), the same license as the MIT Press print edition. It may be read and shared for non-commercial use with attribution; translations and other modified versions need permission. The companion software is open source under its own licenses, listed under License.
|
Star the repository Stars help show universities and foundations that this work matters, and that support funds workshops and hardware kits for classrooms that cannot buy them. |
Fund the mission Contributions go to Open Collective, a transparent fund for educational outreach, where every expense is public. 
|
| I want to... | Go here | |
|---|---|---|
| 📖 | Fix a typo or improve a chapter | Textbook contributing guide |
| 🔥 | Add a TinyTorch module or fix a bug | TinyTorch contributing guide |
| 🛠️ | Improve the hardware kits | Hardware kits guide |
| 🔬 | Improve the labs or simulator models | Labs guide · MLSys·im guide |
| 💼 | Improve assessment or interview content | StaffML guide |
| 🧠 | Improve AI learning tools | SocratiQ guide |
| 🐛 | Report an issue | GitHub Issues |
| 💬 | Ask a question | GitHub Discussions |
Branches. Development happens on dev, which this page shows. The main branch holds what is live on mlsysbook.ai, and the publish workflow promotes dev to main at each release.
This is a multi-component repository, and each component is released under its own license to match its purpose. The file inside each directory (e.g. tinytorch/LICENSE, staffml/app/LICENSE) is authoritative.
| Component | License | What it means |
|---|---|---|
Textbook (books/) |
CC BY-NC-ND 4.0 | The license of the MIT Press edition. Share for non-commercial use with attribution; modified versions, including translations, need permission. |
Labs (labs/), Kits (kits/), Slides (slides/), Instructors (instructors/) |
CC-BY-NC-SA 4.0 | Share and adapt for non-commercial use, with attribution and same-license sharing. |
| TinyTorch | MIT | Permissive — use, modify, redistribute, including commercially. |
| MLSys·im | Apache 2.0 | Permissive with explicit patent grant. |
| StaffML | AGPL v3 | Strong copyleft — modifications to deployed services must be published. Commercial licensing available; contact the authors. |
| StaffML question corpus | CC BY-NC 4.0 | Research and educational use; commercial use requires permission. |
| TinyDigits dataset | BSD 3-Clause | Permissive (matches sklearn ancestry). |
| TinyTalks dataset | CC BY 4.0 | Permissive with attribution; commercial use allowed. |
A user-facing summary lives at mlsysbook.ai/about/license.
If you are an institution considering adoption, or a company interested in commercial terms for a copyleft component, please reach out to [email protected].
Thanks goes to these wonderful people who have contributed to making this resource better for everyone!
Legend: 🪲 Bug Hunter · 🧑💻 Code Contributor · ✍️ Doc Wizard · 🎨 Design Artist · 🧠 Idea Spark · 🔎 Code Reviewer · 🧪 Test Tinkerer · 🛠️ Tool Builder
 Vijay Janapa Reddi 🪲 🧑💻 🎨 ✍️ 🧠 🔎 🧪 🛠️ |
 Rocky 🪲 🧑💻 🎨 ✍️ 🧪 🛠️ |
 kai 🪲 🧑💻 🎨 ✍️ 🧪 |
 Dang Truong 🪲 🧑💻 ✍️ 🧪 |
 Ng Bo Lin 🪲 🧑💻 ✍️ 🧪 |
 Farhan Asghar 🪲 🧑💻 🎨 ✍️ |
 Vedant Joshi 🪲 🧑💻 ✍️ 🧪 |
 Didier Durand 🪲 🧑💻 ✍️ |
 rnjema 🧑💻 ✍️ 🛠️ |
 AndreaMattiaGaravagno 🪲 🧑💻 ✍️ |
 Pratham Chaudhary 🪲 🧑💻 ✍️ |
 asgalon 🪲 🧑💻 ✍️ |
 bdub 🪲 🧑💻 ✍️ |
 Satnam Singh 🧑💻 ✍️ 🧪 |
 Karthik Dani 🪲 🧑💻 |
 Avik De 🪲 🧪 |
 Takosaga 🪲 ✍️ |
 joeswagson 🧑💻 🛠️ |
 Rolds 🪲 🧑💻 |
 Ademola Arigbabuwo 🪲 ✍️ |
 Harish 🧑💻 ✍️ |
 Chufeng JIANG 🪲 🧑💻 |
 Fábio 🧑💻 ✍️ |
 Aadityansha 🪲 🧑💻 |
 euwyngoh 🧑💻 ✍️ |
 Pratyush Adhikari 🧑💻 🧪 |
 diegosaldonid-lab 🪲 🧑💻 |
 Choy Rim 🪲 🧑💻 |
 Nhan Nguyen 🪲 🧑💻 |
 ehhholi ✍️ 🧪 |
 Amir Alasady 🪲 |
 jettythek 🧑💻 |
 wzz 🪲 |
 keo-dara 🪲 |
 Wayne Norman 🪲 |
 Ilham Rafiqin 🪲 |
 Oscar Flores ✍️ |
 Pastor Soto ✍️ |
 Salman Chishti 🧑💻 |
 Aditya Mulik ✍️ |
 Yaroslav Halchenko 🧑💻 |
 Rin 🧑💻 |
Vijay Janapa Reddi 🧑💻 🎨 ✍️ 🧠 |
Rocky 🪲 🧑💻 🎨 ✍️ |
Farhan Asghar 🪲 🧑💻 🎨 ✍️ |
Aadityansha 🪲 🧑💻 🧪 |
 Shivtej Gaikwad 🪲 🧑💻 🧪 |
 Tong Lin 🧑💻 ✍️ 🧪 |
 lynnyulinlin-debug 🪲 🧑💻 🧠 |
Peter Koellner 🪲 ✍️ |
 Zeljko Hrcek 🧑💻 |
 nyxst4ck ✍️ |
Rocky 🪲 🧑💻 🎨 🧪 🛠️ |
Farhan Asghar 🪲 🧑💻 🎨 ✍️ 🧪 |
Vijay Janapa Reddi 🎨 ✍️ 🧠 |
 Pelin Balcı 🎨 🧠 🧪 |
 Agustin Torres 🪲 🧑💻 ✍️ |
 didi 🧑💻 🎨 |
Vijay Janapa Reddi 🪲 🧑💻 🎨 ✍️ 🧪 🛠️ |
 Marcelo Rovai ✍️ 🧑💻 🎨 |
Rocky 🪲 🧑💻 🎨 ✍️ |
Farhan Asghar 🪲 🧑💻 🎨 |
 Gergely Imreh 🪲 ✍️ |
Salman Chishti 🧑💻 |
Pratham Chaudhary 🧑💻 |
Rocky 🪲 🧑💻 🎨 ✍️ 🧪 |
Farhan Asghar 🪲 🧑💻 🎨 ✍️ |
Vijay Janapa Reddi 🧑💻 🎨 ✍️ |
Aadityansha 🪲 🧑💻 🧪 |
Peter Koellner 🪲 🧑💻 |
Salman Chishti 🧑💻 |
Pratham Chaudhary 🧑💻 |
Rocky 🪲 🧑💻 🎨 ✍️ |
Vijay Janapa Reddi 🧑💻 🎨 ✍️ |
Farhan Asghar 🪲 🧑💻 🎨 |
Farhan Asghar 🪲 🧑💻 🎨 ✍️ 🧪 |
Rocky 🪲 🧑💻 🎨 ✍️ 🔎 |
Vijay Janapa Reddi 🧑💻 🎨 ✍️ |
 coyaSONG 🪲 🧑💻 ✍️ |
✉️ Subscribe • 💬 Join discussions • 🌐 Visit mlsysbook.ai
Made with ❤️ for AI engineers
in the making, around the world 🌎
