Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Reliable, minimal and scalable library for pretraining foundation and world models
| Date | Stars |
|---|---|
| 2026-07-24 | 291 |
| 2026-07-25 | 293 |
| 2026-07-28 | 295 |
| 2026-07-30 | 295 |
| 2026-08-06 | 295 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
35.0
growth rate 0.00%/day
# stable-pretraining
[](https://galilai-group.github.io/stable-pretraining/)
[](https://github.com/galilai-group/stable-pretraining/tree/main/benchmarks)
[](https://github.com/galilai-group/stable-pretraining/actions/workflows/testing.yml)
[](https://pytorch.org/get-started/locally/)
[](#jax-backend)
[](https://github.com/astral-sh/ruff)
[](https://opensource.org/licenses/MIT)
[](https://wandb.ai/site)
**PyTorch Lightning for foundation-model research.** Dict-shaped state
so any intermediate tensor is loggable, live evaluation probes that
attach without touching the training loop (`OnlineProbe`, `OnlineKNN`,
`RankMe`, `LiDAR`, …), SLURM-grade requeue + atomic checkpoints +
queryable run registry, GPU-side batched augmentation, and 30+ ready
recipes spanning SSL, supervised, and multi-modal pretraining
(SimCLR, DINO/DINOv2, MAE, BYOL, VICReg, Barlow Twins, LeJEPA, CLIP, …).
An **experimental [JAX / Flax-NNX backend](#jax-backend)** mirrors the same
design (forward-dict, callbacks, SLURM-grade Manager) for users who prefer JAX.
[30-second tour ↓](#30s-tour) · [JAX backend ↓](#jax-backend) ·
[Built-in methods ↓](#built-in-methods) · [Discord](https://discord.gg/adzpqWKM25)
## Table of Contents
- [How?](#how)
- [Quick Setup](#quick-setup)
- [Tutorial Notebook](#quick-setup)
- [30-second tour](#30s-tour)
- [JAX backend (experimental)](#jax-backend)
- [Core Structure](#core-structure)
- [Data](#data)
- [Module](#module)
- [Callbacks](#callbacks)
- [Trainer](#trainer)
- [Global Configuration](#global-configuration)
- [Output Directory (`cache_dir`)](#output-directory)
- [Run Registry](#run-registry)
- [Built-in Methods](#built-in-methods)
- [Backbones](#backbones)
- [Optimizers & Schedulers](#optimizers-schedulers)
- [Complete Example](#complete-example)
- [Quick Start with `spt` CLI](#spt-cli)
- [Installation](#installation)
- [Contributing](#contributing)
- [Citation](#citation)
- [Contributors](#contributors)
- [Benchmarks](#benchmarks)
<a id="how"></a>
## How?
To reach flexibility, scalability and stability, we rely on battle-tested third party libraries: `PyTorch`, `Lightning`, `HuggingFace`, `TorchMetrics` amongst a few others. Those dependencies allow us to focus on assembling everything into a powerful ML framework. ``stable-pretraining`` adopts a flexible and modular design for seamless integration of components from external libraries, including architectures, loss functions, evaluation metrics, and augmentations.
<a id="quick-setup"></a>
## Quick setup
```bash
# Clone the repository
git clone https://github.com/galilai-group/stable-pretraining.git
# Install the framework
cd stable-pretraining
pip install -e .
```
For an interactive walkthrough — data loading, Module, callbacks, training, and evaluation all in one place — open the tutorial notebook:
```bash
jupyter notebook examples/simclr_cifar10_tutorial.ipynb
```
<a id="30s-tour"></a>
## 30-second tour
The whole framework is four components that pass **dicts** to each other.
Once you see the shape, the rest of the README is reference.
```mermaid
%%{init: {'theme':'base', 'themeVariables': {
'fontFamily':'-apple-system, BlinkMacSystemFont, system-ui, sans-serif',
'fontSize':'14px',
'primaryColor':'#fafafa',
'primaryTextColor':'#1f2937',
'primaryBorderColor':Excerpt of 57,081 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:d8a0a3fd669d82b8, topic:deep-learning, topic:pytorch, name:pretraining