Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Official PyTorch Implementation of "Learning to Learn with Generative Models of Neural Network Checkpoints"
| Date | Stars |
|---|---|
| 2026-07-31 | 347 |
| 2026-08-06 | 347 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
## Learning to Learn with Generative Models of Neural Network Checkpoints<br><sub>Official PyTorch Implementation</sub> ### [Paper](http://arxiv.org/abs/2209.12892) | [Project Page](https://www.wpeebles.com/Gpt)   This repo contains training, evaluation, and visualization code for our recent paper exploring loss-conditional diffusion models of neural network parameters. > [**Learning to Learn with Generative Models of Neural Network Checkpoints**](https://www.wpeebles.com/Gpt)<br> > [William Peebles*](https://www.wpeebles.com), [Ilija Radosavovic*](https://people.eecs.berkeley.edu/~ilija/), > [Tim Brooks](https://www.timothybrooks.com), [Alexei A. Efros](http://people.eecs.berkeley.edu/~efros/), > [Jitendra Malik](http://people.eecs.berkeley.edu/~malik/) > <br>University of California, Berkeley<br> Our generative models are conditioned on a starting parameter vector, a starting loss/error/return and a _prompted_ loss/error/return. With these inputs, we can sample an updated parameter vector that ideally achieves the prompt. We call our model G.pt (G and .pt refer to generative models and checkpoint extensions, respectively). The core of G.pt is a transformer model that operates over sequences of parameters from the input neural network parameters. Similar to ViTs, G.pt leverages very few domain-specific inductive biases (only in tokenization and data augmentation). The transformer is trained as a diffusion model directly in parameter space. After training, G.pt can optimize neural networks from random initialization in one step by prompting for a small loss/error or high return. In this paper, we introduce G.pt models for optimizing MNIST MLPs, CIFAR-10 CNNs and Cartpole Gaussian MLPs. This repository contains: * ⚡️ Five pre-trained G.pt DDPM Transformers for vision and RL tasks * 🪐 A dataset containing over 23M neural net checkpoints across 100K+ training runs * 💥 Training and testing scripts for G.pt models ## Setup First, download and set up the repo: ```bash git clone https://github.com/wpeebles/G.pt.git cd G.pt pip install -e . ``` We provide an [`environment.yml`](environment.yml) file that can be used to create a Conda environment: ```bash conda env create -f environment.yml conda activate G.pt ``` If you opt to use your own environment, you'll need Python 3.8 in order to run the IsaacGym RL simulator (newer versions of Python may have compatibility issues). You'll also want to set up a [Weights & Biases](https://wandb.ai/site) account since some of our visualization code uses it. Finally, in order to train or evaluate RL G.pt models, you'll need to install IsaacGym. [Download it here](https://developer.nvidia.com/isaac-gym), then install it: ```bash cd /path/to/isaac-gym/python pip install -e . export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/anaconda3/envs/G.pt/lib ``` ## Checkpoint Pre-training Data We provide three checkpoint datasets, in aggregate containing over 23M checkpoints from 100K+ training runs. Each individual checkpoint contains neural network parameters and any useful task-specific metadata (e.g., test losses and errors for classification, episode returns for RL). If you run our G.pt testing scripts (explained [below](#evaluating-gpt-models)), the relevant checkpoint data will be auto-downloaded. Or, you can download all three checkpoint datasets (and the five pre-trained G.pt models) by running: ```python python Gpt/download.py ``` This will store all three datasets in a folder named `checkpoint_datasets`. The breakdown for each dataset is as follows: | Checkpoint Dataset | # Checkpoints | # Runs | # Checkpoints/Run | Storage (GB) | |------------------------|---------------|---------|-------------------|--------------| | MNIST MLPs | 2.1M | 10728 | 200 | 68 | | CIFAR-10 CNNs | 11.3
Excerpt of 10,050 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:443e3eacc32d1a7c, topic:deep-learning, topic:pytorch
matched fp:443e3eacc32d1a7c, topic:diffusion-models