Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Official code for our CVPR '22 paper "Dataset Distillation by Matching Training Trajectories"
| Date | Stars |
|---|---|
| 2026-07-24 | 441 |
| 2026-07-25 | 441 |
| 2026-07-28 | 441 |
| 2026-07-30 | 441 |
| 2026-08-06 | 441 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Dataset Distillation by Matching Training Trajectories
### [Project Page](https://georgecazenavette.github.io/mtt-distillation/) | [Paper](https://arxiv.org/abs/2203.11932)
<br>
This repo contains code for training expert trajectories and distilling synthetic data from our Dataset Distillation by Matching Training Trajectories paper (CVPR 2022). Please see our [project page](https://georgecazenavette.github.io/mtt-distillation) for more results.
> [**Dataset Distillation by Matching Training Trajectories**](https://georgecazenavette.github.io/mtt-distillation/)<br>
> [George Cazenavette](https://georgecazenavette.github.io/), [Tongzhou Wang](https://ssnl.github.io/), [Antonio Torralba](https://groups.csail.mit.edu/vision/torralbalab/), [Alexei A. Efros](https://people.eecs.berkeley.edu/~efros/), [Jun-Yan Zhu](https://www.cs.cmu.edu/~junyanz/)<br>
> CMU, MIT, UC Berkeley<br>
> CVPR 2022 (Oral)
The task of "Dataset Distillation" is to learn a small number of synthetic images such that a model trained on this set alone will have similar test performance as a model trained on the full real dataset.
<img src='docs/method.gif' width=600>
Our method distills the synthetic dataset by directly optimizing the fake images to induce similar network training dynamics as the full,
real dataset. We train "student" networks for many iterations on the synthetic data,
measure the error in parameter space between the "student" and "expert" networks trained on real data,
and back-propagate through all the student network updates to optimize the synthetic pixels.
## Wearable ImageNet: Synthesizing Tileable Textures
Instead of treating our synthetic data as individual images, we can instead encourage every random crop (with circular padding) on a larger canvas of pixels to induce a good training trajectory. This results in class-based textures that are continuous around their edges.
<img src='docs/penguins1_horizontal.png' width=600>
Given these tileable textures, we can apply them to areas that require such properties, such as clothing patterns.
<img src="docs/flamingo_shirt.jpg" width="150"><img src="docs/penguin_shirt.jpg" width="150"><img src="docs/parrot_dress.jpg" width="150"><img src="docs/eagle_jacket.jpg" width="150">
Visualizations made using <a href="https://tri3d.in/">FAB3D</a>
<br>
### Getting Started
First, download our repo:
```bash
git clone https://github.com/GeorgeCazenavette/mtt-distillation.git
cd mtt-distillation
```
For an express instillation, we include ```.yaml``` files.
If you have an RTX 30XX GPU (or newer), run
```bash
conda env create -f requirements_11_3.yaml
```
If you have an RTX 20XX GPU (or older), run
```bash
conda env create -f requirements_10_2.yaml
```
You can then activate your conda environment with
```bash
conda activate distillation
```
##### Quadro Users Take Note:
```torch.nn.DataParallel``` seems to not work on Quadro A5000 GPUs, and this may extend to other Quadro cards.
If you experience indefinite hanging during training, try running the process with only 1 GPU by prepending ```CUDA_VISIBLE_DEVICES=0``` to the command.
### Generating Expert Trajectories
Before doing any distillation, you'll need to generate some expert trajectories using ```buffer.py```
The following command will train 100 ConvNet models on CIFAR-100 with ZCA whitening for 50 epochs each:
```bash
python buffer.py --dataset=CIFAR100 --model=ConvNet --train_epochs=50 --num_experts=100 --zca --buffer_path={path_to_buffer_storage} --data_path={path_to_dataset}
```
We used 50 epochs with the default learning rate for all of our experts.
Worse (but still interesting) results can be obtained faster through training fewer experts by changing ```--num_experts```. Note that experts need only be trained once and can be re-used for multiple distillation experiments.
### Distillation by Matching Training Trajectories
The following command Excerpt of 8,616 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:f2a425591c5af8b5, topic:synthetic-data, readme:synthetic data
matched fp:f2a425591c5af8b5, topic:computer-vision