Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Ensembling Off-the-shelf Models for GAN Training (CVPR 2022 Oral)
| Date | Stars |
|---|---|
| 2026-07-24 | 421 |
| 2026-07-25 | 421 |
| 2026-07-28 | 421 |
| 2026-07-30 | 421 |
| 2026-08-06 | 421 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Vision-aided GAN
[](https://paperswithcode.com/sota/image-generation-on-lsun-horse-256-x-256?p=ensembling-off-the-shelf-models-for-gan)
[](https://paperswithcode.com/sota/image-generation-on-lsun-cat-256-x-256?p=ensembling-off-the-shelf-models-for-gan)
[](https://paperswithcode.com/sota/image-generation-on-lsun-churches-256-x-256?p=ensembling-off-the-shelf-models-for-gan)
### [video](https://youtu.be/oHdyJNdQ9E4) | [website](https://www.cs.cmu.edu/~vision-aided-gan/) | [paper](https://arxiv.org/abs/2112.09130)
**[NEW!]** Vision-aided GAN training with BigGAN and StyleGAN3
**[NEW!]** Using vision-aided Discriminator in your own GAN training with `pip install vision-aided-loss`
<img src='docs/code.gif' align="center" width=800>
<br>
<div class="gif">
<p align="center">
<img src='docs/vision-aided-gan.gif' align="center" width=800>
</p>
</div>
Can the collective *knowledge* from a large bank of pretrained vision models be leveraged to improve GAN training? If so, with so many models to choose from, which one(s) should be selected, and in what manner are they most effective?
We find that pretrained computer vision models can significantly improve performance when used in an ensemble of discriminators. We propose an effective selection mechanism, by probing the linear separability between real and fake samples in pretrained model embeddings, choosing the most accurate model, and progressively adding it to the discriminator ensemble. Our method can improve GAN training in both limited data and large-scale settings.
Ensembling Off-the-shelf Models for GAN Training <br>
[Nupur Kumari](https://nupurkmr9.github.io/), [Richard Zhang](https://richzhang.github.io/), [Eli Shechtman](https://research.adobe.com/person/eli-shechtman/), [Jun-Yan Zhu](https://www.cs.cmu.edu/~junyanz/)<br>
In CVPR 2022
## Quantitative Comparison
<p align="center">
<img src="docs/lsun_eval.jpg" width="800px"/><br>
</p>
Our method outperforms recent GAN training methods by a large margin, especially in limited sample setting. For LSUN Cat, we achieve similar FID as StyleGAN2 trained on the full dataset using only 0.7\% of the dataset. On the full dataset, our method improves FID by 1.5x to 2x on cat, church, and horse categories of LSUN.
## Example Results
Below, we show visual comparisons between the baseline StyleGAN2-ADA and our model (Vision-aided GAN) for the
same randomly sample latent code on 100-shot [Bridge-of-sighs](https://data-efficient-gans.mit.edu/datasets/100-shot-bridge_of_sighs.zip) and [AnimalFace Dog](https://data-efficient-gans.mit.edu/datasets/AnimalFace-dog.zip) dataset.
<img src="docs/bridge.gif" width="400px"/><img src="docs/animalface_dog.gif" width="400px"/>
## Interpolation Videos
Latent interpolation results of models trained with our method on AnimalFace Cat (160 images), Dog (389 images), and Bridge-of-Sighs (100 photos).
<p align="center">
<img src="docs/interp.gif" width="800px"/>
</p>
## Worst sample visualzation
We randomly sample 5k images and sort them according to Mahalanobis distance using mean and variance of real samples calculated in inception feature space. Below visualization shows the bottom 30 images according to the distance for StyleGAN2-ADA (left) and our model (right).
<details open><summary>AFHQ Dog</summary>
<p>
<div class="images">
<table width=500>
<tr>
<td valign="top"><img src="docs/afhqdog_worst_baseline.jpg"/></td>
<td valign="top"><img src="docs/afhqdog_worst_ours.jpg"/></td>
</tr>
</Excerpt of 8,611 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:ef196af19f8f4478, topic:computer-vision, readme:computer vision
matched fp:ef196af19f8f4478, topic:pytorch
matched fp:ef196af19f8f4478, topic:image-generation, readme:image generation