Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Label-Efficient Semantic Segmentation with Diffusion Models (ICLR'2022)
| Date | Stars |
|---|---|
| 2026-07-31 | 718 |
| 2026-08-04 | 719 |
| 2026-08-06 | 719 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Label-Efficient Semantic Segmentation with Diffusion Models **ICLR'2022** [[Project page]](https://yandex-research.github.io/ddpm-segmentation/) Official implementation of the paper [Label-Efficient Semantic Segmentation with Diffusion Models](https://arxiv.org/pdf/2112.03126.pdf) This code is based on [datasetGAN](https://github.com/nv-tlabs/datasetGAN_release) and [guided-diffusion](https://github.com/openai/guided-diffusion). **Note:** use **--recurse-submodules** when clone. ## Overview The paper investigates the representations learned by the state-of-the-art DDPMs and shows that they capture high-level semantic information valuable for downstream vision tasks. We design a simple semantic segmentation approach that exploits these representations and outperforms the alternatives in the few-shot operating point. <div align="center"> <img width="100%" alt="DDPM-based Segmentation" src="https://storage.yandexcloud.net/yandex-research/ddpm-segmentation/figs/new_ddpm_seg_scheme.png"> </div> ## Updates **3/9/2022:** 1) Improved performance of DDPM-based segmentation by changing:\ Diffusion steps: [50,150,250,350] --> [50,150,250];\ UNet blocks: [6,7,8,9] --> [5,6,7,8,12]; 3) Trained a bit better DDPM on FFHQ-256; 4) Added [MAE](https://github.com/facebookresearch/mae) for comparison. ## Datasets The evaluation is performed on 6 collected datasets with a few annotated images in the training set: Bedroom-18, FFHQ-34, Cat-15, Horse-21, CelebA-19 and ADE-Bedroom-30. The number corresponds to the number of semantic classes. [datasets.tar.gz](https://storage.yandexcloud.net/yandex-research/ddpm-segmentation/datasets.tar.gz) (~47Mb) ## DDPM ### Pretrained DDPMs The models trained on LSUN are adopted from [guided-diffusion](https://github.com/openai/guided-diffusion). FFHQ-256 is trained by ourselves using the same model parameters as for the LSUN models. *LSUN-Bedroom:* [lsun_bedroom.pt](https://openaipublic.blob.core.windows.net/diffusion/jul-2021/lsun_bedroom.pt)\ *FFHQ-256:* [ffhq.pt](https://storage.yandexcloud.net/yandex-research/ddpm-segmentation/models/ddpm_checkpoints/ffhq.pt) (Updated 3/8/2022)\ *LSUN-Cat:* [lsun_cat.pt](https://openaipublic.blob.core.windows.net/diffusion/jul-2021/lsun_cat.pt)\ *LSUN-Horse:* [lsun_horse.pt](https://openaipublic.blob.core.windows.net/diffusion/jul-2021/lsun_horse.pt) ### Run 1. Download the datasets:\ ```bash datasets/download_datasets.sh``` 2. Download the DDPM checkpoint:\ ```bash checkpoints/ddpm/download_checkpoint.sh <checkpoint_name>``` 3. Check paths in ```experiments/<dataset_name>/ddpm.json``` 4. Run: ```bash scripts/ddpm/train_interpreter.sh <dataset_name>``` **Available checkpoint names:** lsun_bedroom, ffhq, lsun_cat, lsun_horse\ **Available dataset names:** bedroom_28, ffhq_34, cat_15, horse_21, celeba_19, ade_bedroom_30 **Note:** ```train_interpreter.sh``` is RAM consuming since it keeps all training pixel representations in memory. For ex, it requires ~210Gb for 50 training images of 256x256. (See [issue](https://github.com/nv-tlabs/datasetGAN_release/issues/34)) **Pretrained pixel classifiers** and test predictions are [here](https://www.dropbox.com/s/kap229jvmhfwh7i/pixel_classifiers.tar?dl=0). ### How to improve the performance * Tune for a particular task what diffusion steps and UNet blocks to use. ## DatasetDDPM ### Synthetic datasets To download DDPM-produced synthetic datasets (50000 samples, ~7Gb) (updated 3/8/2022):\ ```bash synthetic-datasets/ddpm/download_synthetic_dataset.sh <dataset_name>``` ### Run | Option #1 1. Download the synthetic dataset:\ ```bash synthetic-datasets/ddpm/download_synthetic_dataset.sh <dataset_name>``` 2. Check paths in ```experiments/<dataset_name>/datasetDDPM.json``` 3. Run: ```bash scripts/datasetDDPM/train_deeplab.sh <dataset_name>``` ### Run | Option #2 1. Download the datasets:\ &
Excerpt of 11,551 characters
Read on GitHub
Dmitry Baranchuk · Yandex Research
17
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:7b31e2bbf0e31da2, topic:deep-learning