Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
The official implementation of [CVPR2022] Decoupled Knowledge Distillation https://arxiv.org/abs/2203.08679 and [ICCV2023] DOT: A Distillation-Oriented Trainer https://openaccess.thecvf.com/content/ICCV2023/papers/Zhao_DOT_A_Distillation-Oriented_Trainer_ICCV_2023_paper.pdf
| Date | Stars |
|---|---|
| 2026-07-24 | 898 |
| 2026-07-25 | 898 |
| 2026-07-28 | 897 |
| 2026-07-30 | 897 |
| 2026-08-06 | 897 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<div align=center><img src=".github/mdistiller.png" width="40%" ><div align=left> This repo is (1) a PyTorch library that provides classical knowledge distillation algorithms on mainstream CV benchmarks, (2) the official implementation of the CVPR-2022 paper: [Decoupled Knowledge Distillation](https://arxiv.org/abs/2203.08679). (3) the official implementation of the ICCV-2023 paper: [DOT: A Distillation-Oriented Trainer](https://openaccess.thecvf.com/content/ICCV2023/papers/Zhao_DOT_A_Distillation-Oriented_Trainer_ICCV_2023_paper.pdf). # DOT: A Distillation-Oriented Trainer ### Framework <div style="text-align:center"><img src=".github/dot.png" width="80%" ></div> ### Main Benchmark Results On CIFAR-100: | Teacher <br> Student | ResNet32x4 <br> ResNet8x4| VGG13 <br> VGG8| ResNet32x4 <br> ShuffleNet-V2| |:---------------:|:-----------------:|:-----------------:|:-----------------:| | KD | 73.33 | 72.98 | 74.45 | | **KD+DOT** | **75.12** | **73.77** | **75.55** | On Tiny-ImageNet: | Teacher <br> Student |ResNet18 <br> MobileNet-V2|ResNet18 <br> ShuffleNet-V2| |:---------------:|:-----------------:|:-----------------:| | KD | 58.35 | 62.26 | | **KD+DOT** | **64.01** | **65.75** | On ImageNet: | Teacher <br> Student |ResNet34 <br> ResNet18|ResNet50 <br> MobileNet-V1| |:---------------:|:-----------------:|:-----------------:| | KD | 71.03 | 70.50 | | **KD+DOT** | **71.72** | **73.09** | # Decoupled Knowledge Distillation ### Framework & Performance <div style="text-align:center"><img src=".github/dkd.png" width="80%" ></div> ### Main Benchmark Results On CIFAR-100: | Teacher <br> Student |ResNet56 <br> ResNet20|ResNet110 <br> ResNet32| ResNet32x4 <br> ResNet8x4| WRN-40-2 <br> WRN-16-2| WRN-40-2 <br> WRN-40-1 | VGG13 <br> VGG8| |:---------------:|:-----------------:|:-----------------:|:-----------------:|:------------------:|:------------------:|:--------------------:| | KD | 70.66 | 73.08 | 73.33 | 74.92 | 73.54 | 72.98 | | **DKD** | **71.97** | **74.11** | **76.32** | **76.23** | **74.81** | **74.68** | | Teacher <br> Student |ResNet32x4 <br> ShuffleNet-V1|WRN-40-2 <br> ShuffleNet-V1| VGG13 <br> MobileNet-V2| ResNet50 <br> MobileNet-V2| ResNet32x4 <br> MobileNet-V2| |:---------------:|:-----------------:|:-----------------:|:-----------------:|:------------------:|:------------------:| | KD | 74.07 | 74.83 | 67.37 | 67.35 | 74.45 | | **DKD** | **76.45** | **76.70** | **69.71** | **70.35** | **77.07** | On ImageNet: | Teacher <br> Student |ResNet34 <br> ResNet18|ResNet50 <br> MobileNet-V1| |:---------------:|:-----------------:|:-----------------:| | KD | 71.03 | 70.50 | | **DKD** | **71.70** | **72.05** | # MDistiller ### Introduction MDistiller supports the following distillation methods on CIFAR-100, ImageNet and MS-COCO: |Method|Paper Link|CIFAR-100|ImageNet|MS-COCO| |:---:|:---:|:---:|:---:|:---:| |KD| <https://arxiv.org/abs/1503.02531> |✓|✓| | |FitNet| <https://arxiv.org/abs/1412.6550> |✓| | | |AT| <https://arxiv.org/abs/1612.03928> |✓|✓| | |NST| <https://arxiv.org/abs/1707.01219> |✓| | | |PKT| <https://arxiv.org/abs/1803.10837> |✓| | | |KDSVD| <https://arxiv.org/abs/1807.06819> |✓| | | |OFD| <https://arxiv.org/abs/1904.01866> |✓|✓| | |RKD| <https://arxiv.org/abs/1904.05068> |✓| | | |VID| <https://arxiv.org/abs/1904.05835> |✓| | | |SP| <https://arxiv.org/abs/1907.09682> |✓| | | |CRD| <https://arxiv.org/abs/1910.10699> |✓|✓| | |ReviewKD| <https://arxiv.org/abs/2104.09044> |✓|✓|✓| |DKD| <https://arxiv.org/abs/2203.08679> |✓|✓|✓| ### Installation Environments: - Python 3.6 - PyTorch 1.9.0 - torchvision 0.10.0 Install the package: ``` sudo pip3 install -r requirements.txt sudo python3 setup.py develop ``` ### Getting started 0. Wandb as the logger - The registeration: <https://wandb.ai/home>. - If you don't want wandb as your logger, set `CFG.LOG.WANDB` as `Fals
Excerpt of 7,712 characters
Read on GitHub
6
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:bfbea81336d2b0fe, topic:deep-learning, topic:pytorch
matched fp:bfbea81336d2b0fe, topic:knowledge-distillation, desc:knowledge distillation, readme:knowledge distillation
matched fp:bfbea81336d2b0fe, topic:computer-vision