Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
A coding-free framework built on PyTorch for reproducible deep learning studies. PyTorch Ecosystem. 🏆26 knowledge distillation methods presented at TPAMI, CVPR, ICLR, ECCV, NeurIPS, ICCV, AAAI, etc are implemented so far. 🎁 Trained models, training logs and configurations are available for ensuring the reproducibiliy and benchmark.
| Date | Stars |
|---|---|
| 2026-07-24 | 1623 |
| 2026-07-25 | 1623 |
| 2026-07-28 | 1623 |
| 2026-07-30 | 1623 |
| 2026-07-31 | 1623 |
| 2026-08-06 | 1625 |
Today
+2 stars today
This week
+2 stars this week
This month
— stars this month
Momentum
10.0
growth rate 0.12%/day
# torchdistill: A Modular, Configuration-Driven Framework for Knowledge Distillation
[](https://pypi.org/project/torchdistill/)
[](https://travis-ci.com/github/yoshitomo-matsubara/torchdistill)
[](https://github.com/yoshitomo-matsubara/torchdistill/discussions)
[](https://doi.org/10.1007/978-3-030-76423-4_3)
[](https://doi.org/10.18653/v1/2023.nlposs-1.18)
***torchdistill*** (formerly *kdkit*) offers various state-of-the-art knowledge distillation methods
and enables you to design (new) experiments simply by editing a declarative yaml config file instead of Python code.
Even when you need to extract intermediate representations in teacher/student models,
you will **NOT** need to reimplement the models, that often change the interface of the forward, but instead
specify the module path(s) in the yaml file. Refer to [these papers](https://github.com/yoshitomo-matsubara/torchdistill#citation) for more details.
In addition to knowledge distillation, this framework helps you design and perform general deep learning experiments
(**WITHOUT coding**) for reproducible deep learning studies. i.e., it enables you to train models without teachers
simply by excluding teacher entries from a declarative yaml config file.
You can find such examples below and in [configs/sample/](https://github.com/yoshitomo-matsubara/torchdistill/tree/main/configs/sample/).
In December 2023, ***torchdistill*** officially joined [PyTorch Ecosystem](https://pytorch.org/ecosystem/).
When you refer to ***torchdistill*** in your paper, please cite [these papers](https://github.com/yoshitomo-matsubara/torchdistill#citation)
instead of this GitHub repository.
**If you use** ***torchdistill*** **as part of your work, your citation is appreciated and motivates me to maintain and upgrade this framework!**
## Documentation
You can find the API documentation and research projects that leverage ***torchdistill*** at https://yoshitomo-matsubara.net/torchdistill/
## Forward hook manager
Using **ForwardHookManager**, you can extract intermediate representations in model without modifying the interface of its forward function.
[This example notebook](https://github.com/yoshitomo-matsubara/torchdistill/tree/main/demo/extract_intermediate_representations.ipynb) [](https://colab.research.google.com/github/yoshitomo-matsubara/torchdistill/blob/main/demo/extract_intermediate_representations.ipynb) [](https://studiolab.sagemaker.aws/import/github/yoshitomo-matsubara/torchdistill/blob/main/demo/extract_intermediate_representations.ipynb)
will give you a better idea of the usage such as knowledge distillation and analysis of intermediate representations.
E.g., extract intermediate representations (feature map) of ResNet-18 for a random input batch
```python
import torch
from torchvision import models
from torchdistill.core.forward_hook import ForwardHookManager
# Define a model and choose torch device
model = models.resnet18(pretrained=False)
device = torch.device('cpu')
# Register forward hooks for modules of your interest
forward_hook_manager = ForwardHookManager(device)
forward_hook_manager.add_hook(model, 'conv1', requires_input=True, requires_output=False)
forward_hook_manager.add_hook(model, 'layer1.0.bnExcerpt of 24,042 characters
Read on GitHub
1.7k
3
Guocheng
1
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:486346048652bb69, topic:nlp, topic:natural-language-processing, topic:text-classification
matched fp:486346048652bb69, topic:knowledge-distillation, desc:knowledge distillation, readme:knowledge distillation
matched fp:486346048652bb69, topic:object-detection, topic:image-classification