Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Speed up Stable Diffusion with this one simple trick!
| Date | Stars |
|---|---|
| 2026-07-31 | 1405 |
| 2026-08-04 | 1405 |
| 2026-08-06 | 1405 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Token Merging for Stable Diffusion Using nothing but pure python and pytorch, ToMe for SD speeds up diffusion by merging _redundant_ tokens.  This is the official implementation of **ToMe for SD** from our short paper: **[Token Merging for Fast Stable Diffusion](https://arxiv.org/abs/2303.17604)** [Daniel Bolya](https://dbolya.github.io), [Judy Hoffman](https://faculty.cc.gatech.edu/~judy/) _[CVPR '23 ECV Workshop Oral](https://sites.google.com/view/ecv23)_ | _[GitHub](https://github.com/dbolya/tomesd)_ | _[arXiv](https://arxiv.org/abs/2303.17604)_ | _[BibTeX](#citation)_ ToMe for SD is an extension of the original **ToMe**: **[Token Merging: Your ViT but Faster](https://arxiv.org/abs/2210.09461)** [Daniel Bolya](https://dbolya.github.io), [Cheng-Yang Fu](http://www.cs.unc.edu/~cyfu/), [Xiaoliang Dai](https://sites.google.com/view/xiaoliangdai/), [Peizhao Zhang](https://research.facebook.com/people/zhang-peizhao/), [Christoph Feichtenhofer](https://feichtenhofer.github.io/), [Judy Hoffman](https://faculty.cc.gatech.edu/~judy/) _[ICLR '23 Oral (Top 5%)](https://openreview.net/forum?id=JroZRaRw7Eu)_ | _[GitHub](https://github.com/facebookresearch/ToMe)_ | _[arXiv](https://arxiv.org/abs/2210.09461)_ | _[Blog](https://research.facebook.com/blog/2023/2/token-merging-your-vit-but-faster/)_ | _[BibTeX](https://github.com/facebookresearch/ToMe#citation)_ **Note:** this extension of ToMe is not affiliated in any way with Meta. ## What is ToMe for SD?  Token Merging (**ToMe**) speeds up transformers by _merging redundant tokens_, which means the transformer has to do _less work_. We apply this to the underlying transformer blocks in Stable Diffusion in a clever way that minimizes quality loss while keeping most of the speed-up and memory benefits. ToMe for SD _doesn't_ require training and should work out of the box for any Stable Diffusion model. **Note:** this is a lossy process, so the image _will_ change, ideally not by much. Here are results with [FID](https://github.com/mseitzer/pytorch-fid) scores vs. time and memory usage (lower is better) when using Stable Diffusion v1.5 to generate 512x512 images of ImageNet-1k classes on a 4090 GPU with 50 PLMS steps using fp16: | Method | r% | FID ↓ | Time (s/im) ↓ | Memory (GB/im) ↓ | |-----------------------------|----|:------|:--------------------------|:------------------------| | Baseline _(Original Model)_ | 0 | 33.12 | 3.09 | 3.41 | | w/ **ToMe for SD** | 10 | 32.86 | 2.56 (**1.21x** _faster_) | 2.99 (**1.14x** _less_) | | | 20 | 32.86 | 2.29 (**1.35x** _faster_) | 2.17 (**1.57x** _less_) | | | 30 | 32.80 | 2.06 (**1.50x** _faster_) | 1.71 (**1.99x** _less_) | | | 40 | 32.87 | 1.85 (**1.67x** _faster_) | 1.26 (**2.71x** _less_) | | | 50 | 33.02 | 1.65 (**1.87x** _faster_) | 0.89 (**3.83x** _less_) | | | 60 | 33.37 | 1.52 (**2.03x** _faster_) | 0.60 (**5.68x** _less_) | Even with more than half of the tokens merged (60%!), ToMe for SD still produces images close to the originals, while being _**2x** faster_ and using _**~5.7x** less memory_. Moreover, ToMe is not another efficient reimplementation of transformer modules. Instead, it actually _reduces_ the total work necessary to generate an image, so it can function _in conjunction_ with efficient implementations (see [Usage](#tome--xformers--flash-attn--torch-20)). ## News - **[2023.06.19]** I'll be giving a talk at the CVPR 2023 Efficient Deep Learning for Computer Vision workshop. Come on do
Excerpt of 9,125 characters
Read on GitHub
Daniel Bolya
18
1
Ali Hassani · NVIDIA
1
yujun · @ECUST · China
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:be234ab03366fa55, desc:stable diffusion