Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
A PyTorch implementation of the paper "ZigMa: A DiT-Style Mamba-based Diffusion Model" (ECCV 2024)
| Date | Stars |
|---|---|
| 2026-07-24 | 351 |
| 2026-07-25 | 351 |
| 2026-07-28 | 351 |
| 2026-07-30 | 351 |
| 2026-07-31 | 351 |
| 2026-08-08 | 352 |
| 2026-08-21 | 351 |
| 2026-09-20 | 351 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# ZigMa: A DiT-style Zigzag Mamba Diffusion Model (ECCV 2024)
**ECCV 2024**
**[Oral Talk in ICML 2024 Workshop on Long Context Foundation Models (LCFM)](https://icml.cc/virtual/2024/39058)**
This repository represents the official implementation of the paper titled "ZigMa: A DiT-style Zigzag Mamba Diffusion Model (ECCV 2024)".
[](https://taohu.me/zigma)
[](https://arxiv.org/abs/2403.13802)
[](https://huggingface.co/taohu/zigma)
[](https://github.com/CompVis/zigma)
[](https://github.com/CompVis/zigma/issues?q=is%3Aissue+is%3Aclosed)
[](https://twitter.com/_akhaliq/status/1770668624392421512)
[](https://www.apache.org/licenses/LICENSE-2.0)
[Vincent Tao Hu](http://taohu.me),
[Stefan Andreas Baumann](https://scholar.google.de/citations?user=egzbdnoAAAAJ&hl=en),
[Ming Gui](https://www.linkedin.com/in/ming-gui-87b76a16b/?originalSubdomain=de),
[Olga Grebenkova](https://www.linkedin.com/in/grebenkovao/),
[Pingchuan Ma](https://www.linkedin.com/in/pingchuan-ma-492543156/),
[Johannes Schusterbauer](https://www.linkedin.com/in/js-fischer/ ),
[Björn Ommer](https://ommer-lab.com/people/ommer/ )
We present ZigMa, a scanning scheme that follows a zigzag pattern, considering both spatial continuity and parameter efficiency. We further adapt this scheme to video, separating the reasoning between spatial and temporal dimensions, thus achieving efficient parameter utilization. Our design allows for greater incorporation of inductive bias for non-1D data and improves parameter efficiency in diffusion models.
## 🎓 Citation
Please cite our paper:
```bibtex
@InProceedings{hu2024zigma,
title={ZigMa: A DiT-style Zigzag Mamba Diffusion Model},
author={Vincent Tao Hu and Stefan Andreas Baumann and Ming Gui and Olga Grebenkova and Pingchuan Ma and Johannes Schusterbauer and Björn Ommer},
booktitle = {ECCV},
year={2024}
}
```
## :white_check_mark: Updates
* **` May. 24th, 2024`**: 🚀🚀🚀 New checkpoints for FacesHQ1024, landscape1024, Churches256 datasets.
* **` April. 6th, 2024`**: Support for FP16 training, and checkpoint function, and torch.compile to achieve better memory utilization and speed boosting.
* **` April. 2th, 2024`**: Main code released.
## Quick Demo
```python
from model_zigma import ZigMa
img_dim = 32
in_channels = 3
model = ZigMa(
in_channels=in_channels,
embed_dim=640,
depth=18,
img_dim=img_dim,
patch_size=1,
has_text=True,
d_context=768,
n_context_token=77,
device="cuda",
scan_type="zigzagN8",
use_pe=2,
)
x = torch.rand(10, in_channels, img_dim, img_dim).to("cuda")
t = torch.rand(10).to("cuda")
_context = torch.rand(10, 77, 768).to("cuda")
o = model(x, t, y=_context)
print(o.shape)
```
### Improved Training Performance
In comparison to the original implementation, we implement a selection of training speed acceleration and memory saving features including gradient checkpointing
| torch.compile | gradient checkpointing | training speed | memory |
| :-----------: | :--------------------: | :------------: | :----: |
| ❌ | ❌ | 1.05 iters/sec | 18G |
| ❌ | ✔ | 0.93 steps/sec | 9G |
| ✔ | ❌ | 1.8 iters/sec | 18G |
torch.compiles is for indexing operation: [here](https://github.com/CoExcerpt of 9,926 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:18cbb962e8f187bf, topic:diffusion-models, desc:diffusion model, readme:diffusion model
matched fp:18cbb962e8f187bf, topic:state-space-model