Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Official implementation of Ctrl-Adapter: An Efficient and Versatile Framework for Adapting Diverse Controls to Any Diffusion Model (ICLR 2025 Oral)
| Date | Stars |
|---|---|
| 2026-07-31 | 470 |
| 2026-08-06 | 471 |
Today
+1 stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Ctrl-Adapter: An Efficient and Versatile Framework for Adapting Diverse Controls to Any Diffusion Model (ICLR 2025 Oral) Official implementation of **Ctrl-Adapter**, an efficient and versatile framework that adds diverse controls to any image/video diffusion models by adapting pretrained ControlNets. [](https://arxiv.org/abs/2404.09967) [](https://ctrl-adapter.github.io/) [](https://huggingface.co/hanlincs/ctrl-adapter) [Han Lin](https://hl-hanlin.github.io/), [Jaemin Cho](https://j-min.io), [Abhay Zala](https://aszala.com/), [Mohit Bansal](https://www.cs.unc.edu/~mbansal/) <br> <img width="800" src="assets/teaser_update.gif"/> <br> CTRL-Adapter is an efficient and versatile framework for adding diverse spatial controls to any image or video diffusion model. It supports a variety of useful applications, including video control, video control with multiple conditions, video control with sparse frame conditions, image control, zero-shot transfer to unseen conditions, and video editing. # 🔥 News - **Feb 11, 2025**. Ctrl-Adapter has been accepted by ICLR 2025 as **<span style="color:red;">Oral</span>**! - **May 26, 2024**. Check our new [arXiv-v2](https://arxiv.org/abs/2404.09967) for exciting new additions to Ctrl-Adapter! - Support for DiT-based backbones (**[Latte](https://github.com/Vchitect/Latte)**, **[PixArt-α](https://github.com/PixArt-alpha/PixArt-alpha)**) - Fine-grained patch-level MoE router for multi-control composition - Downstream tasks beyond spatial control (video editing, video style transfer, text-guided motion control) - **Apr 30, 2024**. Training code released now! It's time to train Ctrl-Adapter on your desired backbone! 🚀🚀 - **Apr 29, 2024**. **[SDXL](https://stability.ai/news/stable-diffusion-sdxl-1-announcement)**, **[I2VGen-XL](https://i2vgen-xl.github.io/)**, and **[SVD](https://stability.ai/news/stable-video-diffusion-open-ai-video-model)** inference code and checkpoints are all released! # 🔧 Setup ### Environment Setup If you only need to perform inference with our code, please install from ```requirements_inference.txt```. To make our codebase easy to use, the primary libraries that need to be installed are Torch, Diffusers, and Transformers. Specific versions of these libraries are not required; the default versions should work fine :) If you are planning to conduct training, please install from ```requirements_train.txt``` instead, which contains more dependent libraries needed. ```shell conda create -n ctrl-adapter python==3.10 conda activate ctrl-adapter pip install -r requirements_inference.txt # install from this if you only need to perform inference pip install -r requirements_train.txt # install from this if you plan to do some training ``` Here we list several questions that we believe important when you start using this # 🔮 Inference We provde model checkpoints and inference scripts for Ctrl-Adapter trained on SDXL, I2VGen-XL, and SVD. All inference scripts are put under ```./inference_scripts```. ### 📌 Notice Before You Begin Please note that there is usually no single model that excels at generating images/videos for all motion styles across various control conditions. Different image/video generation backbones may perform better with specific types of motion. For instance, we have observed that SVD excels at slide motions, while it generally performs worse than I2VGen-XL with complex motions (this is consistent wtih the findings in [DynamiCrafter](https://www.youtube.com/watch?v=0NfmIsNAg-g)). Additionally, using different control conditions can lead to significantly different results in the generated images/videos, and some control conditions may be more informative than others for certain types of motion. ### 📌 Inference Data Structure We put some s
Excerpt of 13,883 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:d54ca11f29aea64e, name:adapter, desc:adapter