Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Official implementation of "MIMO: Controllable Character Video Synthesis with Spatial Decomposed Modeling"
| Date | Stars |
|---|---|
| 2026-07-24 | 1576 |
| 2026-07-25 | 1576 |
| 2026-07-28 | 1576 |
| 2026-07-30 | 1576 |
| 2026-08-06 | 1576 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# MIMO - Official PyTorch Implementation ### [Project page](https://menyifang.github.io/projects/MIMO/index.html) | [Paper](https://arxiv.org/abs/2409.16160) | [Video](https://www.youtube.com/watch?v=skw9lPKFfcE) | [Online Demo](https://modelscope.cn/studios/iic/MIMO) > **MIMO: Controllable Character Video Synthesis with Spatial Decomposed Modeling**<br> > [Yifang Men](https://menyifang.github.io/), [Yuan Yao](mailto:[email protected]), [Miaomiao Cui](mailto:[email protected]), [Liefeng Bo](https://scholar.google.com/citations?user=FJwtMf0AAAAJ&hl=en)<br> > Institute for Intelligent Computing (Tongyi Lab), Alibaba Group > In: CVPR 2025 MIMO is a generalizable model for controllable video synthesis, which can not only synthesize realistic character videos with controllable attributes (i.e., character, motion and scene) provided by very simple user inputs, but also simultaneously achieve advanced scalability to arbitrary characters, generality to novel 3D motions, and applicability to interactive real-world scenes in a unified framework. ## Demo Animating character image with driving 3D pose from motion dataset https://github.com/user-attachments/assets/3a13456f-9ee5-437c-aba4-30d8c3b6e251 Driven by in-the-wild video with spatial 3D motion and interactive scene https://github.com/user-attachments/assets/4d989e7f-a623-4339-b3d1-1d1a33ad25f2 More results can be found in [project page](https://menyifang.github.io/projects/MIMO/index.html). ## 📢 News (2025-06-11) The code is released! We released a simplified version of full implementation, but it could achieve comparable performance. (2025-02-27) The paper is accepted by CVPR 2025! The full version of the paper is available on [arXiv](https://arxiv.org/abs/2409.16160). (2024-01-07) The online demo (v1.5) supporting custom driving videos is available now! Try out [](https://modelscope.cn/studios/iic/MIMO). (2024-11-26) The online demo (v1.0) is available on ModelScope now! Try out [](https://modelscope.cn/studios/iic/MIMO). The 1.5 version to support custom driving videos will be coming soon. (2024-09-25) The project page, demo video and technical report are released. The full paper version with more details is in process. ## Requirements * python (>=3.10) * pyTorch * tensorflow * cuda 12.1 * GPU (tested on A100, L20) ## 🚀 Getting Started ```bash git clone https://github.com/menyifang/MIMO.git cd MIMO ``` ### Installation ```bash conda create -n mimo python=3.10 conda activate mimo bash install.sh ``` ### Downloads #### Model Weights You can manually download model weights from [ModelScope](https://modelscope.cn/models/iic/MIMO/files) or [Huggingface](https://huggingface.co/menyifang/MIMO/tree/main), or automatically using follow commands. Download from HuggingFace ```python from huggingface_hub import snapshot_download model_dir = snapshot_download(repo_id='menyifang/MIMO', cache_dir='./pretrained_weights') ``` Download from ModelScope ```python from modelscope import snapshot_download model_dir = snapshot_download(model_id='iic/MIMO', cache_dir='./pretrained_weights') ``` #### Prior Model Weights Download pretrained weights of based model and other components: - [StableDiffusion V1.5](https://huggingface.co/runwayml/stable-diffusion-v1-5) - [sd-vae-ft-mse](https://huggingface.co/stabilityai/sd-vae-ft-mse) - [image_encoder](https://huggingface.co/lambdalabs/sd-image-variations-diffusers/tree/main/image_encoder) #### Data Preparation Download examples and resources (`assets.zip`) from [google drive](https://drive.google.com/file/d/1dg0SDAxEARClYq_6L1T1XIfWvC5iA8WD/view?usp=drive_link) and unzip it under `${PROJECT_ROOT}/`. You can also process custom videos following [Process driving templates](#process-driving-templates). After downloading weights and data, the folder of the project st
Excerpt of 6,820 characters
Read on GitHub
10
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:a21952eb64d33a27, topic:video-generation, topic:video-editing, desc:video synthesis
matched fp:a21952eb64d33a27, topic:diffusion-models, readme:stable diffusion