Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
[ECCV 2024] MOFA-Video: Controllable Image Animation via Generative Motion Field Adaptions in Frozen Image-to-Video Diffusion Model.
| Date | Stars |
|---|---|
| 2026-07-31 | 764 |
| 2026-08-06 | 764 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<h2>🦄️ MOFA-Video: Controllable Image Animation via Generative Motion Field Adaptions in Frozen Image-to-Video Diffusion Model (ECCV 2024) </h2>
<div>
<a href='https://myniuuu.github.io/' target='_blank'>Muyao Niu</a> <sup>1,2</sup>
<a href='https://vinthony.github.io/academic/' target='_blank'>Xiaodong Cun</a><sup>2,*</sup>
<a href='https://xinntao.github.io/' target='_blank'>Xintao Wang</a><sup>2</sup>
<a href='https://yzhang2016.github.io/' target='_blank'>Yong Zhang</a><sup>2</sup>
<a href='https://scholar.google.com/citations?user=4oXBp9UAAAAJ&hl=en' target='_blank'>Ying Shan</a><sup>2</sup>
<a href='https://scholar.google.com/citations?user=JD-5DKcAAAAJ&hl=en' target='_blank'>Yinqiang Zheng</a><sup>1,*</sup>
</div>
<div>
<sup>1</sup> The University of Tokyo <sup>2</sup> Tencent AI Lab <sup>*</sup> Corresponding Author
</div>
In *European Conference on Computer Vision (ECCV) 2024*
<a href='https://arxiv.org/abs/2405.20222'><img src='https://img.shields.io/badge/ArXiv-PDF-red'></a> <a href='https://myniuuu.github.io/MOFA_Video'><img src='https://img.shields.io/badge/Project-Page-Green'></a> <a href='https://huggingface.co/MyNiuuu/MOFA-Video-Traj'><img src='https://img.shields.io/badge/🤗 huggingface-MOFA_Traj-blue'></a> <a href='https://huggingface.co/MyNiuuu/MOFA-Video-Hybrid'><img src='https://img.shields.io/badge/🤗 huggingface-MOFA_Hybrid-yellow'></a>
## 🔥🔥🔥 New Features/Updates
- (2024.08.07) We have released the inference script for keypoint-based facial image animation! Please refer to [Here](./MOFA-Video-Keypoint/README.md) for more instructions.
- (2024.07.15) We have released the training code for trajectory-based image animation! Please refer to [Here](./Training/README.md) for more instructions.
- MOFA-Video will be appeared in *ECCV 2024!* 🇮🇹🇮🇹🇮🇹
- We have released the Gradio inference code and the checkpoints for **Hybrid** Controls! Please refer to [Here](./MOFA-Video-Hybrid/README.md) for more instructions.
- *Free online demo via HuggingFace Spaces will be coming soon!*
- If you find this work interesting, please do not hesitate to give a ⭐!
## 📰 CODE RELEASE
- [x] (2024.05.31) Gradio demo and checkpoints for trajectory-based image animation
- [x] (2024.06.22) Gradio demo and checkpoints for image animation with hybrid control
- [x] (2024.07.15) Training scripts for trajectory-based image animation
- [x] (2024.08.07) Inference scripts and checkpoints for keypoint-based facial image animation
- [ ] Training scripts for keypoint-based facial image animation
## TL;DR
<div align="center">
<h3>
Image 🏞️ + Hybrid Controls 🕹️ = Videos 🎬🍿
</h3>
</div>
<table align="center">
<tr>
<td align="center">
<img src="assets/teasers/hybrid/input.gif"/>
<br />
</td>
<td align="center">
<img src="assets/teasers/hybrid/landmark.gif"/>
<br />
</td>
<td align="center">
<img src="assets/teasers/hybrid/output.gif"/>
<br />
</td>
</tr>
<tr>
<td colspan="3" align="center" style="border: none;">
Trajectory + Landmark Control
</td>
</tr>
</table>
<table align="center">
<tr>
<td align="center">
<img src="assets/teasers/traj/input.gif"/>
<br />
</td>
<td align="center">
<img src="assets/teasers/traj/hint.gif"/>
<br />
</td>
<td align="center">
<img src="assets/teasers/traj/flow.gif"/>
<br />
</td>
<td align="center">
<img src="assets/teasers/traj/output.gif"/>
<br />
</td>
</tr>
<tr>
<td colspan="4" align="center" style="border: none;">
Trajectory Control
</td>
</tr>
</table>
<table align="center">
<tr>
<td align="center">
<img src="assets/teasers/ldmk/input.gif" />
<br />
</td>
<td align="center">
<img src="assets/teasers/ldmk/flow.gif" />
<br />
</td>Excerpt of 9,014 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:4e517230dda24bbd, topic:diffusion-models, desc:diffusion model
