Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
[CVPR 2024] | LAMP: Learn a Motion Pattern for Few-Shot Based Video Generation
| Date | Stars |
|---|---|
| 2026-07-24 | 283 |
| 2026-07-25 | 283 |
| 2026-07-28 | 283 |
| 2026-07-30 | 283 |
| 2026-08-06 | 283 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<p align="center">
<img src="assets/LOGO.png" height=170>
</p>
# <p align="center"> [CVPR 2024] | LAMP: Learn A Motion Pattern for Few-Shot-Based Video Generation </p>
 
This repository is the official implementation of [LAMP]()
> **LAMP: Learn A Motion Pattern for Few-Shot Video Generation**<br>
> Ruiqi Wu, Linagyu Chen, Tong Yang, Chunle Guo, Chongyi Li, Xiangyu Zhang
><br>( * indicates corresponding author)
[[Arxiv Paper](https://arxiv.org/abs/2310.10769)]
[[Website Page](https://rq-wu.github.io/projects/LAMP/index.html)]
[[Google Drive](https://drive.google.com/drive/folders/1e409tML98gwouIOxFwcFuGVBSNwsfEtY?usp=share_link)]
[[Baidu Disk (pwd: ffsp)](https://pan.baidu.com/s/1y9L2kfUlaHVZGE6B0-vXnA)]
[[Colab Notebook](https://colab.research.google.com/drive/1Cw2e0VFktVjWC5zIKzv2r7D2-4NtH8xm?usp=sharing)]
:rocket: LAMP is a **few-shot-based** method for text-to-video generation. You only need **8~16 videos 1 GPU (> 15 GB VRAM)** for training!! Then you can generate videos with learned motion pattern.
## News
- [2024/02/27] Our paper is accepted by CVPR2024!
- [2023/11/15] The code for applying LAMP on video editing is released!
- [2023/11/02] The [Colab demo](https://colab.research.google.com/drive/1Cw2e0VFktVjWC5zIKzv2r7D2-4NtH8xm?usp=sharing) is released! Thanks for the PR of @ShashwatNigam99.
- [2023/10/21] We add Google Drive link about our checkpoints and training data.
- [2023/10/17] We release our checkpoints and [Arxiv paper](https://arxiv.org/abs/2310.10769).
- [2023/10/16] Our code is publicly available.
## Preparation
### Dependencies and Installation
- Ubuntu > 18.04
- CUDA=11.3
- Others:
```bash
# clone the repo
git clone https://github.com/RQ-Wu/LAMP.git
cd LAMP
# create virtual environment
conda create -n LAMP python=3.8
conda activate LAMP
# install packages
pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu113
pip install -r requirements.txt
pip install xformers==0.0.13
```
### Weights and Data
1. You can download pre-trained T2I diffusion models on Hugging Face.
In our work, we use [Stable Diffusion v1.4](https://huggingface.co/CompVis/stable-diffusion-v1-4) as our backbone network. Clone the pretrained weights by `git-lfs` and put them in `./checkpoints`
2. Our checkpoint and training data are listed as follows. You can also collect video data by your own (Suggest websites: [pexels](https://pexels.com/), [frozen-in-time](https://meru.robots.ox.ac.uk/frozen-in-time/)) and put .mp4 files in `./training_videos/[motion_name]/`
3. [Update] You can find the training video for video editing demo in `assets/run.mp4`
<table class="center">
<tr>
<td align="center"> Motion Name </td>
<td align="center"> Checkpoint Link </td>
<td align="center"> Training data </td>
</tr>
<tr>
<td align="center">Birds fly</td>
<td align="center"><a href="https://pan.baidu.com/s/1nuZVRj-xRqkHySQQ3jCFkw">Baidu Disk (pwd: jj0o)</a></td>
<td align="center"><a href="https://pan.baidu.com/s/10fi8KoBrGJMpLQKhUIaFSQ">Baidu Disk (pwd: w96b)</a></td>
</tr>
<tr>
<td align="center">Firework</td>
<td align="center"><a href="https://pan.baidu.com/s/1zJnn5bZpGzChRHJdO9x6WA">Baidu Disk (pwd: wj1p)</a></td>
<td align="center"><a href="https://pan.baidu.com/s/1uIyw0Q70svWNM5z7DFYkiQ">Baidu Disk (pwd: oamp)</a></td>
</tr>
<tr>
<td align="center">Helicopter</td>
<td align="center"><a href="https://pan.baidu.com/s/1oj6t_VFo9cX0vTZWDq8q3w">Baidu Disk (pwd: egpe)</a></td>
<td align="center"><a href="https://pan.baidu.com/s/1MYMjIFyFTiLGEX1w0ees2Q">Baidu Disk (pwd: t4ba)</a></td>
</tr>
<tr>
<td align="center">Horse run</td>
<td align="center"><a href="https://pan.baidu.com/s/1lkAFZuEnot4JGruLe6pR3g">Baidu Disk (pwd: 19ldExcerpt of 12,507 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:7cd8f9f9e31b300c, topic:text-to-video, topic:video-editing, desc:video generation
matched fp:7cd8f9f9e31b300c, topic:stable-diffusion, topic:diffusion-models, readme:stable diffusion