Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
General video interaction platform based on LLMs, including Video ChatGPT
| Date | Stars |
|---|---|
| 2026-07-31 | 257 |
| 2026-08-06 | 257 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<p align="center" width="100%">
<img src="https://github-production-user-asset-6210df.s3.amazonaws.com/78398294/237149157-ab45a2aa-7b86-444b-92e0-099eb0289890.png" width="80%" height="80%">
</p>
<div>
<div align="center">
Zehuan Huang 
Haoran Feng 
Chongzhi Zhang
</br>
<a href='https://lucassheng.github.io/' target='_blank'>Lu Sheng</a> 
<a href='https://liuziwei7.github.io/' target='_blank'>Ziwei Liu</a> 
<a href='https://amandajshao.github.io/' target='_blank'>Jing Shao</a>
</div>
<div>
<div align="center">
Beihang University, Nanyang Technological University
</div>
# Dolphin
<!-- <a src="https://img.shields.io/badge/%F0%9F%A4%97-Open%20in%20Spaces-blue" href="" target="_blank">
<img src="https://img.shields.io/badge/%F0%9F%A4%97-Open%20in%20Spaces-blue" alt="Open in Huggingface">
</a>
<a src="https://img.shields.io/badge/GPU%20Demo-Open-green?logo=alibabacloud" href="" target="_blank">
<img src="https://img.shields.io/badge/GPU%20Demo-Open-green?logo=alibabacloud">
</a> -->
<a src="https://img.shields.io/twitter/follow/kaleido_lab?style=social" href="https://twitter.com/kaleido_lab" target="_blank">
<img src="https://img.shields.io/twitter/follow/kaleido_lab?style=social">
</a>
Dolphin is a general video interaction platform based on large language models. Our team is trying to build a chatbot for video understanding, processing and generation.
The project 🐬 Dolphin is still under construction, we will continue to update it and welcome contributions/pull requests from the community.
## 📽️ Demo
<a href="https://www.youtube.com/watch?v=d8giiMAWMLc" target="_blank">
<img src="https://user-images.githubusercontent.com/78398294/236398299-050439e4-b870-44f5-8341-db87ca849748.png" alt="Dolphin, a general video interaction platform based on LLMs, from BUAA & NTU">
</a>
## 🔥 Updates
- 2023/05/06: Code release & Online Demo
- Video understanding: Q&A about the video.
- Video processing: Basic functions such as trimming video, adding subtitles, extracting audio, and adding audio using [moviepy](https://github.com/Zulko/moviepy). Video to pose/depth/canny also included.
- Video generation: Text to video, pose/depth and text to video, and video pix2pix.
## 💬 Example
https://user-images.githubusercontent.com/78398294/236603247-b7381154-743c-4262-ad30-75f11e34a91d.mp4
## 🔨 Quick Start
Prepare the project and environment:
```Bash
# We recommend using conda to manage the environment and use python 3.8
conda create -n dolphin python=3.8
conda activate dolphin
# Clone the respository:
git clone https://github.com/BUAA-PrismGroup/dolphin.git
cd dolphin
# Install dependencies:
pip install -r requirements.txt
```
To start 🐬 Dolphin, you can specify the GPU/CPU assignment by `--load`, the parameter indicates which Video Foundation Model to use and where it will be loaded to. The model and device are separated by underline `_`, while the different models are separated by comma `,`. The available Video Foundation Models can be found in the following table or `configs/backends.yaml`.
For example, if you want to load VideoCaptioning to cuda:0 and MoviepyInterface to cpu, you can use: `VideoCaptioning_cuda:0,MoviepyInterface_cpu`.
Some starting commands are as follows.
```Bash
# Advice for CPU Users
python video_chatgpt.py --load VideoCaptioning_cpu,ImageCaptioning_cpu,ModelscopeT2V_cpu
# Advice for 4 Tesla V100 32GB (Full usage)
# You can specify the device where each model is loaded in `configs/backend.yaml`
python video_chatgpt.py
```
## 💾 GPU memory usage
| **Foundation Model** | **GPU Memory (MB)** |
| - | - |
| VideoCaptioning | 13393 |
| ImageCaptioning | 8429 |
| MoviepyInterface | 0 |
| Video2Canny | 0 |
| Video2Pose | 1361 |
| Video2Depth | 1521 |
| CannyText2Video | 6659 |
| PoseText2Video | 6721 |
| DepthText2Video | 6673 |
| VideoPix2Pix | 5251 |
| ModelscopeT2V | 6535 |
| Text2Audio | 5797 |
## 🛠️ How to expand
Our project framework iExcerpt of 8,082 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:9643c5de06590870, llm:Repository topics: chatgpt, gpt, langchain, video; description: 'General video interaction platform based on LLMs, including Video ChatGPT'
matched fp:9643c5de06590870, llm:Repository topics: chatgpt, gpt, langchain, video; description: 'General video interaction platform based on LLMs, including Video ChatGPT'
matched fp:9643c5de06590870, llm:Repository topics: chatgpt, gpt, langchain, video; description: 'General video interaction platform based on LLMs, including Video ChatGPT'