Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
[CVPR 2024] MovieChat: From Dense Token to Sparse Memory for Long Video Understanding
| Date | Stars |
|---|---|
| 2026-07-24 | 704 |
| 2026-07-25 | 704 |
| 2026-07-28 | 706 |
| 2026-07-30 | 706 |
| 2026-08-06 | 706 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<img src="src/assets/logo.png" height="120px" align="left"> # MovieChat [](https://arxiv.org/abs/2307.16449v4) [](https://arxiv.org/abs/2404.17176) > **MovieChat: From Dense Token to Sparse Memory for Long Video Understanding** > Enxin Song*, Wenhao Chai*, Guanhong Wang*, Yucheng Zhang, Haoyang Zhou, Feiyang Wu, Xun Guo, Tian Ye, Yan Lu, Jenq-Neng Hwang, Gaoang Wang✉️ > _CVPR 2024._ <img width="1155" alt="image" src="https://github.com/user-attachments/assets/4c0412d3-0729-4f56-af0c-1ee3eeac8f99"> MovieChat can handle videos with >10K frames on a 24GB graphics card. MovieChat has a 10000× advantage over other methods in terms of the average increase in GPU memory cost per frame (21.3KB/f to ~200MB/f). <p align="center" width="100%"> <a target="_blank"><img src="src/assets/wave.gif" alt="MovieChat" style="width: 80%; min-width: 200px; display: block; margin: auto;"></a> </p> <h5 align="center"> If you like our project, please give us a star ⭐ on GitHub for the latest update.</h5> ## 🔢 MovieChat-1K leaderboard Feel free to PR your new results! | Model with Link | Comment | Breakpoint Acc | Global Acc | |-----------------------------------------------|------------------------------|------------|----------------| | [Video-LLaMA](https://arxiv.org/pdf/2306.02858) | End-to-end | 39.1 | 51.7 | | [VideoChat](https://arxiv.org/abs/2305.06355) | End-to-end | 46.1 | 57.8 | | [TimeChat](https://arxiv.org/pdf/2406.11333) | CoT, ICL, train on MovieChat| 46.1 | 73.8 | | [VideoChatGPT](https://arxiv.org/pdf/2306.05424) | End-to-end | 48.0 | 47.6 | | [MovieChat](https://arxiv.org/abs/2307.16449v4) (baseline) | End-to-end | 48.3 | 62.3 | | [MovieChat+](https://arxiv.org/abs/2404.17176) (baseline) | End-to-end | 49.6 | 71.2 | | [Long-LLaVA](https://arxiv.org/abs/2411.13093) | Eng-to-end | 54.0 | 69.6 | | [Long-LLaVA + Video-RAG](https://arxiv.org/abs/2411.13093) | Eng-to-end | 54.5 | 72.9 | | [Streaming Long Video](https://arxiv.org/abs/2405.16009) | Train on MovieChat | 54.9 | 90.4 | | [DrVideo](https://arxiv.org/pdf/2406.12846) | RAG | 56.7 | 93.1 | | [ReWind](https://arxiv.org/pdf/2411.15556) | End-to-end | 57.2 | 87.6 | | [HERMES](https://arxiv.org/pdf/2408.17443) | Train on MovieChat | 57.3 | 78.6 | | [Flash-VStream](https://arxiv.org/abs/2406.08085) | Train on MovieChat | 59.6 | 96.0 | | [MM-Screenplayer](https://arxiv.org/pdf/2406.17309) | RAG | 68.8 | 87.5 | | [VILA1.5-8B](https://openreview.net/pdf?id=oS79Tw3G0c) | End-to-end | - | 40.0 | | [FocusChat](https://arxiv.org/pdf/2412.12833) | End-to-end | - | 60.0| | [llavaonevision-MovieChat](https://github.com/rese1f/MovieChat) | End-to-end | - | 79.0 | | [Sullam Jeoung, _et al_](https://arxiv.org/pdf/2410.20252) | Agent | - | 84.8 | | [SEAL](https://arxiv.org/pdf/2412.01798) | Train on MovieChat | - | 86.8 | | [HEM-LLM](https://arxiv.org/pdf/2409.06299) | Unknown training dataset | - | 90.6 | ## 🔢 Evaluation of MovieChat on Existing Benchmarks Sort in alphabetical order. | Benchmark | Results | |-----------|---------| | ActivityNet-QA | Acc. / Score: 45.7 / 3.4 | | Charades-STA | R@1(IOU =0.3): 8.8 • R@1(IOU =0.5): 2.9 • R@1(IOU =0.7): 1.3 | | CineClipQA | Overall: 20.86/2.11 • Description: 23.67/2.41 • Intention: 30.19/2.41 • Perception: 21.80/1.97 • Temporality: 16.32/1.97 • Spaciality: 16.40/1.98 | | CVRR-ES | Average: 16.41 | | EgoSchema | Top 1 Acc: 53.5
Excerpt of 30,246 characters
Read on GitHub
Enxin Song · University of Pennsylvania · China
89
Wenhao Chai · Princeton University
28
3
Ikko Eltociear Ashimine · Japan
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:881bf8994601be89, topic:large-language-models, topic:llama
matched fp:881bf8994601be89, topic:computer-vision
matched fp:881bf8994601be89, topic:dataset, readme:dataset