Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
[ECCV 2024 Oral] Code for paper: An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models
| Date | Stars |
|---|---|
| 2026-07-31 | 592 |
| 2026-08-03 | 593 |
| 2026-08-06 | 593 |
| 2026-08-12 | 594 |
| 2026-08-18 | 593 |
| 2026-08-26 | 594 |
| 2026-08-31 | 593 |
| 2026-09-03 | 594 |
| 2026-09-04 | 595 |
| 2026-09-08 | 596 |
| 2026-09-09 | 596 |
| 2026-09-11 | 597 |
| 2026-09-14 | 598 |
| 2026-09-17 | 599 |
| 2026-09-20 | 599 |
Today
— stars today
This week
+2 stars this week
This month
+6 stars this month
Momentum
0.0
growth rate 0.34%/day
<h1 align="center">FastV</h1>
<p align="center">
<a href="https://arxiv.org/abs/2403.06764">
<img alt="Static Badge" src="https://img.shields.io/badge/arXiv-2403.06764-red"></a>
<a href="https://www.youtube.com/watch?v=V0I6ntvb9pQ">
<img alt="Static Badge" src="https://img.shields.io/badge/Tutorial-Youtube-yellow"></a>
*2024.08.13: FastV is selected as Oral Presentation(2%) at ECCV 2024!*
*2024.07.01: FastV is accepted to ECCV 2024!*
*FastV is a plug-and-play inference acceleration method for large vision language models relying on visual tokens. It could reach 45\% theoretical FLOPs reduction without harming the performance through pruning redundant visual tokens in deep layers.*
<div align=center>
Tutorial video on youtube. Thanks to [
Soroush Mehraban](https://www.youtube.com/@soroushmehraban)!
[](https://www.youtube.com/watch?v=V0I6ntvb9pQ)
https://github.com/pkunlp-icler/FastV/assets/43977888/e6ae44b9-ebb9-4b69-b534-cee3441b1d22
<img width="600" src="./figs/fastv_tradeoff.png"/>
</div>
---
*Scheduled Updates🔥*
0. - [x] Setup
1. - [x] Visualization [Online Demo](https://www.fastv.work/)
2. - [x] LVLM Inefficent Visual Attention Visualization Code
3. - [x] FastV Inference and Evaluation
4. - [x] Latency Test Reproduction Guideline
5. - [x] Support HuggingFace Models (LLaVA)
6. - [x] Support KV Cache
7. - [x] Support lmms-eval and report results
Stay tuned!
## Setup
```bash
conda create -n fastv python=3.10
conda activate fastv
cd src
bash setup.sh
```
## Online Demo
We provide an [online demo](https://www.fastv.work/) for the FastV model. You can upload an image, enter a prompt, and select the number of layers to get the generated response and visualize the attention maps.
If you want to start your own demo, run the following script:
```bash
python demo.py --model-path ./llava-v1.5-7b
```
## Visualization: Inefficient Attention over Visual Tokens
we provide a script (./src/FastV/inference/visualization.sh) to reproduce the visualization result of each LLaVA model layer for a given image and prompt.
```bash
bash ./src/FastV/inference/visualization.sh
```
or
```bash
python ./src/FastV/inference/plot_inefficient_attention.py \
--model-path "PATH-to-HF-LLaVA1.5-Checkpoints" \
--image-path "./src/LLaVA/images/llava_logo.png" \
--prompt "Describe the image in details."\
--output-path "./output_example"\
```
Model output and attention maps for different layers would be stored at "./output_example"
<div align=center>
<img width="600" src="./figs/attn_map.png"/>
</div>
## FastV Inference and Evaluation
We provide code to reproduce the ablation study on K and R values, as shown in figure-7 in the paper. This implementation masks out the discarded tokens (no speed up) in deep layers for convenience and fair performance comparison.
*ocrvqa*
```bash
bash ./src/FastV/inference/eval/eval_ocrvqa_fastv_token_mask.sh
```
<div align=center>
<img width="300" src="./figs/ablation_ocrvqa.png"/><br>
Results
</div>
### Latency Experiment Reproduction
You could use following code to reproduce FastV's latency experiment on aokvqa. We conduct the following experiments on one A100 GPU (80G)
```bash
bash ./src/FastV/inference/eval/eval_aokvqa_latency_fastv_inplace.sh
```
*aokvqa results*
| Model | Score | latency / first output token (A100 80G) | GPU Memory |
| ----------------------------- | ----- | --------------------------------------- | ---------- |
| 7B Vanilla Decoding | 76.8 | 0.138s | 18G |
| 13B Vanilla Decoding | 81.9 | 0.203s | 33G |
| \- 13B FastV (K=2 R=25%) | 81.8 | 0.181s | 29G |
| \- 13B FastV (K=2 R=50%) | 81.3 | 0.155s | 28G |
| \- 13B FastV (K=2 R=75%) | 80.9 | **0.124s** Excerpt of 9,480 characters
Read on GitHub
Liang Chen · UniPat AI · China
91
Haozhe zhao
5
2
Ikko Eltociear Ashimine · Japan
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:224ef68e90ff7999, desc:vision-language