Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
[NeurIPS 2025 Spotlight] A Generalist Diffusion Model for Vision Perception
| Date | Stars |
|---|---|
| 2026-07-31 | 321 |
| 2026-08-06 | 321 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<p align="center">
<img src="assets/logo.png" height=200>
</p>
<hr>
<div align="center">
## 🎯 DICEPTION: A Generalist Diffusion Model for Vision Perception
<p align="center">
<a href="https://aim-uofa.github.io/Diception/"><b>📖 Project Page</b></a> |
<a href="https://arxiv.org/abs/2502.17157"><b>📄 Paper Link</b></a> |
<a href="https://huggingface.co/spaces/Canyu/Diception-Demo"><b>🤗 Huggingface Demo</b></a>
</p>
</div>
> One single model solves multiple perception tasks, on par with SOTA!
## 📰 News
- 2025-09-21: 🚀 Model and inference code released
- 2025-09-19: 🌟 Accepted as NeurIPS 2025 Spotlight
- 2025-02-25: 📝 Paper released
## 🛠️ Installation
```
conda create -n diception python=3.10 -y
conda activate diception
pip install -r requirements.txt
```
## 👾 Inference
### ⚡ Quick Start
#### 🧩 Model Setup
1. **Download SD3 Base Model**:
Download the Stable Diffusion 3 medium model from:
https://huggingface.co/stabilityai/stable-diffusion-3-medium-diffusers
2. **Download Trained Weights**:
Please download the model from Hugging Face: https://huggingface.co/Canyu/DICEPTION
3. **Update Paths**:
Set `--pretrained_model_path` to your SD3 path, and set `--diception_path` to the local path of the downloaded `DICEPTION_v1.pth`.
4. **Sample JSON for Batch Inference**:
We provide several JSON examples for batch inference in the `DATA/jsons/evaluate` directory.
#### ▶️ Option 1: Simple Inference Script
For single image inference:
```bash
python inference.py \
--image path/to/your/image.jpg \
--prompt "[[image2depth]]" \
--pretrained_model_path PATH_TO_SD3 \
--diception_path PATH_TO_DICEPTION_v1.PTH \
--output_dir ./outputs \
--guidance_scale 2 \
--num_inference_steps 28
```
**With coordinate points** (for interactive segmentation):
```bash
python inference.py \
--image path/to/your/image.jpg \
--prompt "[[image2segmentation]]" \
--pretrained_model_path PATH_TO_SD3 \
--diception_path PATH_TO_DICEPTION_v1.PTH \
--output_dir ./outputs \
--guidance_scale 2 \
--num_inference_steps 28 \
--points "0.3,0.5;0.7,0.2"
```
The `--points` parameter accepts coordinates in format `"y1,x1;y2,x2;y3,x3"` where:
- Coordinates are normalized to [0,1] range
- Format is (y,x) where y=height/image_height, x=width/image_width
- Multiple points are separated by semicolons
- Maximum 5 points are supported
#### 📦 Option 2: Batch Inference
For batch processing with a JSON dataset:
```bash
python batch_inference.py \
--pretrained_model_path PATH_TO_SD3 \
--diception_path PATH_TO_DICEPTION_v1.PTH \
--input_path example_batch.json \
--data_root_path ./ \
--save_path ./batch_results \
--batch_size 4 \
--guidance_scale 2 \
--num_inference_steps 28
# --save_npy (for depth and normal value)
```
**JSON Format for Batch Inference**:
The input JSON file should contain a list of tasks in the following format:
```json
[
{
"input": "path/to/image1.jpg",
"caption": "[[image2segmentation]]"
},
{
"input": "path/to/image2.jpg",
"caption": "[[image2depth]]"
},
{
"input": "path/to/image3.jpg",
"caption": "[[image2segmentation]]",
"target": {
"path": "path/to/sa1b.json" (For convenience, randomly select a region for point prompt from the GT json)
}
}
]
```
### 📋 Supported Tasks
DICEPTION supports various vision perception tasks:
- **Depth Estimation**: `[[image2depth]]`
- **Surface Normal Estimation**: `[[image2normal]]`
- **Pose Estimation**: `[[image2pose]]`
- **Interactive Segmentation**: `[[image2segmentation]]`
- **Semantic Segmentation**: `[[image2semantic]] + (category in coco)`, e.g. `[[image2semantic]] person`
- **Entity Segmentation**: `[[image2entity]]`
### 💡 Inference Tips
- **General settings**: For best overall results, use `--num_inference_steps 28` and `--guidance_scale 2.0`.
- **1-step/few-step inference**: We found flow-matching diffusion models naturally Excerpt of 4,964 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:f2e70e6703a696cb, desc:diffusion model