Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Prompt-Free Diffusion: Taking "Text" out of Text-to-Image Diffusion Models, arxiv 2023 / CVPR 2024
| Date | Stars |
|---|---|
| 2026-07-31 | 759 |
| 2026-08-04 | 759 |
| 2026-08-06 | 759 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Prompt-Free Diffusion
[](https://huggingface.co/spaces/shi-labs/Prompt-Free-Diffusion)
[](https://pytorch.org/)
[](https://opensource.org/licenses/MIT)
This repo hosts the official implementation of:
[Xingqian Xu](https://ifp-uiuc.github.io/), Jiayi Guo, Zhangyang Wang, Gao Huang, Irfan Essa, and [Humphrey Shi](https://www.humphreyshi.com/home), **Prompt-Free Diffusion: Taking "Text" out of Text-to-Image Diffusion Models**, [Paper arXiv Link](https://arxiv.org/abs/2305.16223).
## News
- **[2023.06.20]: SDWebUI plugin is created, repo at this [link](https://github.com/xingqian2018/sd-webui-prompt-free-diffusion)**
- [2023.05.25]: Our demo is running on [HuggingFace🤗](https://huggingface.co/spaces/shi-labs/Prompt-Free-Diffusion)
- [2023.05.25]: Repo created
## Introduction
**Prompt-Free Diffusion** is a diffusion model that relys on only visual inputs to generate new images, handled by **Semantic Context Encoder (SeeCoder)** by substituting the commonly used CLIP-based text encoder. SeeCoder is **reusable to most public T2I models as well as adaptive layers** like ControlNet, LoRA, T2I-Adapter, etc. Just drop in and play!
<p align="center">
<img src="assets/figures/reusability.png" width="90%">
</p>
## Performance
<p align="center">
<img src="assets/figures/qualitative_show.png" width="99%">
</p>
## Network
<p align="center">
<img src="assets/figures/prompt_free_diffusion.png" width="60%">
</p>
<p align="center">
<img src="assets/figures/seecoder.png" width="99%">
</p>
## Setup
```
conda create -n prompt-free-diffusion python=3.10
conda activate prompt-free-diffusion
pip install torch==2.0.0+cu117 torchvision==0.15.1 --extra-index-url https://download.pytorch.org/whl/cu117
pip install -r requirements.txt
```
## Demo
We provide a WebUI empowered by [Gradio](https://github.com/gradio-app/gradio). Start the WebUI with the following command:
```
python app.py
```
## Pretrained models
To support the full functionality of our demo. You need the following models located in these paths:
```
└── pretrained
├── pfd
| ├── vae
| │ └── sd-v2-0-base-autokl.pth
| ├── diffuser
| │ ├── AbyssOrangeMix-v2.safetensors
| │ ├── AbyssOrangeMix-v3.safetensors
| │ ├── Anything-v4.safetensors
| │ ├── Deliberate-v2-0.safetensors
| │ ├── OpenJouney-v4.safetensors
| │ ├── RealisticVision-v2-0.safetensors
| │ └── SD-v1-5.safetensors
| └── seecoder
| ├── seecoder-v1-0.safetensors
| ├── seecoder-pa-v1-0.safetensors
| └── seecoder-anime-v1-0.safetensors
└── controlnet
├── control_sd15_canny_slimmed.safetensors
├── control_sd15_depth_slimmed.safetensors
├── control_sd15_hed_slimmed.safetensors
├── control_sd15_mlsd_slimmed.safetensors
├── control_sd15_normal_slimmed.safetensors
├── control_sd15_openpose_slimmed.safetensors
├── control_sd15_scribble_slimmed.safetensors
├── control_sd15_seg_slimmed.safetensors
├── control_v11p_sd15_canny_slimmed.safetensors
├── control_v11p_sd15_lineart_slimmed.safetensors
├── control_v11p_sd15_mlsd_slimmed.safetensors
├── control_v11p_sd15_openpose_slimmed.safetensors
├── control_v11p_sd15s2_lineart_anime_slimmed.safetensors
├── control_v11p_sd15_softedge_slimmed.safetensors
└── preprocess
├── hed
│ └── ControlNetHED.pth
├── midas
│ └── dpt_hybrid-midas-501f0c75.pt
├── mlsd
│ └── mlsd_large_512_fp32.pth
├── openpose
│ ├── body_pose_model.pth
│ ├── facenet.pth
│ └── hand_pose_model.pth
└── pidinetExcerpt of 5,346 characters
Read on GitHub
22
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:2d5c58adc7780a6a, desc:text-to-image