Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Official implementation of OneDiffusion paper (CVPR 2025)
| Date | Stars |
|---|---|
| 2026-07-24 | 662 |
| 2026-07-25 | 662 |
| 2026-07-28 | 662 |
| 2026-07-30 | 662 |
| 2026-08-06 | 662 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# One Diffusion to Generate Them All
<p align="left">
<a href="https://lehduong.github.io/OneDiffusion-homepage/">
<img alt="Build" src="https://img.shields.io/badge/Project%20Page-OneDiffusion-yellow">
</a>
<a href="https://arxiv.org/abs/2411.16318">
<img alt="Build" src="https://img.shields.io/badge/arXiv%20paper-2411.16318-b31b1b.svg">
</a>
<a href="https://huggingface.co/spaces/lehduong/OneDiffusion">
<img alt="License" src="https://img.shields.io/badge/HF%20Demo-🤗-lightblue">
</a>
<a href="https://huggingface.co/lehduong/OneDiffusion">
<img alt="Build" src="https://img.shields.io/badge/HF%20Model-🤗-yellow">
</a>
</p>
<h4 align="left">
<p>
<a href=#news>News</a> |
<a href=#quick-start>Quick start</a> |
<a href=https://github.com/lehduong/OneDiffusion/blob/main/PROMPT_GUIDE.md>Prompt guide & Supported tasks </a> |
<a href=#qualitative-results>Qualitative results</a> |
<a href="#license">License</a> |
<a href="#citation">Citation</a>
<p>
</h4>
<p align="center">
<img src="assets/teaser.png" alt="Teaser Image" width="800">
</p>
This is official repo of OneDiffusion, a versatile, large-scale diffusion model that seamlessly supports bidirectional image synthesis and understanding across diverse tasks.
**For more detail, read our paper [here](https://arxiv.org/abs/2411.16318).**
## News
- 📦 2024/12/11: [Huggingface space](https://huggingface.co/spaces/lehduong/OneDiffusion) is online. Reduce the VRAM requirements for running demo with Molmo to 21GB.
- 📦 2024/12/10: Released [weight](https://huggingface.co/lehduong/OneDiffusion) and inference code.
- ✨ 2024/12/06: Added image editing from instruction.
- ✨ 2024/12/02: Added subject-driven generation
## Installation
```
conda create -n onediffusion_env python=3.8 &&
conda activate onediffusion_env &&
pip install torch==2.3.1 torchvision==0.18.1 torchaudio==2.3.1 --index-url https://download.pytorch.org/whl/cu118 &&
pip install "git+https://github.com/facebookresearch/pytorch3d.git" &&
pip install -r requirements.txt
```
## Quick start
Check `inference.py` for more detailed. For text-to-image, you can use below code snipe.
```
import torch
from onediffusion.diffusion.pipelines.onediffusion import OneDiffusionPipeline
device = torch.device('cuda:0')
pipeline = OneDiffusionPipeline.from_pretrained("lehduong/OneDiffusion").to(device=device, dtype=torch.bfloat16)
NEGATIVE_PROMPT = "monochrome, greyscale, low-res, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, username, blurry, artist name, poorly drawn, bad anatomy, wrong anatomy, extra limb, missing limb, floating limbs, disconnected limbs, mutation, mutated, ugly, disgusting, blurry, amputation"
output = pipeline(
prompt="[[text2image]] A bipedal black cat wearing a huge oversized witch hat, a wizards robe, casting a spell,in an enchanted forest. The scene is filled with fireflies and moss on surrounding rocks and trees",
negative_prompt=NEGATIVE_PROMPT,
num_inference_steps=50,
guidance_scale=4,
height=1024,
width=1024,
)
output.images[0].save('text2image_output.jpg')
```
You can run the gradio demo with:
```
python gradio_demo.py --captioner molmo # [molmo, llava, disable]
```
The demo provides guidance and helps format the prompt properly for each task.
- By default, it loads the **quantized** Molmo for captioning source images. ~~which significantly increases memory usage. You generally need a GPU with at least $40$ GB of memory to run the demo.~~ You generally need a GPU with at least $21$ GB of memory to run the demo.
- Opting to use LLaVA can reduce this requirement to $\approx 27$ GB, though the resulting captions may be less accurate in some cases.
- You can also manually provide the caption for each input image and run with Excerpt of 7,409 characters
Read on GitHub
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:2279cff52a9b4744, topic:stable-diffusion, topic:diffusion-models, topic:text-to-image