Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Official implementation for "Blended Latent Diffusion" [SIGGRAPH 2023]
| Date | Stars |
|---|---|
| 2026-07-24 | 631 |
| 2026-07-25 | 631 |
| 2026-07-28 | 632 |
| 2026-07-30 | 632 |
| 2026-08-06 | 632 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Blended Latent Diffusion [SIGGRAPH 2023] <a href="https://omriavrahami.com/blended-latent-diffusion-page/"><img src="https://img.shields.io/static/v1?label=Project&message=Website&color=blue"></a> <a href="https://arxiv.org/abs/2206.02779"><img src="https://img.shields.io/badge/arXiv-2206.02779-b31b1b.svg"></a> <a href="https://opensource.org/licenses/MIT"><img src="https://img.shields.io/badge/License-MIT-yellow.svg"></a> <a href="https://pytorch.org/"><img src="https://img.shields.io/badge/PyTorch->=2.1.0-Red?logo=pytorch"></a> <a href="https://omriavrahami.com/blended-latent-diffusion-page/"><img src="docs/teaser.png" /></a> > <a href="https://omriavrahami.com/blended-latent-diffusion-page/">**Blended Latent Diffusion**</a> > > Omri Avrahami, Ohad Fried, Dani Lischinski > > Abstract: The tremendous progress in neural image generation, coupled with the emergence of seemingly omnipotent vision-language models has finally enabled text-based interfaces for creating and editing images. Handling *generic* images requires a diverse underlying generative model, hence the latest works utilize diffusion models, which were shown to surpass GANs in terms of diversity. One major drawback of diffusion models, however, is their relatively slow inference time. In this paper, we present an accelerated solution to the task of *local* text-driven editing of generic images, where the desired edits are confined to a user-provided mask. Our solution leverages a recent text-to-image Latent Diffusion Model (LDM), which speeds up diffusion by operating in a lower-dimensional latent space. We first convert the LDM into a local image editor by incorporating Blended Diffusion into it. Next we propose an optimization-based solution for the inherent inability of this LDM to accurately reconstruct images. Finally, we address the scenario of performing local edits using thin masks. We evaluate our method against the available baselines both qualitatively and quantitatively and demonstrate that in addition to being faster, our method achieves better precision than the baselines while mitigating some of their artifacts <div> <img src="docs/object_editing.gif" width="200px"/> <img src="docs/new_object.gif" width="200px"/> <img src="docs/graffiti.gif" width="200px"/> </div> # Applications ### Background Editing <img src="docs/applications/background_edit.png" /> ### Text Generation <img src="docs/applications/text.png" /> ### Multiple Predictions <img src="docs/applications/multiple_predictions.png" /> ### Alter an Existing Object <img src="docs/applications/object_edit.png" /> ### Add a New Object <img src="docs/applications/new_object.png" /> ### Scribble Editing <img src="docs/applications/scribble_edit.png" /> # Installation Install the conda virtual environment: ```bash $ conda env create -f environment.yaml $ conda activate ldm ``` # Usage ## New :fire: - Stable Diffusion Implementation You can use the newer Stable Diffusion implementation based on [Diffusers](https://github.com/huggingface/diffusers) library. For that, you need to install PyTorch 2.1 and Diffusers via the following commands: ```bash $ conda install pytorch==2.1.0 torchvision==0.16.0 pytorch-cuda=11.8 -c pytorch -c nvidia $ pip install -U diffusers==0.19.3 ``` * For using Stable Diffusion XL (requires a stronger GPU), use the following script: ```bash $ python scripts/text_editing_SDXL.py --prompt "a stone" --init_image "inputs/img.png" --mask "inputs/mask.png" ``` You can use smaller `--batch_size` in order to save GPU memory. * For using Stable Diffusion v2.1, use the following script: ```bash $ python scripts/text_editing_SD2.py --prompt "a stone" --init_image "inputs/img.png" --mask "inputs/mask.png" ``` ## Old - Latent Diffusion Model Implementation For using the old implementation, based on the Latent Diffusion Model (LDM), you need first to download the pre-trained weights (5.7GB): ```bash $ mkdir -p models/ldm/text2img-large/ $ wget -O models/ldm/text2
Excerpt of 6,348 characters
Read on GitHub
Omri Avrahami · The Hebrew University of Jerusalem
10
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:d97b6e4c768e4f60, topic:diffusion-models, topic:image-generation, topic:text-to-image
matched fp:d97b6e4c768e4f60, topic:deep-learning, topic:pytorch