Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Official Implementation for "Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models" (SIGGRAPH 2023)
| Date | Stars |
|---|---|
| 2026-07-24 | 771 |
| 2026-07-25 | 771 |
| 2026-07-28 | 771 |
| 2026-07-30 | 771 |
| 2026-07-31 | 770 |
| 2026-08-06 | 770 |
Today
— stars today
This week
-1 stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models (SIGGRAPH 2023) > Hila Chefer*, Yuval Alaluf*, Yael Vinker, Lior Wolf, Daniel Cohen-Or > Tel Aviv University > \* Denotes equal contribution > > Recent text-to-image generative models have demonstrated an unparalleled ability to generate diverse and creative imagery guided by a target text prompt. While revolutionary, current state-of-the-art diffusion models may still fail in generating images that fully convey the semantics in the given text prompt. We analyze the publicly available Stable Diffusion model and assess the existence of catastrophic neglect, where the model fails to generate one or more of the subjects from the input prompt. Moreover, we find that in some cases the model also fails to correctly bind attributes (e.g., colors) to their corresponding subjects. To help mitigate these failure cases, we introduce the concept of Generative Semantic Nursing (GSN), where we seek to intervene in the generative process on the fly during inference time to improve the faithfulness of the generated images. Using an attention-based formulation of GSN, dubbed Attend-and-Excite, we guide the model to refine the cross-attention units to attend to all subject tokens in the text prompt and strengthen — or excite — their activations, encouraging the model to generate all subjects described in the text prompt. We compare our approach to alternative approaches and demonstrate that it conveys the desired concepts more faithfully across a range of text prompts. <a href="https://arxiv.org/abs/2301.13826"><img src="https://img.shields.io/badge/arXiv-2301.13826-b31b1b.svg" height=22.5></a> <a href="https://attendandexcite.github.io/Attend-and-Excite/"><img src="https://img.shields.io/static/v1?label=Project&message=Website&color=red" height=20.5></a> <a href="https://youtu.be/9EWs2IX4cus"><img src="https://img.shields.io/static/v1?label=5-Minute&message=Video&color=darkgreen" height=20.5></a> [](https://huggingface.co/spaces/hysts/Attend-and-Excite) [](https://replicate.com/daanelson/attend-and-excite) <p align="center"> <img src="docs/teaser.jpg" width="800px"/> <br> Given a pre-trained text-to-image diffusion model (e.g., Stable Diffusion) our method, Attend-and-Excite, guides the generative model to modify the cross-attention values during the image synthesis process to generate images that more faithfully depict the input text prompt. Stable Diffusion alone (top row) struggles to generate multiple objects (e.g., a horse and a dog). However, by incorporating Attend-and-Excite (bottom row) to strengthen the subject tokens (marked in blue), we achieve images that are more semantically faithful with respect to the input text prompts. </p> ## Description Official implementation of our Attend-and-Excite paper. ## Setup ### Environment Our code builds on the requirement of the official [Stable Diffusion repository](https://github.com/CompVis/stable-diffusion). To set up their environment, please run: ``` conda env create -f environment/environment.yaml conda activate ldm ``` On top of these requirements, we add several requirements which can be found in `environment/requirements.txt`. These requirements will be installed in the above command. ### Hugging Face Diffusers Library Our code relies also on Hugging Face's [diffusers](https://github.com/huggingface/diffusers) library for downloading the Stable Diffusion v1.4 model. ## Usage <p align="center"> <img src="docs/results.jpg" width="800px"/> <br> Example generations outputted by Stable Diffusion with Attend-and-Excite. </p> To generate an image, you can simply run the `run.py` script. For example, ``` python run.py --prompt "a cat and a dog" --seeds [0] --token_in
Excerpt of 8,113 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:a271725e119a7166, topic:stable-diffusion, topic:diffusion-models, topic:text-to-image
