Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Rich-Text-to-Image Generation
| Date | Stars |
|---|---|
| 2026-07-24 | 800 |
| 2026-07-25 | 800 |
| 2026-07-28 | 800 |
| 2026-07-30 | 800 |
| 2026-07-31 | 800 |
| 2026-08-06 | 800 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Rich-Text-to-Image ### [Project Page](https://rich-text-to-image.github.io/) | [Paper](https://arxiv.org/abs/2304.06720) | [Video](https://youtu.be/ihDbAUh0LXk) | [HuggingFace Demo](https://huggingface.co/spaces/songweig/rich-text-to-image) | [A1111 Extension](https://github.com/songweige/sd-webui-rich-text) **tl;dr:** We use various formatting information from rich text, including font size, color, style, and footnote, to increase control of text-to-image generation. Our method enables explicit token reweighting, precise color rendering, local style control, and detailed region synthesis. https://github.com/songweige/rich-text-to-image/assets/22885450/ccd186d1-f0fc-4e55-80c0-06afd6cb84c0 ***Expressive Text-to-Image Generation with Rich Text*** <br> [Songwei Ge](https://songweige.github.io/), [Taesung Park](https://taesung.me/), [Jun-Yan Zhu](https://www.cs.cmu.edu/~junyanz/), [Jia-Bin Huang](https://jbhuang0604.github.io/)<br> UMD, Adobe, CMU<br> ICCV 2023 ## Updates * [09/26] We initiate an implementation of an [A1111 WebUI extension](https://github.com/songweige/sd-webui-rich-text) for integrating the rich-text editor for text-to-image generation. * [09/24] We now support LoRA checkpoints. Please find the demo and the latest code in [this branch](https://github.com/songweige/rich-text-to-image/tree/lora). * [08/09] Our method now support [SD-XL](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0) with `--model SDXL`, and various fintuned model like [ANIMAGINE-XL](https://huggingface.co/Linaqruf/animagine-xl) with `--model AnimeXL`. * [07/14] Our paper is accepted by ICCV 2023. * [05/03] We update our approach to abtain more robust and accurate token maps and improve the structure preservation from plain-text results. The following images are generated by the new method with the prompt taken from [this issue](https://github.com/SongweiGe/rich-text-to-image/issues/9). * [04/17] We release the [rich-text-to-image demo](https://huggingface.co/spaces/songweig/rich-text-to-image) on HuggingFace Space. Thanks to [HuggingFace](https://huggingface.co/) team for the help with the demo! * [04/13] We release the [rich-text-to-image generation](https://arxiv.org/abs/2304.06720), which leverages the formatting options of a rich-text editor to facilitate controlling the text-to-image generation. ## Setup This code was tested with Python 3.8, [Pytorch](https://pytorch.org/) 1.11 and supports a [Stable Diffusion v1-5](https://huggingface.co/runwayml/stable-diffusion-v1-5) or [Stable Diffusion XL](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0) or [ANIMAGINE-XL](https://huggingface.co/Linaqruf/animagine-xl) through hugginface. ``` git clone https://github.com/SongweiGe/rich-text-to-image.git cd rich-text-to-image/ conda env create -f environment.yaml pip install git+https://github.com/openai/CLIP.git conda activate rich-text ``` ## Usage In general, our pipeline contains two steps. We first input the plain text prompt to the diffusion model and compute the cross-attention maps to associate each token with the spatial region. The rich-text prompts obtained from the editor are stored in JSON format, providing attributes for each token span. We use a new region-based diffusion to render each region’s attributes into a globally coherent image. Below we provide the basic usage of various font formats. ### Rich text to JSON We encode the rich text into JSON format and use it as the input to the rich-text conditioned sampling script `sample.py`. To automatically generate a JSON string based on rich text, you can use our [rich-text-to-json](https://rich-text-to-image.github.io/rich-text-to-json.html) interface, which is a purely static webpage that can be readily incorporated into any rich-text-based application. ### Rich-text JSON to Image  You may start generating images with rich-text JSON via our local gradio demo: ``` python gradio_app.py ``` Or through the command line:
Excerpt of 9,072 characters
Read on GitHub
28
Jun-Yan Zhu · Carnegie Mellon University: · United States
3
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:517a2a475f6263aa, topic:diffusion-models, name:text-to-image, desc:text-to-image