Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
CosmicMan: A Text-to-Image Foundation Model for Humans (CVPR 2024)
| Date | Stars |
|---|---|
| 2026-07-31 | 348 |
| 2026-08-01 | 348 |
| 2026-08-06 | 348 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# CosmicMan: A Text-to-Image Foundation Model for Humans <img src="./assets/1.png" width="96%" height="96%"> [Shikai Li](mailto:[email protected]), [Jianglin Fu](mailto:[email protected]), [Kaiyuan Liu](mailto:[email protected]), [Wentao Wang](mailto:[email protected]), [Kwan-Yee Lin](mailto:[email protected]), [Wayne Wu](mailto:[email protected]) <br> **[[Video Demo]](https://www.youtube.com/watch?v=CsZKA27tQDA)** | **[[Project Page]](https://cosmicman-cvpr2024.github.io/)** | **[[Paper]](https://arxiv.org/abs/2404.01294)** | **[[Huggingface Gradio]](https://huggingface.co/spaces/cosmicman/CosmicMan-SDXL)** **Abstract:** We present **CosmicMan**, a text-to-image foundation model specialized for generating high-fidelity human images. Unlike current general-purpose foundation models that are stuck in the dilemma of inferior quality and text-image misalignment for humans, CosmicMan enables generating photo-realistic human images with meticulous appearance, reasonable structure, and precise text-image alignment with detailed dense descriptions. <br> At the heart of CosmicMan's success are the new reflections and perspectives on data and model: (1) We found that data quality and a scalable data production flow are essential for the final results from trained models. Hence, we propose a new data production paradigm, **Annotate Anyone**, which serves as a perpetual data flywheel to produce high-quality data with accurate yet cost-effective annotations over time. Based on this, we constructed a large-scale dataset **CosmicMan-HQ 1.0**, with 6 Million high-quality real-world human images in a mean resolution of 1488x1255, and attached with precise text annotations deriving from 115 Million attributes in diverse granularities. (2) We argue that a text-to-image foundation model specialized for humans must be pragmatic - easy to integrate into down-streaming tasks while effective in producing high-quality human images. Hence, we propose to model the relationship between dense text descriptions and image pixels in a decomposed manner, and present **D**ecomposed-**A**ttention-**R**efocus**ing** (**Daring**) training framework. It seamlessly decomposes the cross-attention features in existing text-to-image diffusion model, and enforces attention refocusing without adding extra modules. Through Daring, we show that explicitly discretizing continuous text space into several basic groups that align with human body structure is the key to tackling the misalignment problem in a breeze. <br> ## Updates - [18/06/2024] Training code is released! - [14/06/2024] Pretrained models [CosmicMan-SDXL](https://huggingface.co/cosmicman/CosmicMan-SDXL), [CosmicMan-SD](https://huggingface.co/cosmicman/CosmicMan-SD) and inference scripts are released. [Online Huggingface Gradio Demo](https://huggingface.co/spaces/cosmicman/CosmicMan-SDXL) is also released. - [29/04/2024] [CosmicManHQ-1.0 Dataset](https://huggingface.co/datasets/cosmicman/CosmicManHQ-1.0) is released! - [05/04/2024] :fire::fire::fire:CosmicMan is selected as **Highlight Paper** (324 out of 11,532 submissions) at CVPR 2024! - [02/04/2024] [Technical report](https://arxiv.org/abs/2404.01294) has been released. - [01/03/2024] CosmicMan has been accepted by CVPR2024. ## Usage Our CosmicMan-SDXL is based on [stabilityai/stable-diffusion-xl-base-1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0) and UNet checkpoint for CosmicMan-SDXL can be download from huggingface page [cosmicman/CosmicMan-SDXL](https://huggingface.co/cosmicman/CosmicMan-SDXL). Our CosmicMan-SD is based on [runwayml/stable-diffusion-v1-5](https://huggingface.co/runwayml/stable-diffusion-v1-5) and UNet checkpoint for CosmicMan-SD can be download from huggingface page [cosmicman/CosmicMan-SD](https://huggingface.co/cosmicman/CosmicMan-SD). ### Requirements ``` conda create -n cosmicman python=3.10 source activate cosmicman pip install torch torchvision torchaudio --ind
Excerpt of 9,301 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:9c00e37eb74096a1, desc:text-to-image