Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
[CVPR 2025] Aesthetic Post-Training Diffusion Models from Generic Preferences with Step-by-step Preference Optimization
| Date | Stars |
|---|---|
| 2026-07-24 | 271 |
| 2026-07-25 | 271 |
| 2026-07-28 | 271 |
| 2026-07-30 | 271 |
| 2026-07-31 | 271 |
| 2026-08-06 | 271 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<div align="center">
# Aesthetic Post-Training Diffusion Models from Generic Preferences with Step-by-step Preference Optimization
[Zhanhao Liang](https://github.com/RockeyCoss), [Yuhui Yuan](https://www.microsoft.com/en-us/research/people/yuyua/), [Shuyang Gu](https://cientgu.github.io), [Bohan Chen](https://github.com/BHCHENGIT), [Tiankai Hang](https://tiankaihang.github.io/), [Mingxi Cheng](https://sites.google.com/a/usc.edu/mingxicheng/), [Ji Li](https://sites.google.com/a/usc.edu/jili/), [Liang Zheng](https://zheng-lab.cecs.anu.edu.au)
${{\color{Red}\Huge{\textsf{ CVPR\ 2025\ \}}}}\$
<a href="https://arxiv.org/abs/2406.04314"><img src="https://img.shields.io/badge/Paper-arXiv-red?style=for-the-badge" height=22.5></a>
<a href="https://rockeycoss.github.io/spo.github.io/"><img src="https://img.shields.io/badge/Project-Page-blue?style=for-the-badge" height=22.5></a>
<a href="https://huggingface.co/SPO-Diffusion-Models"><img src="https://img.shields.io/badge/Hugging-Face-yellow?style=for-the-badge" height=22.5></a>
</div>
This is the official implementation of SPO, introduced in [Aesthetic Post-Training Diffusion Models from Generic Preferences with Step-by-step Preference Optimization](https://arxiv.org/abs/2406.04314).
## Abstract
<p>
Generating visually appealing images is fundamental to modern text-to-image generation models.
A potential solution to better aesthetics is direct preference optimization (DPO),
which has been applied to diffusion models to improve general image quality including prompt alignment and aesthetics.
Popular DPO methods propagate preference labels from clean image pairs to all the intermediate steps along the two generation trajectories.
However, preference labels provided in existing datasets are blended with layout and aesthetic opinions, which would disagree with aesthetic preference.
Even if aesthetic labels were provided (at substantial cost), it would be hard for the two-trajectory methods to capture nuanced visual differences at different steps.
</p>
<p>
To improve aesthetics economically, this paper uses existing generic preference data and introduces step-by-step preference optimization
(SPO) that discards the propagation strategy and allows fine-grained image details to be assessed. Specifically,
at each denoising step, we 1) sample a pool of candidates by denoising from a shared noise latent,
2) use a step-aware preference model to find a suitable win-lose pair to supervise the diffusion model, and
3) randomly select one from the pool to initialize the next denoising step.
This strategy ensures that diffusion models focus on the subtle, fine-grained visual differences
instead of layout aspect. We find that aesthetics can be significantly enhanced by accumulating these
improved minor differences.
</p>
<p>
When fine-tuning Stable Diffusion v1.5 and SDXL, SPO yields significant
improvements in aesthetics compared with existing DPO methods while not sacrificing image-text alignment
compared with vanilla models. Moreover, SPO converges much faster than DPO methods due to the use of
more correct preference labels provided by the step-aware preference model.
</p>
## Method Overview
## TODO
- [x] Release training code for step-aware preference model
- [x] Release training code for SPO
- [x] Release checkpoints and inference code
- [x] Initialization
## Gallery
<table>
<tr>
<td><img src="assets/teaser/0.png" alt="teaser example 0" width="200"/></td>
<td><img src="assets/teaser/1.png" alt="teaser example 1" width="200"/></td>
<td><img src="assets/teaser/2.png" alt="teaser example 2" width="200"/></td>
<td><img src="assets/teaser/3.png" alt="teaser example 3" width="200"/></td>
</tr>
<tr>
<td><img src="assets/teaser/4.png" alt="teaser example 4" width="200"/></td>
<td><img src="assets/teaser/5.png" alt="teaser eExcerpt of 6,729 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:96f7f77fd04adb1b, topic:diffusion-models, topic:text-to-image, readme:text-to-image
matched fp:96f7f77fd04adb1b, topic:dpo, readme:fine-tuning, readme:fine tuning