This repository collects methods for evaluating visual generation.
Within this repository, we collect works that aim to answer some critical questions in the field of evaluating visual generation, such as:
- Model Evaluation: How does one determine the quality of a specific image or video generation model?
- Sample/Content Evaluation: What methods can be used to evaluate the quality of a particular generated image or video?
- User Control Consistency Evaluation: How to tell how well the generated images and videos align with the user controls or inputs?
This repository is updated periodically. If you have suggestions for additional resources, updates on methodologies, or fixes for expiring links, please feel free to do any of the following:
- raise an Issue,
- nominate awesome related works with Pull Requests,
- We are also contactable via email (
ZIQI002 at e dot ntu dot edu dot sg).
- 1. Evaluation Metrics of Generative Models
- 2. Evaluation Metrics of Condition Consistency
- 3. Evaluation Systems of Generative Models
- 3.1. Evaluation of Unconditional Image Generation
- 3.2. Evaluation of Text-to-Image Generation
- 3.3. Evaluation of Text-Based Image Editing
- 3.4. Evaluation of Neural Style Transfer
- 3.5. Evaluation of Video Generation
- 3.6. Evaluation of Text-to-Motion Generation
- 3.7. Evaluation of Model Trustworthiness
- 3.8. Evaluation of Entity Relation
- 3.9. Agentic Evaluation
- 3.10. Evaluation of Unified Multimodal Models
- 4. Improving Visual Generation with Evaluation / Feedback / Reward
- 5. Quality Assessment for AIGC
- 6. Study and Rethinking
- 7. Other Useful Resources
| Metric | Paper | Code |
|---|---|---|
| Inception Score (IS) | Improved Techniques for Training GANs (NeurIPS 2016) | |
| Fréchet Inception Distance (FID) | GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium (NeurIPS 2017) |   |
| Kernel Inception Distance (KID) | Demystifying MMD GANs (ICLR 2018) |   |
| CLIP-FID | The Role of ImageNet Classes in Fréchet Inception Distance (ICLR 2023) |   |
| Precision-and-Recall | Assessing Generative Models via Precision and Recall (2018-05-31, NeurIPS 2018) Improved Precision and Recall Metric for Assessing Generative Models (NeurIPS 2019) |
  |
| Renyi Kernel Entropy (RKE) | An Information-Theoretic Evaluation of Generative Models in Learning Multi-modal Distributions (NeurIPS 2023) |  |
| CLIP Maximum Mean Discrepancy (CMMD) | Rethinking FID: Towards a Better Evaluation Metric for Image Generation (CVPR 2024) |  |
| Fréchet Wavelet Distance (FWD) | Fréchet Wavelet Distance: A Domain-Agnostic Metric For Image Generation (ICLR 2025) |  |
-
Physics-Based Benchmarking Metrics for Multimodal Synthetic Images (2025-11-19)
-
Making Reconstruction FID Predictive of Diffusion Generation FID (2026-03-05)
Note: Introduces iFID.
-
Gram-MMD: A Texture-Aware Metric for Image Realism Assessment (2026-04-03)
Note: Introduces Gram-MMD (GMMD).
-
Relative Classification Accuracy: A Calibrated Metric for Identity Consistency in Fine-Grained K-pop Face Generation (2026-01-22)
Note: Introduces Relative Classification Accuracy (RCA).
-
Towards a Scalable Reference-Free Evaluation of Generative Models (2024-07-03)
-
FaceScore: Benchmarking and Enhancing Face Quality in Human Generation (2024-06-24)
Note: Face Score introduced
-
Global-Local Image Perceptual Score (GLIPS): Evaluating Photorealistic Quality of AI-Generated Images (2024-05-15)
-
Unifying and extending Precision Recall metrics for assessing generative models (2024-05-02)
-
Enhancing Plausibility Evaluation for Generated Designs with Denoising Autoencoder (2024-03-08)
Note: Fréchet Denoised Distance introduced
-
Virtual Classifier Error (VCE) from Virtual Classifier: A Reversed Approach for Robust Image Evaluation (2024-03-04)
-
An Interpretable Evaluation of Entropy-based Novelty of Generative Models (2024-02-27)
-
Semantic Shift Rate from Discovering Universal Semantic Triggers for Text-to-Image Synthesis (2024-02-12)
-
Optimizing Prompts Using In-Context Few-Shot Learning for Text-to-Image Generative Models (2024-01-01)
Note: Quality Loss introduced
-
Attribute Based Interpretable Evaluation Metrics for Generative Models (2023-10-26)
-
On quantifying and improving realism of images generated with diffusion (2023-09-26)
Note: Image Realism Score introduced
-
Probabilistic Precision and Recall Towards Reliable Evaluation of Generative Models (2023-09-04)
Note: P-precision and P-recall introduced
-
Learning to Evaluate the Artness of AI-generated Images (2023-05-08)
Note: ArtScore, metric for images resembling authentic artworks by artists
-
Training-Free Location-Aware Text-to-Image Synthesis (2023-04-26)
Note: New evaluation metric for control capability of location aware generation task
-
Feature Likelihood Divergence: Evaluating the Generalization of Generative Models Using Samples (2023-02-09)
-
LGSQE: Lightweight Generated Sample Quality Evaluatoin (2022-11-08)
-
SSD: Towards Better Text-Image Consistency Metric in Text-to-Image Generation (2022-10-27)
Note: Semantic Similarity Distance introduced
-
Layout-Bridging Text-to-Image Synthesis (2022-08-12)
Note: Layout Quality Score (LQS), new metric for evaluating the generated layout
-
Rarity Score: A New Metric to Evaluate the Uncommonness of Synthesized Images (2022-06-17)
-
Mutual Information Divergence: A Unified Metric for Multimodal Generative Models (2022-05-25)
Note: evaluates text to image and utilizes vision language models (VLM)
-
TREND: Truncated Generalized Normal Density Estimation of Inception Embeddings for GAN Evaluation (2021-04-30, ECCV 2022)
-
CFID from Conditional Frechet Inception Distance (2021-03-21)
-
On Self-Supervised Image Representations for GAN Evaluation (2021-01-12)
Note: SwAV, self-supervised image representation model
-
Random Network Distillation as a Diversity Metric for Both Image and Text Generation (2020-10-13)
Note: RND metric introduced
-
The Vendi Score: A Diversity Evaluation Metric for Machine Learning (2022-10-05)
-
CIS from Evaluation Metrics for Conditional Image Generation (2020-04-26)
-
Text-To-Image Synthesis Method Evaluation Based On Visual Patterns (2020-04-09)
-
Cscore: A Novel No-Reference Evaluation Metric for Generated Images (2020-03-25)
-
SceneFID from Object-Centric Image Generation from Layouts (2020-03-16)
-
Reliable Fidelity and Diversity Metrics for Generative Models (2020-02-23, ICML 2020)
-
Effectively Unbiased FID and Inception Score and where to find them (2019-11-16, CVPR 2020)
-
On the Evaluation of Conditional GANs (2019-07-11)
Note:Fréchet Joint Distance (FJD), which is able to assess image quality, conditional consistency, and intra-conditioning diversity within a single metric.
-
Quality Evaluation of GANs Using Cross Local Intrinsic Dimensionality (2019-05-02)
CrossLID, assesses the local intrinsic dimensionality
-
A domain agnostic measure for monitoring and evaluating GANs (2018-11-13)
-
Learning to Generate Images with Perceptual Similarity Metrics (2015-11-19)
Multiscale structural-similarity score introduced
-
A No-Reference Image Blur Metric Based on the Cumulative Probability of Blur Detection (CPBD) (2011-03-28)
| Metric | Paper | Code |
|---|---|---|
| FID-vid | GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium (NeurIPS 2017) | |
| Fréchet Video Distance (FVD) | Towards Accurate Generative Models of Video: A New Metric & Challenges (arXiv 2018) FVD: A new Metric for Video Generation (2019-05-04) (Note: ICLR 2019 Workshop DeepGenStruct Program Chairs) |
 |
- Linear Separability & Perceptual Path Length (PPL) from A Style-Based Generator Architecture for Generative Adversarial Networks (2020-01-09)
| Metric | Condition | Pipeline | Code | References |
|---|---|---|---|---|
CLIP Score (a.k.a. CLIPSIM) |
Text | cosine similarity between the CLIP image and text embeddings |  PyTorch Lightning |
CLIP Paper (ICML 2021). Metrics first used in CLIPScore Paper (arXiv 2021) and GODIVA Paper (arXiv 2021) applies it in video evaluation. |
| Mask Accuracy | Segmentation Mask | predict the segmentatio mask, and compute pixel-wise accuracy against the ground-truth segmentation mask | any segmentation method for your setting | |
| DINO Similarity | Image of a Subject (human / object etc) | cosine similarity between the DINO embeddings of the generated image and the condition image |  |
DINO paper. Metric is proposed in DreamBooth. |
-
NexusScore, NaturalScore and GmeScore from OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generation (2025-06-03)
Note: NexusScore - Identity Consistency - image retrieval + cosine similarity; NaturalScore - Identity Naturalness - prompting gpt4o; GmeScore - Text - cosine similarity between the GME image and text embeddings.
-
FaceSim-Cur from Identity-Preserving Text-to-Video Generation by Frequency Decomposition (2024-11-26)
Note: NFaceSim-Cur - Face image of human - cosine similarity between the curricularface embeddings of the generated face and the input face.
-
Manipulation Direction (MD) from Manipulation Direction: Evaluating Text-Guided Image Manipulation Based on Similarity between Changes in Image and Text Modalities (2023-11-20)
-
Semantic Similarity Distance: Towards better text-image consistency metric in text-to-image generation (2022-12-02)
-
On the Evaluation of Conditional GANs (2019-07-11)
Note: Fréchet Joint Distance (FJD), which is able to assess image quality, conditional consistency, and intra-conditioning diversity within a single metric.
-
Classification Accuracy Score for Conditional Generative Models (2019-05-26)
Note: New metric Classification Accuracy Score (CAS)
-
Visual-Semantic (VS) Similarity from Photographic Text-to-Image Synthesis with a Hierarchically-nested Adversarial Network (2018-12-26)
-
Semantically Invariant Text-to-Image Generation (2018-09-06)
Note: They evaluate image-text similarity via image captioning
-
Inferring Semantic Layout for Hierarchical Text-to-Image Synthesis (2018-01-16)
Note: An object detector based metric is proposed.
-
CLPIPS: A Personalized Metric for AI-Generated Image Similarity (2026-03-26)
-
MEt3R: Measuring Multi-View Consistency in Generated Images (2025-01-10)
-
Pathology-Guided Virtual Staining Metric for Evaluation and Training (2025-07-16)
Note: Introduces the Pathology-Aware Perceptual Image Similarity (PaPIS) metric.
| Metrics | Paper | Code |
|---|---|---|
| Learned Perceptual Image Patch Similarity (LPIPS) | The Unreasonable Effectiveness of Deep Features as a Perceptual Metric (2018-01-11) (CVPR 2018) |   |
| Structural Similarity Index (SSIM) | Image quality assessment: from error visibility to structural similarity (TIP 2004) |   |
| Peak Signal-to-Noise Ratio (PSNR) | - | |
| Multi-Scale Structural Similarity Index (MS-SSIM) | Multiscale structural similarity for image quality assessment (SSC 2004) | PyTorch-Metrics |
| Feature Similarity Index (FSIM) | FSIM: A Feature Similarity Index for Image Quality Assessment (TIP 2011) |  |
The community has also been using DINO or CLIP features to measure the semantic similarity of two images / frames.
There are also recent works on new methods to measure visual similarity (more will be added):
-
Synthetic Melanoma Image Generation and Evaluation Using Generative Adversarial Networks (2026-03-13)
-
AesBench: An Expert Benchmark for Multimodal Large Language Models on Image Aesthetics Perception (2024-01-16)
-
A Lightweight Generalizable Evaluation and Enhancement Framework for Generative Models and Generated Samples (2024-04-16)
-
Anomaly Score: Evaluating Generative Models and Individual Generated Images based on Complexity and Vulnerability (2023-12-17, CVPR 2024)
-
Using Skew to Assess the Quality of GAN-generated Image Features (2023-10-31)
Note: Skew Inception Distance introduced
-
StudioGAN: A Taxonomy and Benchmark of GANs for Image Synthesis (2022-06-19)
-
HYPE: A Benchmark for Human eYe Perceptual Evaluation of Generative Models (2019-04-01)
-
An Improved Evaluation Framework for Generative Adversarial Networks (2018-03-20)
Note: Class-Aware Frechet Distance introduced
-
Simile Understanding in Text-to-Image Models: An Evaluation Framework (2026-08-05)
-
Can Text-to-Image Models Draw from the Right Frame of Reference? (2026-08-04)
-
ExpertVerse: A General-Purpose Benchmark for Expert-Level Reasoning in Knowledge-Intensive Visual Synthesis (2026-07-21)
-
DynEval: Holistic Evaluations of T2I Generative Models in the Wild (2026-07-13)
-
TIIF-Bench: How Does Your T2I Model Follow Your Instructions? (2025-06-02)
-
PIPBench: A Profile-Inclusive Framework for Personalized Image Generation Evaluation (2026-07-07)
-
KathaTrace: Diagnosing Semantic Trajectory Collapse in Generated Visual Narratives (2026-07-01)
-
Are Text-to-Image Models Inductivist Turkeys? A Counterfactual Benchmark for Causal Reasoning (2026-06-23)
-
Scaling Multi-Reference Image Generation with Dynamic Reward Optimization (2026-06-25)
-
Qwen-Image-Bench: From Generation to Creation in Text-to-Image Evaluation (2026-05-27)
-
Can AI Draw Science? A Benchmark for Evaluating Scientific Figure Generation by Text-to-Image and Multimodal Models (2026-06-24)
-
GeoFidelity-Bench: Evaluating Segment-Level Geographic Fidelity in Text-to-Image Street-View Generation (2026-06-22)
-
WeGenBench: A Multidimensional Diagnostic Benchmark towards Text-to-Image Model Optimization (2026-06-18)
-
GarmentSketch: Large-scale Sketch-to-Fashion Benchmark (2026-06-12)
-
Faithful, Enriched, and Precise: Benchmarking Natural-Science Illustration Generation by T2I models (2026-06-04)
-
TASTE: A Designer-Annotated Multi-Dimensional Preference Dataset for AI-Generated Graphic Design (2026-05-20)
-
TextAlign: Preference Alignment for Text Rendering with Hierarchical Rewards (2026-05-19)
-
DetailMaster: Can Your Text-to-Image Model Handle Long Prompts? (2025-05-22)
-
Benchmarking and Enhancing Text-to-Image Models for Generating Visual Representations in Early Arithmetic Education (2026-05-29)
-
DyCoRM: Dynamic Criterion-Aware Reward Modeling for Text-to-Image Generation (2026-05-25)
-
MaSC: A Masked Similarity Metric for Evaluating Concept-Driven Generation (2026-05-21)
-
A Framework for Evaluating Zero-Shot Image Generation in Concept-based Explainability (2026-05-19)
-
Benchmarking and Evolving Reason-Reflect-Rectify for Reflective Visual Generation (2026-05-19)
-
ColorConceptBench: A Benchmark for Probabilistic Color-Concept Understanding in Text-to-Image Models (2026-01-23)
-
TIQA: Human-Aligned Perceptual Text Quality Assessment in Generated Images (2026-03-07)
-
Benchmarking Layout-Guided Diffusion Models through Unified Semantic-Spatial Evaluation in Closed and Open Settings (2026-04-28)
-
Knowledge Visualization: A Benchmark and Method for Knowledge-Intensive Text-to-Image Generation (2026-04-24)
-
Pref-GRPO: Pairwise Preference Reward-based GRPO for Stable Text-to-Image Reinforcement Learning (2025-08-28)
-
ViStoryBench: Comprehensive Benchmark Suite for Story Visualization (2025-05-30)
-
When Identities Collapse: A Stress-Test Benchmark for Multi-Subject Personalization (2026-03-27)
-
BizGenEval: A Systematic Benchmark for Commercial Visual Content Generation (2026-03-26)
-
Diffusion Probe: Generated Image Result Prediction Using CNN Probes (2026-02-27)
-
TechImage-Bench: Rubric-Based Evaluation for Technical Image Generation (2025-12-13)
-
Evaluating Generative Models via One-Dimensional Code Distributions (2026-03-09)
-
The Art That Poses Back: Assessing AI Pastiches after Contemporary Artworks (2026-03-06)
-
Toward Early Quality Assessment of Text-to-Image Diffusion Models (2026-03-03)
-
Enhancing Spatial Understanding in Image Generation via Reward Modeling (2026-02-27)
-
UniGenBench++: A Unified Semantic Evaluation Benchmark for Text-to-Image Generation (2025-10-21)
-
Everything in Its Place: Benchmarking Spatial Intelligence of Text-to-Image Models (2026-01-28)
-
Text-to-Image Diffusion Models Cannot Count, and Prompt Refinement Cannot Help (2025-03-10)
-
TempViz: On the Evaluation of Temporal Knowledge in Text-to-Image Models (2026-01-21)
-
Scientific Image Synthesis: Benchmarking, Methodologies, and Downstream Utility (2026-01-17)
-
Architecture inside the mirage: evaluating generative image models on architectural style, elements, and typologies (2026-01-14)
-
GenEval 2: Addressing Benchmark Drift in Text-to-Image Evaluation (2025-12-18)
-
Color Bind: Exploring Color Perception in Text-to-Image Models (2025-08-27)
-
Prompt fidelity of ChatGPT4o / Dall-E3 text-to-image visualisations (2025-10-21)
-
CAP: Evaluation of Persuasive and Creative Image Generation (2024-12-10)
-
TIT-Score: Evaluating Long-Prompt Based Text-to-Image Alignment via Text-to-Image-to-Text Consistency (2025-10-03)
-
AcT2I: Evaluating and Improving Action Depiction in Text-to-Image Models (2025-09-19)
-
STRICT: Stress Test of Rendering Images Containing Text (2025-05-25)
-
MagicMirror: A Large-Scale Dataset and Benchmark for Fine-Grained Artifacts Assessment in Text-to-Image Generation (2025-09-12)
-
FLUX-Reason-6M & PRISM-Bench: A Million-Scale Text-to-Image Reasoning Dataset and Comprehensive Benchmark (2025-09-11)
-
Evaluating Text-to-Image and Text-to-Video Synthesis with a Conditional Fréchet Distance (2025-03-27)
-
T2I-ReasonBench: Benchmarking Reasoning-Informed Text-to-Image Generation (2025-08-24)
-
7Bench: a Comprehensive Benchmark for Layout-guided Text-to-image Models (2025-08-18)
-
VISTAR: A User-Centric and Role-Driven Benchmark for Text-to-Image Evaluation (2025-08-08)
-
Scendi Score: Prompt-Aware Diversity Evaluation via Schur Complement of CLIP Embeddings (2024-12-24)
-
KITTEN: A Knowledge-Intensive Evaluation of Image Generation on Visual Entities (2024-10-15)
-
Enhancing Reward Models for High-quality Image Generation: Beyond Text-Image Alignment (2025-07-25)
-
Evaluating Attribute Confusion in Fashion Text-to-Image Generation (2025-07-09)
-
OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generation (2025-06-09)
-
Q-Eval-100K: Evaluating Visual Quality and Alignment Level for Text-to-Vision Content (2025-03-04)
-
GenSpace: Benchmarking Spatially-Aware Image Generation (2025-05-30)
-
T2I-FactualBench: Benchmarking the Factuality of Text-to-Image Models with Knowledge-Intensive Concepts (2024-12-05)
-
MIBE: Multi-subject Interaction Benchmark and Evaluator for Personalized Image Generation (2026-07-01)
-
CogCanvas: A Benchmark for Evaluating Multi-Subject Reference-Based Image Generation (2026-06-14)
-
GlazyBench: A Benchmark for Ceramic Glaze Property Prediction and Image Generation (2026-05-07)
-
PortraitCraft: A Benchmark for Portrait Composition Understanding and Generation (2026-04-04)
-
AIBench: Evaluating Visual-Logical Consistency in Academic Illustration Generation (2026-03-30)
-
MultiBind: A Benchmark for Attribute Misbinding in Multi-Subject Generation (2026-03-23)
-
Counting Hallucinations in Diffusion Models (2025-10-15)
-
Beyond the Pixels: VLM-based Evaluation of Identity Preservation in Reference-Guided Synthesis (2025-11-11)
-
Mind-the-Glitch: Visual Correspondence for Detecting Inconsistencies in Subject-Driven Generation (2025-09-26)
-
Visual Personalization Turing Test (2026-01-30, CVPR 2026)
-
Finer-Personalization Rank: Fine-Grained Retrieval Examines Identity Preservation for Personalized Generation (2025-12-22)
-
Human Preference-Aligned Concept Customization Benchmark via Decomposed Evaluation (2025-09-03)
-
GenColorBench: A Color Evaluation Benchmark for Text-to-Image Generation Models (2025-10-23)
-
ServImage: An Image Generation and Editing Benchmark from Real-World Commercial Imaging Services (2026-04)
Note: Introduces ServImageScore.
-
LongT2IBench: A Benchmark for Evaluating Long Text-to-Image Generation with Graph-Structured Annotations (2025-12-10)
-
OverLayBench: A Benchmark for Layout-to-Image Generation with Dense Overlaps (2025-09-23)
Note: Introduces OverLayScore.
-
ICE-Bench: A Unified and Comprehensive Benchmark for Image Creating and Editing (2025-03)
Note: Introduces VLLM-QA.
-
MultiHuman-Testbench: Benchmarking Image Generation for Multiple Humans (2025-06-25)
-
MMMG: A Massive, Multidisciplinary, Multi-Tier Generation Benchmark for Text-to-Image Reasoning (2025-06-12)
Note: Introduces MMMG-Score.
-
MMIG-Bench: Towards Comprehensive and Explainable Evaluation of Multi-Modal Image Generation Models (2025-05)
Note: Introduces the Aspect Matching Score (AMS).
-
SciFlow-Bench: Evaluating Structure-Aware Scientific Diagram Generation via Inverse Parsing (2026-02-10)
-
IGenBench: Benchmarking the Reliability of Text-to-Infographic Generation (2026-01-08)
-
Automatic and Reliable Evaluation for Academic Caption-to-Figure Generation with LMMs (ACL 2026)
-
Scendi Score from Scendi Score: Prompt-Aware Diversity Evaluation via Schur Complement of CLIP Embeddings (ICCV 2025)
Note: Introduces the Scendi Score for prompt-aware diversity evaluation.
-
P2P: Automated Paper-to-Poster Generation and Fine-Grained Benchmark (ICLR 2026)
-
Too Vivid to Be Real? Benchmarking and Calibrating Generative Color Fidelity (CVPR 2026)
Note: Introduces the Color Fidelity Metric (CFM).
-
PosterIQ: A Design Perspective Benchmark for Poster Understanding and Generation (2026-03-25)
-
MultiBanana: A Challenging Benchmark for Multi-Reference Text-to-Image Generation (2025-11-28)
-
GenExam: A Multidisciplinary Text-to-Image Exam (2025-09-17)
-
What Makes a Scene ? Scene Graph-based Evaluation and Feedback for Controllable Generation (2024-05-26)
-
Commonsense-T2I Challenge: Can Text-to-Image Generation Models Understand Commonsense? (2024-08-12)
-
WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generation (2025-03-10)
-
Why Settle for One? Text-to-ImageSet Generation and Evaluation (2025-06-29)
-
LMM4LMM: Benchmarking and Evaluating Large-multimodal Image Generation with LMMs (2025-04-11)
-
Robust and Discriminative Speaker Embedding via Intra-Class Distance Variance Regularization (2018-09)
Note: IntraClass Average Distance(ICAD) - Diversity
-
REAL: Realism Evaluation of Text-to-Image Generation Models for Effective Data Augmentation. (2025-02-15)
-
ABHINAW: A method for Automatic Evaluation of Typography within AI-Generated Images (2024-09-18)
-
Finding the Subjective Truth: Collecting 2 Million Votes for Comprehensive Gen-AI Model Evaluation (2024-09-18)
-
Beyond Aesthetics: Cultural Competence in Text-to-Image Models (2024-07-09)
Note: CUBE benchmark introduced
-
MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation? (2024-07-05)
Note: MJ-Bench introduced
-
MIGC++: Advanced Multi-Instance Generation Controller for Image Synthesis (2024-07-02)
Note: Benchmark COCO-MIG and Multimodal-MIG introduced
-
Analyzing Quality, Bias, and Performance in Text-to-Image Generative Models (2024-06-28)
-
DreamBench++: A Human-Aligned Benchmark for Personalized Image Generation (2024-06-24)
-
Six-CD: Benchmarking Concept Removals for Benign Text-to-image Diffusion Models (2024-06-21)
-
Evaluating Numerical Reasoning in Text-to-Image Models (2024-06-20)
Note: GeckoNum introduced
-
Holistic Evaluation for Interleaved Text-and-Image Generation (2024-06-20)
Note: InterleavedBench and InterleavedEval metric introduced
-
GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generation (2024-06-19)
-
Decomposed evaluations of geographic disparities in text-to-image models (2024-06-17)
Note: new metric Decomposed Indicators of Disparities introduced
-
PhyBench: A Physical Commonsense Benchmark for Evaluating Text-to-Image Models (2024-06-17)
Note: PhyBench introduced
-
Make It Count: Text-to-Image Generation with an Accurate Number of Objects (2024-06-14)
-
Commonsense-T2I Challenge: Can Text-to-Image Generation Models Understand Commonsense? (2024-06-11)
Note: Commonsense-T2I, benchmark for real-life commonsense reasoning capabilities of T2I models
-
Unified Text-to-Image Generation and Retrieval (2024-06-09)
Note: TIGeR-Bench, benchmark for evaluation of unified text-to-image generation and retrieval.
-
PQPP: A Joint Benchmark for Text-to-Image Prompt and Query Performance Prediction (2024-06-07)
-
GenAI Arena: An Open Evaluation Platform for Generative Models (2024-06-06)
-
A-Bench: Are LMMs Masters at Evaluating AI-generated Images? (2024-06-05)
-
Multidimensional Preference Score from Learning Multi-dimensional Human Preference for Text-to-Image Generation (2024-05-23)
-
Evolving Storytelling: Benchmarks and Methods for New Character Customization with Diffusion Models (2024-05-20)
Note: NewEpisode benchmark introduced
-
Training-free Subject-Enhanced Attention Guidance for Compositional Text-to-image Generation (2024-05-11)
Note: GroundingScore metric introduced
-
TheaterGen: Character Management with LLM for Consistent Multi-turn Image Generation (2024-04-29)
Note: consistent score r introduced
-
Exposing Text-Image Inconsistency Using Diffusion Models (2024-04-28)
-
Revisiting Text-to-Image Evaluation with Gecko: On Metrics, Prompts, and Human Ratings (2024-04-25)
-
Multimodal Large Language Model is a Human-Aligned Annotator for Text-to-Image Generation (2024-04-23)
-
Infusion: Preventing Customized Text-to-Image Diffusion from Overfitting (2024-04-22)
Note: Latent Fisher divergence and Wasserstein metric introduced
-
TAVGBench: Benchmarking Text to Audible-Video Generation (2024-04-22)
-
Object-Attribute Binding in Text-to-Image Generation: Evaluation and Control (2024-04-21)
-
Magic Clothing: Controllable Garment-Driven Image Synthesis (2024-04-15)
Note: new metric Matched-Points-LPIPS introduced
-
GenAI-Bench: A Holistic Benchmark for Compositional Text-to-Visual Generation (2024-04-09)
Note: GenAI-Bench was introduced in a previous paper 'Evaluating Text-to-Visual Generation with Image-to-Text Generation'
-
Detect-and-Compare from Identity Decoupling for Multi-Subject Personalization of Text-to-Image Models (2024-04-05)
-
Enhancing Text-to-Image Model Evaluation: SVCS and UCICM (2024-04-02)
Note: Evaluation metrics: Semantic Visual Consistency Score and User-Centric Image Coherence Metric
-
Evaluating Text-to-Visual Generation with Image-to-Text Generation (2024-04-01)
-
Measuring Style Similarity in Diffusion Models (2024-04-01)
-
AAPMT: AGI Assessment Through Prompt and Metric Transformer (2024-03-28)
-
FlashEval: Towards Fast and Accurate Evaluation of Text-to-image Diffusion Generative Models (2024-03-25)
-
Refining Text-to-Image Generation: Towards Accurate Training-Free Glyph-Enhanced Image Generation (2024-03-25)
Note: LenCom-Eval introduced
-
Exploring GPT-4 Vision for Text-to-Image Synthesis Evaluation (2024-03-20)
-
DialogGen: Multi-modal Interactive Dialogue System for Multi-turn Text-to-Image Generation (2024-03-13)
Note: DialogBen introduced
-
Evaluating Text-to-Image Generative Models: An Empirical Study on Human Image Synthesis (2024-03-08)
-
An Information-Theoretic Evaluation of Generative Models in Learning Multi-modal Distributions (2024-02-13)
-
MIGC: Multi-Instance Generation Controller for Text-to-Image Synthesis (2024-02-08)
Note: COCO-MIG benchmark introduced
-
CAS: A Probability-Based Approach for Universal Condition Alignment Score (2024-01-16)
Note: Condition alignment of text-to-image, {instruction, image}-to-image, edge-/scribble-to-image, and text-to-audio
-
EmoGen: Emotional Image Content Generation with Text-to-Image Diffusion Models (2024-01-09)
Note: emotion accuracy, semantic clarity and semantic diversity are not core contributions of this paper
-
VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluation (2023-12-22)
-
PIA: Your Personalized Image Animator via Plug-and-Play Modules in Text-to-Image Models (2023-12-21)
Note: AnimateBench, benchmark for comparisons in the field of personalized image animation
-
Stellar: Systematic Evaluation of Human-Centric Personalized Text-to-Image Methods (2023-12-11)
-
The Challenges of Image Generation Models in Generating Multi-Component Images (2023-11-22)
-
SelfEval: Leveraging the discriminative nature of generative models for evaluation (2023-11-17)
-
GPT-4V(ision) as a Generalist Evaluator for Vision-Language Tasks (2023-11-02)
-
Davidsonian Scene Graph: Improving Reliability in Fine-grained Evaluation for Text-to-Image Generation (2023-10-27, ICLR 2024)
-
DEsignBench: Exploring and Benchmarking DALL-E 3 for Imagining Visual Design (2023-10-23)
-
GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment (2023-10-17)
-
Hypernymy Understanding Evaluation of Text-to-Image Models via WordNet Hierarchy (2023-10-13)
-
SingleInsert: Inserting New Concepts from a Single Image into Text-to-Image Models for Flexible Editing (2023-10-12)
Note: New Metric: Editing Success Rate
-
ImagenHub: Standardizing the evaluation of conditional image generation models (2023-10-02)
-
Navigating Text-To-Image Customization: From LyCORIS Fine-Tuning to Model Evaluation (2023-09-26, ICLR 2024)
-
Concept Score from Text-to-Image Generation for Abstract Concepts (2023-09-26)
-
OpenLEAF: Open-Domain Interleaved Image-Text Generation and Evaluation (2023-09-23)
GenAI-Arena
Note: Evaluates task of image and text generation
-
Progressive Text-to-Image Diffusion with Soft Latent Direction (2023-09-18)
Note: Benchmark for text-to-image generation tasks
-
AltDiffusion: A Multilingual Text-to-Image Diffusion Model (2023-08-19, AAAI 2024)
Note: Benchmark with focus on multilingual generation aspect
-
LEICA from Likelihood-Based Text-to-Image Evaluation with Patch-Level Perceptual and Semantic Credit Assignment (2023-08-16)
-
Let's ViCE! Mimicking Human Cognitive Behavior in Image Generation Evaluation (2023-07-18)
-
T2I-CompBench: A Comprehensive Benchmark for Open-world Compositional Text-to-image Generation (2023-07-12)
-
TIAM -- A Metric for Evaluating Alignment in Text-to-Image Generation (2023-07-11, WACV 2024)
-
Divide, Evaluate, and Refine: Evaluating and Improving Text-to-Image Alignment with Iterative VQA Feedback (2023-07-10, NeurIPS 2023)
-
Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis (2023-06-15)
-
ConceptBed: Evaluating Concept Learning Abilities of Text-to-Image Diffusion Models (2023-06-07, AAAI 2024)
-
Visual Programming for Text-to-Image Generation and Evaluation (2023-05-24, NeurIPS 2023)
-
LLMScore: Unveiling the Power of Large Language Models in Text-to-Image Synthesis Evaluation (2023-05-18, NeurIPS 2023)
-
X-IQE: eXplainable Image Quality Evaluation for Text-to-Image Generation with Visual Large Language Models (2023-05-18)
-
What You See is What You Read? Improving Text-Image Alignment Evaluation (2023-05-17, NeurIPS 2023)
-
Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation (2023-05-02)
-
Analysis of Appeal for Realistic AI-Generated Photos (2023-04-17)
-
Appeal and quality assessment for AI-generated images (2023-06-22)
-
Diagnostic Benchmark and Iterative Inpainting for Layout-Guided Image Generation (2023-04-13)
-
HRS-Bench: Holistic, Reliable and Scalable Benchmark for Text-to-Image Models (2023-04-11, ICCV 2023)
-
Human Preference Score: Better Aligning Text-to-Image Models with Human Preference (2023-03-25, ICCV 2023)
-
A study of the evaluation metrics for generative images containing combinational creativity (2023-03-23)
Note: Consensual Assessment Technique and Turing Test used in T2I evaluation
-
TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering (2023-03-21, ICCV 2023)
-
Is This Loss Informative? Faster Text-to-Image Customization by Tracking Objective Dynamics (2023-02-09)
Note: an evaluation approach for early stopping criterion in T2I customization
-
Benchmarking Spatial Relationships in Text-to-Image Generation (2022-12-20)
-
MMI and MOR from from Benchmarking Robustness of Multimodal Image-Text Models under Distribution Shift (2022-12-15)
-
TeTIm-Eval: a novel curated evaluation data set for comparing text-to-image models (2022-12-15)
-
Human Evaluation of Text-to-Image Models on a Multi-Task Benchmark (2022-11-22)
-
UPainting: Unified Text-to-Image Diffusion Generation with Cross-modal Guidance (2022-10-28)
Note: UniBench, benchmark contains prompts for simple-scene images and complex-scene images in Chinese and English
-
Re-Imagen: Retrieval-Augmented Text-to-Image Generator (2022-09-29)
Note: EntityDrawBench, benchmark to evaluates image generation for diverse entities
-
Vision-Language Matching for Text-to-Image Synthesis via Generative Adversarial Networks (2022-08-20)
Note: new metric, Vision-Language Matching Score (VLMS)
-
Scaling Autoregressive Models for Content-Rich Text-to-Image Generation (2022-06-22)
-
GR-GAN: Gradual Refinement Text-to-image Generation (2022-05-23)
Note: new metric Cross-Model Distance introduced
-
DrawBench from Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding (2022-05-23)
-
StyleT2I: Toward Compositional and High-Fidelity Text-to-Image Synthesis (2022-03-29, CVPR 2024)
Note: Evaluation metric for compositionality of T2I models
-
Benchmark for Compositional Text-to-Image Synthesis (2021-07-29)
-
TISE: Bag of Metrics for Text-to-Image Synthesis Evaluation (2021-12-02, ECCV 2022)
-
Improving Generation and Evaluation of Visual Stories via Semantic Consistency (2021-05-20)
-
Leveraging Visual Question Answering to Improve Text-to-Image Synthesis (2020-10-28)
-
Image Synthesis from Locally Related Texts (2020-06-08)
Note: VQA accuracy as a new evaluation metric.
-
Semantic Object Accuracy for Generative Text-to-Image Synthesis (2019-10-29)
Note: new evaluation metric, Semantic Object Accuracy (SOA)
-
GPT-ImgEval: A Comprehensive Benchmark for Diagnosing GPT4o in Image Generation (2025-04-03)
-
R2I-Bench: Benchmarking Reasoning-Driven Text-to-Image Generation (2025-05-28)
-
MIEScore: Human-Aligned Evaluation for Multi-Source Image Editing (2026-08-03)
-
MPIE-Bench: Benchmarking Anatomically Plausible Multi-Person Interaction Editing (2026-07-30)
-
RS-RIE-Bench: Benchmarking Reasoning-Guided Remote Sensing Image Editing (2026-07-22)
-
WeEdit: A Dataset, Benchmark and Glyph-Guided Framework for Text-centric Image Editing (2026-03-12)
-
Do Image Editing Models Understand Lighting? (2026-06-25)
-
ChartSync: A Benchmark for Visuo-Logical Cascading Chart Editing (2026-07-11)
-
PhyEditBench: A Real-World Multi-Stage Benchmark for Physics-Aware Image Editing (2026-06-25)
-
An Extensive Benchmark for Single-round and Multi-round Instruction-based Image Editing (2026-06-14)
-
VDE Bench: Evaluating The Capability of Image Editing Models to Modify Visual Documents (2026-01-27)
-
PaintBench: Deterministic Evaluation of Precise Visual Editing (2026-05-29)
-
TextSculptor: Training and Benchmarking Scene Text Editing (2026-05-20)
-
MULTITEXTEDIT: Benchmarking Cross-Lingual Degradation in Text-in-Image Editing (2026-05-04)
-
DLEBench: Evaluating Small-scale Object Editing Ability for Instruction-based Image Editing Model (2026-02-27)
-
StyleText: A Large-Scale Dataset and Benchmark for Stylized Scene Text Inpainting (2026-05-17)
-
Editor's Choice: Evaluating Abstract Intent in Image Editing through Atomic Entity Analysis (2026-05-14)
-
Do-Undo Bench: Reversibility for Action Understanding in Image Generation (2025-12-15)
-
Edit-Compass & EditReward-Compass: A Unified Benchmark for Image Editing and Reward Modeling (2026-05-13)
-
PlanViz: Evaluating Planning-Oriented Image Generation and Editing for Computer-Use Tasks (2026-02-06)
-
Towards In-Context Tone Style Transfer with A Large-Scale Triplet Dataset (2026-04-17)
-
Is This Edit Correct? A Multi-Dimensional Benchmark for Reasoning-Aware Image Editing (2026-04-16)
-
Redefining Quality Criteria and Distance-Aware Score Modeling for Image Editing Assessment (2026-04-14)
-
AIM-Bench: Benchmarking and Improving Affective Image Manipulation via Fine-Grained Hierarchical Control (2026-04-12)
-
SpatialEdit: Benchmarking Fine-Grained Image Spatial Editing (2026-04-06)
-
MIRAGE: Benchmarking and Aligning Multi-Instance Image Editing (2026-04-06)
-
FED-Bench: A Cross-Granular Benchmark for Disentangled Evaluation of Facial Expression Editing (2026-03-31)
-
GEditBench v2: A Human-Aligned Benchmark for General Image Editing (2026-03-30)
-
ImagenWorld: Stress-Testing Image Generation Models with Explainable Human Evaluation on Open-ended Real-World Tasks (2026-03-29)
-
VIBE: How Well Do Models Follow Visual Instructions? (2026-02-02)
-
Evaluating Image Editing with LLMs: A Comprehensive Benchmark and Intermediate-Layer Probing Approach (2026-03-20)
-
Edit Spillover as a Probe: Do Image Editing Models Implicitly Understand World Relations? (2026-03-18)
-
Scalable Evaluation of the Realism of Synthetic Environmental Augmentations in Images (2026-03-04)
-
InEdit-Bench: Benchmarking Intermediate Logical Pathways for Intelligent Image Editing Models (2026-03-04)
-
Human-Aligned MLLM Judges for Fine-Grained Image Editing Evaluation: A Benchmark, Framework, and Analysis (2026-02-13)
-
WorldEdit: Towards Open-World Image Editing with a Knowledge-Informed Benchmark (2026-02-06)
-
LocateEdit-Bench: A Benchmark for Instruction-Based Editing Localization (2026-02-05)
-
HYPE-EDIT-1: Benchmark for Measuring Reliability in Frontier Image Editing Models (2026-01-25)
-
Fine-Grained Human Pose Editing Assessment via Layer-Selective MLLMs (2026-01-15)
-
PICABench: How Far Are We from Physically Realistic Image Editing? (2025-10-20)
-
RealDrag: The First Dragging Benchmark with Real Target Image (2025-12-13)
-
WiseEdit: Benchmarking Cognition- and Creativity-Informed Image Editing (2025-11-29)
-
UniREditBench: A Unified Reasoning-based Image Editing Benchmark (2025-11-03)
-
MedGEN-Bench: Contextually Entangled Benchmark for Open-ended Multimodal Medical Generation (2025-11-17)
-
MedEBench: Diagnosing Reliability in Text-Guided Medical Image Editing (2025-06-02)
-
SpotEdit: Evaluating Visually-Guided Image Editing Methods (2025-08-25)
-
ADIEE: Automatic Dataset Creation and Scorer for Instruction-Guided Image Editing Evaluation (2025-07-09)
-
GIE-Bench: Towards Grounded Evaluation for Text-Guided Image Editing (2025-05-16)
-
ComplexBench-Edit: Benchmarking Complex Instruction-Driven Image Editing via Compositional Dependencies (2025-06-15)
-
ByteMorph: Benchmarking Instruction-Guided Image Editing with Non-Rigid Motions (2025-06-03)
-
RefEdit: A Benchmark and Method for Improving Instruction-based Image Editing Model on Referring Expressions (2025-06-03)
-
A Systematic Benchmark of Intraoperative Ultrasound-to-MR Synthesis for Brain Tumour Surgery (2026-05-30)
-
PhysEditBench: A Protocol-Conditioned Benchmark for Dense Physical-Map Prediction with Image Editors (2026-05-13)
-
WildRelight: A Real-World Benchmark and Physics-Guided Adaptation for Single-Image Relighting (2026-05-12)
-
StableI2I: Spotting Unintended Changes in Image-to-Image Transition (2026-05-06)
-
ViGoR-Bench: How Far Are Visual Generative Models From Zero-Shot Visual Reasoners? (2026-03-26)
-
GEBench: Benchmarking Image Generation Models as GUI Environments (2026-02-09)
-
Vectra: A New Metric, Dataset, and Model for Visual Quality Assessment in E-Commerce In-Image Machine Translation (2026-01-31)
-
Contrast-X: A Multi-Modal Contrast Image Synthesis Benchmark and Universal Modality Flow Matching (2026-01-22)
-
Human Body Restoration with One-Step Diffusion Model and A New Benchmark (2025-02-03)
-
Probing Visual Planning in Image Editing Models (2026-04-23)
-
EditInspector: A Benchmark for Evaluation of Text-Guided Image Edits (2025-06-11, ACL 2025)
-
Reference-Free Image Quality Assessment for Virtual Try-On via Human Feedback (2026-03-13, ECCV 2026)
Note: Introduces the reference-free VTON-IQA framework and the human-annotated VTON-QBench.
-
VTEdit-Bench: A Comprehensive Benchmark for Multi-Reference Image Editing Models in Virtual Try-On (2026-03-12, ECCV 2026)
Note: Introduces VTEdit-QA to assess model consistency, cloth consistency, and overall image quality.
-
Virtual Try-On for Cultural Clothing: A Benchmarking Study (2026-03-07)
Note: Introduces BD-VITON for benchmarking virtual try-on on culturally diverse Bangladeshi garments.
-
OpenVTON-Bench: A Large-Scale High-Resolution Benchmark for Controllable Virtual Try-On Evaluation (2026-01-30)
Note: Evaluates background consistency, identity fidelity, texture fidelity, shape plausibility, and overall realism.
-
VTONQA: A Multi-Dimensional Quality Assessment Dataset for Virtual Try-on (2026-01-06)
Note: Provides human opinion scores for clothing fit, body compatibility, and overall quality across images from 11 VTON models.
-
Omni IIE Bench: Benchmarking the Practical Capabilities of Image Editing Models (2026-03)
-
I2I-Bench: A Comprehensive Benchmark Suite for Image-to-Image Editing Models (2025-12-04)
-
Inter-Edit: First Benchmark for Interactive Instruction-Based Image Editing (CVPR 2026)
-
LMM4Edit: Benchmarking and Evaluating Multimodal Image Editing with LMMs (2025-07)
Note: Introduces LMM4Edit.
-
CompBench: Benchmarking Complex Instruction-Guided Image Editing (2025-05)
-
Does FLUX Already Know How to Perform Physically Plausible Image Composition? (2025-09)
-
Taming I2V Models for Image HOI Editing: A Cognitive Benchmark and Agentic Self-Correcting Framework (2026-06)
-
MICE-Bench: A Challenging and Comprehensive Benchmark for Multi-Reference Image Creation and Editing (ICML 2026)
-
Learning Action and Reasoning-Centric Image Editing from Videos and Simulations (2024-07-03)
Note: AURORA-Bench introduced
-
GIM: A Million-scale Benchmark for Generative Image Manipulation Detection and Localization (2024-06-24)
-
MultiEdits: Simultaneous Multi-Aspect Editing with Text-to-Image Diffusion Models (2024-06-03)
Note: PIE-Bench++, evaluating image-editing tasks involving multiple objects and attributes
-
DiffUHaul: A Training-Free Method for Object Dragging in Images (2024-06-03)
Note: foreground similarity, object traces and realism metric introduced
-
HQ-Edit: A High-Quality Dataset for Instruction-based Image Editing (2024-04-15)
-
FlexEdit: Flexible and Controllable Diffusion-based Object-centric Image Editing (2024-03-27)
Note: novel automatic mask-based evaluation metric tailored to various object-centric editing scenarios
-
TransformationOriented Paired Benchmark from InstructBrush: Learning Attention-based Instruction Optimization for Image Editing (2024-03-27)
-
ImageNet Concept Editing Benchmark from Editing Massive Concepts in Text-to-Image Diffusion Models (2024-03-20)
-
Editing Massive Concepts in Text-to-Image Diffusion Models (2024-03-20)
Note: ImageNet Concept Editing Benchmark (ICEB), for evaluating massive concept editing for T2I models
-
Make Me Happier: Evoking Emotions Through Image Diffusion Models (2024-03-13)
Note: EMR, ESR, ENRD, ESS metric introduced
-
Diffusion Model-Based Image Editing: A Survey (2024-02-27)
Note: EditEval, benchmark for text-guided image editing and LLM Score
-
Towards Efficient Diffusion-Based Image Editing with Instant Attention Masks (2024-01-15, AAAI 2024)
Note: Editing-Mask, new benchmark to examine the mask accuracy and local editing ability
-
RotationDrag: Point-based Image Editing with Rotated Diffusion Features (2024-01-12)
Note: RotationBench introduced
-
LEDITS++: Limitless Image Editing using Text-to-Image Models (2023-11-28)
Note: TEdBench++, revised benchmark of TEdBench
-
Emu Edit: Precise Image Editing via Recognition and Generation Tasks (2023-11-16)
-
EditVal: Benchmarking Diffusion Based Text-Guided Image Editing Methods (2023-10-03)
-
PIE-Bench from Direct Inversion: Boosting Diffusion-based Editing with 3 Lines of Code (2023-10-02)
-
Iterative Multi-granular Image Editing using Diffusion Models (2023-09-01)
-
DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editing (2023-06-26)
Note: drawbench benchmark introduced
-
DreamEdit: Subject-driven Image Editing (2023-06-22)
Note: DreamEditBench benchmark introduced
-
MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editing (2023-06-16)
Note: dataset only
-
Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpainting (2022-12-13, CVPR 2023)
-
Imagic: Text-Based Real Image Editing with Diffusion Models (2022-10-17)
Note: TEdBench, image editing benchmark
-
Knowledge-Driven Generative Adversarial Network for Text-to-Image Synthesis (2021-09-29)
Note: New evaluation system, Pseudo Turing Test (PTT)
-
ManiGAN: Text-Guided Image Manipulation (2019-12-12)
Note: manipulative precision metric introduced
-
Text Guided Person Image Synthesis (2019-04-10)
Note: VQA perceptual score introduced
-
CultureVidBench: Benchmarking Cultural Understanding in Text-to-Video Generation (2026-08-03)
-
FilmBench: A Film-Grade Benchmark for Cinematic Video Generation (2026-07-27)
-
When Cultures Move: Measuring and Improving Multicultural Text-to-Video Generation (2026-05-16)
-
Multi-Dimensional Quality Assessment for AI-Generated Human-Centric Videos: Dataset and Model (2026-07-18)
-
MultiRef-Compass: Towards Comprehensive Evaluation of Multi-Reference-to-Audio-Video Generation (2026-07-15)
-
VGIF-Score: Interpretable and Diagnostic Evaluation of Spatio-Temporal Instruction Following in Video Generation (2026-07-15)
-
GeoT2V-Bench: Benchmarking 3D Consistency in Text-to-Video Models via 3D Reconstruction (2026-06-23)
-
Measuring 3D Spatial Geometric Consistency in Dynamic Video Generation (2026-03-19)
-
MSAVBench: Towards Comprehensive and Reliable Evaluation of Multi-Shot Audio-Video Generation (2026-05-19)
-
CultureScore: Evaluating Cultural Faithfulness in Video Generation Models (2026-06-05)
-
BRITE: A Benchmark for Reliable and Interpretable T2V Evaluation on Implausible Scenarios (2026-04-24)
-
Evaluating Design Video Generation: Metrics for Compositional Fidelity (2026-05-15)
-
PEDRA: Evaluating the Realism of Pedestrian Dynamics in Video Generation (2025-10-23)
-
T2AV-Compass: Towards Unified Evaluation for Text-to-Audio-Video Generation (2025-12-24)
-
OmniHuman: A Large-scale Dataset and Benchmark for Human-Centric Video Generation (2026-04-20)
-
LoCoT2V-Bench: Benchmarking Long-Form and Complex Text-to-Video Generation (2025-10-30)
-
LongAV-Compass: Towards Unified Evaluation of Minute-Scale Audio-Visual Generation Across T2AV, I2AV, and V2AV (2026-05-25)
-
AVBench: Human-Aligned and Automated Evaluation Benchmark for Audio-Video Generative Models (2026-05-23)
-
EduVQA: Towards Concept-Aware Assessment of Educational AI-Generated Videos (2026-03-03)
-
WorldJen: An End-to-End Multi-Dimensional Benchmark for Generative Video Models (2026-05-05)
-
HumanScore: Benchmarking Human Motions in Generated Videos (2026-04-22)
-
AVGen-Bench: A Task-Driven Benchmark for Multi-Granular Evaluation of Text-to-Audio-Video Generation (2026-04-09)
-
MSG Score: Automated Video Verification for Reliable Multi-Scene Generation (2024-11-28)
-
VABench: A Comprehensive Benchmark for Audio-Video Generation (2025-12-10)
-
MSVBench: Towards Human-Level Evaluation of Multi-Shot Video Generation (2026-02-27)
-
Q-Save: Towards Scoring and Attribution for Generated Video Evaluation (2025-11-24)
-
PhyEduVideo: A Benchmark for Evaluating Text-to-Video Models for Physics Education (2026-01-02)
-
GRADEO: Towards Human-Like Evaluation for Text-to-Video Generation via Multi-Step Reasoning (2025-03-04)
-
Are AI-Generated Driving Videos Ready for Autonomous Driving? A Diagnostic Evaluation Framework (2025-12-06)
-
V-ReasonBench: Toward Unified Reasoning Benchmark Suite for Video Generation Models (2025-11-20)
-
Temporal Realism Evaluation of Generated Videos Using Compressed-Domain Motion Vectors (2025-11-17)
-
SeqBench: Benchmarking Sequential Narrative Generation in Text-to-Video Models (2025-10-14)
-
DynamicEval: Rethinking Evaluation for Dynamic Text-to-Video Synthesis (2025-10-08)
-
Stable Cinemetrics : Structured Taxonomy and Evaluation for Professional Video Generation (2025-09-30)
-
GeneVA: A Dataset of Human Annotations for Generative Text to Video Artifacts (2025-09-10)
-
T2VEval: Benchmark Dataset and Objective Evaluation Method for T2V-generated Videos (2025-01-15)
-
T2VWorldBench: A Benchmark for Evaluating World Knowledge in Text-to-Video Generation (2025-07-24)
-
FIFA: Unified Faithfulness Evaluation Framework for Text-to-Video and Video-to-Text Generation (2025-07-09)
-
Beyond Time Shifts: Adapting Omni-LLM as a Reference-Free Evaluator for Generative Audio-Visual Models (2026-07-10)
-
Artifact-Bench: Evaluating MLLMs on Detecting and Assessing the Artifacts of AI-Generated Videos (2026-05-18)
-
EntityBench: Towards Entity-Consistent Long-Range Multi-Shot Video Generation (2026-05-14)
-
Omni-Judge: Can Omni-LLMs Serve as Human-Aligned Judges for Text-Conditioned Audio-Video Generation? (2026-02-02)
-
VideoASMR-Bench: Can AI-Generated ASMR Videos Fool VLMs and Humans? (2025-12-15)
-
Spotlight: Identifying and Localizing Video Generation Errors Using VLMs (2025-11-22)
-
NarrLV: Towards a Comprehensive Narrative-Centric Evaluation for Long Video Generation (2025-07-15, ICLR 2026)
-
Comparison Drives Preference: Reference-Aware Modeling for AI-Generated Video Quality Assessment (2026-04-18)
Note: Introduces Reference-aware Video Quality Assessment (RefVQA).
-
Towards Holistic Visual Quality Assessment of AI-Generated Videos: A LLM-Based Multi-Dimensional Evaluation Model (2025-06-05, CVPR Workshop 2025)
-
Physics Question Scene Graph: Fine-grained Evaluation of Physical Plausibility in Text-to-Video Generation (2026-06-24)
-
Trimming the Long-Tail of Visual World Modeling Evaluation (2026-06-23)
-
Quantitative Video World Model Evaluation for Geometric-Consistency (2026-05-14)
-
PhyGround: Benchmarking Physical Reasoning in Generative World Models (2026-05-11)
-
Physion-Eval: Evaluating Physical Realism in Generated Video via Human Reasoning (2026-03-20)
-
3DSPA: A 3D Semantic Point Autoencoder for Evaluating Video Realism (2026-02-23)
-
Benchmarking Scientific Understanding and Reasoning for Video Generation using VideoScience-Bench (2025-12-02)
-
PhyDetEx: Detecting and Explaining the Physical Plausibility of T2V Models (2025-12-01)
-
World Consistency Score: A Unified Metric for Video Generation Quality (2025-07-31)
-
"PhyWorldBench": A Comprehensive Evaluation of Physical Realism in Text-to-Video Models (2025-07-17)
-
VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness (2025-03-27)
-
Artifact-Aware Evaluation for High-Quality Video Generation (2026-01-28)
-
How Far are AI-generated Videos from Simulating the 3D Visual World: A Learned 3D Evaluation Approach (2024-06-27)
-
EvalVerse: Pipeline-Aware and Expert-Calibrated Benchmarking for Professional Cinematic Video Generation (2026-05-22)
Note: Introduces a pipeline-aware, expert-calibrated framework for evaluating cinematic quality, acting, aesthetics, multi-shot sequencing, and audio-visual integration.
-
MuSS: A Large-Scale Dataset and Cinematic Narrative Benchmark for Multi-Shot Subject-to-Video Generation (2026-04-26)
Note: Introduces the Cinematic Narrative Benchmark and Anti-Copy-Paste Variance (ACP-Var) metric for evaluating continuous storytelling and 3D structural consistency.
-
VGA-Bench: A Unified Benchmark and Multi-Model Framework for Video Aesthetics and Generation Quality Evaluation (2026-04-11, CVPR 2026)
Note: Jointly evaluates aesthetic quality, aesthetic tagging, and generation quality with VAQA-Net, VTag-Net, and VGQA-Net.
-
Ref4D-VideoBench: Four-Dimensional Reference-Based Evaluation of Text-to-Video Generative Models (CVPR 2026)
-
SVBench: Evaluation of Video Generation Models on Social Reasoning (2025-12-25)
-
FingER: Content Aware Fine-Grained Evaluation with Reasoning for AI-Generated Videos (2025-04)
-
OSCBench: Benchmarking Object State Change in Text-to-Video Generation (2026-03-12)
-
MTAVG-Bench: A Diagnostic Benchmark for Multi-Talker Dialogue-Centric Audio-Video Generation (2026-02)
-
VQ-Insight: Teaching VLMs for AI-Generated Video Quality Understanding via Progressive Visual Reinforcement Learning (2025-06-23)
-
ViStoryBench: Comprehensive Benchmark Suite for Story Visualization (CVPR 2026)
-
LikePhys: Evaluating Intuitive Physics Understanding in Video Diffusion Models via Likelihood Preference (2025-10-13, ICLR 2026)
Note: likelihood-preference benchmark for evaluating intuitive physics understanding in video diffusion models.
-
VideoScore2: Think before You Score in Generative Video Evaluation (2025-09-26)
-
Stable Cinemetrics: Structured Taxonomy and Evaluation for Professional Video Generation (2025-09)
-
Are Synthetic Videos Useful? A Benchmark for Retrieval-Centric Evaluation of Synthetic Videos (2025-07-03)
-
AIGVE-MACS: Unified Multi-Aspect Commenting and Scoring Model for AI-Generated Video Evaluation (2025-07-02)
-
BrokenVideos: A Benchmark Dataset for Fine-Grained Artifact Localization in AI-Generated Videos (2025-06-25)
-
OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generation (2025-06-03)
Note: The first open-sourced infrastructure (OpenS2V-Eval & OpenS2V-5M) for Subject-to-Video generation
-
CineTechBench: A Benchmark for Cinematographic Technique Understanding and Generation (2025-05)
-
LOVE: Benchmarking and Evaluating Text-to-Video Generation and Video-to-Text Interpretation (2025-05-17)
-
On the Consistency of Video Large Language Models in Temporal Comprehension (2025-05-17)
-
AIGVE-Tool: AI-Generated Video Evaluation Toolkit with Multifaceted Benchmark (2025-04-18)
-
VideoGen-Eval: Agent-based System for Video Generation Evaluation (2025-03-30)
-
Video-Bench: Human Preference Aligned Video Generation Benchmark (2025-04-07)
-
Morpheus: Benchmarking Physical Reasoning of Video Generative Models with Real Physical Experiments (2025-04-03)
-
Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual Editing (2025-04-03)
-
VinaBench: Benchmark for Faithful and Consistent Visual Narratives (2025-03-26)
-
ETVA: Evaluation of Text-to-Video Alignment via Fine-grained Question Generation and Answering (2025-03-21)
-
UVE: Are MLLMs Unified Evaluators for AI-Generated Videos? (2025-03-13)
-
Is Your World Simulator a Good Story Presenter? A Consecutive Events-Based Benchmark for Future Long Video Generation (2024-12-17)
Note: focus on storytelling.
-
Neuro-Symbolic Evaluation of Text-to-Video Models using Formal Verification (2024-12-03)
Note: focus on temporally text-video alignment (event order, accuracy)
-
AIGV-Assessor: Benchmarking and Evaluating the Perceptual Quality of Text-to-Video Generation with LMM (2024-11-26)
Note: fuild motion, light change, motion speed, event order.
-
Human-Activity AGV Quality Assessment: A Benchmark Dataset and an Objective Evaluation Metric (2024-11-25)
Note: Introduces the AI-Generated Human Activity Video Quality metric (GHVQ).
-
What You See Is What Matters: A Novel Visual and Physics-Based Metric for Evaluating Video Generation Quality (2024-11-24)
Note: texture evaluation scheme introduced
-
The Dawn of Video Generation: Preliminary Explorations with SORA-like Models (2024-10-10)
-
Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation (2024-10-07)
Note: Comprehensive physical (optical, mechanic, thermal, material) benchmark introduced
-
Benchmarking AIGC Video Quality Assessment: A Dataset and Unified Model (2024-07-31)
-
T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generation (2024-07-19)
-
T2VSafetyBench: Evaluating the Safety of Text-to-Video Generative Models (2024-07-08)
Note: T2VSafetyBench introduced
-
Evaluation of Text-to-Video Generation Models: A Dynamics Perspective (2024-07-01)
-
T2VBench: Benchmarking Temporal Dynamics for Text-to-Video Generation (2024-06)
-
Evaluating and Improving Compositional Text-to-Visual Generation (2024-06)
-
TlTScore: Towards Long-Tail Effects in Text-to-Visual Evaluation with Generative Foundation Models (2024-06)
-
ChronoMagic-Bench: A Benchmark for Metamorphic Evaluation of Text-to-Time-lapse Video Generation (2024-06-26)
Note: Comprehensive time-lapse (biological, human created, meteorological, physical) benchmark introduced
-
VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generation (2024-06-21)
-
TC-Bench: Benchmarking Temporal Compositionality in Text-to-Video and Image-to-Video Generation (2024-06-12)
Note: TC-Bench, TCR and TC-Score introduced
-
VideoPhy: Evaluating Physical Commonsense for Video Generation (2024-06-05)
-
Illumination Histogram Consistency Metric for Quantitative Assessment of Video Sequences (2024-05-15)
-
The Lost Melody: Empirical Observations on Text-to-Video Generation From A Storytelling Perspective (2024-05-13)
Note: New evaluation framework T2Vid2T, Evaluation for storytelling aspects of videos
-
Exposing AI-generated Videos: A Benchmark Dataset and a Local-and-Global Temporal Defect Based Detection Method (2024-05-07)
-
Sora Detector: A Unified Hallucination Detection for Large Text-to-Video Models (2024-05-07)
Note: hallucination detection
-
Exploring AIGC Video Quality: A Focus on Visual Harmony, Video-Text Consistency and Domain Distribution Gap (2024-04-21)
-
Subjective-Aligned Dataset and Metric for Text-to-Video Quality Assessment (2024-03-18)
-
A dataset of text prompts, videos and video quality metrics from generative text-to-video AI models (2024-02-22)
-
Sora Generates Videos with Stunning Geometrical Consistency (2024-02-27)
-
STREAM: Spatio-TempoRal Evaluation and Analysis Metric for Video Generative Models (2024-01-30)
-
Towards A Better Metric for Text-to-Video Generation (2024-01-15)
-
PEEKABOO: Interactive Video Generation via Masked-Diffusion (2023-12-12)
Note: Benchmark for interactive video generation
-
VBench: Comprehensive Benchmark Suite for Video Generative Models (2023-11-29)
-
SmoothVideo: Smooth Video Synthesis with Noise Constraints on Diffusion Models for One-shot Video Tuning (2023-11-29)
-
FETV: A Benchmark for Fine-Grained Evaluation of Open-Domain Text-to-Video Generation (2023-11-03)
-
EvalCrafter: Benchmarking and Evaluating Large Video Generation Models (2023-10-17)
-
Measuring the Quality of Text-to-Video Model Outputs: Metrics and Dataset (2023-09-14)
-
StoryBench: A Multifaceted Benchmark for Continuous Story Visualization (2023-08-22, NeurIPS 2023)
-
Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectives (2023-03-07, ICCV 2023)
Note: Aesthetic View & Technical View
-
CelebV-Text: A Large-Scale Facial Text-Video Dataset (2023-03-26, CVPR 2023)
Note: Benchmark on Facial Text-to-Video Generation
-
Make It Move: Controllable Image-to-Video Generation with Text Descriptions (2021-12-06, CVPR 2022)
-
GAUGE: A Measurement-Grounded Benchmark for Physical Fidelity in Simulation Engines and Video World Models (2026-08-06)
-
MBench: A Comprehensive Benchmark on Memory Capability for Video World Models (2026-05-30)
-
WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation (2026-05-25)
-
VideoVerse: Does Your T2V Generator Have World Model Capability to Synthesize Videos? (2025-10-09)
-
WorldReasonBench: Human-Aligned Stress Testing of Video Generators as Future World-State Predictors (2026-05-11)
-
MME-CoF-Pro: Evaluating Reasoning Coherence in Video Generative Models with Text and Visual Hints (2026-03-20)
-
RISE-Video: Can Video Generators Decode Implicit World Rules? (2026-02-05)
-
AnimeAgent: Is the Multi-Agent via Image-to-Video models a Good Disney Storytelling Artist? (2026-02-24)
-
Thinking in Video: Can Video Generators Really Reason About the Real World? (2026-07-20)
-
KeyFrame-Compass: Towards Comprehensive Evaluation of Keyframe-Conditioned Video Generation (2026-07-15)
-
Beyond the Last Frame: Process-aware Evaluation for Generative Video Reasoning (2025-12-31)
-
Can World Simulators Reason? Gen-ViRe: A Generative Visual Reasoning Benchmark (2025-11-17)
-
Drive&Gen: Co-Evaluating End-to-End Driving and Video Generation Models (2025-10-07)
-
SimWorld: A Unified Benchmark for Simulator-Conditioned Scene Generation via World Model (2025-03-18)
Note: Benchmarks simulator-conditioned world-model image generation for autonomous-driving scenes.
-
MechVerse: Evaluating Physical Motion Consistency in Video Generation Models (2026-05-14)
-
WorldMark: A Unified Benchmark Suite for Interactive Video World Models (2026-04-23)
-
DrivingGen: A Comprehensive Benchmark for Generative Video World Models in Autonomous Driving (2026-01-04)
-
WorldLens: Full-Spectrum Evaluations of Driving World Models in Real World (2025-12-11)
-
Evaluating Newtonian Mechanics in Video Generative Models with Real Physical Systems (2025-04-03, ICML 2026)
-
WorldExam: Benchmarking World Models from Apparent Appearance to Inherent Reactivity (2026-08-03)
Note: Evaluates visual quality, control adherence, spatial consistency, and inherent world reactivity.
-
Learning Explicit Physical Parameter Control and Benchmarking for Video Generation (2026-07-21)
Note: Introduces PhyParam-Bench for temporal dynamics, spatial stability, and semantic-physical alignment in I2V generation.
-
AnimationBench: Are Video Models Good at Character-Centric Animation? (2026-04-16)
Note: Evaluates animation principles, IP preservation, semantic consistency, motion rationality, and camera-motion consistency.
-
Out of Sight, Out of Mind? Evaluating State Evolution in Video World Models (2026-03-13)
Note: Introduces STEVO-Bench to evaluate whether state evolution continues correctly under interrupted observation.
-
TiViBench: Benchmarking Think-in-Video Reasoning for Video Generative Models (2025-11-17)
Note: Benchmarks structural, spatial, logical, and action-planning reasoning in I2V models.
-
UI2V-Bench: An Understanding-based Image-to-video Generation Benchmark (2025-09-29)
Note: Evaluates spatial understanding, attribute binding, category understanding, and reasoning in I2V models.
-
HyperMotionX: The Dataset and Benchmark with DiT-Based Pose-Guided Human Image Animation of Complex Motions (2025-05-29)
Note: Introduces HyperMotionX Bench for pose-guided human image animation under complex motions.
-
VBench++: Comprehensive and Versatile Benchmark Suite for Video Generative Models (2024-11-20)
-
I2V-Bench from ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generation (2024-02-06)
-
AIGCBench: Comprehensive Evaluation of Image-to-Video Content Generated by AI (2024-01-03)
-
A Benchmark for Controllable Text-Image-to-Video Generation (2023-06-12)
-
Temporal Shift GAN for Large Scale Video Generation (2020-04-04)
Note: Symmetric-Similarity-Score introduced
-
Video Imagination from a Single Image with Transformation Generation (2017-06-13)
Note: RIQA metric introduced
-
Temporally-Aligned Evaluation for Audio-Driven Talking Head Generation (2026-05-31)
-
EvalTalker: Learning to Evaluate Real-Portrait-Driven Multi-Subject Talking Humans (2025-12-01)
-
THEval. Evaluation Framework for Talking Head Video Generation (2025-11-06, CVPR 2026 Findings)
-
Assessing Identity Leakage in Talking Face Generation: Metrics and Evaluation Framework (2025-11-05, ICASSP 2026)
-
Preview WB-DH: Towards Whole Body Digital Human Bench for the Generation of Whole-body Talking Avatar Videos (2025-08-12)
-
Who is a Better Talker: Subjective and Objective Quality Assessment for AI-Generated Talking Heads (2025-07-31)
-
JWB-DH-V1: Benchmark for Joint Whole-Body Talking Avatar and Speech Generation Version 1 (2025-07-28)
-
OpFlowTalker: Realistic and Natural Talking Face Generation via Optical Flow Guidance (2024-05-23)
Note: VTCS to measures lip-readability in synthesized videos
-
Audio-Visual Speech Representation Expert for Enhanced Talking Face Video Generation and Evaluation (2024-05-07)
-
VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Time (2024-04-16)
Note: Contrastive Audio and Pose Pretraining (CAPP) score introduced
-
THQA: A Perceptual Quality Assessment Database for Talking Heads (2024-04-13)
-
A Comparative Study of Perceptual Quality Metrics for Audio-driven Talking Head Videos (2024-03-11)
-
Seeing What You Said: Talking Face Generation Guided by a Lip Reading Expert (2023-03-29, CVPR 2023)
Note: Measuring intelligibility of the generated videos
-
Sparse in Space and Time: Audio-visual Synchronisation with Trainable Selectors (2022-10-13)
-
Responsive Listening Head Generation: A Benchmark Dataset and Baseline (2021-12-27, ECCV 2022)
-
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild (2020-08-23)
Note: new metric LSE-D and LSE-C introduced
-
What comprises a good talking-head video generation?: A Survey and Benchmark (2020-05-07)
-
WorldModelBench: Judging Video Generation Models As World Models (2025-02)
-
WorldScore: A Unified Evaluation Benchmark for World Generation (2025-04-01)
-
Decouple and Track: Benchmarking and Improving Video Diffusion Transformers for Motion Transfer (2025-03-21)
Note: Introduces MTBench and a hybrid motion-fidelity metric.
-
A new dataset and comparison for multi-camera frame synthesis (2025-08-12)
-
OmniEdit-Bench: A Comprehensive Benchmark for Instruction-based Video Editing (2026-08-05)
-
MEDit-Bench: A Dataset for Evaluating Message-Driven Narrative Video Editing (2026-07-28)
-
AVE-Compass: Towards Holistic Evaluation for Audio-Video Editing Abilities (2026-07-17)
-
Goku: A Million-Scale Universal Dataset and Benchmark for Instruction-Based Video Editing (2026-06-29)
-
CoVEBench: Can Video Editing Models Handle Complex Instructions? (2026-06-07)
-
V2V-Bench: A Comprehensive Benchmark for Video-to-Video Generation Evaluation (2026-06-04, ICML 2026 Workshop)
-
PROVE: A Perceptual RemOVal cohErence Benchmark for Visual Media (2026-05-14, ACM MM 2026)
-
VEFX-Bench: A Holistic Benchmark for Generic Video Editing and Visual Effects (2026-04-17)
-
UniEditBench: A Unified and Cost-Effective Benchmark for Image and Video Editing via Distilled MLLMs (2026-04-17)
-
Physics-Aware Video Instance Removal Benchmark (2026-04-07)
-
Hi-Light: A Path to high-fidelity, high-resolution video relighting with a Novel Evaluation Paradigm (2026-01-30)
-
Beyond Rigid: Benchmarking Non-Rigid Video Editing (2026-01-26)
-
DirectSwap: Mask-Free Cross-Identity Training and Benchmarking for Expression-Consistent Video Head Swapping (2025-12-10)
-
EgoEdit: Dataset, Real-Time Streaming Model, and Benchmark for Egocentric Video Editing (2025-12-05)
-
IVEBench: Modern Benchmark Suite for Instruction-Guided Video Editing Assessment (2025-10-13, ICLR 2026)
-
FiVE: A Fine-grained Video Editing Benchmark for Evaluating Emerging Diffusion and Rectified Flow Models (2025-03-17)
-
EditBoard: Towards a Comprehensive Evaluation Benchmark for Text-Based Video Editing Models (2024-09)
-
Multi-modal Segment Assemblage Network for Ad Video Editing with Importance-Coherence Reward (2022-09-25, ACCV 2022)
-
Self-Consistency for LLM-Based Motion Trajectory Generation and Verification (2026-03-31, CVPR 2026)
Note: Uses transformation-aware self-consistency to generate and verify motion-graphics trajectories.
-
Bilingual Text-to-Motion Generation: A New Benchmark and Baselines (2026-03-26)
Note: Introduces the bilingual BiHumanML3D benchmark and the BiMD baseline.
-
T2MBench: A Benchmark for Out-of-Distribution Text-to-Motion Generation (2026-02-14)
Note: Evaluates semantic alignment, generalizability, physical quality, and fine-grained accuracy under OOD prompts.
-
IRG-MotionLLM: Interleaving Motion Generation, Assessment and Refinement for Text-to-Motion Generation (2025-12-11)
Note: Interleaves motion generation, assessment, and refinement through iterative text-motion dialogue.
-
ReAlign: Text-to-Motion Generation via Step-Aware Reward-Guided Alignment (2025-11-24, AAAI 2026)
Note: Introduces a step-aware reward model for text-motion semantic consistency and motion realism.
-
The Quest for Generalizable Motion Generation: Data, Model, and Evaluation (2025-10-30, ICLR 2026)
Note: Introduces MBench for motion quality, prompt fidelity, and generalization ability.
-
FineMotion: A Dataset and Benchmark with both Spatial and Temporal Annotation for Fine-grained Motion Generation and Editing (2025-07-26, ICCV 2025)
Note: Provides fine-grained spatial and temporal annotations for motion generation and editing.
-
ReAlign: Bilingual Text-to-Motion Generation via Step-Aware Reward-Guided Alignment (2025-05-08)
Note: Combines the BiHumanML3D benchmark with step-aware reward-guided alignment.
-
VMBench: A Benchmark for Perception-Aligned Video Motion Generation (2024-03-13)
-
MoDiPO: text-to-motion alignment via AI-feedback-driven Direct Preference Optimization (2024-05-06)
-
What is the Best Automated Metric for Text to Motion Generation? (2023-09-19)
-
Text-to-Motion Retrieval: Towards Joint Understanding of Human Motion Data and Natural Language (2023-05-25)
Note: Evaluation protocol for assessing the quality of the retrieved motions
-
Establishing a Unified Evaluation Framework for Human Motion Generation: A Comparative Analysis of Metrics (2024-05-13)
-
Evaluation of text-to-gesture generation model using convolutional neural network (2021-10-11)
-
Innocent Panels, Hateful Stories: Evaluating and Detecting Hateful Intent in Multi-Turn Visual Story Generation (2026-08-05)
-
Child-Oriented AIGC Video Risk Reviewing: A Benchmark and Knowledge-Supported Iterative Reasoning Framework (2026-07-21)
-
NAMESAKES: Probing Identity Memorization in Text-to-Image Models (2026-06-18)
-
Inference-Time Concept Suppression and Video-Centric Evaluation for Text-to-Video Models (2026-07-15)
-
Beyond wheelchairs and blindfolds: Investigating disability stereotypes in T2I models with INCLUDE-BENCH (2026-07-09)
-
AI's Blind Spots: Geographic Knowledge and Diversity Deficit in Generated Urban Scenario (2025-06-20)
-
VPA-Guard: Defending and Benchmarking Image-to-Video Generation Against Visual Prompt Attacks (2026-06-24)
-
Happy Young Women, Grumpy Old Men? Emotion-Driven Demographic Biases in Synthetic Face Generation (2026-01-18)
-
When Safe Concepts Become Unsafe: Multi-Concept Compositional Vulnerabilities in Text-to-Image Models (2026-04-17)
-
BAFIS: Dataset + Framework to assess occupational Bias and Human Preference in modern Text-to-image Models (2026-06-18)
-
Generating Fearful Images: Investigating Potential Emotional Biases in Image-Generation Models (2024-11-08)
-
When Cars Have Stereotypes: Auditing Demographic Bias in Objects from Text-to-Image Models (2025-08-05)
-
CheXGenBench: A Unified Benchmark For Fidelity, Privacy and Utility of Synthetic Chest Radiographs (2025-05-15)
-
Aligned but Stereotypical? How System Prompts Shape Demographic Bias in LLM-Based Text-to-Image Models (2025-12-04)
-
Gender Artifacts from Art History to Text-to-Image Generation (2026-06-04)
-
Toward Trustworthy Portrait Editing: Evaluation of Demographic Misrepresentation in I2I Models (2026-02-18)
-
Exposing Blindspots: Cultural Bias Evaluation in Generative Image Models (2025-10-22)
-
SynCred-Bench: Benchmarking Synthetic Credibility in AI-Generated Visual Misinformation (2026-06-02)
-
RoboTrustBench: Benchmarking the Trustworthiness of Video World Models for Robotic Manipulation (2026-06-01)
-
SafeGen-Bench: Benchmarking Safety in Image-Conditioned Text-to-Video Generation (2026-05-31)
-
Broken Memories: Detecting and Mitigating Memorization in Diffusion Models with Degraded Generations (2026-05-21)
-
Hallucination Behavior in Multimodal LLMs Across Agricultural Image Interpretation and Generation Tasks (2026-05-26)
-
FairJudge: Abstention-Aware Multimodal Judges for Fairness and Alignment Evaluation in Text-to-Image Models (2025-10-26)
-
The Silent Brush: Evaluating Artistic Style Leakage in AI Art Generation (2026-05-17)
-
Red-Teaming Text-to-Image Models via In-Context Experience Replay and Semantic-Preserving Prompt Rewriting (2024-11-25)
-
Latent Space Probing for Adult Content Detection in Video Generative Models (2026-04-25)
-
T2I-BiasBench: A Multi-Metric Framework for Auditing Demographic and Cultural Bias in Text-to-Image Models (2026-04-14)
-
Text-to-Image Models and Their Representation of People from Different Nationalities Engaging in Activities (2025-04-08)
-
GLEaN: A Text-to-image Bias Detection Approach for Public Comprehension (2026-04-10)
-
FlowGuard: Towards Lightweight In-Generation Safety Detection for Diffusion Models via Linear Latent Decoding (2026-04-09)
-
DialectGen: Benchmarking and Improving Dialect Robustness in Multimodal Generation (2025-10-16)
-
Erasure or Erosion? Evaluating Compositional Degradation in Unlearned Text-To-Image Diffusion Models (2026-04-06)
-
From Unlearning to UNBRANDING: A Benchmark for Trademark-Safe Text-to-Image Generation (2025-12-15)
-
PROBE: Diagnosing Residual Concept Capacity in Erased Text-to-Video Diffusion Models (2026-03-23)
-
REFORGE: Multi-modal Attacks Reveal Vulnerable Concept Unlearning in Image Generation Models (2026-03-17)
-
Culture in Action: Evaluating Text-to-Image Models through Social Activities (2025-11-07)
-
Investigating Disability Representations in Text-to-Image Models (2026-02-04)
-
GeoDiv: Framework For Measuring Geographical Diversity In Text-To-Image Models (2026-02-25)
-
Synthetic History: Evaluating Visual Representations of the Past in Diffusion Models (2025-05-18)
-
From Preferences to Prejudice: The Role of Alignment Tuning in Shaping Social Bias in Video Diffusion Models (2025-10-20)
-
Aesthetics as Structural Harm: Algorithmic Lookism Across Text-to-Image Generation and Classification (2026-01-15)
-
Are Video Generation Models Geographically Fair? An Attraction-Centric Evaluation of Global Visual Knowledge (2026-01-26)
-
Wukong Framework for Not Safe For Work Detection in Text-to-Image systems (2025-08-01)
-
CulturalFrames: Assessing Cultural Expectation Alignment in Text-to-Image Models and Evaluation Metrics (2025-06-10)
-
Beautiful Images, Toxic Words: Understanding and Addressing Offensive Text in Generated Images (2025-02-07)
-
Prompting4Debugging: Red-Teaming Text-to-Image Diffusion Models by Finding Problematic Prompts (2023-09-12)
-
Bias Detection and Rotation-Robustness Mitigation in Vision-Language Models and Generative Image Models (2026-01-09)
-
How Many Images Does It Take? Estimating Imitation Thresholds in Text-to-Image Models (2024-10-19)
-
MineTheGap: Automatic Mining of Biases in Text-to-Image Models (2025-12-15)
-
DREAM: Scalable Red Teaming for Text-to-Image Generative Systems via Distribution Modeling (2025-07-22)
-
Where Culture Fades: Revealing the Cultural Gap in Text-to-Image Generation (2025-11-21)
-
The Path to Reconciling Quality and Safety in Text-to-Image Generation: Dataset, Method, and Evaluation (2025-04-19)
-
Prompt-Based Safety Guidance Is Ineffective for Unlearned Text-to-Image Diffusion Models (2025-11-06)
-
Holistic Unlearning Benchmark: A Multi-Faceted Evaluation for Text-to-Image Diffusion Model Unlearning (2024-10-08)
-
AutoPrompt: Automated Red-Teaming of Text-to-Image Models via LLM-Driven Adversarial Prompts (2025-10-28)
-
Comprehensive Evaluation and Analysis for NSFW Concept Erasure in Text-to-Image Diffusion Models (2025-05-21)
-
The Digital Mirror: Gender Bias and Occupational Stereotypes in AI-Generated Images (2025-10-08)
-
MetaLogic: Robustness Evaluation of Text-to-Image Models via Logically Equivalent Prompts (2025-10-01)
-
A Large Scale Analysis of Gender Biases in Text-to-Image Generative Models (2025-03-30)
-
Beyond the Prompt: Gender Bias in Text-to-Image Models, with a Case Study on Hospital Professions (2025-09-27)
-
BiasMap: Leveraging Cross-Attentions to Discover and Mitigate Hidden Social Biases in Text-to-Image Generation (2025-09-16)
-
A Framework for Benchmarking Fairness-Utility Trade-offs in Text-to-Image Models via Pareto Frontiers (2025-08-22)
-
Whose View of Safety? A Deep DIVE Dataset for Pluralistic Alignment of Text-to-Image Models (2025-07-15)
-
Towards Evaluating Robustness of Prompt Adherence in Text to Image Models (2025-07-09)
-
Hidden Bias in the Machine: Stereotypes in Text-to-Image Models (2025-06-09)
-
CuRe: Cultural Gaps in the Long Tail of Text-to-Image Systems (2025-06-09)
-
PRJ: Perception-Retrieval-Judgement for Generated Images (2025-06-04)
-
TEAR: Temporal-Aware Automated Red-Teaming for Text-to-Video Models (2025-11-26)
-
How Bias Binds: Measuring Hidden Associations for Bias Control in Text-to-Image Compositions (2025-11-10)
Note: Introduces the bias adherence score.
-
T2I-RiskyPrompt: A Benchmark for Safety Evaluation, Attack, and Defense on Text-to-Image Model (2025-10-25)
-
OVERT: A Benchmark for Over-Refusal Evaluation on Text-to-Image Models (2025-05)
-
Proactive Defense Benchmark against Deepfake Generation (ICML 2026)
-
HoloFair: Unified T2I Fairness Evaluation and Fair-GRPO Debiasing (2026-05-23)
Note: Introduces the Multi-attribute, Group-wise Bias Index (MGBI).
-
JailbreakDiffBench: A Comprehensive Benchmark for Jailbreaking Diffusion Models (ICCV 2025)
-
EMMA: Concept Erasure Benchmark with Comprehensive Semantic Metrics and Diverse Evaluation (CVPR 2026)
-
Erasing More Than Intended? How Concept Erasure Degrades the Generation of Non-Target Concepts (2025-01-16)
-
Bias in Gender Bias Benchmarks: How Spurious Features Distort Evaluation (2025-09-09)
-
MLLM-as-a-Judge for Image Safety without Human Labeling (2024-12-31)
-
VBench++: Comprehensive and Versatile Benchmark Suite for Video Generative Models (2024-11-20)
-
BIGbench: A Unified Benchmark for Social Bias in Text-to-Image Generative Models Based on Multi-modal LLM (2024-07-21)
-
Towards Understanding Unsafe Video Generation (2024-07-17)
Note: Proposes Latent Variable Defense (LVD) which works within the model's internal sampling process
-
The Factuality Tax of Diversity-Intervened Text-to-Image Generation: Benchmark and Fact-Augmented Intervention (2024-06-29)
-
Note: Normalized Entropy metric introduced
-
Latent Directions: A Simple Pathway to Bias Mitigation in Generative AI (2024-06-10)
-
Evaluating and Mitigating IP Infringement in Visual Generative AI (2024-06-07)
-
Improving Geo-diversity of Generated Images with Contextualized Vendi Score Guidance (2024-06-06)
-
AI-Face: A Million-Scale Demographically Annotated AI-Generated Face Dataset and Fairness Benchmark (2024-06-02)
-
FAIntbench: A Holistic and Precise Benchmark for Bias Evaluation in Text-to-Image Models (2024-05-28)
-
ART: Automatic Red-teaming for Text-to-Image Models to Protect Benign Users (2024-05-24)
-
Condition Likelihood Discrepancy from Membership Inference on Text-to-Image Diffusion Models via Conditional Likelihood Discrepancy (2024-05-23)
-
Could It Be Generated? Towards Practical Analysis of Memorization in Text-To-Image Diffusion Models (2024-05-09)
-
Towards Geographic Inclusion in the Evaluation of Text-to-Image Models (2024-05-07)
-
UnsafeBench: Benchmarking Image Safety Classifiers on Real-World and AI-Generated Images (2024-05-06)
-
Espresso: Robust Concept Filtering in Text-to-Image Models (2024-04-30)
Note: Paper is about filtering unacceptable concepts, not evaluation.
-
Ethical-Lens: Curbing Malicious Usages of Open-Source Text-to-Image Models (2024-04-18)
-
OpenBias: Open-set Bias Detection in Text-to-Image Generative Models (2024-04-11)
-
Survey of Bias In Text-to-Image Generation: Definition, Evaluation, and Mitigation (2024-04-01)
-
Lost in Translation? Translation Errors and Challenges for Fair Assessment of Text-to-Image Models on Multilingual Concepts (2024-03-17, NAACL 2024)
-
Evaluating Text-to-Image Generative Models: An Empirical Study on Human Image Synthesis (2024-03-08)
-
Position: Towards Implicit Prompt For Text-To-Image Models (2024-03-04)
Note: ImplicitBench, new benchmark
-
The Male CEO and the Female Assistant: Probing Gender Biases in Text-To-Image Models Through Paired Stereotype Test (2024-02-16)
-
Multilingual Text-to-Image Generation Magnifies Gender Stereotypes and Prompt Engineering May Not Help You (2024-01-29)
-
Benchmarking the Fairness of Image Upsampling Methods (2024-01-24)
-
ViSAGe: A Global-Scale Analysis of Visual Stereotypes in Text-to-Image Generation (2024-01-02)
-
New Job, New Gender? Measuring the Social Bias in Image Generation Models (2024-01-01)
-
Distribution Bias, Jaccard Hallucination, Generative Miss Rate from Quantifying Bias in Text-to-Image Generative Models (2023-12-20)
-
TIBET: Identifying and Evaluating Biases in Text-to-Image Generative Models (2023-12-03)
Note: CAS and BAV novel metric introduced
-
Holistic Evaluation of Text-To-Image Models (2023-11-07)
-
Sociotechnical Safety Evaluation of Generative AI Systems (2023-10-18)
-
Navigating Cultural Chasms: Exploring and Unlocking the Cultural POV of Text-To-Image Models (2023-10-03)
Note: Evaluate the cultural content of TTI-generated images
-
ITI-GEN: Inclusive Text-to-Image Generation (2023-09-11, ICCV 2023)
-
DIG In: Evaluating Disparities in Image Generations with Indicators for Geographic Diversity (2023-08-11)
-
On the Cultural Gap in Text-to-Image Generation (2023-07-06)
-
Evaluating the Robustness of Text-to-image Diffusion Models against Real-world Attacks (2023-06-16)
-
Disparities in Text-to-Image Model Concept Possession Across Languages (2023-06-12)
Note: Benchmark of multilingual parity in conceptual possession
-
Evaluating the Social Impact of Generative AI Systems in Systems and Society (2023-06-09)
-
Word-Level Explanations for Analyzing Bias in Text-to-Image Models (2023-06-03)
-
Multilingual Conceptual Coverage in Text-to-Image Models (2023-06-02, ACL 2023)
Note: CoCo-CroLa, benchmark for multilingual parity of text-to-image models
-
T2IAT: Measuring Valence and Stereotypical Biases in Text-to-Image Generation (2023-06-01)
-
SneakyPrompt: Jailbreaking Text-to-image Generative Models (2023-05-20)
-
Inspecting the Geographical Representativeness of Images from Text-to-Image Models (2023-05-18)
-
Multimodal Composite Association Score: Measuring Gender Bias in Generative Multimodal Models (2023-04-26)
-
Uncurated Image-Text Datasets: Shedding Light on Demographic Bias (2023-04-06, CVPR 2023)
-
Social Biases through the Text-to-Image Generation Lens (2023-03-30)
-
Stable Bias: Analyzing Societal Representations in Diffusion Models (2023-03-20)
-
Auditing Gender Presentation Differences in Text-to-Image Models (2023-02-07)
-
Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models (2022-11-09, CVPR 2023)
Note: SLD removes and suppresses inappropriate image parts during the diffusion process
-
How well can Text-to-Image Generative Models understand Ethical Natural Language Interventions? (2022-10-27)
-
Exploiting Cultural Biases via Homoglyphs in Text-to-Image Synthesis (2022-09-19)
-
DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Models (2022-02-08, ICCV 2023)
Note: PaintSkills, evaluation for visual reasoning capabilities and social biases
Not for visual generation, but related evaluations of other models like LLMs
-
The African Woman is Rhythmic and Soulful: Evaluation of Open-ended Generation for Implicit Biases (2024-07-01)
-
Extrinsic Evaluation of Cultural Competence in Large Language Models (2024-06-17)
-
Benchmarking Trustworthiness of Multimodal Large Language Models: A Comprehensive Study (2024-06-11)
-
HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal (2024-02-06)
-
FACET: Fairness in Computer Vision Evaluation Benchmark (2023-08-31)
-
Gender Biases in Automatic Evaluation Metrics for Image Captioning (2023-05-24)
-
Fairness Indicators for Systematic Assessments of Visual Feature Extractors (2022-02-15)
-
Scene Graph(SG)-IoU, Relation-IoU, and Entity-IoU (using GPT-4v) from SG-Adapter: Enhancing Text-to-Image Generation with Scene Graph Guidance (2024-05-24)
-
Relation Accuracy & Entity Accuracy from ReVersion: Diffusion-Based Relation Inversion from Images (2023-03-23)
-
Testing Relational Understanding in Text-Guided Image Generation (2022-07-29)
-
DirectorBench: Diagnosing Long-Form Video Generation with Personalized Multi-Agent Evaluation (2026-05-28)
-
VideoArgus: Agentic Rubric-Grounded Unified Evaluation for Video Generation and Editing (2026-08-06)
Note: Introduces VideoArgus-Bench and sample-specific, evidence-grounded rubrics for five video generation and editing settings.
-
VideoWeaver: Evaluating and Evolving Skills for Agentic Long Video Generation (2026-06-06)
Note: Evaluates both agent execution traces and final long videos with an evidence-grounded agent-as-judge.
-
BackTranslation2.0 -- A Linguistically Motivated Metric to Assess Sign Language Production (2026-06-27, ECCV 2026)
Note: Introduces an agentic, linguistically grounded metric for grammatical correctness, phonological accuracy, motion fluency, and generation fidelity in sign language production.
-
Do Joint Audio-Video Generation Models Understand Physics? (2026-05-08)
Note: Introduces AV-Phys Bench and the ReAct-style AV-Phys Agent for evaluating physical commonsense in joint audio-video generation.
-
FAGER: Factually Grounded Evaluation and Refinement of Text-to-Image Models (2026-05-18)
Note: Agentically constructs factual rubrics to evaluate and refine visually verifiable factual correctness in generated images.
-
Note: Introduces EdiVal-IF, EdiVal-CC, and EdiVal-VQ.
-
A Unified Agentic Framework for Evaluating Conditional Image Generation (2025-04-09)
-
Evaluation Agent: Efficient and Promptable Evaluation Framework for Visual Generative Models (2024-12-10)
-
VideoGen-Eval: Agent-based System for Video Generation Evaluation (2025-03-30)
-
Evaluating Hallucination in Text-to-Image Diffusion Models with Scene-Graph based Question-Answering Agent (2024-12-07)
-
UEval: A Benchmark for Unified Multimodal Generation (2026-01-29)
-
A High-Quality Dataset and Reliable Evaluation for Interleaved Image-Text Generation (2025-06-11, ICLR 2026)
Note: Introduces the InterSyn dataset and SynJudge evaluator.
-
Unison: Benchmarking Unified Multimodal Models via Synergistic Understanding and Generation (2026-06-25)
-
Do Text Edits Generalize to Visual Generation? Benchmarking Cross-Modal Knowledge Editing in UMMs (2026-05-30, ICML 2026)
-
AlphaGRPO: Unlocking Self-Reflective Multimodal Generation in UMMs via Decompositional Verifiable Reward (2026-05-12)
-
MICON-Bench: Benchmarking and Enhancing Multi-Image Context Image Generation in Unified Multimodal Models (2026-02-23)
-
FysicsWorld: A Unified Full-Modality Benchmark for Any-to-Any Understanding, Generation, and Reasoning (2025-12-14)
-
WEAVE: Unleashing and Benchmarking the In-context Interleaved Comprehension and Generation (2025-11-14, CVPR 2026)
-
ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generation (2025-11-03)
-
GIR-Bench: Versatile Benchmark for Generating Images with Reasoning (2025-10-13)
-
SUDER: Self-Improving Unified Large Multimodal Models for Understanding and Generation with Dual Self-Rewards (2025-06-09)
-
FactVerse: A Benchmark for Factual Consistency in Interleaved Image–Text Generation (ACL 2026)
-
MME-Unify: A Comprehensive Benchmark for Unified Multimodal Understanding and Generation Models (ICLR 2026)
-
Evaluation-Verification Reward for Consistent Multi-Reference Image Editing (2026-07-31)
-
PRISM: Prompt Refinement via Image-grounded Self-rewarding Mechanism for Text-to-Image Generation (2026-07-27)
-
Read It Back: Pretrained MLLMs Are Zero-Shot Reward Models for Text-to-Image Generation (2026-07-13)
-
Reward Lightning: Fast Video Generation via Homologous Preference Distillation (2026-07-04)
-
Optimizing Visual Generative Models via Distribution-wise Rewards (2026-07-02)
-
PortraitGen: Exemplar-Driven GRPO with Dual-Reward Guidance for Photorealistic Portrait Generation (2026-06-25)
-
DivRL: Disentangled Self-Similarity Rewards for Diverse Subject-Driven Generation (2026-06-22)
-
ProductConsistency: Improving Product Identity Preservation in Instruction-Based Image Editing via SFT and RL (2026-06-17)
-
HPSv3++: Scaling Reward Models Across the Full Spectrum of Diffusion Model Capabilities (2026-06-12)
-
Where, What, Why, and Importance: Structured Defect Grounding for Text-to-Image Feedback (2026-06-04)
-
DRM: Diffusion-based Reward Model With Step-wise Guidance (2026-05-25)
-
Geo-Align: Video Generation Alignment via Metric Geometry Reward (2026-05-22)
-
AutoRubric-T2I: Robust Rule-Based Reward Model for Text-to-Image Alignment (2026-05-17)
-
CreFlow: Corrective Reflow for Sparse-Reward Embodied Video Diffusion RL (2026-05-14)
-
PhyMotion: Structured 3D Motion Reward for Physics-Grounded Human Video Generation (2026-05-14)
-
Arena as Offline Reward: Efficient Fine-Grained Preference Optimization for Diffusion Models (2026-05-07)
-
AesRM: Improving Video Aesthetics with Expert-Level Feedback (2026-04-30)
-
Leveraging Verifier-Based Reinforcement Learning in Image Editing (2026-04-30)
-
HP-Edit: A Human-Preference Post-Training Framework for Image Editing (2026-04-21)
-
PromptEcho: Annotation-Free Reward from Vision-Language Models for Text-to-Image Reinforcement Learning (2026-04-14)
-
ReinDriveGen: Reinforcement Post-Training for Out-of-Distribution Driving Scene Generation (2026-04-01)
-
VGGRPO: Towards World-Consistent Video Generation with 4D Latent Reward (2026-03-27)
-
Policy-based Tuning of Autoregressive Image Models with Instance- and Distribution-Level Rewards (2026-03-24)
-
VIGOR: VIdeo Geometry-Oriented Reward for Temporal Generative Alignment (2026-03-17)
-
GlyphPrinter: Region-Grouped Direct Preference Optimization for Glyph-Accurate Visual Text Rendering (2026-03-16)
-
EditHF-1M: A Million-Scale Rich Human Preference Feedback for Image Editing (2026-03-16)
-
LatSearch: Latent Reward-Guided Search for Faster Inference-Time Scaling in Video Diffusion (2026-03-15)
-
Trust Your Critic: Robust Reward Modeling and Reinforcement Learning for Faithful Image Editing and Generation (2026-03-12)
-
Learning to Generate via Understanding: Understanding-Driven Intrinsic Rewarding for Unified Multimodal Models (2026-03-06)
-
When Rubrics Fail: Error Enumeration as Reward in Reference-Free RL Post-Training for Virtual Try-On (2026-03-05)
-
BeautyGRPO: Aesthetic Alignment for Face Retouching via Dynamic Path Guidance and Fine-Grained Preference Modeling (2026-03-01)
-
DesignSense: A Human Preference Dataset and Reward Modeling Framework for Graphic Layout Generation (2026-02-26)
-
TextPecker: Rewarding Structural Anomaly Quantification for Enhancing Visual Text Rendering (2026-02-24)
-
Beyond VLM-Based Rewards: Diffusion-Native Latent Reward Modeling (2026-02-11)
-
SpatialReward: Bridging the Perception Gap in Online RL for Image Editing via Explicit Spatial Reasoning (2026-02-07)
-
Unified Personalized Reward Model for Vision Generation (2026-02-02)
-
CamPilot: Improving Camera Control in Video Diffusion Model with Efficient Camera Reward Feedback (2026-01-22)
-
Diffusion-DRF: Free, Rich, and Differentiable Reward for Video Diffusion Fine-Tuning (2026-01-07)
-
Understanding Reward Hacking in Text-to-Image Reinforcement Learning (2026-01-06)
-
PhyGDPO: Physics-Aware Groupwise Direct Preference Optimization for Physically Consistent Text-to-Video Generation (2025-12-31)
-
Taming Preference Mode Collapse via Directional Decoupling Alignment in Diffusion Reinforcement Learning (2025-12-30)
-
GeCo: Evaluating Geometric Consistency for Video Generation via Motion and Structure (2025-12-25)
-
MotionEdit: Benchmarking and Learning Motion-Centric Image Editing (2025-12-11)
-
PaCo-RL: Advancing Reinforcement Learning for Consistent Image Generation with Pairwise Reward Modeling (2025-12-02)
-
Taming Camera-Controlled Video Generation with Verifiable Geometry Reward (2025-12-02)
-
McSc: Motion-Corrective Preference Alignment for Video Generation with Self-Critic Hierarchical Reasoning (2025-11-28)
-
RubricRL: Simple Generalizable Rewards for Text-to-Image Generation (2025-11-25)
-
HiCoGen: Hierarchical Compositional Text-to-Image Generation in Diffusion Models via Reinforcement Learning (2025-11-25)
-
Personalized Reward Modeling for Text-to-Image Generation (2025-11-21)
-
PhysCorr: Dual-Reward DPO for Physics-Constrained Text-to-Video Generation with Automated Preference Selection (2025-11-06)
-
M$^{3}$T2IBench: A Large-Scale Multi-Category, Multi-Instance, Multi-Relation Text-to-Image Benchmark (2025-10-27)
-
StableSketcher: Enhancing Diffusion Model for Pixel-based Sketch Generation via Visual Question Answering Feedback (2025-10-23)
-
D2D: Detector-to-Differentiable Critic for Improved Numeracy in Text-to-Image Generation (2025-10-22)
-
Video Consistency Distance: Enhancing Temporal Consistency for Image-to-Video Generation via Reward-Based Fine-Tuning (2025-10-22)
-
Uniworld-V2: Reinforce Image Editing with Diffusion Negative-aware Finetuning and MLLM Implicit Feedback (2025-10-19)
-
VR-Thinker: Boosting Video Reward Models through Thinking-with-Image Reasoning (2025-10-12)
-
ImageDoctor: Diagnosing Text-to-Image Generation via Grounded Image Reasoning (2025-10-01)
-
EditScore: Unlocking Online RL for Image Editing via High-Fidelity Reward Modeling (2025-09-28)
-
RLGF: Reinforcement Learning with Geometric Feedback for Autonomous Driving Video Generation (2025-09-20)
-
RewardDance: Reward Scaling in Visual Generation (2025-09-10)
-
UMO: Scaling Multi-Identity Consistency for Image Customization via Matching Reward (2025-09-08)
-
The Mind's Eye: A Multi-Faceted Reward Framework for Guiding Visual Metaphor Generation (2025-08-26)
-
FantasyTalking2: Timestep-Layer Adaptive Preference Optimization for Audio-Driven Portrait Animation (2025-08-15)
-
Uni-Layout: Integrating Human Feedback in Unified Layout Generation and Evaluation (2025-08-04)
-
Multimodal LLMs as Customized Reward Models for Text-to-Image Generation (2025-07-28)
-
RDPO: Real Data Preference Optimization for Physics Consistency Video Generation (2025-06-23)
-
Doctor Approved: Generating Medically Accurate Skin Disease Images through AI-Expert Feedback (2025-06-14)
-
AssetDropper: Asset Extraction via Diffusion Models with Reward-Driven Optimization (2025-06-06)
-
GoT-R1: Unleashing Reasoning Capability of MLLM for Visual Generation with Reinforcement Learning (2025-05-22)
-
We'll Fix it in Post: Improving Text-to-Video Generation with Neuro-Symbolic Feedback (2025-04-24)
-
Aligning Anime Video Generation with Human Feedback (2025-04-14)
-
Unified Reward Model for Multimodal Understanding and Generation (2025-03-07)
-
Dual-IPO: Dual-Iterative Preference Optimization for Text-to-Video Generation (2025-02-04)
-
Diffusion Model as a Noise-Aware Latent Reward Model for Step-Level Preference Optimization (2025-02-03)
-
Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Step (2025-01-23)
-
Detection-Driven Object Count Optimization for Text-to-Image Diffusion Models (2024-08-21)
-
Debiasing Text-to-Image Evaluation via Implicit Cultural Alignment Reward Modeling (2026-07-17)
Note: Introduces an implicit cultural alignment reward model trained/evaluated on human cultural-preference pairs.
-
IE-Critic-R1: Advancing the Explanatory Measurement of Text-Driven Image Editing for Human Perception Alignment (2025-11-22)
Note: Introduces IE-Bench with human MOS annotations and an RLVR-trained critic model for explainable, human-aligned image-editing quality assessment.
-
CoCoEdit: Content-Consistent Image Editing via Region Regularized Reinforcement Learning (2026-02-15)
Note: Introduces a region-regularized pixel-level similarity reward for content consistency during image editing.
-
ReasonEdit: Towards Interpretable Image Editing Evaluation via Reinforcement Learning (2026-05-08)
Note: Introduces 1.3M human judgments, the human-aligned RE-Reward model, and an interpretable image-editing evaluator.
-
RewardFlow: Generate Images by Optimizing What You Reward (2026-04-09)
Note: Introduces a differentiable VQA reward for semantic alignment, localized grounding, object consistency, and human preference in image editing.
-
GEditBench v2: A Human-Aligned Benchmark for General Image Editing (2026-03-30)
Note: Introduces PVC-Judge and expert-annotated VCReward-Bench for human-aligned visual-consistency assessment.
-
PISCES: Annotation-free Text-to-Video Post-Training via Optimal Transport-Aligned Rewards (2026-02-02)
Note: Introduces quality/temporal-coherence and semantic spatio-temporal rewards aligned through optimal transport.
-
Identity-GRPO: Optimizing Multi-Human Identity-preserving Video Generation via Reinforcement Learning (2025-10-16)
Note: Introduces a human-preference reward model for multi-human identity consistency.
-
PSR: Scaling Multi-Subject Personalized Image Generation with Pairwise Subject-Consistency Rewards (2025-12-01)
Note: Introduces Pairwise Subject-Consistency Rewards and a three-dimensional multi-subject personalization benchmark.
-
PosterOmni: Generalized Artistic Poster Creation via Task Distillation and Unified Reward Feedback (2026-02-12)
Note: Introduces PosterOmni Reward Feedback for entity preservation and aesthetic preference, together with PosterOmni-Bench.
-
USO: Unified Style and Subject-Driven Generation via Disentangled and Reward Learning (2025-08-26)
Note: Introduces style reward learning and USO-Bench for joint style similarity and subject fidelity evaluation.
-
MagicWand: A Universal Agent for Generation and Evaluation Aligned with User Preference (2025-11-23)
Note: Introduces the human-preference-oriented UniPrefer-100K dataset and UniPreferBench for image and video generation.
-
Local Intrinsic Dimension Unveils Hallucinations in Diffusion Models (2026-05-06)
Note: Introduces local intrinsic dimension as a structural-hallucination signal and Intrinsic Quenching as its corrective mechanism.
-
CRAFT: Clinical Reward-Aligned Finetuning for Medical Image Synthesis (2026-05-12)
Note: Introduces the Clinical Alignment Score (CAS) as a clinical reward, supported by a blinded physician preference study.
-
GT-SVJ: Generative-Transformer-Based Self-Supervised Video Judge For Efficient Video Reward Modeling (2026-02-05, CVPR 2026)
Note: Introduces a temporally-aware video reward model aligned with human preferences.
-
Proprio: Latent Self-Scoring and Inference-Time Refinement for Physically Plausible Video Generation (2026-05-27)
-
PhysRVG: Physics-Aware Unified Reinforcement Learning for Video Generative Models (2026-01-16)
-
What about gravity in video generation? Post-Training Newton's Laws with Verifiable Rewards (2025-11-29)
-
WearWow: Native 2K Multi-Garment Virtual Try-On via Adaptive Token Packing and Preference Alignment (2026-07-22)
Note: Introduces the Multi-dimensional Try-on Reward (MTR) system for semantic guidance and cloth-distribution alignment.
-
Hallo4: High-Fidelity Dynamic Portrait Animation via Direct Preference Optimization and Temporal Motion Modulation (2025-05-29)
-
Thinking with Frames: Generative Video Distortion Evaluation via Frame Reward Model (2026-01-07)
-
A Benchmark for Text-to-Image Alignment and Kernelized Direct Preference Optimization (2025-06-17)
Note: Introduces the Alignment Quality Index (AQI).
-
StyleDoctor: Towards Specialist Reward Model for Style-centric Generation Tasks (CVPR 2026)
-
PosterReward: Unlocking Accurate Evaluation for High-Quality Graphic Design Generation (CVPR 2026)
-
HPSv3: Towards Wide-Spectrum Human Preference Score (2025-08-05, ICCV 2025)
-
OneReward: Unified Mask-Guided Image Generation via Multi-Task Human Preference Learning (2025-08-28)
-
Prompt-A-Video: Prompt Your Video Diffusion Model via Preference-Aligned LLM (2024-12-19)
-
Improving Video Generation with Human Feedback (2025-01-23, NeurIPS 2025)
-
LiFT: Leveraging Human Feedback for Text-to-Video Model Alignment (2024-12-24)
-
VideoDPO: Omni-Preference Alignment for Video Diffusion Generation (2024-12-18)
-
Boosting Text-to-Video Generative Model with MLLMs Feedback (2024-09-26,NeurIPS 2024)
-
Direct Unlearning Optimization for Robust and Safe Text-to-Image Models (2024-07-17)
-
Safeguard Text-to-Image Diffusion Models with Human Feedback Inversion (2024-07-17, ECCV 2024)
-
Subject-driven Text-to-Image Generation via Preference-based Reinforcement Learning (2024-07-16)
-
Video Diffusion Alignment via Reward Gradients (2024-07-11)
-
Powerful and Flexible: Personalized Text-to-Image Generation via Reinforcement Learning (2024-07-09)
-
Aligning Human Motion Generation with Human Perceptions (2024-07-02)
-
PopAlign: Population-Level Alignment for Fair Text-to-Image Generation (2024-06-28)
-
Prompt Refinement with Image Pivot for Text-to-Image Generation (2024-06-28, ACL 2024)
-
Diminishing Stereotype Bias in Image Generation Model using Reinforcemenlent Learning Feedback (2024-06-27)
-
Beyond Thumbs Up/Down: Untangling Challenges of Fine-Grained Feedback for Text-to-Image Generation (2024-06-24)
-
Batch-Instructed Gradient for Prompt Evolution: Systematic Prompt Optimization for Enhanced Text-to-Image Synthesis (2024-06-13)
-
InstructRL4Pix: Training Diffusion for Image Editing by Reinforcement Learning (2024-06-14)
-
Diffusion-RPO: Aligning Diffusion Models through Relative Preference Optimization (2024-06-10)
Note: new evaluation metric: style alignment
-
Margin-aware Preference Optimization for Aligning Diffusion Models without Reference (2024-06-10)
-
ReNO: Enhancing One-step Text-to-Image Models through Reward-based Noise Optimization (2024-06-06)
-
Step-aware Preference Optimization: Aligning Preference with Denoising Performance at Each Step (2024-06-06)
-
Improving GFlowNets for Text-to-Image Diffusion Alignment (2024-06-02)
Note: Improves text-to-image alignment with reward function
-
Enhancing Reinforcement Learning Finetuned Text-to-Image Generative Model Using Reward Ensemble (2024-06-01)
-
Boost Your Own Human Image Generation Model via Direct Preference Optimization with AI Feedback (2024-05-30)
-
T2V-Turbo: Breaking the Quality Bottleneck of Video Consistency Model with Mixed Reward Feedback (2024-05-29)
-
Curriculum Direct Preference Optimization for Diffusion and Consistency Models (2024-05-22)
-
Class-Conditional self-reward mechanism for improved Text-to-Image models (2024-05-22)
-
Understanding and Evaluating Human Preferences for AI Generated Images with Instruction Tuning (2024-05-12)
-
Deep Reward Supervisions for Tuning Text-to-Image Diffusion Models (2024-05-01)
-
ID-Aligner: Enhancing Identity-Preserving Text-to-Image Generation with Reward Feedback Learning (2024-04-23)
-
Hyper-SD: Trajectory Segmented Consistency Model for Efficient Image Synthesis (2024-04-21)
Note: Human feedback learning to enhance model performance in low-steps regime
-
Prompt Optimizer of Text-to-Image Diffusion Models for Abstract Concept Understanding (2024-04-17)
-
ControlNet++: Improving Conditional Controls with Efficient Consistency Feedback (2024-04-11)
-
UniFL: Improve Stable Diffusion via Unified Feedback Learning (2024-04-08)
-
YaART: Yet Another ART Rendering Technology (2024-04-08)
-
ByteEdit: Boost, Comply and Accelerate Generative Image Editing (2024-04-07)
Note: ByteEdit, feedback learning framework for Generative Image Editing tasks
-
Aligning Diffusion Models by Optimizing Human Utility (2024-04-06)
-
Dynamic Prompt Optimizing for Text-to-Image Generation (2024-04-05)
-
Pixel-wise RL on Diffusion Models: Reinforcement Learning from Rich Feedback (2024-04-05)
-
CoMat: Aligning Text-to-Image Diffusion Model with Image-to-Text Concept Matching (2024-04-04)
-
VersaT2I: Improving Text-to-Image Models with Versatile Reward (2024-03-27)
-
Improving Text-to-Image Consistency via Automatic Prompt Optimization (2024-03-26)
-
RL for Consistency Models: Faster Reward Guided Text-to-Image Generation (2024-03-25)
-
AGFSync: Leveraging AI-Generated Feedback for Preference Optimization in Text-to-Image Generation (2024-03-20)
-
Reward Guided Latent Consistency Distillation (2024-03-16)
-
Optimizing Negative Prompts for Enhanced Aesthetics and Fidelity in Text-To-Image Generation (2024-03-12)
-
Debiasing Text-to-Image Diffusion Models (2024-02-22)
-
Universal Prompt Optimizer for Safe Text-to-Image Generation (2024-02-16, NAACL 2024)
-
Social Reward: Evaluating and Enhancing Generative AI through Million-User Feedback from an Online Creative Community (2024-02-15, ICLR 2024)
-
A Dense Reward View on Aligning Text-to-Image Diffusion with Preference (2024-02-13, ICML 2024)
-
Confronting Reward Overoptimization for Diffusion Models: A Perspective of Inductive and Primacy Biases (2024-02-13, ICML 2024)
-
PRDP: Proximal Reward Difference Prediction for Large-Scale Reward Finetuning of Diffusion Models (2024-02-13)
-
Human Aesthetic Preference-Based Large Text-to-Image Model Personalization: Kandinsky Generation as an Example (2024-02-09)
-
Divide and Conquer: Language Models can Plan and Self-Correct for Compositional Text-to-Image Generation (2024-01-28)
-
Large-scale Reinforcement Learning for Diffusion Models (2024-01-20)
-
Parrot: Pareto-optimal Multi-Reward Reinforcement Learning Framework for Text-to-Image Generation (2024-01-11)
-
InstructVideo: Instructing Video Diffusion Models with Human Feedback (2023-12-19)
-
Rich Human Feedback for Text-to-Image Generation (2023-12-15, CVPR 2024)
-
iDesigner: A High-Resolution and Complex-Prompt Following Text-to-Image Diffusion Model for Interior Design (2023-12-07)
-
InstructBooth: Instruction-following Personalized Text-to-Image Generation (2023-12-04)
-
DreamSync: Aligning Text-to-Image Generation with Image Understanding Feedback (2023-11-29)
-
Enhancing Diffusion Models with Text-Encoder Reinforcement Learning (2023-11-27)
-
AdaDiff: Adaptive Step Selection for Fast Diffusion (2023-11-24)
-
Using Human Feedback to Fine-tune Diffusion Models without Any Reward Model (2023-11-22)
-
Diffusion Model Alignment Using Direct Preference Optimization (2023-11-21)
-
BeautifulPrompt: Towards Automatic Prompt Engineering for Text-to-Image Synthesis (2023-11-12)
-
Quality Diversity through Human Feedback: Towards Open-Ended Diversity-Driven Optimization (2023-10-18, ICML 2024)
-
Aligning Text-to-Image Diffusion Models with Reward Backpropagation (2023-10-05)
-
Directly Fine-Tuning Diffusion Models on Differentiable Rewards (2023-09-29)
-
LayoutLLM-T2I: Eliciting Layout Guidance from LLM for Text-to-Image Generation (2023-08-09, ACM MM 2023)
-
FABRIC: Personalizing Diffusion Models with Iterative Feedback (2023-07-19)
[
-
Divide, Evaluate, and Refine: Evaluating and Improving Text-to-Image Alignment with Iterative VQA Feedback (2023-07-10, NeurIPS 2023)
-
Censored Sampling of Diffusion Models Using 3 Minutes of Human Feedback (2023-07-06, NeurIPS 2023)
Note: Censored generation using a reward model
-
StyleDrop: Text-to-Image Generation in Any Style (2023-06-01)
Note: Iterative Training with Feedback
-
RealignDiff: Boosting Text-to-Image Diffusion Model with Coarse-to-fine Semantic Re-alignment (2023-05-31)
-
DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Models (2023-05-25, NeurIPS 2023)
-
Training Diffusion Models with Reinforcement Learning (2023-05-22)
Website](https://rl-diffusion.github.io/)
-
ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation (2023-04-12)
-
Confidence-aware Reward Optimization for Fine-tuning Text-to-Image Models (2023-04-02, ICLR 2024)
-
Human Preference Score: Better Aligning Text-to-Image Models with Human Preference (2023-03-25)
-
HIVE: Harnessing Human Feedback for Instructional Visual Editing (2023-03-16)
](https://github.com/salesforce/HIVE)
-
Aligning Text-to-Image Models using Human Feedback (2023-02-23)
-
Optimizing Prompts for Text-to-Image Generation (2022-12-19, NeurIPS 2023)
-
ELIQ: A Label-Free Framework for Quality Assessment of Evolving AI-Generated Images (2026-02-03)
-
HandEval: Taking the First Step Towards Hand Quality Evaluation in Generated Images (2025-10-10)
-
AU-IQA: A Benchmark Dataset for Perceptual Quality Assessment of AI-Enhanced User-Generated Content (2025-08-07)
-
Patch Knowledge Transfer for Efficient AI-Generated Image Quality Assessment (2026-07-06)
-
Decoupling Semantics from Distortions: Multi-Scale Two-Stream Vision-Language Alignment for AI-Generated Image Quality Assessment (2026-06-15, ICME 2026 Spotlight)
-
Plug In, Grade Right: Psychology-Inspired AGIQA (2025-12-28)
-
CAS-IQA: Teaching Vision-Language Models for Synthetic Angiography Quality Assessment (2025-05-23, ICONIP 2025)
-
M3-AGIQA: Multimodal, Multi-Round, Multi-Aspect AI-Generated Image Quality Assessment (2025-02-21)
-
CLIP-AGIQA: Boosting the Performance of AI-Generated Image Quality Assessment with CLIP (2024-08-27)
-
Text-Visual Semantic Constrained AI-Generated Image Quality Assessment (2025-07-14)
-
Towards Explainable Partial-AIGC Image Quality Assessment (2025-04)
-
Multi-Dimensional Text-to-Face Image Quality Assessment Using LLM: Database and Method
-
Descriptive Image Quality Assessment in the Wild (2024-05-29)
-
PKU-AIGIQA-4K: A Perceptual Quality Assessment Database for Both Text-to-Image and Image-to-Image AI-Generated Images (2024-04-29)
-
Large Multi-modality Model Assisted AI-Generated Image Quality Assessment (2024-04-27)
-
Adaptive Mixed-Scale Feature Fusion Network for Blind AI-Generated Image Quality Assessment (2024-04-23)
-
PCQA: A Strong Baseline for AIGC Quality Assessment Based on Prompt Condition (2024-04-20)
-
AIGIQA-20K: A Large Database for AI-Generated Image Quality Assessment (2024-04-04)
-
AIGCOIQA2024: Perceptual Quality Assessment of AI Generated Omnidirectional Images (2024-04-01)
-
Bringing Textual Prompt to AI-Generated Image Quality Assessment (2024-03-27, ICME 2024)
-
TIER: Text-Image Encoder-based Regression for AIGC Image Quality Assessment (2024-01-08)
-
PSCR: Patches Sampling-based Contrastive Regression for AIGC Image Quality Assessment (2023-12-10)
-
Exploring the Naturalness of AI-Generated Images (2023-12-09)
-
PKU-I2IQA: An Image-to-Image Quality Assessment Database for AI Generated Images (2023-11-27)
-
Appeal and quality assessment for AI-generated images (2023-07-18)
-
AGIQA-3K: An Open Database for AI-Generated Image Quality Assessment (2023-06-07)
-
A Perceptual Quality Assessment Exploration for AIGC Images (2023-03-22)
-
SPS: A Subjective Perception Score for Text-to-Image Synthesis (2021-04-27)
-
GIQA: Generated Image Quality Assessment (2020-03-19)
-
ArtFRD: A Fisher-Rao Mixture Metric for Generative Model Aesthetic Evaluation
Note: Introduces ArtFRD.
-
Multi-modal Learnable Queries for Image Aesthetics Assessment (2024-05-02, ICME 2024)
-
Aesthetic Scorer extension for SD Automatic WebUI (2023-01-15)
-
Simulacra Aesthetic-Models (2022-07-09)
-
Rethinking Image Aesthetics Assessment: Models, Datasets and Benchmarks (2022-07-01)
-
LAION-Aesthetics_Predictor V2: CLIP+MLP Aesthetic Score Predictor (2022-06-26)
-
LAION-Aesthetics_Predictor V1 (2022-05-21)
-
Follow-Your-Preference: Rethinking Preference Alignment for Image Inpainting (2025-09-27)
Note: Studies nine existing reward models, preference-data scaling, reward-model bias, and ensemble mitigation for image inpainting.
-
Follow-Your-Preference++: Rethinking Preference Alignment for Image Inpainting (2026-06-02)
Note: Empirically compares nine existing reward models, analyzes evaluator bias and reward hacking, and studies calibrated reward ensembles.
-
Can We Predict The Human Preference For Text-to-Image Content Prior To Generation And Is It Even Useful To Do So? (2026-06-03)
-
DynT2I-Eval: A Dynamic Evaluation Framework for Text-to-Image Models (2026-05-07)
-
Guidance Matters: Rethinking the Evaluation Pitfall for Text-to-Image Generation (2026-02-26, ICLR 2026)
-
GenArena: How Can We Achieve Human-Aligned Evaluation for Visual Generation Tasks? (2026-02-05)
-
AMVICC: A Novel Benchmark for Cross-Modal Failure Mode Profiling for VLMs and IGMs (2026-01-20)
-
FineGRAIN: Evaluating Failure Modes of Text-to-Image Models with Vision Language Model Judges (2025-12-01)
-
Evaluating the Evaluators: Metrics for Compositional Text-to-Image Generation (2025-09-25)
-
Evaluating Compositional Generalisation in VLMs and Diffusion Models (2025-08-28, IWCS 2025)
-
ConceptMix++: Leveling the Playing Field in Text-to-Image Benchmarking via Iterative Prompt Optimization (2025-07-04)
-
MINOS: A Multimodal Evaluation Model for Bidirectional Generation Between Image and Text (2025-06-03)
-
CROC: Evaluating and Training T2I Metrics with Pseudo- and Human-Labeled Contrastive Robustness Checks (2025-05-16)
-
K-Sort Eval: Efficient Preference Evaluation for Visual Generation via Corrected VLM-as-a-Judge (2026-02-10, ICLR 2026)
-
A Pragmatic Note on Evaluating Generative Models with Fréchet Inception Distance for Retinal Image Synthesis (2025-02-24)
-
SLVMEval: Synthetic Meta Evaluation Benchmark for Text-to-Long Video Generation (2026-03-31)
-
Do Existing Testing Tools Really Uncover Gender Bias in Text-to-Image Models? (2025-01-27)
-
What Makes a Good Generated Image? Investigating Human and Multimodal LLM Image Preference Alignment (2025-09-16)
-
GAIA: Rethinking Action Quality Assessment for AI-Generated Videos (2024-06-10)
-
Who Evaluates the Evaluations? Objectively Scoring Text-to-Image Prompt Coherence Metrics with T2IScoreScore (TS2) (2024-04-05)
-
Instruction-based Image Editing: A Survey on Data, Models, Evaluation, and Applications (2026-07-28)
-
Operationalizing Fairness in Text-to-Image Models: A Survey of Bias, Fairness Audits and Mitigation Strategies (2026-04-15, ICLR 2026 AFAA Workshop)
-
A Survey of Body and Face Motion: Datasets, Performance Evaluation Metrics and Generative Techniques (2025-12-09)
-
Right Looks, Wrong Reasons: Compositional Fidelity in Text-to-Image Generation (2025-11-13, AAAI 2026)
-
SoK: Can Synthetic Images Replace Real Data? A Survey of Utility and Privacy of Synthetic Image Generation (2025-06-24, USENIX Security 2025)
-
Alignment and Safety of Diffusion Models via Reinforcement Learning and Reward Modeling: A Survey (2025-05-23)
-
Unveiling Deep Shadows: A Survey and Benchmark on Image and Video Shadow Detection, Removal, and Generation in the Deep Learning Era (2024-09-03, IJCV)
-
Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey (2025-03-23)
-
Survey of Bias In Text-to-Image Generation: Definition, Evaluation, and Mitigation (2024-05-01)
-
Motion Generation: A Survey of Generative Approaches and Benchmarks (2025-07-07)
-
A Survey of Automatic Evaluation Methods on Text, Visual and Speech Generations (2025-06-06)
-
Survey of Video Diffusion Models: Foundations, Implementations, and Applications (2025-04-22)
-
A Survey on Quality Metrics for Text-to-Image Generation (2024-03-18)
-
A Survey of AI-Generated Video Evaluation (2024-10-24)
-
A Survey of Multimodal-Guided Image Editing with Text-to-Image Diffusion Models (2024-06-20)
-
From Sora What We Can See: A Survey of Text-to-Video Generation (2024-05-17)
Note: Refer to Section 3.4 for Evaluation Datasets and Metrics
-
A Survey on Personalized Content Synthesis with Diffusion Models (2024-05-09)
Note: Refere to Section 6 for Evaluation Datasets and Metrics
-
A Survey on Long Video Generation: Challenges, Methods, and Prospects (2024-03-25)
Note: Refer to table 2 for evaluation metrics for long video generation
-
Sora as an AGI World Model? A Complete Survey on Text-to-Video Generation (2024-03-08)
-
State of the Art on Diffusion Models for Visual Computing (2023-10-11)
Note: Refer to Section 9 for Metrics
-
AI-Generated Images as Data Source: The Dawn of Synthetic Era (2023-10-03)
Note: Refer to Section 4.2 for Evaluation Metrics
-
A Survey on Video Diffusion Models (2023-10-06)
Note: Refer to Section 2.3 for Evaluation Datasets and Metrics
-
Text-to-image Diffusion Models in Generative AI: A Survey (2023-03-14)
Note: Refer to Section 5 for Evaulation from Techincal and Ethical Perspective
-
Image synthesis: a review of methods, datasets, evaluation metrics, and future outlook (2023-02-28)
Note: Refer to section 4 for evaluation metrics
-
Adversarial Text-to-Image Synthesis: A Review (2021-01-25)
Note: Refer to Section 5 for Evaluation of T2I Models
-
Generative Adversarial Networks (GANs): An Overview of Theoretical Model, Evaluation Metrics, and Recent Developments (2020-05-27)
Note: Refer to section 2.2 for Evaluation Metrics
-
What comprises a good talking-head video generation?: A Survey and Benchmark (2020-05-07)
-
A Survey and Taxonomy of Adversarial Neural Networks for Text-to-Image Synthesis (2019-10-21)
Note: Refer to Section 5 for Benchmark and Evaluation
-
Recent Progress on Generative Adversarial Networks (GANs): A Survey (2019-03-14)
Note: Refer to section 5 for Evaluation Metrics
-
Video Description: A Survey of Methods, Datasets and Evaluation Metrics (2018-06-01)
Note: Refer to section 5 for Evaluation Metrics
-
Early Failure Detection and Intervention in Video Diffusion Models (2026-03-15)
-
Error Analyses of Auto-Regressive Video Diffusion Models: A Unified Framework (2025-03-12)
-
The FID Lottery: Quantifying Hidden Randomness in Generative-Model Evaluation (2026-06-18)
-
Multimodal Image Colorization: Quantifying the Impact of Text-Conditioned Guidance on Grayscale-to-Color Translation (2026-06-16)
-
Evaluating Reasoning Fidelity in Visual Text Generation (2026-06-03)
-
Train, Test, Re-evaluate: Schedule-Sensitive Evaluation of Generative Data for Hand Detection (2026-06-01)
-
Rethinking FID Through the Geometry of the Reference Dataset (2026-05-28)
-
Skill-Aligned Annotation for Reliable Evaluation in Text-to-Image Generation (2026-05-13)
-
Prompt-to-Gesture: Measuring the Capabilities of Image-to-Video Deictic Gesture Generation (2026-04-16, FG 2026)
-
On Semiotic-Grounded Interpretive Evaluation of Generative Art (2026-04-09)
Note: Introduces SemJudge for symbolic and indexical meaning in generated art.
-
How Do Inpainting Artifacts Propagate to Language? (2026-02-24)
-
Is Visual Realism Enough? Evaluating Gait Biometric Fidelity in Generative AI Human Animation (2025-12-22)
-
An evaluation of SVBRDF Prediction from Generative Image Models for Appearance Modeling of 3D Scenes (2025-12-15)
-
Cost Savings from Automatic Quality Assessment of Generated Images (2025-10-17)
-
FailureAtlas:Mapping the Failure Landscape of T2I Models via Active Exploration (2025-09-26)
-
Hallucination as an Upper Bound: A New Perspective on Text-to-Image Evaluation (2025-09-25)
-
The Iconicity of the Generated Image (2025-09-19)
-
Perceptual Evaluation of GANs and Diffusion Models for Generating X-rays (2025-08-10)
-
Quality Assessment and Distortion-aware Saliency Prediction for AI-Generated Omnidirectional Images (2025-06-27)
-
How to Train your Text-to-Image Model: Evaluating Design Choices for Synthetic Training Captions (2025-06-20)
-
Re-Thinking the Automatic Evaluation of Image-Text Alignment in Text-to-Image Models (2025-06-10)
-
RAISE: Realness Assessment for Image Synthesis and Evaluation (2025-05-25)
Note: Introduces human realness annotations and realness-prediction baselines.
-
Evaluating and Improving the Effectiveness of Synthetic Chest X-Rays for Medical Image Analysis (2024-11-27)
-
The Intricate Dance of Prompt Complexity, Quality, Diversity, and Consistency in T2I Models (2025-10-22)
-
A-Bench: Are LMMs Masters at Evaluating AI-generated Images? (2024-06-05)
-
On the Content Bias in Fréchet Video Distance (2024-04-18, CVPR 2024)
-
On the Evaluation of Generative Models in Distributed Learning Tasks (2023-10-18)
-
Recent Advances in Text-to-Image Synthesis: Approaches, Datasets and Future Research Prospects (2023-08-18)
-
Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models (2023-06-07, NeurIPS 2023)
-
Toward Verifiable and Reproducible Human Evaluation for Text-to-Image Generation (2023-04-04, CVPR 2023)
-
Revisiting the Evaluation of Image Synthesis with GANs (2023-04-04)
-
A Study on the Evaluation of Generative Models (2022-06-22)
-
REALY: Rethinking the Evaluation of 3D Face Reconstruction (2022-03-18)
-
On Aliased Resizing and Surprising Subtleties in GAN Evaluation (2021-04-22)
-
Pros and Cons of GAN Evaluation Measures: New Developments (2021-03-17)
-
On the Robustness of Quality Measures for GANs (2022-01-31, ECCV 2022)
-
Multimodal Image Synthesis and Editing: The Generative AI Era (2021-12-27)
-
An Analysis of Text-to-Image Synthesis (2021-05-25)
-
Pros and Cons of GAN Evaluation Measures (2018-02-09)
-
A Note on the Inception Score (2018-01-06)
-
An empirical study on evaluation metrics of generative adversarial networks (2018-06-19)
-
Are GANs Created Equal? A Large-Scale Study (2017-11-28, NeurIPS 2018)
-
A note on the evaluation of generative models (2015-11-05)
-
Appeal prediction for AI up-scaled Images (2024-12-12)
-
The First Controllable Bokeh Rendering Challenge at NTIRE 2026 (2026-05-06, NTIRE 2026)
-
NTIRE 2024 Quality Assessment of AI-Generated Content Challenge (2024-04-25)
-
CVPR 2023 Text Guided Video Editing Competition (2023-10-24)
-
Stanford Course: CS236 "Deep Generative Models" - Lecture 15 "Evaluation of Generative Models" [slides]
-
ImageBench - Public benchmark and comparison site for text-to-image models with side-by-side outputs, pass-rate summaries, and methodology pages.