Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
| Date | Stars |
|---|---|
| 2026-07-31 | 570 |
| 2026-08-01 | 570 |
| 2026-08-06 | 570 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Awesome-LLM-as-a-judge Survey
[](https://awesome.re)  
## Awesome-LLM-as-a-judge: A Survey
This repo include the papers discussed in our latest survey paper on Awesome-LLM-as-a-judge.
Our official websit: [Website](https://llm-as-a-judge.github.io/)
:book: Read the full paper here: [Paper Link](https://arxiv.org/abs/2411.16594)
## 🔔 News
- **`2025-5` We update our paper list and include papers of LLM-as-a-judge in April & May 2025!**
- **`2025-4` We update our paper list and include papers of LLM-as-a-judge in March 2025!**
- **`2025-3` Want to learn more about risks and safety problems of using LLM-based annotation? Check out our new paper list on [AI supervision risk](https://github.com/David-Li0406/AI-Supervision-Risk)!**
- **`2025-3` We update our paper list and include papers of LLM-as-a-judge in February 2025, together with papers about thinking LLM as a judge!**
- **`2025-2` We update our paper list and include papers of LLM-as-a-judge in January 2025!**
- **`2025-2` Check our new paper about [preference leakage](https://arxiv.org/abs/2502.01534) in LLM-as-a-judge!**
- **`2024-12` Also check our paper list and survey on [LLM-based data annotation and synthesis](https://github.com/Zhen-Tan-dmml/LLM4Annotation)!**
- **`2024-12` We update our paper list and include papers of LLM-as-a-judge in December 2024!**
- **`2024-12` We update the slides, talk and report of our paper, check them in our [Website](https://llm-as-a-judge.github.io/)!**
## Reference
If our survey is useful for your research, please kindly cite our [paper](https://arxiv.org/abs/2411.16594):
```
@article{li2024llmasajudge,
title = {From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge},
author = {Dawei Li and Bohan Jiang and Liangjie Huang and Alimohammad Beigi and Chengshuai Zhao and Zhen Tan and Amrita Bhattacharjee and Yuxuan Jiang and Canyu Chen and Tianhao Wu and Kai Shu and Lu Cheng and Huan Liu},
year = {2024},
journal = {arXiv preprint arXiv: 2411.16594}
}
```
## Overview of Awesome-LLM-as-a-judge:
## Table of Content (ToC)
- [Awesome-LLM-as-a-judge Survey](#awesome-llm-as-a-judge-survey)
- [Awesome-LLM-as-a-judge: A Survey](#awesome-llm-as-a-judge-a-survey)
- [🔔 News](#-news)
- [Reference](#reference)
- [Overview of Awesome-LLM-as-a-judge:](#overview-of-awesome-llm-as-a-judge)
- [Table of Content (ToC)](#table-of-content-toc)
- [Thinking LLM-as-a-judge](#thinking-llm-as-a-judge)
- [Papers](#papers)
- [Update](#update)
- [04/2025 \& 05/2025](#042025--052025)
- [03/2025](#032025)
- [02/2025](#022025)
- [01/2025](#012025)
- [12/2024](#122024)
- [1 Attributes](#1-attributes)
- [1.1 Helpfulness](#11-helpfulness)
- [1.2 Harmlessness](#12-harmlessness)
- [1.3 Reliability](#13-reliability)
- [1.4 Relevance](#14-relevance)
- [1.5 Feasibility](#15-feasibility)
- [1.6 Overall Quality](#16-overall-quality)
- [2 Methodology](#2-methodology)
- [2.1 Tuning](#21-tuning)
- [Data Source](#data-source)
- [Manually-labeled](#manually-labeled)
- [Synthetic Feedback](#synthetic-feedback)
- [Tuning Techiniques](#tuning-techiniques)
- [Supervised Fine-Tuning](#supervised-fine-tuning)
- [Preference Learning](#preference-learning)
- [2.2 Prompting](#22-prompting)
- [Swapping Operation](#swapping-operation)
- [Rule Augmentation](#rule-augmentation)
- [Multi-Agent Collaboration](#multi-agent-collaboration)
- [Demonstration](#demonstration)
- [Multi-Turn Interaction](#multi-turn-interaction)
- [Comparison Acceleration](#comparison-acceleration)
- [3 Application](#3-application)
- [3.1 Evaluation](#31-evaluation)
- [3.2 Alignment](#32-alignment)
- [3.3 Retrieval](Excerpt of 135,143 characters
Read on GitHub
23
21
5
Suhani Nagpal
2
Mohammad Ghiasvand Mohammadkhani
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:6086807c97760820, name:llm-as-a-judge