Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
The official repo for paper, LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods.
| Date | Stars |
|---|---|
| 2026-07-31 | 600 |
| 2026-08-06 | 605 |
Today
+5 stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<div align="center"> <img src="./img/LLMs-as-Judges.png" style="width: 40%;height: 40%"> </div> # 🚀 Awesome-LLMs-as-Judges [](https://github.com/sponsors)      # 🌟 About This Repo With the rapid development of LLMs, LLM-as-a-Judge has garnered widespread attention in both academia and industry. LLM judges are not only capable of serving as flexible evaluators in various fields such as text generation, question answering, and dialogue systems, but also facilitate the self-evolution and performance improvement of models. This repository aims to provide a one-stop resource for developers, researchers, and practitioners, helping them explore how to effectively leverage LLMs-as-Judges technology. This repo include the papers discussed in our latest survey paper: 📝[LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods.](https://arxiv.org/abs/2412.05579) We will continuously track the latest developments in LLMs-as-Judges and regularly update the repository with the newest related papers. If you find this repository helpful, please give us a ⭐! If you notice any work we've missed, please feel free to submit a pull request or contact us at email [email protected]. We will update the repository and our paper. Welcome to discuss and contribute! # 📚 Daily Papers on LLMs-as-Judges [Daily Papers on LLMs-as-Judges](https://github.com/Deriq-Qian-Dong/arXivReporter/tree/main/LLMs-as-Judges) includes the latest paper titles and abstracts related to LLMs-as-Judges on arXiv, with information available in both English and Chinese. # ⚡️ Update 🔥🔥 News: 2024/12/20: We have updated [Daily Papers on LLMs-as-Judges](https://github.com/Deriq-Qian-Dong/arXivReporter/tree/main/LLMs-as-Judges), which automatically retrieves and updates daily papers from arXiv related to LLMs-as-Judges. 🔥🔥 News: 2024/12/14: We compiled papers related to LLMs-as-Judges presented at [NeurIPS 2024](NeurIPS.md). 🔥🔥 News: 2024/12/10: We released the first version of the [full paper](https://arxiv.org/abs/2412.05579). 🔥🔥 News: 2024/11/10: We completed the foundational work for the project and structured the framework. # 🌳 Contents - [🚀 Awesome-LLMs-as-Judges](#-awesome-llms-as-judges) - [🌟 About This Repo](#-about-this-repo) - [📚 Daily arXiv Papers on LLMs-as-Judges](#-daily-arxiv-papers-on-llms-as-judges) - [⚡️ Update](#️-update) - [🌳 Contents](#-contents) - [📖 Cite Our Work](#-cite-our-work) - [📚 Overview of Awesome-LLMs-as-Judges](#-overview-of-awesome-llms-as-judges) - [📑 PaperList](#-paperlist) - [1. Functionality](#1-functionality) - [1.1 Performance Evaluation](#11-performance-evaluation) - [1.1.1 Responses Evaluation](#111-responses-evaluation) - [1.1.2 Model Evaluation](#112-model-evaluation) - [1.2 Model Enhancement](#12-model-enhancement) - [1.2.1 Reward Modeling During Training](#121-reward-modeling-during-training) - [1.2.2 Acting as Verifier During Inference](#122-acting-as-verifier-during-inference) - [1.2.3 Feedback for Refinement](#123-feedback-for-refinement) - [1.3 Data Collection](#13-data-collection) - [1.3.1 Data Annotation](#131-data-annotation) - [1.3.2 Data Synthesize](#132-data-synthesize) - [2. METHODOLOGY](#2-methodology) - [2.1 Single-LLM System](#21-single-llm-system) - [2.1.1 Prompt-based](#211-prompt-based) - [2.1.1.1 In-Context Learning](#2111-in-context-learning) - [2.1.1.2 Step-by-step](#2112-step-by-step) - [2.1.1.3 Definition Augmentation](#2113-definition-augmentation) - [2.1.1.4 Multi-turn Optimization](#21
Excerpt of 54,981 characters
Read on GitHub
36
14
12
9
Ikko Eltociear Ashimine · Japan
1
Mohammad Ghiasvand Mohammadkhani
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:dd02b4f69a275695, llm:Repository description: 'The official repo for paper, LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods.' (survey/paper repository about LLM-based evaluation methods).
matched fp:dd02b4f69a275695, llm:Repository description: 'The official repo for paper, LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods.' (survey/paper repository about LLM-based evaluation methods).