| Challenge: | Recent studies focus on generative judges, but only on their judge ability. |
| Approach: | They propose a method that leverages the generative and reasoning capabilities of large language models to evaluate LLM responses across diverse scenarios, providing accurate preference signals. |
| Outcome: | The proposed model performs on RewardBench with only 2% to 40% of the data required by other training frameworks. |
Similar Papers
Direct Judgement Preference Optimization (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing judge models are largely trained with supervised finetuning on small data scales to perform limited types of evaluation tasks, limiting generalization. |
| Approach: | They propose to train judge models at large data scales with direct preference optimization . they use four training tasks to form three types of preference pairs targeting different aspects of evaluation . |
| Outcome: | The proposed model outperforms GPT-4o and other similar models on 13 benchmarks. |
From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge (2025.emnlp-main)
Copied to clipboard
Dawei Li, Bohan Jiang, Liangjie Huang, Alimohammad Beigi, Chengshuai Zhao, Zhen Tan, Amrita Bhattacharjee, Yuxuan Jiang, Canyu Chen, Tianhao Wu, Kai Shu, Lu Cheng, Huan Liu
| Challenge: | Recent advances in Large Language Models (LLMs) inspire the "LLM-as-a-judge" paradigm . traditional methods of assessment and evaluation fail in dynamic and open-ended scenarios . |
| Approach: | They propose a paradigm where LLMs are leveraged to perform scoring, ranking, or selection for machine learning evaluation scenarios. |
| Outcome: | The proposed model-based judgment and evaluation paradigms are based on large language models and are compared to the current model-driven evaluation paradigm. |
Assistant-Guided Mitigation of Teacher Preference Bias in LLM-as-a-Judge (2025.findings-emnlp)
Copied to clipboard
| Challenge: | LLM-as-a-Judge uses large language models to evaluate the quality of LLM generated responses, but training proxy judge models using evaluation data generated by powerful teacher models introduces a critical yet previously overlooked issue: teacher preference bias. |
| Approach: | They propose a new setting that incorporates an additional assistant model, which is not biased toward the teacher model’s responses, to complement the training data. |
| Outcome: | The proposed model reduces teacher preference bias while maintaining strong performance across six evaluation benchmarks. |
Current Advances in LLM Reasoning (2026.acl-tutorials)
Copied to clipboard
| Challenge: | This tutorial examines comprehensive evaluation strategies to assess the reasoning abilities of large language models (LLMs) advanced inference time methods and post-training methods that aim to make LLMs think more like humans are discussed in this tutorial. |
| Approach: | This tutorial explores comprehensive evaluation strategies to assess the reasoning abilities of large language models (LLMs) and discusses two types of methods to improve models’ reasoning: advanced inference time methods, structured and self-improvement inference methods, and post-training methods, such as RLHF, DPO, and GRPO. |
| Outcome: | This tutorial examines evaluation strategies to assess the reasoning abilities of large language models and discusses two types of methods to improve models’ reasoning. |
An Empirical Study of LLM-as-a-Judge for LLM Evaluation: Fine-tuned Judge Model is not a General Substitute for GPT-4 (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have fine-tuned judge models based on open-source LLMs to evaluate the quality of other LLM. |
| Approach: | They propose to use open-source LLMs to evaluate Large Language Models (LLMs) their empirical results show that the models underperform GPT-4 in several dimensions . |
| Outcome: | The proposed models outperform GPT-4 on several dimensions including generalizability, fairness and adaptability. |
J4R: Learning to Judge with Equivalent Initial State Group Relative Policy Optimization (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly being used for reasoning intensive tasks. |
| Approach: | They propose an algorithm that trains judges to be robust to positional biases . they also propose a benchmark that evaluates judges in diverse reasoning settings . |
| Outcome: | The proposed algorithm outperforms GPT-4o and the next best small judge by 6.7% and 9% on ReasoningJudgeBench and JudgeBench. |
Improving LLM-as-a-Judge Inference with the Judgment Distribution (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Using language models to scalably approximate human preferences on text quality (LLM-as-a-judge) is a standard practice applicable to many tasks. |
| Approach: | They propose to use LLM judges to approximate human preferences on text quality by using distributional output over judgment tokens. |
| Outcome: | The proposed method outperforms taking the mode (i.e. greedy decoding) in all evaluation settings, and incorporating risk aversion improves performance. |
SpeechLLM-as-Judges: Towards General and Interpretable Speech Quality Evaluation (2026.acl-long)
Copied to clipboard
Hui Wang, Jinghua Zhao, Yifan Yang, Shujie Liu, Junyang Chen, Yanzhe Zhang, Shiwan Zhao, Jinyu Li, Jiaming Zhou, Haoqin Sun, Yan Lu, Yong Qin
| Challenge: | Existing methods for evaluating the perceptual quality of synthetic speech are limited due to the complexity of perceptual quality factors and the diversity of speech generation tasks. |
| Approach: | They propose a new paradigm for enabling large language models to conduct structured speech quality evaluation using a large-scale dataset. |
| Outcome: | The proposed model performs well across tasks and languages. |
Multi-Task Reinforcement Learning for Enhanced Multimodal LLM-as-a-Judge (2026.acl-industry)
Copied to clipboard
| Challenge: | Existing MLLMs are optimized for single-task scenarios and struggle to generalize to diverse contexts. |
| Approach: | They propose a framework that integrates multitask reinforcement learning and generalization capabilities of MLLMs to optimize the judge model across multiple tasks. |
| Outcome: | The proposed framework outperforms baseline models in judgment consistency and correlation with human preferences. |
Humans or LLMs as the Judge? A Study on Judgement Bias (2024.emnlp-main)
Copied to clipboard
| Challenge: | Proprietary models such as GPT-4, Claude, Gemini-Pro and others are being democratized to improve evaluations of LLMs. |
| Approach: | They propose a framework that is free from referencing groundtruth annotations for investigating **Misinformation Oversight Bias**, **Gender Bia**,**Authority Bia* and **Beauty Bia's** on LLM and human judges. |
| Outcome: | The proposed framework investigates **Misinformation Oversight Bias**, **Gender Bia**,**Authority Bia* and **Beauty Bia' on LLM and human judges. |