InstaJudge: Aligning Judgment Bias of LLM-as-Judge with Humans in Industry Applications

Challenge:	Automated evaluation using LLM-as-Judge is a viable alternative to human evaluation, but misalignment of judgment biases between humans and LLMs hinders its use in real-world applications.
Approach:	They propose an LLM-as-Judge library that improves alignments of judgment biases through automatic prompt optimization.
Outcome:	The proposed library outperforms existing LLM-as-Judge libraries by a large margin while being more cost efficient.

Similar Papers

Overview

Challenge

Approach

Outcome

SenseJudge: Human-Centric Preference-Driven Judgment Framework (2026.findings-acl)

Copied to clipboard

Rui Li, Junfeng Liu, Xiangwen Kong, Zhifang Sui

Challenge:	Existing judgment approaches rely on trained judgers using fixed preference data . existing judgment approaches neglect diverse user preferences and struggle to adapt to real-world human-AI dialogue scenarios.
Approach:	They propose a customizable judgment framework driven by human preferences and a diverse instruction following benchmark derived from real-world multi-turn interactions.
Outcome:	The proposed framework surpasses other judgment methods and models in two tasks, and achieves model ranking that aligns with real human sense.

SAJA: A Simple Approach to Judge Alignment for LLM-as-a-Judge (2026.acl-industry)

Copied to clipboard

Sneha Kola, Pankaj Kumar Sharma, Soumyadeep Dey, Bamdev Mishra, Mayur Datar

Language Models

Challenge:	Current approaches to evaluate text at scale require multiple calls and per-dataset prompt tuning.
Approach:	They propose a model-agnostic approach to evaluate judge alignment that uses a lightweight calibration head.
Outcome:	a new model with SAJA matches more complex systems across four evaluation paradigms . it outperforms uncalibrated models on MT-Bench pairwise preference and competitive performance on five classification benchmarks compared to uncalibred models .

Humans or LLMs as the Judge? A Study on Judgement Bias (2024.emnlp-main)

Copied to clipboard

Guiming Chen, Shunian Chen, Ziche Liu, Feng Jiang, Benyou Wang

Responsible & Trustworthy NLP

Challenge:	Proprietary models such as GPT-4, Claude, Gemini-Pro and others are being democratized to improve evaluations of LLMs.
Approach:	They propose a framework that is free from referencing groundtruth annotations for investigating Misinformation Oversight Bias, Gender Bia,*Authority Bia and Beauty Bia's on LLM and human judges.
Outcome:	The proposed framework investigates Misinformation Oversight Bias, Gender Bia,*Authority Bia and **Beauty Bia' on LLM and human judges.

LLM Bias Detection and Mitigation through the Lens of Desired Distributions (2025.emnlp-main)

Copied to clipboard

Ingroj Shrestha, Padmini Srinivasan

Language Models

Challenge:	Prior work on bias mitigation has focused on promoting social equality and demographic parity, but less attention has been given to aligning LLM’s outputs to desired distributions.
Approach:	They propose a weighted adaptive loss based fine-tuning method that aligns LLM’s gender–profession output distribution with the desired distribution while preserving language modeling capability.
Outcome:	The proposed method achieves near-complete mitigation under equality and 30–75% reduction under real-world settings.

Aligning Black-box Language Models with Human Judgments (2025.findings-naacl)

Copied to clipboard

Gerrit J.j. Van Den Burg, Gen Suzuki, Wei Liu, Murat Sensoy

Language Models

Challenge:	Large language models (LLMs) are increasingly used as automated judges to evaluate recommendation systems, search engines, and other subjective tasks.
Approach:	They propose a framework to align LLM judgments with individual human evaluators or their aggregated judgments without retraining or fine-tuning the LLM.
Outcome:	The proposed framework achieves 142% improvement in agreement across 29 tasks and exceeds inter-human agreement on four out of six tasks.

CalibraEval: Calibrating Prediction Distribution to Mitigate Selection Bias in LLMs-as-Judges (2025.acl-long)

Copied to clipboard

Haitao Li, Junjie Chen, Qingyao Ai, Zhumin Chu, Yujia Zhou, Qian Dong, Yiqun Liu

Robustness in NLP

Challenge:	Empirical evaluations of large language models demonstrate that they improve performance in a wide range of tasks.
Approach:	They propose a label-free method for mitigating selection bias during inference by reformulating debiasing as an optimization task.
Outcome:	The proposed method mitigates selection bias and improves performance compared to existing methods.

Assistant-Guided Mitigation of Teacher Preference Bias in LLM-as-a-Judge (2025.findings-emnlp)

Copied to clipboard

Zhuo Liu, Moxin Li, Xun Deng, Qifan Wang, Fuli Feng

Challenge:	LLM-as-a-Judge uses large language models to evaluate the quality of LLM generated responses, but training proxy judge models using evaluation data generated by powerful teacher models introduces a critical yet previously overlooked issue: teacher preference bias.
Approach:	They propose a new setting that incorporates an additional assistant model, which is not biased toward the teacher model’s responses, to complement the training data.
Outcome:	The proposed model reduces teacher preference bias while maintaining strong performance across six evaluation benchmarks.

Persona-judge: Personalized Alignment of Large Language Models via Token-level Self-judgment (2025.findings-acl)

Copied to clipboard

Xiaotian Zhang, Ruizhe Chen, Yang Feng, Zuozhu Liu

Language Models

Challenge:	Existing methods for aligning language models with human preferences rely on reward signals and additional annotated data, limiting their scalability and adaptability to diverse human values.
Approach:	They propose a discriminative paradigm that leverages the intrinsic preference judgment capabilities of the model to align language models with human preferences.
Outcome:	The proposed model is scalable and efficient, paving the way for more adaptive personalized alignment.

Improving LLM-as-a-Judge Inference with the Judgment Distribution (2025.findings-emnlp)

Copied to clipboard

Victor Wang, Michael JQ Zhang, Eunsol Choi

Challenge:	Using language models to scalably approximate human preferences on text quality (LLM-as-a-judge) is a standard practice applicable to many tasks.
Approach:	They propose to use LLM judges to approximate human preferences on text quality by using distributional output over judgment tokens.
Outcome:	The proposed method outperforms taking the mode (i.e. greedy decoding) in all evaluation settings, and incorporating risk aversion improves performance.

Justice in Judgment: Unveiling (Hidden) Bias in LLM-assisted Peer Reviews (2026.findings-acl)

Copied to clipboard

Sai Suresh Macharla Vasu, Ivaxi Sheth, Hui-Po Wang, Ruta Binkyte, Mario Fritz

Responsible & Trustworthy NLP

Challenge:	Existing studies show that large language models carry implicit biases across race, gender, and religion . prior studies documented such biase based on text generation and classification tasks .
Approach:	They investigate bias in large language models by controlling metadata on author metadata . authors found affiliation bias favoring authors from highly ranked institutions .
Outcome:	The proposed model favors authors from highly ranked institutions, the authors show . the model also favors author affiliations from highly-ranked institutions .