Learning Personalized Alignment for Evaluating Open-ended Text Generation

Challenge:	Traditional evaluation metrics rely heavily on lexical similarity with human-written references, showing poor correlation with human judgments and failing to account for alignment with the diversity of human preferences.
Approach:	They propose an interpretable evaluation framework that evaluates alignment with specific human preferences by providing detailed comments and fine-grained scoring.
Outcome:	The proposed framework outperforms GPT-4 in Kendall correlation and accuracy with zero-shot reviewers.

Similar Papers

Overview

Challenge

Approach

Outcome

A Survey on Personalized Alignment—The Missing Piece for Large Language Models in Real-World Applications (2025.findings-acl)

Copied to clipboard

Jian Guan, Junfei Wu, Jia-Nan Li, Chuanqi Cheng, Wei Wu

Semantic Text Processing

Challenge:	Large Language Models (LLMs) have demonstrated remarkable capabilities, yet their transition to real-world applications reveals a critical limitation: the inability to adapt to individual preferences while maintaining alignment with universal human values.
Approach:	They propose a framework that enables LLMs to adapt their behavior within ethical boundaries based on individual preferences.
Outcome:	The proposed framework analyzes implementation approaches and evaluates their effectiveness across various scenarios.

ExPerT: Effective and Explainable Evaluation of Personalized Long-Form Text Generation (2025.findings-acl)

Copied to clipboard

Alireza Salemi, Julian Killingback, Hamed Zamani

Explainability & Interpretability in NLP

Challenge:	Evaluating personalized text generated by large language models is challenging, as only the LLM user, i.e. prompt author, can reliably assess the output.
Approach:	They propose an explainable reference-based evaluation framework that leverages an LLM to extract atomic aspects and their evidences from the generated and reference texts, match the aspects, and evaluate their alignment based on content and writing style.
Outcome:	The proposed framework achieves a 7.2% improvement in alignment with human judgments compared to the state-of-the-art evaluation methods.

Benchmarking and Improving LLM Robustness for Personalized Generation (2025.findings-emnlp)

Copied to clipboard

Chimaobi Okite, Naihao Deng, Kiran Bodipati, Huaidian Hou, Joyce Chai, Rada Mihalcea

Robustness in NLP

Challenge:	Existing evaluations focus on whether a model’s responses align with a user’s preferences, but factuality is an important yet overlooked dimension.
Approach:	They propose a scalable framework for evaluating robustness of large language models in personalization and a new dataset, PERGData.
Outcome:	The proposed framework improves robustness by 25% across models.

Aligning LLMs with Individual Preferences via Interaction (2025.coling-main)

Copied to clipboard

Shujin Wu, Yi R. Fung, Cheng Qian, Jeonghwan Kim, Dilek Hakkani-Tur, Heng Ji

Challenge:	Existing studies on LLMs alignment focus on generalizing their behavior to generalized values such as helpfulness, harmlessness, and honesty.
Approach:	They train large language models to "interact to align" to implicitly infer user preferences . they use a multi-turn preference dataset to generate a personalized alignment .
Outcome:	The proposed method enables dynamic, personalized alignment via interaction with a multi-turn preference dataset.

Disentangling Preference Representation and Text Generation for Efficient Individual Preference Alignment (2025.coling-main)

Copied to clipboard

Jianfei Zhang, Jun Bai, Bei Li, Yanmeng Wang, Rumei Li, Chenghua Lin, Wenge Rong

Semantic Text Processing

Challenge:	Human values are inherently diverse, making it insufficient to align LLMs solely with general preferences.
Approach:	They propose a flexible paradigm for individual preference alignment that disentangles preference representation from text generation in LLMs.
Outcome:	The proposed method produces aligned quality and better than PEFT-based methods while reducing training time for each new individual preference by 80% to 90%.

Rethinking the Evaluation of Alignment Methods: Insights into Diversity, Generalisation, and Safety (2026.eacl-srw)

Copied to clipboard

Denis Janiak, Julia Moska, Dawid Motyka, Karolina Seweryn, Paweł Walkowiak, Bartosz Żuk, Arkadiusz Janz

Challenge:	Existing studies focus on individual techniques or specific dimensions, lacking a holistic assessment of the inherent trade-offs.
Approach:	They propose a framework that compares LLM alignment methods across five axes . they use a validated LLM-as-judge prompt to compare the results .
Outcome:	The proposed framework compares LLM alignment methods across factuality, safety, conciseness, proactivity, diversity and safety axes . it provides insights into trade-offs of common alignment methods, guiding the development of more balanced and reliable LLMs.

Persona-judge: Personalized Alignment of Large Language Models via Token-level Self-judgment (2025.findings-acl)

Copied to clipboard

Xiaotian Zhang, Ruizhe Chen, Yang Feng, Zuozhu Liu

Language Models

Challenge:	Existing methods for aligning language models with human preferences rely on reward signals and additional annotated data, limiting their scalability and adaptability to diverse human values.
Approach:	They propose a discriminative paradigm that leverages the intrinsic preference judgment capabilities of the model to align language models with human preferences.
Outcome:	The proposed model is scalable and efficient, paving the way for more adaptive personalized alignment.

Uncovering Factor-Level Preference to Improve Human-Model Alignment (2025.findings-emnlp)

Copied to clipboard

Juhyun Oh, Eunsu Kim, Jiseon Kim, Wenda Xu, Inha Cha, William Yang Wang, Alice Oh

Challenge:	Large language models exhibit tendencies that diverge from human preferences, such as favoring certain writing styles or producing overly verbose outputs.
Approach:	They propose a framework to uncover and measure factor-level preference alignment of humans and large language models (LLMs)
Outcome:	The proposed framework uncovers and measures factor-level preference alignment of humans and large language models.

OpenGenAlign: A Preference Dataset and Benchmark for Trustworthy Reward Modeling in Open-Ended, Long-Context Generation (2026.findings-acl)

Copied to clipboard

Hanning Zhang, Juntong Song, Juno Zhu, Yuanhao Wu, Tong Zhang, Cheng Niu

Responsible & Trustworthy NLP

Challenge:	Existing reward models perform suboptimal on held-out benchmarks, resulting in poor quality outputs.
Approach:	They propose a framework and a high-quality dataset to evaluate reward models . they define four key metrics to assess generation quality and develop a pipeline to evaluate outputs .
Outcome:	The proposed framework and dataset improves hallucination-free, comprehensive, reliable, and efficient open-ended long-context generation.

WildFeedback: Aligning LLMs With In-situ User Interactions And Feedback (2026.acl-long)

Copied to clipboard

Taiwei Shi, Zhuoer Wang, Longqi Yang, Ying-Chun Lin, Zexue He, Mengting Wan, Pei Zhou, Sujay Kumar Jauhar, Sihao Chen, Shan Xia, Hongfei Zhang, Jieyu Zhao, Xiaofeng Xu, Xia Song, Jennifer Neville

Challenge:	Traditional alignment methods rely on human annotations and are subjective and misalignment with real-world user preferences.
Approach:	They propose a framework that leverages in-situ user feedback during conversations with LLMs to create preference datasets automatically.
Outcome:	The proposed framework identifies and classifies user feedback to LLM responses between conversation turns and creates examples of preferred and dispreferred responses according to user preferences.