PLLuM-Align: Polish Preference Dataset for Large Language Model Alignment (2025.emnlp-main)
Copied to clipboard
Karolina Seweryn, Anna Kołos, Agnieszka Karlińska, Katarzyna Lorenc, Katarzyna Dziewulska, Maciej Chrabaszcz, Aleksandra Krasnodebska, Paula Betscher, Zofia Cieślińska, Katarzyna Kowol, Julia Moska, Dawid Motyka, Paweł Walkowiak, Bartosz Żuk, Arkadiusz Janz
| Challenge: | Large language models generate preferred responses while avoiding harmful or inappropriate outputs, despite their ability to generate cross-language transferability. |
| Approach: | They introduce the first Polish preference dataset PLLuM-Align, created entirely through human annotation to reflect Polish language and cultural nuances. |
| Outcome: | The proposed dataset lays the groundwork for more aligned Polish LLMs and contributes to the broader goal of multilingual alignment in underrepresented languages. |
Similar Papers
CM-Align: Consistency-based Multilingual Alignment for Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Current large language models (LLMs) show a significant performance gap in alignment between English and other languages. |
| Approach: | They propose a consistency-based method to construct high-quality multilingual preference data for improving multilingual alignment. |
| Outcome: | The proposed method is based on three LLMs and three common tasks and shows that it performs better than current methods. |
Alignment Data Map for Efficient Preference Data Selection and Diagnosis (2026.findings-acl)
Copied to clipboard
| Challenge: | constructing high-quality preference datasets faces scalability challenges due to prohibitive cost and complexity of human annotation. |
| Approach: | They propose a tool to identify and select effective preference data by LLM-as-a-judge, explicit reward model, and reference-based approaches. |
| Outcome: | The proposed tool reduces annotation costs while preserving alignment effectiveness. |
Pedagogical Alignment of Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are often used without pedagogical fine-tuning and provide immediate answers rather than guiding students through the problem-solving process. |
| Approach: | They propose a method for constructing large-scale preference datasets using synthetic data generation techniques that eliminates the need for manual annotation. |
| Outcome: | The proposed methods outperform standard supervised fine-tuning (SFT) and improve alignment accuracy by 13.1% and 8.7% respectively. |
A Survey on Training-free Alignment of Large Language Models (2025.findings-emnlp)
Copied to clipboard
Birong Pan, Yongqi Li, Weiyu Zhang, Wenpeng Lu, Mayi Xu, Shen Zhou, Yuanyuan Zhu, Ming Zhong, Tieyun Qian
| Challenge: | a survey of large language models (LLMs) aims to ensure outputs adhere to human values, ethical standards, and legal norms. |
| Approach: | They present the first systematic review of TF alignment methods . they categorize them by stages of pre-decoding, in-decoder and post-decoration . |
| Outcome: | The proposed methods are based on training-free (TF) alignment techniques . they are able to be used in open-source and closed-source environments without retraining . |
MetaAlign: Align Large Language Models with Diverse Preferences during Inference Time (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to align large language models with human preferences often result in a static alignment that cannot account for the diversity of human preferences in practical applications. |
| Approach: | They propose a method to help large language models dynamically align with various explicit or implicit preferences specified at inference time. |
| Outcome: | The proposed method can help LLMs dynamically align with various explicit or implicit preferences specified at the inference stage, validating the feasibility of MetaAlign. |
Align2LLaVA: Cascaded Human and Large Language Model Preference Alignment for Multi-modal Instruction Curation (2025.findings-acl)
Copied to clipboard
Hongzhe Huang, Jiang Liu, Zhewen Yu, Li Cai, Dian Jiao, Wenqiao Zhang, Siliang Tang, Juncheng Li, Hao Jiang, Haoyuan Li, Yueting Zhuang
| Challenge: | Recent advances in Multi-modal Large Language Models (MLLMs) introduce significant variability in data quality. |
| Approach: | They propose to use human and LLM preference alignment to compress large corpus of machine-generated multimodal instructions into a compact and high-quality form. |
| Outcome: | The proposed algorithm outperforms LLaVA-series models in MLLM benchmarks by 90% . it uses human and LLM preference alignment to compress a large dataset . |
Multilingual Refusal Alignment for Safer Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly used globally, but their safety and alignment can vary unpredictably between languages. |
| Approach: | They propose a multilingual refusal alignment dataset to investigate whether alignment transfers cross-lingually and how language consistency is preserved during training. |
| Outcome: | The proposed model can be trained on multilingual datasets without affecting general performance. |
From 1,000,000 Users to Every User: Scaling Up Personalized Preference for User-level Alignment (2026.acl-long)
Copied to clipboard
| Challenge: | Current approaches to align large language models assume uniform human preferences, overlooking the diversity inherent in human populations. |
| Approach: | They propose a framework for scalable personalized alignment of large language models . they establish a preference space characterizing psychological and behavioral dimensions . |
| Outcome: | The proposed framework improves on existing methods with an average of 17.06% accuracy gain across four benchmarks and a strong adaptation capability to novel preferences. |
AlignBench: Benchmarking Chinese Alignment of Large Language Models (2024.acl-long)
Copied to clipboard
Xiao Liu, Xuanyu Lei, Shengyuan Wang, Yue Huang, Andrew Feng, Bosi Wen, Jiale Cheng, Pei Ke, Yifan Xu, Weng Lam Tam, Xiaohan Zhang, Lichao Sun, Xiaotao Gu, Hongning Wang, Jing Zhang, Minlie Huang, Yuxiao Dong, Jie Tang
| Challenge: | Effective evaluation of alignment for emerging Chinese LLMs is still significantly lacking, calling for real-scenario grounded, open-ended, challenging and automatic evaluations tailored for alignment. |
| Approach: | They propose a multi-dimensional benchmark for evaluating LLMs’ alignment in Chinese with 8 main categories, 683 real-scenario rooted queries and corresponding human verified references. |
| Outcome: | The benchmark uses a human-in-the-loop data curation pipeline, 683 real-scenario rooted queries and human verified references. |
AutoAlign: Get Your LLM Aligned with Minimal Annotations (2025.acl-demo)
Copied to clipboard
Xinyu Lu, Dong Xu, Chunkang Zhang, Xinyan Guan, Junxiang Wang, Qingyu Zhang, Pengbo Wang, Yingzhi Mao, Hao Xiang, Xueru Wen, Zichao Li, Yaojie Lu, Hongyu Lin, Le Sun, Xianpei Han
| Challenge: | Automated Alignment (ALM) is a set of algorithms designed to align Large Language Models (LLMs) with human intentions and values while minimizing manual intervention. |
| Approach: | They propose an open-source toolkit that integrates mainstream automated algorithms through a consistent interface and an accessible workflow supporting one-click execution for prompt synthesis and automatic alignment signal construction. |
| Outcome: | The proposed framework enables easy reproduction of existing results through extensive benchmarks and facilitates the development of novel approaches via modular components. |