When Do Language Models Endorse Limitations on Human Rights Principles? (2026.findings-eacl)
Copied to clipboard
Keenan Samway, Miu Nicole Takagi, Rada Mihalcea, Bernhard Schölkopf, Ilias Chalkidis, Daniel Hershcovich, Zhijing Jin
| Challenge: | a recent study evaluated how large language models navigate trade-offs involving the Universal Declaration of Human Rights. |
| Approach: | They evaluate how large language models navigate trade-offs involving the Universal Declaration of Human Rights (UDHR) they use 1,152 synthetically generated scenarios across 24 rights articles and eight languages . |
| Outcome: | The proposed models accept limiting economic, social, and cultural rights more often than political and civil rights, the authors show . their models show significant cross-linguistic variation with elevated endorsement rates of rights-limiting actions in Chinese and Hindi compared to English or Romanian . |
Similar Papers
Safety of Large Language Models Beyond English: A Systematic Literature Review of Risks, Biases, and Safeguards (2026.eacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have a growing number of applications that generate harmful, biased, or unsafe content. |
| Approach: | They synthesize findings from recent studies that evaluate their robustness across languages . they highlight gaps in multilingual safety research and recommend future work . |
| Outcome: | The systematic review examines the multilingual safety of large language models in English . it identifies challenges such as dataset availability and evaluation biases . |
Large Human Language Models: A Need and the Challenges (2024.naacl-long)
Copied to clipboard
| Challenge: | a growing recognition of the importance of modeling human and social factors into human-centered NLP models . authors advocate for three positions toward creating large human language models based on psychological and behavioral sciences . |
| Approach: | et al. advocate for three positions toward creating large human language models . they argue that LM training should include the human context and recognize that people are more than their group . |
| Outcome: | a new study shows that learning language from linguistic signals alone is not adequate, according to a recent paper . authors advocate for three positions toward creating large human language models . a human-centered model should include the human context, and account for the dynamic nature of the human environment, they say . |
A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendations (2024.emnlp-main)
Copied to clipboard
Md Tahmid Rahman Laskar, Sawsan Alqahtani, M Saiful Bari, Mizanur Rahman, Mohammad Abdullah Matin Khan, Haidar Khan, Israt Jahan, Amran Bhuiyan, Chee Wei Tan, Md Rizwan Parvez, Enamul Hoque, Shafiq Joty, Jimmy Huang
| Challenge: | Large Language Models (LLMs) have gained significant attention due to their capabilities in performing diverse tasks across domains. |
| Approach: | They review the primary challenges and limitations causing inconsistencies in evaluations . early models could generate coherent text but limited to simple tasks . |
| Outcome: | The proposed evaluations are reproducible, reliable, and robust. |
Dissecting Human and LLM Preferences (2024.acl-long)
Copied to clipboard
| Challenge: | a recent study shows that human and Large Language Model preferences are important for model fine-tuning and evaluation. |
| Approach: | They dissect the preferences of human and 32 different Large Language Models to understand their quantitative composition. |
| Outcome: | The proposed model is compared with 32 different large language models using real-world user-model conversations. |
The Impossibility of Fair LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Existing frameworks for evaluating large language models do not extend to general-purpose AI contexts or are infeasible in practice. |
| Approach: | They analyze a variety of technical fairness frameworks to find inherent challenges . they find that each framework does not logically extend to the general-purpose AI context . |
| Outcome: | The proposed frameworks do not logically extend to the general-purpose AI context or are infeasible in practice due to large amounts of unstructured training data and potential combinations of human populations, use cases, and sensitive attributes. |
Large Language Models: The Need for Nuance in Current Debates and a Pragmatic Perspective on Understanding (2023.emnlp-main)
Copied to clipboard
| Challenge: | Current Large Language Models (LLMs) are unparalleled in their ability to generate grammatically correct, fluent text. |
| Approach: | They argue that LLMs only parrot statistical patterns in training data and that language learning in LLM cannot inform human language learning. |
| Outcome: | The proposed model can generate grammatically correct, fluent text without requiring human intervention. |
The Linguistic Connectivities Within Large Language Models (2025.findings-acl)
Copied to clipboard
Dan Wang, Boxi Cao, Ning Bian, Xuanang Chen, Yaojie Lu, Hongyu Lin, Jia Zheng, Le Sun, Shanshan Jiang, Bin Dong, Xianpei Han
| Challenge: | Recent studies have discovered notable disparities in their performance across different languages. |
| Approach: | They conduct a systematic investigation into the behaviors of large language models across 27 different languages on 3 different scenarios and reveals a Linguistic Map correlates with the richness of available resources and linguistic family relations. |
| Outcome: | The proposed model demonstrates that there are significant disparities in performance across languages across 27 different languages on 3 different scenarios. |
Context Limitations Make Neural Language Models More Human-Like (2022.emnlp-main)
Copied to clipboard
| Challenge: | Language models (LMs) have been used in cognitive modeling and engineering studies to simulate human cognitive load during reading. |
| Approach: | They propose to constrain LMs' context access to improve their simulation of human reading behavior by incorporating syntactic biases into their context access. |
| Outcome: | The proposed model improves the simulation of human reading behavior by incorporating syntactic biases into their context access. |
Systematic Biases in LLM Simulations of Debates (2024.emnlp-main)
Copied to clipboard
| Challenge: | Current research suggests that LLM-based agents become increasingly human-like in their performance, sparking interest in using these AI agents as substitutes for human participants in behavioral studies. |
| Approach: | They propose to use LLMs to simulate political debates on topics that are important aspects of people’s day-to-day lives and decision-making processes. |
| Outcome: | The proposed model can simulate political debates on topics that are important aspects of people’s day-to-day lives and decision-making processes. |
The LLM Effect: Are Humans Truly Using LLMs, or Are They Being Influenced By Them Instead? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models have shown capabilities close to human performance in various analytical tasks. |
| Approach: | They investigate the efficiency and accuracy of Large Language Models in specialized tasks . they integrate LLMs with expert annotators to observe the impact of LLM suggestions . |
| Outcome: | The proposed model improves task completion speed but introduces anchoring bias . the proposed model is not suitable for open-ended analysis, but is capable of handling specialized tasks. |