CORRECTNESS

Unsupervised Information Refinement Training of Large Language Models for Retrieval-Augmented Generation (2024.acl-long)

Challenge:	Recent advances in Large Language Models have demonstrated their proficiency in answering natural language queries.
Approach:	They propose a system that augments Large Language Models with domain-specific knowledge graphs . they evaluate a medical KG and use a KG-based retrieval approach to enhance factual correctness .
Outcome:	The proposed system surpasses a standalone LLM in accuracy and completeness on a medical KG dataset.

Copied to clipboard

Shicheng Xu, Liang Pang, Mo Yu, Fandong Meng, Huawei Shen, Xueqi Cheng, Jie Zhou

Position Paper: How Should We Responsibly Adopt LLMs in the Peer Review Process? (2026.findings-eacl)

Challenge:	Existing studies show that LLMs face challenges in effectively using retrieved information . authors propose a method that considers LLM as "Information Refiner"
Approach:	They propose a method that considers LLMs as "Information Refiners" they propose INFO-RAG, which is low-cost and general across various tasks .
Outcome:	The proposed method improves performance of LLaMA2 by 9.39% relative points . it is low-cost and general across various tasks, and is robust and in-context learning is possible .

Copied to clipboard

Juhwan Choi, JungMin Yun, Changhun Kim, YoungBin Kim

Knowledge-centered conversational agents with a drive to learn (2024.naacl-srw)

Challenge:	a recent paper criticizes the current use of Large Language Models (LLMs) for simple review text generation.
Approach:	They propose to use Large Language Models to support key aspects of the review process . they argue that this approach overlooks more meaningful applications of LLMs . authors argue that the increased reviewing burden per reviewer is a factor .
Outcome:	The proposed approach would support reproducibility, correctness and relevance of citations and ethics review flagging.

Copied to clipboard

Selene Baez Santamaria

Towards Understanding of Medical Randomized Controlled Trials by Conclusion Generation (D19-62)

Challenge:	Unlike traditional task-oriented dialogue agents, knowledgeable agents can autonomously determine what they know and do not know, what is the epistemic status of what they do not understand, and what they need to learn.
Approach:	They propose an adaptive conversational agent that assesses the quality of its knowledge and is driven to become more knowledgeable.
Outcome:	The proposed agent can learn effective policies to acquire the knowledge needed by assessing the efficiency of these capabilities during interaction.

Copied to clipboard

Alexander Te-Wei Shieh, Yung-Sung Chuang, Shang-Yu Su, Yun-Nung Chen

Extraction of Message Sequence Charts from Software Use-Case Descriptions (N19-2)

Challenge:	Using machine learning to interpret large amounts of data can be over-whelming for clinicians.
Approach:	They propose to use PubMed 200k RCT sentence classification dataset to generate RCT conclusion generation task.
Outcome:	The proposed model improves quality and correctness in generated conclusions compared to baseline model . the proposed model is not suitable for all RCTs, but it could be improved .

Copied to clipboard

Girish Palshikar, Nitin Ramrakhiyani, Sangameshwar Patil, Sachin Pawar, Swapnil Hingmire, Vasudeva Varma, Pushpak Bhattacharyya

Leveraging Product Catalog Patterns for Multilingual E-commerce Product Attribute Prediction (2025.emnlp-industry)

Challenge:	Software Requirement Specification documents provide natural language descriptions of the core functional requirements as a set of use-cases.
Approach:	They propose a linguistic knowledge-based approach to extract software requirements from use-cases using a textual representation of the core functional requirements.
Outcome:	The proposed method performs better than existing techniques and improves performance.

Copied to clipboard

Bryan Zhang, Suleiman A. Khan, SteCphan Walter

Multilinguality

Challenge:	E-commerce stores increasingly use Large Language Models to improve catalog data quality . a critical challenge is accurately predicting missing structured attribute values .
Approach:	They propose a retrieval-augmented system that leverages existing product catalog entries to guide LLM predictions for missing attributes.
Outcome:	The proposed system improves catalog data quality by 34% and accuracy by 0.8% . the proposed model can predict missing attributes in multilingual product catalogs .

Representation and Generation of Machine Learning Test Functions (2024.eacl-srw)

Copied to clipboard

Souha Hassine, Steven Wilson

Fluent Response Generation for Conversational Question Answering (2020.acl-main)

Challenge:	Large Language Models (LLMs) have been adopted for ML code generation but their implications are relatively unexplored.
Approach:	They examine the use of Large Language Models to extract representations of ML source code and tests to understand the semantic relationships between human-written tests and LLM-generated ones.
Outcome:	The proposed models can be used to extract representations of ML source code and tests and annotate them for usefulness, documentation, and correctness.

Copied to clipboard

Ashutosh Baheti, Alan Ritter, Kevin Small

Automatic Comment Generation for Chinese Student Narrative Essays (2022.emnlp-demos)

Challenge:	Question answering (QA) is an important aspect of open-domain conversational agents, garnering specific research focus in the conversational QA subtask.
Approach:	They propose a method for situating QA responses within a SEQ2SEQ NLG approach to generate fluent grammatical answer responses while maintaining correctness.
Outcome:	The proposed model outperforms baseline CoQA and QuAC models in generating conversational responses.

Copied to clipboard

Zhexin Zhang, Jian Guan, Guowei Xu, Yixiang Tian, Minlie Huang

GREEN: Generative Radiology Report Evaluation and Error Notation (2024.findings-emnlp)

Challenge:	Existing studies focus on giving discrete scores for holistic quality or distinct traits, but real-world teachers usually provide detailed comments in natural language, which are more informative than single scores.
Approach:	They propose a model which generates comments for specified segments from given student narrative essays using a human-written Chinese dataset.
Outcome:	The proposed model outperforms baselines and has 91% success rate.

Copied to clipboard

Sophie Ostmeier, Justin Xu, Zhihong Chen, Maya Varma, Louis Blankemeier, Christian Bluethgen, Arne Md, Michael Moseley, Curtis Langlotz, Akshay Chaudhari, Jean-Benoit Delbrouck

Explainability & Interpretability in NLP

Challenge:	Existing automated evaluation metrics fail to consider factual correctness or are limited in their interpretability.
Approach:	They propose a radiology report evaluation metric that leverages natural language understanding of language models to identify and explain clinically significant errors.
Outcome:	The proposed method demonstrates higher correlation with expert error counts and higher alignment with expert preferences when compared to previous methods.

An LLM-Based Approach for Insight Generation in Data Analysis (2025.naacl-long)

Copied to clipboard

Alberto Sánchez Pérez, Alaa Boukhary, Paolo Papotti, Luis Castejón Lozano, Adam Elwood

PREME: Preference-based Meeting Exploration through an Interactive Questionnaire (2023.findings-eacl)

Challenge:	Existing approaches to generate insightful data from databases are time-consuming and resource-intensive.
Approach:	They propose a method that leverages Large Language Models to automatically generate textual insights from databases.
Outcome:	The proposed approach generates more insightful insights than other approaches while maintaining correctness.

Copied to clipboard

Negar Arabzadeh, Ali Ahmadvand, Julia Kiseleva, Yang Liu, Ahmed Hassan Awadallah, Ming Zhong, Milad Shokouhi

Challenge:	Recent studies show that providing meeting summaries does not align with current approaches to document summarization.
Approach:	They propose a framework for generating questionnaires for preference-based meeting exploration . they measure how much questions are answerable to ensure factual correctness .
Outcome:	The proposed framework provides a list of suggested questions reflecting user preferences . it measures how much questions are answerable to ensure factual correctness .

Multilingual CheckList: Generation and Evaluation (2022.findings-aacl)

Copied to clipboard

Karthikeyan K, Shaily Bhatt, Pankaj Singh, Somak Aditya, Sandipan Dandapat, Sunayana Sitaram, Monojit Choudhury

Multilinguality

Challenge:	Multilingual evaluation benchmarks usually contain limited high-resource languages and do not test models for specific linguistic capabilities.
Approach:	They propose an algorithm for automatically extracting target language CheckList templates from machine translated instances of a source language templates.
Outcome:	The proposed algorithm compares with CheckLists created with human verification in Hindi and 9 other languages.

Impeding LLM-assisted Cheating in Introductory Programming Assignments via Adversarial Perturbation (2024.emnlp-main)

Copied to clipboard

Saiful Salim, Rubin Yang, Alexander Cooper, Suryashree Ray, Saumya Debray, Sazzadur Rahaman

Entailment Tree Explanations via Iterative Retrieval-Generation Reasoner (2022.findings-naacl)

Challenge:	Large language model (LLM)-based programming assistants can also facilitate cheating in introductory computer science courses.
Approach:	They propose to use Large Language Models to detect and penalize cheating and modify problem statements to impede cheating.
Outcome:	The proposed methods reduce correctness scores by 77% and detectability by perturbations.

Copied to clipboard

Danilo Neves Ribeiro, Shen Wang, Xiaofei Ma, Rui Dong, Xiaokai Wei, Henghui Zhu, Xinchi Chen, Peng Xu, Zhiheng Huang, Andrew Arnold, Dan Roth

MTO: A Multi-turn Conversational Text-to-OverpassQL Dataset for Enhanced OpenStreetMap Query Generation (2026.findings-acl)

Challenge:	Large language models have achieved high performance on various natural language benchmarks, but the explainability of their output remains elusive.
Approach:	They propose an architecture called iterative retrieval-generation reasoner that generates an entailment tree that explains a given hypothesis by using premises from C.
Outcome:	The proposed model outperforms existing benchmarks on premise retrieval and entailment tree generation with around 300% gain in overall correctness.

Copied to clipboard

Haodi Zhang, Xinrui Zhu, Mingze Kong, Zhidan Liu, Tao Fan, Kaishun Wu, Yuanfeng Song

FastV-RAG: Towards Fast and Fine-Grained Video QA with Retrieval-Augmented Generation (2026.acl-long)

Challenge:	a framework for constructing multi-turn Text-to-OverpassQL dialogue datasets is proposed . a dataset of over 7,800 dialogues contains more than 20,000 individual utterances .
Approach:	They propose a framework for constructing multi-turn Text-to-OverpassQL dialogue datasets . they convert Overpass queries into syntax trees using a custom parser based on OverpassQl .
Outcome:	The proposed dataset includes over 7,800 dialogues, each containing 2 to 4 user utterances . it is the first multi-turn Text-to-OverpassQL dataset built upon the OverpassNL corpus .

Copied to clipboard

Gen Li, Peiyu Liu

Nemotron-CrossThink: Scaling Self-Learning beyond Math Reasoning (2026.eacl-long)

Challenge:	Existing methods for retrieval-augmented generation are inefficient and often fail to maintain high answer quality.
Approach:	They propose an efficient VLM-based RAG framework built on a speculative decoding pipeline and a similarity-based filtering strategy to mitigate errors.
Outcome:	The proposed framework reduces inference latency without sacrificing correctness . it achieves comparable or higher accuracy than standard approaches while speeding up inference by approximately 2x .

Copied to clipboard

Syeda Nahida Akter, Shrimai Prabhumoye, Matvei Novikov, Seungju Han, Ying Lin, Evelina Bakhturina, Eric Nyberg, Yejin Choi, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro

A Continued Pretrained LLM Approach for Automatic Medical Note Generation (2024.naacl-short)

Challenge:	Prior work has successfully applied Reinforcement Learning (RL) to mathematical reasoning, but generalization to broader domains remains challenging due to limited data and lack of verifiable rewards for unstructured domains.
Approach:	They propose a framework that integrates multi-domain corpora into RL training to improve generalization across diverse reasoning tasks.
Outcome:	The proposed framework improves generalization across diverse reasoning tasks.

Copied to clipboard

Dong Yuan, Eti Rastogi, Gautam Naik, Sree Prasanna Rajagopal, Sagar Goyal, Fen Zhao, Bharath Chintagunta, Jeffrey Ward

Neural Text Summarization: A Critical Evaluation (D19-1)

Challenge:	HEAL is the first continuously trained LLaMA2-based LLM for medical conversations . despite the success of LLMs in general capabilities, they often fall short in niche domains like healthcare .
Approach:	They propose a 13B LLaMA2-based LLM that is purpose-built for medical conversations and measured on automated scribing.
Outcome:	The HEAL LLM outperforms GPT-4 and PMC-LLaMA in PubMedQA with 78.4% accuracy and parity with GPT-LLAMA in generating medical notes.

Copied to clipboard

Wojciech Kryscinski, Nitish Shirish Keskar, Bryan McCann, Caiming Xiong, Richard Socher

How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering (2021.tacl-1)

Challenge:	Current approaches to text summarization use advanced attention and copying mechanisms, multi-task and multi-reward training techniques.
Approach:	They evaluate datasets, evaluation metrics, and models for text summarization . they highlight three primary shortcomings: 1) datasets leave task underconstrained; 2) models overfit layout biases .
Outcome:	The current evaluation protocol is weakly correlated with human judgment and does not account for factual correctness.

Copied to clipboard

Zhengbao Jiang, Jun Araki, Haibo Ding, Graham Neubig

eC-Tab2Text: Aspect-Based Text Generation from e-Commerce Product Tables (2025.naacl-industry)

Challenge:	Recent studies have shown that language models capture different types of knowledge regarding facts or commonsense knowledge.
Approach:	They examine how language models can be calibrated to make their confidence scores correlate better with the likelihood of correctness.
Outcome:	The proposed calibration methods improve confidence scores on QA tasks and improve accuracy.

Copied to clipboard

Luis Antonio Gutierrez Guanilo, Mir Tafseer Nayeem, Cristian Jose Lopez Del Alamo, Davood Rafiei

Hospitality-VQA: Decision-Oriented Informativeness Evaluation for Vision–Language Models (2026.eacl-srw)

Challenge:	eC-Tab2Text dataset is designed to capture product attributes and user-specific queries.
Approach:	They propose a novel dataset to capture the intricacies of e-commerce including detailed product attributes and user-specific queries.
Outcome:	The proposed dataset outperforms existing generalpurpose LLMs in generating accurate product reviews.

Copied to clipboard

Jeongwoo Lee, Baek Duhyeong, Eungyeol Han, Soyeon Shin, Gukin Han, Seungduk Kim, Jaehyun Jeon, Taewoo Jeong

Visual Data in NLP

Challenge:	Existing VQA benchmarks focus on factual correctness but rarely capture what information users actually find useful.
Approach:	They propose a framework to quantify how much information an image–question pair provides . they conduct experiments with several state-of-the-art VLMs to determine their reliability .
Outcome:	The proposed framework quantifies how much information an image–question pair provides in hospitality contexts.

On the Impacts of Contexts on Repository-Level Code Generation (2025.findings-naacl)

Copied to clipboard

Nam Le Hai, Dung Manh Nguyen, Nghi D. Q. Bui

Judging the Judges: Can Large Vision-Language Models Fairly Evaluate Chart Comprehension and Reasoning? (2025.acl-industry)

Challenge:	CodeLLMs are widely used for code generation, but their ability to handle repository-level dependencies remains underexplored.
Approach:	They propose a benchmark for evaluating repository-level code generation based on dependency contexts.
Outcome:	The proposed model improves dependency handling and introduces a new metric, Dependency Invocation Rate (DIR), to measure context utilization.

Copied to clipboard

Md Tahmid Rahman Laskar, Mohammed Saidul Islam, Ridwan Mahbub, Ahmed Masry, Mizanur Rahman, Amran Bhuiyan, Mir Tafseer Nayeem, Shafiq Joty, Enamul Hoque, Jimmy Huang

Semantic Consistency-Based Uncertainty Quantification for Factuality in Radiology Report Generation (2025.findings-naacl)

Challenge:	Large Vision-Language Models (LVLMs) are expensive and time-consuming to evaluate . however, they are limited in their use in industrial settings due to their limited availability and limited resources.
Approach:	They evaluate 13 open-source LVLMs as judges for diverse chart comprehension and reasoning tasks.
Outcome:	The proposed models can be used to assess chart comprehension and reasoning tasks, but they are expensive and time-consuming.

Copied to clipboard

Chenyu Wang, Weichao Zhou, Shantanu Ghosh, Kayhan Batmanghelich, Wenchao Li

AfriMed-QA: A Pan-African, Multi-Specialty, Medical Question-Answering Benchmark Dataset (2025.acl-long)

Challenge:	Radiology report generation has shown great potential in assisting radiologists . generative medical Vision Large Language Models (VLLMs) are prone to hallucinations and can produce inaccurate diagnostic information.
Approach:	They propose a framework that provides both report-level and sentence-level uncertainties.
Outcome:	The proposed method improves factuality scores by 10% by rejecting 20% of reports on the MIMIC-CXR dataset.

Copied to clipboard

Charles Nimo, Tobi Olatunji, Abraham Toluwase Owodunni, Tassallah Abdullahi, Emmanuel Ayodele, Mardhiyah Sanni, Ezinwanne C. Aka, Folafunmi Omofoye, Foutse Yuehgoh, Timothy Faniran, Bonaventure F. P. Dossou, Moshood O. Yekini, Jonas Kemp, Katherine A Heller, Jude Chidubem Omeke, Chidi Asuzu Md, Naome A Etori, Aïmérou Ndiaye, Ifeoma Okoh, Evans Doe Ocansey, Wendy Kinara, Michael L. Best, Irfan Essa, Stephen Edward Moore, Chris Fourie, Mercy Nyamewaa Asiedu

Evaluating Vision-Language Models for Emotion Recognition (2025.findings-naacl)

Challenge:	Recent advances in large language models (LLMs) performance on medical multiplechoice question (MCQ) benchmarks have stimulated interest from healthcare providers and patients globally.
Approach:	They introduce AfriMed-QA, the first largescale Pan-African English multi-specialty medical Question-Answering (QA) dataset, with 15,000 questions sourced from over 60 medical schools across 16 countries.
Outcome:	The proposed model outperforms other models in the medical field and is compared with other models.

Copied to clipboard

Sree Bhattacharyya, James Z. Wang

Selene: Pioneering Automated Proof in Software Verification (2024.acl-long)

Challenge:	Large Vision-Language Models (VLMs) have been used for objective multimodal reasoning tasks for decades.
Approach:	They present a comprehensive evaluation of large vision-language models for recognizing evoked emotions from images.
Outcome:	The proposed model performs well in evoked emotion recognition task and is robust to human errors.

Copied to clipboard

Lichen Zhang, Shuai Lu, Nan Duan

Dissecting Human and LLM Preferences (2024.acl-long)

Challenge:	Currently, software verification is resource-intensive and manpower-consuming.
Approach:	They propose a project-level automated proof benchmark based on the seL4 operating system . they propose augmentations to enhance the flexibility of the framework and lightweight verification environment .
Outcome:	The proposed framework provides a comprehensive framework for end-to-end proof generation and a lightweight verification environment.

Copied to clipboard

Junlong Li, Fan Zhou, Shichao Sun, Yikai Zhang, Hai Zhao, Pengfei Liu

ConCodeEval: Evaluating Large Language Models for Code Constraints in Domain-Specific Languages (2025.acl-industry)

Challenge:	a recent study shows that human and Large Language Model preferences are important for model fine-tuning and evaluation.
Approach:	They dissect the preferences of human and 32 different Large Language Models to understand their quantitative composition.
Outcome:	The proposed model is compared with 32 different large language models using real-world user-model conversations.

Copied to clipboard

Mehant Kammakomati, Sameer Pimparkhede, Srikanth G. Tamilselvam, Prince Kumar, Pushpak Bhattacharyya

Multi-Stage Prompting for Knowledgeable Dialogue Generation (2022.findings-acl)

Challenge:	Large Language Models (LLMs) have demonstrated potential in code generation and natural language understanding, but they struggle with code constraints.
Approach:	They propose to use Large Language Models to handle constraints represented in code . they use JSON, YAML, XML, Python, and natural language to test their effectiveness .
Outcome:	The proposed benchmark shows that LLMs can handle code constraints better than natural language . the results suggest that conscious choice of representations can lead to optimal use of LLM in enterprise use cases involving code constraints.

Copied to clipboard

Zihan Liu, Mostofa Patwary, Ryan Prenger, Shrimai Prabhumoye, Wei Ping, Mohammad Shoeybi, Bryan Catanzaro

CARMO: Dynamic Criteria Generation for Context Aware Reward Modelling (2025.findings-acl)

Challenge:	Existing knowledge-grounded dialogue systems typically use finetuned versions of a pretrained language model and large-scale knowledge bases.
Approach:	They propose a multi-stage prompting approach to generate knowledgeable responses from a single pretrained LM.
Outcome:	The proposed model outperforms the state-of-the-art retrieval-based model in terms of knowledge relevance and correctness by 5.8% and 5%, respectively.

Copied to clipboard

Taneesh Gupta, Shivam Shandilya, Xuchao Zhang, Rahul Madhavan, Supriyo Ghosh, Chetan Bansal, Huaxiu Yao, Saravan Rajmohan

Challenge:	Reward modeling in large language models is susceptible to reward hacking . flawed reward signals often lead to outputs that optimize for spurious correlates .
Approach:	They propose a new approach that generates dynamic, context-relevant criteria to ground the reward model prior to producing reward scores.
Outcome:	The proposed approach generates dynamic, context-relevant criteria to ground the model prior to producing reward scores.

Measuring Sycophancy of Language Models in Multi-turn Dialogues (2025.findings-emnlp)

Copied to clipboard

Jiseung Hong, Grace Byun, Seungone Kim, Kai Shu

Dialogue Systems & Conversational Agents

Challenge:	Prior research on sycophancy has focused on single-turn factual correctness, overlooking the dynamics of real-world interactions.
Approach:	They propose a new evaluation suite that assesses sycophantic behavior in multi-turn, free-form conversational settings.
Outcome:	The proposed evaluation suite measures how quickly a model conforms to the user and how frequently it shifts its stance under sustained user pressure.

Ensure the Correctness of the Summary: Incorporate Entailment Knowledge into Abstractive Sentence Summarization (C18-1)

Copied to clipboard

Haoran Li, Junnan Zhu, Jiajun Zhang, Chengqing Zong

Summarization

Challenge:	Existing approaches focus on improving the informativeness of the summary, but ignore the correctness.
Approach:	They propose an entailment-aware encoder and an aML-based decoder to improve the correctness of the sentence summarization task.
Outcome:	The proposed model outperforms baselines on informativeness and correctness.

ReMedQA: Are We Done With Medical Multiple-Choice Benchmarks? (2026.eacl-long)

Copied to clipboard

Alessio Cocchieri, Luca Ragazzi, Giuseppe Tagliavini, Gianluca Moro

In Search of the Long-Tail: Systematic Generation of Long-Tail Inferential Knowledge via Logical Rule Guided Search (2024.emnlp-main)

Challenge:	Multiple-choice question answering (MCQA) benchmarks show near-human accuracy . but a single accuracy score is a poor proxy for competence .
Approach:	They propose a medical multiple-choice question answering (MCQA) benchmark that augments three standard medical MCQA datasets with open-ended answers and systematically perturbed options.
Outcome:	The proposed benchmarks show that high MCQA accuracy masks low reliability . MCQ is the dominant paradigm for assessing medical knowledge in large language models .

Copied to clipboard

Huihan Li, Yuting Ning, Zeyi Liao, Siyuan Wang, Xiang Li, Ximing Lu, Wenting Zhao, Faeze Brahman, Yejin Choi, Xiang Ren

Evaluating Legal Reasoning Traces with Legal Issue Tree Rubrics (2026.acl-long)

Challenge:	Logic-Induced-Knowledge-Search (LINK) is a framework for generating factually-correct yet long-tail inferential knowledge.
Approach:	They introduce a framework to obtain factually-correct yet long-tail inferential statements using variable-wise prompting grounded on symbolic rules.
Outcome:	The proposed framework is able to obtain factually-correct yet long-tail inferential statements while ensuring factual correctness.

Copied to clipboard

Jinu Lee, Kyoung-Woon On, Sophia Simeng Han, Arman Cohan, Julia Hockenmaier

Learning to Generate Move-by-Move Commentary for Chess Games from Large-Scale Social Forum Data (P18-1)

Challenge:	Evaluating the quality of LLM-generated reasoning traces in expert domains is essential for ensuring credibility and explainability, yet remains challenging due to the inherent complexity of such reasoning tasks.
Approach:	They propose a large-scale legal reasoning dataset with an emphasis on reasoning trace evaluation that converts court judgments into hierarchical trees of opposing parties’ arguments and the court’s conclusions.
Outcome:	The proposed model improves the quality of LLM-generated reasoning traces in legal domains, whereas RL improves correctness albeit with reduced coverage.

Copied to clipboard

Harsh Jhamtani, Varun Gangal, Eduard Hovy, Graham Neubig, Taylor Berg-Kirkpatrick

Challenge:	Using a large-scale chess commentary dataset, we generate a set of comments for individual moves in a game.
Approach:	They propose a large-scale chess commentary dataset and a method to generate commentary for individual moves in a chessian game.
Outcome:	The proposed method is rated similar to ground truth commentary texts in terms of correctness and fluency.

Counter-Argument Generation by Attacking Weak Premises (2021.findings-acl)

Copied to clipboard

Milad Alshomary, Shahbaz Syed, Arkajit Dhar, Martin Potthast, Henning Wachsmuth

An Investigation of Evaluation Methods in Automatic Medical Note Generation (2023.findings-acl)

Challenge:	a recent work explores the generation of counter-arguments by undermining one of its premises . identifying the argument's weak premises is key to effective countering, we hypothesize .
Approach:	They propose a pipeline approach that first assesses the argument's weak premises and generates a counter-argument undermining the weakest among them.
Outcome:	The proposed approach undermins arguments by attacking weak premises . human annotators favor the proposed approach over state-of-the-art approaches .

Copied to clipboard

Asma Ben Abacha, Wen-wai Yim, George Michalopoulos, Thomas Lin

Performance and Risk Trade-offs for Multi-word Text Prediction at Scale (2023.findings-eacl)

Challenge:	Recent studies show that doctors can save significant amounts of time when using automatic note generation.
Approach:	They propose task-specific metrics for automatic note generation from medical conversation summarization and generation, including knowledge-graph embedding-based metrics, customized model-based measures with domain-specific weights, and ensemble metrics.
Outcome:	The proposed evaluation metrics are compared to existing models and can have different behaviors on different types of clinical notes datasets.

Copied to clipboard

Aniket Vashishtha, S Sai Prasad, Payal Bajaj, Vishrav Chaudhary, Kate Cook, Sandipan Dandapat, Sunayana Sitaram, Monojit Choudhury

Challenge:	Large Language Models (LLMs) generate ethically inappropriate texts even for seemingly innocuous contexts.
Approach:	They propose to use large language models to detect and filter toxic content in text prediction tasks by evaluating their toxicity detection approaches against a manually crafted CheckList of harms.
Outcome:	The proposed methods are compared against a checklist of harms targeted at different groups and different levels of severity in English.

SAGE: Steerable Agentic Data Generation for Deep Search with Execution Feedback (2026.findings-eacl)

Copied to clipboard

Fangyuan Xu, Rujun Han, Yanfei Chen, Zifeng Wang, I-Hung Hsu, Jun Yan, Vishy Tirumalashetty, Eunsol Choi, Tomas Pfister, Chen-Yu Lee

HelpSteer: Multi-attribute Helpfulness Dataset for SteerLM (2024.naacl-long)

Challenge:	High-quality, complex question-answer pairs are pivotal for training and evaluating capable deep search agents.
Approach:	They propose a pipeline that generates high-quality, difficulty-controlled deep search question-answer pairs for a given corpus and a target difficulty level.
Outcome:	The proposed pipeline generates high-quality, difficulty-controlled deep search question-answer pairs for a given corpus and a target difficulty level.

Copied to clipboard

Zhilin Wang, Yi Dong, Jiaqi Zeng, Virginia Adams, Makesh Narsimhan Sreedhar, Daniel Egert, Olivier Delalleau, Jane Scowcroft, Neel Kant, Aidan Swope, Oleksii Kuchaiev

Challenge:	Existing helpfulness preference datasets do not specify what makes some responses more helpful and others less helpful.
Approach:	They use a dataset that has annotated for correctness, coherence, complexity, and verbosity.
Outcome:	The dataset has annotations for correctness, coherence, complexity, and verbosity in addition to overall helpfulness of responses.

GEAR: A Scalable and Interpretable Evaluation Framework for RAG-Based Car Assistant Systems (2025.emnlp-industry)

Copied to clipboard

Niloufar Beyranvand, Hamidreza Dastmalchi, Aijun An, Heidar Davoudi, Winston Chan, Ron DiCarlantonio

Explainability & Interpretability in NLP

Challenge:	Large language models (LLMs) increasingly power car assistants, but evaluating response quality remains a challenge.
Approach:	They propose a framework that uses large language models as evaluators to compare assistant responses against ground-truth counterparts.
Outcome:	The proposed framework compares assistant responses against ground-truth counterparts, assessing coverage, correctness, and other dimensions of answer quality.

Towards Fine-Grained and Multi-Granular Contrastive Language-Speech Pre-training (2026.acl-long)

Copied to clipboard

Yifan Yang, Bing Han, Hui Wang, Wei Wang, Ziyang Ma, Long Zhou, Zengrui Jin, Guanrou Yang, Tianrui Wang, Xu Tan, Xie Chen

Conformity in Large Language Models (2025.acl-long)

Challenge:	Existing models for fine-grained speaking styles are limited in terms of accuracy, coverage, and naturalness.
Approach:	They propose a model that pre-trains with coarse captions and annotates with a pipeline that grounds captions in audio.
Outcome:	The proposed model outperforms existing models with fine-grained style annotations . it integrates global and fine-granular supervision, enabling unified representations based on the proposed model .

Copied to clipboard

Xiaochen Zhu, Caiqi Zhang, Tom Stafford, Nigel Collier, Andreas Vlachos

Characterizing LLM Abstention Behavior in Science QA with Context Perturbations (2024.findings-emnlp)

Challenge:	Conformity is a form of social influence that affects the way people respond to information.
Approach:	They adapt psychological experiments to examine the extent of conformity in large language models.
Outcome:	The proposed interventions mitigate conformity by reducing the naturalness of majority tones and reducing instruction-tuned models.

Copied to clipboard

Bingbing Wen, Bill Howe, Lucy Wang

Deliberative Searcher: Improving LLM Reliability via Reinforcement Learning with Constraints (2026.acl-long)

Challenge:	Prior work has investigated the ability of LLMs to abstain from answering context-dependent questions when provided insufficient or inconsistent context is provided.
Approach:	They propose to improve abstention when provided insufficient or incorrect context . they probed the ability of LLMs to abstain from answering context-dependent science questions .
Outcome:	The proposed models abstain from answering science questions when provided insufficient or incorrect context.

Copied to clipboard

Zhenyun Yin, Shujie Wang, Xuhong Wang, Xingjun Ma, Yingchun Wang

What Else Do I Need to Know? The Effect of Background Information on Users’ Reliance on QA Systems (2023.emnlp-main)

Challenge:	Large language models with search capabilities often exhibit miscalibrated confidence, causing incorrect answers with high certainty.
Approach:	They propose a reasoning-primary framework that integrates search operations into chain-of-thought generation while maintaining explicit confidence calibration.
Outcome:	The proposed framework improves accuracy and reliability of large language models with search capabilities.

Copied to clipboard

Navita Goyal, Eleftheria Briakou, Amanda Liu, Connor Baumler, Claire Bonial, Jeffrey Micher, Clare Voss, Marine Carpuat, Hal Daumé III

MR-ALIGN: Meta-Reasoning Informed Factuality Alignment for Large Reasoning Models (2026.findings-acl)

Challenge:	Existing NLP systems can only access the retrieved context to determine the answer, resulting in a knowledge gap between the information that is required to answer the question and the information available to assess the model’s correctness.
Approach:	They ask whether adding relevant background helps mitigate users’ over-reliance on predictions.
Outcome:	The proposed approach reduces over-reliance on model predictions even in the absence of sufficient information to assess their correctness.

Copied to clipboard

Xinming Wang, Jian Xu, Bin Yu, Sheng Lian, yi Chen, Boran Wang, Yingjian Zhu, Hongzhu Yi, Hong-Ming Yang, Han Hu, Cheng-Lin Liu, Xu-Yao Zhang

Marking Code Without Breaking It: Code Watermarking for Detecting LLM-Generated Code (2026.findings-eacl)

Challenge:	Large reasoning models (LRMs) show strong capabilities in complex reasoning, yet their marginal gains on evidence-dependent factual questions are limited.
Approach:	They propose a Meta-Reasoning informed alignment framework that quantifies state-transition probabilities along the model’s thinking process and constructs a transition-aware implicit reward that reinforces beneficial reasoning patterns while suppressing defective ones at the atomic thinking segments.
Outcome:	Empirical evaluations of four factual QA datasets and one long-form factuality benchmark show that MR-ALIGN consistently improves accuracy and truthfulness while reducing misleading reasoning.

Copied to clipboard

Jungin Kim, Shinwoo Park, Yo-Sub Han

ComparisonQA: Evaluating Factuality Robustness of LLMs Through Knowledge Frequency Control and Uncertainty (2025.findings-acl)

Challenge:	Existing methods for identifying LLM-generated code are limited by syntax-critical tokens, which can introduce syntax errors.
Approach:	They propose a syntax-aware watermarking method that embeds watermarks only in non-syntactic tokens and preserves code integrity.
Outcome:	The proposed method outperforms baseline methods on Python, C++, and Java.

Copied to clipboard

Qing Zong, Zhaowei Wang, Tianshi Zheng, Xiyu Ren, Yangqiu Song

EcoSafeRAG: Efficient Security through Context Analysis in Retrieval-Augmented Generation (2025.findings-emnlp)

Challenge:	Existing studies on LLMs' factual knowledge are unreliable since the questions can vary not only in entity frequency but also in difficulty themselves.
Approach:	They propose a benchmark to study the role of knowledge frequency in the performance of large language models (LLMs) it aims to avoid possible semantic shortcuts which is a serious problem of current QA study.
Outcome:	The proposed method avoids possible semantic shortcuts and improves on existing proofs.

Copied to clipboard

Ruobing Yao, Yifei Zhang, Shuang Song, Neng Gao, Chenyang Tu

Green & Sustainable NLP

Challenge:	Existing defense methods rely on internal knowledge of the model, which conflicts with the design concept of Retrieval-Augmented Generation (RAG).
Approach:	EcoSafeRAG uses sentence-level processing and bait-guided context diversity detection to identify malicious content .
Outcome:	EcoSafeRAG uses sentence-level processing and bait-guided context diversity detection to identify malicious content.

MoleculeQA: A Dataset to Evaluate Factual Accuracy in Molecular Comprehension (2024.findings-emnlp)

Copied to clipboard

Xingyu Lu, He Cao, Zijing Liu, Shengyuan Bai, Leqing Chen, Yuan Yao, Hai-Tao Zheng, Yu Li

Rule or Story, Which is a Better Commonsense Expression for Talking with Large Language Models? (2024.acl-long)

Challenge:	Existing models generate erroneous information and evaluations fail to assess factual correctness of models.
Approach:	They propose to use MoleculeQA to evaluate molecular factual correctness in large language models by organizing molecules into a taxonomy and building QA pairs through human and LLM efforts.
Outcome:	The proposed model improves the factual correctness of generated information and enables the development of new models.

Copied to clipboard

Ning Bian, Xianpei Han, Hongyu Lin, Yaojie Lu, Ben He, Le Sun

Improving In-Context Learning with Prediction Feedback for Sentiment Analysis (2024.findings-acl)

Challenge:	Experimental results show that stories outperform rules as the expression for retrieving commonsense from LLMs, exhibiting higher generation confidence and commonsensense accuracy.
Approach:	They investigate the commonsense ability of large language models expressed through stories and rules to retrieve commonsensing knowledge from LLMs.
Outcome:	The stories outperform rules as commonsense expressions on 28 commonsensense QA datasets, exhibiting higher generation confidence and commonsence accuracy.

Copied to clipboard

Hongling Xu, Qianlong Wang, Yice Zhang, Min Yang, Xi Zeng, Bing Qin, Ruifeng Xu

Low-Resource NLP

Challenge:	Large language models (LLMs) have achieved promising results in sentiment analysis through the in-context learning paradigm.
Approach:	They propose a framework that incorporates prior predictions and feedback to improve sentiment understanding by incorporating prior feedback and leveraging a feedback-driven prompt.
Outcome:	The proposed framework improves on nine sentiment analysis datasets with an average improvement of 5.95% over conventional methods.

Beyond the Crowd: LLM-Augmented Community Notes for Governing Health Misinformation (2026.acl-long)

Copied to clipboard

Jiaying Wu, Zihang Fu, Haonan Wang, Fanxiao Li, Jiafeng Guo, Preslav Nakov, Min-Yen Kan

Improving Reward Models with Synthetic Critiques (2025.findings-naacl)

Challenge:	X (formerly Twitter) users can flag misleading posts, attach contextual notes, and rate the notes’ helpfulness, but there is a significant latency in Community Notes, which is unable to provide accurate notes.
Approach:	They propose a framework that augments Community Notes for faster and more reliable health misinformation governance.
Outcome:	The proposed framework outperforms human contributors in correctness, helpfulness, and evidence utility in health misinformation surges.

Copied to clipboard

Zihuiwen Ye, Fraser David Greenlee, Max Bartolo, Phil Blunsom, Jon Ander Campos, Matthias Gallé

Understanding the Modality Gap: An Empirical Study on the Speech-Text Alignment Mechanism of Large Speech Language Models (2025.emnlp-main)

Challenge:	a recent study shows that reward models overfit on superficial features, hindering generalization performance . prevailing approach to training preference-based reward models presents several challenges .
Approach:	They propose a method that uses synthetic natural language critiques to provide additional feedback to large language models.
Outcome:	The proposed approach improves performance and data efficiency of RMs initialized from different pretrained models, reducing the reliance on costly human annotations.

Copied to clipboard

Bajian Xiang, Shuaijiang Zhao, Tingwei Guo, Wei Zou

Efficient Citer: Tuning Large Language Models for Enhanced Answer Quality and Verification (2024.findings-naacl)

Challenge:	LSLMs have impressive conversational generation abilities, but consistently fall short of traditional pipeline systems on semantic understanding benchmarks.
Approach:	They propose to analyze the performance gap between speech and text inputs through a systematic experiment . they find that representation similarity is strongly correlated with the modality gap .
Outcome:	The proposed models improve the accuracy of speech inputs and their semantic understanding benchmarks.

Copied to clipboard

Marzieh Tahaei, Aref Jafari, Ahmad Rashid, David Alfonso-Hermelo, Khalil Bibi, Yimeng Wu, Ali Ghodsi, Boxing Chen, Mehdi Rezagholizadeh

UCoder: Unsupervised Code Generation by Internal Probing of Large Language Models (2026.findings-acl)

Challenge:	Existing models with explicit citations lack the ability to verify information generated by these models.
Approach:	They construct a citation training dataset and fine-tune two models to address the challenge of explicit citations efficiently.
Outcome:	The proposed models surpass ChatGPT and exhibit exceptional out-of-domain generalization in both human and automatic evaluation.

Copied to clipboard

Jiajun Wu, Jian Yang, Wei Zhang, Linzheng Chai, Yuchi Ma, Ensheng Shi, Yuqing Ma, Zhoujun Li, Xianglong Liu

From spoken dialogue to formal summary: An utterance rewriting for dialogue summarization (2022.naacl-main)

Challenge:	Large language models (LLMs) have demonstrated remarkable capabilities in code generation tasks, but their effectiveness relies on supervised training with extensive labeled data and computational resources.
Approach:	They propose an unsupervised method that leverages Internal Probing of Large language models for Code generation without any external corpus, even unlabeled code snippets.
Outcome:	The proposed method can achieve competitive performance compared to supervised approaches while reducing the dependency on labeled data and computational resources.

Copied to clipboard

Yue Fang, Hainan Zhang, Hongshen Chen, Zhuoye Ding, Bo Long, Yanyan Lan, Yanquan Zhou

Tree-of-Quote Prompting Improves Factuality and Attribution in Multi-Hop and Medical Reasoning (2025.emnlp-main)

Challenge:	Existing models focus more on the structure of summary, not on the personal and logical inconsistency problem.
Approach:	They propose a model to solve the problem of personal and logical inconsistency . they use an utterance rewriter to complete the ellipsis content of dialogue content .
Outcome:	The proposed model outperforms baseline models on both SAMSum and DialSum datasets.

Copied to clipboard

Justin Xu, Yiming Li, Zizheng Zhang, Augustine Yui Hei Luk, Mayank Jobanputra, Samarth Oza, Ashley Murray, Meghana Reddy Kasula, Andrew Parker, David W Eyre

Low-Resource NLP

Challenge:	Large language models (LLMs) produce fluent but factually incorrect outputs, a phenomenon commonly referred to as hallucination.
Approach:	They propose a Tree-of-Quote framework that decomposes complex questions into subquestions and generates quotes to support each step without retrieval.
Outcome:	Experiments on StrategyQA, 2WikiMultiHopQA, MuSiQue, MoreHopQ, and MedQA show that ToQ improves factuality and attribution over baselines.

LLMs as Cultural Archives: Cultural Commonsense Knowledge Graph Extraction (2026.eacl-long)

Copied to clipboard

Junior Cedric Tonga, Chen Cecilia Liu, Iryna Gurevych, Fajri Koto

EPT-X: An Expression-Pointer Transformer model that generates eXplanations for numbers (2022.acl-long)

Challenge:	Large language models encode rich cultural knowledge, but it remains mostly implicit and unstructured, limiting its interpretability and use.
Approach:	They propose an iterative framework for constructing a Cultural Commonsense Knowledge Graph using a prompt-based framework.
Outcome:	The proposed framework improves cultural reasoning and story generation on non-English cultures.

Copied to clipboard

Bugeun Kim, Kyung Seo Ki, Sangkyu Rhim, Gahgene Gweon

Step Potential Advantage Estimation: Harnessing Intermediate Confidence and Correctness for Efficient Mathematical Reasoning (2026.findings-acl)

Challenge:	Existing neural approaches to solve algebraic word problems have a plausible answer, but this belief has less been verified due to Q.
Approach:	They propose a neural model EPT-X which utilizes natural language explanations to solve an algebraic word problem.
Outcome:	The proposed model achieves an average performance of 69.59% on a PEN dataset and produces explanations with quality comparable to human output.

Copied to clipboard

Fei Wu, Zhenrong Zhang, Qikai Chang, Jianshu Zhang, Quan Liu, Jun Du

Improving the Factual Correctness of Radiology Report Generation with Semantic Rewards (2022.findings-emnlp)

Challenge:	Existing approaches to RLVR provide sparse supervision since reward arrives only after the full generation is complete.
Approach:	They propose a step-level reward system that extracts confidence and correctness and combines them into a Step Potential signal that explicitly estimates reasoning state at each step.
Outcome:	The proposed method outperforms existing methods on multiple benchmarks and improves accuracy while reducing response length.

Copied to clipboard

Jean-Benoit Delbrouck, Pierre Chambon, Christian Bluethgen, Emily Tsai, Omar Almusa, Curtis Langlotz

Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Framework (2023.acl-long)

Challenge:	Neural image-to-text radiology report generation systems have been successful on NLG metrics, but they are not factually complete or consistent due to inadequate training and evaluation.
Approach:	They propose a method to improve the factual completeness and correctness of generated radiology reports by using a dataset containing annotated chest X-ray images.
Outcome:	The proposed method significantly improves factual completeness and correctness of generated radiology reports on two open radiology report datasets.

Copied to clipboard

Ruochen Zhao, Xingxuan Li, Shafiq Joty, Chengwei Qin, Lidong Bing

Rethinking Hallucinations: Correctness, Consistency, and Prompt Multiplicity (2026.eacl-long)

Challenge:	Large language models (LLMs) have a number of shortcomings, including lack of factual correctness.
Approach:	They propose a framework to increase prediction factuality by post-editing reasoning chains . they propose to use large language models to generate interpretable reasoning chains.
Outcome:	The proposed framework leads to accuracy improvements in open-domain question-answering tasks.

Copied to clipboard

Prakhar Ganesh, Reza Shokri, Golnoosh Farnadi

ET-Agent: Incentivizing Effective Tool-Integrated Reasoning Agent via Behavior Calibration (2026.acl-long)

Challenge:	Existing hallucination evaluations focus only on correctness and often overlook consistency . a significant inconsistency in benchmarks like Med-HALT suggests hallucianation-related harms have been misunderstood.
Approach:	They propose a framework for quantifying consistency in hallucination evaluations . they find that detection techniques detect consistency, not correctness .
Outcome:	The proposed framework uncovers critical limitations in hallucination evaluations.

Copied to clipboard

Yifei Chen, Guanting Dong, Zhicheng Dou

Beemo: Benchmark of Expert-edited Machine-generated Outputs (2025.naacl-long)

Challenge:	Existing training frameworks for Large Language Models (LLMs) focus on answers’ accuracy, overlooking specific alignment for behavior patterns.
Approach:	They propose a training framework for calibrating agent’s tool-use behavior through two synergistic perspectives: self-evolving data flywheel and behavior calibration training.
Outcome:	The proposed framework improves the accuracy, efficiency, reasoning conciseness, and tool execution accuracy of large language models.

Copied to clipboard

Ekaterina Artemova, Jason S Lucas, Saranya Venkatraman, Jooyoung Lee, Sergei Tilga, Adaku Uchendu, Vladislav Mikhailov

Challenge:	Existing benchmarks for machine-generated texts (MGTs) include single-author texts (human-written and machine-generated).
Approach:	They propose to benchmark machine-generated outputs (Beemo) which includes 6.5k texts written by humans, generated by ten instruction-finetuned LLMs, and edited by experts for various use cases.
Outcome:	The proposed benchmark includes 6.5k texts written by humans, generated by ten instruction-finetuned LLMs, and edited by experts for various use cases, ranging from creative writing to summarization.

Explaining Length Bias in LLM-Based Preference Evaluations (2025.findings-emnlp)

Copied to clipboard

Zhengyu Hu, Linxin Song, Jieyu Zhang, Zheyuan Xiao, Tianfu Wang, Zhengyu Chen, Nicholas Jing Yuan, Jianxun Lian, Kaize Ding, Hui Xiong

LegalReasoner: Step-wised Verification-Correction for Legal Judgment Reasoning (2025.acl-long)

Challenge:	a preference evaluation metric is often biased towards longer responses, revealing a reliability problem . a decomposition of the preference evaluation into two components is needed to understand this bias.
Approach:	They propose to decompose the preference evaluation metric into two key components . the first component is length-dependent and related to trustworthiness .
Outcome:	The proposed evaluation metric is based on two components: desirability and information mass.

Copied to clipboard

Weijie Shi, Han Zhu, Jiaming Ji, Mengze Li, Jipeng Zhang, Ruiyuan Zhang, Jia Zhu, Jiajie Xu, Sirui Han, Yike Guo

ReCoSa: Detecting the Relevant Contexts with Self-Attention for Multi-turn Dialogue Generation (P19-1)

Challenge:	Existing legal judgment prediction methods struggle with logical errors when conducting complex legal reasoning.
Approach:	They propose a method which enhances LJP reliability through step-wise verification and correction of the reasoning process.
Outcome:	The proposed model significantly improves concordance with court decisions from 72.37 to 80.27 on LLAMA-3.1-70B.

Copied to clipboard

Hainan Zhang, Yanyan Lan, Liang Pang, Jiafeng Guo, Xueqi Cheng

Dialogue Systems & Conversational Agents

Challenge:	Existing hierarchical recurrent encoder-decoder models treat all contexts indiscriminately, which may hurt the following response generation process.
Approach:	They propose a hierarchical recurrent encoder-decoder model that treats all contexts indiscriminately and uses a word level LSTM encoder to obtain the initial representation of each context.
Outcome:	The proposed model outperforms baseline models on Chinese customer services and English Ubuntu dialogue datasets in terms of both metric-based and human evaluations.

Turning the Tide: Repository-based Code Reflection (2025.findings-emnlp)

Copied to clipboard

Wei Zhang, Jian Yang, Jiaxi Yang, Ya Wang, Zhoujun Li, Zeyu Cui, Binyuan Hui, Junyang Lin

Interpretable Math Word Problem Solution Generation via Step-by-step Planning (2023.acl-long)

Challenge:	Code large language models (LLMs) enhance programming by understanding and generating code across languages.
Approach:	a new benchmark evaluates code understanding and generation in repositories using code large language models.
Outcome:	The proposed model improves code understanding and generation in repositories by evaluating 1,888 test cases across 6 programming languages.

Copied to clipboard

Mengxue Zhang, Zichao Wang, Zhichao Yang, Weiqi Feng, Andrew Lan

Tandem Training for Language Models (2026.eacl-long)

Challenge:	Existing approaches to solving math word problems focus on obtaining the correct answer.
Approach:	They propose a step-by-step planning approach for intermediate solution generation that strategically plans the generation of the next solution step based on the MWP and the previous solution steps.
Outcome:	The proposed approach improves the accuracy and interpretability of the solution on automatic metrics and human evaluation.

Copied to clipboard

Robert West, Ashton Anderson, Ece Kamar, Eric Horvitz

Text Simplification via Adaptive Teaching (2024.findings-acl)

Challenge:	As language models improve, their actions and reasoning will become difficult or impossible for weaker agents and humans to follow, undermining interpretability and oversight.
Approach:	They propose a tandem training paradigm that allows models to adapt their language to weaker partners by intermittently and randomly sampling a frozen weak model instead of the strong model being trained.
Outcome:	The proposed model is able to produce solutions that are intelligible to weaker agents and humans while keeping task accuracy high.

Copied to clipboard

Seyed Ali Bahrainian, Jonathan Dou, Carsten Eickhoff

Enabling Large Language Models to Generate Text with Citations (2023.emnlp-main)

Challenge:	Text simplification is the process of rewriting a text using simpler vocabulary and grammatical structure in order to make it more accessible and understandable for a larger audience.
Approach:	They propose a model for text simplification based on adaptive teaching using a teacher network and a text generation network.
Outcome:	The proposed model outperforms the current state-of-the-art model on the Wiki-Doc and D-Wikipedia datasets and performs well on human evaluations in terms of text simplicity, correctness, and fluency.

Copied to clipboard

Tianyu Gao, Howard Yen, Jiatong Yu, Danqi Chen

SimplifyUR: Unsupervised Lexical Text Simplification for Urdu (2020.lrec-1)

Challenge:	Existing work relies on commercial search engines and human evaluation, making it difficult to reproduce and compare different modeling approaches.
Approach:	They propose a new generation paradigm that requires large language models to provide citations to one or a few text passages for any statement they generate.
Outcome:	The proposed model improves factual correctness and verifiability of large language models by providing citations to a set of questions and retrieval corpora and generating answers with citation.

Copied to clipboard

Namoos Hayat Qasmi, Haris Bin Zia, Awais Athar, Agha Ali Raza

Accounting for Sycophancy in Language Model Uncertainty Estimation (2025.findings-naacl)

Challenge:	Existing methods for text simplification for Urdu rely on manual lexicons and simplified corpora, but are not applicable to the language.
Approach:	They propose an unsupervised method for automatic text simplification for Urdu using word embeddings and morphological features.
Outcome:	The proposed method achieves BLEU score of 80.15 and SARI score of 42.02 on simple text generated on simplified corpora and human evaluations for correctness, grammaticality, meaning-preservation and simplicity.

Copied to clipboard

Anthony Sicilia, Mert Inan, Malihe Alikhani

Mixture of Decoding: An Attention-Inspired Adaptive Decoding Strategy to Mitigate Hallucinations in Large Vision-Language Models (2025.findings-acl)

Challenge:	Effective human-machine collaboration requires machine learning models to externalize uncertainty.
Approach:	They propose a generalization of the definition of sycophancy bias and a new algorithm to account for scophancies in uncertainty estimation.
Outcome:	The proposed algorithm can account for sycophancy in uncertainty estimation process.

Copied to clipboard

Xinlong Chen, Yuanxing Zhang, Qiang Liu, Junfei Wu, Fuzheng Zhang, Tieniu Tan

DREAM: Deep Research Evaluation with Agentic Metrics (2026.acl-long)

Challenge:	Large Vision-Language Models (LVLMs) have impressive capabilities across visual tasks, yet they remain hindered by the persistent challenge of hallucinations.
Approach:	They propose a novel approach that dynamically adapts decoding strategies by evaluating the correctness of the model’s attention on image tokens to distinguish the correct attention.
Outcome:	Extensive experiments show that the proposed approach outperforms existing decoding methods across multiple mainstream benchmarks, effectively mitigating hallucinations in LVLMs.

Copied to clipboard

Elad Ben Avraham, ChangHao Li, Ron Dorfman, Roy Ganz, Oren Nuriel, Amir Dudai, Aviad Aberdam, Noah Flynn, Elman Mansimov, Aditya Kalyanpur, Ron Litman

Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology Reports (2020.acl-main)

Challenge:	Recent benchmarks propose distinct methodologies, yet they suffer from the Mirage of Synthesis . static evaluators lack the tool-use capabilities required to assess temporal validity and factual correctness .
Approach:	They propose a framework that instantiates the principle of capability parity by making evaluation agentic.
Outcome:	The proposed framework is more sensitive to factual decay than existing benchmarks . large language models increasingly support autonomous, tool-using agents .

Copied to clipboard

Yuhao Zhang, Derek Merck, Emily Tsai, Christopher D. Manning, Curtis Langlotz

Towards Real-World Writing Assistance: A Chinese Character Checking Benchmark with Faked and Misspelled Characters (2024.acl-long)

Challenge:	Existing abstractive summarization models do not guarantee factual correctness of summaries .
Approach:	They propose a framework where models evaluate factual correctness by fact-checking it against its reference using an information extraction module.
Outcome:	The proposed method significantly improves the factual correctness and overall quality of outputs over a competitive neural summarization system, producing radiology summaries that approach the quality of human-authored ones.

Copied to clipboard

Yinghui Li, Zishan Xu, Shaoshen Chen, Haojing Huang, Yangning Li, Shirong Ma, Yong Jiang, Zhongli Li, Qingyu Zhou, Hai-Tao Zheng, Ying Shen

Visual Data in NLP

Challenge:	Existing studies focus on misspelled characters, ignoring faked characters which are more common and difficult to correct.
Approach:	They propose to use Chinese character checking to identify and correct wrong characters in texts by human annotation.
Outcome:	The proposed dataset is the first real-world visual and the largest human-crafted dataset for the Chinese character checking scenario.

DuetSim: Building User Simulator with Dual Large Language Models for Task-Oriented Dialogues (2024.lrec-main)

Copied to clipboard

Xiang Luo, Zhiwen Tang, Jin Wang, Xuejie Zhang

Dialogue Systems & Conversational Agents

Challenge:	User Simulators are used to train task-oriented dialogue systems . traditional training paradigms rely on human-engineered agendas resulting in generated responses that lack diversity and spontaneity.
Approach:	They propose a framework that leverages large language models to generate diverse responses . they use two LLMs to generate and verify responses, which are preferred by users .
Outcome:	The proposed framework produces responses that exhibit diversity and are preferred by human users.

CoQuIR: A Comprehensive Benchmark for Code Quality-Aware Information Retrieval (2026.acl-long)

Copied to clipboard

Jiahui Geng, Fengyu Cai, Shaobo Cui, Qing Li, Liangwei Chen, Chenyang Lyu, Haonan Li, Derui Zhu, Alexander Pretschner, Heinz Koeppl, Fakhri Karray

Self-Training Meets Consistency: Improving LLMs’ Reasoning with Consistency-Driven Rationale Evaluation (2025.naacl-long)

Challenge:	Existing benchmarks focus on functional relevance while neglecting code quality.
Approach:	They propose a multilingual benchmark to evaluate quality-aware code retrieval . they include fine-grained quality annotations over 42,725 queries and 134,907 code snippets .
Outcome:	The proposed benchmarks show that state-of-the-art models fail to separate buggy or insecure code from robust counterparts.

Copied to clipboard

Jaehyeok Lee, Keisuke Sakaguchi, JinYeong Bak

Aligning Offline Metrics and Human Judgments of Value for Code Generation Models (2023.findings-acl)

Challenge:	Existing approaches labeled rationales that produce correct answers as appropriate for training but one measure risks misjudging rationale quality, leading models to learn flawed reasoning patterns.
Approach:	They propose a framework that evaluates rationales through follow-up questions and leverages this evaluation to guide its training.
Outcome:	The proposed framework improves robustness and correctness of rationales and reasoning abilities compared to previous self-training approaches.

Copied to clipboard

Victor Dibia, Adam Fourney, Gagan Bansal, Forough Poursabzi-Sangdeh, Han Liu, Saleema Amershi

A Systematic Approach to Derive a Refined Speech Corpus for Sinhala (2022.lrec-1)

Challenge:	Large language models have shown impressive capabilities on code generation tasks.
Approach:	They propose a metric that combines functional correctness and syntactic similarity to measure the productivity gains generated by large language models.
Outcome:	The proposed model achieves a 14% stronger correlation with value and better represents real-world gains when evaluating and comparing models.

Copied to clipboard

Disura Warusawithana, Nilmani Kulaweera, Lakshan Weerasinghe, Buddhika Karunarathne

R3-RAG: Learning Step-by-Step Reasoning and Retrieval for LLMs via Reinforcement Learning (2025.findings-emnlp)

Challenge:	Despite being large and generic, some languages such as Sinhala are left to underutilize the technology due to the lack of adequate resources.
Approach:	They propose to derive a corpus from a publicly available corpus for Sinhala speech recognition using crowdsourcing and web scraping techniques.
Outcome:	The proposed corpus reduces the Word-Error-Rate by 15.9%.

Copied to clipboard

Yuan Li, Qi Luo, Xiaonan Li, Bufan Li, Qinyuan Cheng, Bo Wang, Yining Zheng, Yuxin Wang, Zhangyue Yin, Xipeng Qiu

Structured List-Grounded Question Answering (2025.coling-main)

Challenge:	RAG systems that integrate external knowledge with Large Language Models often become bottlenecks due to their limited parameters compared to LLMs and their inability to perform step-by-step reasoning.
Approach:	They propose a model that integrates external knowledge with Large Language Models to enhance factual correctness and mitigate hallucination.
Outcome:	The proposed model outperforms baselines and can transfer well to different retrievers.

Copied to clipboard

Mujeen Sung, Song Feng, James Gung, Raphael Shu, Yi Zhang, Saab Mansour

Natural Language Deduction with Incomplete Information (2022.emnlp-main)

Challenge:	Document-grounded dialogue systems aim to answer user queries by leveraging external information.
Approach:	They propose a dataset to evaluate QA systems' ability to interpret and use structured lists . they use language models and model-based filtering processes to enhance data quality .
Outcome:	The proposed model outperforms baselines on the LIST2QA dataset . it shows that the proposed model is more accurate and complete than baselines .

Copied to clipboard

Zayne Sprague, Kaj Bostrom, Swarat Chaudhuri, Greg Durrett

ASQA: Factoid Questions Meet Long-Form Answers (2022.emnlp-main)

Challenge:	Existing systems for reasoning given incomplete information are inadequate . current approaches to reasoning are based on latent reasoning by large language models .
Approach:	They propose a system that generates a natural language "proof" by abductively inferring a premise from another premise and a conclusion.
Outcome:	The proposed system can handle the underspecified setting where not all premises are stated at the outset; additional assumptions need to be materialized to prove a claim.

Copied to clipboard

Ivan Stelmakh, Yi Luan, Bhuwan Dhingra, Ming-Wei Chang

Mathematical Proof as a Litmus Test: Revealing Failure Modes of Advanced Large Reasoning Models (2026.acl-long)

Challenge:	Recent progress on factoid question answering (QA) does not easily transfer to the task of long-form QA where the goal is to generate detailed explanations.
Approach:	They propose a task that focuses on ambiguous factoid questions which have different correct answers depending on interpretation.
Outcome:	The proposed metric is reliable and demonstrates agreement between this metric and human judgments, and reveals a considerable gap between human performance and strong baselines.

Copied to clipboard

Dadi Guo, Jiayu Liu, Zhiyuan Fan, Zhitao He, Haoran Li, Yuxin Li, Yumeng Wang, Yi R. Fung

Where Fact Ends and Fairness Begins: Redefining AI Bias Evaluation through Cognitive Biases (2025.findings-emnlp)

Challenge:	Large reasoning models have demonstrated remarkable mathematical problem-solving abilities, but their true reasoning shortcomings are often hidden.
Approach:	They propose to leverage the rigor and methodological complexity of mathematical proofs as a diagnostic tool to expose hidden failures.
Outcome:	The proposed model evaluation exploits the rigor and complexity of proof problems to uncover 10 fine-grained errors.

Copied to clipboard

Jen-tse Huang, Yuhang Yan, Linqi Liu, Yixin Wan, Wenxuan Wang, Kai-Wei Chang, Michael R. Lyu

Diagnosing LLMs via Information Spectrum Analysis: Tail Behavior and the Effects of Side Information (2026.findings-acl)

Challenge:	Existing benchmarks conflate factual correctness and normative fairness . a model may generate responses that are factually accurate but socially unfair .
Approach:	They propose a benchmark to examine the boundary between fact and fair . they draw on representativeness bias, attribution bias and ingroup–outgroup bias to explain why models often misalign fact and faireness.
Outcome:	The proposed model is based on ten frontier models and is available on github . it is compared with a standard model that generates people of color in Nazi-era uniforms .

Copied to clipboard

Yuuki Tachioka

Challenge:	Large language models exhibit non-stationary generation because of variability in output distributions . authors propose a framework that treats LLMs as general sources without stationarity or ergodicity .
Approach:	They propose a diagnostic framework that treats large language models as general sources without stationarity, ergodicity, or the asymptotic equipartition property.
Outcome:	The proposed framework treats large language models as general sources without stationarity, ergodicity, or the asymptotic equipartition property.

Reinforced Informativeness Optimization for Long-Form Retrieval-Augmented Generation (2026.acl-long)

Copied to clipboard

Yuhao Wang, Ruiyang Ren, Yucheng Wang, Xin Zhao, Jing Liu, Hua Wu, Haifeng Wang

ReCEval: Evaluating Reasoning Chains via Correctness and Informativeness (2023.emnlp-main)

Challenge:	Existing reinforcement learning systems lack verifiable reward mechanisms for long-form question answering . current systems lack reliable long-term answers due to lack of factual content .
Approach:	They propose a framework for reinforced verifiable informativeness optimization . it defines informativeness as measurable and externally verifier objective for RL .
Outcome:	Experiments show that RioRAG achieves higher factual recall and faithfulness . the proposed framework is based on a framework that uses nugget-centric verification with cross-source checks .

Copied to clipboard

Archiki Prasad, Swarnadeep Saha, Xiang Zhou, Mohit Bansal

Rethinking Reward Model Evaluation Through the Lens of Reward Overoptimization (2025.acl-long)

Challenge:	Existing methods focus on whether the reasoning chain leads to the correct conclusion, but this view may confound reasoning quality with other spurious shortcuts to predict the answer.
Approach:	They propose a framework that evaluates reasoning chains via two key properties: (1) correctness, i.e., each step makes a valid inference based on information contained within the step, preceding steps, and input context, and (2) informativeness, respectively.
Outcome:	The proposed framework evaluates reasoning chains via two key properties: (1) correctness, i.e., each step makes a valid inference based on information contained within the step, preceding steps, and input context, and (2) informativeness, which is helpful towards deriving the generated answer.

Copied to clipboard

Sunghwan Kim, Dongjin Kang, Taeyoon Kwon, Hyungjoo Chae, Dongha Lee, Jinyoung Yeo

Challenge:	Existing benchmarks for reward models show a weak correlation with performance of optimized policies . existing benchmarks do not accurately assess the true capabilities of reward models .
Approach:	They explore how reward overoptimization captures how well a reward model aligns with human preferences and the dynamics of the learning signal it provides to the policy.
Outcome:	The proposed benchmarks show that reward overoptimization is a weak factor . the high correlation with degree of overoptimalization leads to lower correlation with downstream performance .

A Framework for Evaluation of Machine Reading Comprehension Gold Standards (2020.lrec-1)

Copied to clipboard

Viktor Schlegel, Marco Valentino, Andre Freitas, Goran Nenadic, Riza Batista-Navarro

Towards Modeling Revision Requirements in wikiHow Instructions (2020.emnlp-main)

Challenge:	Existing literature on machine reading comprehension (MRC) data is limited on the data design of gold standards.
Approach:	They propose a framework to investigate linguistic features, lexical cues and ambiguity in MRC gold standards.
Outcome:	The proposed framework investigates the present linguistic features, required reasoning and background knowledge and factual correctness on the one hand, and the presence of lexical cues as a lower bound for the requirement of understanding on the other.

Copied to clipboard

Irshad Bhat, Talita Anthonio, Michael Roth

Challenge:	wikiHow is a collaboratively edited platform of how-to guides . authors extend existing textual edits with 4 million sentences that remain unedited .
Approach:	They extend existing textual edits with a set of 4 million sentences that remain unedited over time.
Outcome:	The proposed model can predict the need for edits in wikiHow guides . the authors extend an existing resource of textual edits with a complementary set of 4 million sentences that remain unedited over time .

FAITH: Factuality Alignment through Integrating Trustworthiness and Honestness (2026.findings-acl)

Copied to clipboard

Xiaoning Dong, Chengyan Wu, Yajie Wen, Yu Chen, Yun Xue, Zhang Jing, Wei Xu, Bolei Ma

ReasonerRank: Redefining Language Model Evaluation with Ground-Truth-Free Ranking Frameworks (2025.findings-acl)

Challenge:	Existing approaches to correct factually inaccurate outputs are lacking the semantic richness needed to properly understand its internal states of trustworthiness and honesty.
Approach:	They propose a framework for factuality alignment that integrates natural-language uncertainty signals with external knowledge and computes confidence scores and semantic entropy from LLM outputs.
Outcome:	Extensive experiments on four knowledge-intensive benchmarks show that FAITH improves the factual accuracy and truthfulness of Large Language Models (LLMs).

Copied to clipboard

Jiamu Zhang, Jiayi Yuan, Andrew Wen, Hoang Anh Duy Le, Yu-Neng Chuang, Soo-Hyun Choi, Rui Chen, Xia Hu

Halwasa: Quantify and Analyze Hallucinations in Large Language Models: Arabic as a Case Study (2024.lrec-main)

Challenge:	Large Language Models (LLMs) are increasingly adopted across real-world applications . traditional evaluations rely on expensive, domain-specific ground-truth labels . obtaining labeled data is expensive, time-consuming, and often requires domain expertise .
Approach:	They propose a ground-truth-free evaluation framework focused on reasoning consistency and instruction following.
Outcome:	The proposed framework outperforms existing label-free methods, including majority voting, triplet ranking, and peer-review approaches.

Copied to clipboard

Hamdy Mubarak, Hend Al-Khalifa, Khaloud Suliman Alkhalefah

Not All Tokens Matter: Towards Efficient LLM Reasoning via Token Significance in Reinforcement Learning (2026.acl-long)

Challenge:	Large Language Models (LLMs) generate text that is factually incorrect, nonsensical, or misleading.
Approach:	They create a large Arabic dataset that contains 10K of LLM generated sentences and annotate it for factuality and correctness.
Outcome:	The proposed dataset analyzes 10K of generated sentences and finds 25% of them are factually incorrect.

Copied to clipboard

Hanbing Liu, Lang Cao, Yuanyi Ren, Mengyu Zhou, Haoyu Dong, Xiaojun Ma, Shi Han, Dongmei Zhang

Logits-Based Finetuning (2025.emnlp-main)

Challenge:	Large language models (LLMs) often produce unnecessarily long explanations that reduce efficiency.
Approach:	They propose a length-aware reward that selectively penalizes insignificance tokens . they also propose 'dynamic length control' that encourages more detailed reasoning .
Outcome:	The proposed method reduces response length while maintaining correctness, the authors show . it selectively penalizes insignificance tokens while maintaining accuracy .

Copied to clipboard

Jingyao Li, Senqiao Yang, Sitong Wu, Han Shi, Chuanyang Zheng, Hong Xu, Jiaya Jia

When LLMs Read Tables Carelessly: Measuring and Reducing Data Referencing Errors (2026.acl-long)

Challenge:	Existing methods for developing compact and efficient large language models lack token-level dependencies and linguistic diversity.
Approach:	They propose a logits-based fine-tuning framework that integrates supervised learning and knowledge distillation to build enriched training targets using teacher logits and ground truth labels.
Outcome:	The proposed method outperforms existing methods on a large-scale logits dataset and a series of science-focused models.

Copied to clipboard

Yuqing Yang, Qi Zhu, Zhen Han, Boran Han, Zhengyuan Shen, Shuai Wang, Vassilis N. Ioannidis, Huzefa Rangwala

CodeDPO: Aligning Code Models with Self Generated and Verified Source Code (2025.acl-long)

Challenge:	Large language models (LLMs) perform well on table tasks, but they still make data referencing errors (DREs) prior studies have only offered limited, small-scale analyses.
Approach:	They propose inference-time strategies and lightweight critics to mitigate data referencing errors.
Outcome:	The proposed model achieves an average F1 score of 78.2% in detecting both in-distribution and out-of-difference DREs and assists inference for larger models.

Copied to clipboard

Kechi Zhang, Ge Li, Yihong Dong, Jingjing Xu, Jun Zhang, Jing Su, Yongfei Liu, Zhi Jin

Improving Language Model Reasoning with Self-motivated Learning (2024.lrec-main)

Challenge:	Existing training methods for code generation do not improve code correctness and efficiency.
Approach:	They propose a framework that integrates preference learning into code generation to improve code correctness and efficiency.
Outcome:	The proposed framework improves code correctness and efficiency by integrating preference learning into code generation.

Copied to clipboard

Yunlong Feng, Yang Xu, Libo Qin, Yasheng Wang, Wanxiang Che

SciDoc2Diagrammer-MAF: Towards Generation of Scientific Diagrams from Documents guided by Multi-Aspect Feedback Refinement (2024.findings-emnlp)

Challenge:	Large-scale high-quality training data is important for improving the performance of models.
Approach:	They propose a framework that motivates the model to automatically generate rationales on existing datasets and improves the performance of reasoning through reinforcement learning.
Outcome:	The proposed model outperforms InstructGPT on multiple reasoning datasets and outperformed InstructGPT on other datasets.

Copied to clipboard

Ishani Mondal, Zongxia Li, Yufang Hou, Anandhavelu Natarajan, Aparna Garimella, Jordan Boyd-Graber

GENIE: Toward Reproducible and Standardized Human Evaluation for Text Generation (2022.emnlp-main)

Challenge:	Current text-to-image models struggle with generating accurate diagrams from long-context inputs.
Approach:	They propose a task that extracts relevant information from scientific papers and generates diagrams based on user intentions using intermediate code generation.
Outcome:	The proposed task outperforms existing models on factual correctness and visual appeal and outperfies existing ones on automatic and human judgement.

Copied to clipboard

Daniel Khashabi, Gabriel Stanovsky, Jonathan Bragg, Nicholas Lourie, Jungo Kasai, Yejin Choi, Noah A. Smith, Daniel Weld

More DWUGs: Extending and Evaluating Word Usage Graph Datasets in Multiple Languages (2024.emnlp-main)

Challenge:	Effective human evaluation of text generation tasks remains an important, open area for research.
Approach:	They propose a system for running standardized human evaluations across different generation tasks.
Outcome:	The proposed system produces standardized human evaluations across tasks . it crowdsources predictions and ranks systems on leaderboards . the proposed system is not reproducible over time and different annotator populations .

Copied to clipboard

Dominik Schlechtweg, Pierluigi Cassotti, Bill Noble, David Alfter, Sabine Schulte Im Walde, Nina Tahmasebi

Prompt-R1: Collaborative Automatic Prompting Framework via End-to-end Reinforcement Learning (2026.findings-acl)

Challenge:	Word Usage Graphs (WUGs) represent word sense clusters from simple pairwise word use judgments.
Approach:	They propose to use a weighted graph to represent human semantic proximity judgments for pairs of word uses to infer word sense clusters from simple pairwise word use judgments.
Outcome:	The proposed approach can be applied in a Word Sense Induction (WSI) setting or for Word sense disambiguation (WSD) it is the first and to date largest manually annotated, diachronic WUG dataset.

Copied to clipboard

Wenjin Liu, Haoran Luo, Xueyuan Lin, Haoming Liu, Tiesunlong Shen, Jiapu Wang, Rui Mao, Erik Cambria

Low-Resource NLP

Challenge:	Existing large language models are limited in understanding, reasoning, calculation, and generation, limiting their performance in complex reasoning and dynamic tasks.
Approach:	They propose a plug-and-play framework that integrates a small-scale LLM (as agent) with large-scale large-level LLMs (a as environment) they propose generating prompts that are used to interact with LLM, and a double constraint reward that optimizes correctness and quality of generation.
Outcome:	The proposed framework significantly outperforms baseline large-scale large-language models across various tasks.

Can Automatic Metrics Assess High-Quality Translations? (2024.emnlp-main)

Copied to clipboard

Sweta Agrawal, António Farinhas, Ricardo Rei, Andre Martins

Machine Translation

Challenge:	a recent human evaluation study found that translations produced by current MT systems achieve very high-quality scores when judged by humans on a direct assessment scale of 0 to 100.
Approach:	They stress-test the ability of current translation quality metrics to detect correct translations . they show that current metrics often over or underestimate translation quality .
Outcome:	The proposed method overestimates translation quality, the authors show . they show that current metrics often overestimate translation quality .

FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation (2024.findings-acl)

Copied to clipboard

Tu Vu, Mohit Iyyer, Xuezhi Wang, Noah Constant, Jerry Wei, Jason Wei, Chris Tar, Yun-Hsuan Sung, Denny Zhou, Quoc Le, Thang Luong

ActPlan-1K: Benchmarking the Procedural Planning Ability of Visual Language Models in Household Activities (2024.emnlp-main)

Challenge:	Modern large language models often "hallucinate" plausible but factually incorrect information, which reduces their trustworthiness especially in settings where accurate and up-to-date information is critical.
Approach:	They develop a human evaluation procedure to measure correctness and hallucination and use it to benchmark both closed and open-source LLMs.
Outcome:	The proposed method outperforms both competing search engine-augmented prompting methods and commercial systems on search-augmented QA.

Copied to clipboard

Ying Su, Zhan Ling, Haochen Shi, Cheng Jiayang, Yauwai Yim, Yangqiu Song

A Needle in a Haystack: An Analysis of High-Agreement Workers on MTurk for Summarization (2023.acl-long)

Challenge:	Large language models (LLMs) have been adopted to process textual task description and accomplish procedural planning in embodied AI tasks because of their powerful reasoning ability.
Approach:	They propose to evaluate the planning ability of large language models and multi-modal counterfactual vision language models (VLMs) using a multi-factual household activity simulator and a chatGPT task description to evaluate their reasoning ability.
Outcome:	The proposed benchmark evaluates the planning ability of multi-modal and counterfactual vision language models on a household activity simulator and a chatGPT task description.

Copied to clipboard

Lining Zhang, Simon Mille, Yufang Hou, Daniel Deutsch, Elizabeth Clark, Yixin Liu, Saad Mahamood, Sebastian Gehrmann, Miruna Clinciu, Khyathi Raghavi Chandu, João Sedoc

GeoDRL: A Self-Learning Framework for Geometry Problem Solving using Reinforcement Learning in Deductive Reasoning (2023.findings-acl)

Challenge:	Using crowdsourcing, it is difficult to obtain high-quality annotations for difficult tasks.
Approach:	They propose a recruitment pipeline to recruit high-quality Amazon Mechanical Turk workers . they filter out subpar workers before they carry out the evaluations .
Outcome:	The proposed method can filter out subpar workers before they carry out evaluations and obtain high-agreement annotations with similar constraints on resources.

Copied to clipboard

Shuai Peng, Di Fu, Yijun Liang, Liangcai Gao, Zhi Tang

Why Uncertainty Estimation Methods Fall Short in RAG: An Axiomatic Analysis (2025.findings-acl)

Challenge:	Existing methods for automated geometry problem solving lack labeled data.
Approach:	They propose a framework that integrates logic graph deduction and deep reinforcement learning to optimize geometry reasoning as a Markov Decision Process.
Outcome:	The proposed framework improves accuracy and interpretability in the Geometry3K dataset while maintaining correctness.

Copied to clipboard

Heydar Soudani, Evangelos Kanoulas, Faegheh Hasibi

Calibrating LLMs for Text-to-SQL Parsing by Leveraging Sub-clause Frequencies (2025.emnlp-main)

Challenge:	Existing UE methods cannot reliably estimate the correctness of LLM responses in Retrieval-Augmented Generation (RAG) . Existing methods generate low uncertainty values without considering relevance of context to query .
Approach:	They propose an axiomatic framework to identify deficiencies in existing UE methods and introduce five constraints that an effective UE method should meet after incorporating retrieved documents into the LLM’s prompt.
Outcome:	The proposed framework satisfies all the axioms and improves correlation between uncertainty estimates and correctness.

Copied to clipboard

Terrance Liu, Shuyi Wang, Daniel Preotiuc-Pietro, Yash Chandarana, Chirag Gupta

Generate, Discriminate, Evolve: Enhancing Context Faithfulness via Fine-Grained Sentence-Level Self-Evolution (2025.findings-acl)

Challenge:	Large language models (LLMs) exhibit unexpected failures in which they are confidently incorrect.
Approach:	They propose a method for calibrating SQL outputs that leverages structured nature to provide more granular signals of correctness.
Outcome:	The proposed method improves on two popular text-to-SQL datasets and provides a confidence score that is calibrated.

Copied to clipboard

Kun Li, Tianhua Zhang, Yunxiang Li, Hongyin Luo, Abdalla Mohamed Salama Sayed Moustafa, Xixin Wu, James R. Glass, Helen M. Meng

Challenge:	Existing methods to improve context faithfulness in large language models are either inadequate or overlook the potential for self-improvement.
Approach:	They propose a framework that enhances context faithfulness through fine-grained sentence-level optimization.
Outcome:	Experiments on ASQA and ConFiQA datasets show that GenDiE surpasses baselines in faithfulness and correctness and exhibits robust performance for domain adaptation.

Temporally Consistent Factuality Probing for Large Language Models (2024.emnlp-main)

Copied to clipboard

Ashutosh Bajpai, Aaryan Goyal, Atif Anwer, Tanmoy Chakraborty

Can LLMs replace Neil deGrasse Tyson? Evaluating the Reliability of LLMs as Science Communicators (2024.emnlp-main)

Challenge:	Large Language Models (LLMs) are used as an alternative knowledge base for many tasks.
Approach:	They propose a temporally consistent factuality probe task that extends the consistency probe in the temporal dimension.
Outcome:	The proposed task extends the definitions of existing metrics to represent consistent factuality across temporal dimension.

Copied to clipboard

Prasoon Bajpai, Niladri Chatterjee, Subhabrata Dutta, Tanmoy Chakraborty

Zero-shot Graph Reasoning via Retrieval Augmented Framework with LLMs (2025.findings-emnlp)

Challenge:	Large Language Models (LLMs) and AI assistants are experiencing exponential growth in usage among expert and amateur users.
Approach:	They propose to assess the reliability of current Large Language Models as science communicators . they use a dataset comprising 742 Yes/No queries embedded in complex scientific concepts .
Outcome:	The proposed model outperforms open-access models in scientific question-answering tasks . the model outpersforms GPT-4 Turbo models in many evaluation aspects .

Copied to clipboard

Hanqing Li, Sharika Mahadevan, Kiran Jyothi Sheena, Henry Liang, Diego Klabjan

Following Occam’s Razor: Dynamic Combination of Structured Knowledge for Multi-Hop Question Answering using LLMs (2025.findings-emnlp)

Challenge:	Existing methods that require extensive finetuning or depend on predefined algorithms are limited by training.
Approach:	a new retrieval-augmented framework is proposed that harnesses retrieval and large language models to address graph reasoning tasks.
Outcome:	The proposed method achieves 100% accuracy on most graph reasoning tasks while maintaining consistent token costs regardless of graph sizes.

Copied to clipboard

Wei Chen, Zhi Zheng, Lili Zhao, Huijun Hou, Tong Xu

Multi-Attribute Steering of Language Models via Targeted Intervention (2025.acl-long)

Challenge:	Multi-hop question answering is a challenging task that requires capturing information from multiple positions in multiple documents.
Approach:	They propose a framework for integrating text-based and triple-based paradigms that incorporates structured knowledge into large-scale question answering.
Outcome:	The proposed framework improves multi-hop question answering by incorporating structured knowledge into the models.

Copied to clipboard

Duy Nguyen, Archiki Prasad, Elias Stengel-Eskin, Mohit Bansal

STRICT: Stress-Test of Rendering Image Containing Text (2025.emnlp-main)

Challenge:	Existing approaches for steering large language models fail to scale to multi-attribute settings with conflicts, such as enhancing helpfulness while also reducing toxicity.
Approach:	They propose a steering framework for selective token-level intervention across multiple attributes that enforcing sparsity and orthogonality among vectors for different attributes.
Outcome:	The proposed framework outperforms existing ITI and parameter-efficient fine-tuning approaches across question answering tasks and generative tasks.

Copied to clipboard

Tianyu Zhang, Xinyu Wang, Lu Li, Zhenghan Tai, Jijun Chi, Jingrui Tian, Hailin He, Suyuchen Wang

Step-by-Step Reasoning to Solve Grid Puzzles: Where do LLMs Falter? (2024.emnlp-main)

Challenge:	Despite the advances in diffusion models, the generation of coherent text remains a major bottleneck.
Approach:	They propose a benchmark to test the ability of diffusion models to render coherent text in images.
Outcome:	The proposed model fails to generate coherent and legible text in images despite its iterative nature . the model fails in both the maximum length of readable text and correctness and legibility of the generated text .

Copied to clipboard

Nemika Tyagi, Mihir Parmar, Mohith Kulkarni, Aswin Rrv, Nisarg Patel, Mutsumi Nakamura, Arindam Mitra, Chitta Baral

Unsupervised Hallucination Detection by Inspecting Reasoning Processes (2025.emnlp-main)

Challenge:	Existing studies evaluate only the final predicted answer of a puzzle, without providing any finer metrics to evaluate them.
Approach:	They propose to use a grid-based evaluation dataset to evaluate LLMs' reasoning abilities and a new error taxonomy to evaluate their reasoning chains.
Outcome:	The proposed model outperforms existing prompting methods on a wide range of natural language understanding tasks previously thought to be exclusive to humans.

Copied to clipboard

Ponhvoan Srey, Xiaobao Wu, Anh Tuan Luu

Neural Chain-of-Thought Search: Searching the Optimal Reasoning Path to Enhance Large Language Models (2026.findings-acl)

Challenge:	Unsupervised hallucination detection aims to identify hallucines generated by large language models without relying on labeled data.
Approach:	They propose an unsupervised method to detect hallucinated content by large language models . they use internal representations intrinsic to factual correctness to prompt the model to verify the truthfulness of a given statement .
Outcome:	The proposed framework outperforms existing unsupervised methods and is fully unsupervised and low cost.

Copied to clipboard

Guoming Ling, Zhongzhan Huang, Yupei Lin, Junxin Li, Shanshan Zhong, Hefeng Wu, Liang Lin

MEDEC: A Benchmark for Medical Error Detection and Correction in Clinical Notes (2025.findings-acl)

Challenge:	Recent research indicates that Large Reasoning Models suffer from a strategic bottleneck at reasoning path planning.
Approach:	They propose a framework that reformulates reasoning as a dynamic search for the optimal thinking strategy.
Outcome:	The proposed framework improves accuracy and computational cost while reducing generation length by over 22%.

Copied to clipboard

Asma Ben Abacha, Wen-wai Yim, Yujuan Fu, Zhaoyi Sun, Meliha Yetisgen, Fei Xia, Thomas Lin

SLM-Bench: A Comprehensive Benchmark of Small Language Models on Environmental Impacts (2025.findings-emnlp)

Challenge:	Several studies have shown that large language models can answer medical questions correctly, outperforming the average human score in some medical exams.
Approach:	They introduce MEDEC, the first publicly available benchmark for medical error detection and correction in clinical notes.
Outcome:	The proposed model outperforms medical doctors in errors detection and correction tasks.

Copied to clipboard

Nghiem Thanh Pham, Tung Kieu, Duc Manh Nguyen, Son Ha Xuan, Nghia Duong-Trung, Danh Le-Phuoc

ReLook: Vision-Grounded RL with a Multimodal LLM Critic for Agentic Web Coding (2026.acl-long)

Challenge:	SLMs offer computational efficiency and accessibility, yet a systematic evaluation of their performance and environmental impact remains lacking.
Approach:	SLM-Bench evaluates 15 SLMs on 9 NLP tasks using 23 datasets . compared accuracy, computational efficiency, and sustainability metrics .
Outcome:	SLM-Bench evaluates 15 SLMs on 9 NLP tasks using 23 datasets spanning 14 domains.

Copied to clipboard

Yuhang Li, Chenchen Zhang, Ruilin Lv, Ao Liu, Ken Deng, Yuanxing Zhang, Jiaheng Liu, Bo Zhou

Diffuse Thinking: Exploring Diffusion Language Models as Efficient Thought Proposers for Reasoning (2026.acl-long)

Challenge:	Large Language Models (LLMs) excel at algorithmic code generation, but front-end development is lacking in visual fidelity and interaction.
Approach:	They propose an agentic, vision-grounded reinforcement learning framework that closes a loop by invoking a multimodal LLM as a tool.
Outcome:	The proposed framework outperforms baselines in front-end code generation.

Copied to clipboard

Chenyang Shao, Sijian Ren, Fengli Xu, Yong Li

From “Thinking” to “Justifying”: Aligning High-Stakes Explainability with Professional Communication Standards (2026.findings-acl)

Challenge:	Large language models (LLMs) have demonstrated strong capabilities in complex reasoning tasks, yet their autoregressive generation paradigm makes it computationally prohibitive to explore diverse reasoning paths.
Approach:	They propose a framework that combines diffusion-based generation with autoregressive evaluation to efficiently generate diverse intermediate reasoning thoughts and employ LLMs as evaluators to assess and select candidates based on their plausibility and correctness.
Outcome:	The proposed framework improves inference efficiency while maintaining competitive or superior reasoning accuracy.

Copied to clipboard

Chen Qian, Yimeng Wang, Yu Chen, Lingfei Wu, Andreas Stathopoulos

Explainability & Interpretability in NLP

Challenge:	Existing explanations for large language models (LLMs) need to be able to verify outputs.
Approach:	They propose a method that constrains output communication to present a conclusion before its structured justification.
Outcome:	The proposed approach achieves 83.9% accuracy and correctness over CoT.

LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMs (2025.findings-acl)

Copied to clipboard

Omkar Thawakar, Dinura Dissanayake, Ketan Pravin More, Ritesh Thawkar, Ahmed Heakl, Noor Ahsan, Yuhao Li, Ilmuz Zaman Mohammed Zumri, Jean Lahoud, Rao Muhammad Anwer, Hisham Cholakkal, Ivan Laptev, Mubarak Shah, Fahad Shahbaz Khan, Salman Khan

LLM Agents Making Agent Tools (2025.acl-long)

Challenge:	Existing approaches do not emphasize step-wise problem-solving.
Approach:	They propose a visual reasoning chain benchmark and a fine-grained reasoning metric that evaluates correctness and logical coherence at each step.
Outcome:	The proposed framework outperforms existing models in six benchmarks and is 5x faster during inference scaling.

Copied to clipboard

Georg Wölflein, Dyke Ferber, Daniel Truhn, Ognjen Arandjelovic, Jakob Nikolas Kather

From RAG to Agentic RAG for Faithful Islamic Question Answering (2026.findings-acl)

Challenge:	Large language models (LLMs) can perform multi-step tasks by dynamically utilising external software components.
Approach:	They propose an agentic framework that autonomously transforms papers with code into LLM-compatible tools.
Outcome:	The proposed framework outperforms current state-of-the-art software engineering agents in 80% of tasks and is openly available on GitHub.

Copied to clipboard

Gagan Bhatia, Hamdy Mubarak, Mustafa Jarrar, George Mikros, Fadi Zaraket, Mahmoud Alhirthani, Mutaz al-Khatib, Logan Cochrane, Kareem Mohamed Darwish, Rashid Yahiaoui, Firoj Alam

CheMM-R1: Enhancing Chemical Structure Recognition and Elucidation with Reasoning Multimodal Large Language Models (2026.findings-acl)

Challenge:	Large Language Models (LLMs) are increasingly used for Islamic question answering, where ungrounded responses may carry serious religious consequences.
Approach:	They propose a bilingual, bilingual, Arabic/English benchmark with atomic single-gold answers that measures hallucination and abstention.
Outcome:	The proposed model improves accuracy and robustness even with a small model.

Copied to clipboard

Liting Huang, Zhihao Zhang, Shoujin Wang

CLARity: Reasoning Consistency Alone Can Teach Reinforced Experts (2026.acl-long)

Challenge:	Existing multimodal large language models lack domain-specific expertise to perform chemical tasks.
Approach:	They propose a benchmark dataset for evaluating multi-step multimodal reasoning capacities in the chemistry domain.
Outcome:	The proposed model surpasses existing models in all CheMM-Bench tasks.

Copied to clipboard

Jiuheng Lin, Cong Jiang, Zirui Wu, Jiarui Sun, Yansong Feng

A Framework of Reflective Agents with Adaptive Collaboration for Attributed Summary Generation (2026.findings-acl)

Challenge:	Existing solutions to supervise the reasoning process are prohibitively expensive.
Approach:	They propose a cost-effective reinforcement learning framework that enhances reasoning quality using a small, general-purpose LLM only.
Outcome:	Experiments show that CLARity improves reasoning quality by 16.5% over standard outcome-based reinforcement learning (RL) human evaluations confirm substantial gains in factual correctness and reasoning coherence, leading to more trustworthy model outputs.

Copied to clipboard

Yu Chen, Peng Chen, Ziwei Zheng, Bang Wang

Reasoning-Guided Exploration for Online DPO (2026.findings-acl)

Challenge:	Existing prompt-based summarization approaches face limitations such as positional preference, poor citation quality and sensitivity to uninformative documents.
Approach:	They propose a framework of Reflective Agents with Adaptive Collaboration for attributed summarization that performs iterative summarizing via reflective agents’ collaboration.
Outcome:	The proposed framework outperforms baselines on the ALCE benchmark in factual correctness and citation quality.

Copied to clipboard

Zetian Hu, Shunyu Liu, Ting-En Lin, Fei Huang, Yongbin Li, Dacheng Tao

EviReport: From Reasoned Outlines to Evidence Tracked Long-Form Reports (2026.findings-acl)

Challenge:	Recent work has aimed to enhance reasoning capabilities of language models, but these methods are limited to domains with objectively verifiable answers.
Approach:	They propose a self-play framework to improve reasoning on general-domain data.
Outcome:	Experiments show that the proposed framework improves reasoning performance on general-domain data while maintaining competitive performance on verifiable academic benchmarks.

Copied to clipboard

Zihan Liu, Jianhui li, Zexin Wang, Fei Sun, Jingjing LI, Zheyuan Li, Ke Xiang, Hang Cui, Houhua Gong, Changhua Pei, Gaogang Xie

CPR-RAG: Clinical Prior-Regularized Retrieval for Anatomy-Aware 3D CT Report Generation (2026.acl-long)

Challenge:	Evidence-intensive reports often produce fluent but under-supported drafts . eviReport is an evidence-grounded workflow for automated long-form report generation .
Approach:	They propose an evidence-tracked workflow that organizes corpus evidence into compact, traceable units and retrieves query-relevant subgraphs into retrieval-ready packages.
Outcome:	The proposed workflow outperforms baselines in factual coverage, factual accuracy and visual evidence integration.

Copied to clipboard

Sungkyu Yang, Kang-Min Kim, Mansu Kim

Forge: Quality-Aware Reinforcement Learning for NP-Hard Optimization in LLMs (2026.findings-acl)

Challenge:	Existing approaches to grounding radiology reports from 3D volumetric data are limited due to visual-semantic ambiguity and lack of "normal" context.
Approach:	They propose a model-agnostic retrieval-augmented generation framework that integrates clinical priors into the retrieval process.
Outcome:	The proposed model improves clinical efficacy across state-of-the-art models.

Copied to clipboard

Xiaozhe Li, Xinyu Fang, Shengyuan Ding, Yang Li, Linyang Li, Haodong Duan, Qingwen Liu, Kai Chen

DiFRa: A Unified Framework for Harmonizing Semantic Diversity and Factual Consistency in Question-Answer Generation (2026.findings-acl)

Challenge:	Existing benchmarks focus on correctness, overlooking optimality . large language models excel at math, coding, logic and puzzles .
Approach:	They propose a framework for training and evaluating Large Language Models on NP-hard optimization problems through quality-aware RLVR.
Outcome:	The proposed framework outperforms existing benchmarks on math, coding, logic and puzzles.

Copied to clipboard

Zhenqin Li, ShengYong Ding, Shuangyin Li

Where to show Demos in Your Prompt: A Positional Bias of In-Context Learning (2025.emnlp-main)

Challenge:	Question-Answer Generation (QAG) is essential for domain-specific large language models post-training.
Approach:	They propose a framework that balances semantic diversity and factual consistency . they propose entropy and consistency scores that harmonize the trade-off between diversity and correctness .
Outcome:	The proposed framework outperforms baseline models in generating diverse QA pairs . the proposed framework harmonizes semantic entropy and consistency scores to quantify trade-off between diversity and correctness.

Copied to clipboard

Kwesi Adu Cobbina, Tianyi Zhou

How Do Answer Tokens Read Reasoning Traces? Self-Reading Patterns in Thinking LLMs for Quantitative Reasoning (2026.findings-acl)

Challenge:	In-context learning (ICL) is a critical emerging capability of large language models (LLMs), enabling few-shot learning during inference by including a few demonstrations in the prompt.
Approach:	They propose to use positional bias to study ICL's performance for the first time by examining the positional variation in demos, system prompt, and user message in LLM input.
Outcome:	The proposed model can predict accuracy and accuracy when demos are placed at different positions in the input prompt and in the user message.

Copied to clipboard

Haoyang Chen, Yi Liu, Jianzhi Shao, Tao Zhang, Chengfu Huo, Wei Hu

Schoenfeld’s Anatomy of Mathematical Reasoning by Language Models (2026.acl-long)

Challenge:	Prior work on activation steering has focused on shaping reasoning traces, but it remains unclear how answer tokens actually read and integrate the reasoning to produce reliable outcomes.
Approach:	They propose a training-free steering method that uses self-reading quality scores to guide inference toward benign self-readiness and away from uncertain and disorganized reading.
Outcome:	The proposed method yields consistent accuracy gains in the reasoning traces generated by thinking LLMs.

Copied to clipboard

Ming Li, Chenrui Fan, Yize Cheng, Soheil Feizi, Tianyi Zhou

AscendKernelGen: LLM-Driven Kernel Generation for NPUs (2026.findings-acl)

Challenge:	Large language models expose reasoning traces, yet their underlying cognitive structure and steps remain difficult to identify and analyze beyond surface-level statistics.
Approach:	They propose a framework that explicitly abstracts reasoning traces into functional reasoning steps such as Analysis, Explore, Implement, Verify, etc.
Outcome:	The proposed framework reveals reproducible thinking dynamics and structural differences between reasoning and non-reasoning models, which are not apparent from token-level views.

Copied to clipboard

Xinzi Cao, Jianyang Zhai, Pengfei Li, Zhiheng Hu, Cen Yan, null Mubingxu, Guanghuan Fang, Bin She, Jiayu Li, Yihan Su, Dongyang Tao, Feidiao Yang, Chang-Dong Wang, Yutong Lu, Weicheng Xue, Bin Zhou, Yonghong Tian

Challenge:	Neural Processing Units (NPUs) are critical for AI infrastructure, but their development remains a bottleneck due to vendor-specific Domain-Specific Languages (DSLs).
Approach:	They propose a framework for NPU kernel development that bridges the gap in hardware-specific coding . compiler success on complex Level-2 kernels improves from 0% to 95.5%, they say .
Outcome:	The proposed framework bridges the gap in hardware-specific coding, showing a near-zero success rate on complex kernels.

ChemReason-Bench: Benchmarking Large Language Models for Procedural Reasoning in Experimental Chemistry (2026.acl-long)

Copied to clipboard

Jinwei Zhang, Xucheng Liang, Yu Zhang, Ruijie Yu, Xiaokang Yang, Yaohui Jin, Yanyan Xu

VEHME: A Vision-Language Model For Evaluating Handwritten Mathematics Expressions (2025.emnlp-main)

Challenge:	Experimental protocols in organic synthesis specify not only the intended transformation, but also an executable sequence of operations and conditions.
Approach:	They propose a human-validated benchmark for verifiable experimental procedure reasoning . they instantiate 7306 benchmark tasks across six complementary formats .
Outcome:	The proposed benchmarks show that the evaluations are less diagnostic of procedure-level decision making.

Copied to clipboard

Thu Phuong Nguyen, Duc M. Nguyen, Hyotaek Jeon, Hyunwook Lee, Hyunmin Song, Sungahn Ko, Taehwan Kim

Rethinking Text-to-SQL: Dynamic Multi-turn SQL Interaction for Real-world Database Exploration (2026.findings-acl)

Challenge:	VEHME is a vision language model for assessing handwritten math answers . traditional methods of assessing student work are limited by time constraints, class sizes and cognitive load .
Approach:	They propose a Vision-Language Model for Evaluating Handwritten Mathematics Expressions to assess handwritten math responses with high accuracy and interpretable reasoning traces.
Outcome:	VEHME achieves state-of-the-art performance among open-source models and approaches accuracy of proprietary systems.

Copied to clipboard

Linzhuang Sun, Tianyu Guo, Hao Liang, Ruitong Liu, Yuying Li, Qifeng Cai, Jingxuan Wei, Yuchen Wu, Bihui Yu, Xiangxiang Zhang, Wentao Zhang, Bin Cui

Demystifying Multi-Agent Debate: The Role of Confidence and Diversity (2026.findings-acl)

Challenge:	Structured Query Language (SQL) is the cornerstone for data-driven decision-making.
Approach:	They propose a benchmark to rigorously evaluate Large Language Models within a dynamic interaction framework.
Outcome:	The proposed benchmark aims to rigorously evaluate LLMs within a dynamic interaction framework.

Copied to clipboard

Xiaochen Zhu, Caiqi Zhang, Yizhou Chi, Tom Stafford, Nigel Collier, Andreas Vlachos

Challenge:	Multi-agent debate (MAD) is widely used to improve large language models' (LLMs) reasoning and test-time scaling.
Approach:	They propose a diversity-aware initialisation that selects a more diverse pool of candidate answers, increasing the likelihood that a correct hypothesis is present at the start of debate.
Outcome:	The proposed protocol outperforms vanilla MAD and majority vote on six reasoning-oriented QA benchmarks.

Example Quality Matters: Multi-Aspects Example Augmentation for Private Library Programming (2026.acl-long)

Copied to clipboard

Yuhao Li, Haifeng Sun, Xuesong Zhang, Shu Yao, Haoyu Zheng, Yvchuan Wang, Huazheng Wang, Zirui Zhuang, Qi Qi, Jianxin Liao, Jingyu Wang

TounsiBench: Benchmarking Large Language Models for Tunisian Arabic (2025.emnlp-main)

Challenge:	Existing approaches to code generation fail to consider the quality of retrieved examples.
Approach:	They propose a retrieval-augmented generation method that combines existing API examples to improve complexity and readability.
Outcome:	The proposed method achieves up to 22% accuracy improvement over baseline methods.

Copied to clipboard

Souha Ben Hassine, Asma Arrak, Marouene Addhoum, Steven R Wilson

POSTCONDBENCH: Benchmarking Correctness and Completeness in Formal Postcondition Inference (2026.findings-acl)

Challenge:	a dataset of Tunisian Arabic instructions and prompts is used to evaluate LLMs' ability to understand and generate responses in Tunisia . we assess the quality, correctness, relevance, and dialectal adherence of LLM responses .
Approach:	They propose a benchmark for evaluating the capabilities of large language models in Tunisian Arabic . they use a dataset of Tunisia Arabic instructions and prompts to evaluate their models .
Outcome:	The proposed model can judge quality, correctness, relevance, and dialectal adherence . the model can also generate a leaderboard for the Tunisian Arabic language .

Copied to clipboard

Gehao Zhang, Juan Zhai

Challenge:	Existing benchmarks emphasize correctness under limited evaluation settings . evaluation of formal specifications is time-consuming, errorprone and requires substantial expertise.
Approach:	They propose a multilingual benchmark for evaluating method-level postcondition generation from real-world software.
Outcome:	The proposed benchmarks show that evaluation remains a key bottleneck . 420 Python and Java tasks are paired with a high-quality postcondition set .

Thinking Traps in Long Chain-of-Thought: A Measurable Study and Trap-Aware Adaptive Restart (2026.findings-acl)

Copied to clipboard

null Chenkang, Fan Yu, Junjie Nian, Sihan Zhao, Zhuoka Feng, Zijun Yao, Wang Heng, Yu Minshen, Yixin Cao

Reducing Hallucinations in LLMs via Factuality-Aware Preference Learning (2026.findings-acl)

Challenge:	Experiments show that extended generation does not guarantee correctness . a recurring pattern in Long-CoT failures is a problem for large reasoning models .
Approach:	They propose a test-time control framework that truncates the trajectory before the trap segment and adaptively restarts decoding.
Outcome:	Experiments show that TAAR improves reasoning performance without fine-tuning model parameters.

Copied to clipboard

Sindhuja Chaduvula, Ahmed Y. Radwan, Azib Farooq, Yani Ioannou, Shaina Raza

Entropy-Aware Reshaping of Reinforcement Signals for Multi-Answer Reasoning (2026.findings-acl)

Challenge:	Preference alignment methods can reinforce hallucinations when preference judgments reward fluency and confidence over factual correctness.
Approach:	They propose a method that corrects misordered preference pairs and adds a factuality-aware margin to emphasize pairs with clear correctness differences.
Outcome:	The proposed method improves factuality and reduces hallucination rates across seven open-weight LLMs.

Copied to clipboard

Zhi Li, Huidan Xu, Zhen Hu, Yali Du, Ying Liu

LLMs as Lab Engineers: A Benchmark for Analytical Method Lifecycle Management (2026.findings-acl)

Challenge:	Reinforcement learning with verifiable rewards (RLVR) is a standard post-training paradigm for large language models.
Approach:	They propose a framework that reshapes how learning signals are normalized and aggregated.
Outcome:	Experiments on MCTACO and MMLU-Multi show that the proposed framework improves accuracy, training stability and cross-dataset transfer performance.

Copied to clipboard

Xiaoyi Chen, Mahsa Monshizadeh, Chaoqi Zhang, Jianjun Lang, Yang Wu, Genevieve Mortensen, Xiaozhong Liu, Haixu Tang

Automatic Paper Analysis and Categorisation for Systematic Reviews with Combined Reasoning-Augmented SFT and DAPO RL (2026.findings-acl)

Challenge:	General-purpose commercial models outperform domain-specialized ones, while RAG and reasoning significantly improve performance.
Approach:	They propose a benchmark to evaluate LLMs' capabilities in analytical chemistry scenarios.
Outcome:	The proposed framework outperforms existing benchmarks focused on factual knowledge and provides practical guidance for analytical chemistry challenges.

Copied to clipboard

Michela Lorandi, Anya Belz, Simon Mille, Craig Thomson

AIPO: Adaptive Information Guided Token-Level Reinforcement Learning for Large Language Model Reasoning (2026.acl-long)

Challenge:	Automating systematic reviews is expensive and time consuming, a study finds . automatic approaches are being explored but their performance has been poor .
Approach:	They propose to use reasoning-enhanced fine-tuning and DAPO reinforcement learning to automate systematic reviews.
Outcome:	The proposed methods significantly improve the performance of LLMs, the authors find . they find that reasoning-enhanced fine-tuning reduces time required for annotation by 80% .

Copied to clipboard

Bin Chen, Hongfei Ye, Huiyang Wang, Wenxi Liu, Yu Zhang, Furui Liu

The Confidence Paradox: Unveiling the Latent Discriminative Power of Diffusion Large Language Models in Mathematical Reasoning (2026.findings-acl)

Challenge:	Existing RLVR methods focus on all generated tokens rather than on which tokens contribute to reasoning.
Approach:	They propose to use a Random–Fourier approximation of the Hilbert–Schmidt Independence Criterion to focus updates on decisive tokens discovered on the fly to improve the efficiency of mutual-information estimation.
Outcome:	The proposed approach yields +20% accuracy over strong RLVR baselines while updating merely 10% of tokens, demonstrating superior efficiency and effectiveness.

Copied to clipboard

Yansi Li, Gongshen Liu, Zhuosheng Zhang