Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers)

ChaI-TeA: A Benchmark for Evaluating Autocompletion of Interactions with LLM-based Chatbots (2025.naacl-short)

Challenge:	Existing work proposes dialect adaptation for encoder models or encoder-decoder models.
Approach:	They propose to use MD-3 to combine task adapters and dialect adapters to decoder models using a masked word game-playing conversation.
Outcome:	The proposed architecture outperforms baselines on Indian English and Nigerian English on a masked conversation with two models.

Copied to clipboard

Shani Goren, Oren Kalinsky, Tomer Stav, Yuri Rapoport, Yaron Fairstein, Ram Yazdi, Nachshon Cohen, Alexander Libov, Guy Kushilevitz

Natural Language Interfaces

Challenge:	Large Language Models (LLMs) have revolutionized many NLP applications.
Approach:	They propose an autocomplete evaluation framework for LLM-based chatbot interactions that includes a formal definition of the task and suitable metrics.
Outcome:	The proposed framework evaluates 11 models on a task that performs fairly but still lacks the ranking of the generated suggestions.

Cross-Lingual Transfer Learning for Speech Translation (2025.naacl-short)

Copied to clipboard

Rao Ma, Mengjie Qian, Yassir Fathullah, Siyuan Tang, Mark Gales, Kate Knill

Reverse Question Answering: Can an LLM Write a Question so Hard (or Bad) that it Can’t Answer? (2025.naacl-short)

Challenge:	Increasing interest in building multilingual foundation models for NLP and speech research has led to limited data collection for training ST systems.
Approach:	They propose to use Whisper to explore the behavior of multilingual speech foundation models with restricted data.
Outcome:	The proposed model can translate to Chinese with a single language, and it can perform transcriptions in other languages.

Copied to clipboard

Nishant Balepur, Feng Gu, Abhilasha Ravichander, Shi Feng, Jordan Lee Boyd-Graber, Rachel Rudinger

Natural Language Interfaces

Challenge:	Question answering (QA) is a popular task, but we test both separately . a recent study found that LLMs are less accurate in numerical RQA than RQA .
Approach:	We run 16 LLMs on QA and RQA with trivia questions/answers . they find question and answer types that lead to RQA errors and suggest improvements .
Outcome:	The results show that LLMs are less accurate in RQA for numerical answers than RQA . RQA errors correlate with question difficulty and inversely correlate with answer frequencies .

Personalized Help for Optimizing Low-Skilled Users’ Strategy (2025.naacl-short)

Copied to clipboard

Feng Gu, Wichayaporn Wongkamjan, Jordan Lee Boyd-Graber, Jonathan K. Kummerfeld, Denis Peskoff, Jonathan May

Challenge:	a natural language agent generates moves and messages based on player intentions . a dozen games with novice and experienced players generate useful advice .
Approach:	a team of researchers augment a natural language agent to generate move and message advice . they use a game to simulate the intentions of novice and experienced players .
Outcome:	The enhanced agent generates move and message advice based on player intentions . the agent helps novices compete with experienced players and even surpass them .

Local Prompt Optimization (2025.naacl-short)

Copied to clipboard

Yash Jain, Vishal Chowdhary

Cross-lingual Transfer of Reward Models in Multilingual Alignment (2025.naacl-short)

Challenge:	Existing prompt optimization methods optimize prompts globally, but they lack the correct words for a task.
Approach:	They propose a local prompt optimization process that integrates with any general automatic prompt engineering method to optimize a prompt over a large vocabulary.
Outcome:	The proposed method improves on Math Reasoning and BIG-bench Hard benchmarks and shows that it can converge to the optimal prompt faster than global methods.

Copied to clipboard

Jiwoo Hong, Noah Lee, Rodrigo Martínez-Castaño, César Rodríguez, James Thorne

Inference-Time Selective Debiasing to Enhance Fairness in Text Classification Models (2025.naacl-short)

Challenge:	Recent studies in reward modeling schemes are skewed towards English, limiting the applicability of RLHF in multilingual alignments.
Approach:	They investigate cross-lingual transfer of English RMs by representation shifts . they also analyze cross-linguistic transfer of RM through the representation shift .
Outcome:	The results show that English RMs can be transferred across languages by 34% .

Copied to clipboard

Gleb Kuzmin, Neemesh Yadav, Ivan Smirnov, Timothy Baldwin, Artem Shelmanov

Automatic Evaluation of Healthcare LLMs Beyond Question-Answering (2025.naacl-short)

Challenge:	Several studies have investigated and promoted fairness, and a variety of definitions have been proposed to address this problem.
Approach:	They propose a selective debiasing method that removes bias from model predictions instead of discarding them at inference time.
Outcome:	The proposed method achieves better results than standard uncertainty quantification methods on text classification datasets with encoder-based classification models.

Copied to clipboard

Anna Arias-Duart, Pablo Agustin Martin-Torres, Daniel Hinjos, Pablo Bernabeu-Perez, Lucia Urcelay Ganzabal, Marta Gonzalez Mallo, Ashwin Kumar Gururajan, Enrique Lopez-Cuena, Sergio Alvarez-Napagao, Dario Garcia-Gasulla

Question Answering

Challenge:	Current Large Language Models (LLMs) benchmarks are often based on open-ended or close-ended QA evaluations, avoiding the requirement of human labor.
Approach:	They propose a multi-axis suite for healthcare LLM evaluation, exploring correlations between open and close benchmarks and metrics.
Outcome:	The proposed framework explores correlations between open and close benchmarks and metrics in the healthcare domain, with blind spots and overlaps in existing methodologies.

STRUX: An LLM for Decision-Making with Structured Explanations (2025.naacl-short)

Copied to clipboard

Yiming Lu, Yebowen Hu, Hassan Foroosh, Wei Jin, Fei Liu

Improving Vietnamese-English Cross-Lingual Retrieval for Legal and General Domains (2025.naacl-short)

Challenge:	a new LLM decision-making framework is designed to help users understand how and why decisions are made.
Approach:	They introduce a new LLM decision-making framework called STRUX that provides structured explanations for LLM decisions.
Outcome:	The proposed framework improves decision-making by providing structured explanations . it has been evaluated on the task of forecasting stock investment decisions based on earnings call transcripts - superior performance against strong baselines compared with previous frameworks based upon earnings call transcriptions demonstrating superior performance .

Copied to clipboard

Toan Ngoc Nguyen, Nam Le Hai, Nguyen Doan Hieu, Dai An Nguyen, Linh Ngo Van, Thien Huu Nguyen, Sang Dinh

Information Retrieval

Challenge:	Existing document retrieval systems focus on a single language, targeting resource-rich languages like English or Chinese.
Approach:	They propose auxiliary loss function and symmetrical training strategy for cross-lingual retrieval between Vietnamese and English . they propose a dataset that covers the general domain and extends to the legal field .
Outcome:	The proposed dataset significantly improves state-of-the-art models on cross-lingual retrieval tasks.

Computational Discovery of Chiasmus in Ancient Religious Text (2025.naacl-short)

Copied to clipboard

Hope McGovern, Hale Sirin, Tom Lippincott

Representation Learning

Challenge:	chiasmus, or chiastic units, is a debated literary device in biblical texts . a computational approach to detect chiastes is shown to be efficient, but not efficient .
Approach:	They propose a computational approach to detect chiasmus within Biblical passages . they leverage neural embeddings to capture lexical and semantic patterns associated with chiastics - using annotators to review a subset of the detected patterns.
Outcome:	The proposed method achieves high inter-annotator agreement and system accuracy of 0.80 at verse level and 0.60 at half-verse level.

Characterizing the Effects of Translation on Intertextuality using Multilingual Embedding Spaces (2025.naacl-short)

Copied to clipboard

Hope McGovern, Hale Sirin, Tom Lippincott

LLM2: Let Large Language Models Harness System 2 Reasoning (2025.naacl-short)

Challenge:	a new study characterizes the preservation of intertextuality across human and machine translations . intertextual references can range from direct quotation to semantic resemblance, both within and between texts .
Approach:	They use multilingual embedding spaces to characterize preservation of intertextuality . they use biblical texts, which are both full of inter textual references .
Outcome:	The proposed method characterizes preservation of intertextuality across human and machine translations.

Copied to clipboard

Cheng Yang, Chufan Shi, Siheng Li, Bo Shui, Yujiu Yang, Wai Lam

Context-Efficient Retrieval with Factual Decomposition (2025.naacl-short)

Challenge:	Empirical results on mathematical reasoning benchmarks substantiate the efficacy of Large language models (LLMs).
Approach:	They propose a framework that combines an LLM with a process-based verifier to generate plausible candidates and provide timely process-driven feedback to distinguish desirable and undesirable outputs.
Outcome:	Empirical results show that LLM2 improves accuracy on GSM8K and self-consistency increases major@20 accuracy.

Copied to clipboard

Yanhong Li, David Yunis, David McAllester, Jiawei Zhou

Green & Sustainable NLP

Challenge:	Existing models that use dynamically expanding text can be incorporated into large language models.
Approach:	They show that pre-processing external corpus into semi-structured "atomic facts" reduces the size of the context and improves inference efficiency.
Outcome:	The proposed form of atomic facts improves on question answering tasks when the amount of retrieved text is limited.

Sports and Women’s Sports: Gender Bias in Text Generation with Olympic Data (2025.naacl-short)

Copied to clipboard

Laura Biester

Alligators All Around: Mitigating Lexical Confusion in Low-resource Machine Translation (2025.naacl-short)

Challenge:	Large Language Models (LLMs) generate text that is stereotypical or not representative of the viewpoints and values of historically marginalized demographic groups.
Approach:	They propose to use data from the Olympic Games to investigate gender bias in large language models.
Outcome:	The proposed model consistently biased against women when the gender is ambiguous in the prompt, revealing pervasive gender bias in LLMs in the context of athletics.

Copied to clipboard

Elizabeth Nielsen, Isaac Rayburn Caswell, Jiaming Luo, Colin Cherry

Text Generation

Challenge:	Current machine translation systems for low-resource languages have a particular failure mode: they tend to confuse words within a domain.
Approach:	They propose a recall-based metric to measure the failure mode of machine translation systems for low-resource languages.
Outcome:	The proposed model outperforms a lexicon-based translator in 122 low-resource languages.

PROM: Pivoted and Regulated Optimization for Multilingual Instruction Learning (2025.naacl-short)

Copied to clipboard

Jaeseong Lee, Seung-won Hwang, Hojin Lee, Yunju Bak, Changmin Lee

Concept-Reversed Winograd Schema Challenge: Evaluating and Improving Robust Reasoning in Large Language Models via Abstraction (2025.naacl-short)

Challenge:	Existing solutions to large language models (LLMs) are English-centric, hindering their application to 6500+ existing languages.
Approach:	They propose to append English tuning data with its translated pair to solve this problem . they identify English as an internal pivot language and propose to regulate between them .
Outcome:	The proposed model is able to generalize on multiple benchmarks across different languages.

Copied to clipboard

Kaiqiao Han, Tianqing Fang, Zhaowei Wang, Yangqiu Song, Mark Steedman

Robustness in NLP

Challenge:	Recent research has revealed that Large Language Models (LLMs) often experience issues with hallucinations and unreliable reasoning due to semantic associations and superficial logical chains.
Approach:	They propose a concept-reversed Winograd Schema Challenge dataset to evaluate the robustness of Large Language Models (LLMs) they propose Abstraction-of-Thought (AoT) method for recovering adversarial cases to normal cases using conceptual abstraction to improve LLMs’ robustness and consistency in reasoning.
Outcome:	The proposed method improves LLMs’ robustness and consistency in reasoning under adversarial and long-tail scenarios.

Defense against Prompt Injection Attacks via Mixture of Encodings (2025.naacl-short)

Copied to clipboard

Ruiyi Zhang, David Sullivan, Kyle Jackson, Pengtao Xie, Mei Chen

Watching the AI Watchdogs: A Fairness and Robustness Analysis of AI Safety Moderation Classifiers (2025.naacl-short)

Challenge:	Large Language Models (LLMs) have emerged as a dominant approach for a wide range of NLP tasks, but external content embeds malicious instructions that manipulate the LLM’s output.
Approach:	They propose a mixture of encodings defense mechanism which utilizes multiple character encodes to degrade LLM performance on certain NLP tasks.
Outcome:	The proposed method achieves one of the lowest attack success rates under prompt injection attacks while maintaining high performance across all NLP tasks.

Copied to clipboard

Akshit Achara, Anshuman Chhabra

CoRAG: Collaborative Retrieval-Augmented Generation (2025.naacl-short)

Challenge:	ASM classifiers are designed to moderate content on social media platforms and serve as guardrails that prevent Large Language Models (LLMs) from being fine-tuned on unsafe inputs.
Approach:	They examine the fairness and robustness of four widely-used, closed-source ASM classifiers: OpenAI Moderation API, Perspective API, Google Cloud Natural Language (GCNL) API, and Clarifai API.
Outcome:	The classifiers do not unfairly classify content belonging to minority groups as unsafe compared to those belonging to majority groups and their behavior remains robust and consistent across similar inputs.

Copied to clipboard

Aashiq Muhamed, Mona T. Diab, Virginia Smith

Information Retrieval

Challenge:	Existing research on Retrieval-Augmented Generation models has focused on centralized settings where a single entity controls both the model and the datastore.
Approach:	They propose a framework for RAG where clients jointly train a shared model using a collaborative passage store.
Outcome:	The proposed framework outperforms parametric learning methods and locally trained models in low-resource scenarios.

Is It Navajo? Accurate Language Detection for Endangered Athabaskan Languages (2025.naacl-short)

Copied to clipboard

Ivory Yang, Weicheng Ma, Chunhui Zhang, Soroush Vosoughi

Text Classification

Challenge:	Endangered languages are significantly underrepresented in contemporary language technologies, exacerbating the challenges of their preservation and revitalization.
Approach:	They propose a random forest classifier trained on Navajo and 20 erroneously suggested languages by Google's Language Identification tool.
Outcome:	The proposed classifier achieves near-perfect accuracy across other Athabaskan languages suggesting its potential for broader application.

Don’t Touch My Diacritics (2025.naacl-short)

Copied to clipboard

Kyle Gorman, Yuval Pinter

Challenge:	a recent paper examines the effects of preprocessing text with diacritics on model performance . we show that inconsistent encoding of diacritized characters and removing diacritical characters can have detrimental downstream effects .
Approach:	They propose to improve the handling of diacritized text by preserving diacritics and removing them altogether.
Outcome:	The proposed approach reduces the number of errors in the preprocessing process, the authors argue . they show that the proposed approach can reduce the number and complexity of errors .

Pretrained Image-Text Models are Secretly Video Captioners (2025.naacl-short)

Copied to clipboard

Chunhui Zhang, Yiren Jian, Zhongyu Ouyang, Soroush Vosoughi

Reverse Modeling in Large Language Models (2025.naacl-short)

Challenge:	Current video captioning methods often incorporate intricate designs tailored to video inputs.
Approach:	They adapt an image-based captioning model to address dynamic video sequences without modifications.
Outcome:	The proposed model outperforms specialised captioning systems on major benchmarks.

Copied to clipboard

Sicheng Yu, Xu Yuanchen, Cunxiao Du, Yanying Zhou, Minghui Qiu, Qianru Sun, Hao Zhang, Jiawei Wu

Preserving Multilingual Quality While Tuning Query Encoder on English Only (2025.naacl-short)

Challenge:	Using pre-trained LLMs with reversed text inputs can improve their performance across multiple languages.
Approach:	They propose a way to determine whether LLMs can understand reversed text inputs by reversing entire paragraphs or documents at the token level.
Outcome:	The proposed model can be used to improve understanding across multiple languages.

Copied to clipboard

Oleg Vasilyev, Randy Sawaya, John Bohannon

Using Contextually Aligned Online Reviews to Measure LLMs’ Performance Disparities Across Language Varieties (2025.naacl-short)

Challenge:	Xiong et al., 2020, 2021b) and Dong eet., 2022) . Switching from one query encoder to another is easily feasible .
Approach:	They propose a general tuning technique that can be used to modify query representations for specific types of queries or domains while keeping precomputed and stored documents intact.
Outcome:	The proposed model preserves multilingual qualities and improves embedding qualities on different datasets.

Copied to clipboard

Zixin Tang, Chieh-Yang Huang, Tsung-che Li, Ho Yin Sam Ng, Hen-Hsen Huang, Ting-Hao Kenneth Huang

Challenge:	Of the world's 7,000 languages, sixty (60) million people speak British English, 23 million speak Taiwan Mandarin, and 10 million speak European Portuguese.
Approach:	They propose a contextually aligned dataset that captures comments in different languages from real-world scenarios.
Outcome:	The proposed approach shows that large language models underperform in Taiwan Mandarin in a sentiment analysis task.

Towards Federated Low-Rank Adaptation of Language Models with Rank Heterogeneity (2025.naacl-short)

Copied to clipboard

Yuji Byun, Jaeho Lee

Zenghao Duan, Wenbin Duan, Zhiyi Yin, Yinghan Shen, Shaoling Jing, Jie Zhang, Huawei Shen, Xueqi Cheng

Challenge:	Low-rank adaptation (LoRA) is an efficient alternative to full-weight adaptation in federated fine-tuning of language models, significantly reducing computational costs.
Approach:	They propose a low-rank adaptation method that freezes original weights and trains only the update parametrized as a product of two low-ranked matrices.
Outcome:	The proposed method accelerates convergence and enhances the global model’s predictive performance.

Copied to clipboard

STEP: Staged Parameter-Efficient Pre-training for Large Language Models (2025.naacl-short)

Challenge:	Existing knowledge editing methods struggle when tasked with editing multiple related knowledge pieces for the same subject.
Approach:	They propose a benchmark to assess the effectiveness of knowledge editing methods . they use same-subject edits to ensure comprehensive updates to entity-centric knowledge .
Outcome:	The proposed method over-relys on subject information, neglecting other critical factors, resulting in reduced editing effectiveness.

Copied to clipboard

Kazuki Yano, Takumi Ito, Jun Suzuki

Language Models Encode Numbers Using Digit Representations in Base 10 (2025.naacl-short)

Challenge:	Recent LLM development trends involve pre-training models with a vast number of parameters on massive datasets.
Approach:	They propose a method that integrates parameter-efficient tuning techniques with model growth to reduce memory requirements while maintaining equivalent performance.
Outcome:	The proposed method reduces memory requirements by 53.9% while maintaining equivalent performance to vanilla pre-trained models on downstream tasks.

Copied to clipboard

Amit Arnold Levy, Mor Geva

A Systematic Study of Cross-Layer KV Sharing for Efficient LLM Inference (2025.naacl-short)

Challenge:	Large language models (LLMs) often make errors when handling simple numerical tasks . a natural hypothesis is that these errors stem from how LLMs represent numbers .
Approach:	They propose to examine how LLMs represent numbers with circular representations per digit . they propose to use digit-wise representations to shed light on errors on numerical tasks .
Outcome:	The proposed model is internally represented with individual circular representations per-digit in base 10 . the proposed model could be used to analyze numerical mechanisms in large language models .

Copied to clipboard

You Wu, Haoyi Wu, Kewei Tu

AMPS: ASR with Multimodal Paraphrase Supervision (2025.naacl-short)

Challenge:	Recent studies have shown that sharing key-value (KV) cache across layers is effective in efficient inference of large language models.
Approach:	They propose a unified framework that covers several recent methods and their novel variants to investigate cross-layer KV sharing.
Outcome:	The proposed framework achieves higher throughput and better performance when reducing the size of the key-value cache by 2 while maintaining competitive performance.

Copied to clipboard

Abhishek Gupta, Amruta Parulekar, Sameep Chattopadhyay, Preethi Jyothi

Text Generation

Challenge:	Spontaneous or conversational multilingual speech presents many challenges for state-of-the-art automatic speech recognition systems.
Approach:	They propose a technique that augments a multilingual multimodal ASR system with paraphrase-based supervision for improved conversational ASR in multiple languages.
Outcome:	The proposed technique reduces word error rates by up to 5% on a state-of-the-art multimodal model .

Taxi1500: A Dataset for Multilingual Text Classification in 1500 Languages (2025.naacl-short)

Copied to clipboard

Chunlan Ma, Ayyoob Imani, Haotian Ye, Renhao Pei, Ehsaneddin Asgari, Hinrich Schuetze

Information Extraction & Text Mining

Challenge:	a large-scale text classification dataset encompassing 1504 languages is needed to address this gap . low-resource languages are often overlooked due to the scarcity of evaluation datasets.
Approach:	They propose to use translations of the Bible to construct a large-scale text classification dataset that covers 1504 languages and annotate them using crowdsourcing.
Outcome:	The proposed dataset covers 1504 languages and is available to the public.

GameTox: A Comprehensive Dataset and Analysis for Enhanced Toxicity Detection in Online Gaming Communities (2025.naacl-short)

Copied to clipboard

Usman Naseem, Shuvam Shiwakoti, Siddhant Bikram Shah, Surendrabikram Thapa, Qi Zhang

FaithBench: A Diverse Hallucination Benchmark for Summarization by Modern LLMs (2025.naacl-short)

Challenge:	Existing methods to detect toxic behavior in online gaming environments are limited by utterance-level annotation.
Approach:	They propose to annotate game chat utterances for toxicity detection through intent classification and slot filling.
Outcome:	The proposed model improves the detection of toxic speech in online gaming environments and reveals limitations of current models.

Copied to clipboard

Forrest Sheng Bao, Miaoran Li, Renyi Qu, Ge Luo, Erana Wan, Yujia Tang, Weisi Fan, Manveer Singh Tamber, Suleman Kazi, Vivek Sourabh, Mike Qi, Ruixuan Tu, Chenyu Xu, Matthew Gonzales, Ofer Mendelevitch, Amin Ahmad

Text Generation

Challenge:	Existing evaluations of hallucinations in large language models suffer from a lack of diversity and recency in the LLM and LLM families considered.
Approach:	They propose a summarization hallucination benchmark that challenges models to disagree on hallucines . they use models to generate answers or summaries from textual input .
Outcome:	The proposed model combines the best of 10 modern LLMs with ground truth annotations.

Debate-Feedback: A Multi-Agent Framework for Efficient Legal Judgment Prediction (2025.naacl-short)

Copied to clipboard

Xi Chen, Mao Mao, Shuo Li, Haotian Shangguan

Green & Sustainable NLP

Challenge:	Comparative experiments show that our model outperforms several general-purpose and domain-specific legal models.
Approach:	They propose a legal judgment prediction model that integrates LLMs with argumentative reasoning techniques to simulate the debate phase of real courtroom trials.
Outcome:	The proposed model outperforms several general-purpose and domain-specific legal models and offers a dynamic reasoning process.

Great Memory, Shallow Reasoning: Limits of kNN-LMs (2025.naacl-short)

Copied to clipboard

Shangyi Geng, Wenting Zhao, Alexander M Rush

Reasoning

Challenge:	Existing models trained on poor quality data have shown strong performance in language modeling and some downstream benchmarks.
Approach:	They evaluate kNN-LMs on a diverse set of tasks and evaluate their performance.
Outcome:	The proposed extension could improve on a variety of tasks, but it fails to perform on reasoning tasks that require integrating multiple pieces of information.

Repetition Neurons: How Do Language Models Produce Repetitions? (2025.naacl-short)

Copied to clipboard

Tatsuya Hiraoka, Kentaro Inui

STAR: Spectral Truncation and Rescale for Model Merging (2025.naacl-short)

Challenge:	Existing studies on text generation with LLMs focus on attention heads, framing repetition as a key mechanism for in-context learning.
Approach:	They introduce repetition neurons that are regarded as "skill neurons" responsible for the repetition problem in text generation tasks.
Outcome:	The authors identify repetition neurons as "skill neurons" that perceive repetition as a task to copy the previous context repeatedly, similar to in-context learning.

Copied to clipboard

Yu-Ang Lee, Ching-Yun Ko, Tejaswini Pedapati, I-Hsin Chung, Mi-Yen Yeh, Pin-Yu Chen

Task-driven Layerwise Additive Activation Intervention (2025.naacl-short)

Challenge:	Model merging is an efficient way of obtaining a multi-task model from several pretrained models without further fine-tuning.
Approach:	They propose a model merging technique that aims at mitigating "merging conflicts" by truncating small components in the respective spectral spaces and then an automatic parameter rescaling scheme to retain the nuclear norm of the original matrix.
Outcome:	The proposed model outperforms baseline models on flan-T5 by 4.2% and is robust to hyperparamater choice.

Copied to clipboard

Hieu Trung Nguyen, Bao Nguyen, Binh Nguyen, Viet Anh Nguyen

Scaling Multi-Document Event Summarization: Evaluating Compression vs. Full-Text Approaches (2025.naacl-short)

Challenge:	Existing approaches to task adaptation rely heavily on heuristic rules or prompt inputs.
Approach:	They propose a layer-wise additive activation intervention framework that steers the LMs’ generation process by identifying and manipulating the activations.
Outcome:	The proposed framework improves the accuracy of pretrained LMs and competing baselines on various datasets, demonstrating improvements in the accuracy and sample efficiency of the proposed framework.

Copied to clipboard

Adithya Pratapa, Teruko Mitamura

Summarization

Challenge:	Summarizing large text collections is a valuable tool for document research . a multi-stage pipeline and lack of global context are challenges for large-scale summarization systems.
Approach:	They compare compression and full-text systems for large-scale multi-document summarization . they find that compression-based methods outperform full-context methods .
Outcome:	The proposed methods outperform compression-based methods on three datasets . however, they suffer information loss due to their multi-stage pipeline and lack of global context.

Black-Box Visual Prompt Engineering for Mitigating Object Hallucination in Large Vision Language Models (2025.naacl-short)

Copied to clipboard

Sangmin Woo, Kang Zhou, Yun Zhou, Shuai Wang, Sheng Guan, Haibo Ding, Lin Lee Cheong

A Layered Debating Multi-Agent System for Similar Disease Diagnosis (2025.naacl-short)

Challenge:	Existing methods to mitigate object hallucination are impractical for proprietary LVLMs.
Approach:	They propose a framework to identify optimal visual prompts that enhance LVLM responses without access to model internals.
Outcome:	The proposed approach is model-agnostic and can be used on open-source and proprietary LVLMs.

Copied to clipboard

Yutian Zhao, Huimin Wang, Yefeng Zheng, Xian Wu

Challenge:	Traditional classification, contrastive learning, and large language models fail to detect subtle clues necessary for differentiation.
Approach:	They propose a framework that leverages Large Language Models to achieve accurate disease diagnosis . they structure patient information and integrate extensive medical knowledge to guide the analysis .
Outcome:	The proposed framework aims to identify subtle differences between similar diseases . the proposed framework can be used in clinical practice to improve accuracy .

The Geometry of Numerical Reasoning: Language Models Compare Numeric Properties in Linear Subspaces (2025.naacl-short)

Copied to clipboard

Ahmed Oumar El-Shangiti, Tatsuya Hiraoka, Hilal AlQuabeh, Benjamin Heinzerling, Kentaro Inui

AlignFreeze: Navigating the Impact of Realignment on the Layers of Multilingual Models Across Diverse Languages (2025.naacl-short)

Challenge:	Existing studies have focused on simple factual recall, but we have not explored how this is used in more complex queries.
Approach:	They propose to identify low-dimensional subspaces which encode numerical attributes associated with entities in comparison prompts.
Outcome:	The proposed model can answer numeric comparison questions using a low-dimensional subspace of theembedding space.

Copied to clipboard

Steve Bakos, David Guzmán, Riddhi More, Kelly Chutong Li, Félix Gaschi, En-Shiun Annie Lee

FLIQA-AD: a Fusion Model with Large Language Model for Better Diagnose and MMSE Prediction of Alzheimer’s Disease (2025.naacl-short)

Challenge:	Realignment techniques are often employed to enhance cross-lingual transfer in multilingual language models, but can degrade performance in languages that differ significantly from the fine-tuned source language.
Approach:	They propose a method that freezes either the lower half or upper half of the layers during realignment to prevent performance degradation.
Outcome:	The proposed method improves Part-of-Speech (PoS) tagging performance in languages where realignment fails.

Copied to clipboard

Junhao Chen, Zhiyuan Ding, Yan Liu, Xiangzhu Zeng, Ling Wang

Transform Retrieval for Textual Entailment in RAG (2025.naacl-short)

Challenge:	Existing classification and regression models that only extract finer-grained information from magnetic resonance imaging (MRI) may not be effective for Alzheimer's disease (AD).
Approach:	They propose to use a 3D Adapter in a Vision Transformer to extract the patient's EHR information and questions related to the disease as text prompts.
Outcome:	The proposed model can discriminate and predict the corresponding MMSE score based on the extracted brain structural information and textual content .

Copied to clipboard

Quan Guo, Xin Liang

How do Multimodal Foundation Models Encode Text and Speech? An Analysis of Cross-Lingual and Cross-Modal Representations (2025.naacl-short)

Challenge:	Existing retrieval methods prioritize relevance without ensuring the retrieved documents semantically support answering the queries.
Approach:	They propose a novel approach to improve Textual Entailment Retrieval within the framework of Retri-Augmented Generation (RAG) they transform query embeddings to better align with semantic entailment without re-encoding the document corpus.
Outcome:	The proposed approach consistently approaches the skyline across multiple datasets, demonstrating its strength in many-to-many retrieval scenarios.

Copied to clipboard

Hyunji Lee, Danni Liu, Supriti Sinhamahapatra, Jan Niehues

Explore the Reasoning Capability of LLMs in the Chess Testbed (2025.naacl-short)

Challenge:	Recent advances in foundation models have sparked growing interest in expanding their text processing capabilities to speech.
Approach:	They analyze the model activations from semantically equivalent sentences across languages in the text and speech modalities and examine how text and spoken are represented in recent multimodal foundation models.
Outcome:	The proposed models exhibit cross-lingual differences, but are not explicitly trained for modality-agnostic representations.

Copied to clipboard

Shu Wang, Lei Ji, Renxi Wang, Wenxiao Zhao, Haokun Liu, Yifan Hou, Ying Nian Wu

Auto-Cypher: Improving LLMs on Cypher generation via LLM-supervised generation-verification framework (2025.naacl-short)

Challenge:	a recent study shows that large language models struggle with long-term, complex reasoning tasks.
Approach:	They propose to integrate annotated strategy and tactic into large language models to improve reasoning capability.
Outcome:	The proposed model performs better than GPT, Claude, and Gemini models . it integrates annotated strategy and tactic into the model .

Copied to clipboard

Aman Tiwari, Shiva Krishna Reddy Malay, Vikas Yadav, Masoud Hashemi, Sathwik Tejaswi Madhusudhan

Leveraging Moment Injection for Enhanced Semi-supervised Natural Language Inference with Large Language Models (2025.naacl-short)

Challenge:	Graph databases like Neo4j are gaining popularity for handling complex, interconnected data, over traditional relational databases.
Approach:	They propose an automated pipeline to generate Cypher queries for Neo4j using LLM-As-Database-Filler, a novel strategy for ensuring Cyphere query correctness.
Outcome:	The proposed pipeline generates high quality Cypher data containing 29.8k instances across various domains and queries with varying complexities.

Copied to clipboard

Seo Yeon Park

A Fair Comparison without Translationese: English vs. Target-language Instructions for Multilingual LLMs (2025.naacl-short)

Challenge:	Existing studies have used class-specific fine-tuned large language models to generate hypotheses and assign pseudo-labels but discarded many LLM-constructed samples to ensure the quality.
Approach:	They propose to leverage LLM-constructed samples by injecting the moments of labeled samples during training to properly adjust the level of noise.
Outcome:	The proposed method outperforms strong baselines on multiple NLI datasets in low-resource settings.

Copied to clipboard

Taisei Enomoto, Hwichan Kim, Zhousi Chen, Mamoru Komachi

Evaluating Multimodal Generative AI with Korean Educational Standards (2025.naacl-short)

Challenge:	Prior studies suggested that English instructions are more effective for non-English tasks . however, these studies often use datasets and instructions translated from English .
Approach:	They conduct a fair comparison between English and target-language instructions by eliminating translationese effects.
Outcome:	The results show that the advantage of adopting English instructions is not overwhelming . the results also show that instruction-following abilities are improved when using respective instructions.

Copied to clipboard

Sanghee Park, Geewook Kim

ScratchEval: Are GPT-4o Smarter than My Child? Evaluating Large Multimodal Models with Visual Programming Challenges (2025.naacl-short)

Challenge:	Current benchmarks focus on English, overlooking the linguistic diversity worldwide and offering limited insights into low-resource languages like Korean.
Approach:	They propose to use Korean national educational tests to evaluate AI systems using a benchmark dataset.
Outcome:	The proposed benchmarks evaluate models in less-explored languages and open-source code and dataset builder will be fully open-sourced.

Copied to clipboard

Rao Fu, Ziyang Luo, Hongzhan Lin, Zhen Ye, Jing Ma

Interpret and Control Dense Retrieval with Sparse Latent Features (2025.naacl-short)

Challenge:	Recent advances in large multimodal models (LMMs) have demonstrated impressive code generation capabilities, primarily evaluated through image-to-code benchmarks.
Approach:	They propose a visual programming reasoning benchmark based on Scratch, a block-based visual programming language widely used in children’s programming education.
Outcome:	The proposed framework evaluates the visual programming ability of large multimodal models by integrating visual elements and embedded programming logic.

Copied to clipboard

Hao Kang, Tevin Wang, Chenyan Xiong

Explainability & Interpretability in NLP

Challenge:	Dense embeddings deliver strong retrieval performance but lack interpretability and controllability.
Approach:	They propose a novel approach using sparse autoencoders to interpret and control dense embeddings via latent sparsity.
Outcome:	The proposed approach retains the same retrieval accuracy as the original dense vectors, affirming their faithfulness.

DART: An AIGT Detector using AMR of Rephrased Text (2025.naacl-short)

Copied to clipboard

Hyeonchu Park, Byungjun Kim, Bugeun Kim

Scaling Graph-Based Dependency Parsing with Arc Vectorization and Attention-Based Refinement (2025.naacl-short)

Challenge:	Existing methods for detecting AIGTs focus on probabilistic features, causing problems . performance of black-box detectors is low, and it is difficult to detect black- box models .
Approach:	They propose a detector that can discriminate multiple black-box LLMs without probabilistic features and the origin of AIGT.
Outcome:	The proposed method can discriminate multiple black-box LLMs without probabilistic features and the origin of AIGT.

Copied to clipboard

Nicolas Floquet, Joseph Le Roux, Nadi Tomeh, Thierry Charnois

Syntactic Text Processing

Challenge:	Existing graph-based dependency parsers use a standard two-pipeline approach that only scores arcs and labels .
Approach:	They propose a graph-based dependency parsing architecture that explicitly constructs vectors from which both arcs and labels are scored.
Outcome:	The proposed model outperforms state-of-the-art models on PTB and UD in accuracy and efficiency.

Language Models “Grok” to Copy (2025.naacl-short)

Copied to clipboard

Ang Lv, Ruobing Xie, Xingwu Sun, Zhanhui Kang, Rui Yan

Evaluating LLMs for Quotation Attribution in Literary Texts: A Case Study of LLaMa3 (2025.naacl-short)

Challenge:	We examine the pre-training dynamics of language models, focusing on their ability to copy text from preceding context.
Approach:	They propose that Transformer-based language models develop copying abilities similarly to grokking . they argue that the connection between groking and context copying can improve in-context performance.
Outcome:	The proposed model development is similar to grokking, but the speed is independent of tokens trained.

Copied to clipboard

Gaspard Michel, Elena V. Epure, Romain Hennequin, Christophe Cerisara

Challenge:	Large Language Models (LLMs) have shown promising results in literary tasks . however, quotation attribution remains a challenging task and methods that generalize across writing styles are lacking analysis regarding book memorization and annotation contamination.
Approach:	They evaluate the ability of Llama-3 to attribute utterances of direct-speech to their speaker in novels by assessing the impact of book memorization and annotation contamination.
Outcome:	The proposed model outperforms existing models on a corpus of 28 novels and shows that book memorization and annotation contamination do not explain the performance gain.

Beyond Literal Token Overlap: Token Alignability for Multilinguality (2025.naacl-short)

Copied to clipboard

Katharina Hämmerl, Tomasz Limisiewicz, Jindřich Libovický, Alexander Fraser

IdentifyMe: A Challenging Long-Context Mention Resolution Benchmark for LLMs (2025.naacl-short)

Challenge:	Existing studies have shown that token overlap is a strong predictor of multilinguality and cross-lingual knowledge transfer between languages with different scripts.
Approach:	They propose a subword token alignability metric to understand the impact and quality of multilingual tokenisation.
Outcome:	The proposed metric predicts multilinguality much better when scripts are disparate and the overlap of literal tokens is low.

Copied to clipboard

Kawshik Manikantan, Makarand Tapaswi, Vineet Gandhi, Shubham Toshniwal

Challenge:	Recent evaluations of LLMs on coreference resolution have revealed that traditional output formats and evaluation metrics do not fully capture the models’ referential understanding.
Approach:	They propose a benchmark for mention resolution presented in a multiple-choice question format and a curated mixture of different mention types and corresponding entities.
Outcome:	The proposed model achieves 81.9% accuracy while the open model achieve 80%.

kNN Retrieval for Simple and Effective Zero-Shot Multi-speaker Text-to-Speech (2025.naacl-short)

Copied to clipboard

Karl El Hajal, Ajinkya Kulkarni, Enno Hermann, Mathew Magimai Doss

CORD: Balancing COnsistency and Rank Distillation for Robust Retrieval-Augmented Generation (2025.naacl-short)

Challenge:	Neural text-to-speech (TTS) models typically rely on extensive transcribed speech datasets and intricate training pipelines.
Approach:	They propose a framework for zero-shot multi-speaker text-to-speech using retrieval methods which leverage the linear relationships between SSL features.
Outcome:	The proposed framework achieves comparable performance to state-of-the-art models trained on large training datasets.

Copied to clipboard

Youngwon Lee, Seung-won Hwang, Daniel F Campos, Filip Graliński, Zhewei Yao, Yuxiong He

GraphLSS: Integrating Lexical, Structural, and Semantic Features for Long Document Extractive Summarization (2025.naacl-short)

Challenge:	Existing methods to ground large language models fail to adequately attend to all contexts . position bias is hindered by retrieval-augmented generation, which requires constant attention .
Approach:	They propose to augment and distill training instances with their perturbed positions to encourage consistent predictions . they also propose to balance COnsistency and Rank Distillation by combining noise-controlled perturbations with augmentation and distillation.
Outcome:	The proposed method outperforms existing methods in diverse RAG benchmarks.

Copied to clipboard

Margarita Bugueño, Hazem Abou Hamdan, Gerard De Melo

Information Extraction & Text Mining

Challenge:	Graph-based methods for extracting documents have been popular, but they often require external tools or additional machine learning models to define graph components.
Approach:	They propose a heterogeneous graph construction for extractive summarization that defines two levels of information and four types of edges without any need for auxiliary learning models.
Outcome:	The proposed graph construction outperforms previous graph-based models on two datasets and is available on GitHub.

Step-by-Step Fact Verification System for Medical Claims with Explainable Reasoning (2025.naacl-short)

Copied to clipboard

Juraj Vladika, Ivana Hacajova, Florian Matthes

Developing multilingual speech synthesis system for Ojibwe, Mi’kmaq, and Maliseet (2025.naacl-short)

Challenge:	Fact verification (FV) aims to assess the veracity of a claim based on relevant evidence.
Approach:	They propose to use iterative fact verification to assess the veracity of a claim based on relevant evidence.
Outcome:	The proposed system improves on three medical fact-checking datasets and evaluates with multiple settings including different LLMs, external web search, and structured reasoning using logic predicates.

Copied to clipboard

Shenran Wang, Changbing Yang, Michael l Parkhill, Chad Quinn, Christopher Hammerly, Jian Zhu

Bottom-Up Synthesis of Knowledge-Grounded Task-Oriented Dialogues with Iteratively Self-Refined Prompts (2025.naacl-short)

Challenge:	In general, speech synthesis for Indigenous languages is underdeveloped compared to the majority of languages.
Approach:	They propose to train a multilingual model on three typologically similar languages to improve performance over monolingual models.
Outcome:	The proposed model can train on three similar languages with high performance and is highly competitive with self-attention architectures with higher memory efficiency.

Copied to clipboard

Kun Qian, Maximillian Chen, Siyan Li, Arpit Sharma, Zhou Yu

Natural Language Interfaces

Challenge:	Training conversational question-answering systems requires in-domain data, which is often scarce in practice.
Approach:	They propose a bottom-up approach where QA pairs are generated first and combined into a coherent dialogue.
Outcome:	The proposed approach produces more realistic and higher-quality dialogues compared to top-down methods.

Sociodemographic Prompting is Not Yet an Effective Approach for Simulating Subjective Judgments with LLMs (2025.naacl-short)

Copied to clipboard

Huaman Sun, Jiaxin Pei, Minje Choi, David Jurgens

Identifying Power Relations in Conversations using Multi-Agent Social Reasoning (2025.naacl-short)

Challenge:	Large language models (LLMs) are widely used to simulate human responses, but their ability to account for demographic differences in subjective tasks remains uncertain.
Approach:	They evaluate large language models' ability to understand demographic differences in two subjective judgment tasks: politeness and offensiveness.
Outcome:	The proposed models perform better in politeness and offensiveness tasks, while sociodemographic prompting does not improve and worsens their ability to perceive language from sub-populations.

Copied to clipboard

Zhaoqing Wu, Dan Goldwasser, Maria Leonor Pacheco, Leora Morgenstern

Dialogue Systems & Conversational Agents

Challenge:	Existing approaches to understanding power relationships in conversations are based on task-specific supervised learning.
Approach:	They propose a multi-agent social reasoning framework that leverages social science tools to generate and evaluate reasons from multiple perspectives and construct a factor graph for inference.
Outcome:	The proposed framework outperforms standard prompting baselines on power dynamics in conversations.

Examining Spanish Counseling with MIDAS: a Motivational Interviewing Dataset in Spanish (2025.naacl-short)

Copied to clipboard

Aylin Ece Gunal, Bowen Yi, John D. Piette, Rada Mihalcea, Veronica Perez-Rosas

Challenge:	Cultural and language factors influence counseling, but research has not explored whether this applies to other languages.
Approach:	They introduce a Spanish-language counseling dataset that contains expert annotations for counseling reflections and questions.
Outcome:	The proposed dataset explores language-based differences in counselor behavior in English and Spanish and develops classifiers in monolingual and multilingual settings.

Self-Debiasing Large Language Models: Zero-Shot Recognition and Reduction of Stereotypes (2025.naacl-short)

Copied to clipboard

Isabel O. Gallegos, Ryan Aponte, Ryan A. Rossi, Joe Barrow, Mehrab Tanjim, Tong Yu, Hanieh Deilamsalehy, Ruiyi Zhang, Sungchul Kim, Franck Dernoncourt, Nedim Lipka, Deonna Owens, Jiuxiang Gu

EqualizeIR: Mitigating Linguistic Biases in Retrieval Models (2025.naacl-short)

Challenge:	Large language models exhibit harmful social biases, but they are often difficult to train and modify.
Approach:	They leverage the zero-shot capabilities of large language models to reduce stereotyping . they introduce a technique called zero- shot self-debiasing to reduce bias .
Outcome:	The proposed technique reduces stereotyping across nine different social groups while relying on the LLM itself and a simple prompt.

Copied to clipboard

Jiali Cheng, Hadi Amiri

Do Audio-Language Models Understand Linguistic Variations? (2025.naacl-short)

Challenge:	Existing information retrieval models show significant linguistic biases based on the linguistic complexity of queries.
Approach:	They propose a framework to mitigate linguistic biases in IR models by using a linguistically biased weak learner to capture biased queries and then train a robust model by regularizing and refining its predictions.
Outcome:	The proposed framework reduces performance disparities across simple and complex queries while improving overall retrieval performance.

Copied to clipboard

Ramaneswaran Selvakumar, Sonal Kumar, Hemant Kumar Giri, Nishit Anand, Ashish Seth, Sreyan Ghosh, Dinesh Manocha

Robustness in NLP

Challenge:	Existing open-vocabulary audio language models struggle to generalize to linguistic variations in textual queries.
Approach:	They propose a novel technique to learn audio-language representations agnostic to linguistic variations by reformulating contrastive loss used in CLAP architectures.
Outcome:	The proposed approach improves the performance of the open-vocabulary audio language models by 0.8%-13% across benchmarks and enhances robustness to linguistic variation.

Giving the Old a Fresh Spin: Quality Estimation-Assisted Constrained Decoding for Automatic Post-Editing (2025.naacl-short)

Copied to clipboard

Sourabh Deoghare, Diptesh Kanojia, Pushpak Bhattacharyya