Papers with inference
Enhancing Ethical Explanations of Large Language Models through Iterative Symbolic Refinement (2024.eacl-long)
Copied to clipboard
| Challenge: | Recent studies have focused on the application and evaluation of Large Language Models (LLMs) but LLMs are still prone to factual errors and inconsistencies in their explanations, offering limited control and interpretability for inference in complex domains. |
| Approach: | They propose an abductive-deductive framework that integrates Large Language Models with an external backward-chaining solver to refine step-wise natural language explanations. |
| Outcome: | The proposed framework improves explanations generated via in-context learning methods and Chain-of-Thought (CoT) on ethical NLI tasks while producing formal proofs describing and supporting models’ reasoning. |
Neural Network based Extreme Classification and Similarity Models for Product Matching (N18-3)
Copied to clipboard
| Challenge: | Matching a seller listed item to an appropriate product has become a fundamental step for e-commerce platforms. |
| Approach: | They propose to use a shallow neural network to match a seller's item to an appropriate product . they also propose a similarity approach based on deep siamese network to train and infer product information. |
| Outcome: | The proposed models outperform the baseline models by more than 5% in terms of accuracy and are capable of efficient training and inference. |
SpiRit-LM: Interleaved Spoken and Written Language Model (2025.tacl-1)
Copied to clipboard
Tu Anh Nguyen, Benjamin Muller, Bokai Yu, Marta R. Costa-jussa, Maha Elbayad, Sravya Popuri, Christophe Ropers, Paul-Ambroise Duquenne, Robin Algayres, Ruslan Mavlyutov, Itai Gat, Mary Williamson, Gabriel Synnaeve, Juan Pino, Benoît Sagot, Emmanuel Dupoux
| Challenge: | SpiRit-LM is a foundation multimodal language model that freely mixes text and speech. |
| Approach: | They propose a multimodal language model that freely mixes text and speech . they extend the model to the speech modality by continuously training it on text and language units. |
| Outcome: | The proposed model can learn new tasks in a few-shot fashion across modalities. |
What BERT Is Not: Lessons from a New Suite of Psycholinguistic Diagnostics for Language Models (2020.tacl-1)
Copied to clipboard
| Challenge: | Pretraining by language modeling has become popular but we have yet to understand what language models learn during that process. |
| Approach: | They propose diagnostics that ask questions about information used by language models for generating predictions in context. |
| Outcome: | The proposed diagnostics can be used to study the popular BERT model . they show that the model can distinguish good from bad completions, but struggles with inference and role-based event prediction. |
GenTranslate: Large Language Models are Generative Multilingual Speech and Machine Translators (2024.acl-long)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have stepped forward the development of multilingual speech and machine translation by its reduced representation errors and incorporated external knowledge. |
| Approach: | They propose a generative paradigm for translation tasks that integrates the diverse translation versions in N-best list. |
| Outcome: | The proposed model outperforms the state-of-the-art model on speech and machine translation benchmarks on various languages. |
Diversifying Reply Suggestions Using a Matching-Conditional Variational Autoencoder (N19-2)
Copied to clipboard
| Challenge: | Automated reply suggestions (SR) are becoming common in many popular applications such as Gmail (2016) . |
| Approach: | They propose a constrained-sampling approach to make the variational inference efficient for a commercial instant-messaging system. |
| Outcome: | The proposed model increases diversity without losing relevance in offline experiments. |
WinoDict: Probing language models for in-context word acquisition (2023.eacl-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are unable to reflect the way language changes over time as their training corpus is frozen in time. |
| Approach: | They propose a new in-context learning paradigm to measure Large Language Models' ability to learn novel words during inference. |
| Outcome: | The proposed model improves on Winograd-style co-reference resolution problems by replacing the key concept word with a plausible word that the model must understand to complete the task. |
Abstractive Text Summarization Using the BRIO Training Paradigm (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing abstractive summarization models rely heavily on reference summaries and lack control over their performance. |
| Approach: | They propose a BRIO paradigm to reduce the dependence on reference summaries by fine-tuning pre-trained language models and training them with the paradigm. |
| Outcome: | The proposed paradigm outperforms existing models on Vietnamese and CNNDM datasets while maintaining the main content of the original text. |
Improving and Simplifying Template-Based Named Entity Recognition (2023.eacl-srw)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is traditionally approached as a sequence labeling task where a tag is predicted for each token. |
| Approach: | They propose to convert a Named Entity Recognition task into a seq2seq task by generating synthetic sentences using templates. |
| Outcome: | The proposed model outperforms the current state-of-the-art approach in resource-rich, low resource and domain transfer settings and the negative examples play an important role in its performance. |
dLLM: Simple Diffusion Language Modeling (2026.acl-demo)
Copied to clipboard
| Challenge: | diffusion language models (DLMs) are evolving rapidly but many lack transparent implementations or are scattered across codebases. |
| Approach: | They propose an open-source framework that unifies diffusion language modeling components while remaining flexible enough to support new methods and architectures. |
| Outcome: | dLLM unifies the core components of diffusion language modeling and makes them easy to customize for new designs. |
DEED: Dynamic Early Exit on Decoder for Accelerating Encoder-Decoder Transformer Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Encoder-decoder transformer models suffer from high inference latency due to auto-regressive decoding . Typically, the decoder takes up most of the latency because of the auto-decoding - a problem that is not solved by the current model. |
| Approach: | They propose an approach to perform Dynamic Early Exit on Decoder to reduce inference latency by 20%-74% by using a multi-exit encoder-decoder transformer model trained with deep supervision. |
| Outcome: | The proposed model reduces inference latency by 20%-74% with comparable or even higher accuracy compared to baseline models. |
Attention Temperature Matters in Abstractive Summarization Distillation (2022.acl-long)
Copied to clipboard
| Challenge: | Recent progress of abstractive text summarization relies on large pre-trained sequence-to-sequence Transformer models, which are computationally expensive. |
| Approach: | They propose to distill large Transformer summarization models into smaller ones with minimal performance loss by manipulating attention temperatures in Transformers. |
| Outcome: | The proposed method outperforms vanilla pseudo-labeling based methods on three summarization datasets and is shorter and more abstractive. |
Geometric Interpretation of Layer Normalization and a Comparative Analysis with RMSNorm (2026.findings-eacl)
Copied to clipboard
| Challenge: | a novel geometric interpretation of LayerNorm is presented . layer normalization is a crucial yet often overlooked component of the transformer architecture . |
| Approach: | They propose a geometric interpretation of LayerNorm and explore how LayerNorm influences the norm and orientation of hidden vectors in the representation space. |
| Outcome: | The proposed interpretation of LayerNorm shows that it is redundant to remove a component along the uniform vector during training and inference. |
AdapterDistillation: Non-Destructive Task Composition with Knowledge Distillation (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Recent work on learning from multiple tasks has shown that adding an extra fusion layer to implement knowledge composition is non-scalable for some applications. |
| Approach: | They propose a two-stage knowledge distillation algorithm to extract task specific knowledge by using local data to train a student adapter. |
| Outcome: | Experiments on frequently asked question retrieval in task-oriented dialog systems validate the efficiency of AdapterDistillation. |
Eider: Empowering Document-level Relation Extraction with Efficient Evidence Extraction and Inference-stage Fusion (2022.findings-acl)
Copied to clipboard
| Challenge: | Document-level relation extraction (DocRE) aims to extract semantic relations among entity pairs in a document. |
| Approach: | They propose an evidence-enhanced framework that empowers document-level relation extraction (DocRE) Eider efficiently extracts evidence and effectively fuses extracted evidence in inference. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on three benchmark datasets. |
GrocLM: Grocery Category Recommendation in E-Commerce with Large Language Models (2026.acl-industry)
Copied to clipboard
| Challenge: | a growing number of online grocery shoppers are using category-level recommendation systems . traditional item-level methods face scalability and accuracy challenges . |
| Approach: | a new language model is developed to encode cyclical purchasing patterns into model parameters . the model is scalable and more business-aligned than traditional item-level methods . |
| Outcome: | a new language model outperforms standard methods in a live production environment . the proposed model achieves a 7.5% relative improvement in cart-adds per impression . |
A Context-aware Framework for Translation-mediated Conversations (2026.tacl-1)
Copied to clipboard
| Challenge: | Existing systems that bridge language barriers can introduce errors leading to misunderstandings and conversation breakdown. |
| Approach: | They propose a framework to integrate contextual information into automatic translation systems . they validate the framework on customer chat and user-assistant interaction . |
| Outcome: | The proposed framework consistently produces better translations than state-of-the-art systems on two task-oriented domains. |
COGEN: Abductive Commonsense Language Generation (2023.acl-short)
Copied to clipboard
| Challenge: | Existing training methods for NLP models to perform on two main tasks are needed to introduce these capabilities into the field of reasoning. |
| Approach: | They propose a model that integrates commonsense reasoning with contextual filtering to improve the inference. |
| Outcome: | The proposed model outperforms existing models and sets new state-of-the-art in regards to alphaNLI and alphaNGG tasks. |
Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation (P19-3)
Copied to clipboard
Zhiting Hu, Haoran Shi, Bowen Tan, Wentao Wang, Zichao Yang, Tiancheng Zhao, Junxian He, Lianhui Qin, Di Wang, Xuezhe Ma, Zhengzhong Liu, Xiaodan Liang, Wanrong Zhu, Devendra Sachan, Eric Xing
| Challenge: | Texar is an open-source text generation toolkit that supports a broad set of text generation tasks. |
| Approach: | They introduce Texar, an open-source text generation toolkit that supports text generation tasks. |
| Outcome: | Texar supports machine translation, summarization, dialog, content manipulation, and more. |
Discourse-Aware In-Context Learning for Temporal Expression Normalization (2024.naacl-short)
Copied to clipboard
| Challenge: | Temporal expression (TE) normalization is a well-studied problem, but upcoming machine learning approaches suffer from a lack of labeled data. |
| Approach: | They propose to use in-context learning to inject task, document, and example information into a large language model for temporal expression normalization. |
| Outcome: | The proposed model performs better in non-standard settings by dynamically including relevant examples during inference. |
Flambé: A Customizable Framework for Machine Learning Experiments (P19-3)
Copied to clipboard
| Challenge: | Flambé is a machine learning experimentation framework built to accelerate the entire research life cycle. |
| Approach: | They propose a framework that allows users to write custom code but include that code as a component in a larger system. |
| Outcome: | The proposed framework enables users to write custom code but include that code as a component in a larger system which is represented by a concise configuration file format. |
SpeechNet: Weakly Supervised, End-to-End Speech Recognition at Industrial Scale (2022.emnlp-industry)
Copied to clipboard
Raphael Tang, Karun Kumar, Gefei Yang, Akshat Pandey, Yajie Mao, Vladislav Belyaev, Madhuri Emmadi, Craig Murray, Ferhan Ture, Jimmy Lin
| Challenge: | End-to-end automatic speech recognition systems require thousands of hours of manual annotation and heavyweight computation to perform inference. |
| Approach: | They propose to use a third-party ASR system as a weak supervision source and labeling functions derived from implicit user feedback to reduce human labor. |
| Outcome: | The proposed system improves word-error rate and speed up 600% over third-party ASR. |
PASS: Perturb-and-Select Summarizer for Product Reviews (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work on product reviews summarization focuses on generating concise, coherent and informative summaries, but this task is challenging. |
| Approach: | They propose a product reviews summarization task that employs a large pre-trained Transformer-based model and a method for ranking these summaries according to desired criteria. |
| Outcome: | The proposed system avoids the problem of self-contradiction by ranking the summaries according to desired criteria. |
Sharing, Teaching and Aligning: Knowledgeable Transfer Learning for Cross-Lingual Machine Reading Comprehension (2023.findings-emnlp)
Copied to clipboard
| Challenge: | X-STA is a new approach for cross-lingual machine reading comprehension . the variation of answer span positions in different languages makes it difficult to transfer knowledge across languages. |
| Approach: | They propose a method that leverages an attentive teacher to subtly transfer the answer spans of the source language to the answer output space of the target. |
| Outcome: | The proposed method outperforms state-of-the-art approaches on three multi-lingual datasets. |
MOLEMAN: Mention-Only Linking of Entities with a Mention Annotation Network (2021.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to entity linking represent each entity with a single vector, but instead use a contextualized mention-encoder that learns to place similar mentions of the same entity closer in vector space than mentions from different entities. |
| Approach: | They propose an instance-based nearest neighbor approach to entity linking that allows for a contextualized mention-encoder to learn to place similar mentions of the same entity closer in vector space than mentions from different entities. |
| Outcome: | The proposed approach outperforms all other systems on two multilingual benchmarks and is simpler to train and interpretable. |
RecInDial: A Unified Framework for Conversational Recommendation with Pretrained Language Models (2022.aacl-main)
Copied to clipboard
| Challenge: | Existing generative methods to recommend items are shallowly integrated into the model training and have poor chit-chat ability. |
| Approach: | They propose a framework that integrates recommendation into the dialog generation by introducing a vocabulary pointer. |
| Outcome: | The proposed framework outperforms the state-of-the-art models on a benchmark dataset. |
Learning to Route for Dynamic Adapter Composition in Continual Learning with Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work shows that PEFT methods can be competitive with, or even superior to, full fine-tuning of PLMs. |
| Approach: | They propose a method that isolates the training of new PEFT modules to ensure their task specialization and learns to compose them by training a network of routers that leverages a small memory containing examples of previously seen tasks. |
| Outcome: | The proposed method improves generalization and performance in two CL setups. |
Towards Better Graph-based Cross-document Relation Extraction via Non-bridge Entity Enhancement and Prediction Debiasing (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on relation extraction ignore non-bridge entities, leading to bias during inference. |
| Approach: | They propose a graph-based cross-document Relation Extraction model with non-bridge entity enhancement and prediction debiasing that integrates non-cross entities with target entities and bridge entities. |
| Outcome: | The proposed model outperforms baseline models on open and closed datasets. |
Accelerating Dense LLMs via L0-regularized Mixture-of-Experts (2025.acl-short)
Copied to clipboard
| Challenge: | Existing methods for accelerating large language models (LLMs) suffer from slow and costly inference. |
| Approach: | They propose a lightweight MoE approach using cluster confusion matrix and dynamic batching to accelerate dense LLMs. |
| Outcome: | The proposed method achieves 2.5x speedup over dense models while maintaining competitive performance. |
Measuring the Effect of Influential Messages on Varying Personas (2023.acl-short)
Copied to clipboard
| Challenge: | a new task estimates the response a persona might have upon seeing a news message . a first benchmark dataset is used to evaluate the performance of the proposed task . |
| Approach: | They propose a task to estimate the response a persona might have upon seeing a news message. |
| Outcome: | The proposed task estimates the response a persona might have upon seeing a news message. |
Augmenting Compliance-Guaranteed Customer Service Chatbots: Context-Aware Knowledge Expansion with Large Language Models (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Retrieval-based chatbots leverage human-verified Q&A knowledge to deliver accurate, verifiable responses. |
| Approach: | They propose a similar question generation task for LLM training and inference to enable comprehensive semantic exploration and enhanced alignment with source question-answer relationships. |
| Outcome: | The proposed methods achieve 92% user satisfaction rate in a deployed chatbot system, reflecting an 18% improvement over the baseline. |
Memory-Efficient Backpropagation for Fine-Tuning LLMs on Resource-Constrained Mobile Devices (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing work on memory-efficient on-device fine-tuning of large language models with backpropagation has focused on approximating gradients with zeroth-order optimization (ZO). |
| Approach: | They propose a memory-efficient implementation of backpropagation on mobile devices that allows flexible trade-offs between memory usage and compute time while converging faster. |
| Outcome: | The proposed method can fine-tune LLMs with backpropagation using less than 1GB of memory while achieving better performance than the baseline. |
BIPro: Zero-shot Chinese Poem Generation via Block Inverse Prompting Constrained Generation Framework (2025.acl-long)
Copied to clipboard
| Challenge: | generative pre-trained models face challenges on constrained writing tasks like poem generation . brian mccartney: BIPro improves the zero-shot generation quality on constricted writing tasks . |
| Approach: | They propose a framework that leverages two block inverse prompting methods to improve the quality of constrained writing tasks. |
| Outcome: | BIPro significantly improves the quality of Chinese poem generation without priming or training. |
Correctable-DST: Mitigating Historical Context Mismatch between Training and Inference for Improved Dialogue State Tracking (2022.emnlp-main)
Copied to clipboard
Hongyan Xie, Haoxiang Su, Shuangyong Song, Hao Huang, Bo Zou, Kun Deng, Jianghua Lin, Zhihui Zhang, Xiaodong He
| Challenge: | Existing dialogue state tracking approaches predict the dialogue state of a target turn sequentially based on the ground-truth previous dialogue state. |
| Approach: | They propose a method that predicts dialogue state sequentially based on previous dialogue state . they propose generating a previously “predicted” dialogue state using ground-truth previous dialogue states . |
| Outcome: | The proposed method achieves 67.51%, 68.24%, 70.30%, 71.38%, and 81.27% joint goal accuracy on MultiWOZ 2.0-2.4 datasets. |
DisComp: A Two-Stage Prompt Optimization Framework Combining Task-Agnostic and Task-Aware Compression (2025.findings-naacl)
Copied to clipboard
| Challenge: | Extended prompts can lead to substantial computational overhead and increased hardware demands, limiting the scalability and efficiency of large language models. |
| Approach: | They propose a two-stage prompt compression framework that combines task-agnostic and task-based strategies to efficiently compress prompt length without compromising performance. |
| Outcome: | The proposed framework outperforms task-agnostic and task-specific compression methods on three benchmark datasets and is up to 6.56 faster at inference compared to the best token-level compression method. |
A Meaning-Based Statistical English Math Word Problem Solver (N18-1)
Copied to clipboard
| Challenge: | Experimental results show that the proposed approach understands the meaning of each quantity in the text more. |
| Approach: | They propose a meaning-based approach for solving English math word problems . they analyze text, transform body and question parts into corresponding logic forms . Statistical models are proposed to select operator and operands . |
| Outcome: | The proposed approach outperforms existing systems on benchmark and noisy datasets. |
PE-QAT: Parameter-Efficient Quantization-Aware Training for Large Language Models (2026.acl-srw)
Copied to clipboard
| Challenge: | Quantization Aware Training (QAT) is expensive to train and unscalable to large models. |
| Approach: | They propose a parameter-efficient framework targeting per-channel 4-bit weight-activation quantization of large language models. |
| Outcome: | The proposed framework preserves accuracy within 0.11 percentage points of the full-precision baseline on Llama-2-7B zero-shot tasks while training only 1.26% of total parameters. |
Diffusion Directed Acyclic Transformer for Non-Autoregressive Machine Translation (2025.acl-short)
Copied to clipboard
| Challenge: | Non-autoregressive transformers (NATs) often encounter performance challenges due to the multi-modality problem. |
| Approach: | They propose a direct-acyclic transformer (DAT) that captures multiple translation modalities to paths in a Directed Acyclic Graph (DAG) this allows the model to integrate latent variables into the model, which is crucial for DAT to achieve state-of-the-art performance. |
| Outcome: | The proposed model captures multiple translation modalities to paths in a Directed Acyclic Graph (DAG) but the collaboration with the latent variable introduced through the Glancing training is crucial for the model to attain state-of-the-art performance. |
Hashing based Efficient Inference for Image-Text Matching (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent work on image-text matching has focused on exploring interactions between images and sentences to improve performance without considering inference efficiency. |
| Approach: | They propose a hashing-based efficient inference module which can be plugged into existing frameworks to speed up inference step without reducing retrieval performance. |
| Outcome: | The proposed module can be plugged into existing framework to speed up inference step without reducing retrieval performance. |
Alleviating Exposure Bias in Abstractive Summarization via Sequentially Generating and Revising (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing approaches to abstractive summarization suffer from exposure bias . Existing solutions bridge this gap through un- or semi-supervised holistic learning . |
| Approach: | They propose to reformat abstractive summarization to sequential generation and revision (SeGRe) this allows the model to assess the flawed summary from a global perspective and modify inappropriate expressions. |
| Outcome: | The proposed model can assess the flawed summary from a global view and modify inappropriate expressions. |
GrEmLIn: A Repository of Green Baseline Embeddings for 87 Low-Resource Languages Injected with Multilingual Graph Knowledge (2025.findings-naacl)
Copied to clipboard
| Challenge: | Contextualized word embeddings are available for many languages, but their coverage is limited for low resourced languages. |
| Approach: | They propose a method that integrates multilingual graph knowledge into the embeddings to make them green. |
| Outcome: | The proposed method outperforms state-of-the-art embeddings on lexical similarity task while being parameter-free at inference time. |
ThinkBooster: A Unified Framework for Seamless Test-Time Scaling of LLM Reasoning (2026.acl-demo)
Copied to clipboard
Vladislav Smirnov, Quang-Chieu Nguyen, Sergey Senichev, Minh Ngoc Ta, Ekaterina Fadeeva, Artem Vazhentsev, Daria Galimzianova, Nikolai Rozanov, Viktor Mazanov, Jingwei Ni, Tianyi Wu, Igor Kiselev, Mrinmaya Sachan, Iryna Gurevych, Preslav Nakov, Timothy Baldwin, Artem Shelmanov
| Challenge: | Existing TTC scaling strategies and reasoning scorers are fragmented and evaluated under inconsistent protocols. |
| Approach: | They propose a framework for seamless test-time compute scaling of large language model reasoning . they use a modular Python library to implement state-of-the-art scaling strategy and scorer families . |
| Outcome: | The proposed framework evaluates performance and computational efficiency on mathematical and coding tasks. |
Adjusting the Precision-Recall Trade-Off with Align-and-Predict Decoding for Grammatical Error Correction (2022.acl-short)
Copied to clipboard
| Challenge: | Modern writing assistance applications always contain a Grammatical Error Correction (GEC) model to correct errors in user-entered sentences. |
| Approach: | They propose a simple yet effective approach to Align-and-Predict Decoding for most popular sequence-to-sequence models to offer more flexibility for the precision-recall trade-off. |
| Outcome: | The proposed model can be used in both English and Chinese GEC models and achieve state-of-the-art results. |
FaiRR: Faithful and Robust Deductive Reasoning over Natural Language (2022.acl-long)
Copied to clipboard
| Challenge: | Currently, black-box models generate both the proof graph and intermediate inferences within the same model and thus may be unfaithful. |
| Approach: | They propose a transformer-based model that can perform deductive reasoning on a logical rulebase containing rules and statements written in natural language. |
| Outcome: | The proposed model is robust to language perturbations and faster at inference than previous models on existing reasoning datasets. |
[MASK] Insertion: a robust method for anti-adversarial attacks (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing studies have focused on adversarial defenses against pretrained language models. |
| Approach: | They propose an adversarial defensing algorithm that inserts tokens into input sequences . they show an improvement in accuracy between 3.2 and 11.1 absolute points . |
| Outcome: | The proposed algorithm improves model accuracy on clean and polluted inputs compared with state-of-the-art models . |
Agentic Economic Modeling (2026.acl-industry)
Copied to clipboard
| Challenge: | AEM is a framework that aligns synthetic LLM choices with small-sample human evidence for reliable econometric inference. |
| Approach: | They introduce a framework that aligns synthetic LLM choices with small-sample human evidence for reliable econometric inference. |
| Outcome: | The proposed framework improves RCT efficiency and establishes a foundation method for LLM-based counterfactual generation. |
Feature-Adaptive and Data-Scalable In-Context Learning (2024.acl-long)
Copied to clipboard
| Challenge: | In-context learning (ICL) is a popular way to stimulate LLM capabilities for downstream tasks due to context length constraints. |
| Approach: | They propose a feature-adaptive and data-scalable in-context learning framework which leverages task-adaptives to promote inference on the downstream task. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on 10 datasets under different data settings and LLM scale. |
Enhancing Event Causality Identification with Counterfactual Reasoning (2023.acl-short)
Copied to clipboard
| Challenge: | Existing methods for event causality identification (ECI) focus on mining potential causal signals, but causal signals are ambiguous, which may lead to the context-keywords bias and the event-pairs bias. |
| Approach: | They propose a method that explicitly estimates the influence of context keywords and event pairs in training to eliminate biases in inference. |
| Outcome: | The proposed method eliminates biases in inference on two datasets. |
LongSpec: Long-Context Lossless Speculative Decoding with Efficient Drafting and Verification (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) can process extremely long contexts, requiring efficient inference over extended inputs. |
| Approach: | They propose a model that uses a constant-sized key-value cache to train long-context models. |
| Outcome: | Experimental results show that LongSpec achieves 3.26x speedup over strong Flash Attention baselines and 2.34x wall clock time on four math reasoning tasks. |
LingGen: Scalable Multi-Attribute Linguistic Control via Power-Law Masking (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing methods for controlling coarse attributes are less effective for finer-grained attributes and suffer from inefficiencies when many attributes must be handled jointly. |
| Approach: | They propose a controlled text generation model that allows fine-grained control over a large number of real-valued linguistic attributes. |
| Outcome: | The proposed model achieves the lowest average control error among evaluated methods while remaining efficient at inference and receiving the highest fluency scores in human evaluation. |
SeNsER: Learning Cross-Building Sensor Metadata Tagger (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Sensor metadata tagging is a key component of smart building applications. |
| Approach: | They propose a framework that learns a sensor metadata tagger for a new building based on its raw metadata and some existing fully annotated building. |
| Outcome: | The proposed framework learns a sensor metadata tagger for a new building based on its raw metadata and some existing fully annotated building. |
Parameter-Efficient Neural Reranking for Cross-Lingual and Multilingual Retrieval (2022.coling-1)
Copied to clipboard
| Challenge: | State-of-the-art neural rankers are notoriously data-hungry and rarely used in multilingual and cross-lingual retrieval settings. |
| Approach: | They propose to use Sparse Fine-Tuning Masks and Adapters to transfer rankers trained on English data to other languages and cross-lingual setups by means of multilingual encoders. |
| Outcome: | The proposed methods outperform standard zero-shot transfer with full MMT fine-tuning while being more modular and reducing training times. |
Zero-shot Generalization in Dialog State Tracking through Generative Question Answering (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for Dialog State Tracking do not generalize well to new domains and unseen slots. |
| Approach: | They propose an ontology-free framework that queries for unseen constraints and slots in multi-domain task-oriented dialogs using a conditional language model pre-trained on substantive English sentences. |
| Outcome: | The proposed framework improves goal accuracy in zero-shot domain adaptation settings by up to 9% over the previous state-of-the-art on the MultiWOZ 2.1 dataset. |
Efficient Hierarchical Domain Adaptation for Pretrained Language Models (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to allow domain adaptation to diverse domains are expensive and require continuing training in-domain. |
| Approach: | They propose a method to permit domain adaptation to many diverse domains using a computationally efficient adapter approach. |
| Outcome: | The proposed method allows domain adaptation to many diverse domains while avoiding negative interference between unrelated domains. |
Explainable Multi-hop Verbal Reasoning Through Internal Monologue (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing state-of-the-art language models do not make intermediate reasoning steps explicit . large pretrained language models such as BERT and RoBERTa have been successfully used in multi-hop reasoning problems . |
| Approach: | They propose to decompose multi-hop reasoning problems into several simple ones and use natural language to guide intermediate reasoning hops. |
| Outcome: | The proposed model can generate subgoals and perform inference in natural language at each reasoning step. |
Generative Text Modeling through Short Run Inference (2021.eacl-main)
Copied to clipboard
| Challenge: | Latent variable models for text capture global semantic and syntactic features when trained correctly. |
| Approach: | They propose a short run dynamics for inference that initializes from the prior distribution of the latent variable and runs a small number of Langevin dynamics steps guided by its posterior distribution. |
| Outcome: | The proposed model is able to generate coherent sentences with smooth transition and shows no sign of posterior collapse. |
L4Q: Parameter Efficient Quantization-Aware Fine-Tuning on Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Quantization-aware PEFT methods have been developed to reduce memory and computational costs associated with large language models. |
| Approach: | They propose a method that integrates Quantization-Aware Training (QAT) with LoRA to reduce memory overhead and improve model accuracy. |
| Outcome: | The proposed method significantly reduces QAT’s memory overhead while preserving the advantage of QAT in producing fully quantized LLMs with high accuracy. |
Exploring End-to-End Differentiable Natural Logic Modeling (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to integrate natural logic with neural networks are brittle and prone to fail in the presence of noise and uncertainty. |
| Approach: | They propose to integrate natural logic with neural networks to create differentiable models that integrate natural reasoning with subsymbolic vector representations and neural components. |
| Outcome: | The proposed model can model monotonicity-based reasoning, compared to baseline models without inductive bias. |
When Facts Change: Temporal Knowledge Conflict Resolution in LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models are increasingly used in retrieval-augmented generation systems to reconcile knowledge conflicts between parametric memory and contextual inputs. |
| Approach: | They propose to use mutability to resolve temporal misalignment in large language models to compare stable and recently updated facts from Wikidata to determine if mutable models can serve as a mediating signal in this process. |
| Outcome: | The proposed model can produce reasoning for facts that actually changed but rarely for stable ones, whereas smaller models rarely detect conflict, while larger models detect it but fail to act on mutability judgments. |
mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval (2024.emnlp-industry)
Copied to clipboard
Xin Zhang, Yanzhao Zhang, Dingkun Long, Wen Xie, Ziqi Dai, Jialong Tang, Huan Lin, Baosong Yang, Pengjun Xie, Fei Huang, Meishan Zhang, Wenjie Li, Min Zhang
| Challenge: | Existing models for text retrieval are based on a multi-stage process that involves retrieving documents from a large corpus. |
| Approach: | They propose to build a multilingual text representation model and a cross-encoder reranker from scratch for text retrieval. |
| Outcome: | The proposed models outperform the state-of-the-art models on long-context retrieval benchmarks. |
The Impact of Reasoning Step Length on Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Long reasoning steps in LLMs improve reasoning abilities, but the correlation between their effectiveness and the length of reasoning steps remains largely unknown. |
| Approach: | They conducted experiments that expand and compress the rationale reasoning steps within CoT demonstrations while keeping all other factors constant. |
| Outcome: | The results show that lengthening the reasoning steps in prompts significantly enhances LLMs’ reasoning abilities across multiple datasets. |
UniRAG: Universal Retrieval Augmentation for Large Vision Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large Vision Language Models (LVLMs) have unlocked many complex use cases that require Multi-Modal (MM) understanding and MM generation. |
| Approach: | They propose a plug-and-play technique that adds relevant retrieved information to prompts as few-shot examples during inference. |
| Outcome: | The proposed method significantly improves the output quality of large vision language models when input prompts are augmented with relevant information retrieved by Vision-Language retrievers like UniRAG. |
Gaussian Mixture Latent Vector Grammars (P18-1)
Copied to clipboard
| Challenge: | Existing models of latent variable grammars are not observable in treebanks, so latent variables are learned using expectation-maximization. |
| Approach: | They propose a new framework that extends latent variable grammars such that each nonterminal symbol is associated with a continuous vector space representing the set of (infinitely many) subtypes of the nonterminals. |
| Outcome: | The proposed framework can achieve competitive accuracies in part-of-speech tagging and constituency parsing. |
Zero-Shot Contextual Embeddings via Offline Synthetic Corpus Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Context-aware embedding methods boost retrieval accuracy by conditioning on corpus statistics extracted from neighboring documents. |
| Approach: | They propose a zero-shot contextual adaptation framework that replaces real corpus access with a one-time offline synthesis of a compact proxy. |
| Outcome: | The proposed framework replaces real corpus access with offline synthesis of a compact proxy. |
Unveiling Internal Reasoning Modes in LLMs: A Deep Dive into Latent Reasoning vs. Factual Shortcuts with Attribute Rate Ratio (2025.emnlp-main)
Copied to clipboard
Yiran Yang, Haifeng Sun, Jingyu Wang, Qi Qi, Zirui Zhuang, Huazheng Wang, Pengfei Ren, Jing Wang, Jianxin Liao
| Challenge: | Existing research in multi-hop questions has identified two reasoning modes, but has not investigated how these modes differ during inference. |
| Approach: | They propose a classification metric that compares latent reasoning and factual shortcuts in multi-hop questions. |
| Outcome: | The proposed metric achieves 90% accuracy on the proposed datasets and demonstrates effectiveness in RAG conflict scenarios. |
Towards Example-Based NMT with Multi-Levenshtein Transformers (2023.emnlp-main)
Copied to clipboard
| Challenge: | Retrieval-augmented machine translation (RAMT) is attracting growing attention . it is assumed to implement some form of domain adaptation . |
| Approach: | They propose a retrieval-augmented version of the Levenshtein Transformer to make it more transparent . they propose to perform training and inference in this model, based on multi-way alignment algorithms and imitation learning. |
| Outcome: | The proposed architecture improves translation performance and improves consistency of translations compared to previous models. |
FOL-Traces: Verified First-Order Logic Reasoning Traces at Scale (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing approaches to evaluate language models fail to provide structural clarity and verifiable inference. |
| Approach: | They propose to use a large-scale dataset of programmatically verified reasoning traces to evaluate structured logical inference. |
| Outcome: | The proposed model achieves 45.7% accuracy on masked operation prediction and 27% on two-step completion. |
An Unsupervised Framework for Adaptive Context-aware Simplified-Traditional Chinese Character Conversion (2024.lrec-main)
Copied to clipboard
| Challenge: | Traditional Chinese characters are still widely used in many areas of China . traditional methods to convert between simplified characters are ineffective . |
| Approach: | They propose an unsupervised adaptive context-aware conversion model that learns to convert between simplified and traditional Chinese characters under a denoising auto-encoder framework. |
| Outcome: | The proposed model outperforms strong unsupervised baselines and yields better conversion result for one-to-many cases. |
Syntactically Supervised Transformers for Faster Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Standard decoders for neural machine translation generate a single token per timestep, which slows inference . a series of controlled experiments demonstrates that SynST decodes sentences 5x faster than the baseline autoregressive Transformer. |
| Approach: | They propose a syntactically supervised Transformer that generates all target tokens in one shot . synST is a variant of the Transformer architecture that autoregressively predicts a chunked parse tree . |
| Outcome: | The proposed method decodes sentences 5x faster than the baseline method on En-De and En-Fr datasets while achieving higher BLEU scores. |
Self-Regulated Sample Diversity in Large Language Models (2024.findings-naacl)
Copied to clipboard
Mingyue Liu, Jonathan Frawley, Sarah Wyer, Hubert P. H. Shum, Sara Uckelman, Sue Black, Chris Willcocks
| Challenge: | Existing methods that require expensive setups or maintain static values during inference are inflexible and require expensive training. |
| Approach: | They propose a self-regulating approach that adjusts sample diversity parameters dynamically based on the input prompt. |
| Outcome: | The proposed method significantly improves the quality of responses generically without model retraining or fine-tuning. |
A Novel Matching Paradigm: Unified Generative and Discriminative LLM with Prompt Compression for Relevance Learning (2026.acl-industry)
Copied to clipboard
Guoliang Zhao, Zixin Cui, Chao Ye, Dengwu He, Fei Huang, Yubo Liu, Shuanglong Li, Tzungren Kuo, Bin Ding, Shuang Zhang, null KunhongZhu, Zhi Guo, Liu Lin
| Challenge: | Existing approaches to matching use Large Language Models as feature extractors, underutilizing their full modeling capabilities. |
| Approach: | They propose a matching paradigm that integrates two-tower, single-towing, and generative tasks within a unified LLM framework via attention-mask partitioning. |
| Outcome: | The proposed model achieves superior performance and strong practical value in an industrial search engine. |
SWAN: An Efficient and Scalable Approach for Long-Context Language Modeling (2025.emnlp-main)
Copied to clipboard
Krishna C Puvvada, Faisal Ladhak, Santiago Akle Serano, Cheng-Ping Hsieh, Shantanu Acharya, Somshubra Majumdar, Fei Jia, Samuel Kriman, Simeng Sun, Dima Rekesh, Boris Ginsburg
| Challenge: | Existing decoder-only models struggle with context lengths beyond their training distribution. |
| Approach: | They propose a causal Transformer architecture that generalizes robustly to sequence lengths longer than seen during training. |
| Outcome: | The proposed decoder-only architecture can generalize robustly to longer contexts . it is more computationally efficient than the standard Transformer architecture, the authors say . |
Byte Pair Encoding for Symbolic Music (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent research has focused on reducing the overall sequence length by merging embeddings or combining tokens. |
| Approach: | They propose to convert music into a sequence of discrete tokens using Byte Pair Encoding . they propose to combine tokens with embeddings to improve the results . |
| Outcome: | The proposed method reduces the sequence length while increasing the vocabulary size. |
Modeling Hierarchical Reasoning Chains by Linking Discourse Units and Key Phrases for Reading Comprehension (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods of logical reasoning focus on entity-aware information but ignore hierarchical relations that may even have mutual effects. |
| Approach: | They propose a holistic graph network that deals with context at both discourse-level and word-level as the basis for logical reasoning. |
| Outcome: | The proposed method improves on logical reasoning QA datasets and natural language inference datasets. |
Logic-Driven Context Extension and Data Augmentation for Logical Reasoning of Text (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for logical reasoning of text focus on contextual semantics while struggling to explicitly model the logical inference process. |
| Approach: | They propose a logic-driven context extension framework and a data-driven augmentation algorithm that uses contrastive learning to better capture logical information. |
| Outcome: | The proposed framework outperforms existing methods on two benchmark datasets, ReClor and LogiQA. |
Found in the Middle: Permutation Self-Consistency Improves Listwise Ranking in Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models exhibit positional bias in how they use context, which affects listwise ranking. |
| Approach: | They propose a method to marginalize out different list orders in the prompt to produce an order-independent ranking with less positional bias. |
| Outcome: | The proposed method improves on five datasets in sorting and passage reranking by 34-52% . it marginalizes out different list orders in the prompt to produce an order-independent ranking . |
Probing Structured Pruning on Multilingual Pre-trained Models: Settings, Algorithms, and Efficiency (2022.acl-long)
Copied to clipboard
| Challenge: | Structured pruning has been extensively studied on monolingual pre-trained models . but little attention has been paid to evaluating the effectiveness of structured pruning on multilingual models. |
| Approach: | They investigate settings, algorithms, and efficiency of structured pruning on multilingual models . authors propose a simple approach that allows training the model once and adapting to different model sizes at inference . |
| Outcome: | The proposed approach allows training the model once and adapting to different model sizes at inference. |
VAULT: VAriable Unified Long Text Representation for Machine Reading Comprehension (2021.acl-short)
Copied to clipboard
| Challenge: | Existing models on Machine Reading Comprehension (MRC) require complex model architecture for effectively modeling long texts with paragraph representation and classification, making inference computationally inefficient for production use. |
| Approach: | They propose a novel Gaussian distribution-based paragraph representation for Machine Reading Comprehension (MRC) that is light-weight and parallel-efficient. |
| Outcome: | The proposed model can achieve comparable performance on Wikipedia-based (NQ) and TechNotes (TechQA) with a state-of-the-art (SOTA) complex document modeling approach while being 16 times faster, demonstrating the efficiency of the proposed model. |
Classic4Children: Adapting Chinese Literary Classics for Children with Large Language Model (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent large language models (LLMs) overlook children’s reading preferences, which poses challenges in CLA. |
| Approach: | They propose a method that augments large language models with children's reading preferences for adaptation by obtaining characters' personalities and narrative structure as additional information for fine-grained instruction tuning. |
| Outcome: | The proposed method significantly improves performance in automatic and human evaluation. |
Coupling Local Context and Global Semantic Prototypes via a Hierarchical Architecture for Rhetorical Roles Labeling (2026.eacl-long)
Copied to clipboard
Anas Belfathi, Nicolas Hernandez, Monceaux Laura, Warren Bonnard, Mary Catherine Lavissière, Christine Jacquin, Richard Dufour
| Challenge: | Hierarchical models capture local dependencies but lack global, corpus-level representations. |
| Approach: | They propose two prototype-based methods that integrate local context with global representations to address this limitation. |
| Outcome: | The proposed methods integrate local context with global representations. |
NeuroPrune: A Neuro-inspired Topological Sparse Training Algorithm for Large Language Models (2024.findings-acl)
Copied to clipboard
Amit Dhurandhar, Tejaswini Pedapati, Ronny Luss, Soham Dan, Aurelie Lozano, Payel Das, Georgios Kollias
| Challenge: | Transformer-based Language Models have become ubiquitous in natural language processing due to impressive performance on various tasks. |
| Approach: | They explore how sparsity affects network topology by exploiting mechanisms seen in biological networks . they show that model-agnostic sparsities are performant across diverse NLP tasks . |
| Outcome: | The proposed model-agnostic sparsity approaches are performant and efficient across NLP tasks. |
RedApt: An Adaptor for wav2vec 2 EncodingFaster and Smaller Speech Translation without Quality Compromise (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained speech Transformers in speech translation systems have facilitated state-of-the-art (SotA) results, but their computational cost is high. |
| Approach: | They propose a Reducer Adaptor block that could be seamlessly integrated within any Transformer-based speech encoding architecture. |
| Outcome: | The proposed Reducer Adaptor block outperforms the existing SotA architecture by an average of 0.68 BLEU score on 8 language pairs from Must-C. |
Learning from Sibling Mentions with Scalable Graph Inference in Fine-Grained Entity Typing (2022.acl-long)
Copied to clipboard
| Challenge: | Existing models struggle to handle hard mentions due to insufficient contexts, limiting their overall typing performance. |
| Approach: | They propose to exploit sibling mentions to enhance the mention representations by adding unseen test mentions as new nodes for inference. |
| Outcome: | The proposed model outperforms ten strong baseline models and outperformed strong baselines. |
On the Robustness of Question Rewriting Systems to Questions of Varying Hardness (2022.acl-long)
Copied to clipboard
| Challenge: | entailment : absence of questions classified based on their rewriting hardness or difficulty . enactment of QR system to rewrite context-dependent questions in CQA requires context knowledge . |
| Approach: | They propose a heuristic method to automatically classify questions into subsets of varying hardness . they then conduct a human evaluation to annotate the rewriting hardness of questions . |
| Outcome: | The proposed learning framework improves the overall performance compared to baselines. |
Great Power, Great Responsibility: Recommendations for Reducing Energy for Training Language Models (2022.findings-naacl)
Copied to clipboard
| Challenge: | Recent studies show that the energy requirements of current NLP models are growing at a rapid, unsustainable pace. |
| Approach: | They investigate ways to measure energy usage and different hardware settings that can be tuned to reduce energy consumption for training and inference for language models. |
| Outcome: | The proposed techniques can reduce energy consumption for training and inference for language models. |
UniverSLU: Universal Spoken Language Understanding for Diverse Tasks with Natural Language Instructions (2024.naacl-long)
Copied to clipboard
Siddhant Arora, Hayato Futami, Jee-weon Jung, Yifan Peng, Roshan Sharma, Yosuke Kashiwagi, Emiru Tsunoo, Karen Livescu, Shinji Watanabe
| Challenge: | Recent studies leverage large language models with multi-tasking capabilities, using natural language prompts to guide the model’s behavior and surpassing performance of task-specific models. |
| Approach: | They adapt a pre-trained automatic speech recognition model to additional tasks using single-token task specifiers. |
| Outcome: | The proposed model can generalize to new datasets and languages for seen task types. |
Probing Pre-trained Auto-regressive Language Models for Named Entity Typing and Recognition (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing studies have focused on auto-regressive models for generalization in named entity (NE) typing (NET) and recognition (NER) . however, little has been done in this direction for auto-Regressive LMs despite their popularity and potential to express a wide variety of NLP tasks in the same unified format. |
| Approach: | They propose to probe auto-regressive LMs for NET and NER generalization by resorting to meta-learning to assess the model's memorization of NEs. |
| Outcome: | The proposed model performs well on NET and NER generalization tasks, while relying more on NE than contextual cues in few-shot NER. |
Chain of Thought with Explicit Evidence Reasoning for Few-shot Relation Extraction (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to few-shot relation extraction require training. |
| Approach: | They propose a method for few-shot relation extraction using large language models, called CoT-ER, chain-of-thought with explicit evidence reasoning. |
| Outcome: | The proposed approach achieves competitive performance compared to the fully-supervised state-of-the-art approach on the FewRel1.0 and FewRela2.0 datasets. |
Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters (2023.acl-long)
Copied to clipboard
| Challenge: | Chain-of-Thought (CoT) prompting can dramatically improve the multi-step reasoning abilities of large language models (LLMs). |
| Approach: | They propose to use Chain-of-Thought (CoT) prompting to encourage the LLM to generate intermediate rationales for solving a problem by providing a series of reasoning steps in the demonstrations. |
| Outcome: | The proposed model can generate coherent lines of reasoning even with invalid demonstrations while still generating coherent lines during inference. |
FLAT-LLM: Fine-grained Low-rank Activation Space Transformation for Large Language Model Compression (2026.findings-eacl)
Copied to clipboard
| Challenge: | Low-rank decomposition methods suffer from accuracy degradation and expensive calibration procedures. |
| Approach: | They propose a fast and accurate, training-free structural compression method based on fine-grained low-rank transformations in the activation space. |
| Outcome: | The proposed method outperforms pruning baselines in generalization and downstream performance while delivering inference speedups. |
Dynamic and Efficient Inference for Text Generation via BERT Family (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods to improve performance of pre-trained language models are limited due to large-scale parameters and the universal autoregressive decoding paradigm. |
| Approach: | They propose a novel fine-tuning method which can make a single pre-trained model support Dynamic and Efficient infERence and achieve an adaptive trade-off between model performance and latency. |
| Outcome: | The proposed method achieves higher BLEU scores than the strong autoregressive Transformer model on translation tasks with 3 12 times speedup and faster inference speed compared with the BART model on four GLGE benchmark tasks. |
Are Generative Models Underconfident? Better Quality Estimation with Boosted Model Probability (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown that text-generation models can be overconfident when there are multiple correct options. |
| Approach: | They propose a QE approach called BoostedProb which boosts the model’s confidence in cases where there are multiple viable output options. |
| Outcome: | The proposed approach achieves on average +0.194 improvement in Pearson correlation to ground-truth quality and outperforms more costly approaches like supervised or ensemble-based QE in certain settings. |
Bridging the Gap between Synthetic and Authentic Images for Multimodal Machine Translation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing models require associated image with input sentence, which is difficult to satisfy at inference. |
| Approach: | They propose to use synthetic and authentic images to generate translations using text-to-image generation models. |
| Outcome: | The proposed model achieves state-of-the-art performance on En-De and En-Fr datasets while remaining independent of authentic images during inference. |
Subword Segmental Machine Translation: Unifying Segmentation and Target Sentence Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Subword segmenters are used in neural machine translation, but are not used in high-resource settings. |
| Approach: | They propose a subword segmental machine translation (SSMT) that unifies subword and MT in a single trainable model. |
| Outcome: | The proposed model improves chrF scores for morphologically rich agglutinative languages and is more robust on a test set constructed for evaluating morphology generalisations. |
Empowering Character-level Text Infilling by Eliminating Sub-Tokens (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for character-level infilling relied on predicting sub-tokens, but this strategy was ineffective. |
| Approach: | They propose a method to fill-in-the-mid with Starting and Ending character constraints that avoids predicting sub-tokens in inference. |
| Outcome: | The proposed method surpasses existing methods and offers significant performance gains. |
Learning to Predict Persona Information for Dialogue Personalization without Explicit Persona Description (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to personalize dialogue agents rely on explicit persona descriptions during inference, which severely limits their application in real-world scenarios. |
| Approach: | They propose a method that learns to predict persona information based on the dialogue history to personalize dialogue agents without relying on explicit persona descriptions during inference. |
| Outcome: | The proposed method improves the consistency and engagingness of generated responses when conditioning on the predicted profile of the dialogue agent. |
VoCoT: Unleashing Visually Grounded Multi-Step Reasoning in Large Multi-Modal Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Despite the impressive capabilities of large multi-modal models, their effectiveness in handling complex tasks has been limited by the prevailing singlestep reasoning paradigm. |
| Approach: | They propose a visuallygrounded object-centric Chain-of-Thought reasoning framework for LMMs that is based on a multi-modal interleaved and aligned representation of object concepts. |
| Outcome: | The proposed model outperforms SOTA models in CLEVR and EmbSpatial benchmarks. |
PyramidInfer: Pyramid KV Cache Compression for High-throughput LLM Inference (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to reduce memory usage for large language models neglect inter-layer dependency between layers and huge memory consumption in pre-computation. |
| Approach: | They propose a method that compresses the KV cache by layer-wise retaining crucial context. |
| Outcome: | The proposed method reduces memory usage by layer-wise retaining crucial context . it can improve 2.2x throughput compared to Accelerate with over 54% memory reduction . |
Single Model Ensemble for Subword Regularized Models in Low-Resource Machine Translation (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing subword regularizations use multiple segmentations during training but only use one segmentation in inference. |
| Approach: | They propose an inference strategy that uses multiple subword segmentations to solve this discrepancy in the training process and inference. |
| Outcome: | The proposed strategy reduces the cost of training and improves the performance of models trained with subword regularization in low-resource machine translation tasks. |
SWITCH: Studying with Teacher for Knowledge Distillation of Large Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Knowledge Distillation (KD) has emerged as a popular method for compressing large language models due to high inference costs and memory requirements. |
| Approach: | They propose a method that integrates the teacher model during the student's sequence generation to reduce misguidance from the teacher. |
| Outcome: | Experiments on three model families and five instruction-following datasets show that SWITCH surpasses traditional methods, especially in the generation of long sequential data. |
Thought2Text: Text Generation from EEG Signal using Large Language Models (LLMs) (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent advances in NLP driven by powerful Large Language Models such as Ope-nAI GPT-4 have been demonstrated in ALS and stroke patients. |
| Approach: | They propose to use instruction-tuned Large Language Models (LLMs) with EEG data to decode and express brain activity in a comprehensible form. |
| Outcome: | The proposed approach enables multimodal description generation from EEG data and further refinement on embeddings to generate text directly from EMG during inference. |
Efficient Low-rank Multimodal Fusion With Modality-Specific Factors (P18-1)
Copied to clipboard
Zhun Liu, Ying Shen, Varun Bharadhwaj Lakshminarasimhan, Paul Pu Liang, AmirAli Bagher Zadeh, Louis-Philippe Morency
| Challenge: | Multimodal research is a growing field of artificial intelligence, and fusion is one of the main research problems. |
| Approach: | They propose a low-rank multimodal fusion method which integrates multiple unimodal representations into one compact multimodal representation. |
| Outcome: | The proposed method achieves competitive results on multimodal sentiment analysis, speaker trait analysis, and emotion recognition tasks while reducing computational complexity. |
Lossless Acceleration of Large Language Models with Hierarchical Drafting based on Temporal Locality in Speculative Decoding (2025.findings-naacl)
Copied to clipboard
Sukmin Cho, Sangjin Choi, Taeho Hwang, Jeongyeon Seo, Soyeong Jeong, Huije Lee, Hoyun Song, Jong C. Park, Youngjin Kwon
| Challenge: | Existing methods for drafting and verifying tokens require significant fine-tuning or have inconsistent performance across tasks. |
| Approach: | They propose a lossless drafting approach that organizes various token sources into multiple databases in a hierarchical framework based on temporal locality. |
| Outcome: | The proposed method outperforms existing database drafting methods on Spec-Bench using 7B and 13B parameters. |
Uncertainty-Aware Semantic Augmentation for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for neural machine translation only observe one source sentence at training time . this discrepancy in data distribution leads to a formidable learning challenge . |
| Approach: | They propose an uncertainty-aware semantic augmentation approach to capture universal semantic information among multiple source sentences and enhance hidden representations with this information. |
| Outcome: | The proposed approach outperforms baseline and existing methods on translation tasks. |
Towards Order Fairness: Mitigating LLMs Order Sensitivity through Dual Group Advantage Optimization (2026.acl-long)
Copied to clipboard
| Challenge: | Recent studies attempt to obtain optimal or suboptimal arrangements based on statistical results or using dataset-based search, but these methods increase inference overhead while leaving the model’s inherent order bias unresolved. |
| Approach: | They propose Dual Group Advantage Optimization (DGAO) which aims to improve model accuracy and order stability simultaneously. |
| Outcome: | The proposed method improves model accuracy and order stability while penalizing order-sensitive or incorrect responses. |
A Knowledge Hunting Framework for Common Sense Reasoning (D18-1)
Copied to clipboard
| Challenge: | a new system that uses common sense to solve a common sense problem is developed . a winograd schema challenge and a choice of plausible alternatives are popular tests . |
| Approach: | They propose an automatic system that achieves state-of-the-art results on the Winograd Schema Challenge . they use a knowledge hunting module to gather web text for problem resolutions . |
| Outcome: | The proposed system achieves state-of-the-art on the Winograd Schema Challenge . it improves F1 performance on the full WSC by 0.21 over the previous best . |
Watermarking LLMs with Weight Quantization (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models are being deployed at an astonishing speed, exposing users to high risks. |
| Approach: | They propose a method that plants watermarks in quantization process of large language models without pre-defined triggers during inference. |
| Outcome: | The proposed method protects model weights without pre-defined triggers . it works when the model is used in the fp32 mode and remains hidden when the models are quantized to int8 . |
DecoupleSearch: Decouple Planning and Search via Hierarchical Reward Modeling (2025.emnlp-main)
Copied to clipboard
Hao Sun, Zile Qiao, Bo Wang, Guoxin Chen, Yingyan Hou, Yong Jiang, Pengjun Xie, Fei Huang, Yan Zhang
| Challenge: | Retrieval-Augmented Generation (RAG) systems have emerged as a pivotal methodology for enhancing Large Language Models (LLMs). |
| Approach: | They propose a framework that decouples planning and search processes using dual value models, enabling independent optimization of plan reasoning and search grounding. |
| Outcome: | The proposed framework decouples planning and search processes using dual value models, enabling independent optimization of plan reasoning and search grounding. |
Wanda++: Pruning Large Language Models via Regional Gradients (2025.findings-acl)
Copied to clipboard
Yifan Yang, Kai Zhen, Bhavana Ganesh, Aram Galstyan, Goeric Huybrechts, Markus Müller, Jonas M. Kübler, Rupak Vignesh Swaminathan, Athanasios Mouchtaris, Sravan Babu Bodapati, Nathan Susanj, Zheng Zhang, Jack FitzGerald, Abhishek Kumar
| Challenge: | Existing pruning methods suffer from accuracy degradation without full-model sparsity-aware fine-tuning. |
| Approach: | They propose a pruning framework that uses decoder-block-level regional gradients to improve pruning accuracy. |
| Outcome: | The proposed pruning framework outperforms the state-of-the-art pruning frameworks by utilizing decoder-block-level regional gradients. |
Attention-Guided Answer Distillation for Machine Reading Comprehension (D18-1)
Copied to clipboard
| Challenge: | Existing approaches to reading comprehension systems are vulnerable to adversarial attacks. |
| Approach: | They propose to use knowledge distillation to transfer knowledge from an ensemble to a single model. |
| Outcome: | The proposed methods outperform the teacher on adversarial datasets and NarrativeQA benchmarks. |
Entropy-Gated Branching for Efficient Test-Time Reasoning (2026.eacl-long)
Copied to clipboard
| Challenge: | Empirical results show that branching at low uncertainty points can improve reasoning capabilities of large language models . however, these methods require substantially more computational resources, causing errors in high-stakes domains . |
| Approach: | They propose an inference technique that selectively expands prediction sequences at points of high uncertainty. |
| Outcome: | Empirical results show that the proposed method improves accuracy by 22.6% over standard inference while operating 31%-75% faster across math benchmarks. |
Exploring Hybrid Sampling Inference for Aspect-based Sentiment Analysis (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for inference require multiple sampling with preset size . however, it is a high-cost method that requires multiple sampling . |
| Approach: | They propose a method that combines multiple and single sampling to greatly reduce the cost of multiple sampling without sacrificing performance. |
| Outcome: | The proposed method greatly reduces the cost of multiple sampling without sacrificing performance. |
TextMixer: Mixing Multiple Inputs for Privacy-Preserving Inference (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) are often deployed as cloud services, enabling users to upload textual data and perform inference remotely. |
| Approach: | They propose a privacy-preserving inference framework called MixPi which aims to obfuscate a user's private input by mixing it with multiple other inputs. |
| Outcome: | The proposed framework surpasses existing privacy-preserving methods on token and sentence classification tasks. |
COMETA: A Corpus for Medical Entity Linking in the Social Media (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing datasets for Entity Linking (EL) fail to address the complex nature of health terminology in layman’s language. |
| Approach: | They propose to use a corpus of 20k English biomedical entity mentions from Reddit expert-annotated with links to a widely-used medical knowledge graph to investigate the ability of these systems to perform complex inference on entities and concepts. |
| Outcome: | The proposed corpus satisfies a combination of desirable properties, from scale and coverage to diversity and quality, that to the best of our knowledge has not been met by existing resources in the field. |
Type-Driven Multi-Turn Corrections for Grammatical Error Correction (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing studies focus on data augmentation to combat exposure bias . but data augmented models lack the ability to recognize the procedure of gradual corrections . |
| Approach: | They propose a type-driven multi-turn corrections approach that uses multiple training instances to train dominant models. |
| Outcome: | The proposed model achieves state-of-the-art single-model performance on English GEC benchmarks. |
McPhraSy: Multi-Context Phrase Similarity and Clustering (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for estimating phrase similarity use the phrase context only during training, instead relying on the phrase itself. |
| Approach: | They propose a novel algorithm that leverages multiple contexts during inference to estimate the similarity of phrases based on multiple context. |
| Outcome: | The proposed method outperforms existing models on two phrase similarity datasets by 13.3% and a new task that relies on phrase similarities in the product reviews domain. |
Inferring symmetry in natural language (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Empirical work on predicate symmetry has taken two main approaches: feature-based approach and context-based one denies the existence of absolute symmetry. |
| Approach: | They propose a methodological framework for inferring symmetry of verb predicates in natural language. |
| Outcome: | The proposed framework is based on a dataset of 400 naturalistic verbs spanning the spectrum of symmetry-asymmetry. |
Enabling Autoregressive Models to Fill In Masked Tokens (2026.findings-eacl)
Copied to clipboard
| Challenge: | Autoregressive (AR) and masked language modeling (MLM) models are incapable of mucked infilling, which is the ability to predict mangled tokens between past and future context. |
| Approach: | They propose a method that leverages the strengths of autoregressive and masked language modeling to achieve state-of-the-art mucked infilling performance. |
| Outcome: | The proposed approach outperforms existing methods on masked infilling tasks. |
DiPair: Fast and Accurate Distillation for Trillion-Scale Text Matching and Pair Modeling (2020.findings-emnlp)
Copied to clipboard
Jiecao Chen, Liu Yang, Karthik Raman, Michael Bendersky, Jung-Jung Yeh, Yun Zhou, Marc Najork, Danyang Cai, Ehsan Emadzadeh
| Challenge: | Existing knowledge distillation models are not optimized for dealing with pairs (or tuples) of texts. |
| Approach: | They propose a framework for distilling fast and accurate models on text pair tasks using a scalable end-to-end training strategy. |
| Outcome: | Empirical studies on academic and real-world e-commerce benchmarks show the proposed framework can achieve speedups of over 350x and minimal quality drop relative to the cross-attention teacher BERT model. |
Knowledge-Enhanced Evidence Retrieval for Counterargument Generation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models lack the reasoning abilities needed to find complex counterevidence. |
| Approach: | They propose a natural language inference model that finds counterevidence from diverse sources on the Web. |
| Outcome: | The proposed model outperforms baseline models for NLI tasks and finds complex counterevidence better. |
Learning to Rewrite for Non-Autoregressive Neural Machine Translation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing non-autoregressive neural machine translations have poor inference speed but weak recognition of erroneous translation pieces. |
| Approach: | They propose an architecture to explicitly learn to rewrite the erroneous translation pieces. |
| Outcome: | The proposed architecture can achieve better performance while significantly reducing decoding time. |
Bridging the Language Gaps in Large Language Models with Inference-Time Cross-Lingual Intervention (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to address performance gaps in LLMs rely on pretraining or fine-tuning, which are resource-intensive. |
| Approach: | They propose a framework that aligns LLMs' internal representations with those of high-performing languages during inference. |
| Outcome: | The proposed framework improves performance on low-performing (source) languages by aligning their internal representations with those of high-performing languages during inference. |
QuZO: Quantized Zeroth-Order Fine-Tuning for Large Language Models (2025.emnlp-main)
Copied to clipboard
Jiajun Zhou, Yifan Yang, Kai Zhen, Ziyue Liu, Yequan Zhao, Ershad Banijamali, Athanasios Mouchtaris, Ngai Wong, Zheng Zhang
| Challenge: | Large Language Models (LLMs) are quantized to lower precision to reduce memory cost and latency in inference. |
| Approach: | They propose a quantized zeroth-order framework for fine-tuning Large Language Models (LLMs) using low-precision forward passes. |
| Outcome: | The proposed method achieves comparable results to first-order methods in FP8 and superior accuracy in INT8 and INT4 training. |
Out-of-Scope Intent Detection with Self-Supervision and Discriminative Training (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for out-of-scope intent detection rely on strong assumptions on data distribution and confidence threshold selection. |
| Approach: | They propose a method to train an out-of-scope intent classifier in a fully end-to-end manner by simulating the test scenario in training. |
| Outcome: | The proposed method improves on four benchmark dialogue datasets and improves over state-of-the-art methods. |
Adaptable Adapters (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing work uses the same adapter architecture for every dataset regardless of the properties of the dataset or the amount of training data. |
| Approach: | They propose to use adaptable adapters to finetune lightweight neural network layers on top of pretrained weights. |
| Outcome: | The proposed adapters achieve on-par performances with the standard adapter architecture while using a considerably smaller number of adapter layers. |
Inferences for Lexical Semantic Resource Building with Less Supervision (2020.lrec-1)
Copied to clipboard
| Challenge: | lexical semantic resources may be built using various approaches such as extraction from corpora, integration of relevant pieces of knowledge from pre-existing knowledge resources and endogenous inference. |
| Approach: | They propose a method where the resource building process appears as a self learning process . they propose lexical and semantic resource building based on inference . |
| Outcome: | The proposed method reduces the human effort needed for lexical semantic resource building. |
Grammatical Error Correction as GAN-like Sequence Labeling (2021.findings-acl)
Copied to clipboard
| Challenge: | Traditional GEC models learn from sentences with fixed error rates . sequence labeling approaches suffer from a couple of key problems . |
| Approach: | They propose a GAN-like sequence labeling model with a grammatical error detector and a generator to correct grammamatical errors. |
| Outcome: | The proposed model improves the state-of-the-art in GEC and improves on benchmarks. |
Scalable Collapsed Inference for High-Dimensional Topic Models (N19-1)
Copied to clipboard
| Challenge: | Existing methods have achieved two out of three criteria simultaneously, but never all three at once. |
| Approach: | They propose an online inference algorithm which leverages stochasticity to scale well in the number of documents and sparsity to achieve accurate inference. |
| Outcome: | The proposed algorithm scales well in the number of documents and topics while achieving accurate inference. |
Benchmarking a transformer-FREE model for ad-hoc retrieval (2021.eacl-main)
Copied to clipboard
| Challenge: | a recent study compares transformer-based models with a greener and more sustainable alternative. |
| Approach: | They compare transformer-based models with a "greener and more sustainable" alternative . they show that transformer-like models can be used in real-world retrieval applications . |
| Outcome: | The lighter model achieves a speedup of 20 times in training and 7 to 47 times in inference while maintaining a comparable retrieval performance. |
Anchor-based Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) require massive GPU memory due to their size and parameter count. |
| Approach: | They propose to use anchor-based self-attention network and anchor-basic inference strategy to compress sequence information into an anchor token, reducing the keys/values cache and enhancing inference efficiency. |
| Outcome: | The proposed model reduces the key/value cache and improves inference efficiency by 99% while maintaining similar accuracy levels. |
Generative Biomedical Entity Linking via Knowledge Base-Guided Pre-training and Synonyms-Aware Fine-tuning (2022.naacl-main)
Copied to clipboard
| Challenge: | Generative methods for biomedical entity linking (EL) use synonyms knowledge from knowledge bases (KB) this is not trivial to inject into a generative method, but it is cost-effective. |
| Approach: | They propose to inject synonyms knowledge into a generative model of biomedical EL by constructing synthetic samples with synonyms and definitions from KB and requiring the model to recover concept names. |
| Outcome: | The proposed method achieves state-of-the-art results on several biomedical EL tasks without candidate selection. |
TReX: Tokenizer Regression for Optimal Data Mixture (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing approaches to train and inference tokenizers rely on heuristics or large-scale searches to determine optimal data mixtures. |
| Approach: | They propose a regression-based framework that efficiently predicts the optimal data mixture for tokenizer training. |
| Outcome: | The proposed model outperforms mixtures based on LLaMA3 and uniform distributions by up to 12% in both in- and out-of-distribution compression efficiency. |
BRIEF: Bridging Retrieval and Inference for Multi-hop Reasoning via Compression (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to augment language models with external knowledge but they are limited by static nature of pre-training data. |
| Approach: | They propose a lightweight approach that compresses retrieved documents into highly dense textual summaries to integrate into in-context RAG. |
| Outcome: | The proposed approach reduces latency and costs while achieving high performance in open-domain questions. |
Dialogue Summaries as Dialogue States (DS2), Template-Guided Summarization for Few-shot Dialogue State Tracking (2022.findings-acl)
Copied to clipboard
| Challenge: | Annotating task-oriented dialogues is notorious for the expensive and difficult data collection process. |
| Approach: | They propose to reformulate dialogue state tracking as a dialogue summarization problem by using synthetic dialogue summaries generated by a set of rules. |
| Outcome: | The proposed method outperforms previous studies on few-shot dialogue state tracking in MultiWoZ 2.0 and 2.1 in cross-domain and multi-domain settings. |
Interactive Learning for LLM Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing multi-agent learning approaches foster collaboration among Large Language Models (LLMs) yet they still rely on re-executing the MAS during inference. |
| Approach: | They propose a co-learning framework that integrates Dynamic Interaction and Perception Calibration to enhance LLMs' independent problem-solving ability. |
| Outcome: | The proposed framework integrates Dynamic Interaction and Perception Calibration to improve LLMs' independent problem-solving ability. |
Se2: Sequential Example Selection for In-Context Learning (2024.findings-acl)
Copied to clipboard
| Challenge: | Prior work has explored the selection of examples for in-context learning, neglecting the internal relationships between examples and exist an inconsistency between training and inference. |
| Approach: | They propose a sequential-aware method that leverages the LLM’s feedback on varying context, aiding in capturing inter-relationships and sequential information among examples. |
| Outcome: | Experiments on 23 NLP tasks show that Se2 surpasses baselines and achieves 42% relative improvement over random selection. |
Slender-Mamba: Fully Quantized Mamba in 1.58 Bits From Head to Toe (2025.coling-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have achieved significant performance improvements in natural language processing domain, but require large computational resources for training and inference. |
| Approach: | They propose to use a language model architecture based on State-Space Models to quantify embedding and projection layers of a model with 150 B tokens from scratch. |
| Outcome: | The proposed language model architecture reduces costs by compressing context windows during inference while reducing the cost of training and inference. |
Twist Decoding: Diverse Generators Guide Each Other (2022.emnlp-main)
Copied to clipboard
Jungo Kasai, Keisuke Sakaguchi, Ronan Le Bras, Hao Peng, Ximing Lu, Dragomir Radev, Yejin Choi, Noah A. Smith
| Challenge: | Using a variety of language generation models, ensembling models is challenging during inference. |
| Approach: | They propose a method that decodes text models that do not assume a shared vocabulary, tokenization or generation order. |
| Outcome: | The proposed method outperforms models decoded in isolation over various scenarios. |
GVdoc - Graph-based Visual DOcument Classification (2023.findings-acl)
Copied to clipboard
| Challenge: | Visual document classifiers have shown impressive performance on in-distribution test sets, but they have a hard time classifying out-of-difference examples. |
| Approach: | They propose a graph-based document classification model that generates a document graph based on its layout and trains a neural network to learn node and graph embeddings. |
| Outcome: | The proposed model outperforms state-of-the-art models on out-of distribution data while maintaining comparable performance on in-distribution data. |
Loss in Translation: Learning Bilingual Word Mapping with a Retrieval Criterion (D18-1)
Copied to clipboard
| Challenge: | Existing approaches to learn orthogonal matrix aligning bilingual lexicons are suboptimal . resulting models suffer from "hubness problem" because word vectors tend to be nearest neighbors of abnormally high number of other words. |
| Approach: | They propose a unified formulation that directly optimizes a retrieval criterion in an end-to-end fashion. |
| Outcome: | The proposed approach outperforms the state-of-the-art on word translation on standard benchmarks. |
Understanding and Improving the Robustness of Terminology Constraints in Neural Machine Translation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing terminology constraint test sets are blind to this issue due to oversimplified settings . PH methods retain high constraint accuracy but lower translation quality . |
| Approach: | They propose a method that replaces terminology terms with ordered labels . placeholder methods are better at retaining high constraint accuracy but lower translation quality . |
| Outcome: | The proposed method achieves high accuracy and translation quality regardless of the number or length of constraints. |
TOME: A Two-stage Approach for Model-based Retrieval (2023.acl-long)
Copied to clipboard
| Challenge: | Recent research has focused on model-based retrieval, which discards the index in the traditional retrieval model and memorizes the candidate corpora using model parameters. |
| Approach: | They propose a model-based retrieval approach that discards the index in the traditional retrieval model and memorizes the candidate corpora using model parameters. |
| Outcome: | The proposed approach eliminates the index in the traditional retrieval model and memorizes the candidate corpora using model parameters. |
PEMA: An Offsite-Tunable Plug-in External Memory Adaptation for Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) show impressive performance in various downstream NLP tasks. |
| Approach: | They propose a Parameter-Efficient Fine-Tuning method that integrates with context representations from test data to perform downstream tasks. |
| Outcome: | The proposed method outperforms other methods in memory and latency efficiency and maintains sentence meaning and generating appropriate language and styles. |
Learning to Search Effective Example Sequences for In-Context Learning (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods address these factors in isolation, overlooking their interdependencies. Existing approaches focus on sequence selection, while focusing on the sequence of examples. |
| Approach: | They propose a method that considers key factors involved in sequence selection and incrementally builds the sequence. |
| Outcome: | Experiments across various datasets and language models show that the proposed method significantly reduces the search space and improves performance. |
Compressing Language Models for Specialized Domains (2026.eacl-long)
Copied to clipboard
| Challenge: | Language models (LMs) excel at tasks across diverse domains, yet require substantial computational resources during inference. |
| Approach: | They propose a calibration method to improve the in-domain performance of compressed LMs in a post-training setting. |
| Outcome: | The proposed method outperforms existing methods on domain-specific tasks while preserving general performance. |
The Cost of Compression: Investigating the Impact of Compression on Parametric Knowledge in Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing research on LLM compression focuses on general metrics like perplexity or downstream task accuracy. |
| Approach: | They propose to quantify the effect of pruning and quantization on model quality . they use the LAMA and LM-Harness benchmarks to quantify compression techniques . |
| Outcome: | The proposed compression techniques provide faster inference, smaller memory footprints, and enables local deployment. |
Learning from Natural Language Explanations for Generalizable Entity Matching (2024.emnlp-main)
Copied to clipboard
| Challenge: | Entity matching is the task of linking records from different sources that refer to the same real-world entity. |
| Approach: | They propose to "distill" LLM reasoning into smaller entity matching models via natural language explanations. |
| Outcome: | The proposed model distillation approach achieves strong performance on out-of-domain generalization tests (10.85% F-1). |
Dynamic Programming in Rank Space: Scaling Structured Inference with Low-Rank HMMs and PCFGs (2022.naacl-main)
Copied to clipboard
| Challenge: | Hidden Markov Models (HMMs) and Probabilistic Context-Free Grammars (PCFGs) are widely used structured models. |
| Approach: | They use tensor rank decomposition to reduce computational complexities for a subset of FGGs subsuming HMMs and PCFGs. |
| Outcome: | The proposed model performs better on HMM modeling and unsupervised PCFG parsing than previous work. |
HyperExpan: Taxonomy Expansion with Hyperbolic Representation Learning (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing taxonomies have limited coverage due to expensive manual curation process. |
| Approach: | They propose an algorithm that expands existing taxonomies to preserve their structure in a more expressive hyperbolic embedding space and learns to represent concepts and their relations with a hyperbolical Graph Neural Network. |
| Outcome: | The proposed algorithm outperforms baseline models with representation learning in a Euclidean feature space and achieves state-of-the-art performance on the taxonomy expansion benchmarks. |
Personalized Text Generation with Contrastive Activation Steering (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to personalized text generation rely on retrieval-augmented generation and parameter-efficient fine-tuning. |
| Approach: | They propose a training-free framework that disentangles and represents personalized writing style as a vector in LLM’s activation-space. |
| Outcome: | The proposed framework achieves 8% relative improvement in personalized generation while reducing storage requirements by 1700 over PEFT method. |
Enhancing Hierarchical Text Classification through Knowledge Graph Integration (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to hierarchical text classification are limited by lack of domain knowledge, which leads to mistakes in a variety of situations. |
| Approach: | They propose a Knowledge-enabled Hierarchical Text Classification model which integrates knowledge graphs into HTC to address the knowledge limitations of traditional methods. |
| Outcome: | The proposed model integrates knowledge graphs into the hierarchical text classification process, addressing the knowledge limitations of traditional methods. |
If You Want to Go Far Go Together: Unsupervised Joint Candidate Evidence Retrieval for Multi-hop Question Answering (2021.naacl-main)
Copied to clipboard
| Challenge: | et al. : evidence retrieval is highly dependent on partial, incorrect or no supporting knowledge. |
| Approach: | They propose a method that retrieves and reranks evidence facts jointly . they propose to account for links between sentences and coverage with the given query . |
| Outcome: | The proposed approach achieves state-of-the-art evidence retrieval performance on two multi-hop question answering datasets. |
Alternative Input Signals Ease Transfer in Multilingual Machine Translation (2022.acl-long)
Copied to clipboard
| Challenge: | Recent work in multilingual machine translation (MMT) has focused on the potential of positive transfer between languages. |
| Approach: | They propose to augment training data with alternative signals that unify different writing systems, such as phonetic, romanized, and transliterated input. |
| Outcome: | The proposed model outperforms strong ensemble baselines on Indic and Turkic languages by 1.3 BLEU points on both languages. |
Jam or Cream First? Modeling Ambiguity in Neural Machine Translation with SCONES (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing neural machine translation models learn the probability P (y|x) of the target sentence given the source sentence x. |
| Approach: | They propose to replace softmax activation with a multi-label classification layer that can model ambiguity more effectively. |
| Outcome: | The proposed multi-label classification layer can model ambiguity more effectively . it yields consistent BLEU score gains across six translation directions . |
LaCo: Large Language Model Pruning via Layer Collapse (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for model quantization, knowledge distillation, and model pruning are limited by hardware support limitations and the need for extensive training. |
| Approach: | They propose a layer-wise structured pruner that collapses rear model layers into a prior layer and enables a rapid reduction in model size while preserving the model structure. |
| Outcome: | The proposed pruner outperforms state-of-the-art pruning methods at pruning ratios of 25-30% and maintains an average task performance of over 80% at different pruning ratio. |
The Reasoning Trap: How Enhancing LLM Reasoning Amplifies Tool Hallucination (2026.acl-long)
Copied to clipboard
| Challenge: | Recent studies suggest that strengthening reasoning often coincides with increased hallucination . however, no prior work has examined whether reasoning enhancement itself causes tool hallucinism . |
| Approach: | They propose a diagnostic benchmark measuring tool hallucination in two failure modes . they demonstrate a causal relationship between enhancing reasoning and tool hallubulation . |
| Outcome: | The proposed benchmark measures tool hallucination in two failure modes: no tool available, and (ii) only distractor tools available. |
HERMES: KV Cache as Hierarchical Memory for Efficient Streaming Video Understanding (2026.acl-long)
Copied to clipboard
| Challenge: | Existing models struggle to maintain stable understanding performance and low GPU memory overhead. |
| Approach: | They propose a training-free architecture for real-time and accurate understanding of video streams . HERMES reuses a compact KV cache, enabling efficient streaming understanding . |
| Outcome: | The proposed architecture achieves 10 faster TTFT compared to prior SOTA. |
Test-Time Scaling in Multimodal Foundation Models: A Comprehensive Survey of Generation and Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent advances have adapted this paradigm to Multimodal Foundation Models (MFMs), unlocking their potential in multimodal reasoning and generation. |
| Approach: | They propose a taxonomy framework that categorizes existing methodologies into three distinct strategies: sampling-based, feedback-based and search-based approaches. |
| Outcome: | The proposed framework categorizes existing methodologies into three distinct strategies: sampling-based, feedback-based and search-based approaches. |
On the Multilingual Ability of Decoder-based Pre-trained Language Models: Finding and Controlling Language-Specific Neurons (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing decoder-based pre-trained language models demonstrate excellent multilingual capabilities, but it is unclear how they handle multilingualism. |
| Approach: | They propose to examine the neuron-level internal behavior of decoder-based PLMs by finding neurons that fire “uniquely for each language” within decoded PLM models. |
| Outcome: | The proposed models fire “uniquely for each language” and show that language-specific neurons are unique, with a slight overlap (5%) between languages. |
Consistent Accelerated Inference via Confident Adaptive Transformers (2021.emnlp-main)
Copied to clipboard
| Challenge: | Amortized or approximate computational methods increase efficiency, but can result in unpredictable performance costs. |
| Approach: | They propose a method that increases computational efficiency while guaranteeing a specifiable degree of consistency with the original model with high confidence. |
| Outcome: | The proposed method improves on four classification and regression tasks and can be used to predict the performance of the proposed model. |
Nearest Neighbor Knowledge Distillation for Neural Machine Translation (2022.naacl-main)
Copied to clipboard
| Challenge: | k-nearest-neighbor machine translation (kNN-MT) is a state-of-the-art machine translation technique . however, it requires conducting kNN searches for each decoding step, which increases the cost of decoding . |
| Approach: | They propose to move the time-consuming kNN search forward to the preprocessing phase and introduce k Nearest Neighbor Knowledge Distillation (kNN-KD) that trains the base NMT model to directly learn the knowledge of kN. |
| Outcome: | The proposed method improves over the state-of-the-art model while maintaining the same training and decoding speed as the standard model. |
DEMix Layers: Disentangling Domains for Modular Language Modeling (2022.naacl-main)
Copied to clipboard
| Challenge: | Extensive experiments with autoregressive transformer LMs show that DEMix layers reduce test-time perplexity and increase training efficiency. |
| Approach: | They introduce a new domain expert mixture layer that enables conditioning a language model on the domain of the input text. |
| Outcome: | Experiments with 1.3B LMs show that DEMix layers reduce test-time perplexity, increase training efficiency, and enable rapid adaptation. |
Iterative Knowledge Graph Refinement and Integration for Medical Question Answering (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing graph-based RAG methods heuristically retrieve and refine question-relevant subgraphs, potentially introducing redundant and noisy factual information that is difficult for LLMs to process. |
| Approach: | They propose to integrate knowledge graphs (KGs) through retrieval-augmented generation methods to improve LLM reasoning by incorporating external trustworthy knowledge resources. |
| Outcome: | The proposed framework achieves state-of-the-art against baseline competitors on three medical QA benchmark datasets. |
Accelerating Inference for Pretrained Language Models by Unified Multi-Perspective Early Exiting (2022.coling-1)
Copied to clipboard
| Challenge: | Existing competitive methods to accelerate inference of pretrained language models are limited by their complexity and computational consumption. |
| Approach: | They propose a unified horizontal and vertical multi-perspective early exiting framework to accelerate inference of transformer-based models. |
| Outcome: | Experiments show that MPEE can achieve higher acceleration inference with competent performance than existing competitive methods. |
Modeling Label Semantics for Predicting Emotional Reactions (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for predicting how events induce emotions ignore the semantics of the labels themselves. |
| Approach: | They propose that the semantics of emotion labels can guide a model’s attention when representing the input story. |
| Outcome: | The proposed model can model the semantics of emotion labels and track correlations on unlabeled data. |
NormBank: A Knowledge Bank of Situational Social Norms (2023.acl-long)
Copied to clipboard
| Challenge: | NormBank is a knowledge bank of 155k situational norms that can be used to ground flexible normative reasoning for interactive, assistive, and collaborative AI systems. |
| Approach: | They propose a new scheme for hierarchically organizing the seemingly unbounded social norms within a multivalent sociocultural frame. |
| Outcome: | The proposed framework can be used to ground flexible reasoning for interactive, assistive, and collaborative AI systems. |
Inductive Relation Inference of Knowledge Graph Enhanced by Ontology Information (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to inference knowledge graphs lack ontology information, which is often too sparse. |
| Approach: | They propose a knowledge graph inductive inference method that fuses ontology information to learn the semantic information of entities. |
| Outcome: | The proposed method outperforms large language models like ChatGPT on two benchmark datasets and improves the MRR metrics by 15.4% and 44.1%, respectively. |
Investigating the Benefits of Free-Form Rationales (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that crowdsourced rationales provide additional background knowledge to models . a qualitative study shows generated rationale is not as useful for humans as crowdsourced ones . |
| Approach: | They investigate whether crowdsourced rationales provide additional background knowledge to models . they find that ECQA rationale provides additional background information to understand a decision . |
| Outcome: | The results show that ECQA rationales provide additional background knowledge to understand a decision . compared to crowdsourced rationale, generated rationale is not as useful for humans . |
Extremely Low Bit Transformer Quantization for On-Device Neural Machine Translation (2020.findings-emnlp)
Copied to clipboard
Insoo Chung, Byeongwook Kim, Yoonjung Choi, Se Jung Kwon, Yongkweon Jeon, Baeseong Park, Sangha Kim, Dongsoo Lee
| Challenge: | Quantization is an effective technique to address heavy computation load and memory overhead during inference. |
| Approach: | They propose a low-bit quantization strategy to represent Transformer weights by an extremely low number of bits. |
| Outcome: | The proposed model achieves 11.8 smaller model size than baseline model, with less than -0.5 BLEU. |
DocNLI: A Large-scale Dataset for Document-level Natural Language Inference (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing studies focus on sentence-level inference, which limits its application in downstream NLP problems. |
| Approach: | They propose to construct a large-scale dataset for document-level NLI that can be used to study NLP problems. |
| Outcome: | The proposed model performs well on popular sentence-level benchmarks and generalizes well to out-of-domain NLP tasks that rely on inference at document granularity. |
Adversarial Learning on the Latent Space for Diverse Dialog Generation (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for dialog generation generate generic utterances, e.g., always generating "I don't know" |
| Approach: | They propose a framework that uses generative adversarial nets to generate conditioned responses in dialogs. |
| Outcome: | The proposed model generates more fluent, relevant, and diverse responses than state-of-the-art methods. |
Amphista: Bi-directional Multi-head Decoding for Accelerating LLM Inference (2025.naacl-long)
Copied to clipboard
Zeping Li, Xinlong Yang, Ziheng Gao, Ji Liu, Guanchen Li, Zhuang Liu, Dong Li, Jinzhang Peng, Lu Tian, Emad Barsoum
| Challenge: | Existing methods such as Medusa lack adequate information interaction between different drafting heads. |
| Approach: | They propose an enhanced speculative decoding framework that builds upon Medusa and integrates a drafting block capable of parallel inference. |
| Outcome: | The proposed framework outperforms Medusa in terms of head accuracy and latency. |
DiffuseDef: Improved Robustness to Adversarial Attacks via Iterative Denoising (2025.acl-long)
Copied to clipboard
| Challenge: | Existing adversarial defense methods for natural language processing still pose challenges to adversarials. |
| Approach: | They propose a novel adversarial defense method that incorporates a diffusion layer as a denoiser between the encoder and the classifier. |
| Outcome: | The proposed method improves over existing adversarial defense methods and achieves state-of-the-art performance against black-box and white-box adversarials. |
Gradient-guided Attention Map Editing: Towards Efficient Contextual Hallucination Mitigation (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) often experience “contextual hallucination” where they prioritize self-generated content over input context, leading to a disregard for pertinent details. |
| Approach: | They propose a method that dynamically adjusts attention maps to enhance contextual relevance by using a trained classifier to identify attention maps likely to induce hallucinations. |
| Outcome: | The proposed approach reduces hallucinations across open-source models on summarization and open-book QA tasks. |
VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent research has focused on pushing weight-only quantization to extremely low-bit due to numerical representation limitations. |
| Approach: | They propose a vector-based quantization approach that pushes LLMs to extremely low-bit . they propose scalar-based weight quantization that reduces memory requirements and optimizes storage costs . |
| Outcome: | The proposed method reduces model quantization perplexity by 0.01-0.34 on LLaMA-2, 0.38-0.68 on mistral-7B, 4.41-7.34, on llaMA-3 on QA tasks on average. |
Contrastive Training Improves Zero-Shot Classification of Semi-structured Documents (2023.findings-acl)
Copied to clipboard
| Challenge: | Xu et al., 2020 focus on semi-structured document classification in a zero-shot setting . positional, layout, and style information play a vital role in interpreting such documents . |
| Approach: | They propose a matching-based approach that relies on a pairwise contrastive objective for pretraining and fine-tuning. |
| Outcome: | The proposed method significantly improves Macro F1 in the zero-shot learning setting. |
Distilling Large Embeddings via Hyperspherical Householder Quantization (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for quantizing large embeddings rely on Euclidean quantization, which is poorly aligned with the angular geometry induced by contrastive embeddment training. |
| Approach: | They propose a geometry-aware distillation method that compresses large embeddings into short discrete representations via iterative Householder transformations on the unit hypersphere. |
| Outcome: | The proposed method reduces decoding cost and maintains strong semantic retrieval accuracy. |
Efficient Shapley Values Estimation by Amortization for Text Classification (2023.acl-long)
Copied to clipboard
| Challenge: | Shapley Values are often estimated with a small number of stochastic model evaluations, but this can only be mitigated by aggregating thousands of model evaluation. |
| Approach: | They propose to combine a model with thousands of model evaluations to estimate Shapley Values without additional model evaluation. |
| Outcome: | The proposed model estimates Shapley Values accurately with up to 60 times speedup compared to traditional methods and does not suffer from stability issues as inference is deterministic. |
Joint Biomedical Entity and Relation Extraction with Knowledge-Enhanced Collective Inference (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for information extraction from biomedical texts do not utilize external knowledge . despite the exponential growth of biomedically published articles, many existing methods fall behind . |
| Approach: | They propose a framework that utilizes external knowledge for entity and relation extraction . KECI uses an initial span graph to construct a knowledge graph containing relevant background knowledge . |
| Outcome: | The proposed framework achieves state-of-the-art results in two biomedical datasets . it achieves 4.59% and 4.91% improvement in F1 scores over the state- of-the art methods . |
Are Visual-Linguistic Models Commonsense Knowledge Bases? (2022.coling-1)
Copied to clipboard
| Challenge: | PTLMs are used to extract knowledge from text on demand. |
| Approach: | They compare visual-linguistic and language-only visual-language models in a zero-shot commonsense question answering inference task. |
| Outcome: | The proposed models are highly promising on certain types of commonsense knowledge associated with the visual world. |
Supervised and Unsupervised Probing of Shortcut Learning: Case Study on the Emergence and Evolution of Syntactic Heuristics in BERT (2025.findings-acl)
Copied to clipboard
| Challenge: | Contemporary language models (LMs) rely on shortcut learning, using superficial cues that are spuriously correlated with labels. |
| Approach: | They propose to use syntactic heuristics to learn shortcuts in BERT when performing a task in Natural Language Understanding to investigate where these shortcuts emerge, how they evolve and how they impact the latent knowledge of the LM. |
| Outcome: | The proposed model rely on syntactic heuristics when performing a task in Natural Language Understanding. |
Multi-Aspect Controllable Text Generation with Disentangled Counterfactual Augmentation (2024.acl-long)
Copied to clipboard
| Challenge: | Existing studies neglect attribute correlations formed by the intertwining of different attributes. |
| Approach: | They propose a multi-aspect controllable text generation method with disentangled counterfactual augmentation that alleviates imbalanced attribute correlations during training by disentanglement. |
| Outcome: | The proposed method outperforms state-of-the-art methods in imbalanced and balanced attribute correlation scenarios. |
Stop Looking for “Important Tokens” in Multimodal Language Models: Duplication Matters More (2025.emnlp-main)
Copied to clipboard
Zichen Wen, Yifeng Gao, Shaobo Wang, Junyuan Zhang, Qintong Zhang, Weijia Li, Conghui He, Linfeng Zhang
| Challenge: | Vision tokens in multimodal large language models often dominate computational overhead due to excessive length compared to linguistic modality. |
| Approach: | They propose a token pruning method which defines an importance criterion for vision tokens and prunes the unimportant vision token during inference. |
| Outcome: | The proposed method can prune 88.9% of vision tokens while maintaining comparable performance. |
Bi-Chainer: Automated Large Language Models Reasoning with Bidirectional Chaining (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing unidirectional chaining methods suffer from low prediction accuracy and efficiency. |
| Approach: | They propose a bidirectional chaining method which dynamically switches to depth-first reasoning in the opposite reasoning direction when it encounters multiple branching options within the current direction. |
| Outcome: | The proposed method achieves sizable accuracy boots over unidirectional chaining frameworks on four challenging logical reasoning datasets. |
Linear Recency Bias During Training Improves Transformers’ Fit to Reading Times (2025.coling-main)
Copied to clipboard
| Challenge: | Recent research has shown a strong fit between surprisal values from Transformers and reading times. |
| Approach: | They evaluate a Transformer model that uses a recency bias added to attention scores to improve the fit to human reading times. |
| Outcome: | The proposed model improves on a Transformer that includes a recency bias added to attention scores. |
Learning Dense Representations of Phrases at Scale (2021.acl-long)
Copied to clipboard
| Challenge: | Existing phrase retrieval models rely on sparse representations and still underperform retriever-reader approaches. |
| Approach: | They propose a method to learn phrase representations from reading comprehension tasks using negative sampling methods. |
| Outcome: | The proposed model improves over previous models by 15%-25% absolute accuracy and matches the performance of state-of-the-art retrieval models. |
Beyond Recognising Entailment: Formalising Natural Language Inference from an Argumentative Perspective (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for recognizing textual entailment lack a standardized definition of inference, making it difficult to compare methods trained on different datasets. |
| Approach: | They propose a rigorous approach to align entailment recognition with argumentation theory by using a tool to assist humans in annotating arguments according to the PTA. |
| Outcome: | The proposed model is based on a human-trained dataset and provides insights into non-expert annotator training. |
An Imitation Learning Curriculum for Text Editing with Non-Autoregressive Models (2022.acl-long)
Copied to clipboard
| Challenge: | et al. (2017) show that imitation learning algorithms for machine translation introduce mismatches between training and inference that lead to undertraining and poor generalization in editing scenarios. |
| Approach: | They propose a framework for training non-autoregressive sequence-to-sequence models for editing tasks where the original input sequence is iteratively edited to produce the output. |
| Outcome: | The proposed framework significantly improves output quality and controls complexity better on the simplification task. |
Bit-Flip Error Resilience in LLMs: A Comprehensive Analysis and Defense Framework (2025.emnlp-main)
Copied to clipboard
Yuhang Chen, Zhen Tan, Ajay Kumar Jaiswal, Huaizhi Qu, Xinyu Zhao, Qi Lin, Yu Cheng, Andrew Kwong, Zhichao Cao, Tianlong Chen
| Challenge: | Bit-flip errors (BFEs) are hardware faults where individual bits in memory or processing units are unintentionally flipped. |
| Approach: | They propose a novel defense strategy to mitigate bit-flip errors (BFEs) they propose bfe protection and a self-correction mechanism to minimize performance degradation . |
| Outcome: | The proposed defense strategy minimizes performance degradation while significantly improving robustness against BFEs. |
PrivacyRestore: Privacy-Preserving Inference in Large Language Models via Privacy Removal and Restoration (2025.acl-long)
Copied to clipboard
| Challenge: | Existing privacy protection methods for large language models suffer from performance degradation or large inference time overhead. |
| Approach: | They propose a plug-and-play method to protect the privacy of user inputs during LLM inference . they use offline restoration vectors to train restoration vector for each privacy span type . |
| Outcome: | The proposed method can prevent the linear growth of the privacy budget. |
Hybrid Self-evolving Structured Memory for Computer-Use Agents (2026.findings-acl)
Copied to clipboard
| Challenge: | despite advances in vision–language models, real-world computer-use tasks remain challenging due to long-horizon workflows, diverse interfaces, and frequent intermediate errors. |
| Approach: | They propose a graph-based memory that couples discrete symbolic nodes with continuous trajectory embeddings. |
| Outcome: | The proposed system outperforms closed-source models in Qwen2.5-VL-7B and Gemini2.5-Pro-Vision on desktop and mobile platforms. |
Dynamic Parallel Tree Search for Efficient LLM Reasoning (2025.acl-long)
Copied to clipboard
Yifu Ding, Wentao Jiang, Shunyu Liu, Yongcheng Jing, Jinyang Guo, Yingjie Wang, Jing Zhang, Zengmao Wang, Ziwei Liu, Bo Du, Xianglong Liu, Dacheng Tao
| Challenge: | Recent methods focus on search accuracy while overlooking computational efficiency. |
| Approach: | They propose a parallelism framework that dynamically optimizes reasoning path in inference. |
| Outcome: | The proposed framework improves efficiency by 2-4 on average while maintaining or even surpassing existing reasoning algorithms in accuracy. |
Momentum Posterior Regularization for Multi-hop Dense Retrieval (2025.coling-main)
Copied to clipboard
| Challenge: | Current methods for knowledge distillation in one-time retrieval are ineffective for multi-hop QA . posterior information is often defined as the response, which may not connect to the query without intermediate retrieval . |
| Approach: | They propose to distill knowledge from a posterior retrieval into a prior retrieval for multi-hop QA . they propose to use momentum moving average method to update posterior information along with prior retrievals . |
| Outcome: | Experiments on HotpotQA and StrategyQA show that MoPo outperforms baselines in retrieval and downstream QA tasks. |
Beyond Single-Event Extraction: Towards Efficient Document-Level Multi-Event Argument Extraction (2024.findings-acl)
Copied to clipboard
Wanlong Liu, Li Zhou, DingYi Zeng, Yichen Xiao, Shaohuan Cheng, Chen Zhang, Grandee Lee, Malu Zhang, Wenyu Chen
| Challenge: | mainstream event argument extraction methods process each event in isolation, resulting in inefficient inference and ignoring correlations among multiple events. |
| Approach: | They propose a multi-event argument argument extraction model which extracts arguments from all events simultaneously. |
| Outcome: | The proposed model performs better on four public datasets while saving time. |
Multimodal Transformer Networks for End-to-End Video-Grounded Dialogue Systems (P19-1)
Copied to clipboard
| Challenge: | Existing work on video-grounded dialogue systems is limited by feature space and semantic information. |
| Approach: | They propose multimodal transformer networks to encode videos and incorporate information from different modalities. |
| Outcome: | The proposed system generates appropriate conversational response to queries of humans based on visual and audio aspects of a given video . it also generalizes to another multimodal visual-grounded dialogue task, and obtains promising performance. |
Unifying Continuous and Discrete Text Diffusion with Non-simultaneous Diffusion Processes (2025.acl-long)
Copied to clipboard
| Challenge: | Experimental results demonstrate NeoDiff’s superior performance compared to baselines of non-autoregressive continuous and discrete diffusion models, iterative-based methods and autoregressive diffusion-based approaches. |
| Approach: | They propose a discrete and continuous diffusion model that integrates the strengths of discrete, continuous and continuous approaches. |
| Outcome: | The proposed model unifies the theories of discrete and continuous diffusion models, offering a more principled and effective framework for text generation. |
Sparsifying Transformer Models with Trainable Representation Pooling (2022.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to sparsify attention in the Transformer model are based on quadratic memory complexity and a lack of information for each word. |
| Approach: | They propose a method to sparsify attention in a Transformer model by learning to select the most-informative token representations during the training process. |
| Outcome: | The proposed model performs better than the current SOTA model while being 1.8 faster during training, 4.5 faster inference and 13 more efficient in the decoder. |
The Right Tool for the Job: Matching Model and Instance Complexities (2020.acl-main)
Copied to clipboard
| Challenge: | a large increase in the size of NLP models can increase production costs and reduce adoption on real-time devices. |
| Approach: | They propose a modification to contextual representation fine-tuning which allows for an early exit from neural network calculations for simple instances and late exit for hard instances. |
| Outcome: | The proposed method produces models which are up to five times faster than the state of the art while preserving their accuracy. |
Accelerating Multiple Intent Detection and Slot Filling via Targeted Knowledge Distillation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing non-autoregressive Spoken Language Understanding models suffer from multi-modality problem . current methods have little prior knowledge about the reference during inference . |
| Approach: | They propose a Targeted Knowledge Distillation Framework (TKDF) for multi-intent SLU that utilizes the knowledge distillation method to improve the performance. |
| Outcome: | The proposed model outperforms existing models on two public multi-intent datasets while speeding up by over 4.5 times. |
Optima: Optimizing Effectiveness and Efficiency for LLM-Based Multi-Agent System (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have emerged as powerful tools for a wide range of tasks, from * Equal Contribution. |
| Approach: | They propose a framework that enhances communication efficiency and task effectiveness in LLM-based multi-agent systems through training. |
| Outcome: | The proposed framework improves communication efficiency and task effectiveness on multi-agent tasks with 2.8x performance gain with less than 10% tokens on tasks requiring heavy information exchange. |
Energy and Carbon Considerations of Fine-Tuning BERT (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work quantifying energy costs and associated carbon emissions has focused on pretraining and fine-tuning. |
| Approach: | They perform an empirical study to quantify the energy requirements of language model fine-tuning in the context of pretraining and inference. |
| Outcome: | The proposed model fine-tuning energy and carbon footprints are compared with pre-training and inference energy requirements and outline recommendations for NLP researchers and practitioners. |
Distilled Dual-Encoder Model for Vision-Language Understanding (2022.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that the proposed cross-modal attention distillation is crucial to the success of our framework. |
| Approach: | They propose a framework that distills knowledge of fusion-encoder teacher into dual-encoding student model. |
| Outcome: | The proposed model is competitive with the fusion-encoder teacher model in performance, but suffers from a lack of deep cross-modal interactions. |
HARP: Hesitation-Aware Reframing in Transformer Inference Pass (2025.naacl-long)
Copied to clipboard
| Challenge: | a recent study has shown that inference steps are not equally challenging, with some being "harder" and others "easier." |
| Approach: | They propose a modified Transformer forward pass that selectively applies additional computation when the model encounters uncertainty during token generation. |
| Outcome: | The proposed method achieves performance gains while maintaining inference times twice faster than beam search. |
Towards Inter-character Relationship-driven Story Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent story generation methods can generate stories based on open-ended prompts and planners but can neither encode character relationships nor give explicit control over the characters and their relationships. |
| Approach: | They propose a model that uses relationships as latent variables for story generation and propose 'relationship-driven' story generation. |
| Outcome: | The proposed model generates stories sentence by sentence with relationships that are more faithful to desired relationships while maintaining the content quality. |
Language Model Prior for Low-Resource Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Neural machine translation is based on large parallel corpora and requires expensive training and training. |
| Approach: | They propose to incorporate a LM as prior in a neural translation model (TM) they add a regularization term which pushes the output distributions to be probable under the LM prior . |
| Outcome: | The proposed approach does not compromise decoding speed, because the LM is used only at training time, unlike previous work that requires it during inference. |
TroL: Traversal of Layers for Large Language and Vision Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing open-source LLVMs that perform comparably to closed-source models such as GPT-4V are often considered too large, having a larger number of layers. |
| Approach: | They propose a new efficient LLVM family with 1.8B, 3.8B, and 7B LLM model sizes, Traversal of Layers, which enables the reuse of layers in a token-wise manner. |
| Outcome: | The proposed model outperforms open-source models with larger model sizes and outperformed closed-source LLVMs with substantial models. |
Disentangle to Decay: Linear Attention with Trainable Decay Factor (2025.coling-main)
Copied to clipboard
| Challenge: | Existing linear attention models use a decay factor based positional encoding (PE), but the decay factor is manually designed and non-trainable, limiting further optimization. |
| Approach: | They propose a PE-based positional encoding that disentangles decay factor into two parts to achieve further optimization and stable training. |
| Outcome: | The proposed model achieves stable training of decay factor and improves inference efficiency in normal context and extrapolation scenarios. |
GAProtoNet: A Multi-head Graph Attention-based Prototypical Network for Interpretable Text Classification (2025.coling-main)
Copied to clipboard
| Challenge: | Existing models with black-box nature obscure decision-making process and lack interpretability. |
| Approach: | They propose a multi-head graph attention-based prototypical network that uses a vector and prototypes to learn an interpretable prototypical representation. |
| Outcome: | The proposed model achieves superior results without sacrificing the accuracy of the original black-box LMs. |
Parallel Refinements for Lexically Constrained Text Generation with BART (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing work injects lexical constraints into the output, which generates generic or ungrammatical sentences and has high computational complexity. |
| Approach: | They propose a model that incorporates pre-specified keywords into the output to control the generated text. |
| Outcome: | The proposed model decomposes the generated text into two sub-tasks and improves the sentence quality. |
FastMem: Fast Memorization of Prompt Improves Context Awareness of Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models struggle with context awareness, leading to inaccuracies in tasks requiring faithful adherence to provided information. |
| Approach: | They propose a method to enhance LLMs' context awareness by updating only the last Feed-Forward Network module to maximize the likelihood of the prompt before inference . |
| Outcome: | The proposed method improves the accuracy of Llama 3-8B-Inst on the NQ-SWAP dataset from 59.1% to 71.6% and reduces the output structure failure rate of Qwen 1.5-4B-Chat from 34.9% to 25.5%. |
Bridging the Granularity Gap for Acoustic Modeling (2023.findings-acl)
Copied to clipboard
Chen Xu, Yuhao Zhang, Chengbo Jiao, Xiaoqian Liu, Chi Hu, Xin Zeng, Tong Xiao, Anxiang Ma, Huizhen Wang, Jingbo Zhu
| Challenge: | Despite the success of speech recognition, how to encode the speech features effectively remains an open problem. |
| Approach: | They propose a Progressive Down-Sampling technique which compresses acoustic features into coarser-grained units containing more complete semantic information, like text-level representation. |
| Outcome: | The proposed method yields comparable or better results on the speech recognition task and inference speedups ranging from 1.20x to 1.47x. |
Structured Pruning for Efficient Generative Pre-trained Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Large-scale generative Pre-trained Language Models (PLMs) are limited in their deployment in real-world applications. |
| Approach: | They propose to prune the feed-forward networks of generative pre-trained language models to smaller widths without designing extra operators. |
| Outcome: | The proposed method achieves 1.51x/6.96x inference speedup on GPU/CPU with 67% size reduction. |
Zero-shot Sharpness-Aware Quantization for Pre-trained Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing zero-shot quantization methods are based on overfitting problem in adversarial learning process, leading to sub-optimal performance. |
| Approach: | They propose a zero-shot sharpness-aware quantization framework for the quantization of various PLMs by optimizing a minimax problem. |
| Outcome: | The proposed framework can achieve significant performance gains on discriminative and generative PLMs. |
How Far Is Too Far? Studying the Effects of Domain Discrepancy on Masked Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Pre-trained masked language models perform strongly on a wide variety of NLP tasks. |
| Approach: | They propose a mechanism to quantify the difference in domains between the pre-trained model and the task and partition it using a cloze task. |
| Outcome: | The proposed model performs better on openly available e-commerce datasets than the original model on scientific and biomedical datasets. |
Can Diffusion Model Achieve Better Performance in Text Generation ? Bridging the Gap between Training and Inference ! (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing models for text generation use a discrete data embedding module to map the data into the continuous space. |
| Approach: | They propose two methods to bridge the gap between training and inference by mapping the discrete text into the continuous space. |
| Outcome: | The proposed methods can achieve 100 200 speedup with better performance on 6 generation tasks. |
Prediction-Augmented Generation for Automatic Diagnosis Tasks (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) adopt autoregressive architecture, predicting the next word token based on the preceding context. |
| Approach: | They propose a method that integrates task-specific predictive models as external tools to improve model generation quality and accuracy. |
| Outcome: | The proposed method improves the generation quality and predictive accuracy of large language models in inference-driven tasks. |
Efficient Long Context Language Model Retrieval with Compression (2025.acl-long)
Copied to clipboard
| Challenge: | Long Context Language Models (LCLMs) are a new paradigm for Information Retrieval . however, processing large number of passages within in-context for retrieval is computationally expensive . a proposed compression approach for LCLM retrieval maximizes retrieval performance while minimizing the length of the compressed passages. |
| Approach: | They propose a new compression approach tailored to maximize retrieval performance while minimizing the length of compressed passages. |
| Outcome: | The proposed approach maximizes retrieval performance while minimizing the length of compressed passages while reducing the in-context size by 1.91. |
Domain Generalizable AI Guardrails with Augmented Policy Training (2026.acl-long)
Copied to clipboard
| Challenge: | Current guardrails overfit the training policies, preventing adaptation to new domains and policies. |
| Approach: | They propose a training recipe that uses a suite of policy perturbation strategies to reduce overfitting and increase generalization to guardrails. |
| Outcome: | The proposed training recipe reduces overfitting and increases generalization on unseen policies and achieves comparable or better performance than existing 8B guardrails on unsen policies. |
LlmLink: Dual LLMs for Dynamic Entity Linking on Long Narratives with Collaborative Memorisation and Prompt Optimisation (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods focus on supervised fine-tuning or limited to one-off prediction, which poses a challenge where the context is long. |
| Approach: | They propose a dynamic approach to CoREFerence resolution in chunked long narratives by deploying dual Large Language Models. |
| Outcome: | The proposed model achieves performance gains over existing models and fine-tuning approaches on long narrative datasets, significantly reducing the resources required for inference and training. |
LLM Critics Help Catch Bugs in Mathematics: Towards a Better Mathematical Verifier with Natural Language Feedback (2025.findings-acl)
Copied to clipboard
Bofei Gao, Zefan Cai, Runxin Xu, Peiyi Wang, Ce Zheng, Runji Lin, Keming Lu, Dayiheng Liu, Chang Zhou, Wen Xiao, Tianyu Liu, Baobao Chang
| Challenge: | Existing mathematical verifiers are trained with binary classification labels, which are not informative enough for the model to accurately assess the solutions. |
| Approach: | They propose a natural language feedback-enhanced verifier that can validate the correctness of response generated by policy models by constructing automatically generated training data and a two-stage training paradigm. |
| Outcome: | The proposed verifier significantly improves in verification and reinforcement learning and alleviates data-demanding problems of the reward model. |
Masks Can be Learned as an Alternative to Experts (2025.acl-long)
Copied to clipboard
| Challenge: | a recent study shows that sparse activation techniques can reduce inference performance without sacrificing performance. |
| Approach: | They propose to sparsify a pre-trained dense large language model into a mixture-of-experts architecture for faster inference. |
| Outcome: | The proposed approach is more efficient than one-shot sparsification techniques . it achieves 97% performance retention on downstream tasks with only 50% of parameters activated . |
Information Extraction with Differentiable Beam Search on Graph RNNs (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing approaches to information extraction suffer from exposure bias due to discrepancy between training and decoding. |
| Approach: | They propose to cast graph generation as auto-regressive sequence labeling and make it aware of decoding procedure by using differentiable beam search. |
| Outcome: | The proposed model outperforms its non-decoding-aware version on ACE05 and ConLL04 datasets. |
TempTool-R1: Tool-Augmented Reinforcement Learning for Temporal Knowledge Graph Question Answering (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to temporal knowledge graph question answering struggle with multi-hop reasoning and implicit temporal constraints. |
| Approach: | They propose a temporal tool-based API capable of transforming implicit temporal cues into executable operations and supervised fine-tuning teaches the model to interweave chain-of-thought reasoning with think-then-tool usage. |
| Outcome: | The proposed framework outperforms existing methods on three challenging questions. |
Jointly Reparametrized Multi-Layer Adaptation for Efficient and Private Tuning (2023.findings-acl)
Copied to clipboard
| Challenge: | Efficient finetuning of pretrained language transformers requires a large number of tunable parameters. |
| Approach: | They propose a language transformer finetuning strategy that introduces task-specific parameters in multiple transformer layers. |
| Outcome: | The proposed method outperforms other methods with 4,100 parameters on GLUE tasks with 5% of full finetuning performance. |
Prompt-R1: Collaborative Automatic Prompting Framework via End-to-end Reinforcement Learning (2026.findings-acl)
Copied to clipboard
Wenjin Liu, Haoran Luo, Xueyuan Lin, Haoming Liu, Tiesunlong Shen, Jiapu Wang, Rui Mao, Erik Cambria
| Challenge: | Existing large language models are limited in understanding, reasoning, calculation, and generation, limiting their performance in complex reasoning and dynamic tasks. |
| Approach: | They propose a plug-and-play framework that integrates a small-scale LLM (as agent) with large-scale large-level LLMs (a as environment) they propose generating prompts that are used to interact with LLM, and a double constraint reward that optimizes correctness and quality of generation. |
| Outcome: | The proposed framework significantly outperforms baseline large-scale large-language models across various tasks. |
SliceMoE: Routing Embedding Slices Instead of Tokens for Fine-Grained and Balanced Transformer Scaling (2025.emnlp-main)
Copied to clipboard
| Challenge: | Token-level routing assigns an entire semantic spectrum to each expert, creating capacity bottlenecks, load-balancing pathologies, and limited specialisation. |
| Approach: | They propose an architecture that routes contiguous slices of a token’s hidden vector and a lightweight shared router predicts the top-k experts. |
| Outcome: | The proposed architecture achieves 1.7x faster inference than dense baselines, 12–18% lower perplexity than parameter-matched token-MoE, and improved expert balance. |
Studying word order through iterative shuffling (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on large language models has made this hypothesis popular . but, word order is not important enough to make sentence structure relevant . |
| Approach: | They propose an efficient procedure that finds word order having highest likelihood under a fixed language model. |
| Outcome: | The proposed procedure can be used to find the ordering of a bag of words having the highest likelihood under a fixed language model. |
Are Human Explanations Always Helpful? Towards Objective Evaluation of Human Natural Language Explanations (2023.acl-long)
Copied to clipboard
| Challenge: | Human-annotated labels and explanations are critical for training explainable NLP models. |
| Approach: | They propose a metric that measures the usefulness of an explanation for model performance at both fine-tuning and inference. |
| Outcome: | The proposed metric can evaluate the quality of human-annotated explanations, while Simulatability falls short. |
Multivalent Entailment Graphs for Question Answering (2021.emnlp-main)
Copied to clipboard
Nick McKenna, Liane Guillou, Mohammad Javad Hosseini, Sander Bijl de Vroe, Mark Johnson, Mark Steedman
| Challenge: | a recent study shows that drawing inferences between open domain predicates is a necessity for true language understanding. |
| Approach: | They propose to reinterpret the Distributional Inclusion Hypothesis to model entailment between predicates of different valencies. |
| Outcome: | The proposed graphs are more useful than using the same valency evidence, the authors show . they show that drawing on evidence across valencies answers more questions than using only the same evidence. |
Beyond Semantic Similarity: Appraisal-Guided Chain-of-Thought Reasoning and Retrieval for Multimodal Emotional Support Conversations (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing retrieval-augmented generation paradigms rely on semantic similarity to retrieve historical dialogues that are surface analogous but therapeutically incongruent. |
| Approach: | They propose to use appraisal-guided reasoning chains to generate appraisal-based reasoning chains and apply a dual-signal verification mechanism to verify and correct them. |
| Outcome: | Extensive experiments on two ESC benchmarks show that the proposed model significantly outperforms state-of-the-art models. |
Identifying Factual Inconsistencies in Summaries: Grounding LLM Inference via Task Taxonomy (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have focused on specialized BERT-variants and recent LLMs to reason inconsistencies. |
| Approach: | They propose to incorporate task-specific taxonomy into inferences to facilitate both zero-shot and supervised paradigms. |
| Outcome: | The proposed model outperforms specialized non-LLM and recent LLM models in a number of domains. |
AVG-LLaVA: An Efficient Large Multimodal Model with Adaptive Visual Granularity (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing large multimodal models typically divide high-resolution images into multiple local images and a global image, leading to a large number of visual tokens. |
| Approach: | They propose an LMM that can adaptively select the appropriate visual granularity based on the input image and instruction. |
| Outcome: | The proposed model significantly reduces visual tokens and speeds up inference on 11 benchmarks. |
Inferring Functionality of Attention Heads from their Parameters (2025.acl-long)
Copied to clipboard
| Challenge: | Prior work on attention heads focused on analyzing their behavior during inference for specific circuits or tasks. |
| Approach: | They propose a framework that maps the functions of attention heads without any model training or inference. |
| Outcome: | The proposed framework infers the functionality of attention heads without any model training or inference. |
Annotation-Inspired Implicit Discourse Relation Classification with Auxiliary Discourse Connective Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Discourse connectives are words or phrases that signal the presence of a discourse relation. |
| Approach: | They propose a model that generates discourse connectives between arguments and predicts discourse relations based on the generated connectives. |
| Outcome: | The proposed model outperforms baselines on three datasets and is highly accurate. |
SMoP: Towards Efficient and Effective Prompt Tuning with Sparse Mixture-of-Prompts (2023.emnlp-main)
Copied to clipboard
| Challenge: | Prompt tuning has emerged as a successful parameter-efficient alternative to the full fine-tuning of language models. |
| Approach: | They propose a prompt tuning method that utilizes short soft prompts for efficient training and inference while maintaining performance gains typically induced by longer soft prompt. |
| Outcome: | The proposed method outperforms baseline methods while preserving memory usage. |
A Frustratingly Easy Post-Training Quantization Scheme for LLMs (2023.emnlp-main)
Copied to clipboard
| Challenge: | Efficient inference is crucial for hyper-scale AI models, including large language models, as their parameter count continues to increase for enhanced performance. |
| Approach: | They propose a quantization scheme that fully utilizes the Transformer structure used in large language models to minimize the frequency of DRAM access while exploiting the parallelism of operations. |
| Outcome: | The proposed method minimizes the frequency of DRAM access while exploiting the parallelism of operations through a dense matrix format. |
Self-Ensemble: Mitigating Confidence Distortion for Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models exhibit a confidence distortion problem on multichoice question-answering . Self-Ensemble solves this problem by splitting the choices into several groups . |
| Approach: | They propose a method that splits LLM choices into several groups and ensembles them to reach a final decision. |
| Outcome: | The proposed method outperforms standard inference and baseline methods on MCQA. |
DeepMed: Building a Medical DeepResearch Agent via Multi-hop Med-Search Data and Turn-Controlled Agentic Training & Inference (2026.findings-acl)
Copied to clipboard
Zihan Wang, Hao Wang, Shi Feng, Xiaocui Yang, Daling Wang, Yiqun Zhang, Jinghao Lin, Xiaozhong Ji, Haihua Yang
| Challenge: | Medical reasoning models are constrained by parametric knowledge and can induce hallucinations and spurious attributions. |
| Approach: | They propose a model that uses a multi-hop med-search QA synthesis method to apply the DR paradigm in medical contexts. |
| Outcome: | The proposed model outperforms larger medical reasoning models on medical benchmarks. |
Deep Exploration of Cross-Lingual Zero-Shot Generalization in Instruction Tuning (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have focused on instruction tuning to show cross-lingual generalization . a novel non-English meta-dataset is used to study instruction tuning . |
| Approach: | They perform instruction tuning individually for two distinct language meta-datasets and assess the performance on unseen tasks in a non-English language. |
| Outcome: | The proposed model outperforms baseline training in English and Korean by 20.7% and 13.6%. |
Step-level Verifier-guided Hybrid Test-Time Scaling for Large Language Models (2025.emnlp-main)
Copied to clipboard
Kaiyan Chang, Yonghao Shi, Chenglong Wang, Hang Zhou, Chi Hu, Xiaoqian Liu, Yingfeng Luo, Yuan Ge, Tong Xiao, JingBo Zhu
| Challenge: | Recent training-based TTS methods, such as continued reinforcement learning, have surged in popularity, while training-free TTS approaches are gradually fading from prominence. |
| Approach: | They propose a fine-grained sequential scaling method guided by process verification that integrates training-free TTS methods with other classical parallel scaling methods at the step level. |
| Outcome: | Experiments on five instruction-tuned large language models (LLMs) show that training-free TTS methods can extend reasoning performance boundaries. |
RWKV: Reinventing RNNs for the Transformer Era (2023.findings-emnlp)
Copied to clipboard
Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Leon Derczynski, Xingjian Du, Matteo Grella, Kranthi Gv, Xuzheng He, Haowen Hou, Przemyslaw Kazienko, Jan Kocon, Jiaming Kong, Bartłomiej Koptyra, Hayden Lau, Jiaju Lin, Krishna Sri Ipsit Mantri, Ferdinand Mom, Atsushi Saito, Guangyu Song, Xiangru Tang, Johan Wind, Stanisław Woźniak, Zhenyuan Zhang, Qinghua Zhou, Jian Zhu, Rui-Jie Zhu
| Challenge: | recurrent neural networks struggle to match the performance of Transformers due to limitations in parallelization and scalability. |
| Approach: | They propose a model architecture that combines the efficient parallelizable training of transformers with the efficient inference of RNNs. |
| Outcome: | The proposed model performs on par with similarly sized RNNs, suggesting future work can leverage this architecture to create more efficient models. |
Arabic Mini-ClimateGPT : A Climate Change and Sustainability Tailored Arabic LLM (2023.findings-emnlp)
Copied to clipboard
Sahal Mullappilly, Abdelrahman Shaker, Omkar Thawakar, Hisham Cholakkal, Rao Anwer, Salman Khan, Fahad Khan
| Challenge: | Recent large language models like ChatGPT and Bard excel in a wide variety of NLP tasks but are not specifically tailored for climate related domain specific information. |
| Approach: | They propose a lightweight Arabic Mini-ClimateGPT that is built on an open-source LLM and specifically fine-tuned on a conversational-style instruction tuning curated Arabic dataset Clima500-Instruct. |
| Outcome: | The proposed model surpasses the baseline LLM in 88.3% of cases during ChatGPT-based evaluation and human expert prefers it over other open-source models. |
Guideline Learning for In-Context Information Extraction (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models can perform a task by conditioning on task instructions and a few input-output examples without optimizing any parameters. |
| Approach: | They propose a guideline learning framework for In-context IE which reflectively learns and follows guidelines. |
| Outcome: | The proposed framework improves the performance of in-context IE by synthesizing and following guidelines. |
MATTER: Memory-Augmented Transformer Using Heterogeneous Knowledge Sources (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing retrieval-augmented models typically retrieve information from a single type of knowledge source. |
| Approach: | They propose an efficient memory-augmented transformer to retrieve relevant knowledge from multiple knowledge sources. |
| Outcome: | The proposed model outperforms existing retrieval-augmented models on popular QA benchmarks in terms of accuracy and speed. |
Make Prompt-based Black-Box Tuning Colorful: Boosting Model Generalization from Three Orthogonal Perspectives (2024.lrec-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown increasing power on NLP tasks. however, tuning these models for downstream tasks usually requires exorbitant costs. |
| Approach: | They propose a black-box tuning technique that optimizes task-specific prompts without accessing gradients and hidden representations. |
| Outcome: | The proposed method improves performance under few-shot learning scenarios. |
Distributed LLM Serving on Consumer-Grade GPUs by Reconciling Computation and Communication (2025.findings-emnlp)
Copied to clipboard
Lewei Jin, Kui Zhang, Yongqi Chen, null Zhuoyifan, Renjie Li, Yi Gao, Bowei Yang, Zhengong Cai, Wei Dong
| Challenge: | Large language models are reshaping internet services, and serving them is costly. |
| Approach: | They propose an efficient distributed LLM serving system that splits prefill and decode requests into smaller chunks . |
| Outcome: | The proposed system reduces TTFT, TPOT, and latency compared to the state-of-the-art system. |
Profiling-Free Mixed-Precision Quantization for MoE LLMs via Fuzzy Rule Interpolation (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models are scaling in size and capability, driving substantial computational and memory costs. |
| Approach: | They propose a mixed-precision quantization framework that uses fuzzy rule interpolation to predict quantization error from only sparse samples. |
| Outcome: | The proposed framework accelerates the profiling phase by up to 15.7 on DeepSeek-V2 while achieving comparable or slightly superior zero-shot accuracy. |
Looking Beyond Text: Reducing Language Bias in Large Vision-Language Models via Multimodal Dual-Attention and Soft-Image Guidance (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large vision-language models (LVLMs) have been criticized for their language bias. |
| Approach: | They propose to use a dual-attention mechanism to construct separate attention for visual and text inputs to enhance integration of visual inputs across models. |
| Outcome: | Experiments show that the proposed model debiases LVLMs from their language bias, enhancing visual comprehension and reducing hallucinations without additional resources. |
Efficient and Interpretable Grammatical Error Correction with Mixture of Experts (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Error type information has been widely used to improve the performance of grammatical error correction models. |
| Approach: | They propose a mixture-of-experts model for grammatical error correction that uses error type information to generate corrections and combine models. |
| Outcome: | The proposed model achieves the performance of T5-XL with three times fewer effective parameters and produces interpretable corrections by also identifying the error type during inference. |
RACER: Retrieval-Augmented Contextual Rapid Speculative Decoding (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for decoding large language models generate one token per step, causing high inference latency. |
| Approach: | They propose a method that integrates retrieved exact patterns with logit-driven future cues. |
| Outcome: | Experiments on Spec-Bench, HumanEval, and MGSM-ZH show that RACER outperforms training-free methods and accelerates inference. |
CARVQ: Corrective Adaptor with Group Residual Vector Quantization for LLM Embedding Compression (2025.findings-emnlp)
Copied to clipboard
Dayin Gou, Sanghyun Byun, Nilesh Malpeddi, Gabrielle De Micheli, Prathamesh Vaste, Jacob Song, Woo Seong Chung
| Challenge: | Large Language Models typically rely on a large number of parameters for token embedding, leading to substantial storage requirements and memory footprints. |
| Approach: | They propose a corrective Adaptor with group Residual Vector Quantization that can be used to compress the embedding layer without requiring specialized hardware. |
| Outcome: | The proposed corrective adaptor can achieve lower average bitwidth-per-parameter while maintaining reasonable perplexity and accuracy compared to scalar quantization. |
Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospects (2026.findings-acl)
Copied to clipboard
Jun Zhang, Yicheng Ji, Feiyang Ren, Yihang Li, Bowen Zeng, Zonghao Chen, Ke Chen, Lidan Shou, Gang Chen, Huan Li
| Challenge: | Large Vision-Language Models are hindered by a systemic efficiency barrier known as visual token dominance. |
| Approach: | They propose a systematic taxonomy of efficiency techniques structured around the inference lifecycle . they examine visual encoding, prefilling, and decoding to understand bottlenecks . |
| Outcome: | The proposed techniques reveal how upstream decisions dictate downstream bottlenecks . the proposed techniques include hybrid compression and modality-aware decoding . |
BTC-LLM: Efficient Sub-1-Bit LLM Quantization via Learnable Transformation and Binary Codebook (2026.acl-long)
Copied to clipboard
Hao Gu, Lujun Li, Hao Wang, Lei Wang, Zheyu Wang, Bei Liu, Jiacheng Liu, Qiyuan Zhu, Sirui Han, Yike Guo
| Challenge: | Recent sparsity-aware binarization approaches can achieve sub-1-bit compression, but they face performance degradation, mask-management overhead, and limited hardware compatibility. |
| Approach: | They propose a binary quantization framework that leverages binary pattern clustering and weight transformation to overcome performance degradation and mask-management overhead. |
| Outcome: | The proposed framework achieves state-of-the-art compression (1.11–0.7 bits) it maintains high performance with only a 3.1% accuracy drop in zero-shot benchmarks while delivering a 1.6 speedup over FP16. |
MMAG: Multimodal Learning for Mucus Anomaly Grading in Nasal Endoscopy via Semantic Attribute Prompting (2025.emnlp-main)
Copied to clipboard
| Challenge: | Accurate grading of rhinitis severity relies heavily on the characterization of key secretions, notably clear nasal discharge (CND) and purulent nasal secretion (PUS). |
| Approach: | They propose a framework that integrates structured prompts with rank-aware vision-language modeling for joint detection and grading. |
| Outcome: | The proposed model improves AUC and F1 scores on CND and PUS datasets by 6.31% and 4.79%. |
Out-of-Context Reasoning in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a lightweight technique trains only new token embeddings on axioms and evaluates them on unseen tasks. |
| Approach: | They propose a lightweight technique that trains only new token embeddings on axioms . they train only new embeddables and evaluate them on unseen tasks . |
| Outcome: | The proposed technique trains only new token embeddings on axioms and evaluates them on unseen tasks. |
Step Guided Reasoning: Improving Mathematical Reasoning using Guidance Generation and Step Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to improve mathematical reasoning require extensive datasets for training or depend on few-shot methods that compromise computational accuracy. |
| Approach: | They propose a training-free adaptation framework that efficiently equips general-purpose pre-trained language models with enhanced mathematical reasoning capabilities. |
| Outcome: | The proposed framework outperforms Qwen2.5-72B-Math-Instruct on MMLU-STEM with a score of 90.9%, compared to 87.3%. |
CLaS-Bench: A Cross-Lingual Alignment and Steering Benchmark (2026.findings-acl)
Copied to clipboard
Daniil Gurgurov, Yusser Al Ghussin, Tanja Baeumel, Cheng-Ting Chou, Patrick Schramowski, Marius Mosbach, Josef Van Genabith, Simon Ostermann
| Challenge: | Understanding and controlling behavior of large language models (LLMs) is an important topic in multilingual NLP. |
| Approach: | They propose a lightweight parallel-question benchmark for evaluating language-forcing behavior in large language models across 32 languages. |
| Outcome: | The proposed benchmark measures language steering in 32 languages across 32 languages. |
s3: You Don’t Need That Much Data to Train a Search Agent via RL (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to optimize retrieval using search-only metrics ignore downstream utility and fine-tune entire LLM to jointly reason and retrieve limit retrieval utility and compatibility with frozen or proprietary models. |
| Approach: | They propose a lightweight, model-agnostic framework that decouples the searcher from the generator and trains the search user using a Gain Beyond RAG reward. |
| Outcome: | The proposed framework outperforms baselines trained on over 70 more data with 2.4k training samples. |
Beyond Logits: Aligning Feature Dynamics for Effective Knowledge Distillation (2025.acl-long)
Copied to clipboard
Guoqiang Gong, Jiaxing Wang, Jin Xu, Deping Xiang, Zicheng Zhang, Leqi Shen, Yifeng Zhang, JunhuaShu JunhuaShu, ZhaolongXing ZhaolongXing, Zhen Chen, Pengzhang Liu, Ke Zhang
| Challenge: | Knowledge distillation (KD) compresses large language models into lightweight versions called student models. |
| Approach: | They propose to align the entire feature dynamics between teacher and student models by using two additional loss terms to achieve this. |
| Outcome: | The proposed method matches the entire feature dynamics between teacher and student models rather than just the final states. |
MS-RAG: Simple and Effective Multi-Semantic Retrieval-Augmented Generation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for large language models suffer from poor indexing and inference speed . graph-based RAGs heavily rely on LLM for retrieval thus inference slow . |
| Approach: | They propose retrieval-augmented generation (RAG) which integrates knowledge with dense vectors to build a multi-semantic RAG. |
| Outcome: | The proposed method achieves state-of-the-art performance with faster inference speed compared to existing methods . |
Rule-Guided Extraction: A Hierarchical Rule Optimization Framework for Document-Level Event Argument Extraction (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Document-level event argument extraction (EAE) is a critical task in natural language processing. |
| Approach: | They propose an LLM-driven HiErarchical Rule Optimization framework that iteratively generates and selects optimal hierarchical rules. |
| Outcome: | The proposed framework outperforms few-shot supervised methods and outperformed state-of-the-art prompting baselines. |
Prophecy Distillation for Boosting Abstractive Summarization (2024.lrec-main)
Copied to clipboard
| Challenge: | Abstractive summarization models with maximum likelihood estimation generate unfaithful facts alongside ambiguous focus. |
| Approach: | They propose a framework which learns a regular summarization model to mimic the behavior of being guided by prophecy for boosting abstractive summaries. |
| Outcome: | The proposed model achieves new or matched state-of-the-art on four well-known datasets. |
Representation Bending for Large Language Model Safety (2025.acl-long)
Copied to clipboard
Ashkan Yousefpour, Taeheon Kim, Ryan Sungmo Kwon, Seungbeen Lee, Wonje Jeung, Seungju Han, Alvin Wan, Harrison Ngan, Youngjae Yu, Jonghyun Choi
| Challenge: | Existing safety-enhancing techniques, such as fine-tuning with human feedback or adversarial training, are still vulnerable as they address specific threats and fail to generalize across unseen attacks. |
| Approach: | They propose a new approach that disrupts representations underlying harmful behaviors in Large Language Models by using loss-based fine-tuning. |
| Outcome: | The proposed approach outperforms existing methods such as Circuit Breaker, RMU, and NPO with 95% reduction in attack success rates across diverse jailbreak benchmarks. |
PVTNL: Prompting Vision Transformers with Natural Language for Generalizable Person Re-identification (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Domain generalization person re-identification (DG-ReID) aims to train models on source domains and generalize to unseen target domains. |
| Approach: | They propose a framework to generalize person re-identification using a vision-language model . body-part cues are used to segment images into semantically coherent regions . |
| Outcome: | The proposed framework can generalize to unseen domains and generalize semantics to people . it leverages the pre-trained vision-language model BLIP to extract aligned visual and textual embeddings. |
Mixture-of-Experts with Intermediate CTC Supervision for Accented Speech Recognition (2026.acl-long)
Copied to clipboard
| Challenge: | Accented speech remains a persistent challenge for automatic speech recognition (ASR) Accent-agnostic approaches improve robustness but struggle with heavily accented or unseen varieties . |
| Approach: | They propose a Mixture-of-Experts architecture with intermediate CTC supervision that promotes expert specialization and generalization. |
| Outcome: | Experiments show that the proposed architecture improves on accented speech . the proposed framework is based on a mixture-of-experts architecture with intermediate supervision . |
From Isolated Scoring to Collaborative Ranking: A Comparison-Native Framework for LLM-Based Paper Evaluation (2026.findings-acl)
Copied to clipboard
Pujun Zheng, Jiacheng Yao, Jinquan Zheng, Chenyang Gu, Guoxiu He, Jiawei Liu, Yong Huang, Tianrui Guo, Wei Lu
| Challenge: | Large language models (LLMs) are currently used to evaluate scientific papers by assigning an absolute score to each paper independently. |
| Approach: | They propose a comparison-native framework for paper evaluation that integrates comparison into both data construction and model learning. |
| Outcome: | The proposed framework achieves an average relative improvement of 21.8% over the strong baseline DeepReview-14B, while exhibiting robust generalization to five previously unseen datasets. |
Sprout: Green Generative AI with Carbon-Efficient LLM Inference (2024.emnlp-main)
Copied to clipboard
| Challenge: | Sprout reduces the carbon footprint of inference in large language models by over 40% in real-world evaluations. |
| Approach: | Sprout introduces "generation directives" to guide autoregressive generation process . et al. cites Llama model and global electricity grid data as examples . |
| Outcome: | Sprout reduces the carbon footprint of generative AI models by over 40% in real-world evaluations using the Llama model and global electricity grid data. |
RevMUX: Data Multiplexing with Reversible Adapters for Efficient LLM Batch Inference (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have brought a great breakthrough to the natural language processing community, but their high throughput demands make them difficult to handle concurrent queries. |
| Approach: | They propose a parameter-efficient data multiplexing framework that integrates a reversible design in the multiplexer and can be reused to perform reverse operations and restore individual samples for classification. |
| Outcome: | The proposed framework improves inference efficiency while maintaining satisfactory classification performance. |
LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMs (2025.findings-acl)
Copied to clipboard
Omkar Thawakar, Dinura Dissanayake, Ketan Pravin More, Ritesh Thawkar, Ahmed Heakl, Noor Ahsan, Yuhao Li, Ilmuz Zaman Mohammed Zumri, Jean Lahoud, Rao Muhammad Anwer, Hisham Cholakkal, Ivan Laptev, Mubarak Shah, Fahad Shahbaz Khan, Salman Khan
| Challenge: | Existing approaches do not emphasize step-wise problem-solving. |
| Approach: | They propose a visual reasoning chain benchmark and a fine-grained reasoning metric that evaluates correctness and logical coherence at each step. |
| Outcome: | The proposed framework outperforms existing models in six benchmarks and is 5x faster during inference scaling. |
TituLLMs: A Family of Bangla LLMs with Comprehensive Benchmarking (2025.findings-acl)
Copied to clipboard
Shahriar Kabir Nahin, Rabindra Nath Nandi, Sagor Sarker, Quazi Sarwar Muhtaseem, Md Kowsher, Apu Chandraw Shill, Md Ibrahim, Mehadi Hasan Menon, Tareq Al Muntasir, Firoj Alam
| Challenge: | Existing benchmarking datasets for Bangla LLMs are not available for all languages. |
| Approach: | They present TituLLMs, the first large pretrained Bangla LLMs, available in 1b and 3b parameter sizes. |
| Outcome: | The proposed model outperforms existing models in Bangla, but not always in the first place. |
Rethinking the Role of Entropy in Optimizing Tool-Use Behaviors for Large Language Model Agents (2026.acl-long)
Copied to clipboard
Zeping Li, Hongru Wang, Yiwen Zhao, Guanhua Chen, Yixia Li, Keyang Chen, Yixin Cao, Guangnan Ye, Hongfeng Chai, Zhenfei Yin
| Challenge: | Large language models excel in mathematical reasoning and multi-hop question answering tasks, but in long trajectories, agents often invoke tools excessively or inappropriately, increasing computation cost and derailing the reasoning process. |
| Approach: | They propose to use entropy reduction as a supervisory signal to reduce tool calls . they propose to design two reward strategies to address the needs of optimizing tool-use behavior. |
| Outcome: | The proposed reward strategies reduce tool calls by 72.07% and improve performance by 22.27%. |
LLM-MC-Affect: LLM-Based Monte Carlo Modeling of Affective Trajectories and Latent Ambiguity for Interpersonal Dynamic Insight (2026.acl-long)
Copied to clipboard
Yu-Zheng Lin, Bono Po-Jen Shih, John Paul Martin Encinas, Elizabeth Victoria Achom, Karan Patel, Jesus Horacio Pacheco, Sicong Shao, Jyotikrishna Dass, Soheil Salehi, Pratik Satam
| Challenge: | Emotional coordination is a core property of human interaction that shapes relational meaning . prior approaches treat sentiment as a deterministic point estimate for individual speakers . scalable and deployable approach extends beyond education to broader social and behavioral research . |
| Approach: | They propose a probabilistic framework that characterizes emotion as a latent probability distribution defined over an affective space. |
| Outcome: | The proposed framework characterizes emotion as a latent probability distribution defined over affective space. |
Enhancing RAG Efficiency with Adaptive Context Compression (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods apply fixed compression rates, over-compressing simple queries or under-compressed complex ones. |
| Approach: | a new framework uses a hierarchical compressor and a context selector to optimize inference efficiency . a framework that dynamically adjusts compression rates based on input complexity optimizes inference without loss of accuracy. |
| Outcome: | Adaptive Context Compression for RAG outperforms fixed-rate methods on Wikipedia and five QA datasets . |
HELENE: Hessian Layer-wise Clipping and Gradient Annealing for Accelerating Fine-tuning LLM with Zeroth-order Optimization (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) face memory challenges due to the high cost of backpropagation. |
| Approach: | They propose a zeroth-order (ZO) optimization that matches memory usage to inference . they propose scalable and memory-efficient zeroth order (ZE) optimizer that integrates annealed A-GNB gradients with diagonal Hessian estimation and layer-wise clipping as a second-order pre-conditioner. |
| Outcome: | The proposed algorithm outperforms state-of-the-art methods with an average speedup of 20 over MeZO on RoBERTa-large and OPT-1.3B. |
Modal Dependency Parsing as Structured Prediction over Source-Cue Scope (2026.acl-long)
Copied to clipboard
| Challenge: | Existing work on identifying sources only focuses on defining source-introducing cues . a structured model focuses learning at the source-cue level and constrains event-level decisions to a small, scope-defined candidate set. |
| Approach: | They propose a framework that leverages large language models to explicitly identify source-cue pairs and their respective scope to define modal contexts. |
| Outcome: | The proposed framework surpasses state-of-the-art results by 3 and 4% for English and Chinese datasets. |
Learning from Mistakes: Negative Reasoning Samples Enhance Out-of-Domain Generalization (2026.acl-long)
Copied to clipboard
Tian Xueyun, MingHua Ma, Bingbing Xu, Nuoyan Lyu, Wei Li, Heng Dong, Zheng Chu, Yuanzhuo Wang, Huawei Shen
| Challenge: | Recent studies show that supervised fine-tuning (SFT) is a common approach for reasoning in large language models. |
| Approach: | They propose to use supervised fine-tuning (SFT) on chain-of-thought trajectories demonstrations . they find that incorporating negative traxories yields substantial OOD generalization gains . |
| Outcome: | The proposed scheme yields 5.51% OOD gain over positive-only training. |
Diversity Explains Inference Scaling Laws: Through a Case Study of Minimum Bayes Risk Decoding (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies show that MBR decoding improves model generation performance . however, the theoretical underpinnings of these results remain uncertain . |
| Approach: | They propose a theoretical interpretation of MBR decoding from the perspective of bias–diversity decomposition. |
| Outcome: | The proposed method improves the quality estimation of hypotheses by decomposing bias and diversity into two main factors. |
Reducing Peak Memory Usage for Modern Multimodal Large Language Model Pipelines (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to reduce memory usage of multimodal large language models rely on storing large numbers of vision tokens in the key–value cache . however, such compression is typically only applied after all inputs are processed, resulting in high peak memory usage during the prefill stage. |
| Approach: | They propose a sequential input-compression mechanism that enforces a fixed memory budget by performing structure-aware key–value cache compression during the prefill stage. |
| Outcome: | The proposed approach reduces peak memory usage while maintaining generative performance with only minimal degradation, enabling more practical and memory-efficient multimodal inference. |
ERRV: Eliciting Efficient Reasoning through Reasoning Vectors for Policy Optimization in Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing efforts to improve reasoning efficiency of large language models focus on modifying the reinforcement learning reward, such as adding length penalties. |
| Approach: | They propose a training framework that elicits efficient reasoning through reasoning vectors and a framework that allows the model to generate high-quality responses during reinforcement learning. |
| Outcome: | The proposed framework reduces reasoning length by 30% while maintaining stability, while retaining high accuracy. |
Video-RTS: Rethinking Reinforcement Learning and Test-Time Scaling for Efficient and Enhanced Video Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Despite advances in reinforcement learning, data collection and fine-tuning remain costly and hard to scale. |
| Approach: | They propose a video-adaptive test-time scaling strategy that combines RL with a supervised fine-tuning strategy to improve video reasoning capability. |
| Outcome: | The proposed method surpasses existing models by 2.4% in accuracy using only 3.6% training samples. |
CARO: Chain-of-Analogy Reasoning Optimization for Robust Content Moderation (2026.findings-acl)
Copied to clipboard
| Challenge: | Current large language models struggle with ambiguous content moderation cases due to misleading "decision shortcuts" . authors propose a two-stage training framework to induce robust analogical reasoning in LLMs . |
| Approach: | They propose a two-stage training framework to induce robust analogical reasoning in LLMs . they bootstrap analogy reasoning chains via retrieval-augmented generation and SFT . |
| Outcome: | The proposed framework outperforms state-of-the-art reasoning models and specialized moderation models on ambiguous moderation benchmarks. |
COUNTDOWN: Contextually Sparse Activation Filtering Out Unnecessary Weights in Down Projection (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models have created significant computational inefficiencies due to their size and complexity. |
| Approach: | They propose to use a linear combination to deactivate non-essential parameters during inference to reduce computational costs. |
| Outcome: | The proposed methods can omit 90% of computations with performance loss as low as 5.5% ideally, while M-COUNTDOWN provides a predictor-free solution with up to 29.4% better performance preservation compared to existing methods. |
TestNUC: Enhancing Test-Time Computing Approaches and Scaling through Neighboring Unlabeled Data Consistency (2025.acl-long)
Copied to clipboard
Henry Peng Zou, Zhengyao Gu, Yue Zhou, Yankai Chen, Weizhi Zhang, Liancheng Fang, Yibo Wang, Yangning Li, Kay Liu, Philip S. Yu
| Challenge: | Test-time computing approaches that leverage additional computational resources during inference have been proven effective in enhancing large language model performance. |
| Approach: | They propose a linearly scaling approach that leverages local consistency of neighboring unlabeled data to improve test-time predictions. |
| Outcome: | The proposed approach outperforms baseline methods such as prompting and self-consistency across eight datasets and performs robustly across embedding models. |
TACLR: A Scalable and Efficient Retrieval-based Method for Industrial Product Attribute Value Identification (2025.acl-long)
Copied to clipboard
Yindu Su, Huike Zou, Lin Sun, Ting Zhang, Haiyang Yang, Chen Li Yu, David Lo, Qingheng Zhang, Shuguang Han, Jufeng Chen
| Challenge: | Existing methods for product attribute value identification face critical challenges . seller-provided attribute values are often incomplete or inaccurate . |
| Approach: | They propose a retrieval-based method that uses taxonomy-aware contrastive learning . they use product profiles and candidate values to encode and retrieve attributes based on similarity . |
| Outcome: | The proposed method is based on a taxonomy-aware, hard negative sampling and adaptive inference with dynamic thresholds. |
Towards Self-Evolving Agents: Enabling Autonomy through Interactive Experience Refinement (2026.findings-acl)
Copied to clipboard
Cheng Yang, Xuemeng Yang, Licheng Wen, Daocheng Fu, Jianbiao Mei, Rong Wu, Pinlong Cai, Yufan Shen, Nianchen Deng, Jia Xu, Botian Shi, Yu Qiao, Haifeng Li
| Challenge: | Large Language Models struggle with complex, multi-step operational tasks because they remain static during inference and cannot learn from past experience. |
| Approach: | They propose a framework that organizes cross-domain insights to facilitate orchestration of long-horizon workflows. |
| Outcome: | The proposed framework outperforms existing methods on the TAC productivity benchmark and shows strong cross-task transferability. |
Split-Merge: Scalable and Memory-Efficient Merging of Expert LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | a zero-shot merging framework for large language models consolidates specialized domain experts into a single model without any further training. |
| Approach: | They propose a zero-shot merging framework that consolidates specialized domain experts into a single model without further training. |
| Outcome: | Experiments on code generation, mathematical reasoning, medical question answering, and instruction-following benchmarks confirm the versatility and effectiveness of the proposed framework. |
Generalizable Prompt Tuning for Audio-Language Models via Semantic Expansion (2026.findings-acl)
Copied to clipboard
| Challenge: | Prompt tuning has achieved remarkable progress in vision–language models, but its generalization ability in ALMs remains underexplored. |
| Approach: | They propose a plug-and-play framework that regularizes the prompt embedding space . they propose introducing a semantic expansion loss with margin constraints that promote compactness . |
| Outcome: | The proposed framework regularizes the prompt embedding space by incorporating semantic neighbors generated by large language models. |
Leveraging Label Semantics and Entity Description Generation for LLM-based Fine-grained Entity Typing (2026.findings-acl)
Copied to clipboard
| Challenge: | Fine-grained entity typing (FET) aims to assign semantically rich and contextually appropriate types to entity mentions. |
| Approach: | They propose a descriptor-based retrieval-augmented framework that reduces effective label space . they propose to use natural language descriptores as an intermediate semantic representation . |
| Outcome: | The proposed framework outperforms existing methods under noisy supervision. |
RAG+: Enhancing Retrieval-Augmented Generation with Application-Aware Reasoning (2025.emnlp-main)
Copied to clipboard
Yu Wang, Shiwan Zhao, Zhihu Wang, Ming Fan, Xicheng Zhang, Yubo Zhang, Zhengfan Wang, Heyuan Huang, Ting Liu
| Challenge: | Existing RAG paradigms often overlook the cognitive step of applying knowledge, leaving a gap between retrieved facts and task-specific reasoning. |
| Approach: | They introduce a module extension that integrates application-aware reasoning into the RAG pipeline. |
| Outcome: | Experiments show that RAG+ outperforms standard RAG variants and achieves gains of 3–5% in complex scenarios. |
River-LLM: Large Language Model Seamless Exit Based on KV Share (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods to reduce latency and speed up early exits are costly and impose significant cost and energy consumption. |
| Approach: | They propose a lightweight KV-Shared Exit River framework that allows the backbone’s missing KV cache to be naturally generated and preserved during the exit process. |
| Outcome: | The proposed framework achieves 1.71 to 2.16 speedup while maintaining high generation quality. |
CAMEC: Complexity-Aware Multi-Expert Collaboration for Reliable Chinese Medical Question Answering (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models are promising for medical question answering in china, but remain unreliable due to hallucinations, weak factual grounding and difficulty handling clinically complex cases. |
| Approach: | They propose a framework that combines hierarchical medical adaptation with complexity-aware expert routing for reliable Chinese medical QA. |
| Outcome: | The proposed framework outperforms strong general and medical LLM baselines on four Chinese medical benchmarks. |
PDTrim: Targeted Pruning for Prefill-Decode Disaggregation in Inference (2026.acl-long)
Copied to clipboard
| Challenge: | Existing pruning methods ignore prefill-decode (PD) disaggregation in practice. |
| Approach: | They propose a pruning method that is highly integrated with prefill-decode (PD) disaggregation, enabling more precise pruning of blocks. |
| Outcome: | The proposed method achieves strong performance in both PD disaggregation and PD unified settings, and can be extended to other non-block pruning methods. |
Too Long, Do Re-weighting for Efficient LLM Reasoning Compression (2026.acl-long)
Copied to clipboard
Zhong-Zhi Li, Xiao Liang, Zihao Tang, Lei Ji, Peijie Wang, Haotian Xu, Xing W, Haizhen Huang, Weiwei Deng, Yeyun Gong, Zhijiang Guo, Xiao Liu, Fei Yin, Cheng-Lin Liu
| Challenge: | Large Language Models (LLMs) have recently achieved remarkable progress on complex reasoning tasks by leveraging extended Chain-of-Thought (CoT) techniques. |
| Approach: | They propose a method that uses Extended Chain-of-Thought (EFT) to reduce the number of output tokens by nearly 40% while maintaining the accuracy of the reasoning. |
| Outcome: | The proposed method reduces the number of output tokens by nearly 40% while maintaining the accuracy of the reasoning. |
Masked Text-to-Audio Flow-Matching and Reward Feedback Optimization (2026.findings-acl)
Copied to clipboard
Rongjie Huang, Dongchao Yang, Wenxiang Guo, Huadai Liu, Xize Cheng, Zehan Wang, Zhou Zhao, Xixin Wu, Helen M. Meng
| Challenge: | Experimental results show that Flow-matching generative models can scale training by increasing data, computational resources, and model size. |
| Approach: | They propose a flow-matching transformer with masked generative modeling for scaling text-to-audio inference-time prediction. |
| Outcome: | The proposed model scales inference-time computations by masking generation and re-predicting them through iterative decoding. |
AdapShot: Adaptive Many-Shot In-Context Learning with Semantic-Aware KV Cache Reuse (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for In-Context Learning (ICL) rely on a predetermined number of shots, leading to insufficient context or noise. |
| Approach: | They propose a probe-based evaluation mechanism that utilizes output entropy to determine the optimal number of shots and leverages KV cache reuse for efficient inference. |
| Outcome: | The proposed model achieves an average performance gain of 10% and a 4.64 speedup compared to state-of-the-art DBSA. |
Sparse-RL: Breaking the Memory Wall in LLM Reinforcement Learning via Stable Sparse Rollouts (2026.acl-long)
Copied to clipboard
Sijia Luo, Xiaokang Zhang, Yuxuan Hu, Bohan Zhang, Ke Wang, Jinbo Su, Mengshu Sun, Lei Liang, Jing Zhang
| Challenge: | Existing methods for storing key-value caches during long-horizon rollouts cause performance collapses. |
| Approach: | They propose a new training paradigm that empowers stable RL training under sparse rollouts. |
| Outcome: | The proposed model reduces rollout overhead while maintaining the performance. |
Grounded Concreteness: Human-Like Concreteness Sensitivity in Vision–Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | a long tradition in cognitive science treats concreteness as a graded dimension of conceptual representation . concrete words benefit from richer sensory codes and exhibit robust behavioral advantages over abstract words . |
| Approach: | They compare vision-language models with text-only large language models to test their concreteness . they find that VLMs show more human-like sensitivity to concreteness than LLMs . |
| Outcome: | The proposed model-based training improves on the Llama text backbones and Llma Vision counterparts. |
GRAD: Generalizing RAG Adaptation with Decoding (2026.acl-long)
Copied to clipboard
| Challenge: | Using GRAD, we can steer Retrieval-augmented generation objectives without retraining large language models. |
| Approach: | They propose an adaptive decoding-time framework that keeps the base generator fixed and composes small, objective-specific guidance at inference. |
| Outcome: | The proposed framework improves accuracy with favorable latency across public benchmarks and private settings with no in-domain labels while reliably activating helpful objectives and suppressing harmful ones, adaptively to tasks. |
Self-Guided Alignment: Adaptive Preference Sensing for Multi-Objective Generation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to align LLMs with diverse human values rely on ground-truth scores . existing approaches implicitly approximate an average-user preference, thereby failing to capture heterogeneity of human values or accommodate conflicting user needs. |
| Approach: | They propose a framework that transforms passive reward dependency into an intrinsic adaptive sensing capability. |
| Outcome: | The proposed framework outperforms state-of-the-art models in multiple model scales and improves preference alignment. |
SELECting over Tokens: Curating Pre-training Data at Scale via Token Classification (2026.acl-long)
Copied to clipboard
Xin Tong, Weidong Zhang, Jiaang Li, Haibin Chen, Shilei Liu, Langming Liu, Kangtao Lv, Yujin Yuan, Wenbo Su, Bo Zheng
| Challenge: | Existing pipelines rely on expert-crafted heuristic rules, which lack content-aware, fine-grained noise detection. |
| Approach: | They propose a framework that reframes data refinement as a highly efficient token classification task. |
| Outcome: | The proposed framework outperforms existing pipelines on benchmarks and is 2.5x faster at inference. |