Papers with decoding
Computational Linguistics for Brain Encoding and Decoding: Principles, Practices and Beyond (2024.acl-tutorials)
Copied to clipboard
| Challenge: | This tutorial will explore the potential of computational linguistics to help understand brain language processing. |
| Approach: | This tutorial will explore the principles and practices of using computational linguistics methods for brain encoding and decoding. |
| Outcome: | This tutorial will explore the principles and practices of using computational linguistics methods for brain encoding and decoding. |
Transformer-Based Direct Hidden Markov Model for Machine Translation (2021.acl-srw)
Copied to clipboard
| Challenge: | Recent studies have found that word alignments produced by the multi-head cross-attention weights are poor. |
| Approach: | They propose to introduce the hidden Markov model to the transformer architecture and introduce alignment components while keeping the system monolithic. |
| Outcome: | The proposed model outperforms the baseline model but is slower in training and decoding. |
A Call for Clarity in Beam Search: How It Works and When It Stops (2024.lrec-main)
Copied to clipboard
| Challenge: | Empirical results show that a modified beam decoding implementation improves decoding performance of strong, neural language generation models. |
| Approach: | They propose a modification to a beam decoding implementation that generalizes the stopping criterion and provides flexibility to the depth of search. |
| Outcome: | The proposed method improves decoding performance of strong models on news text summarization and machine translation over diverse language pairs with negligible inference slowdown. |
Probing Word Translations in the Transformer and Trading Decoder for Encoder Layers (2021.naacl-main)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) has attracted wide attention in recent years. |
| Approach: | They propose a probing-based approach to measure word translation accuracy using transformer layers. |
| Outcome: | The proposed model outperforms previous probing-based translation models. |
Subset Retrieval Nearest Neighbor Machine Translation (2023.acl-long)
Copied to clipboard
| Challenge: | k-nearest-neighbor machine translation (kNN-MT) is a new approach to improve NMT performance without additional training. |
| Approach: | They propose a method that integrates example-search into the decoding algorithm to improve neighbor token retrieval. |
| Outcome: | The proposed method achieves a speed-up of up to 132.2 times and an improvement in BLEU score of up 1.6 compared with kNN-MT in the WMT’19 translation task and the domain adaptation tasks in De-En and En-Ja. |
Thesis Proposal: Efficient KV Cache Reuse for Multi-Document Retrieval-Augmented Generation (2026.eacl-srw)
Copied to clipboard
| Challenge: | Retrieval-Augmented Generation (RAG) systems face efficiency bottlenecks in prefill due to attention mechanism, and traditional KV cache only accelerates decoding. |
| Approach: | They propose a multi-document KV cache reuse framework for multi-doc RAG workloads . they propose to resolve position and context misalignment while eliminating document-specific quadratic complexity in prefill. |
| Outcome: | The proposed framework solves position and context misalignment issues while eliminating document-specific quadratic complexity in prefill. |
Multi-Prompting Decoder Helps Better Language Understanding (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to adapt Pre-trained Language Models to downstream tasks are limited by their inference APIs. |
| Approach: | They propose a multi-prompting decoding framework that query PLMs with multiple prompts . they propose to query Plms with optimal transport for hidden states and calibrated decoding for class scores . |
| Outcome: | The proposed framework achieves state-of-the-art results on multiple natural language understanding datasets under the few-shot setting. |
How Important is a Language Model for Low-resource ASR? (2024.findings-acl)
Copied to clipboard
| Challenge: | Using an n-gram language model in ASR may seem obvious, but its absence in most implementations suggests otherwise. |
| Approach: | They examine whether using an n-gram language model in ASR can improve accuracy in low-resource languages. |
| Outcome: | The proposed model is absent in most implementations, but it does improve accuracy in English and Mandarin. |
Token Level Routing Inference System for Edge Devices (2025.acl-demo)
Copied to clipboard
| Challenge: | Large language models (LLMs) have been gaining in performance but deployment in edge devices faces significant hurdles due to their high computational complexity. |
| Approach: | They propose a collaborative decoding system that allows small models to perform on-device inference while selectively consulting a cloud-based large model for critical token generation. |
| Outcome: | The proposed system achieves 60% performance gain on CommonsenseQA using a 0.5B model on an M1 MacBook, with under 7% of tokens generation uploaded to the large model in the cloud. |
Long-Form Speech Translation through Segmentation with Finite-State Decoding Constraints on Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a challenge in speech translation is that plenty of spoken content is long-form, but short units are necessary for obtaining high-quality translations. |
| Approach: | They propose a large language model to split long ASR transcripts into segments that can be independently translated to maximize translation quality. |
| Outcome: | The proposed model improves the average BLEU by 2.9 points for English–German, English–Spanish, and English–Arabic TED talk translation in 9 sets. |
Transformer and seq2seq model for Paraphrase Generation (D19-56)
Copied to clipboard
| Challenge: | Existing methods for generating paraphrases fall into one of these broad categories -rule-based, seq2seq, deep generative models and a varied combination. |
| Approach: | They propose a framework that combines transformer and sequence-to-sequence models for better quality of generated paraphrases. |
| Outcome: | The proposed framework improves on two datasets-QUORA and MSCOCO using transformer and sequence-to-sequence models. |
Mixup Decoding for Diverse Machine Translation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for generating multiple translations for source and target languages neglect the one-to-many mapping between the source and the target languages. |
| Approach: | They propose a method to generate different translations for the input sentence by linearly interpolating it with different sentence pairs sampled from the training corpus during decoding. |
| Outcome: | Experiments on WMT’16 en-ro, WMT'14 en de, and WMT ‘17 zh-en show that the proposed method outperforms all previous diverse machine translation methods. |
I already said that! Degenerating redundant questions in open-domain dialogue systems. (2023.acl-srw)
Copied to clipboard
| Challenge: | Neural text generation models have been successful in short open-domain conversations, but their performance degrades significantly in the long term. |
| Approach: | They propose a method to generate training data without crowdsourcing . they adapt negative training, decoding, and classification methods to mitigate redundancy problem . |
| Outcome: | The proposed method reduces the rate of redundant questions from 27.2% to 8.7% while improving the quality of the original model. |
ReEx-SQL: Reasoning with Execution-Aware Reinforcement Learning for Text-to-SQL (2026.acl-long)
Copied to clipboard
Yaxun Dai, Wenxuan Xie, Xialie Zhuang, Tianyu Yang, Ziyi Liu, Haiqin Yang, Yiying Yang, Yuhang Zhao, Pingfu Chao, Wenhao Jiang
| Challenge: | Current Text-to-SQL reasoning models lack integrated execution feedback during generation. |
| Approach: | They propose a text-to-SQL framework that interacts with the SQL execution engine during decoding and dynamically adjusts reasoning based on execution feedback. |
| Outcome: | The proposed framework achieves 89.1% accuracy on Spider and 65.3% on BIRD at the 7B scale. |
From Short Video to Clickable Search: RLVR-Enabled Listwise Query Suggestion with Retrieval-Augmented Context (2026.acl-industry)
Copied to clipboard
| Challenge: | Existing methods to generate short-video bottom-bar queries are largely retrieval-based. |
| Approach: | They propose to reformulate the task as one-shot list generation, producing multiple queries per video . they also build multi-query ground truth from exposure and CTR logs, and redesign offline evaluation . |
| Outcome: | The proposed system yields strong offline and online improvements . it is deployed on Kuaishou to serve hundreds of millions of users daily . |
Predicting and Using Target Length in Neural Machine Translation (2020.aacl-main)
Copied to clipboard
| Challenge: | Current NMT systems do not model the length of the output explicitly . length normalization is a common technique used in the beam search of NMT to enable a fair comparison of partial hypotheses with different lengths. |
| Approach: | They propose to use length prediction as an auxiliary task to obtain length information from the encoder. |
| Outcome: | The proposed sub-network improves over the baseline system and the predicted length can be used as an alternative to length normalization during decoding. |
Learning to Translate Ambiguous Terminology by Preference Optimization on Post-Edits (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Ambiguous terminology can make translation difficult, especially in corporate contexts. |
| Approach: | They propose to learn how to disambiguate terminology based on human post-edits . they use preference optimization to optimize for correctness using the term post-Edit . |
| Outcome: | The proposed framework improves term accuracy over a translation oriented LLM without significant losses in COMET score. |
Code-Switching for Enhancing NMT with Pre-Specified Translation (N19-1)
Copied to clipboard
| Challenge: | Existing methods to constrain NMT use placeholder tags for lexicon words and hard constraints during decoding. |
| Approach: | They propose to use placeholder tags to replace lexicon words with target translations . they use a data augmentation method to make code-switched training data . |
| Outcome: | The proposed method improves translation quality without hurting unconstrained words. |
Sparse-to-Dense: A Free Lunch for Lossless Acceleration of Video Understanding in LLMs (2025.acl-short)
Copied to clipboard
| Challenge: | Recent advances in Video Large Language Models (Video-LLMs) have achieved exceptional performance on tasks like video question answering and captioning. |
| Approach: | They propose a decoding strategy that leverages sparse top-K attention and dense full attention to accelerate Video-LLMs without loss. |
| Outcome: | The proposed approach achieves a 1.94 walltime speedup in video processing. |
Generating Repetitions with Appropriate Repeated Words (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing studies have focused on general response generation with neural network-based approaches, but none have addressed specific types of repetitions. |
| Approach: | They propose a weighted label smoothing method for explicitly learning which words to repeat during fine-tuning and a repetition scoring method that can output more appropriate repetitions during decoding. |
| Outcome: | The proposed method outperforms baselines in automatic and human evaluations on a pre-trained language model for generating repetitions. |
PrefixNLI: Detecting Factual Inconsistencies as Soon as They Arise (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are prone to factually inconsistent statements, known as hallucinations. |
| Approach: | They propose to train a specialized model that detects inconsistencies over text prefixes to improve generation faithfulness by 5-14 F1 points. |
| Outcome: | The proposed model outperforms baseline models by 5-14 F1 points in prefix-level entailment. |
NAG-NER: a Unified Non-Autoregressive Generation Framework for Various NER Tasks (2023.acl-industry)
Copied to clipboard
| Challenge: | Existing models for general NER tasks require entities to be generated in a predefined order, causing error propagation and inefficient decoding. |
| Approach: | They propose a non-autoregressive generation framework for general NER tasks that generates entities as a set instead of a sequence, avoiding error propagation and inefficient decoding. |
| Outcome: | The proposed model outperforms state-of-the-art models on three benchmark NER datasets and two of our proprietary NER tasks. |
Look-back Decoding for Open-Ended Text Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to decode open-ended text have addressed degeneration problems in large-scale language models (LLMs) |
| Approach: | They propose an improved decoding algorithm that leverages the Kullback–Leibler divergence to track the distribution distance between current and historical decoding steps. |
| Outcome: | The proposed algorithm outperforms existing methods in document continuation and story generation. |
Learning Hard Retrieval Decoder Attention for Transformers (2021.findings-emnlp)
Copied to clipboard
| Challenge: | In this paper, we show that learning a hard retrieval attention that attends to a single token in a sentence is 1.43 times faster than the standard scaled dot-product attention. |
| Approach: | They propose a method to learn hard retrieval attention where an attention head attends to a single token in a sentence rather than all tokens. |
| Outcome: | The proposed method is 1.43 times faster in decoding while preserving translation quality on a wide range of MT tasks. |
A Syntactically Constrained Bidirectional-Asynchronous Approach for Emotional Conversation Generation (D18-1)
Copied to clipboard
| Challenge: | Existing neural language models generate generic responses with poor logic and no emotion. |
| Approach: | They propose a syntactically constrained bidirectional-asynchronous approach for emotional conversation generation using pre-generated emotion keywords and topic keywords. |
| Outcome: | The proposed approach improves the diversity of responses and boosts logic and emotion compared with baselines. |
ZoomR: Memory Efficient Reasoning through Multi-Granularity Key Value Retrieval (2026.acl-long)
Copied to clipboard
David H. Yang, Yuxuan Zhu, Mohammad Mohammadi Amiri, Keerthiram Murugesan, Tejaswini Pedapati, Subhajit Chaudhury, Pin-Yu Chen
| Challenge: | Large language models generate long chain of thoughts but memory footprint grows with output length . prior work on KV cache optimization focused on compressing long input context . |
| Approach: | They propose a new approach that compresses verbose reasoning thoughts into summaries . they use a dynamic KV cache selection policy that leverages these summary keys . |
| Outcome: | The proposed approach reduces memory usage while avoiding full-cache attention at each step. |
Lexically Aware Semi-Supervised Learning for OCR Post-Correction (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing methods for digitizing text in endangered languages rely on manual data curated by the user. |
| Approach: | They propose a semi-supervised learning method that utilizes raw images to improve performance. |
| Outcome: | The proposed method reduces errors by 15%–29% on four endangered languages. |
Characterizing and addressing the issue of oversmoothing in neural autoregressive sequence modeling (2022.aacl-main)
Copied to clipboard
| Challenge: | Neural autoregressive sequence models assign high probability to unreasonably short sequences . authors propose to minimize oversmoothing rate during training . |
| Approach: | They propose to minimize the oversmoothing rate during training by tuning the regularization strength. |
| Outcome: | The proposed regularization can control the oversmoothing rate and improve decoding performance. |
Dynamic Past and Future for Neural Machine Translation (D19-1)
Copied to clipboard
| Challenge: | Neural machine translation models can benefit from modeling translated and untranslated source contents as recurrent states, but this less interpretable recurrence hinders their power to model dynamic updating of and contents during decoding. |
| Approach: | They propose to model the dynamic updating of and contents during decoding by explicitly separating source words into groups of translated and untranslated contents through parts-to-wholes assignment. |
| Outcome: | The proposed method achieves significant improvements over both Rnmt and Transformer by producing more adequate translations. |
Joint Transformer/RNN Architecture for Gesture Typing in Indic Languages (2020.coling-main)
Copied to clipboard
| Challenge: | Gesture typing is a method of typing words on a touch-based keyboard by drawing a continuous trace passing through the relevant keys. |
| Approach: | They propose a keyboard that supports gesture typing in Indic languages by drawing a continuous trace over the keyboard and the finger needs to be lifted only once a word is completed. |
| Outcome: | The proposed model performs path decoding, transliteration and transliterations correction. |
Neural Search Space in Gboard Decoder (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Gboard decoder uses context, a lexicon and language models to provide a user-friendly keyboard. |
| Approach: | They propose a Neural Search Space which replaces an N-gram LM with a neural network LM and dynamically constructs the search space during decoding. |
| Outcome: | The proposed system improves the quality of the decoded keyboards on various locales with acceptable latency increases. |
Gaussian Process Optimization for Adaptable Multi-Objective Text Generation using Linearly-Weighted Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Multi-objective text generation requires a method to optimize for dynamic requirements of the overall objective. |
| Approach: | They propose a linear combination of objective-specific language models to efficiently adapt the decoding process and optimize for the desired overall objective without retraining one or more language models. |
| Outcome: | The proposed method outperforms other weighting schemes and standard baselines in a few iterations of decoding. |
Second Language (Arabic) Acquisition of LLMs via Progressive Vocabulary Expansion (2025.acl-long)
Copied to clipboard
Jianqing Zhu, Huang Huang, Zhihang Lin, Juhao Liang, Zhengyang Tang, Khalid Almubarak, Mosen Alharthi, Bang An, Juncai He, Xiangbo Wu, Fei Yu, Junying Chen, Ma Zhuoheng, Yuhao Du, He Zhang, Saied Alshahrani, Emad A. Alghamdi, Lian Zhang, Ruoyu Sun, Haizhou Li, Benyou Wang, Jinchao Xu
| Challenge: | In the evolving landscape of large language models, the predominant focus has been on English and Chinese. |
| Approach: | They propose to utilize Arabic-specific vocabulary in the tokenizer to accelerate decoding. |
| Outcome: | The proposed model achieves decent performance comparable to the best Arabic LLMs across various Arabic benchmarks. |
TaDA: Training-free recipe for Decoding with Adaptive KV Cache Compression and Mean-centering (2025.acl-industry)
Copied to clipboard
| Challenge: | key-value caches in large language models consume memory, posing a major challenge for scalable deployment. |
| Approach: | They propose a training-free recipe for KV cache compression with quantization precision that adapts to error sensitivity across layers and a mean centering to eliminate separate outlier handling. |
| Outcome: | The proposed technique reduces the KV cache memory footprint to 27% of the original 16-bit baseline while achieving comparable accuracy. |
On Finding the K-best Non-projective Dependency Trees (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work on finding the one-best dependency tree has not extended this to finding the K-best tree. |
| Approach: | They propose to simplify the K-best spanning tree algorithm by decoding the K best dependency trees with a root constraint. |
| Outcome: | The proposed algorithm can be used to find the K-best dependency trees without root constraint. |
Language Model Decoding as Likelihood–Utility Alignment (2023.findings-eacl)
Copied to clipboard
Martin Josifoski, Maxime Peyrard, Frano Rajič, Jiheng Wei, Debjit Paul, Valentin Hartmann, Barun Patra, Vishrav Chaudhary, Emre Kiciman, Boi Faltings
| Challenge: | Existing studies only compare decoding algorithms in narrow scenarios, and their findings do not generalize across tasks. |
| Approach: | They propose a taxonomy of misalignment mitigation strategies to provide a unifying view of decoding as a tool for alignment. |
| Outcome: | The proposed taxonomy combines likelihood and utility assumptions to provide general statements about decoding as a tool for alignment across tasks. |
Refining Source Representations with Relation Networks for Neural Machine Translation (C18-1)
Copied to clipboard
| Challenge: | Existing neural machine translation frameworks that forget distant information and disregard relationship between source and target words are not effective. |
| Approach: | They propose to use relation networks to learn better representations of the source . they propose to associate source words with each other to help retain their relationships . |
| Outcome: | Experiments show that the proposed approach outperforms the encoder-decoder framework on several datasets. |
FELIX: Flexible Text Editing Through Tagging and Insertion (2020.findings-emnlp)
Copied to clipboard
| Challenge: | FELIX is efficient in low-resource settings and fast at inference time, while being capable of modeling flexible input-output transformations. |
| Approach: | They propose a flexible text-editing approach that decomposes a text-generating task into two sub-tasks: tagging and insertion. |
| Outcome: | The proposed model is efficient in low-resource settings and fast at inference time while being capable of modeling flexible input-output transformations. |
Autoregressive Score Generation for Multi-trait Essay Scoring (2024.findings-eacl)
Copied to clipboard
| Challenge: | Existing holistic approaches to score essays using pre-trained BERT-based models are inefficient, leading to inferior qualities in data-scarce traits. |
| Approach: | They propose an autoregressive prediction of multi-trait scores using pre-trained T5 models. |
| Outcome: | The proposed model shows over 5% improvement in prompts and traits compared to previous models . |
DYLE: Dynamic Latent Extraction for Abstractive Long-Input Summarization (2022.acl-long)
Copied to clipboard
Ziming Mao, Chen Henry Wu, Ansong Ni, Yusen Zhang, Rui Zhang, Tao Yu, Budhaditya Deb, Chenguang Zhu, Ahmed Awadallah, Dragomir Radev
| Challenge: | Existing models struggle with summarizing long text due to high memory complexity of the full self-attention. |
| Approach: | They propose a dynamic latent extraction approach for abstractive long-input summarization that treats extracted text snippets as latent variables and allows dynamic attention weights during decoding. |
| Outcome: | The proposed method outperforms existing methods on GovReport, QMSum, and arXiv while yielding strong results on arX. |
CTC Alignments Improve Autoregressive Translation (2023.eacl-main)
Copied to clipboard
Brian Yan, Siddharth Dalmia, Yosuke Higuchi, Graham Neubig, Florian Metze, Alan W Black, Shinji Watanabe
| Challenge: | Connectionist Temporal Classification (CTC) is widely used for automatic speech recognition (ASR) but lags behind attentional decoder approaches in terms of translation quality. |
| Approach: | They propose to use a CTC/attention framework to validate this hypothesis by modifying the Hybrid CTC-Attention model proposed for automatic speech recognition to support text-to-text translation (MT) and speech-totext translation. |
| Outcome: | The proposed model outperforms pure-attention baselines across six translation tasks. |
A Compare Aggregate Transformer for Understanding Document-grounded Dialogue (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have focused on KS in unstructured documents, but dialogue history that is not related to the current dialogue may introduce noise in the KS processing. |
| Approach: | They propose a Compare Aggregate Transformer to jointly denoise the dialogue context and aggregate the document information for response generation. |
| Outcome: | The proposed model outperforms the state-of-the-art approach and strong baselines on a CMU_DoG dataset. |
Fast and Accurate Reordering with ITG Transition RNN (C18-1)
Copied to clipboard
| Challenge: | Attention-based sequence-to-sequence neural networks learn to jointly align and translate. |
| Approach: | They propose to use a reordering RNN that shares the input encoder with the decoder to decouple re-ordering from translation. |
| Outcome: | The proposed model can achieve superior reordering accuracy without feature engineering and is 2.5x faster in decoding. |
Hexatagging: Projective Dependency Parsing as Tagging (2023.acl-short)
Copied to clipboard
| Challenge: | Using a pretrained language model, we can train language models on increasingly large amounts of data. |
| Approach: | They propose a dependency parser that constructs dependency trees by tagging words with elements from a finite set of possible tags. |
| Outcome: | The proposed approach achieves state-of-the-art performance of 96.4 LAS and 97.4 UAS on the Penn Treebank test set. |
Self-Attentive Residual Decoder for Neural Machine Translation (N18-1)
Copied to clipboard
| Challenge: | Neural sequence-to-sequence networks with attention have been used for machine translation . however, the target-side context is limited and the model lacks the ability to capture non-syntactic dependencies among words. |
| Approach: | They propose a sequence-to-sequence network with attention that captures contextual information at each time-step prediction through an attention mechanism. |
| Outcome: | The proposed model outperforms a neural MT baseline and memory and self-attention network on three language pairs. |
MEDA: Dynamic KV Cache Allocation for Efficient Multimodal Long-Context Inference (2025.naacl-long)
Copied to clipboard
| Challenge: | Long-context Multimodal Large Language Models (MLLMs) require substantial computational resources as their multimodal Key-Value (KV) cache grows with increasing input lengths, challenging memory and time efficiency. |
| Approach: | They propose a dynamic multimodal KV cache allocation strategy that dynamically allocating KV size based on attention entropy to better adapt to multimodal interactions. |
| Outcome: | The proposed model achieves up to 72% KV cache memory reduction and 2.82 faster decoding speeds while maintaining or enhancing performance on various multimodal tasks in a long context. |
Imitation Learning for Non-Autoregressive Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Existing non-autoregressive translation models lack parallel decoding, which is a bottleneck for NMT decoding. |
| Approach: | They propose a framework for non-autoregressive machine translation that emulates the autoregressive model by sampling sentence length in parallel. |
| Outcome: | The proposed model achieves 31.85 BLEU on WMT16 RoEn and 30.68 BLUE on IWSLT16 EnDe on the IWSLD16, WMT14 and WMT15 datasets. |
Fast and Accurate Non-Projective Dependency Tree Linearization (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for decoding dependency trees are 10 times faster than current ones. |
| Approach: | They propose a graph-based method to tackle a dependency tree linearization task . they propose to solve a Traveling Salesman Problem and combine the solution into a projective tree . |
| Outcome: | The proposed method outperforms the state-of-the-art linearizer while being 10 times faster in training and decoding. |
BiTIIMT: A Bilingual Text-infilling Method for Interactive Machine Translation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing IMT systems relying on lexical constrained decoding (LCD) are limited in translation efficiency and quality due to LCD. |
| Approach: | They propose a novel interactive neural machine translation system that uses lexical constraints to decode missing words in a manually revised translation. |
| Outcome: | The proposed system performs significantly better and faster than state-of-the-art IMT on three translation tasks. |
Short Text Topic Modeling with Topic Distribution Quantization and Negative Sampling Decoder (2020.emnlp-main)
Copied to clipboard
| Challenge: | Topic models for short texts suffer from data sparsity because of limited word co-occurrences. |
| Approach: | They propose a neural topic model with a new topic distribution quantization approach that generates peakier distributions that are more appropriate for modeling short texts. |
| Outcome: | The proposed model outperforms both strong traditional and neural baselines under extreme data sparsity scenes, producing high-quality topics. |
Dynamic Oracle for Neural Machine Translation in Decoding Phase (L18-1)
Copied to clipboard
| Challenge: | Existing methods to improve NMT performance but there is a discrepancy between training and inference when decoding. |
| Approach: | They propose to use Scheduled Sampling to reduce the discrepancy between training and inference in NMT when decoding to mitigate the discrépancy. |
| Outcome: | The proposed methods improve translation quality over standard NMT system. |
Parallel Sentence Mining by Constrained Decoding (2020.acl-main)
Copied to clipboard
| Challenge: | Using a neural machine translation system to extract parallel sentences from two monolingual corpora is challenging. |
| Approach: | They propose a method to extract parallel sentences from two monolingual corpora by constraining the decoding by a prefix tree built on the other corpus. |
| Outcome: | The proposed method achieves comparable results to other submissions on the BUCC shared task. |
Align-Refine: Non-Autoregressive Speech Recognition via Iterative Realignment (2021.naacl-main)
Copied to clipboard
| Challenge: | Non-autoregressive encoder-decoder models improve decoding speed, but generation quality suffers . editing at the level of output sequences limits model flexibility. |
| Approach: | They propose *iterative realignment* which iteratively realigns connectionist temporal alignments. |
| Outcome: | The proposed model matches an autoregressive baseline with a 14x speedup on the WSJ dataset; on LibriSpeech, it achieves an LM-free test-other WER of 9.0% (19% relative improvement on comparable work). |
Eliciting In-context Retrieval and Reasoning for Long-context Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks like LOFT often overestimate LCLM performance by providing overly simplified contexts. |
| Approach: | They propose to use retrieval-attention-probing to filter and de-noise long contexts during decoding and joint retrieval head training alongside the generation head to improve LCLM performance. |
| Outcome: | The proposed approach outperforms RAG and GPT-4-Turbo on most tasks despite being a much smaller model. |
Accelerating Neural Transformer via an Average Attention Network (P18-1)
Copied to clipboard
| Challenge: | Using parallelizable attention networks, the neural Transformer is slow to train due to auto-regressive architecture and self-attention in the decoder. |
| Approach: | They propose an average attention network to replace the original self-attention model in the decoder of the neural Transformer. |
| Outcome: | The proposed network can decode sentences over four times faster than the original version with almost no loss in training time and translation performance. |
If beam search is the answer, what was the question? (2020.emnlp-main)
Copied to clipboard
| Challenge: | surprisingly, beam search results on language generation tasks are low-quality . despite its high error rate, beam searches can be used to decode models with high probability . |
| Approach: | They frame beam search as the exact solution to a different decoding objective . they propose a set of decoding objectives that explicitly enforce this property . |
| Outcome: | The proposed method enforces uniform information density in text, a property motivated by cognitive science. |
Critic-Driven Decoding for Mitigating Hallucinations in Data-to-text Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Hallucination of text lacking grounding in input data is a problem in neural data-to-text generation. |
| Approach: | They propose to combine probabilistic output of a generator language model with the output of an “text critic” classifier which guides the generation by assessing the match between the input data and the generated text. |
| Outcome: | The proposed method improves on the WebNLG and OpenDialKG benchmarks. |
ReaRev: Adaptive Reasoning for Question Answering over Knowledge Graphs (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Knowledge Graph Question Answering (KGQA) involves retrieving entities as answers from a Knowledge Flow using natural language queries. |
| Approach: | They propose a method to decode a question into instructions that are dense question representations used to guide the KG traversals. |
| Outcome: | The proposed method improves instruction decoding and execution by using a KG-aware information to update the initial instructions. |
Expressing Visual Relationships via Language (P19-1)
Copied to clipboard
| Challenge: | Current studies on image captioning focus on single image, but there are no effective models for generating relational captions for two images. |
| Approach: | They propose a language-guided image editing dataset that contains real image pairs with corresponding editing instructions. |
| Outcome: | The proposed model outperforms baseline and existing methods on two datasets. |
Efficient Tool Use with Chain-of-Abstraction Reasoning (2025.coling-main)
Copied to clipboard
Silin Gao, Jane Dwivedi-Yu, Ping Yu, Xiaoqing Ellen Tan, Ramakanth Pasunuru, Olga Golovneva, Koustuv Sinha, Asli Celikyilmaz, Antoine Bosselut, Tianlu Wang
| Challenge: | Recent large language models have made progress at interpreting and executing instructions. |
| Approach: | They propose a method to decouple general reasoning from specialized knowledge . they propose to use abstract reasoning chains and domain tools to reify each chain . |
| Outcome: | The proposed method outperforms baseline methods on QA and mathematical reasoning domains. |
Icon2: Aligning Large Language Models Using Self-Synthetic Preference Data via Inherent Regulation (2025.emnlp-main)
Copied to clipboard
Qiyuan Chen, Hongsen Huang, Qian Shao, Jiahe Chen, Jintai Chen, Hongxia Xu, Renjie Hua, Ren Chuan, Jian Wu
| Challenge: | Large Language Models (LLMs) require high quality preference datasets to align with human preferences. |
| Approach: | They propose a framework that leverages inherent regulation of LLMs’ representation space for efficient and tailored preference dataset construction, named Icon2. |
| Outcome: | The proposed framework improves performance on benchmarks like AlpacaEval 2.0 and Arena-Hard while reducing computational costs by up to 48.1%. |
SetGNER: General Named Entity Recognition as Entity Set Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is a fundamental task in the field of information extraction and has played an important role in the development of natural language processing. |
| Approach: | They propose a method that treats each entity as a sequence and is capable of recognizing discontinuous mentions. |
| Outcome: | The proposed model outperforms state-of-the-art generative NER models on two discontinuous NER datasets, two nested NER and one flat NER. |
ParaLS: Lexical Substitution via Pretrained Paraphraser (2023.acl-long)
Copied to clipboard
| Challenge: | Lexical substitution (LS) is an extremely powerful technology that can be used as a backbone of various NLP applications such as writing assistance. |
| Approach: | They propose two simple decoding strategies that focus on the variations of the target word during decoding to generate substitutes from a paraphraser. |
| Outcome: | The proposed methods outperform state-of-the-art LS methods based on pre-trained language models on three benchmarks. |
Silencing the Guardrails: Inference-Time Jailbreaking via Dynamic Contextual Representation Ablation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing strategies to circumvent safety constraints face significant trade-offs between effectiveness and efficiency. |
| Approach: | They propose a framework that allows to infer model refusal behaviors without expensive parameter updates or training. |
| Outcome: | The proposed framework outperforms baselines in multiple safety-aligned open-source LLMs. |
Informed Sampling for Diversity in Concept-to-Text NLG (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to encourage lexical diversity for language generation tasks produce repetitive outputs, but this often comes at a cost to the perceived fluency and adequacy of the output. |
| Approach: | They propose to augment the decoding process with a meta-classifier trained to distinguish which words at any given timestep will lead to high-quality output. |
| Outcome: | The proposed method achieves a high level of diversity with minimal effect on the output’s fluency and adequacy. |
RealTranS: End-to-End Simultaneous Speech Translation with Convolutional Weighted-Shrinking Transformer (2021.findings-acl)
Copied to clipboard
| Challenge: | End-to-end simultaneous speech translation (SST) is useful in many scenarios but has not been fully investigated. |
| Approach: | They propose an end-to-end simultaneous speech translation model called RealTranS . they use interleaved convolution and unidirectional Transformer layers to downsample input speech . |
| Outcome: | The proposed model outperforms existing models on public and widely-used datasets in multiple latency settings. |
Incremental Beam Manipulation for Natural Language Generation (2021.eacl-main)
Copied to clipboard
| Challenge: | a larger beam size can lead to deteriorating performance of natural language generation systems due to model errors . performance of NLG systems can plateau or even decrease when beam sizes larger than 10 are used . |
| Approach: | They propose to rerank the output of beam search to produce a good set of hypotheses . they propose incremental beam manipulation to discarded hypothese . |
| Outcome: | The proposed method outperforms a strong reranker on the E2E and WebNLG datasets while being on par with the existing method. |
Improvement in Sign Language Translation Using Text CTC Alignment (2025.coling-main)
Copied to clipboard
| Challenge: | Current sign language translation (SLT) approaches rely on gloss-based supervision with Connectionist Temporal Classification (CTC) limiting their ability to handle non-monotonic alignments between sign language video and spoken text. |
| Approach: | They propose a method that integrates CTC/Attention with the attention mechanism during decoding and integrates it with the sign language video and spoken text. |
| Outcome: | The proposed method outperforms the pure-attention baseline and achieves comparable results to state-of-the-art methods. |
Multi-Input Attention for Unsupervised OCR Correction (P18-1)
Copied to clipboard
| Challenge: | Existing methods for OCR correction are mostly supervised methods that correct recognition errors in a single output. |
| Approach: | They propose a sequence-to-sequence model with attention and a decoder with attention averaging to search for consensus among multiple sequences. |
| Outcome: | The proposed methods cut the character and word error rates nearly in half on single inputs and can rival supervised methods. |
Discerning and Resolving Knowledge Conflicts through Adaptive Decoding with Contextual Information-Entropy Constraint (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing decoding methods for large language models (LLMs) are specialized in resolving knowledge conflicts and could inadvertently deteriorate performance in absence of conflicts. |
| Approach: | They propose an adaptive decoding method to discern whether knowledge conflicts occur and resolve them by a contextual information-entropy constraint decoding technique. |
| Outcome: | The proposed method improves the model’s faithfulness to conflicting context and maintains high performance among non-conflicting contexts. |
LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context Inference (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Long-context Multimodal Large Language Models (MLLMs) require substantial computational resources for inference . the growth of their multimodal Key-Value (KV) cache challenges memory and time efficiency. |
| Approach: | They propose a fine-tuning-free approach that efficiently reduces the multimodal KV cache size while maintaining performance comparable to a full cache. |
| Outcome: | The proposed method reduces the multimodal KV cache size while maintaining performance comparable to a full cache. |
CodecLM: Aligning Language Models with Tailored Synthetic Data (2024.findings-naacl)
Copied to clipboard
Zifeng Wang, Chun-Liang Li, Vincent Perot, Long Le, Jin Miao, Zizhao Zhang, Chen-Yu Lee, Tomas Pfister
| Challenge: | Recent work on generating diverse instructions and applying LLM to increase instruction complexity neglects downstream use cases. |
| Approach: | They propose a framework for generating high-quality synthetic data for LLM alignment with different downstream instruction distributions and LLMs. |
| Outcome: | Experiments on four open-domain instruction using the proposed framework validate the effectiveness of CodecLM over the current state-of-the-art. |
Constrained Decoding with Speculative Lookaheads (2025.naacl-long)
Copied to clipboard
Nishanth Sridhar Nakshatri, Shamik Roy, Rajarshi Das, Suthee Chaidaroon, Leonid Boytsov, Rashmi Gangadharaiah
| Challenge: | Constrained decoding with lookahead heuristics is effective for aligning LLM generations to human preferences, but the extensive lookaheaded roll-out operations for each generated token make it prohibitively expensive. |
| Approach: | They propose a technique that uses lookaheads to align LLMs to human preferences . they propose 2.2x to 12.15x speedup over greedy decoding . |
| Outcome: | The proposed technique achieves 2.2x to 12.15x speedup over greedy decoding without significant performance reduction. |
Tree-structured Decoding for Solving Math Word Problems (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to solve math word problems do not consider an abstract syntax tree. |
| Approach: | They propose a tree-structured decoding method that generates an abstract syntax tree of an equation in a top-down manner and can stop during decoding without a redundant stop token. |
| Outcome: | The proposed method achieves state-of-the-art performance on the largest dataset on this task. |
Learning Cooperative Interactions for Multi-Overlap Aspect Sentiment Triplet Extraction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for aspect sentiment triplet extraction focus on the single interactions between an aspect and an opinion. |
| Approach: | They propose a multi-overlap triplet extraction method which decodes the complex relations between multiple aspects and opinions by learning their cooperative interactions. |
| Outcome: | The proposed method outperforms baselines, especially multi-overlap triplets. |
On Sparsifying Encoder Outputs in Sequence-to-Sequence Models (2021.findings-acl)
Copied to clipboard
| Challenge: | Using sequence-to-sequence models, encoder outputs are usually transferred to the decoder for generation, but in this study, encoded outputs can be compressed to shorten the sequence for decoding. |
| Approach: | They propose to use a stochastic gate-based algorithm to mask encoder outputs to shorten the sequence delivered for decoding. |
| Outcome: | The proposed model can be used to shorten encoder outputs to short a sequence . the proposed model yields a speedup of up to 1.65 on document summarization and 1.20 on character-based machine translation tasks. |
Graph-to-Tree Neural Networks for Learning Structured Input-Output Translation with Applications to Semantic Parsing and Math Word Problem (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Graph2Tree model encodes graph-structured input and decodes tree-structures output. |
| Approach: | They propose a novel Graph-to-Tree Neural Network consisting of a graph encoder and a hierarchical tree decoder that encodes an augmented graph-structured input and decodes a tree-structure-output. |
| Outcome: | The proposed model outperforms or matches the performance of other state-of-the-art models on two problems, neural semantic parsing and math word problem. |
SH2: Self-Highlighted Hesitation Helps You Decode More Truthfully (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have made great progress in text generation but suffer from hallucinations during reasoning and generation. |
| Approach: | They propose an inference-time method to help LLMs decode truthfully by selecting tokens with the lowest probabilities and concatenating them to the original context. |
| Outcome: | The proposed method improves LLaMA-7b, LLama2-7b and Mistral-7b on hallucination tasks. |
Zero-shot Sonnet Generation with Discourse-level Planning and Aesthetics Features (2022.naacl-main)
Copied to clipboard
| Challenge: | a sonnet is a fourteen-line poem with rigorous meter-and-rhyme constraints. |
| Approach: | They propose a framework which plans the poem sketch before decoding a sonnet without training on poems . they use a rhyme module, polishing module and a constrained decoding algorithm to impose the meter-and-rhyme constraint . |
| Outcome: | The proposed framework generates sonnets that are coherent and poetic without training on poems . the proposed framework is based on a framework that plans the poem sketch before decoding . |
Group is better than individual: Exploiting Label Topologies and Label Relations for Joint Multiple Intent Detection and Slot Filling (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent joint multiple intent detection and slot filling models ignore the dependencies among labels and label embeddings. |
| Approach: | They propose to construct a Heterogeneous Label Graph (HLG) containing two kinds of topologies and a novel model termed ReLa-Net which captures beneficial correlations among the labels from HLG. |
| Outcome: | The proposed model outperforms the previous model by over 20% on MixATIS dataset. |
Attention Can Reflect Syntactic Structure (If You Let It) (2021.eacl-main)
Copied to clipboard
| Challenge: | a recent study has attempted to decode linguistic structure from the Transformer . but, much of the work focused on English, a language with rigid word order and a lack of inflectional morphology. |
| Approach: | They propose to fine-tune a feature encoder for BERT to learn linguistic structure from its multi-head attention mechanism. |
| Outcome: | The proposed model can decode full trees above baseline accuracy from single attention heads across languages. |
Deconvolution-Based Global Decoding for Neural Machine Translation (C18-1)
Copied to clipboard
| Challenge: | Existing models for Neural Machine Translation (NMT) use Recurrent Neural Network (RNN) to generate translation word by word following a sequential order. |
| Approach: | They propose a Neural Machine Translation (NMT) model that decodes the sequence with the guidance of its structural prediction of the target-side context. |
| Outcome: | The proposed model is more competitive compared with the state-of-the-art methods and reduces repetition with the instruction from the target-side context for decoding. |
Non-Autoregressive Sentence Ordering (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing sentence ordering approaches only leverage unilateral dependencies during decoding and cannot fully explore the semantic dependency between sentences. |
| Approach: | They propose a non-autoregressive ordering network that explores bilateral dependencies between sentences and predicts sentences for each position in parallel. |
| Outcome: | The proposed model outperforms existing autoregressive sentence ordering approaches and yields competitive performance compared with the state-of-the-arts. |
HS-STaR: Hierarchical Sampling for Self-Taught Reasoners via Difficulty Estimation and Budget Reallocation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have incorporated reward models to guide response selection or decoding, aiming to obtain higher-quality data. |
| Approach: | They propose a Hierarchical Sampling framework for self-taught reasoners that allocates a fixed sampling budget to problem boundary-level problems and then reallocates the remaining budget toward high-utility problems during a re-sampling phase. |
| Outcome: | The proposed framework outperforms baseline models without additional sampling budgets across multiple reasoning benchmarks and backbone LLMs. |
HALP: Detecting Hallucinations in Vision-Language Models without Generating a Single Token (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing methods for detection of hallucinations operate after text generation, making intervention costly and untimely. |
| Approach: | They examine whether hallucination risk can instead be predicted before any token is generated by probing a model's internal representations in a single forward pass. |
| Outcome: | The proposed model can detect hallucinations before token generation, while query-token representations can be more accurate. |
A Framework for Bidirectional Decoding: Case Study in Morphological Inflection (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing encoder-decoders that generate sequences from left to right are prone to errors due to the "snowballing" effect. |
| Approach: | They propose a transformer-based encoder-decoder framework that produces sequences from the "outside-in" they argue that this approach is more principled than prior bidirectional decoders . |
| Outcome: | The proposed model beats the current system by over 4.7 and 2.7 points in accuracy on 2022 and 2023 tasks. |
Look, Compare, Decide: Alleviating Hallucination in Large Vision-Language Models via Multi-View Multi-Path Reasoning (2025.coling-main)
Copied to clipboard
| Challenge: | Large Vision-Language Models (LVLMs) have impressive capabilities in multi-modal context comprehension, but they still suffer from hallucination problems due to inconsistent outputs with the image content. |
| Approach: | They propose a training-free framework MVP to reduce hallucinations in Large Vision-Language Models . they propose multi-view information-seeking strategy to perceive the comprehensive information in the image . |
| Outcome: | The proposed framework reduces hallucinations in large vision-language models by combining multi-view multi-path reasoning with multi-vision multi-path reasoning. |
Almost Free Semantic Draft for Neural Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Empirical experiments show that the presented method can achieve competitive performance in common language pairs with a clear advantage in inference efficiency. |
| Approach: | They propose a method to sample and consider a semantic draft as global information from semantic space for decoding with almost free of cost. |
| Outcome: | Empirical results show that the proposed method can achieve competitive performance in common language pairs with a clear advantage in inference efficiency. |
SummaReranker: A Multi-Task Mixture-of-Experts Re-ranking Framework for Abstractive Summarization (2022.acl-long)
Copied to clipboard
| Challenge: | Sequence-to-sequence neural networks have enabled great progress in abstractive summarization. |
| Approach: | They propose to train a second-stage model performing re-ranking on a set of summary candidates by using a mixture of experts. |
| Outcome: | The proposed model outperforms the base model on CNN- DailyMail, XSum and Reddit TIFU with a base PEGASUS. |
Beyond Hard Masks: Progressive Token Evolution for Diffusion Language Models (2026.acl-long)
Copied to clipboard
Linhao Zhong, Linyu Wu, Bozhen Fang, Tianjian Feng, Chenchen Jing, Wen Wang, Jiaheng Zhang, Hao Chen, Chunhua Shen
| Challenge: | Existing Diffusion Language Models rely on hard binary masking and discrete token assignments, which hinder the revision of early decisions. |
| Approach: | They propose a diffusion-based language modeling approach that replaces hard binary masks with evolving soft token distributions. |
| Outcome: | The proposed approach outperforms existing DLMs on multiple benchmarks. |
Viterbi Decoding of Directed Acyclic Transformer for Non-Autoregressive Machine Translation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Non-autoregressive models lack the ability to capture sequential dependency . Existing approaches to model sequential dependency have to apply a sequential decision process at inference time . |
| Approach: | They propose a Viterbi decoding framework to capture sequential dependency . they propose to find the optimal translation path under any length constraint . |
| Outcome: | The proposed framework improves the performance of DA-Transformer while maintaining similar speedup. |
Joint Extraction of Entities, Relations, and Events via Modeling Inter-Instance and Inter-Label Dependencies (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing models that perform information extraction tasks manually assume heuristic dependency between the task instances and mean-field factorization for the joint distribution of instance labels. |
| Approach: | They propose to induce a dependency graph among task instances to boost representation learning by estimating their joint distribution via Conditional Random Fields. |
| Outcome: | The proposed model outperforms previous models on multiple IE tasks across 5 datasets and 2 languages. |
A Plug-and-Play Method for Controlled Text Generation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for controlling language generation are not able to produce fluent text . current methods require additional models or fine-tuning to ensure specific words are included . |
| Approach: | They propose a plug-and-play decoding method that allows for controlled language generation . they add a shift in the probability distribution over our vocabulary towards semantically similar words . |
| Outcome: | The proposed method outperforms competing methods in human evaluations and does not impact fluency. |
NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraints (2021.naacl-main)
Copied to clipboard
| Challenge: | Conditional text generation often requires lexical constraints, i.e., which words should or shouldn't be included in the output text. |
| Approach: | They propose an algorithm that enables neural language models to generate fluent text while satisfying complex lexical constraints. |
| Outcome: | The proposed algorithm outperforms existing methods on four benchmarks and shows that it handles any set of lexical constraints expressible under predicate logic while its asymptotic runtime is equivalent to conventional beam search. |
Diverse Retrieval-Augmented In-Context Learning for Dialogue State Tracking (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work has demonstrated that in-context learning for dialogue state tracking outperforms training methods in the few-shot setting. |
| Approach: | They propose a method for in-context learning for dialogue state tracking that takes into account probabilities of competing surface forms and produces a more accurate dialogue state prediction. |
| Outcome: | The proposed method outperforms trained methods in the few-shot setting and requires little data and zero parameter updates. |
Narrowing the Knowledge Evaluation Gap: Open-Domain Question Answering with Multi-Granularity Answers (2024.acl-long)
Copied to clipboard
| Challenge: | Standard question answering evaluation protocols do not take factual errors into account and compare a predicted answer against answers of a single granularity level. |
| Approach: | They propose a new evaluation setting where a predicted answer is evaluated in terms of accuracy and informativeness against a set of multi-granularity answers. |
| Outcome: | The proposed evaluation setting compares a predicted answer against a set of multi-granularity answers. |
A Streaming Approach For Efficient Batched Beam Search (2020.emnlp-main)
Copied to clipboard
| Challenge: | During decoding, candidates terminate or are pruned according to heuristics, a streaming method is used to "refill" the batch after it finishes translating some fraction of the current batch. |
| Approach: | They propose an efficient batching strategy for variable-length decoding on GPU architectures by streamlining the batching process. |
| Outcome: | The proposed method reduces runtime by 71% compared to a fixed-width beam search baseline and 17% compared with a variable-widness baseline while matching baselines’ BLEU. |
Vocabulary Pyramid Network: Multi-Pass Encoding and Decoding with Multi-Level Vocabularies for Response Generation (P19-1)
Copied to clipboard
| Challenge: | Conventional methods employ a fixed vocabulary and one-pass decoding, which make them prone to safe and general responses and lack further refinement to the first generated raw sequence. |
| Approach: | They propose a Vocabulary Pyramid Network which integrates multi-pass encoding and decoding with multi-level vocabularies into response generation. |
| Outcome: | The proposed system outperforms strong baselines on English Twitter and Chinese Weibo datasets. |
On the Copying Behaviors of Pre-Training for Neural Machine Translation (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that initializing NMT models with pre-trained language models (LM) can speed up the model training and boost the model performance. |
| Approach: | They propose a method to control copying behaviors in NMT models by initializing them with pre-trained language models (LM) they propose to use a metric called copy ratio to control the copying behavior in decoding. |
| Outcome: | The proposed method improves translation performance by controlling copying behaviors for pre-training based models. |
Global Reasoning over Database Structures for Text-to-SQL Parsing (D19-1)
Copied to clipboard
| Challenge: | Existing semantic parsers only select a set of database constants at training time . current models only consider local information, not global ones . |
| Approach: | They propose a semantic parser that globally reasons about the structure of the query to make a more contextually-informed selection of database constants. |
| Outcome: | The proposed model increases accuracy from 39.4% to 47.4% on a zero-shot semantic parsing dataset with complex databases. |
Methods for Numeracy-Preserving Word Embeddings (2020.emnlp-main)
Copied to clipboard
Dhanasekar Sundararaman, Shijing Si, Vivek Subramanian, Guoyin Wang, Devamanyu Hazarika, Lawrence Carin
| Challenge: | Word embedding models capture semantic relationships between words but fail to capture numerical properties associated with numbers. |
| Approach: | They propose a method to assign and learn embeddings for numbers using word embedders. |
| Outcome: | The proposed model outperforms pre-trained word embedding models across multiple examples of two tasks. |
Mutual Information Alleviates Hallucinations in Abstractive Summarization (2022.emnlp-main)
Copied to clipboard
| Challenge: | Abstractive summarization models exhibit the tendency to hallucinate, i.e., output content not supported by the source document. |
| Approach: | They propose a decoding strategy that optimizes for pointwise mutual information of source and target tokens when models exhibit uncertainty. |
| Outcome: | The proposed method decreases the probability of hallucinated tokens while maintaining the Rouge and BERT-S scores of top-performing decoding strategies. |
Decoding in Latent Spaces for Efficient Inference in LLM-based Recommendation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Light Latent-space Decoding (L2D) is an efficient and efficient latent- space decoding method. |
| Approach: | They propose to bypass language-space decoding by matching candidate items with LLM's internal thought representations in the latent space. |
| Outcome: | The proposed method is 10x faster than language-space decoding while maintaining or enhancing performance. |
AutoEQA: Auto-Encoding Questions for Extractive Question Answering (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Extractive question answering models are reliant on annotations of answer-spans in the corresponding passages. |
| Approach: | They propose a method that auto-encodes a question and generates corresponding questions from it. |
| Outcome: | The proposed method performs well in a zero-shot setting and can provide an additional loss to boost performance for extractive question answering (EQA). |
Stacked AMR Parsing with Silver Data (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Lack of large-scale annotated data is one main challenge for abstract meaning representation (AMR) parsing. |
| Approach: | They propose to use silver data to train a pre-trained abstract meaning representation model. |
| Outcome: | The proposed model outperforms previous models on the AMR2.0 dataset and is faster than the SOTA model. |
An Empirical Investigation of Beam-Aware Training in Supertagging (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Structured prediction is often approached by training a local normalized model with maximum likelihood and decoding approximately with beam search. |
| Approach: | They propose a meta-algorithm that captures beam-aware training algorithms and suggests new ones. |
| Outcome: | The proposed algorithm improves performance for both models and the simpler model . it also improves the model which must manage uncertainty during decoding . |
Rethinking Model Selection and Decoding for Keyphrase Generation with Pre-trained Sequence-to-Sequence Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Keyphrase generation is a longstanding task in NLP with widespread applications. |
| Approach: | They propose a likelihood-based decode-select algorithm for seq2seq PLMs that improves greedy search by an average of 4.7% semantic F1 across five datasets. |
| Outcome: | The proposed algorithm improves greedy search by an average of 4.7% semantic F1 across five datasets. |
Sentence-Permuted Paragraph Generation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing models produce similar contents from homogenized contexts due to the fixed left-to-right sentence order. |
| Approach: | They propose a framework permuting sentence orders to improve content diversity of multi-sentence paragraphs by permutating the sentence orders. |
| Outcome: | The proposed framework produces more diverse outputs with higher quality than existing models. |
KCVR: Knowledge-Centric Video Reconstruction for Structured Pedagogical Summarization via Dynamic Graph Planning (2026.acl-long)
Copied to clipboard
Jingjiang Liu, Jia Zhu, Hanghui Guo, Weijie Shi, Yue Cui, Xiaokang Jin, Yilin Wang, Qingyu Niu, Jiawei Shen, Guoqing Ma, Yidan Liang, Shimin Di, Jiajie Xu
| Challenge: | Existing summarization methods compress content for gist browsing, but they break prerequisite logic in instructional videos. |
| Approach: | They propose a framework that decouples epistemic planning from content generation. |
| Outcome: | The proposed framework outperforms strong end-to-end baselines on Knowledge Progression Consistency and Learning Objective Coverage. |
Gold Panning in Vocabulary: An Adaptive Method for Vocabulary Expansion of Domain-Specific LLMs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) struggle when it comes to specialized domains due to limited domain-specific knowledge. |
| Approach: | They propose an adaptive method that automatically identifies valuable words from a given domain vocabulary. |
| Outcome: | The proposed method has been validated on three Chinese datasets and performed on general tasks. |
Neural Quality Estimation with Multiple Hypotheses for Grammatical Error Correction (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing GEC models produce spurious corrections or fail to detect lots of errors. |
| Approach: | They propose a neural network for GEC quality estimation with multiple hypotheses . VERNet establishes interactions among hypothese based on reasoning graph . |
| Outcome: | The proposed model achieves state-of-the-art grammatical error detection performance and best quality estimation results on four GEC datasets. |
Abstractive Text-Image Summarization Using Multi-Modal Attentional Hierarchical RNN (D18-1)
Copied to clipboard
| Challenge: | Recent research shows the strength of the Encoder-Decoder model in text summarization. |
| Approach: | They propose to use the attentional hierarchical Encoder-Decoder model to summarize a text document and its accompanying images simultaneously and then to align the sentences and images in summaries. |
| Outcome: | The proposed model outperforms the existing methods that do not consider images . it can generate informative summaries of images, and it can be used to summarize documents . |
Entropy Guided Extrapolative Decoding to Improve Factuality in Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) exhibit impressive natural language capabilities but suffer from hallucination – generating content that does not align with realworld facts. |
| Approach: | They propose to extrapolate critical token probabilities beyond the last layer to improve decoding by manipulating the predicted distributions at inference time. |
| Outcome: | The proposed methods surpass state-of-the-art on multiple datasets by large margins. |
Rewriter-Evaluator Architecture for Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to improve neural machine translation models with multiple decoding passes lack proper policies to terminate multi-pass processes. |
| Approach: | They propose a novel architecture of Rewriter-Evaluator to terminate multi-pass decoding . they propose prioritized gradient descent to jointly and efficiently train rewriter and evaluator . |
| Outcome: | The proposed architecture significantly outperforms existing methods on three translation tasks and reduces performance gaps to oracle policies. |
Improving Language Model Integration for Neural Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to integrate external language models into machine translation systems have been based on the assumption that the external model learns an implicit target-side language model at decoding time. |
| Approach: | They transfer this concept to the task of machine translation and compare it with the most prominent way of including additional monolingual data - namely back-translation. |
| Outcome: | The proposed approach outperforms the most prominent way of including additional monolingual data, namely back-translation. |
Label-Synchronous Neural Transducer for E2E Simultaneous Speech Translation (2024.acl-long)
Copied to clipboard
| Challenge: | LS-Transducer-SST dynamically decides when to emit translation tokens based on an Auto-regressive Integrate-and-Fire (AIF) mechanism. |
| Approach: | They propose a label-synchronous neural transducer for simultaneous speech translation which dynamically decides when to emit translation tokens based on an Auto-regressive Integrate-and-Fire mechanism. |
| Outcome: | The proposed model can predict translation tokens on monolingual text-only data or can be used in both decoding and training. |
OpenPI-C: A Better Benchmark and Stronger Baseline for Open-Vocabulary State Tracking (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work on state tracking assumes both entities and state space are known, which limits their applicability. |
| Approach: | They propose a two-stage model that refines the state change prediction conditioned on entities predicted from the first stage. |
| Outcome: | The proposed model improves on the cleaned dataset and the evaluation metric on the proposed model. |
Accelerate Parallelizable Reasoning via Parallel Decoding within One Sequence (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for parallelizable reasoning tasks are inefficient, says a new study . generating lengthy reasoning sequences is computationally expensive and time-consuming, says the study authors . |
| Approach: | They propose a method that decodes multiple tokens per forward pass using a tree-like attention mask . their method achieves nearly 100% speedup in decoding while basically maintaining the answer quality . |
| Outcome: | Experimental results show that the method achieves nearly 100% speedup in decoding while maintaining the answer quality. |
Alleviating Hallucinations of Large Language Models through Induced Hallucinations (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies have shown that large language models generate inaccurate or fabricated information, a phenomenon known as hallucinations. |
| Approach: | They propose a simple strategy to induce-then-contrast decode LLMs to enhance their factuality . they first induce hallucinations from the original model and penalize them . |
| Outcome: | The proposed strategy improves factuality of large language models across task formats, model sizes, and model families. |
Task-Oriented Conversation Generation Using Heterogeneous Memory Networks (D19-1)
Copied to clipboard
| Challenge: | Existing memory networks do not perform well when leveraging heterogeneous information from different sources. |
| Approach: | They propose to use user utterances, dialogue history and background knowledge tuples to integrate external knowledge into a neural dialogue model. |
| Outcome: | The proposed model outperforms the state-of-the-art data-driven task-oriented dialogue models on real-world datasets. |
POS-Constrained Parallel Decoding for Non-autoregressive Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing non-autoregressive generation systems face multimodality problem due to conditionally independent decoding. |
| Approach: | They propose to incorporate linguistic structure into NAG inference instead of teacher AG . they propose a method that provides a specific POS sequence to constrain the NAG model . |
| Outcome: | The proposed method improves NAG models on four text generation tasks to a greater extent compared to knowledge distillation. |
Regularized Best-of-N Sampling with Minimum Bayes Risk Objective for Language Model Alignment (2025.naacl-long)
Copied to clipboard
| Challenge: | Best-of-N (BoN) sampling is an effective strategy for aligning Large Language Models (LLMs) to human preferences at the time of decoding. |
| Approach: | They propose a variant of BoN that incorporates the Minimum Bayes Risk objective as a proximity regularizer for BoN sampling. |
| Outcome: | The proposed method outperforms both BoN sampling and MBR decoding on the AlpacaFarm and Anthropic datasets. |
Alignment-Based Decoding Policy for Low-Latency and Anticipation-Free Neural Japanese Input Method Editors (2024.findings-acl)
Copied to clipboard
| Challenge: | Japanese input method editors (IMEs) allow users to input Japanese text using a limited set of characters such as the kana syllabary. |
| Approach: | They propose a simple decoding policy to enable simultaneous kana-kanji conversion in Japanese IMEs inspired by simultaneous machine translation. |
| Outcome: | The proposed approach achieves a better quality-latency trade-off than baselines while being more practical due to its ability to directly handle streaming input. |
A Thorough Examination of Decoding Methods in the Era of LLMs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Decoding methods are essential for converting language models from next-token predictors into practical task solvers. |
| Approach: | They propose to evaluate decoding methods in general-purpose large language models . they find that decoding method performance is notably task-dependent . |
| Outcome: | The proposed methods perform task-dependently and are influenced by alignment, model size, and quantization. |
BASS: Batched Attention-optimized Speculative Sampling (2024.findings-acl)
Copied to clipboard
Haifeng Qian, Sujan Kumar Gonugondla, Sungsoo Ha, Mingyue Shang, Sanjay Krishna Gouda, Ramesh Nallapati, Sudipta Sengupta, Xiaofei Ma, Anoop Deoras
| Challenge: | Speculative decoding has emerged as a powerful method to improve latency and throughput in hosting large language models. |
| Approach: | They propose a batched speculative decoding system that generates sequences at an average speed of 5.8ms per token and a batch size of 8 at a 2.15 speed-up over optimized regular decoding. |
| Outcome: | The proposed system achieves state-of-the-art latency and speed-up over optimized regular decoding. |
CoMIF: Modeling of Complex Multiple Interaction Factors for Conversation Generation (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for generating human-like dialogues lack implicit correlations among factors . different factors may alternately dominate token-level response generation during decoding . |
| Approach: | They propose a framework that can model complex multiple interaction factors to generate human-like conversations. |
| Outcome: | The proposed framework generates human-like conversations with multiple factors compared to state-of-the-art methods . et al. show that the proposed framework produces superior results over existing methods compared with the state- of-the art methods based on multiple datasets . |
OjaKV: Context-Aware Online Low-Rank KV Cache Compression (2026.findings-acl)
Copied to clipboard
Yuxuan Zhu, David H. Yang, Mohammad Mohammadi Amiri, Keerthiram Murugesan, Tejaswini Pedapati, Pin-Yu Chen
| Challenge: | Existing methods for inference use static, offline-learned subspaces that perform poorly under distribution shifts. |
| Approach: | They propose a framework that integrates a storage policy with an online subspace adaptation to preserve key-value tokens in full rank as high-fidelity anchors. |
| Outcome: | Experiments show that OjaKV maintains or improves zero-shot accuracy at high compression ratios, achieving the strongest gains on long-context benchmarks requiring complex reasoning. |
WatME: Towards Lossless Watermarking Through Lexical Redundancy (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for text watermarking rely on arbitrary vocabulary partitioning during decoding, which compromises the availability of suitable tokens and significantly degrades the quality of responses. |
| Approach: | They propose a method that leverages linguistic prior knowledge of lexical redundancies in LLM vocabularies to seamlessly integrate watermarks. |
| Outcome: | The proposed approach preserves the expressive power of large language models while preserving watermark detectability. |
Enhancing Neural Machine Translation Through Target Language Data: A kNN-LM Approach for Domain Adaptation (2025.acl-long)
Copied to clipboard
Abudurexiti Reheman, Hongyu Liu, Junhao Ruan, Abudukeyumu Abudula, Yingfeng Luo, Tong Xiao, JingBo Zhu
| Challenge: | Neural machine translation (NMT) has made significant progress in recent years, yet often suffers from translating in new domains, which is called domain adaptation. |
| Approach: | They propose a method that leverages semantically similar target language sentences in the kNN framework and generates a probability distribution over these sentences during decoding. |
| Outcome: | The proposed method generates a probability distribution over similar target language sentences and then interpolates with the model’s distribution. |
P3LM: Probabilistically Permuted Prophet Language Modeling for Generative Pre-Training (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing autoregressive left-to-right (L2R) models are limited to unidirectional information and constrained on strong local dependencies. |
| Approach: | They propose a probabilistically permuted prophet language model which strengthens the modeling of bidirectional information and long token dependencies for sequence generation. |
| Outcome: | Experiments on GLGE dataset show that P3LM improves on natural language generation tasks. |
Focus on Your Question! Interpreting and Mitigating Toxic CoT Problems in Commonsense Reasoning (2024.acl-long)
Copied to clipboard
| Challenge: | Large language models exhibit high-level commonsense reasoning abilities, especially with enhancement methods like Chain-of-Thought (CoT). |
| Approach: | They propose a chain-of-thought-like method to elicit models' potential abilities to generate rationales and answers that are based on attribution tracing and causal tracers to probe the internal working mechanism of the LLM. |
| Outcome: | The proposed method eliminates Toxic CoT problems and improves the model’s overall commonsense reasoning performance by 5.5%. |
STEER: Unified Style Transfer with Expert Reinforcement (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Experimental results show unified style transfer models outperform the 175B instruction-tuned GPT-3 on overall style transfer quality. |
| Approach: | They propose a unified style transfer framework that can transfer to multiple target styles from an arbitrary source style. |
| Outcome: | The proposed method outperforms the 175B instruction-tuned GPT-3 on overall style transfer quality despite being 226 times smaller in size . |
Dodo: Dynamic Contextual Compression for Decoder-only LMs (2024.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to NLP are sparsifying attention patterns or approximating the attention computation with kernel methods. |
| Approach: | They propose a method for dynamic contextual compression for decoder-only LMs. |
| Outcome: | The proposed method reduces the cost of self-attention to a fraction of typical time and space. |
Levenshtein Training for Word-level Quality Estimation (2021.emnlp-main)
Copied to clipboard
| Challenge: | a novel scheme to perform word-level quality estimation is proposed for word-based quality estimation . authors propose a two-stage transfer learning procedure on augmented and human data . a Levenshtein Transformer can learn to post-edit without explicit supervision. |
| Approach: | They propose a novel scheme to use a Levenshtein Transformer to perform word-level quality estimation. |
| Outcome: | The proposed method performs better under data-constrained and unconstrained conditions than existing methods. |
CTC-based Non-autoregressive Textless Speech-to-Speech Translation (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing direct speech-to-speech translation models require text supervision during training, which is not feasible for numerous unwritten languages. |
| Approach: | They propose a non-autoregressive (NAR) model that generates discrete units from the source speech and employs a unit-based vocoder to synthesize the target. |
| Outcome: | The proposed model achieves translation quality comparable to the autoregressive model while preserving up to 26.81 decoding speedup. |
DecoCal: Decoding with Calibration in Diffusion Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Diffusion Large Language Models (DLLMs) generate text via iterative token denoising . but decoding is challenging, with many tokens appearing predictable early . |
| Approach: | They propose a Decoding framework that performs Calibration of token-level confidence across diffusion steps and leverages the calibrated results to guide decoding decisions. |
| Outcome: | Experiments on multiple DLLMs and benchmarks show that DecoCal improves generation accuracy compared to existing strategies. |
Learning Programmatic Idioms for Scalable Semantic Parsing (D19-1)
Copied to clipboard
| Challenge: | In state-of-the-art semantic parsers map natural language instructions to source code . idioms improve the accuracy of semantic parses, allowing for faster decoding . |
| Approach: | They propose an iterative method to extract code idioms from large source code corpora . they use most-frequent subtrees of their syntax trees to train semantic parsers to apply them . |
| Outcome: | The proposed method improves the state-of-the-art semantic parsers' accuracy and training time by more than 50%. |
DELTA: Dynamic Layer-Aware Token Attention for Efficient Long-Context Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Large reasoning models generate long chains of intermediate steps, but their inference cost is dominated by decoding, where each new token must attend to the entire growing sequence. |
| Approach: | They propose a training-free sparse attention mechanism that reduces inference cost by evicting entries from the key-value cache. |
| Outcome: | The proposed model matches or surpasses full attention on reasoning benchmarks . it reduces the number of attended tokens by up to 4.25 and delivers 1.54 speedup . |
Sample Efficient Alignment Learning With Episodic Control (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing parametric methods for aligning large language models with task objectives are limited. |
| Approach: | They propose a non-parametric framework that aligns large language models with task objectives . they use a key-value memory to store associations between generated text and its corresponding values . |
| Outcome: | The proposed framework outperforms state-of-the-art baselines on harmless, helpful, and summarization tasks. |
Dynamic Evil Score-Guided Decoding: An Efficient Decoding Framework For Red-Team Model (2025.findings-acl)
Copied to clipboard
Cong Gao, Bo Zhang, Linkang Yang, Minghao Hu, Zhunchen Luo, Xiaoying Bai, Guotong Geng, Jun Zhang, Yunhua Xue
| Challenge: | Existing red-teaming methods require expensive fine-tuning, especially for large LLMs. |
| Approach: | They propose a red-teaming method that uses an ‘evil score’ to evaluate the potential of tokens to contribute to harmful outputs during decoding. |
| Outcome: | The proposed method achieves an ASR of 92.83% on the Llama-3.2-3B-Instruct model, compared to 83.48% with adversarial fine-tuning while using less computational resources. |
Hint-Based Training for Non-Autoregressive Machine Translation (D19-1)
Copied to clipboard
| Challenge: | AutoRegressive Translation models have to generate tokens sequentially during decoding and thus suffer from high inference latency. |
| Approach: | They propose to use hidden states and word alignments to help train NART models. |
| Outcome: | The proposed model improves on the WMT14 En-De and De-En datasets but is faster in inference than the current models. |
Improving NMT Quality Using Terminology Injection (2020.lrec-1)
Copied to clipboard
| Challenge: | a recent study has explored the use of vetted terminology in neural machine translation . a number of organizations use domain- or organization-specific words and phrases . |
| Approach: | They propose a method for injecting terminology and for evaluating terminology injection. |
| Outcome: | The proposed method is based on the long-term memory (LSTM) attention mechanism prevalent in state-of-the-art systems . it also introduces a new translation metric more sensitive to approved terminological content in MT output. |
HybridKV: Hybrid KV Cache Compression for Efficient Multimodal Large Language Model Inference (2026.acl-long)
Copied to clipboard
| Challenge: | Multimodal Large Language Models (MLLMs) are hindered by the rapid growth of key–value (KV) caches. |
| Approach: | They propose a hybrid KV cache compression framework that reduces KV memory by 7.9 and speeds up decoding by 1.52. |
| Outcome: | Experiments on 11 multimodal benchmarks show that HYBRIDKV cuts KV cache memory by 7.9 and speeds up decoding by 1.52. |
On Parsing as Tagging (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to reduce constituency parsing to tagging are based on linearization, learning, and decoding . linearization of the derivation tree is the most critical factor in achieving accurate parsers as taggers . |
| Approach: | They propose a pipeline with three steps for reducing constituency parsing to tagging . they find that linearization and learning are critical factors for accurate parsers . |
| Outcome: | The proposed pipelines are linearized, learning, and decoded, and have three steps to achieve accurate parsing as taggers. |
Learning Rhyming Constraints using Structured Adversaries (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to text generation fail to capture higher-level structure in text, for example, rhyming patterns. |
| Approach: | They propose a method that uses a structured discriminator to learn rhyming constraints from poetry . the discriminator compares two English poetry datasets based on a learned similarity matrix . |
| Outcome: | The proposed method can learn rhyming patterns in English poetry without explicit phonetic information. |
Nudging: Inference-time Alignment of LLMs via Guided Decoding (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) require alignment to effectively and safely follow user instructions. |
| Approach: | They propose a simple, training-free algorithm that aligns any base model at inference time using a small aligned model. |
| Outcome: | The proposed algorithm outperforms large aligned models on open-instruction tasks without training. |
HINT: Hypernetwork Instruction Tuning for Efficient Zero- and Few-Shot Generalisation (2023.acl-long)
Copied to clipboard
| Challenge: | Recent NLP models have shown the remarkable ability to generalise ‘zero-shot’ to new tasks using only natural language instructions as guidance. |
| Approach: | They introduce Hypernetworks for INstruction Tuning (HINT) which converts task instructions and examples into parameter-efficient modules inserted into an underlying model using a pretrained text encoder. |
| Outcome: | The proposed models outperform strong state-of-the-art models by over 10% when controlling for compute. |
Mask-Predict: Parallel Decoding of Conditional Masked Language Models (D19-1)
Copied to clipboard
| Challenge: | a masked language model is used to train a model to predict subsets of mangled words . a parallel decoding algorithm can be used to generate translations in a constant number of iterations. |
| Approach: | They propose a model and a parallel decoding algorithm which train a machine to predict any subset of target words . they introduce conditional masked language models (CMLMs) which are trained with a mangled language model objective . |
| Outcome: | The proposed model improves state-of-the-art performance levels for non-autoregressive and parallel decoding models by over 4 BLEU on average. |
Compositional Mathematical Encoding for Math Word Problems (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing MWP encoders work in a unimodal setting and map problem description to latent representation, then for decoding. |
| Approach: | They propose a Compositional Math Word Problem Solver which maps problem description to latent representation and decodes it in an interactive way. |
| Outcome: | Extensive experiments show that the proposed model outperforms state-of-the-art models on public benchmarks. |
Adaptive Token Biaser: Knowledge Editing via Biasing Key Entities (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to update parametric knowledge of large language models (LLMs) are outdated and incontext editing (KE) is not effective due to the substantial cost associated with retraining. |
| Approach: | They propose a new decoding technique that enhances in-context editing (ICE) they propose to use parametric knowledge to update the models' knowledge . |
| Outcome: | The proposed technique improves ICE performance while incurring only half the latency. |
Speculative Safety-Aware Decoding (2025.emnlp-main)
Copied to clipboard
| Challenge: | Speculative Safety-Aware Decoding (SSD) equips large language models with desired safety property while accelerating inference. |
| Approach: | They propose a lightweight decoding-time approach that equips large models with the desired safety property while accelerating inference. |
| Outcome: | Experimental results show that a small language model has the desired safety property while accelerating inference. |
TrajGuard: Streaming Hidden-state Trajectory Detection for Decoding-time Jailbreak Defense (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing jailbreak defense paradigms rely on static detection of prompts, outputs, or internal states . hidden states in critical layers during decoding carry stronger and more stable risk signals . |
| Approach: | They propose a decoding-time defense framework that aggregates hidden-state trajectories via a sliding window to quantify risk in real time. |
| Outcome: | The proposed framework achieves an average defense rate of 95% in 12 jailbreak attacks and open-source LLMs. |
Well Begun is Half Done: Low-resource Preference Alignment by Weak-to-Strong Decoding (2025.findings-acl)
Copied to clipboard
| Challenge: | Low-resource methods for LLM alignment have been popular, but still face challenges in obtaining high-quality and aligned content. |
| Approach: | They propose a framework to enhance alignment ability of base models by the guidance of a small aligned model. |
| Outcome: | The proposed framework outperforms baseline methods while avoiding degradation on downstream tasks. |
PipeSpec: Breaking Stage Dependencies in Hierarchical LLM Decoding (2025.findings-acl)
Copied to clipboard
| Challenge: | Speculative decoding is limited by sequential stage dependencies that prevent full hardware utilization. |
| Approach: | They propose a framework that generalizes speculative decoding to use multiple models arranged in a hierarchical pipeline and enables asynchronous execution with lightweight coordination for prediction verification and rollback. |
| Outcome: | The proposed framework achieves 2.25 tokens/unit through pipelined parallelism with multiple models arranged in a hierarchical pipeline. |
HSCR: Hierarchical Self-Contrastive Rewarding for Aligning Medical Vision Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for medical vision-language models overlook modality misalignment . HSCR generates high-quality preference data with higher sampling probability . |
| Approach: | They propose a hierarchical self-contrastive reward approach that addresses two challenges in alignment . they leverage the inherent capability of Med-VLMs to generate dispreferred responses . |
| Outcome: | The proposed approach improves accuracy and trustworthiness of medical vision-label models with 2,000 training entries. |
Learning to Decode Collaboratively with Multiple Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Using a latent variable model, multiple large language models can be trained to collaborate at the token level. |
| Approach: | They propose a method to teach multiple large language models to collaborate by interleaving their generations at the token level. |
| Outcome: | The proposed method improves on instruction-following, domain-specific QA, and reasoning tasks and shows that the model trained with the method exhibits several interesting collaboration patterns. |
ClusterAttn: KV Cache Compression under Intrinsic Attention Clustering (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for sparse attention apply the same pattern across different attention heads and inputs, but fail to capture the intrinsic attention clustering in large language models. |
| Approach: | They propose a training-free sparse attention method that provides an efficient prompt cache compression scheme under intrinsic attention clustering for efficient LLM inference. |
| Outcome: | The proposed method reduces memory usage by 10%–65% and increases throughput by 2.6–4.8 times with no accuracy loss. |
UniCoRN: Unified Cognitive Signal ReconstructioN bridging cognitive signals and human language (2023.acl-long)
Copied to clipboard
| Challenge: | Existing studies focus on decoding word-level fMRI volumes from a restricted vocabulary. |
| Approach: | They propose an open-vocabulary task to bridge fMRI time series and human language . they use a pre-trained language model to construct a robust encoder for cognitive signals . |
| Outcome: | The proposed task bridges fMRI time series and human language with a baseline model. |
Ouroboros: Generating Longer Drafts Phrase by Phrase for Faster Speculative Decoding (2024.emnlp-main)
Copied to clipboard
Weilin Zhao, Yuxiang Huang, Xu Han, Wang Xu, Chaojun Xiao, Xinrong Zhang, Yewei Fang, Kaihuo Zhang, Zhiyuan Liu, Maosong Sun
| Challenge: | Speculative decoding is a widely used method that accelerates the generation process of large language models (LLMs) drafting efficiency has become a bottleneck in the final speedup of speculative drafting, therefore generating longer drafts at less cost can lead to better speedup. |
| Approach: | They propose a method that uses existing model to drafting and target LLM to verify draft in a low-cost parallel manner. |
| Outcome: | The proposed method can achieve speedups of up to 2.4 over speculative decoding and 3.9 over vanilla decoding without fine-tuning draft and target models. |
Hybrid of Spans and Table-Filling for Aspect-Level Sentiment Triplet Extraction (2024.lrec-main)
Copied to clipboard
| Challenge: | Aspect Sentiment Triplet Extraction (ASTE) is an emerging task in sentiment analysis research. |
| Approach: | They propose a model which combines span with table-filling to extract triplets from words . they use syntactic and contextual features to generate word-pair tables and convert them to span tables . |
| Outcome: | The proposed model achieves competitive results on a dataset with a large dataset. |
Efficient Beam Search for Large Language Models Using Trie-Based Decoding (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) face memorybound performance bottlenecks due to their high memory requirements. |
| Approach: | They propose a trie-based parallel decoding method that shares a single KV cache across beams with common prefixes to dramatically reduce memory usage and enables efficient decoding. |
| Outcome: | The proposed method significantly reduces memory usage and enables efficient decoding without compromising generation quality. |
Bridging Semantic and Modality Gaps in Zero-Shot Captioning via Retrieval from Synthetic Data (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for zero-shot image captioning fail to bridge the semantic gap between images and textual inputs. |
| Approach: | They propose a retrieval-based framework that leverages only existing synthetic image-text pairs as its search corpus to bridge the gap when using synthetic data for captioning. |
| Outcome: | The proposed method bridges the semantic gap between a synthetic image and its input text . it extracts image-related textual descriptions to mitigate the modality gap during decoding . |
FLRC: Fine-grained Low-Rank Compressor for Efficient LLM Inference (2025.emnlp-main)
Copied to clipboard
| Challenge: | Low-rank compression can reduce memory usage and computational demand, but results are poor during decoding. |
| Approach: | They propose a fine-grained low-rank compression algorithm that determines optimal rank allocation for each layer and incorporates progressive low-ranked decoding to maintain text generation quality. |
| Outcome: | The proposed approach outperforms state-of-the-art methods on summarization tasks and on understanding tasks. |
Temperature-Centric Investigation of Speculative Decoding with Knowledge Distillation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Speculative decoding is a novel method to expedite inference in autoregressive (large) language models. |
| Approach: | They propose to use a smaller model as a draft model to speculate a block of tokens, which the target model then evaluates for acceptance. |
| Outcome: | The proposed method can be used to accelerate inference in autoregressive (large) language models by using smaller models as draft models to speculate tokens for multiple inference steps. |
Revisiting Higher-Order Dependency Parsers (2020.acl-main)
Copied to clipboard
| Challenge: | Neural encoders have allowed dependency parsers to shift from higher-order structured models to simpler first-order ones, making decoding faster and still achieving better accuracy than non-neural parser. |
| Approach: | They found that neural parsers may benefit from higher-order features when employing a powerful pre-trained encoder, such as BERT. |
| Outcome: | Using a pre-trained encoder, we found that higher-order models are more accurate on full sentence parses and match of modifier lists. |
Information Extraction with Differentiable Beam Search on Graph RNNs (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing approaches to information extraction suffer from exposure bias due to discrepancy between training and decoding. |
| Approach: | They propose to cast graph generation as auto-regressive sequence labeling and make it aware of decoding procedure by using differentiable beam search. |
| Outcome: | The proposed model outperforms its non-decoding-aware version on ACE05 and ConLL04 datasets. |
Accelerating Prefilling via Decoding-time Contribution Sparsity (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing acceleration methods exploit attention score sparsity by estimating blocks with high attention scores and applying dynamic sparse attention. |
| Approach: | They propose a method which replaces dense attention with Triangle attention in a subset of layers to reduce the time needed to decode. |
| Outcome: | Experiments show that TriangleMix achieves near-lossless performance on long-context and long-constrast reasoning benchmarks while significantly improving efficiency. |
On Search Strategies for Document-Level Neural Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Document-level neural machine translation models produce a more consistent output across a document . however, the exact decoding strategy is often not described and not mentioned at all. |
| Approach: | They propose to use standard automatic metrics and specific linguistic phenomena to compare different decoding schemes. |
| Outcome: | The proposed decoding strategies perform similar to each other on three standard document-level translation benchmarks. |
QaRL: Rollout-Aligned Quantization-Aware RL for Fast and Stable Training under Training–Inference Mismatch (2026.findings-acl)
Copied to clipboard
Hao Gu, Hao Wang, Jiacheng Liu, Lujun Li, Qiyuan Zhu, Bei Liu, Binxing Xu, Lei Wang, Xintong Yang, Sida Lin, Sirui Han, Yike Guo
| Challenge: | Recent work has shown that reinforcement learning with simple rule-based reward functions (RLVR) can induce emergent reasoning behaviors and yield gains in challenging domains such as math problem solving. |
| Approach: | They propose a rollout-alignment-quantization-aware RL which aligns training-side forward with the quantized rollout to minimize mismatch. |
| Outcome: | The proposed approach outperforms quantized-rollout training by +5.5 on Qwen3-30B-A3B MoE for math problems while maintaining low-bit throughput. |
Adaptive Contrastive Search: Uncertainty-Guided Decoding for Open-Ended Text Generation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to decode text to the most probable sequence have been proposed to address these challenges by improving coherence, diversity, and resemblance to human-generated text. |
| Approach: | They propose a novel decoding strategy that extends contrastive search by incorporating an adaptive degeneration penalty informed by the model’s estimated uncertainty at each generation step. |
| Outcome: | The proposed approach improves creativity and coherence while maintaining coherency across model architectures, languages, and datasets. |
Self-Ensemble of N-best Generation Hypotheses by Lexically Constrained Decoding (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have improved generation quality by explicitly reranking N-best candidates. |
| Approach: | They propose a method that ensembles N-best hypotheses to improve natural language generation by combining high-quality fragments of N- best hypothese . they use tokens that should or should not be present in the final output as lexical constraints to improve quality of generation. |
| Outcome: | Empirical results show that the proposed method outperforms strong N-best reranking methods on paraphrase generation, summarisation, and constrained text generation. |
RetentiveKV: State-Space Memory for Uncertainty-Aware Multimodal KV Cache Eviction (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for evicting KV pairs rely on the "persistence of importance" hypothesis . visual tokens display "deferred importance" but become pivotal during later decoding, authors say . |
| Approach: | They propose an entropy-driven method that reformulates KV eviction from "discrete context truncation" to "continuous memory evolution" they propose to prune visual tokens with "deferred importance" visual token exhibiting low salience but becoming pivotal during later decoding . |
| Outcome: | The proposed method achieves 5.0 KV cache compression and 1.5 decoding acceleration. |
KVPR: Efficient LLM Inference with I/O-Aware KV Cache Partial Recomputation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to inference large language models are limited by CPU capabilities and memory constraints. |
| Approach: | They propose an efficient I/O-aware LLM inference method that overlaps GPU computation with KV cache transfer to minimize idle GPU time. |
| Outcome: | The proposed method reduces the cost of auto-regressive decoding by 35.8% . it also achieves 46.2% higher throughput during decoding compared to state-of-the-art methods. |
GRACE: Discriminator-Guided Chain-of-Thought Reasoning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing language models (LMs) can assign a high likelihood to incorrect steps . Existing models (LLMs), however, struggle with complex multi-step reasoning. |
| Approach: | They propose a stepwise decoding approach that steers the decoding process towards producing correct reasoning steps. |
| Outcome: | The proposed approach outperforms existing methods on math and symbolic reasoning tasks. |
A Simple and Effective L_2 Norm-Based Strategy for KV Cache Compression (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to reduce the KV cache size involve fine-tuning the model to learn a compression strategy or leveraging attention scores to reduce sequence length. |
| Approach: | They find a correlation between the L2 norm and attention scores over cached KV pairs . they compress the KV cache based on the L1 norm of key embeddings . |
| Outcome: | The proposed approach reduces the KV cache size by 50% on language modelling and needle-in-a-haystack tasks and 90% on passkey retrieval tasks without losing accuracy. |
DocCGen: Document-based Controlled Code Generation (2024.emnlp-main)
Copied to clipboard
Sameer Pimparkhede, Mehant Kammakomati, Srikanth Tamilselvam, Prince Kumar, Ashok Kumar, Pushpak Bhattacharyya
| Challenge: | Large language models (LLMs) produce state-of-the-art performance on natural language to code generation for resource-rich general-purpose languages like C++, Java, and Python. |
| Approach: | They propose a framework that breaks the NL-to-Code generation task into two steps . they use library documentation to detect the correct libraries and schema rules extracted from the documentation to constrain the decoding . |
| Outcome: | The proposed framework improves different sized language models across all six evaluation metrics, reducing syntactic and semantic errors in structured code. |
Learning Semantic Role Labeling from Compatible Label Sequences (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work has shown that cross-task interaction helps, but only explored multitask learning so far. |
| Approach: | They propose a framework that jointly models VerbNet and PropBank labels as one sequence and enforcing Semlink constraints during decoding improves the overall F1 . |
| Outcome: | The proposed model outperforms the prior best in-domain model by 3.5 (VerbNet) and 0.8 (PropBank). |
TailorKV: A Hybrid Framework for Long-Context Inference via Tailored KV Cache Optimization (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing work mitigates memory overhead by offloading or compressing the Key-Value cache. |
| Approach: | They propose a method that integrates quantization and offloading into a generative large language model by using a hybrid compression method. |
| Outcome: | The proposed method outperforms the state-of-the-art in long-context evaluations. |
Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospects (2026.findings-acl)
Copied to clipboard
Jun Zhang, Yicheng Ji, Feiyang Ren, Yihang Li, Bowen Zeng, Zonghao Chen, Ke Chen, Lidan Shou, Gang Chen, Huan Li
| Challenge: | Large Vision-Language Models are hindered by a systemic efficiency barrier known as visual token dominance. |
| Approach: | They propose a systematic taxonomy of efficiency techniques structured around the inference lifecycle . they examine visual encoding, prefilling, and decoding to understand bottlenecks . |
| Outcome: | The proposed techniques reveal how upstream decisions dictate downstream bottlenecks . the proposed techniques include hybrid compression and modality-aware decoding . |
Opinions Are Not Always Positive: Debiasing Opinion Summarization with Model-Specific and Model-Agnostic Methods (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing opinion summarization frameworks are reluctant to generate negative summaries given input of negative opinions. |
| Approach: | They propose to disentangle input into sentiment-relevant and sentiment-irrelevant components through adversarial loss. |
| Outcome: | The proposed approaches reduce sentiment bias in the existing opinion summarization dataset . the proposed approaches generate better summaries with a more balanced emotional polarity distribution . |
Why These Documents? Explainable Generative Retrieval with Hierarchical Category Paths (2026.findings-acl)
Copied to clipboard
| Challenge: | Generative retrieval directly decodes a document identifier, making it impossible to provide explanations for its retrieval decision. |
| Approach: | They propose a hierarchical category path-Enhanced Generative Retrieval that generates category paths step-by-step and decodes docid. |
| Outcome: | The proposed method provides explanations for retrieval decision by generating hierarchical category paths step-by-step and decoding docid. |
Looking Beyond the One: Operationalizing and Eliciting Visual Ambiguity in VLLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Visual question answering systems typically collapse ambiguity, committing to a single interpretation during decoding and evaluation. |
| Approach: | They operationalize ambiguity as the existence of multiple answer-supporting regions in an image . they show that ambiguities are already encoded in their internal representations . |
| Outcome: | The proposed approach makes ambiguity observable without exhaustive annotations . ambiguities are already encoded in models, but not reliably expressed in outputs despite hidden states . |
CoViPAL: Layer-wise Contextualized Visual Token Pruning for Large Vision-Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to prune redundant vision tokens struggle in shallow layers due to the lack of contextual information. |
| Approach: | They propose a layer-wise contextualized visual token pruning method that uses a plug-and-play Pruning Module to prune redundant vision tokens. |
| Outcome: | The proposed method outperforms training-free pruning methods under equal token budgets and surpasses training based methods with comparable supervision. |
FiRST: Finetuning Router-Selective Transformers for Input-Adaptive Latency Reduction (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to improve latency via skipping layers have limitations . fiRST is a model-agnostic framework that reduces inference latency while maintaining quality . |
| Approach: | They propose a model-agnostic framework that skips transformer layers during decoding . it is fully compatible with KV caching, enabling faster decoding while maintaining quality . |
| Outcome: | a new framework reduces inference latency by using layer-specific routers to skip transformer layers during decoding. |
Local Normalization Distortion and the Thermodynamic Formalism of Decoding Strategies for Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing decoding strategies for language models are based on heuristics and are difficult to improve. |
| Approach: | They propose a theory of decoding strategies for language models by expressing popular decoding algorithms as equilibrium states and stating objective functions they optimize. |
| Outcome: | The proposed method is based on ergodic theory and is difficult to improve . it shows that local normalization distortion is a fundamental defect of decoding strategies . |
Estimating Privacy Leakage of Augmented Contextual Knowledge in Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Prior work has shown that privacy leakage of parametric knowledge often occurs from memorized pre-training data. |
| Approach: | They propose a metric that builds on differential privacy to estimate the privacy leakage of contextual knowledge during decoding by comparing parametric and contextual knowledge. |
| Outcome: | The proposed method overestimates the privacy leakage of parametric knowledge while separating parametric and contextual knowledge. |
Watching the Watchers: Exposing Gender Disparities in Machine Translation Quality Estimation (2025.acl-long)
Copied to clipboard
| Challenge: | Qualitative estimation (QE) metrics have been optimized to align with human quality judgments, but whether they encode social biases has been largely overlooked. |
| Approach: | They define and investigate gender bias of QE metrics and discuss its downstream implications for machine translation (MT) when a human entity’s gender in the source is undisclosed, masculine-inflected translations score higher than feminine-infflectes translations are penalized. |
| Outcome: | The proposed measures are based on gender-based quality estimation metrics across multiple domains, datasets, and languages. |
Latent Attention Denoising: A Training-Free Energy-Based Framework for Mitigating Hallucinations in Vision-Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing models for large vision-Language Models lack reliable visual hallucination . despite advances in multimodal perception, they are still limited in real-world applications . |
| Approach: | They propose a framework that recasts attention calibration as a one-step score-based denoising process. |
| Outcome: | The proposed framework achieves superior performance on generative and discriminative tasks while maintaining efficiency comparable to standard decoding. |
Less Languages, Less Tokens: An Efficient Unified Logic Cross-lingual Chain-of-Thought Reasoning Framework (2026.acl-long)
Copied to clipboard
Chenyuan Zhang, Qiguang Chen, Xie Chen, Zhuotao Tian, Bowen Xing, Meishan Zhang, Libo Qin, Baotian Hu, Min Zhang
| Challenge: | Existing methods for cross-lingual chain-of-thought (XCoT) with self-consistency are costly due to extensive sampling of full trajectories across languages. |
| Approach: | They propose a cross-lingual chain-of-thought framework that minimizes redundancy in token usage and latency. |
| Outcome: | Experiments on polymath show that UL-XCoT reduces decoding token costs and latency by 50% . UL XCot also aggregates remaining high-quality reasoning paths via voting . |
MM-ShiftKV: Decode-Aware Prefill-Stage KV Selection for Multimodal Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent work suggests a prefill-stage KV cache selection method to estimate KV importance from prefilling statistics. |
| Approach: | They propose a training-free, decode-aware and strictly prefill-only KV selection method that retains key-value caching for decoding . |
| Outcome: | The proposed method outperforms existing methods under tight cache budgets on multimodal benchmarks. |
ReasoningGuard: Safeguarding Large Reasoning Models with Inference-time Safety Aha Moments (2026.acl-long)
Copied to clipboard
| Challenge: | Existing defenses for Large Reasoning Models (LRMs) depend on costly fine-tuning and additional expert knowledge, which limits their scalability. |
| Approach: | They propose an inference-time safeguard for Large Reasoning Models that injects safety aha moments into the reasoning process to guide the model towards harmless yet helpful reasoning. |
| Outcome: | The proposed safeguard outperforms nine existing safeguards while avoiding common exaggerated safety issues. |
Which Sense Dominates Multisensory Semantic Understanding? A Brain Decoding Study (2024.lrec-main)
Copied to clipboard
| Challenge: | Decoding semantic meanings from brain activity is open to multisensory stimulation, as word meanings can be delivered by both auditory and visual inputs. |
| Approach: | They aim to develop a computational model to probing what information from the act of language understanding is represented in human brain. |
| Outcome: | The proposed model dissociates multisensory integration of word understanding into written text, spoken text and image perception respectively, exploring the decoding efficiency and reliability of unisensory information in the brain representation. |
Word-level Commonsense Knowledge Selection for Event Detection (2024.lrec-main)
Copied to clipboard
| Challenge: | Event Detection (ED) is a task of automatically extracting multi-class trigger words . Xie and Tu, 2022, use a Context-specific Knowledge Selector to select commonsense knowledge of words based on living contexts . |
| Approach: | They use a Context-specific Knowledge Selector to select the exact commonsense knowledge of words from a large knowledge base. |
| Outcome: | The proposed approach achieves the F1-score of about 78.3% on the ACE-2005 dataset. |
LeanK: Learnable K Cache Channel Pruning for Efficient Decoding (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing efforts to optimize the key-value (KV) cache include: (1) Eviction, which discards cache of less important tokens; (2) Selection, which retains the full KV cache but selectively reads relevant entries. |
| Approach: | They propose a learning-based method that prunes unimportant key (K) cache channels by leveraging static channel sparsity. |
| Outcome: | Experiments show that LeanK reduces GPU memory and accelerates decoding without sacrificing accuracy. |
FastKV: Decoupling of Context Reduction and KV Cache Compression for Prefill-Decoding Acceleration (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) excel at handling long-context sequences, but require substantial prefill computation and key-value (KV) cache. |
| Approach: | They propose a KV cache compression framework that decouples prefill computation from decoding KV budget. |
| Outcome: | The proposed framework reduces latency in prefill and decoding by leveraging the stabilization of token importance in later layers. |
Parallel Context-of-Experts Decoding for Retrieval Augmented Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Retrieval Augmented Generation relies on concatenating documents into a long context prompt, causing prefill bottlenecks. |
| Approach: | They propose a training-free framework that shifts evidence aggregation from attention to decoding . they treat retrieved documents as isolated "experts", synchronizing their predictions via a retrieval-aware extension of context-awful decoding. |
| Outcome: | The proposed framework shifts evidence aggregation from attention to decoding . it treats retrieved documents as isolated experts, synchronizing their predictions . |
The Impact of Language Mixing on Bilingual LLM Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies show multilingual speakers intentionally switch languages during reasoning . enforcing monolingual decoding reduces accuracy by 5.6 percentage points . |
| Approach: | They find that multilingual speakers intentionally switch languages during reasoning . enforcing monolingual decoding reduces accuracy by 5.6 percentage points . authors suggest that language mixing is not merely a byproduct of multilingual training . |
| Outcome: | The proposed model can be used to predict whether a language switch would benefit or harm reasoning. |
Lookahead Q-Cache: Achieving More Consistent KV Cache Eviction via Pseudo Query (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing KV cache eviction methods prune tokens using prefilling-stage attention scores, causing inconsistency with actual inference queries. |
| Approach: | They propose a lookahead q-cache framework that generates low-cost pseudo lookaheaded queries to better approximate the true decoding-stage queries. |
| Outcome: | The proposed framework outperforms existing methods on LongBench and Needle-in-a-Haystack benchmarks and can be flexibly combined to yield further improvements. |
The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Sparse attention is a promising strategy to extend long-context capabilities in LLMs . but its efficiency–accuracy trade-offs remain unclear due to the lack of comprehensive evaluation . |
| Approach: | They evaluate sparse attention methods across multiple model families and sizes . they find larger sparser models outperform smaller dense ones at equivalent cost . |
| Outcome: | The proposed methods outperform smaller sparse models at equivalent cost and improve the Pareto frontier. |
HeteroCache: A Dynamic Retrieval Approach to Heterogeneous KV Cache Compression for Long-Context LLM Inference (2026.acl-long)
Copied to clipboard
| Challenge: | Existing static compression methods suffer from coarse-grained caching and high I/O overhead. |
| Approach: | They propose a training-free dynamic compression framework that uses a sparse attention mechanism to categorize attention heads based on stability and similarity. |
| Outcome: | The proposed framework achieves state-of-the-art performance on long-context benchmarks and accelerates decoding by up to 3 compared to the original model with a 224K context. |
Through the Magnifying Glass: Adaptive Perception Magnification for Hallucination-Free VLM Decoding (2026.acl-long)
Copied to clipboard
| Challenge: | Existing vision-language models suffer from visual hallucination, where the generated responses contain inaccuracies that are not grounded in the visual input. |
| Approach: | They propose a visual decoding method that iteratively isolates relevant visual tokens based on attention and magnifies the corresponding regions. |
| Outcome: | The proposed method reduces language biases and amplifies weights of visual embedding during decoding, while still preserving strong reasoning capabilities. |
EDSD: Entropy-Driven Design for Faster Speculative Decoding (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for speculative decoding incur substantial training overhead to mitigate information misalignment between autoregressive draft model training and decoding. |
| Approach: | They propose an Entropy-Driven Speculative Decoding framework that uses entropy as a unified, interpretable signal for both draft model training and architectural design. |
| Outcome: | Experiments on seven large language models show that EDSD improves training efficiency by 24.8% and increases acceptance length by 4.0% compared to state-of-the-art methods. |
DREAM-S: Speculative Decoding with Searchable Drafting and Target-Aware Refinement for Multimodal Generation (2026.acl-long)
Copied to clipboard
| Challenge: | Speculative decoding (SD) has proven to be effective for autoregressive generation in large language models (LLMs), however its application to vision-language models (VLMs) remains relatively unexplored. |
| Approach: | They propose a Speculative Decoding framework for vision-language models that integrates a neural architecture search framework and target-aware supernet training to identify optimal interaction strategies. |
| Outcome: | DREAM-S achieves 3.85 speedup compared to baselines on well-established vision-language models. |
Preference Heads in Large Language Models: A Mechanistic Framework for Interpretable Personalization (2026.acl-long)
Copied to clipboard
Weixu Zhang, Ye Yuan, Changjiang Han, Yuxing Tian, Zipeng Sun, Linfeng Du, Jikun Kang, Hong Kang, Xue Liu, Haolun Wu
| Challenge: | Large Language Models exhibit strong implicit personalization ability, but most approaches treat this behavior as a black box. |
| Approach: | They propose a mechanistic interpretation perspective and propose 'sparse' set of Preference Heads . they compute a Preference Contribution Score for each attention head and compare their predictions . |
| Outcome: | The proposed framework computes a Preference Contribution Score (PCS) for each attention head and measures its causal impact on user aligned outputs. |