Papers with training
Neural Network based Extreme Classification and Similarity Models for Product Matching (N18-3)
Copied to clipboard
| Challenge: | Matching a seller listed item to an appropriate product has become a fundamental step for e-commerce platforms. |
| Approach: | They propose to use a shallow neural network to match a seller's item to an appropriate product . they also propose a similarity approach based on deep siamese network to train and infer product information. |
| Outcome: | The proposed models outperform the baseline models by more than 5% in terms of accuracy and are capable of efficient training and inference. |
A Hybrid Approach to Cross-lingual Product Review Summarization (2022.emnlp-industry)
Copied to clipboard
| Challenge: | Existing methods for summarizing product reviews with thousands of reviews are inefficient and time consuming. |
| Approach: | They propose an unsupervised extractive step and a supervised abstractive step to generate a short summary in any language. |
| Outcome: | The proposed model is as good as human written summaries in coherence, informativeness, non-redundancy, and fluency as human summary summators. |
Towards the TopMost: A Topic Modeling System Toolkit (2024.acl-demos)
Copied to clipboard
| Challenge: | Current topic models adopt totally different datasets, implementations, and evaluations, hindering their research progress and applications. |
| Approach: | They propose a Topic Modeling System Toolkit that covers a broader spectrum of topic modeling scenarios with their complete lifecycles. |
| Outcome: | The proposed toolkit covers a broader spectrum of topic modeling scenarios with their complete lifecycles, including datasets, preprocessing, models, training, and evaluations. |
Learning to request guidance in emergent language (D19-64)
Copied to clipboard
Benjamin Kolb, Leon Lang, Henning Bartsch, Arwin Gansekoele, Raymond Koopmanschap, Leonardo Romor, David Speck, Mathijs Mul, Elia Bruni
| Challenge: | Previous research into agent communication has shown that a pre-trained guide can speed up the learning process of an imitation learning agent. |
| Approach: | They extend one-directional communication by a one-bit communication channel from the learner back to the guide and limit the guidance by penalizing the learners for these requests. |
| Outcome: | The proposed guide can speed up the learning process of an imitation learning agent by providing the learner with discrete messages in an emerged language about how to solve the task. |
Multi-Task, Multi-Channel, Multi-Input Learning for Mental Illness Detection using Social Media Text (D19-62)
Copied to clipboard
| Challenge: | Existing methods for mental illness detection have limited data available for training . lack of sufficient annotated data and inability to extract explanations on the derived outcome have restricted researchers to use traditional methods. |
| Approach: | They propose to use emotional patterns identified by clinical practitioners to enhance the prediction capabilities of a mental illness detection model built using a deep neural network architecture. |
| Outcome: | The proposed method achieves a task-specific AUC higher than 0.90 . it compares multi-task learning with multi-channel convolutional neural network and multiple inputs to methods such as multi-class classification . |
dLLM: Simple Diffusion Language Modeling (2026.acl-demo)
Copied to clipboard
| Challenge: | diffusion language models (DLMs) are evolving rapidly but many lack transparent implementations or are scattered across codebases. |
| Approach: | They propose an open-source framework that unifies diffusion language modeling components while remaining flexible enough to support new methods and architectures. |
| Outcome: | dLLM unifies the core components of diffusion language modeling and makes them easy to customize for new designs. |
EMCompress: Video-LLMs with Endomorphic Multimodal Compression (2026.findings-acl)
Copied to clipboard
| Challenge: | Static, sparse frame sampling either dilutes evidence across task-irrelevant segments at significant cost or misses fine-grained temporal semantics altogether. |
| Approach: | They propose a novel task that compresses multimodal input while preserving answer invariance across reasonable downstream models. |
| Outcome: | The proposed task surpasses prior methods by 0.40 F-1 with competitive query rewriting. |
Controllable Neural Dialogue Summarization with Personal Named Entity Planning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that our proposed framework generates fluent and factually consistent summaries under various planning controls using both objective metrics and human evaluations. |
| Approach: | They propose a controllable neural generation framework that can guide dialogue summarization with personal named entity planning. |
| Outcome: | The proposed framework generates fluent and factually consistent summaries under various planning controls using objective metrics and human evaluations. |
Automatic Machine Translation Evaluation in Many Languages via Zero-Shot Paraphrasing (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing metrics for machine translation evaluation are causing the correlation between human judgments and automatic metrics to break down. |
| Approach: | They propose to train a multilingual NMT system to score machine translation output with a sequence-to-sequence paraphraser, conditioned on a human reference. |
| Outcome: | The proposed model outperforms or statistically ties with all prior metrics on the WMT 2019 segment-level shared metrics task in all languages (excluding Gujarati where the model had no training data). |
Teach the Rules, Provide the Facts: Targeted Relational-knowledge Enhancement for Textual Inference (2021.starsem-1)
Copied to clipboard
| Challenge: | InferBERT is a method to enhance transformer-based inference models with relevant relational knowledge. |
| Approach: | They propose to enhance transformer-based inference models with relevant relational knowledge by injecting relevant facts at test time into the model. |
| Outcome: | The proposed method outperforms existing models on the challenge datasets while outperforming existing models. |
On the Importance of the Kullback-Leibler Divergence Term in Variational Autoencoders for Text Generation (D19-56)
Copied to clipboard
| Challenge: | Variational Autoencoders suffer from learning uninformative latent representations due to issues such as approximated posterior collapse or entanglement of the latent space. |
| Approach: | They propose to impose an explicit constraint on the Kullback-Leibler divergence term inside the VAE objective function to understand the significance of the KL term in controlling the information transmitted through the VAe channel. |
| Outcome: | The proposed constraint avoids posterior collapse, but it also controls the information transmitted through the VAE channel. |
Coreference Resolution through a seq2seq Transition-Based System (2023.tacl-1)
Copied to clipboard
| Challenge: | Recent coreference resolution systems use search algorithms to identify mentions and resolve coreference. |
| Approach: | They propose a text-to-text coreference resolution system that uses a semantic paradigm to predict mentions and links jointly. |
| Outcome: | The proposed system achieves state-of-the-art accuracy on CoNLL-2012 datasets with 83.3 F1-score for English, 68.5 F1 score for Arabic, and 74.3 F1 scores for Chinese. |
TaoType: Predicting Fine-Grained Typing Intent for Faster Search (2026.acl-industry)
Copied to clipboard
| Challenge: | Large-scale industrial ranking systems operate under stringent real-time performance requirements. |
| Approach: | They propose a client-side framework that determines whether a user’s query is complete at each typing . this method leverages client-based typing behavior for real-time early prediction . |
| Outcome: | The proposed framework achieves offline precision/recall/accuracy of 0.7936/0.8196/0.7742 and decreases online response time by 640.5193.65 milliseconds. |
Alexa Conversations: An Extensible Data-driven Approach for Building Task-oriented Dialogue Systems (2021.naacl-demos)
Copied to clipboard
Anish Acharya, Suranjit Adhikari, Sanchit Agarwal, Vincent Auvray, Nehal Belgamwar, Arijit Biswas, Shubhra Chandra, Tagyoung Chung, Maryam Fazel-Zarandi, Raefer Gabriel, Shuyang Gao, Rahul Goel, Dilek Hakkani-Tur, Jan Jezabek, Abhay Jha, Jiun-Yu Kao, Prakash Krishnan, Peter Ku, Anuj Goyal, Chien-Wei Lin, Qing Liu, Arindam Mandal, Angeliki Metallinou, Vishal Naik, Yi Pan, Shachi Paul, Vittorio Perera, Abhishek Sethi, Minmin Shen, Nikko Strom, Eddie Wang
| Challenge: | Traditional goal-oriented dialogue systems require annotations which are hard to obtain for every new domain, limiting scalability. |
| Approach: | They propose a data-driven approach to building goal-oriented dialogue systems . they use a seed dialogue simulator to generate annotated conversations instead of collecting annotations . |
| Outcome: | The proposed system improves turn-level action signature prediction accuracy by 50% . the system is scalable, extensible and data efficient . |
MixtureKit: A General Framework for Composing, Training, and Visualizing Mixture-of-Experts Models (2026.acl-demo)
Copied to clipboard
| Challenge: | MixtureKit is a modular open-source framework for constructing, training, and analyzing Mixture-of-Experts (MoE) models from arbitrary pre-trained or fine-tuned checkpoints. |
| Approach: | They propose a modular open-source framework for constructing, training, and analyzing Mixture-of-Experts (MoE) models from arbitrary pre-trained or fine-tuned checkpoints. |
| Outcome: | Experiments on multilingual code-switched (Arabic–Latin) show that BTX models built with MixtureKit outperform dense baselines across multiple benchmarks. |
Augmenting Training Data for Massive Semantic Matching Models in Low-Traffic E-commerce Stores (2022.naacl-industry)
Copied to clipboard
Ashutosh Joshi, Shankar Vishwanath, Choon Teo, Vaclav Petricek, Vishy Vishwanathan, Rahul Bhagat, Jonathan May
| Challenge: | Existing methods to augment training data for e-commerce stores using behavioral data are limited in low-traffic stores . eXtreme multi-label classification systems require large amounts of customer behavior data . |
| Approach: | They propose a technique that augments behavioral training data via query reformulation . they use an example semantic matching model from the e-commerce store AL-XMC . |
| Outcome: | The proposed method improves quality of the AL-XMC model over a baseline model. |
When Language Model Meets Private Library (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing language models have been pre-trained on large-scale code corpora and generate decent code snippets. |
| Approach: | They propose a framework that can provide pre-trained language models with the ability to generate code using private libraries. |
| Outcome: | The proposed framework can generate code using private libraries using off-the-shelf language models or pre-trained models on code corpus containing API information. |
A Strong and Robust Baseline for Text-Image Matching (P19-2)
Copied to clipboard
| Challenge: | Text-image matching is one of the most popular methods for training text-image embeddings. |
| Approach: | They propose to use a kNN-margin loss that utilizes hard negatives and is robust to noise . they advocate using Inverted Softmax and Cross-modal Local Scaling during inference . |
| Outcome: | The proposed loss function is robust to noise and pseudo negatives are tolerable . the proposed loss functions improve scores of all metrics by a large margin . |
Auto-Sizing the Transformer Network: Improving Speed, Efficiency, and Performance for Low-Resource Machine Translation (D19-56)
Copied to clipboard
| Challenge: | Neural sequence-to-sequence models are sensitive to architecture and hyperparameter settings. |
| Approach: | They incorporate architecture search into a single training run through auto-sizing . they show that auto-size can improve BLEU scores by up to 3.9 points . |
| Outcome: | The proposed algorithm improves BLEU scores on low-resource language pairs while removing one-third of the parameters from the model. |
COGEN: Abductive Commonsense Language Generation (2023.acl-short)
Copied to clipboard
| Challenge: | Existing training methods for NLP models to perform on two main tasks are needed to introduce these capabilities into the field of reasoning. |
| Approach: | They propose a model that integrates commonsense reasoning with contextual filtering to improve the inference. |
| Outcome: | The proposed model outperforms existing models and sets new state-of-the-art in regards to alphaNLI and alphaNGG tasks. |
Preserving Multilingual Quality While Tuning Query Encoder on English Only (2025.naacl-short)
Copied to clipboard
| Challenge: | Xiong et al., 2020, 2021b) and Dong eet., 2022) . Switching from one query encoder to another is easily feasible . |
| Approach: | They propose a general tuning technique that can be used to modify query representations for specific types of queries or domains while keeping precomputed and stored documents intact. |
| Outcome: | The proposed model preserves multilingual qualities and improves embedding qualities on different datasets. |
SpeechNet: Weakly Supervised, End-to-End Speech Recognition at Industrial Scale (2022.emnlp-industry)
Copied to clipboard
Raphael Tang, Karun Kumar, Gefei Yang, Akshat Pandey, Yajie Mao, Vladislav Belyaev, Madhuri Emmadi, Craig Murray, Ferhan Ture, Jimmy Lin
| Challenge: | End-to-end automatic speech recognition systems require thousands of hours of manual annotation and heavyweight computation to perform inference. |
| Approach: | They propose to use a third-party ASR system as a weak supervision source and labeling functions derived from implicit user feedback to reduce human labor. |
| Outcome: | The proposed system improves word-error rate and speed up 600% over third-party ASR. |
Unsupervised KB-to-Text Generation with Auxiliary Triple Extraction using Dual Learning (2020.aacl-main)
Copied to clipboard
| Challenge: | Existing methods to generate text from KB triples are limited and expensive . a novel approach is proposed to train the generation model in unsupervised way . |
| Approach: | They propose a method which trains the generation model in a completely unsupervised way with unaligned raw text data and KB triples. |
| Outcome: | The proposed method outperforms existing methods and is cost-effective. |
AutoQA: From Databases To QA Semantic Parsers With Only Synthetic Training Data (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to generate semantic parsers that answer questions on databases require large amounts of annotated data. |
| Approach: | They propose a method to generate semantic parsers that answer questions on databases . they use automatic paraphrasing and template-based parsing to find alternative expressions . |
| Outcome: | The proposed method achieves 69.8% answer accuracy on natural questions, 16.4% higher than state-of-the-art models and 5.2% lower than the same model trained with human data. |
ESPnet-ST: All-in-One Speech Translation Toolkit (2020.acl-demos)
Copied to clipboard
Hirofumi Inaguma, Shun Kiyono, Kevin Duh, Shigeki Karita, Nelson Yalta, Tomoki Hayashi, Shinji Watanabe
| Challenge: | ESPnet-ST is a new project for the quick development of speech-to-speech translation systems. |
| Approach: | They propose a framework for rapid development of speech-to-speech translation systems . they provide all-in-one recipes including data pre-processing, feature extraction, training, and decoding pipelines . |
| Outcome: | The proposed model outperforms the current state-of-the-art models on a wide range of benchmark datasets. |
Enhancing User Safety: Context-Aware Detection of Offensive Query-Ad Pairs in Multimodal Search Advertising (2026.eacl-industry)
Copied to clipboard
Gaurav Kumar, Qiangjian Xi, Tanmaya Shekhar Dabral, Hooshang Ghasemi, Abishek Krishnamoorthy, Danqing Fu, Rui Min, Emilio Antunez, Zhongli Ding, Pradyumna Narayana
| Challenge: | Multi-modal online advertisements require robust content moderation to ensure user safety . key challenges include nuanced, multi-modal nature of ads, severe data scarcity and class imbalance due to the rarity of offensive content . |
| Approach: | They propose a framework that detects offensive content only when a user's search query is paired with a specific ad . |
| Outcome: | The proposed framework reduces the serving of offensive query-ad pairs by more than 80% while maintaining the efficiency required for real-time advertising systems. |
Hierarchical Curriculum Learning for AMR Parsing (2022.acl-short)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) parsing aims to translate sentences to semantic representation with a hierarchical structure, but there is a gap between their flat training objective and the hierarchic structure, which limits the model generalization. |
| Approach: | They propose a Hierarchical Curriculum Learning framework with Structure-level (SC) and Instance-level curricula (IC) that aims to translate sentences to semantic representation with a hierarchical structure. |
| Outcome: | Experiments on AMR2.0, AMR3.0, structure-complex and out-of-distribution situations confirm the effectiveness of the proposed framework. |
Lipschitz Constrained Parameter Initialization for Deep Transformers (2020.acl-main)
Copied to clipboard
| Challenge: | Existing studies show that deep Transformers have difficulty in training even with residual connection and layer normalization. |
| Approach: | They propose a method that leverages the Lipschitz constraint on the initialization of Transformer parameters to ease the optimization difficulties caused by its multi-layer encoder/decoder structure. |
| Outcome: | The proposed model outperforms previous RNN/CNN models but fails to converge with the original computation order. |
Accelerating Dense LLMs via L0-regularized Mixture-of-Experts (2025.acl-short)
Copied to clipboard
| Challenge: | Existing methods for accelerating large language models (LLMs) suffer from slow and costly inference. |
| Approach: | They propose a lightweight MoE approach using cluster confusion matrix and dynamic batching to accelerate dense LLMs. |
| Outcome: | The proposed method achieves 2.5x speedup over dense models while maintaining competitive performance. |
An Analysis under a Unified Formulation of Learning Algorithms with Output Constraints (2024.acl-srw)
Copied to clipboard
| Challenge: | Existing work on NN models with output constraints has not been able to categorize them in a unified manner. |
| Approach: | They propose new algorithms to integrate the information of main task and constraint injection . they use the H-score as a metric for considering main task metric and constrain infringement simultaneously . |
| Outcome: | The proposed algorithms integrate the information of main task and constraint injection, inspired by continual-learning algorithms. |
Parallel Continuous Chain-of-Thought with Jacobi Iteration (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to continuous CoT rely on sequential decoding of latent thought tokens, which leads to long training time and low inference speed. |
| Approach: | They propose a parallel continuous chain-of-thought which updates latent thought tokens iteratively in parallel instead of sequentially and improves both training and inference efficiency. |
| Outcome: | The proposed method saves 50% of training and inference time while maintaining stability and robustness in training. |
GTA: Supervised-Guided Reinforcement Learning for Text Classification with Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Reinforcement learning fine-tuning methods suffer from inefficient exploration and slow convergence . supervised fine- tuning methods have limited performance ceiling and less solid theoretical foundation . |
| Approach: | They propose a Guess-Think-Answer framework that combines supervised and supervised learning in a unified training paradigm. |
| Outcome: | The proposed framework outperforms both standalone SFT and RL training models on three text classification benchmarks. |
Noisy Self-Knowledge Distillation for Text Summarization (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to summarize text using a single reference and noisy datasets are ill-suited to summarising on single reference datasets. |
| Approach: | They propose to use self-knowledge distillation to improve text summarization by generating smoothed labels for students and teachers to reduce model uncertainty. |
| Outcome: | The proposed framework improves on pretrained and non-pretrained models on three benchmarks. |
Measuring and Mitigating Shortcut Reliance in Language Models with Probe-Based Representation Entanglement (2026.acl-srw)
Copied to clipboard
| Challenge: | Shortcut learning remains a major obstacle to robust NLP systems. |
| Approach: | They propose to fine-tune Gemma 3 1B Instruct and Llama 3.2 1B on two synthetic sentiment shortcuts in SST-2 and one natural shortcut in MNLI based on lexical overlap. |
| Outcome: | The proposed model improves on two synthetic sentiment shortcuts and one natural shortcut in MNLI with a 99% shortcut ratio, while Gemma drops from 91.8% to 60.2%. |
Can Public Large Language Models Help Private Cross-device Federated Learning? (2024.findings-naacl)
Copied to clipboard
| Challenge: | Recent studies have shown that public data can be used to improve privacy-utility trade-offs for large and small language models. |
| Approach: | They propose to use large-scale public data to help differentially private FL training . they propose a distribution matching algorithm with theoretical grounding to sample public data close to private data distribution . |
| Outcome: | The proposed method is efficient and effective for training private models by taking advantage of public data. |
PrefixNLI: Detecting Factual Inconsistencies as Soon as They Arise (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are prone to factually inconsistent statements, known as hallucinations. |
| Approach: | They propose to train a specialized model that detects inconsistencies over text prefixes to improve generation faithfulness by 5-14 F1 points. |
| Outcome: | The proposed model outperforms baseline models by 5-14 F1 points in prefix-level entailment. |
A Cross-Domain Transferable Neural Coherence Model (P19-1)
Copied to clipboard
Peng Xu, Hamidreza Saghir, Jin Sung Kang, Teng Long, Avishek Joey Bose, Yanshuai Cao, Jackie Chi Kit Cheung
| Challenge: | Existing coherence models do not generalize to unseen categories of text . previous work advocates for generative models for cross-domain generalization . |
| Approach: | They propose a local discriminative neural model with a smaller negative sampling space that can discriminate against incorrect orderings. |
| Outcome: | The proposed model outperforms state-of-the-art methods on a standard benchmark dataset on the Wall Street Journal corpus and multiple challenging settings on Wikipedia articles. |
Frequency Effects on Syntactic Rule Learning in Transformers (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models perform well on a variety of linguistic tasks that require symbolic reasoning, raising the question of whether such models implicitly represent abstract symbols and rules. |
| Approach: | They investigate the performance of BERT on English subject–verb agreement by analyzing word frequency and absolute frequency of verb forms. |
| Outcome: | The proposed model generalizes well to subject–verb pairs that never occurred in training, suggesting a degree of rule-governed behavior. |
Learning to Paraphrase Sentences to Different Complexity Levels (2023.tacl-1)
Copied to clipboard
| Challenge: | Using unsupervised datasets, we train models on sentence complexification and same-level paraphrasing tasks. |
| Approach: | They compare two unsupervised datasets with a single supervised dataset to train models on sentence complexification and same-level paraphrasing tasks. |
| Outcome: | The proposed models outperform previous work on sentence-level targeting and improve on the ASSET simplification benchmark. |
A Multimodal In-Context Tuning Approach for E-Commerce Product Description Generation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for generating product descriptions from images are inaccurate and generic . e-commerce product descriptions are important for content marketing and increasing engagement . |
| Approach: | They propose a new setting for generating product descriptions from images, augmented by marketing keywords. |
| Outcome: | The proposed approach improves the accuracy and diversity of product descriptions by up to 3.3% on Rouge-L and 9.4% on D-5. |
Grounding Open-Domain Instructions to Automate Web Support Tasks (2021.naacl-main)
Copied to clipboard
| Challenge: | RUSS is a task and dataset to ground natural language instructions on the web to perform previously unseen tasks. |
| Approach: | They build a task and dataset to ground AI agents from open-domain, step-by-step instructions on the web. |
| Outcome: | The proposed model outperforms existing models that map instructions to actions without WebLang. |
Neural Attention-Aware Hierarchical Topic Model (2021.emnlp-main)
Copied to clipboard
| Challenge: | Neural topic models (NTMs) use deep neural networks to learn topic information. |
| Approach: | They propose a variational autoencoder model that reconstructs sentence and document word counts using bag-of-words embeddings and pre-trained semantic embedders. |
| Outcome: | The proposed model lowers reconstruction errors at sentence and document levels and finds more coherent topics from real-world datasets. |
Context-Aware Monolingual Repair for Neural Machine Translation (D19-1)
Copied to clipboard
| Challenge: | et al., 2018) show that human raters prefer corrected translations over the baseline ones. |
| Approach: | They propose a monolingual model to correct inconsistencies between sentences . they use monolingual document-level data to train the model . |
| Outcome: | The proposed model improves translations of contextual phenomena in English-Russian translation task. |
LingGen: Scalable Multi-Attribute Linguistic Control via Power-Law Masking (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing methods for controlling coarse attributes are less effective for finer-grained attributes and suffer from inefficiencies when many attributes must be handled jointly. |
| Approach: | They propose a controlled text generation model that allows fine-grained control over a large number of real-valued linguistic attributes. |
| Outcome: | The proposed model achieves the lowest average control error among evaluated methods while remaining efficient at inference and receiving the highest fluency scores in human evaluation. |
Description-Based Zero-shot Fine-Grained Entity Typing (N19-1)
Copied to clipboard
| Challenge: | Existing systems consider a small set of coarse types, but fine-grained Entity Typing can be used for a variety of tasks. |
| Approach: | They propose a zero-shot entity typing approach that utilizes the type description available from Wikipedia to build a distributed semantic representation of the types. |
| Outcome: | The proposed method is able to recognize novel types without additional training on a public benchmark dataset. |
Neural Latent Extractive Document Summarization (D18-1)
Copied to clipboard
| Challenge: | Existing summarization paradigms focus on extractive summarizing based on sentence level labels . |
| Approach: | They propose a latent variable extractive model where sentences are viewed as latent variables and sentences with activated variables are used to infer gold summaries. |
| Outcome: | The proposed model outperforms a strong extractive baseline trained on rule-based labels and performs competitively with several recent models. |
Beyond Token Length: Step Pruner for Efficient and Accurate Reasoning in Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing reinforcement learning methods for large reasoning models suffer from excessive verbosity, known as "overthinking." Existing models penalize generated tokens to promote conciseness, but these methods encounter two challenges: they may develop hacking behavior in later stages of training by discarding reasoning steps. |
| Approach: | They propose a framework that steers large reasoning models toward more efficient reasoning . they prioritize correctness while imposing penalties for redundant steps . |
| Outcome: | The proposed framework reduces token usage by 69.7% on AIME24. |
You Only Read Once (YORO): Learning to Internalize Database Knowledge for Text-to-SQL (2025.naacl-long)
Copied to clipboard
Hideo Kobayashi, Wuwei Lan, Peng Shi, Shuaichen Chang, Jiang Guo, Henghui Zhu, Zhiguo Wang, Patrick Ng
| Challenge: | Existing text-to-SQL systems encode the same schema for every question, resulting in unnecessary high inference cost and missing crucial database knowledge. |
| Approach: | They propose a paradigm that directly internalizes database knowledge into the parametric knowledge of a text-to-SQL model during training and eliminates the need for schema encoding during inference. |
| Outcome: | The proposed paradigm significantly reduces the input token length by 66%-98% and outperforms traditional systems on three benchmarks. |
WSDMS: Debunk Fake News via Weakly Supervised Detection of Misinforming Sentences with Contextualized Social Wisdom (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for debunking fake news rely on blending of authentic and fabricated content by creators. |
| Approach: | They propose a model that detects misinformation at sentence-level using social media conversations . they use a bag-level annotation system to train the model . |
| Outcome: | The proposed model outperforms existing state-of-the-art models on three real-world benchmarks and outperformed existing state of the art models in debunking fake news at sentence and article levels. |
L4Q: Parameter Efficient Quantization-Aware Fine-Tuning on Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Quantization-aware PEFT methods have been developed to reduce memory and computational costs associated with large language models. |
| Approach: | They propose a method that integrates Quantization-Aware Training (QAT) with LoRA to reduce memory overhead and improve model accuracy. |
| Outcome: | The proposed method significantly reduces QAT’s memory overhead while preserving the advantage of QAT in producing fully quantized LLMs with high accuracy. |
Graph-Based Multilingual Label Propagation for Low-Resource Part-of-Speech Tagging (2022.emnlp-main)
Copied to clipboard
| Challenge: | Part-of-Speech (POS) tagging is an important component of the NLP pipeline, but many low-resource languages lack labeled training data. |
| Approach: | They propose a method for transferring labels from high-resource sources to low-resourced target languages using a graph-based label propagation method. |
| Outcome: | The proposed method achieves state-of-the-art for unsupervised POS tagging of low-resource languages. |
mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval (2024.emnlp-industry)
Copied to clipboard
Xin Zhang, Yanzhao Zhang, Dingkun Long, Wen Xie, Ziqi Dai, Jialong Tang, Huan Lin, Baosong Yang, Pengjun Xie, Fei Huang, Meishan Zhang, Wenjie Li, Min Zhang
| Challenge: | Existing models for text retrieval are based on a multi-stage process that involves retrieving documents from a large corpus. |
| Approach: | They propose to build a multilingual text representation model and a cross-encoder reranker from scratch for text retrieval. |
| Outcome: | The proposed models outperform the state-of-the-art models on long-context retrieval benchmarks. |
Controlling Text Edition by Changing Answers of Specific Questions (2021.findings-acl)
Copied to clipboard
| Challenge: | In many situations, we need to change specific content in a document. |
| Approach: | They propose a task where we take a long text, a question, and a target answer as input and output a minimally modified text so that it fits the target answer. |
| Outcome: | The proposed task is based on the existing dataset WIKIBIO and tests on a test set. |
Towards Example-Based NMT with Multi-Levenshtein Transformers (2023.emnlp-main)
Copied to clipboard
| Challenge: | Retrieval-augmented machine translation (RAMT) is attracting growing attention . it is assumed to implement some form of domain adaptation . |
| Approach: | They propose a retrieval-augmented version of the Levenshtein Transformer to make it more transparent . they propose to perform training and inference in this model, based on multi-way alignment algorithms and imitation learning. |
| Outcome: | The proposed architecture improves translation performance and improves consistency of translations compared to previous models. |
Multilingual Neural Machine Translation: Can Linguistic Hierarchies Help? (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Multilingual Neural Machine Translation (MNMT) trains a single model that supports translation between multiple languages . transferring knowledge from a diverse set of languages degrades the translation performance due to negative transfer. |
| Approach: | They propose a hierarchical knowledge distillation approach to train multilingual models . they use typological features and phylogeny to overcome negative transfer issue . |
| Outcome: | The proposed approach avoids negative transfer effect by capitalising on language groups generated according to typological features and phylogeny of languages. |
SLAM-Omni: Timbre-Controllable Voice Interaction System with Single-Stage Training (2025.findings-acl)
Copied to clipboard
Wenxi Chen, Ziyang Ma, Ruiqi Yan, Yuzhe Liang, Xiquan Li, Ruiyang Xu, Zhikang Niu, Yanqiao Zhu, Yifan Yang, Zhanxun Liu, Kai Yu, Yuxuan Hu, Jinyu Li, Yan Lu, Shujie Liu, Xie Chen
| Challenge: | a new spoken dialogue system with single-stage training is demonstrating its low latency and high quality . SLAM-Omni achieves zero-shot timbre control by modeling spoken language with semantic tokens . |
| Approach: | They propose a timbre-controllable, end-to-end voice interaction system with single-stage training. |
| Outcome: | The proposed system outperforms previous models on 4 GPUs with limited data. |
CMD: a framework for Context-aware Model self-Detoxification (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods of text detoxification fail to achieve a decent balance between effectiveness and generation quality. |
| Approach: | They propose a text detoxification framework that pays attention to both context and detoxification process. |
| Outcome: | Experiments on various LLMs show that the proposed framework can yield the best performance compared to baselines. |
A Distant Supervision Corpus for Extracting Biomedical Relationships Between Chemicals, Diseases and Genes (2022.lrec-1)
Copied to clipboard
| Challenge: | Biomedical researchers have used manual curation to extract biomedical interactions from research texts to improve coverage. |
| Approach: | They propose a new dataset for training and evaluating multi-class multi-label biomedical relation extraction models using human annotations and the CTD database. |
| Outcome: | The proposed dataset is substantially larger and cleaner than existing datasets and includes annotations linking mentions to their entities. |
Fine-Tuned LLMs are “Time Capsules” for Tracking Societal Bias Through Books (2025.naacl-long)
Copied to clipboard
| Challenge: | We develop a corpus comprising 593 fictional books across seven decades (1950-2019) to track bias evolution. |
| Approach: | They develop a method to trace and quantify bias evolution using fine-tuned LLMs on fictional books across seven decades to track bias evolution. |
| Outcome: | The proposed method traces and quantifies bias evolution in a corpus of 593 fictional books across seven decades. |
Extract and Edit: An Alternative to Back-Translation for Unsupervised Neural Machine Translation (N19-1)
Copied to clipboard
| Challenge: | Back-translation has been used in previous approaches for unsupervised neural machine translation, but pseudo sentences are of low quality as translation errors accumulate during training. |
| Approach: | They propose an approach to extract and edit real sentences from monolingual corpora and introduce a comparative translation loss to evaluate the translated target sentences. |
| Outcome: | The proposed approach outperforms state-of-the-art translation systems across two benchmarks and two low-resource language pairs by more than 2 BLEU points. |
Controlling Learned Effects to Reduce Spurious Correlations in Text Classifiers (2023.acl-long)
Copied to clipboard
| Challenge: | toxicity and IMDB review datasets show that pre-trained NLP classifiers learn spurious correlations between input features and label . |
| Approach: | They propose an algorithm to regularize the learnt effect of features on the model’s prediction to the estimated effect of a feature on label. |
| Outcome: | The proposed method minimises spurious correlations and improves minority group accuracy while improving total accuracy compared to standard training. |
Effects of Parameter Norm Growth During Transformer Training: Inductive Bias from Gradient Descent (2021.emnlp-main)
Copied to clipboard
| Challenge: | Evidence is emerging that neural networks learn due to inductive bias in the training routine, typically a variant of gradient descent (GD). |
| Approach: | They propose to characterize GD as an inductive bias in transformer training . they document norm growth in transformer language models and show they are saturated . |
| Outcome: | Empirically, we document norm growth in the training of transformer language models . the results suggest saturation is a new characterization of an inductive bias implicit in GD . |
Length-aware Byte Pair Encoding for Mitigating Over-segmentation in Korean Machine Translation (2024.findings-acl)
Copied to clipboard
Jungseob Lee, Hyeonseok Moon, Seungjun Lee, Chanjun Park, Sugyeong Eo, Hyunwoong Ko, Jaehyung Seo, Seungyoon Lee, Heuiseok Lim
| Challenge: | Byte Pair Encoding (BPE) is an effective approach in machine translation across several languages, but it is prone to over-segmentation in Korean, an agglutinative and morphologically rich language. |
| Approach: | They propose a new method that incorporates long words into the Korean vocabulary by strategically preserving morphological information and reducing semantic confusion. |
| Outcome: | The proposed method outperforms BPE and surpasses state-of-the-art morpheme-aware tokenization methods. |
A Unified Approach to Entity-Centric Context Tracking in Social Conversations (2022.lrec-1)
Copied to clipboard
| Challenge: | Context Tracking is a computational task for human-human conversations . it involves identifying important entities and keeping track of their properties and relationships . |
| Approach: | They propose to use a human-human conversation corpus for context tracking with people and location annotations to model the conversation's context. |
| Outcome: | The proposed model is based on a large human-human conversation corpus with people and location annotations. |
Sentiment Adaptive End-to-End Dialog Systems (P18-1)
Copied to clipboard
| Challenge: | Existing methods to train dialog systems only consider semantic inputs and under-utilize other user information. |
| Approach: | They propose to include user sentiment in the end-to-end learning framework to make dialog systems more user-adaptive and effective. |
| Outcome: | The proposed system improves on a bus information search task with sentiment information. |
Investigating Content Planning for Navigating Trade-offs in Knowledge-Grounded Dialogue (2024.eacl-long)
Copied to clipboard
| Challenge: | Knowledge-grounded dialogues require a balance between being specific to what the conversation partner has said and being attributable to an underlying source document. |
| Approach: | They propose a framework that allows to experiment with various plan variables supported by prior work . they show that metric-aware planning mechanisms are better at automatic evaluations but underperform in human judgment compared to metric agnostic mechanisms. |
| Outcome: | The proposed framework supports metric-agnostic and metric aware content planning, but it underperforms in human judgment. |
TechING: Towards Real World Technical Image Understanding via VLMs (2026.findings-eacl)
Copied to clipboard
| Challenge: | Modern day vision language models struggle when it comes to understanding technical diagrams . a large synthetically generated corpus is needed to train and evaluate VLMs on hand-drawn images . |
| Approach: | They propose a large synthetically generated corpus for training VLMs and evaluate them on hand-drawn images. |
| Outcome: | The proposed model improves ROUGE-L performance of Llama 3.2 11B-instruct by 2.14x on synthetic images on real-world images. |
Self-Distillation with Meta Learning for Knowledge Graph Completion (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing knowledge graph completion frameworks for knowledge graphs are far from complete and require missing triples to be added to them. |
| Approach: | They propose a dynamic pruning technique to obtain a pruned model from a large source model, where the pruning mask of the pruned models could be updated adaptively per epoch after the model weights are updated. |
| Outcome: | The proposed framework achieves competitive performance compared to strong baselines, while being 10x smaller than baselines. |
A Neural Approach to Pun Generation (P18-1)
Copied to clipboard
| Challenge: | generating puns with artificial intelligence techniques requires manual training and templates. |
| Approach: | They propose neural network models for homographic pun generation that can generate puns without requiring any pun data for training. |
| Outcome: | The proposed models generate homographic puns of good readability and quality without training. |
Automatic Multiple-Choice Question Generation and Evaluation Systems Based on LLM: A Study Case With University Resolutions (2025.coling-main)
Copied to clipboard
Sérgio Silva Mucciaccia, Thiago Meireles Paixão, Filipe Wall Mutz, Claudine Santos Badue, Alberto Ferreira de Souza, Thiago Oliveira-Santos
| Challenge: | Multiple choice questions (MCQs) are often used in employee selection and training, but their creation is resource-intensive and requires significant effort and investment. |
| Approach: | They propose to use large language models and prompt engineering techniques to automate the generation and validation of MCQs. |
| Outcome: | The proposed system reduces the burden on human resources and enables scalable, cost-effective MCQ generation. |
RMLM: A Flexible Defense Framework for Proactively Mitigating Word-level Adversarial Attacks (2023.acl-long)
Copied to clipboard
| Challenge: | Existing defenses focus on improving robustness of the victim model in training, but neglect to mitigate adversarial attacks during inference. |
| Approach: | They propose a framework that confuses attackers and corrects adversarial contexts . their framework helps improve the robustness of the victim model during inference . |
| Outcome: | The proposed framework improves the robustness of the victim model in training . it also corrects abnormal contexts in the representation level and filtering out examples . |
Mitigating Exposure Bias in Grammatical Error Correction with Data Augmentation and Reweighting (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing approaches to grammatical error correction (GEC) use sequence-to-sequence models, but there is an exposure bias problem. |
| Approach: | They propose a data manipulation approach to overcome the exposure bias problem in seq2seq GEC . they propose augmentation methods to mimic decoder input and reweighting methods to automatically balance the importance of each kind of augmented samples. |
| Outcome: | The proposed method improves on benchmark GEC datasets. |
Should I try multiple optimizers when fine-tuning a pre-trained Transformer for NLP tasks? Should I tune their hyperparameters? (2024.eacl-long)
Copied to clipboard
| Challenge: | NLP research has explored different neural model architectures and sizes, datasets, training objectives, and transfer-learning techniques. |
| Approach: | They propose to use a variant of Stochastic Gradient Descent (SGD) to select among numerous variants, often with minimal or no tuning of the optimizer’s hyperparameters. |
| Outcome: | Experiments with five GLUE datasets, two models and seven popular optimizers show that tuning just the learning rate is as good as tuning all the hyperparameters. |
WSLLN:Weakly Supervised Natural Language Localization Networks (D19-1)
Copied to clipboard
| Challenge: | Existing methods to learn correspondence between visual segments and texts require temporal coordinates for training, which leads to high costs of annotation. |
| Approach: | They propose weakly supervised language localization networks to detect events in untrimmed videos . they train with only video-sentence pairs without accessing to temporal locations of events . |
| Outcome: | Experiments on ActivityNet Captions and DiDeMo show that WSLLN performs state-of-the-art. |
Analyzing Bayesian Crosslingual Transfer in Topic Models (N19-1)
Copied to clipboard
| Challenge: | a theoretical analysis of crosslingual transfer in probabilistic topic models is presented . we use Gibbs sampling to quantify the loss of knowledge across languages . |
| Approach: | They propose a method to quantify the loss of knowledge across languages during crosslingual transfer in probabilistic topic models. |
| Outcome: | The proposed model quantifies the loss of knowledge across languages during this process . it is validated on a diverse set of five languages and discusses best practices for data collection and model design . |
Boosting Summarization with Normalizing Flows and Aggressive Training (2023.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that FlowSUM improves the quality of generated summaries with minimal impact on inference time. |
| Approach: | They propose a normalizing flows-based variational encoder-decoder framework for Transformer-based summarization. |
| Outcome: | The proposed model improves the quality of generated summaries and reduces inference time. |
Multimodal Prompt Learning for Product Title Generation with Extremely Limited Labels (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to generate informative titles for products with limited labels are inadequate for novel products. |
| Approach: | They propose a prompt-based approach to generate attractive titles for novel products . they use multimodal prompts to preserve characteristics and writing styles of novel products. |
| Outcome: | The proposed approach achieves state-of-the-art results on novel product categories with limited labels. |
Weakly Supervised Semantic Parsing with Abstract Examples (P18-1)
Copied to clipboard
| Challenge: | training semantic parsers from weak supervision complicates training in two ways . spurious programs that accidentally lead to a correct denotation add noise to training . |
| Approach: | They propose to use tokens in both language utterance and program to map denotations to executable programs. |
| Outcome: | The proposed method improves performance and reaches 82.5% accuracy compared to the best reported accuracy so far. |
LLM-Based Offline Learning for Embodied Agents via Consistency-Guided Reward Ensemble (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Employing large language models (LLMs) to enable embodied agents has become popular, yet it presents several limitations in practice. |
| Approach: | They propose a consistency-guided reward ensemble framework to train agents offline via offline reinforcement learning (RL) they use spatio-temporally consistent rewards to derive domain-grounded rewards from training datasets. |
| Outcome: | The proposed framework outperforms state-of-the-art LLM-based agents with 8B parameters and has 117M parameters for agent policy network and only for training. |
A Multi-Format Transfer Learning Model for Event Argument Extraction via Variational Information Bottleneck (2022.coling-1)
Copied to clipboard
| Challenge: | Event argument extraction (EAE) aims to extract arguments with given roles from texts. |
| Approach: | They propose a multi-format transfer learning model with variational information bottleneck to learn from existing datasets. |
| Outcome: | The proposed model improves on three benchmark datasets and obtains state-of-the-art performance on EAE. |
Self-Supervised Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) methods relied on the availability of high-quality parallel corpora. |
| Approach: | They propose a method where an emergent NMT system is used for selecting training data and learning internal NMT representations. |
| Outcome: | The proposed method achieves BLEU scores of 29.21 (en2fr) and 27.36 (fr2en) on newstest2014 using English and French Wikipedia data for training. |
Multilingual Word Segmentation: Training Many Language-Specific Tokenizers Smoothly Thanks to the Universal Dependencies Corpus (L18-1)
Copied to clipboard
| Challenge: | Towards language scalability, major progress has been achieved in multilingual language technology in recent years. |
| Approach: | They propose a tokenizer that can be trained from any Universal Dependencies corpus dataset . they argue that tokenization should be seen as a supervised task and scalability requires a software engineering process across languages. |
| Outcome: | The proposed tokenizer can be trained from any dataset in the corpus UD2 . the proposed software tool relies on elephant to perform the training . |
Static Embeddings as Efficient Knowledge Bases? (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent research investigates factual knowledge stored in large pretrained language models . masked sentences such as “Paris is the capital of [MASK]” are used as probes . |
| Approach: | They use masked sentences to test whether a language model can capture factual knowledge . they show that static embeddings perform better than PLMs when restricted to a candidate set . |
| Outcome: | The results show that static embeddings perform better than PLMs when restricted to a candidate set . |
Lessons from Natural Language Inference in the Clinical Domain (D18-1)
Copied to clipboard
| Challenge: | State of the art models with deep neural networks lack generalization capabilities in specialized domains where training data is limited. |
| Approach: | They propose a dataset annotated by doctors performing a natural language inference task grounded in the medical history of patients. |
| Outcome: | The proposed model outperforms existing models in the clinical domain by incorporating domain knowledge from external data and lexical sources. |
NASH: Toward End-to-End Neural Architecture for Generative Semantic Hashing (P18-1)
Copied to clipboard
Dinghan Shen, Qinliang Su, Paidamoyo Chapfuwa, Wenlin Wang, Guoyin Wang, Ricardo Henao, Lawrence Carin
| Challenge: | Existing approaches to fast similarity search require two-stage training and the binary constraints are handled ad-hoc. |
| Approach: | They propose an end-to-end neural architecture for semantic hashing where binary hash codes are treated as Bernoulli latent variables. |
| Outcome: | The proposed approach outperforms state-of-the-art models on unsupervised and supervised scenarios on three public datasets. |
Context-aware Entity Typing in Knowledge Graphs (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for knowledge graph entity typing are embedding-based and graph convolutional networks (GCNs) . Existing approaches for knowledge Graph Entity Typing (KGET) are incomplete and require multiple inference mechanisms. |
| Approach: | They propose a method that uses entities’ contextual information to infer missing types in knowledge graphs by using two inference mechanisms: N2T and Agg2T. |
| Outcome: | The proposed method can infer entities' missing types by completing two real-world KGs. |
Single Model Ensemble for Subword Regularized Models in Low-Resource Machine Translation (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing subword regularizations use multiple segmentations during training but only use one segmentation in inference. |
| Approach: | They propose an inference strategy that uses multiple subword segmentations to solve this discrepancy in the training process and inference. |
| Outcome: | The proposed strategy reduces the cost of training and improves the performance of models trained with subword regularization in low-resource machine translation tasks. |
Eliciting Affective Events from Language Models by Multiple View Co-prompting (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to generate training data using weakly labeled data are costly and limited . |
| Approach: | They propose a method for acquiring and labeling affective events with multiple view co-prompting using pre-trained language models. |
| Outcome: | The proposed approach improves state-of-the-art affective event classifier on two datasets. |
Opinion Summarization by Weak-Supervision from Mix-structured Data (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for opinion summarization of multiple reviews lack reference summaries . OAs and ISs are often mismatched between review input and summary . |
| Approach: | They propose a method to generate mixed-structured synthetic training data for opinion summarization. |
| Outcome: | The proposed method outperforms existing methods on Yelp, Amazon and RottenTomatos datasets. |
LIRE: listwise reward enhancement for preference alignment (2024.findings-acl)
Copied to clipboard
| Challenge: | prevailing approaches to preference alignment focus on pairwise comparisons, with limited exploration into multi-response scenarios. |
| Approach: | They propose a listwise reward enhancement approach that integrates offline rewards of multiple responses into a streamlined listwise framework. |
| Outcome: | The proposed approach outperforms existing methods on dialogue and summarization tasks with good transferability to out-of-distribution data. |
Self-Induced Curriculum Learning in Self-Supervised Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on curriculum learning focus on selecting the best distribution of data to train a system. |
| Approach: | They propose a self-supervised neural machine translation model that self-selects data without being told to do so. |
| Outcome: | The proposed model self-selects samples of increasing complexity and task relevance without being told to do so, and performs a denoising curriculum. |
Negative Sample is Negative in Its Own Way: Tailoring Negative Sentences for Image-Text Retrieval (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to retrieve hard negative sentences are limited in the scale of the dataset thus fail to identify negative samples of high difficulty for every image. |
| Approach: | They propose to use a model to generate synthetic negative sentences with higher difficulty by masking and refilling the images and performing word discrimination and word correction tasks to improve retrieval and generation. |
| Outcome: | The proposed model generates synthetic negative sentences with higher difficulty on MS-COCO and Flickr30K and is robust and faithful to state-of-the-art training. |
Entropy Scheduling in Reinforcement Learning for Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | entropy in reinforcement learning functions analogously to the learning rate in LLMs. |
| Approach: | They propose an entropy scheduling system that optimizes different pre-set goals by controlling and scheduling entropicy at each step of the RL process. |
| Outcome: | The proposed method improves AIME2024 from 50.9 to 54.9 within 40 training steps. |
Unsupervised Discovery of Multimodal Links in Multi-image, Multi-sentence Documents (D19-1)
Copied to clipboard
| Challenge: | a structured training objective based on identifying whether collections of images and sentences co-occur in documents can suffice to predict links between specific images and specific sentences. |
| Approach: | They propose algorithms that discover image-sentence relationships without explicit annotation . they experiment on seven datasets of varying difficulty . |
| Outcome: | The proposed algorithms can predict links between images and sentences without explicit multimodal annotation. |
Exploring the Generalizability of Factual Hallucination Mitigation via Enhancing Precise Knowledge Utilization (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to improve LLMs' ability to align their responses with objective facts suffer from poor generalization and trade-offs in other different capabilities. |
| Approach: | They propose to introduce PKUE (Precise Knowledge Utilization Enhancement) which fine-tunes the model on self-generated responses to precise and simple factual questions through preference optimization. |
| Outcome: | The proposed enhancements improve LLM’s ability to precisely leverage its knowledge and improve FactualBench, a comprehensive and precise factual QA dataset containing 181k Chinese data spanning 21 domains. |
Towards Quantifiable Dialogue Coherence Evaluation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing automatic dialogue coherence evaluation metrics are expensive and high-latency, which cannot meet the requirements of a dialogue system. |
| Approach: | They propose a framework to train a quantifiable dialogue coherence metric that can reflect actual human rating standards. |
| Outcome: | Experimental results show that the model trained by QuantiDCE presents stronger correlations with human judgements than the other state-of-the-art metrics. |
Semi-supervised Formality Style Transfer using Language Model Discriminator and Mutual Information Maximization (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Formality style transfer is the task of converting informal sentences to grammatically-correct formal sentences. |
| Approach: | They propose a semi-supervised formality style transfer model that utilizes a language model-based discriminator to maximize the likelihood of the output sentence being formal. |
| Outcome: | The proposed model outperforms state-of-the-art models in terms of automated metrics and human judgement. |
Conversation Disentanglement with Bi-Level Contrastive Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods focus on pairwise utterance relations but pay inadequate attention to utterant-to-context relation modeling. |
| Approach: | They propose a general disentangle model based on bi-level contrastive learning that brings closer utterances in the same session while encouraging each utterrance to be near its clustered session prototypes in representation space. |
| Outcome: | The proposed model achieves state-of-the-art performance on both settings across public datasets. |
LM-Infinite: Zero-Shot Extreme Length Generalization for Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Currently, large language models (LLMs) train on short text segments due to the computational overhead quadratic in the input lengths of their Transformer architectures. |
| Approach: | They propose a method that allows LLMs pre-trained with 2K or 4K-long segments to generalize to up to 200M length inputs while retaining perplexity. |
| Outcome: | The proposed method achieves 2.7 decoding speed up and 7.5 memory saving over the original model. |
One-Shot Relational Learning for Knowledge Graphs (D18-1)
Copied to clipboard
| Challenge: | Existing studies on knowledge graph completion require a large number of positive examples for each relation, but long-tail relations are more common in KGs and those newly added relations do not have many known triples for training. |
| Approach: | They propose a one-shot relational learning framework that utilizes the knowledge distilled by embedding models and learns a matching metric by considering both the learned embeddments and one-hop graph structures. |
| Outcome: | The proposed framework improves on existing embedding models and eliminates the need for retraining when dealing with newly added relations. |
Semi-Supervised Dependency Parsing with Arc-Factored Variational Autoencoding (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for dependency parsing use unlabelled data to compensate for the lack of training corpora. |
| Approach: | They propose semi-supervised dependency parsing methods that utilize unlabelled data to compensate for the scarcity of labelled training corpora. |
| Outcome: | The proposed model overcomes the tree constraint and the complexity of the training procedure while avoiding the challenges brought by the tree constraints. |
Controlled Neural Sentence-Level Reframing of News Articles (2021.findings-emnlp)
Copied to clipboard
| Challenge: | a news article is framed from a specific perspective, but reframing can be difficult . a framed article can be used to communicate with opposing camps of audiences . |
| Approach: | They propose to reframe news articles using a media frame corpus to achieve this . they propose three strategies to train neural models for reframing . |
| Outcome: | The proposed techniques maintain coherence of sentences and reframe them correctly . the proposed techniques are effective but have tradeoffs . |
Two Parents, One Child: Dual Transfer for Low-Resource Neural Machine Translation (2021.findings-acl)
Copied to clipboard
| Challenge: | Neural machine translation suffers when parallel data is scarce for training . a new framework to transfer multiple sources of auxiliary data is proposed . |
| Approach: | They propose a framework to transfer multiple sources of auxiliary data from high-resource parallel data to low-resourced translation models using pretrained language models. |
| Outcome: | The proposed approach yields consistent improvements over strong competitors for multiple translation directions. |
Do not let the history haunt you: Mitigating Compounding Errors in Conversational Question Answering (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing approaches employ human-written ground-truth answers for answering conversational questions at test time, but in a realistic scenario, the CoQA model will not have access to ground-Truth answers. |
| Approach: | They propose a sampling strategy that dynamically selects between target answers and model predictions during training, closely simulating the situation at test time. |
| Outcome: | The proposed sampling strategy closely simulates the situation at test time and significantly lowers the performance of CoQA systems. |
kFolden: k-Fold Ensemble for Out-Of-Distribution Detection (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies studying OOD detection in NLP often rely on external data to diversify model predictions. |
| Approach: | They propose a framework which mimics OOD detection behavior without external data . they take text classification as an archetype and compare them to existing datasets . |
| Outcome: | The proposed framework can resolve in- and out-distribution examples in a natural way using existing datasets. |
Logical Reasoning with Span-Level Predictions for Interpretable and Robust NLI Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Current models learn from annotation artefacts and dataset biases, but it is unclear to what extent they are learning the task of NLI. |
| Approach: | They propose a logical reasoning framework that allows models to learn from annotation artefacts and dataset biases. |
| Outcome: | The proposed model outperforms humans on in-distribution test sets without using span labels . the model is more robust in a reduced data setting, and out-of-disturbance performance is improved . |
Graph-tree Fusion Model with Bidirectional Information Propagation for Long Document Classification (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for document classification struggle with token limits and fail to adequately model hierarchical relationships within documents. |
| Approach: | They propose a novel model leveraging a graph-tree structure to capture local and global dependencies. |
| Outcome: | The proposed model captures syntactic relationships and broader document contexts without token limits and can handle arbitrarily long contexts. |
Enabling Autoregressive Models to Fill In Masked Tokens (2026.findings-eacl)
Copied to clipboard
| Challenge: | Autoregressive (AR) and masked language modeling (MLM) models are incapable of mucked infilling, which is the ability to predict mangled tokens between past and future context. |
| Approach: | They propose a method that leverages the strengths of autoregressive and masked language modeling to achieve state-of-the-art mucked infilling performance. |
| Outcome: | The proposed approach outperforms existing methods on masked infilling tasks. |
Test-Time Code-Switching for Cross-lingual Aspect Sentiment Triplet Extraction (2025.naacl-long)
Copied to clipboard
| Challenge: | Aspect Sentiment Triplet Extraction (ASTE) is a thriving research area . current code-switching methods suffer from term boundary detection issues and out-of-dictionary problems. |
| Approach: | They propose a test-time code-switching framework which bridges the gap between bilingual training and monolingual test- time prediction. |
| Outcome: | The proposed framework achieves an average improvement of 3.7% on four cross-lingual datasets. |
Deep Dominance - How to Properly Compare Deep Neural Models (P19-1)
Copied to clipboard
| Challenge: | Existing methods for comparing DNNs on unseen data are not suitable for this task. |
| Approach: | They propose to adapt a test for the Almost Stochastic Dominance relation between two distributions to the problem by comparing their performance on unseen data. |
| Outcome: | The proposed method meets all criteria while previously proposed methods fail to do so. |
Policy Shaping and Generalized Update Equations for Semantic Parsing from Denotations (D18-1)
Copied to clipboard
| Challenge: | Existing learning approaches for parsing from denotations (SpFD) do not provide access to correct representations, so there are two steps for every training example. |
| Approach: | They propose a framework for parsing from denotations that generalizes three different learning algorithms. |
| Outcome: | The proposed framework outperforms previous work by 5.0% absolute on exact match accuracy on a question answering dataset. |
General Purpose Text Embeddings from Pre-trained Language Models for Scalable Inference (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Large pre-trained language models are currently used for many NLP tasks . however, inference for these models requires significant computational resources . |
| Approach: | They propose to use a shared text encoder to amortize the computational cost of inference over multiple tasks. |
| Outcome: | The proposed method reduces the size of the extracted representations by a factor of 16 to store them for later use. |
Enhancing Sequence-to-Sequence Neural Lemmatization with External Resources (2021.eacl-main)
Copied to clipboard
| Challenge: | a hybrid approach to lemmatization enhances the seq2seq neural model with additional lemmas extracted from an external lexicon or a rule-based system. |
| Approach: | They propose a hybrid approach that enhances a seq2seq neural model with additional lemmas extracted from an external lexicon or a rule-based system. |
| Outcome: | The proposed model achieves statistically significant improvements on 23 UD languages, compared to baseline models not utilizing additional lemma information. |
Learning to Model and Ignore Dataset Bias with Mixed Capacity Ensembles (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work has shown that datasets contain incidental correlations created by idiosyncrasies in the data collection process. |
| Approach: | They propose a method that detects and ignores dataset-specific correlations by introducing a new method that makes them conditionally independent. |
| Outcome: | The proposed method detects and ignores these kinds of dataset-specific correlations, and does not require the bias to be known in advance. |
Out-of-Scope Intent Detection with Self-Supervision and Discriminative Training (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for out-of-scope intent detection rely on strong assumptions on data distribution and confidence threshold selection. |
| Approach: | They propose a method to train an out-of-scope intent classifier in a fully end-to-end manner by simulating the test scenario in training. |
| Outcome: | The proposed method improves on four benchmark dialogue datasets and improves over state-of-the-art methods. |
Self-Taught Agentic Long Context Understanding (2025.acl-long)
Copied to clipboard
Yufan Zhuang, Xiaodong Yu, Jialian Wu, Ximeng Sun, Ze Wang, Jiang Liu, Yusheng Su, Jingbo Shang, Zicheng Liu, Emad Barsoum
| Challenge: | Extensive experiments across seven long-context tasks demonstrate that AgenticLU significantly outperforms state-of-the-art prompting methods and specialized long-consumer LLMs. |
| Approach: | They propose a framework to enhance an LLM's understanding of long-context questions by integrating targeted self-clarification with contextual grounding within an agentic workflow. |
| Outcome: | The proposed framework outperforms state-of-the-art prompting methods and specialized long-context LLMs in seven long-constitut tasks. |
Semantic Parsing for Technical Support Questions (C18-1)
Copied to clipboard
| Challenge: | Automating technical support is a task of providing answers to complex problems . traditional approaches to this task rely on information retrieval and are keyword based . |
| Approach: | They propose a semantic parsing approach that uses grammatical structure to extract technical questions' attributes as a baseline and a CRF-based model that can improve performance in the presence of annotated data. |
| Outcome: | The proposed model outperforms retrieval baselines in annotated data for training. |
LaPraDoR: Unsupervised Pretrained Dense Retriever for Zero-Shot Text Retrieval (2022.findings-acl)
Copied to clipboard
| Challenge: | Experimental results show that LaPraDoR is state-of-the-art compared with supervised dense retrieval models. |
| Approach: | They propose a pretrained dual-tower dense retriever that does not require supervised data for training. |
| Outcome: | The proposed method achieves state-of-the-art performance on 18 datasets of 9 zero-shot text retrieval tasks. |
Discriminative Learning of Open-Vocabulary Object Retrieval and Localization by Negative Phrase Augmentation (D18-1)
Copied to clipboard
| Challenge: | Existing methods for object detection only handle pre-specified classes, requiring large amounts of visual samples for training. |
| Approach: | They propose a method to retrieve and localize objects specified by a textual query from one million images in 0.5 seconds with high precision. |
| Outcome: | The proposed method can retrieve and localize objects specified by a textual query from one million images in 0.5 seconds with high precision. |
Unsupervised Graph-Text Mutual Conversion with a Unified Pretrained Language Model (2023.acl-long)
Copied to clipboard
| Challenge: | Existing unsupervised approaches for learning knowledge graphs require multiple modules and require entity information or relation type for training. |
| Approach: | They propose a method that uses a unified pretrained language model to achieve fully unsupervised graph-text mutual conversion for the first time. |
| Outcome: | The proposed method outperforms state-of-the-art methods for G2T and T2G tasks by fine-tuning only one pretrained model. |
Does it Really Generalize Well on Unseen Data? Systematic Evaluation of Relational Triple Extraction Methods (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing extraction models memorize and recall already seen triples but cannot generalize effectively for unseen triples. |
| Approach: | They propose a method to generalize existing extraction models by rearranging datasets and augmenting test sets. |
| Outcome: | The proposed method can significantly increase the generalization performance of existing models. |
Transparency at the Source: Evaluating and Interpreting Language Models With Access to the True Distribution (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a new approach to train, evaluate and interpret neural language models uses artificial, language-like data. |
| Approach: | They propose a setup for training, evaluating and interpreting neural language models that uses artificial, language-like data. |
| Outcome: | The proposed model is based on a massive probabilistic grammar and a large natural language corpus, and provides complete control over the generative process. |
Klexikon: A German Dataset for Joint Summarization and Simplification (2022.lrec-1)
Copied to clipboard
| Challenge: | Traditionally, Text Simplification is a monolingual translation task where individual sentences are "translated" into a simplified version. |
| Approach: | They propose to use a dataset to jointly simplify long source documents by combining sentences from a source and their simplified counterparts. |
| Outcome: | The proposed system can summarize and simplify long source documents using almost 2,900 documents. |
Grammatical Error Correction as GAN-like Sequence Labeling (2021.findings-acl)
Copied to clipboard
| Challenge: | Traditional GEC models learn from sentences with fixed error rates . sequence labeling approaches suffer from a couple of key problems . |
| Approach: | They propose a GAN-like sequence labeling model with a grammatical error detector and a generator to correct grammamatical errors. |
| Outcome: | The proposed model improves the state-of-the-art in GEC and improves on benchmarks. |
Benchmarking a transformer-FREE model for ad-hoc retrieval (2021.eacl-main)
Copied to clipboard
| Challenge: | a recent study compares transformer-based models with a greener and more sustainable alternative. |
| Approach: | They compare transformer-based models with a "greener and more sustainable" alternative . they show that transformer-like models can be used in real-world retrieval applications . |
| Outcome: | The lighter model achieves a speedup of 20 times in training and 7 to 47 times in inference while maintaining a comparable retrieval performance. |
Is LLM an Overconfident Judge? Unveiling the Capabilities of LLMs in Detecting Offensive Language with Annotation Disagreement (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing studies focus on evaluating large language models' ability to handle disagreement cases. |
| Approach: | They evaluate the performance of large language models in detecting offensive language at varying levels of agreement. |
| Outcome: | The proposed model improves detection accuracy and model alignment with human judgment by using disagreement samples in training. |
Agent Trading Arena: A Study on Numerical Understanding in LLM-Based Agents (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to large language models are limited to historical backtesting and static data. |
| Approach: | a new large-language model is developed to simulate real-time trading in a virtual stock market . the agent trading arena simulates real-world bid-ask interactions and provides real-life trading scenarios . |
| Outcome: | The Agent Trading Arena simulates real-world market conditions and directly impacts price dynamics. |
CLHA: A Simple Yet Effective Contrastive Learning Framework for Human Alignment (2024.lrec-main)
Copied to clipboard
Feiteng Fang, Liang Zhu, Xi Feng, Jinchang Hou, Qixuan Zhao, Chengming Li, Xiping Hu, Ruifeng Xu, Min Yang
| Challenge: | Large language models (LLMs) have attracted considerable attention from academic and industrial communities due to their outstanding performance in various natural language processing tasks. |
| Approach: | They propose a Contrastive Learning Framework for Human Alignment to evaluate the noise within the data and dynamically adjust the training process. |
| Outcome: | The proposed framework surpasses other algorithms in terms of reward model scores, automatic evaluations, and human assessments on the widely used dataset "Helpful and Harmless" |
Generative Biomedical Entity Linking via Knowledge Base-Guided Pre-training and Synonyms-Aware Fine-tuning (2022.naacl-main)
Copied to clipboard
| Challenge: | Generative methods for biomedical entity linking (EL) use synonyms knowledge from knowledge bases (KB) this is not trivial to inject into a generative method, but it is cost-effective. |
| Approach: | They propose to inject synonyms knowledge into a generative model of biomedical EL by constructing synthetic samples with synonyms and definitions from KB and requiring the model to recover concept names. |
| Outcome: | The proposed method achieves state-of-the-art results on several biomedical EL tasks without candidate selection. |
Lattice-Based Transformer Encoder for Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) takes deterministic sequences for source representations. However, word-level or subword-level segmentation has multiple choices to split a source sequence with different word segmentors or different subword vocabulary sizes. |
| Approach: | They propose lattice-based encoders to explore effective word or subword representations in an automatic way during training. |
| Outcome: | The proposed encoders can explore effective word or subword representation in an automatic way during training. |
Text-Only Training for Image Captioning using Noise-Injected CLIP (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a new approach to image captioning requires large datasets of captioned images and is difficult to collect. |
| Approach: | They propose to use a decoder to translate CLIP textual embeddings back into text . they show that this intuition is “almost correct” because of a gap between the embeddable spaces . |
| Outcome: | The proposed approach shows that the intuition is “almost correct” because of a gap between the embedding spaces, and rectifies this via noise injection during training. |
Evaluating Approaches to Personalizing Language Models (2020.lrec-1)
Copied to clipboard
| Challenge: | a large amount of text is not available for training a user-specific language model, which suggests a need to personalize language models with only a small amount of data. |
| Approach: | They propose three approaches to personalize a language model that was trained on a large background corpus using a relatively small amount of text from an individual user. |
| Outcome: | The proposed techniques outperform language model adaptation based on demographic factors. |
Dialogue Summaries as Dialogue States (DS2), Template-Guided Summarization for Few-shot Dialogue State Tracking (2022.findings-acl)
Copied to clipboard
| Challenge: | Annotating task-oriented dialogues is notorious for the expensive and difficult data collection process. |
| Approach: | They propose to reformulate dialogue state tracking as a dialogue summarization problem by using synthetic dialogue summaries generated by a set of rules. |
| Outcome: | The proposed method outperforms previous studies on few-shot dialogue state tracking in MultiWoZ 2.0 and 2.1 in cross-domain and multi-domain settings. |
BERT-kNN: Adding a kNN Search Component to Pretrained Language Models for Better QA (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained language models (PLMs) capture a diverse range of linguistic and factual knowledge without the use of finetuning. |
| Approach: | They combine BERT with a traditional information retrieval step and a kNN search to improve the recall of facts encountered during training. |
| Outcome: | The proposed approach outperforms baselines on cloze-style QA by large margins without further training. |
Translation Deserves Better: Analyzing Translation Artifacts in Cross-lingual Visual Question Answering (2024.findings-acl)
Copied to clipboard
ChaeHun Park, Koanho Lee, Hyesu Lim, Jaeseok Kim, Junmo Park, Yu-Jung Heo, Du-Seong Chang, Jaegul Choo
| Challenge: | Recent studies have employed machine translation systems for cross-lingual VQA tasks . however, translated texts contain unique characteristics distinct from human-written ones, referred to as translation artifacts. |
| Approach: | They propose a machine translation system that can train models in multiple languages . they propose augmentation strategies that reduce translation artifacts in translated texts . |
| Outcome: | The proposed approach reduces translation artifacts in models across languages and languages. |
Mixture Content Selection for Diverse Sequence Generation (D19-1)
Copied to clipboard
| Challenge: | Generating diverse sequences exhibit semantically one-to-many relationships between source and target sequences. |
| Approach: | They propose to separate diversification from generation using a general plug-and-play module that wraps around and guides an existing encoder-decoder model. |
| Outcome: | The proposed method shows that diversification and generation are separate steps in the same model and that the model is robust. |
JOLT-SQL: Joint Loss Tuning of Text-to-SQL with Confusion-aware Noisy Schema Sampling (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models have improved Text-to-SQL methods . however, they still face challenges such as complex multi-stage pipelines and poor robustness to noisy schema information. |
| Approach: | They propose a single-stage SFT framework that optimizes schema linking and SQL generation via a unified loss. |
| Outcome: | Experiments on the Spider and BIRD benchmarks show that JOLT-SQL achieves state-of-the-art execution accuracy among comparable-size open-source models. |
Why Is MBTI Personality Detection from Texts a Difficult Task? (2021.eacl-main)
Copied to clipboard
| Challenge: | Automatic detection of the four MBTI personality dimensions from texts has attracted noticeable attention from the natural language processing and computational linguistic communities. |
| Approach: | They propose to use a questionnaire-based personality assessment to provide more objective assessment of one's personality than traditional questionnaires. |
| Outcome: | The proposed systems rarely outperform the majority-class baseline despite large datasets and high levels of noise in training datasets. |
Narrative Text Generation with a Latent Discrete Plan (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work on story generation has focused on generating consistent stories via story outlines using keywords or key phrases. |
| Approach: | They propose a deep latent variable model that first samples a sequence of anchor words, one per sentence in the story, as part of its generative process. |
| Outcome: | The proposed model gets favorable scores when evaluated on perplexity, diversity, and control of story via discrete plan. |
LSTDial: Enhancing Dialogue Generation via Long- and Short-Term Measurement Feedback (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing dialogue systems do not utilize quality dimensions specifically designed for dialogue evaluation to guide the response generation during training. |
| Approach: | They propose a two-stage framework which generates and utilizes conversation evaluation as explicit feedback during training. |
| Outcome: | The proposed framework generates and utilizes conversation evaluation as explicit feedback during training. |
A Sequence-to-Sequence&Set Model for Text-to-Table Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing models for text-to-table generation are order-insensitive, but suffer from errors . a novel sequence-tosequence&set model generates table body rows in parallel . |
| Approach: | They propose a sequence-to-sequence generation task that serializes each table into a token sequence during training by concatenating all rows in a top-down order. |
| Outcome: | The proposed model outperforms baselines on commonly-used datasets. |
Adapting Meta Knowledge Graph Information for Multi-Hop Reasoning over Few-Shot Relations (D19-1)
Copied to clipboard
| Challenge: | Existing methods for multi-hop reasoning assume that every relation has enough triples for training . however, performance drops significantly on few-shot relations . |
| Approach: | They propose a meta-based multi-hop reasoning method that learns meta parameters from high-frequency relations that could quickly adapt to few-shot scenarios. |
| Outcome: | The proposed method outperforms state-of-the-art methods in few-shot scenarios on two public datasets from Freebase and NELL. |
Are Larger Pretrained Language Models Uniformly Better? Comparing Performance at the Instance Level (2021.findings-acl)
Copied to clipboard
| Challenge: | Larger models have higher out-of-distribution robustness, while smaller models have lower accuracy on rare subgroups. |
| Approach: | They develop statistically rigorous methods to investigate whether large models are better on every instance . they find that individual predictions are highly sensitive to noise in the randomness in training . |
| Outcome: | The proposed model is worse than BERT-MINI on 1-4% of instances across MNLI, SST-2, and QQP, compared to the overall accuracy improvement of 2-10%. |
PEMA: An Offsite-Tunable Plug-in External Memory Adaptation for Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) show impressive performance in various downstream NLP tasks. |
| Approach: | They propose a Parameter-Efficient Fine-Tuning method that integrates with context representations from test data to perform downstream tasks. |
| Outcome: | The proposed method outperforms other methods in memory and latency efficiency and maintains sentence meaning and generating appropriate language and styles. |
On Learning Text Style Transfer with Direct Rewards (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for text style transfer lack parallel corpora, which makes it impossible to train supervised models. |
| Approach: | They propose to use semantic similarity metrics to explicitly assess the preservation of content between system outputs and inputs. |
| Outcome: | The proposed methods provide significant gains in automatic and human evaluation over strong baselines. |
Continual Few-shot Event Detection via Hierarchical Augmentation Networks (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for continual few-shot event detection use labeled data, but in real-world applications, new event types emerge continually. |
| Approach: | They propose a memory-based framework for continual few-shot event detection . they incorporate prototypical augmentation into the memory set to memorize previous event types . |
| Outcome: | The proposed method outperforms existing methods in multiple continual few-shot event detection tasks. |
A Pre-training Strategy for Zero-Resource Response Selection in Knowledge-Grounded Conversations (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to train retrieval-based dialogue systems rely on crowd-sourced data . however, it is difficult to collect large-scale dialogues that are grounded on background knowledge . |
| Approach: | They propose to decompose training of knowledge-grounded response selection into three tasks . they propose to combine query-passage matching task with query-dialogue history matching task . |
| Outcome: | Experimental results show that the proposed model can perform comparable to existing methods . the retrieval-based system can leverage background knowledge when conversing with humans . |
Boundary-Guided Policy Optimization for Memory-efficient RL of Diffusion Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for reinforcement learning (RL) require a large sample size to be implemented. |
| Approach: | They propose a memory-efficient RL algorithm that maximizes a lower bound of the ELBO-based objective. |
| Outcome: | Experiments show that BGPO outperforms previous RL algorithms for diffusion large language models in math problem solving, code generation, and planning tasks. |
APOLLO: A Simple Approach for Adaptive Pretraining of Language Models for Logical Reasoning (2023.acl-long)
Copied to clipboard
Soumya Sanyal, Yichong Xu, Shuohang Wang, Ziyi Yang, Reid Pryzant, Wenhao Yu, Chenguang Zhu, Xiang Ren
| Challenge: | Existing methods to improve logical reasoning skills require complex data processing. |
| Approach: | They propose an adaptive pretraining approach to improve logical reasoning over text . they use a subset of Wikipedia sentences for pretraining and a sentence-level classification loss . |
| Outcome: | The proposed model outperforms baselines on LogiQA and ReClor. |
MultiTabQA: Generating Tabular Answers for Multi-Table Question Answering (2023.acl-long)
Copied to clipboard
| Challenge: | Recent tabular question answering models only answer questions over a single table . multi-table operations often result in tabular outputs . |
| Approach: | They propose a model that answers questions over multiple tables and generalizes to generate tabular answers. |
| Outcome: | The proposed model outperforms state-of-the-art single table QA models on a multi-table QA setting. |
ECG-QALM: Entity-Controlled Synthetic Text Generation using Contextual Q&A for NER (2023.findings-acl)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) requires high-quality labeled datasets. |
| Approach: | They propose a method that uses pre-trained language models to generate entity-controlled text to augment small labeled datasets for downstream NER tasks. |
| Outcome: | The proposed method produces full text samples with desired entities appearing in a controllable way while retaining sentence coherence closest to the real world data. |
Dynamic Curriculum Learning for Low-Resource Neural Machine Translation (2020.coling-main)
Copied to clipboard
| Challenge: | Recent work on neural machine translation (NMT) has demonstrated impressive performance improvements and became the de-facto standard. |
| Approach: | They propose a dynamic curriculum learning method to reorder training samples in training using a Transformer-based system. |
| Outcome: | The proposed method outperforms baselines on three low-resource machine translation benchmarks and different sized data of WMT’16 En-De. |
Data-to-text Generation by Splicing Together Nearest Neighbors (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on data-to-text generation relies on retrieved "neighbors" but instead generates text token-by-token, left-to right. |
| Approach: | They propose to splice together retrieved segments of text from "neighbor" source-target pairs to generate text token-by-token, left-to-right. |
| Outcome: | The proposed method performs on par with strong baselines in terms of automatic and human evaluation, but allows for more interpretable and controllable generation. |
KPDROP: Improving Absent Keyphrase Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to generate absent keyphrases have limited results in generating present keyphrase. |
| Approach: | They propose a model-agnostic approach to generate absent keyphrases by randomly dropping present keyphrase from a document and turning them into artificial absent key phrases during training. |
| Outcome: | The proposed approach improves absent keyphrase generation in both supervised and resource-constrained settings. |
INVITE: a Testbed of Automatically Generated Invalid Questions to Evaluate Large Language Models for Hallucinations (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in Large language models have enabled them to hold free form conversations over multiple turns, but they exhibit a tendency to make unfounded and incorrect statements, commonly labeled as hallucinations. |
| Approach: | They propose a framework to test large language models for hallucinations using automatically generated INValId questions. |
| Outcome: | The proposed framework is based on a testbed of automatically generated INValId questions to evaluate large language models for hallucinations. |
A Surprisingly Effective Fix for Deep Latent Variable Modeling of Text (D19-1)
Copied to clipboard
| Challenge: | Variational Autoencoders are powerful language models and effective representation learning frameworks. |
| Approach: | They propose a fix for posterior collapse which improves held-out likelihood, reconstruction and latent representation learning . |
| Outcome: | The proposed fix significantly improves held-out likelihood, reconstruction, and latent representation learning compared with previous state-of-the-art methods. |
LaCo: Large Language Model Pruning via Layer Collapse (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for model quantization, knowledge distillation, and model pruning are limited by hardware support limitations and the need for extensive training. |
| Approach: | They propose a layer-wise structured pruner that collapses rear model layers into a prior layer and enables a rapid reduction in model size while preserving the model structure. |
| Outcome: | The proposed pruner outperforms state-of-the-art pruning methods at pruning ratios of 25-30% and maintains an average task performance of over 80% at different pruning ratio. |
Parameter-Efficient Transfer Learning with Diff Pruning (2021.acl-long)
Copied to clipboard
| Challenge: | Pretrained networks are difficult to deploy for multiple tasks in storage-constrained settings. |
| Approach: | Diff pruning enables parameter-efficient transfer learning that scales well with new tasks. |
| Outcome: | Diff pruning can match the performance of finetuned baselines on the GLUE benchmark while only modifying 0.5% of the pretrained model’s parameters per task. |
Invariant Language Modeling (2022.emnlp-main)
Copied to clipboard
Maxime Peyrard, Sarvjeet Ghotra, Martin Josifoski, Vidhan Agarwal, Barun Patra, Dean Carignan, Emre Kiciman, Saurabh Tiwary, Robert West
| Challenge: | Existing methods to remove spurious correlations and biases involve expensive domain alignment. |
| Approach: | They propose a framework for learning invariant representations that generalize better across environments . they adapt a game-theoretic implementation of IRM to language models . |
| Outcome: | The proposed framework can remove structured noise, ignore correlations and achieve better generalization across environments. |
Beyond Rejection Sampling: Trajectory Fusion for Scaling Mathematical Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) fine-tuned using rejection sampling retain only correct reasoning trajectories . however, this paradigm treats supervision as a binary filter that systematically excludes teacher-generated errors, leaving a gap in how reasoning failures are modeled during training. |
| Approach: | They propose a fine-tuning strategy that reframes rejection sampling as a structured supervision construction process. |
| Outcome: | The proposed approach outperforms RFT on multiple math benchmarks while retaining only correct reasoning trajectories. |
Distilling Calibrated Knowledge for Stance Detection (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for stance detection ignore meaningful signals among categories offered by hard labels. |
| Approach: | They propose to use knowledge distillation to calibrate teacher predictions in each generation step. |
| Outcome: | The proposed method can calibrate teacher predictions in each generation step and improves stance detection accuracy. |
HCLD: A Hierarchical Framework for Zero-shot Cross-lingual Dialogue System (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods to train task-oriented dialogue systems in monolingual datasets are expensive to build. |
| Approach: | They propose a hierarchical framework to classify intents in high-level and slot filling in low-level . they incorporate sentence-level alignment among different languages to enhance intent detection . |
| Outcome: | The proposed framework achieves the performance on a public task-oriented dialog dataset. |
Debiasing Multi-Entity Aspect-Based Sentiment Analysis with Norm-Based Data Augmentation (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent research has explored strategies for reduce measurable biases in NLP predictions while maintaining prediction accuracy on held-out test sets. |
| Approach: | They propose to augment training data with norm-based language templates derived from previous language resources to reduce biases in NLP models. |
| Outcome: | The proposed model reduces topical bias to less than half while maintaining prediction quality on held-out test sets. |
A Weak Supervision Approach for Predicting Difficulty of Technical Interview Questions (2022.coling-1)
Copied to clipboard
| Challenge: | Existing models require large volumes of candidate response data to train . Existing approaches require large amounts of candidate data to generate questions and generate models. |
| Approach: | They create a dataset of interview questions with difficulty scores for deep learning and use it to evaluate SOTA models trained using weak supervision. |
| Outcome: | The proposed model improves the difficulty and promise of weak supervision for interview questions and identifies the potential for weak supervision. |
Speculative Sampling in Variational Autoencoders for Dialogue Response Generation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have tried to improve variational models but they fail to learn proper mappings. |
| Approach: | They propose to use a variable-based sampling technique to find the most probable one from redundantly sampled latent variables to tie up the variable with a given response. |
| Outcome: | The proposed method is effective in response generation with massive dialogue data constructed from Twitter posts. |
Rethinking Complex Neural Network Architectures for Document Classification (N19-1)
Copied to clipboard
| Challenge: | Neural network models for many NLP tasks have grown increasingly complex in recent years . authors of recent papers question the necessity of such architectures and find them quite effective . |
| Approach: | They propose to use regularization techniques borrowed from language modeling to improve model accuracy . they find that a simple biLSTM architecture with appropriate regularization yields competitive results . |
| Outcome: | a simple biLSTM model outperforms the state-of-the-art on four benchmark datasets . authors say that improvements are not real, but are attributed to mundane reasons . |
FreeCtrl: Constructing Control Centers with Feedforward Layers for Learning-Free Controllable Text Generation (2024.acl-long)
Copied to clipboard
| Challenge: | Controllable text generation (CTG) focuses on crafting texts adhering to specific attributes . studies show learning-based methods require extensive computational and data resources . |
| Approach: | They propose a learning-free approach that dynamically adjusts the weights of selected feedforward neural network vectors to steer the outputs of large language models. |
| Outcome: | The proposed approach outperforms learning-based and learning-free methods on multi-attribute control. |
Teaching Models to Balance Resisting and Accepting Persuasion (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are susceptible to persuasion, which can pose risks when faced with an adversarial interlocutor. |
| Approach: | They propose a method to balance positive and negative persuasion by using recursive dialogue trees to train models to accept persulasion. |
| Outcome: | The proposed model-based training improves resistance to misinformation and resilience to being challenged while also resulting in the best overall performance on multi-agent debates across two domains. |
Enhancing Attributed Question Answering using Tailored Progressive Curriculum Learning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Attributed Question Answering models are not yet leveraged to enhance their essential capabilities, including evidence identification, cross-source relation recognition and anti-distraction reasoning. |
| Approach: | They propose a progressive progressive curriculum learning approach that optimizes both encoder-decoder and decoder-only AQA models. |
| Outcome: | The proposed approach improves both encoder-decoder and decoder-only AQA models on the quotesum benchmark. |
ZEN: Pre-training Chinese Text Encoder Enhanced by N-gram Representations (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Experimental results show that pre-trained text encoders can perform many NLP tasks with less resource. |
| Approach: | They propose a BERT-based Chinese text encoder enhanced by n-gram representations . they show reasonable performance when ZEN is trained on a small corpus . |
| Outcome: | The proposed encoder incorporates the comprehensive information of both the character sequence and words or phrases it contains. |
Shortformer: Better Language Modeling using Shorter Inputs (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods require computationally expensive relative position embeddings. |
| Approach: | They propose two methods that decrease input length to improve perplexity and perplexability. |
| Outcome: | The proposed methods speed up training by a factor of 1.65 and reduce memory usage. |
OIE@OIA: an Adaptable and Efficient Open Information Extraction Framework (2022.acl-long)
Copied to clipboard
| Challenge: | Different Open Information Extraction (OIE) tasks require different types of information. |
| Approach: | They propose to adapt an OIE Graph to different OIE tasks with simple rules . they implement an end-to-end OIA generator and make it open-accessible . |
| Outcome: | The proposed system achieves new SOTA performance on three popular OIE tasks. |
Investigating the Benefits of Free-Form Rationales (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that crowdsourced rationales provide additional background knowledge to models . a qualitative study shows generated rationale is not as useful for humans as crowdsourced ones . |
| Approach: | They investigate whether crowdsourced rationales provide additional background knowledge to models . they find that ECQA rationale provides additional background information to understand a decision . |
| Outcome: | The results show that ECQA rationales provide additional background knowledge to understand a decision . compared to crowdsourced rationale, generated rationale is not as useful for humans . |
Will It Still Be True Tomorrow? Multilingual Evergreen Question Classification to Improve Trustworthy QA (2025.emnlp-main)
Copied to clipboard
Sergey Pletenev, Maria Marina, Nikolay Ivanov, Daria Galimzianova, Nikita Krayko, Mikhail Salnikov, Vasily Konovalov, Alexander Panchenko, Viktor Moskvoretskii
| Challenge: | Large Language Models (LLMs) often struggle with question answering due to hallucinated answers. |
| Approach: | They propose a multilingual QA dataset with evergreen labels that can be used to evaluate and train large language models. |
| Outcome: | The proposed model performs well on 12 modern LLMs and EG-E5 classifiers. |
LEMON: Reviving Stronger and Smaller LMs from Larger LMs with Linear Parameter Fusion (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods to train a stronger and smaller model with the help of large models are limited by the model size and performance. |
| Approach: | They propose to learn competent initial points for smaller models by fusing parameters from larger models and introduce controllable receptive fields to model prior parameter characteristics. |
| Outcome: | The proposed method outperforms baselines in terms of effectiveness and efficiency. |
Learning to Ask Conversational Questions by Optimizing Levenshtein Distance (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for estimating maximum likelihood are limited by easily learned tokens . Existing systems that generate questions based on dialogue context are limited in their ability to learn tokens. |
| Approach: | They propose a framework that optimizes the minimum Levenshtein distance through explicit editing actions. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on two benchmark datasets and generalizes well on unseen data. |
Zero- and Few-Shot Event Detection via Prompt-Based Meta Learning (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for event detection often fail to detect unseen or rare events due to the lack of domain knowledge. |
| Approach: | They propose a meta learning-based framework for zero-shot event detection that uses a prompt-based prompt and a trigger-aware soft verbalizer to efficiently project output to unseen tasks. |
| Outcome: | The proposed framework performs state-of-the-art in zero-shot and few-shot scenarios on benchmark datasets FewEvent and MAVEN. |
Improving Language Model Integration for Neural Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to integrate external language models into machine translation systems have been based on the assumption that the external model learns an implicit target-side language model at decoding time. |
| Approach: | They transfer this concept to the task of machine translation and compare it with the most prominent way of including additional monolingual data - namely back-translation. |
| Outcome: | The proposed approach outperforms the most prominent way of including additional monolingual data, namely back-translation. |
Phrase-Level Action Reinforcement Learning for Neural Dialog Response Generation (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for dialog agent training lack a robust action space for entangled information, which can cause bias and deviate from natural human language. |
| Approach: | They propose phrase-level action reinforcement learning which allows the model to alter the sentence structure and content with the sequential action selection. |
| Outcome: | The proposed model achieves competitive results with state-of-the-art models on the MultiWOZ dataset, indicating that it is effective for solving task-oriented dialogs. |
Agents generalize to novel levels of abstraction by using adaptive linguistic strategies (2025.findings-acl)
Copied to clipboard
| Challenge: | Abstract: Abstracts are fundamental to building well-generalizing models. |
| Approach: | They propose to use a concept-level reference game to generalize concepts . they find that agents can learn robust concepts based on which they can generalize . |
| Outcome: | The proposed model can generalize from generic to very specific concepts, while reusing many messages from training. |
LLM-OREF: An Open Relation Extraction Framework Based on Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies focus on building models that can only handle predefined relations . however, their reliance on human annotation limits their practicality . |
| Approach: | They propose an open relation extraction framework that can generalize to new relations not encountered during training. |
| Outcome: | The proposed framework can generalize to new relations not encountered during training. |
Modeling Noise in Paraphrase Detection (2022.lrec-1)
Copied to clipboard
| Challenge: | Noisy labels in training data are challenging and can lead to incorrect decisions . large pre-trained language models have achieved great results in many NLP tasks . |
| Approach: | They propose to use a linear noise model to augment pre-trained language models to account for label noise in fine-tuning. |
| Outcome: | The proposed model can be applied without further knowledge about annotation quality and label confidence of training examples and their results are compared with other models. |
Unsupervised Opinion Summarization as Copycat-Review Generation (2020.acl-main)
Copied to clipboard
| Challenge: | Recent work on opinion summarization has focused on extracting fragments from reviews, but we use novel sentences to generate abstractive summaries. |
| Approach: | They propose an abstractive summarizer which does not use summaries in training and is trained end-to-end on a large collection of reviews. |
| Outcome: | The proposed model produces fluent and coherent summaries reflecting consensus opinions on Amazon and Yelp reviews. |
R2D2: Robust Data-to-Text with Replacement Detection (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to mitigate unfaithful text generation are inadequate . data-to-text generation requires a structured input format . |
| Approach: | They propose a training framework that addresses unfaithful Data-to-Text generation by training a system as a generator and faithfulness discriminator with additional replacement detection and unlikelihood learning tasks. |
| Outcome: | The proposed training framework improves FeTaQA, LogicNLG, and ToTTo fidelity on D2T systems. |
Lost in Inference: Rediscovering the Role of Natural Language Inference for Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | In the recent past, a popular way of evaluating natural language understanding was to consider a model’s ability to perform natural language inference (NLI) tasks. |
| Approach: | They focus on five different NLI benchmarks across six models of different scales and examine how their accuracies develop during training. |
| Outcome: | The softmax distributions of models align with human label distributions in cases where statements are ambiguous or vague. |
Diverse Distributions of Self-Supervised Tasks for Meta-Learning in NLP (2021.emnlp-main)
Copied to clipboard
| Challenge: | Meta-learning considers learning as an efficient learning process that can leverage its past experience to accurately solve new tasks. |
| Approach: | They propose to provide task distributions for meta-learning by considering self-supervised tasks automatically proposed from unlabeled text to enable large-scale meta- learning in NLP. |
| Outcome: | The proposed distributions show that human learning models perform better on the few-shot benchmark than previous methods. |
Zero-Shot Learners for Natural Language Understanding via a Unified Multiple Choice Perspective (2022.emnlp-main)
Copied to clipboard
Ping Yang, Junjie Wang, Ruyi Gan, Xinyu Zhu, Lin Zhang, Ziwei Wu, Xinyu Gao, Jiaxing Zhang, Tetsuya Sakai
| Challenge: | Existing approaches to zero-shot learning are format-agnostic and can address new learning tasks without additional training. |
| Approach: | They propose a new paradigm for zero-shot learning that is format agnostic and compatible with any format and applicable to a list of language tasks. |
| Outcome: | The proposed model shows state-of-the-art performance on several benchmarks and produces satisfactory results on tasks such as text classification and commonsense reasoning. |
Learning to Rank for Plausible Plausibility (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to encoding sentences using contextualized encoders are inconsistent . |
| Approach: | They propose to use a cross entropy log-loss objective to improve plausibility . they propose a margin-based loss leads to a more plausible model of plausability . |
| Outcome: | The proposed loss is intuitively wrong when applied to plausibility tasks . the proposed loss leads to a more plausible model of plausability . |
Precisely the Point: Adversarial Augmentations for Faithful and Informative Text Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for text generation are weak enough to handle perturbations in inputs, leading to degeneration in faithfulness and informativeness. |
| Approach: | They propose a framework for improving faithfulness and informativeness of Seq2Seq models by perturbing word representations and word swapping. |
| Outcome: | The proposed framework improves faithfulness and informativeness of Seq2Seq models under automatic and human evaluation settings. |
Exploiting Narrative Context and A Priori Knowledge of Categories in Textual Emotion Classification (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for recognizing the mental state of characters in text are limited by their use of character-specific contexts. |
| Approach: | They propose a method that encodes the preceding context of the target sentence along with the target phrase using a BERT-based text encoder. |
| Outcome: | The proposed method improves the accuracy of emotion classification by encoding the preceding context of the target sentence along with the target phrase using a BERT encoder. |
Simple and Effective Curriculum Pointer-Generator Networks for Reading Comprehension over Long Narratives (P19-1)
Copied to clipboard
Yi Tay, Shuohang Wang, Anh Tuan Luu, Jie Fu, Minh C. Phan, Xingdi Yuan, Jinfeng Rao, Siu Cheung Hui, Aston Zhang
| Challenge: | Using a pointer-generator framework for reading/sampling over large documents, we propose a framework for learning over long narratives where documents easily span over thousands of tokens. |
| Approach: | They propose a curriculum learning (CL) based pointer-generator framework for reading/sampling over large documents, enabling diverse training of the neural model based on the notion of alternating contextual difficulty. |
| Outcome: | The proposed framework improves on the NarrativeQA reading comprehension benchmark and reaches state-of-the-art performance. |
FIRST: Faster Improved Listwise Reranking with Single Token Decoding (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing listwise LLMs lack efficiency as they provide ranking output in the form of a generated ordered sequence of candidate passage identifiers. |
| Approach: | They propose a listwise LLM reranking approach that leverages the first generated identifier to obtain a ranked ordering of the candidates. |
| Outcome: | The proposed approach accelerates inference by 50% while maintaining robust ranking performance with gains across BEIR benchmark. |
Tutoring Helps Students Learn Better: Improving Knowledge Distillation for BERT with Tutor Network (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge distillation approaches for language models have overlooked the difficulty of training examples. |
| Approach: | They propose a framework that controls difficulty of training examples during pre-training by a tutor network. |
| Outcome: | The proposed framework outperforms state-of-the-art KD methods with student models on the GLUE benchmark. |
Synthesize, Prompt and Transfer: Zero-shot Conversational Question Generation with Pre-trained Language Model (2023.acl-long)
Copied to clipboard
| Challenge: | Existing research on QG focuses on generating single-turn questions, which are formalized as independent interactions. |
| Approach: | They propose a multi-stage knowledge transfer framework to leverage knowledge from single-turn question generation instances. |
| Outcome: | The proposed framework achieves 14.81 BLEU-4 (88.2% absolute improvement compared to T5) in CoQA with knowledge transferred from three single-turn datasets. |
OpinionDigest: A Simple Framework for Opinion Summarization (2020.acl-main)
Copied to clipboard
| Challenge: | Abstractive opinion summarization framework outperforms competitors' summarizing frameworks . extractive approaches produce well-formed text, but selecting the most popular opinions is challenging . |
| Approach: | They propose an abstractive opinion summarization framework that trains a Transformer model to reconstruct reviews from extracted opinions. |
| Outcome: | The proposed framework outperforms baselines on Yelp and shows promising customization capabilities. |
QA-NatVer: Question Answering for Natural Logic-based Fact Verification (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has focused on natural logic, which operates directly on natural language by capturing the semantic relation of spans between an aligned claim and its evidence via set-theoretic operators. |
| Approach: | They propose to use question answering to predict natural logic operators using generalization capabilities of instruction-tuned language models. |
| Outcome: | The proposed approach outperforms the best baseline on a Danish verification dataset by 4.3 accuracy points. |
Self-Training Meets Consistency: Improving LLMs’ Reasoning with Consistency-Driven Rationale Evaluation (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing approaches labeled rationales that produce correct answers as appropriate for training but one measure risks misjudging rationale quality, leading models to learn flawed reasoning patterns. |
| Approach: | They propose a framework that evaluates rationales through follow-up questions and leverages this evaluation to guide its training. |
| Outcome: | The proposed framework improves robustness and correctness of rationales and reasoning abilities compared to previous self-training approaches. |
Fast Forwarding Low-Rank Training (2024.emnlp-main)
Copied to clipboard
| Challenge: | Modern optimizers provide a spectacular array of tweaks to stabilize training trajectories and accelerate Stochastic Gradient Descent (SGD). |
| Approach: | They propose a fast-forward approach to accelerate large segments of SGD training . they alternate between Adam SGD for burn-in and accelerating by line search . |
| Outcome: | The proposed approach speeds up training without compromising model performance. |
Learning Non-Autoregressive Models from Search for Unsupervised Sentence Summarization (2022.acl-long)
Copied to clipboard
| Challenge: | Text summarization aims to generate a short summary for an input text. |
| Approach: | They propose a non-autoregressive unsupervised summarization approach which performs edit-based search towards a heuristicically defined score and generates a summary as pseudo-groundtruth. |
| Outcome: | The proposed approach achieves state-of-the-art performance for unsupervised summarization, while improving inference efficiency. |
CommonIT: Commonality-Aware Instruction Tuning for Large Language Models via Data Partitions (2024.emnlp-main)
Copied to clipboard
| Challenge: | Current studies have focused on fine-tuning, but the use of instruction tuning is not as effective as fine-cuning. |
| Approach: | They propose a commonality-aware instruction tuning strategy to cluster instruction datasets into distinct groups with three proposed metrics Task, Embedding and Length. |
| Outcome: | The proposed strategy boosts an average improvement of 2.1% on the general domain and 5.2% on the special domain. |
Assessing Word Importance Using Models Trained for Semantic Tasks (2023.findings-acl)
Copied to clipboard
| Challenge: | Many NLP tasks require to automatically identify the most significant words in a text. |
| Approach: | They propose to use attribution methods to explain the predictions of two NLP tasks to derive word significance from models trained to solve semantic tasks. |
| Outcome: | The proposed method is robust to the initial task and is able to identify important words in sentences without explicit word importance labeling in training. |
Bootstrapping LLM-based Task-Oriented Dialogue Agents via Self-Talk (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are powerful dialogue agents, but specializing them towards fulfilling a specific function can be prohibitive in terms of feasibility, time, and resources. |
| Approach: | They propose a method for training large language models by enabling "self-talk" they propose supervised fine-tuning of LLMs to improve quality of dialogues . |
| Outcome: | The proposed method generates training data via "self-talk" of LLMs that can be refined and utilized for supervised fine-tuning. |
Generalizable Implicit Hate Speech Detection Using Contrastive Learning (2022.coling-1)
Copied to clipboard
| Challenge: | Hate speech detection is challenging when there are insufficient lexical cues. |
| Approach: | They propose a contrastive learning method that pulls an implication and its corresponding posts close in representation space. |
| Outcome: | The proposed method improves on BERT and HateBERT benchmarks on three implicit hate speech benchmarks. |
Simultaneous Translation with Flexible Policy via Restricted Imitation Learning (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to simultaneous translation have been limited and use fixed-latency policies or a complicated two-staged model. |
| Approach: | They propose a single model that adds a “delay” token to the target vocabulary and a restricted dynamic oracle to greatly simplify training. |
| Outcome: | The proposed model achieves better BLEU scores and lower latencies compared to fixed and RL-learned policies on Chinese -> English simultaneous translation. |
Sparsifying Transformer Models with Trainable Representation Pooling (2022.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to sparsify attention in the Transformer model are based on quadratic memory complexity and a lack of information for each word. |
| Approach: | They propose a method to sparsify attention in a Transformer model by learning to select the most-informative token representations during the training process. |
| Outcome: | The proposed model performs better than the current SOTA model while being 1.8 faster during training, 4.5 faster inference and 13 more efficient in the decoder. |
CriticSearch: Fine-Grained Credit Assignment for Search Agents via a Retrospective Critic (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing search agent pipelines rely on sparse outcome rewards, leading to inefficient exploration and unstable training. |
| Approach: | They propose a tool-integrated reasoning framework that provides turn-level feedback via a retrospective critic mechanism. |
| Outcome: | The proposed framework outperforms baselines in multi-hop reasoning benchmarks and achieves faster convergence and training stability. |
Low-Resource Generation of Multi-hop Reasoning Questions (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods to generate valid and fluent questions from text are limited and insufficient for training. |
| Approach: | They propose to generate multi-hop reasoning questions from the raw text in a low resource circumstance by deducing over multiple relations on several sentences in the text. |
| Outcome: | The proposed model can be applied to the task of machine reading comprehension and achieve significant performance improvements. |
Retrieving, Rethinking and Revising: The Chain-of-Verification Can Improve Retrieval Augmented Generation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent Retrieval Augmented Generation (RAG) aims to enhance Large Language Models . however, such approach can generate inconsistent answer with external references . |
| Approach: | They propose to integrate the verification module into the RAG to improve external retrieval correctness and internal generation consistency. |
| Outcome: | The proposed model can significantly surpass the state-of-the-art baselines using different LLM backbones. |
LM-Critic: Language Models for Unsupervised Grammatical Error Correction (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent work casts GEC as a translation problem using encoder-decoder models to map bad (ungrammatical) sentences into good (grammatically) sentences. |
| Approach: | They propose to use a pretrained language model to define an LM-Critic that judges a sentence to be grammatical if the LM assigns it a higher probability than its local perturbations. |
| Outcome: | The proposed method outperforms existing methods in both the unsupervised and supervised setting. |
FalAI: A Dataset for End-to-end Spoken Language Understanding in a Low-Resource Scenario (2024.lrec-main)
Copied to clipboard
Andres Pineiro-Martin, Carmen Garcia-Mateo, Laura Docio-Fernandez, Maria del Carmen Lopez-Perez, Jose Gandarela-Rodriguez
| Challenge: | End-to-end (E2E) Spoken Language Understanding systems extract structured information from speech signals using a single model. |
| Approach: | They propose to use a dataset to extract structured information from speech signals . they define splits for noisy audio, hesitant audio and audio where sentence has changed . |
| Outcome: | The proposed model exploits acoustic information and avoids cascading errors . falAI dataset is the largest public SLU dataset in Galician and first to be obtained in low-resource scenario. |
Teaching According to Talents! Instruction Tuning LLMs with Competence-Aware Curriculum Learning (2025.findings-emnlp)
Copied to clipboard
Yangning Li, Tingwei Lu, Yinghui Li, Yankai Chen, Wei-Chieh Huang, Wenhao Jiang, Hui Wang, Hai-Tao Zheng, Philip S. Yu
| Challenge: | Efficient instruction tuning aims to enhance the ultimate performance of large language models (LLMs) current methods suffer from the curriculum rigidity, resulting in a fixed and potentially sub-optimal learning trajectory. |
| Approach: | a framework for efficient instruction tuning is proposed to address the issue of curriculum rigidity . current methods rely on static heuristic difficulty metrics and fail to adapt to evolving capabilities . |
| Outcome: | Efficient instruction tuning aims to enhance the ultimate performance of large language models . current methods suffer from the curriculum rigidity, resulting in a fixed learning trajectory . |
Unveiling the Role of Pretraining in Direct Speech Translation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to train direct speech-to-text translation systems are pretraining the encoder on automatic speech recognition, thus losing efficiency in the training process. |
| Approach: | They propose to change the decoder cross-attention to integrate source information from earlier steps in training. |
| Outcome: | The proposed model can achieve comparable performance to the pretrained model while reducing training time. |
TroL: Traversal of Layers for Large Language and Vision Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing open-source LLVMs that perform comparably to closed-source models such as GPT-4V are often considered too large, having a larger number of layers. |
| Approach: | They propose a new efficient LLVM family with 1.8B, 3.8B, and 7B LLM model sizes, Traversal of Layers, which enables the reuse of layers in a token-wise manner. |
| Outcome: | The proposed model outperforms open-source models with larger model sizes and outperformed closed-source LLVMs with substantial models. |
Honey or Poison? Solving the Trigger Curse in Few-shot Event Detection via Causal Intervention (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent supervised ED approaches have achieved promising performance but require large number of manually annotated event data. |
| Approach: | They propose to overfit the trigger confounder of the context and the result . they propose to intervene on the context via backdoor adjustment during training . |
| Outcome: | The proposed method significantly improves the FSED on ACE05 and MAVEN datasets. |
DictDis: Dictionary Constrained Disambiguation for Improved NMT (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to domain-specific neural machine translation (NMT) are lexically constrained and draw from domain- specific dictionaries. |
| Approach: | They propose a lexically constrained neural machine translation system that disambiguates between multiple dictionary candidates. |
| Outcome: | The proposed system disambiguates between multiple candidate translations derived from dictionaries on English-Hindi, English-German, and English-French datasets. |
Sequence Length is a Domain: Length-based Overfitting in Transformer Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Current Transformer-based sequence-to-sequence architectures can suffer from overfitting during training. |
| Approach: | They propose to use Transformer-based sequence-to-sequence architectures to overcome overfitting problems when generating very long sequences. |
| Outcome: | The proposed model performs worse on very long sequences than previous approaches on string editing and translation tasks when faced with sequences of length diverging from the length distribution in training data. |
Evaluating and Improving Factuality in Multimodal Abstractive Summarization (2022.emnlp-main)
Copied to clipboard
| Challenge: | Current factuality metrics do not account for vision modality, thus are not adequate for vision-and-language summarization. |
| Approach: | They propose a weighted combination of CLIPScore and BERTScore to evaluate factuality for abstractive document summarization. |
| Outcome: | The proposed metric outperforms existing factuality metrics on four factuity metric-evaluation benchmarks and is robust to human judgments. |
CE-DA: Custom Embedding and Dynamic Aggregation for Zero-Shot Relation Extraction (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods to predict relationships with given entity pairs are lacking in supervised methods. |
| Approach: | They propose a framework for zero-shot Relation Extraction that includes two modules: Custom Embedding and Dynamic Aggregation. |
| Outcome: | The proposed framework shows competitive performance on two ZSRE datasets. |
Language Modeling, Lexical Translation, Reordering: The Training Process of NMT through the Lens of Classical SMT (2021.emnlp-main)
Copied to clipboard
| Challenge: | Statistical MT decomposes the translation task into distinct components that are learned separately. |
| Approach: | They show that neural machine translation models acquire different competences over the course of training . previous work shows how to improve some of the competences in NMT by using lexical translation probabilities, phrase memories, alignment information. |
| Outcome: | The proposed model improves translation quality and word-by-word translation, while learning complex reordering patterns. |
Multi-Level Cross-Modal Alignment for Speech Relation Extraction (2024.emnlp-main)
Copied to clipboard
Liang Zhang, Zhen Yang, Biao Fu, Ziyao Lu, Liangying Shao, Shiyu Liu, Fandong Meng, Jie Zhou, Xiaoli Wang, Jinsong Su
| Challenge: | Existing studies use synthetic speech to train and evaluate SpeechRE models, hindering their development . modality gap issue limits performance of existing models, limiting future researches . |
| Approach: | They propose to use speech data to train and evaluate SpeechRE models by using real speech . they propose to train a cross-modal alignment model to bridge the modality gap . |
| Outcome: | The proposed model can train to bridge the modality gap between speech encoder and text decoder . the proposed model is based on two real SpeechRE datasets . |
Agent Lumos: Unified and Modular Training for Open-Source Language Agents (2024.acl-long)
Copied to clipboard
Da Yin, Faeze Brahman, Abhilasha Ravichander, Khyathi Chandu, Kai-Wei Chang, Yejin Choi, Bill Yuchen Lin
| Challenge: | Lumos is a framework for training open-source agents on complex interactive tasks. |
| Approach: | They propose a framework for training open-source LLM-based agents called Lumos . Lumos features a learnable, unified and modular architecture with a planning module that learns high-level subgoal generation and a grounding module trained to translate these into the actions using various tools in the execution module. |
| Outcome: | The framework outperforms open-source agents on QA and web tasks. |
Curating Datasets for Better Performance with Example Training Dynamics (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to improve data quality but rely on data quantity to improve performance are not effective. |
| Approach: | They propose a method for weighing the relative importance of examples in a dataset based on their Example Training dynamics (ETD) they propose an active learning approach for computing ETD during training rather than as a preprocessing step. |
| Outcome: | The proposed method can be used to improve performance in in-distribution and out-of-distortion testing. |
FAITH: Factuality Alignment through Integrating Trustworthiness and Honestness (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to correct factually inaccurate outputs are lacking the semantic richness needed to properly understand its internal states of trustworthiness and honesty. |
| Approach: | They propose a framework for factuality alignment that integrates natural-language uncertainty signals with external knowledge and computes confidence scores and semantic entropy from LLM outputs. |
| Outcome: | Extensive experiments on four knowledge-intensive benchmarks show that FAITH improves the factual accuracy and truthfulness of Large Language Models (LLMs). |
Multi-task Adversarial Attacks against Black-box Model with Few-shot Queries (2025.acl-long)
Copied to clipboard
| Challenge: | Existing adversarial text attacks rely on abundant access to shared internal features and numerous queries, limited to a single task type. |
| Approach: | They propose a black-box attack that exploits the transferability of adversarial texts . they use a deep-level substitute model trained in a plug-and-play manner for text classification . |
| Outcome: | The proposed attack can target multiple tasks with minimal perturbations . it can target commercial APIs, large language models, and image-generation models . |
MuTIS: Enhancing Reasoning Efficiency through Multi Turn Intervention Sampling in Reinforcement Learning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for training large reasoning models with long chain-of-thought (CoT) are limited by the number of parameters and the complexity of the model. |
| Approach: | They propose a framework that leverages multi-turn interventions to produce concise reasoning chains and demonstrates strong scalability. |
| Outcome: | The proposed framework breaks the accuracy-efficiency trade-off by producing concise reasoning chains and demonstrating strong scalability on 7B models. |
Using Context in Neural Machine Translation Training Objectives (2020.acl-main)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) training is based on document-level metrics, not sentence-level BLEU. |
| Approach: | They propose to merge document-level metrics with batch-level documents to improve NMT training. |
| Outcome: | The proposed training is more robust for document-level metrics than sequence MRT and maximum-likelihood training. |
CapWAP: Image Captioning with a Purpose (2020.emnlp-main)
Copied to clipboard
| Challenge: | a traditional image captioning task uses generic reference captions to provide textual information about images. |
| Approach: | They propose a task that uses question-answer pairs to provide visual information instead of generic reference captions. |
| Outcome: | The proposed captioning with a purpose task can be tailored to meet user needs . question-answer pairs are used as a source of supervision for learning visual information needs a new task is proposed . |
From Selection to Generation: A Survey of LLM-based Active Learning (2025.acl-long)
Copied to clipboard
Yu Xia, Subhojyoti Mukherjee, Zhouhang Xie, Junda Wu, Xintong Li, Ryan Aponte, Hanjia Lyu, Joe Barrow, Hongjie Chen, Franck Dernoncourt, Branislav Kveton, Tong Yu, Ruiyi Zhang, Jiuxiang Gu, Nesreen K. Ahmed, Yu Wang, Xiang Chen, Hanieh Deilamsalehy, Sungchul Kim, Zhengmian Hu, Yue Zhao, Nedim Lipka, Seunghyun Yoon, Ting-Hao Kenneth Huang, Zichao Wang, Puneet Mathur, Soumyabrata Pal, Koyel Mukherjee, Zhehao Zhang, Namyong Park, Thien Huu Nguyen, Jiebo Luo, Ryan A. Rossi, Julian McAuley
| Challenge: | Large Language Models (LLMs) have been used for selection and training of data for active learning. |
| Approach: | They propose an intuitive taxonomy that categorizes LLM-based active learning techniques and discuss the transformative roles they can play in the active learning loop. |
| Outcome: | The proposed model can generate entirely new data instances and provide more cost-effective annotations with fewer labeled data instances. |
PATIENT-𝜓: Using Large Language Models to Simulate Patients for Training Mental Health Professionals (2024.emnlp-main)
Copied to clipboard
Ruiyi Wang, Stephanie Milani, Jamie Chiu, Jiayin Zhi, Shaun Eack, Travis Labrum, Samuel Murphy, Nev Jones, Kate Hardy, Hong Shen, Fei Fang, Zhiyu Chen
| Challenge: | Mental illness remains one of the most critical public health issues. |
| Approach: | They propose a patient simulation framework for cognitive behavior therapy training that uses large language models to act as a simulated therapy patient. |
| Outcome: | The proposed framework improves the skill acquisition and confidence of mental health trainees beyond textbooks, videos, and role-play with non-patients. |
mAggretriever: A Simple yet Effective Approach to Zero-Shot Multilingual Dense Retrieval (2023.emnlp-main)
Copied to clipboard
| Challenge: | MLIR requires human annotations in multiple languages, making training labor-intensive. |
| Approach: | They propose a multilingual information retrieval model that leverages pre-trained multilingual transformers for dense retrieval. |
| Outcome: | Empirical results show that mAggretriever outperforms state-of-the-art models fine-tuned on English training data. |
Minos: A Multimodal Evaluation Model for Bidirectional Generation Between Image and Text (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluation models struggle to achieve consistent performance across image-to-text (I2T) and text-to image (T2I) tasks. |
| Approach: | They construct a multimodal evaluation model using a large multimodal dataset and rigorous quality control strategies to train it. |
| Outcome: | The proposed model achieves state-of-the-art evaluation performance across 16 out-of domain datasets covering both I2T and T2I tasks among all open-source multimodal evaluation models and remain competitive with closed-source models. |
Domain Generalizable AI Guardrails with Augmented Policy Training (2026.acl-long)
Copied to clipboard
| Challenge: | Current guardrails overfit the training policies, preventing adaptation to new domains and policies. |
| Approach: | They propose a training recipe that uses a suite of policy perturbation strategies to reduce overfitting and increase generalization to guardrails. |
| Outcome: | The proposed training recipe reduces overfitting and increases generalization on unseen policies and achieves comparable or better performance than existing 8B guardrails on unsen policies. |
Multi-Domain Named Entity Recognition with Genre-Aware and Agnostic Inference (2020.acl-main)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is a key component of many text processing pipelines. |
| Approach: | They propose a new architecture tailored to the task of identifying named entities with data from multiple genres. |
| Outcome: | The proposed architecture outperforms baseline and competitive methods on all three setups with differences ranging between +1.95 to +3.11 average F1 across multiple genres when compared to standard approaches. |
LlmLink: Dual LLMs for Dynamic Entity Linking on Long Narratives with Collaborative Memorisation and Prompt Optimisation (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods focus on supervised fine-tuning or limited to one-off prediction, which poses a challenge where the context is long. |
| Approach: | They propose a dynamic approach to CoREFerence resolution in chunked long narratives by deploying dual Large Language Models. |
| Outcome: | The proposed model achieves performance gains over existing models and fine-tuning approaches on long narrative datasets, significantly reducing the resources required for inference and training. |
LLM Critics Help Catch Bugs in Mathematics: Towards a Better Mathematical Verifier with Natural Language Feedback (2025.findings-acl)
Copied to clipboard
Bofei Gao, Zefan Cai, Runxin Xu, Peiyi Wang, Ce Zheng, Runji Lin, Keming Lu, Dayiheng Liu, Chang Zhou, Wen Xiao, Tianyu Liu, Baobao Chang
| Challenge: | Existing mathematical verifiers are trained with binary classification labels, which are not informative enough for the model to accurately assess the solutions. |
| Approach: | They propose a natural language feedback-enhanced verifier that can validate the correctness of response generated by policy models by constructing automatically generated training data and a two-stage training paradigm. |
| Outcome: | The proposed verifier significantly improves in verification and reinforcement learning and alleviates data-demanding problems of the reward model. |
Contrastive Domain Adaptation for Question Answering using Limited Text Corpora (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing question generation methods rely on large amounts of synthetically generated datasets and costly computational resources. |
| Approach: | They propose a framework for domain adaptation that combines question generation and domain-invariant learning to answer out-of-domain questions in settings with limited text corpora. |
| Outcome: | The proposed framework improves on state-of-the-art questions in a domain with limited text corpora. |
OneNet: A Fine-Tuning Free Framework for Few-Shot Entity Linking via Large Language Model Prompting (2024.emnlp-main)
Copied to clipboard
| Challenge: | Entity Linking (EL) is the process of associating ambiguous textual mentions to specific entities in a knowledge base. |
| Approach: | They propose a framework that utilizes the few-shot learning capabilities of Large Language Models without the need for fine-tuning to improve the accuracy of EL. |
| Outcome: | The framework outperforms current state-of-the-art methods in a few-shot entity linking task. |
Large Corpus of Czech Parliament Plenary Hearings (2020.lrec-1)
Copied to clipboard
| Challenge: | a corpus of Czech parliament plenary sessions is a valuable resource for future research . only a few public datasets are available in the Czech language . end-to-end approaches require extensive training data to produce competitive results . |
| Approach: | They present a corpus of Czech parliament plenary sessions which is a large corpus . they combine a traditional approach with a more traditional approach . |
| Outcome: | The proposed model architectures can be used to train and evaluate speech recognition systems on a large corpus of speech data and transcripts. |
Aggregating Multiple Heuristic Signals as Supervision for Unsupervised Automated Essay Scoring (2023.acl-long)
Copied to clipboard
| Challenge: | Automated Essay Scoring (AES) aims to evaluate the quality score of input essays without human intervention. |
| Approach: | They propose an unsupervised approach to evaluate the quality of input essays . they use multiple heuristic quality signals as pseudo-groundtruths to train a neural AES model . |
| Outcome: | The proposed approach achieves state-of-the-art performance on eight prompts of ASPA dataset compared with previous unsupervised methods . |
Paraphrasing as Zero-shot Translation with Feature-guided Diversity Enhancement (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies use parallel corpora for training, which results in less diverse paraphrases. |
| Approach: | They train a bidirectional multilingual neural machine translation model on a bilingual parallel corpus and use it as a paraphrasing model. |
| Outcome: | The proposed method generates paraphrases with higher semantic consistency, literal fluency and sentential diversity than existing parabanks and LLMs. |
Exploring Quantization for Efficient Pre-Training of Transformer Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Quantization has proven to be effective after pre-training and during fine-tuning, but its effects on pre-trainer performance have remained unexplored. |
| Approach: | They propose a linear quantization strategy to be applied during the pre-training of Transformers to improve model efficiency and stability. |
| Outcome: | The proposed method improves model efficiency, stability, and performance while maintaining language modeling ability. |
Generating Structured Pseudo Labels for Noise-resistant Zero-shot Video Sentence Localization (2023.acl-long)
Copied to clipboard
| Challenge: | Existing zero-shot pipelines generate event proposals and then generate a pseudo query for each event proposal. |
| Approach: | They propose a Structure-based Pseudo Label generation (SPL) that generates free-form interpretable pseudo queries before constructing query-dependent event proposals. |
| Outcome: | The proposed method learns with only video data without any annotation . it generates free-form interpretable pseudo queries before constructing query-dependent event proposals . |
DATA-CUBE: Data Curriculum for Instruction-based Sentence Representation Learning (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to improve sentence representation learning (SRL) ignore the potential interference problems across tasks and instances. |
| Approach: | They propose a multi-task instruction tuning method that arranges the order of multi- task data for training to minimize interference risks. |
| Outcome: | The proposed method can boost the performance of state-of-the-art methods. |
Learning Language-Specific Layers for Multilingual Machine Translation (2023.acl-long)
Copied to clipboard
| Challenge: | Multilingual Machine Translation (MNMT) is a promising new approach to improve translation quality between non-English languages. |
| Approach: | They propose a language-specific transformer layer to increase model capacity while keeping computation and parameters constant. |
| Outcome: | The proposed approach improves translation quality by 1.3 chrF (1.5 spBLEU) over not using LSLs on a separate decoder architecture. |
PRiSM: Benchmarking Phone Realization in Speech Models (2026.acl-long)
Copied to clipboard
Shikhar Bharadwaj, Chin-Jou Li, Yoonjae Kim, Kwanghee Choi, Eunjung Yeo, Ryan Soh-Eun Shim, Hanyu Zhou, Brendon Boldt, Karen Rosero, Kalvin Chang, Darsh Agrawal, Keer Xu, Chao-Han Huck Yang, Jian Zhu, Shinji Watanabe, David R. Mortensen
| Challenge: | Existing evaluations of phone recognition systems only measure surface-level transcription accuracy. |
| Approach: | They propose to standardize transcription-based evaluation and assess downstream utility in clinical, educational, and multilingual settings with transcription and representation probes. |
| Outcome: | The proposed system outperforms LALMs in clinical, educational, and multilingual settings. |
Token-Level Metrics for Detecting Incorrect Gold Annotations in Named Entity Recognition (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Annotated datasets for supervised learning often contain incorrect labels, i.e. label noise. |
| Approach: | They compare popular sample metrics for detecting incorrect annotations in named entity recognition (NER) they find that training dynamics metrics work the best overall, and they detect errors that the model has not yet memorized . |
| Outcome: | The proposed measures reduce label noise across noise types by detecting errors in trained models. |
BOSE: A Systematic Evaluation Method Optimized for Base Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluation methods for large language models (LLMs) are inadequate to provide solid conclusions for key experiments such as data ablation and scaling law. |
| Approach: | They propose a method specifically designed to optimize the evaluation of base models by incorporating two innovations: In-Context Light-instruction Prompt and Blank-ppl for multi-choice tasks with candidate options. |
| Outcome: | The proposed method significantly improves stability and consistency of evaluations during pre-training and consistency between base and instruct models. |
Balanced Data Sampling for Language Model Training with Clustering (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are a fundamental part of the training process. |
| Approach: | They propose to use clustering to balance the text distribution of training data for better model training. |
| Outcome: | Extensive experiments validate the effectiveness of ClusterClip Sampling under various training datasets and large language models. |
SCOP: Evaluating the Comprehension Process of Large Language Models from a Cognitive View (2025.acl-long)
Copied to clipboard
| Challenge: | despite the potential of large language models, it is difficult to fully count on them in real-world scenarios. |
| Approach: | They propose to examine how LLMs perform during the comprehension process from a cognitive perspective. |
| Outcome: | The proposed model analyzes how LLMs perform during the comprehension process from a cognitive perspective. |
Ladder: A Model-Agnostic Framework Boosting LLM-based Machine Translation to the Next Level (2024.emnlp-main)
Copied to clipboard
| Challenge: | General-purpose Large Language Models (LLMs) like GPT-4 have exhibited strong translation abilities. |
| Approach: | They propose to use a model-agnostic model to refine the performance of general-purpose large-language models for machine translation (MT) by utilizing Gemma-2B/7B as the backbone. |
| Outcome: | The proposed model-agnostic and cost-effective tool improves the performance of general-purpose large-language models for machine translation (MT) by integrating it with any general-use LLM. |
Governance in Motion: Co-evolution of Constitutions and AI models for Scalable Safety (2025.emnlp-main)
Copied to clipboard
Chenhao Huang, Ziyu Shen, Yicong Ren, Huiyuan Zheng, Jiazheng Zhang, Mingxu Chai, Ming Zhang, Shihan Dou, Fan Mo, Jie Shi, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Existing approaches to align large language models with human preferences lack flexibility . static alignment preferences lack the ability to correct misaligned behaviors as they emerge . |
| Approach: | They propose a framework that enables dynamic and continuous alignment of large language models with human preferences. |
| Outcome: | The proposed framework improves safety and accuracy of a 7B model with human annotations. |
Automatic Transmission for LLM Tiers: Optimizing Cost and Accuracy in Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are powerful tools for a wide range of natural language tasks. |
| Approach: | They propose an LLM automatic transmission framework that automatically selects LLM tiers without training. |
| Outcome: | The proposed framework achieves superior performance while reducing costs. |
Sparsity-Accelerated Training for Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated proficiency across various NLP tasks but often require additional training, such as continual pre-training and supervised fine-tuning. |
| Approach: | They propose to leverage sparsity in pre-trained LLMs to accelerate training by disregarding computations for unimportant neurons. |
| Outcome: | The proposed framework achieves comparable or superior performance to standard training while significantly accelerating the process. |
SMoP: Towards Efficient and Effective Prompt Tuning with Sparse Mixture-of-Prompts (2023.emnlp-main)
Copied to clipboard
| Challenge: | Prompt tuning has emerged as a successful parameter-efficient alternative to the full fine-tuning of language models. |
| Approach: | They propose a prompt tuning method that utilizes short soft prompts for efficient training and inference while maintaining performance gains typically induced by longer soft prompt. |
| Outcome: | The proposed method outperforms baseline methods while preserving memory usage. |
DeepMed: Building a Medical DeepResearch Agent via Multi-hop Med-Search Data and Turn-Controlled Agentic Training & Inference (2026.findings-acl)
Copied to clipboard
Zihan Wang, Hao Wang, Shi Feng, Xiaocui Yang, Daling Wang, Yiqun Zhang, Jinghao Lin, Xiaozhong Ji, Haihua Yang
| Challenge: | Medical reasoning models are constrained by parametric knowledge and can induce hallucinations and spurious attributions. |
| Approach: | They propose a model that uses a multi-hop med-search QA synthesis method to apply the DR paradigm in medical contexts. |
| Outcome: | The proposed model outperforms larger medical reasoning models on medical benchmarks. |
Deep Exploration of Cross-Lingual Zero-Shot Generalization in Instruction Tuning (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have focused on instruction tuning to show cross-lingual generalization . a novel non-English meta-dataset is used to study instruction tuning . |
| Approach: | They perform instruction tuning individually for two distinct language meta-datasets and assess the performance on unseen tasks in a non-English language. |
| Outcome: | The proposed model outperforms baseline training in English and Korean by 20.7% and 13.6%. |
Soundwave: Less is More for Speech-Text Alignment in LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Existing end-to-end speech large language models rely on large-scale annotated data for training, while data-efficient training has not been discussed in depth. |
| Approach: | They propose a training strategy and a novel architecture to address representation space gap and sequence length inconsistency in speech and text. |
| Outcome: | The proposed model outperforms other advanced speech LLMs in speech translation and AIR-Bench speech tasks with only a fraction of the training data. |
Group-SAE: Efficient Training of Sparse Autoencoders for Large Language Models via Layer Groups (2025.emnlp-main)
Copied to clipboard
| Challenge: | Sparse Autoencoders (SAEs) are a promising unsupervised approach for understanding the representations of layers of Large Language Models (LLMs). |
| Approach: | They propose a method that groups similar models and trains a single SAE per group based on representational similarity across layers. |
| Outcome: | Experiments on Pythia family models show that the proposed method significantly accelerates training with minimal impact on reconstruction quality and comparable downstream task performance and interpretability over baseline SAEs trained layer by layer. |
BSFA: Leveraging the Subspace Dichotomy to Accelerate Neural Network Training (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies highlight a fundamental dichotomy in deep learning optimization: parameter updates along the top eigendirections of the loss Hessian (Dom-space) capture most of the update magnitude, while updates in the orthogonal component (Bulk-space), have smaller magnitudes but drive most learning progress. |
| Approach: | They propose a plug-and-play framework that scales update components projected onto distinct subspaces and a block-wise strategy that applies this estimation on a per-parameter-block basis. |
| Outcome: | The proposed framework accelerates training by differentially scaling update components projected onto distinct subspaces, while enhancing stability by moderating updates in dominant subspace and boosting convergence speed by amplifying updates in bulk-space. |
Hop, skip, jump to Convergence: Dynamics of Learning Rate Transitions for Improved Training of Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Modern deep neural networks have achieved stateof-the-art performance across a wide range of machine learning tasks. |
| Approach: | They propose to switch the learning rate at a predetermined time during training to improve the performance of large language models. |
| Outcome: | The proposed model shows that switching the learning rate causes the loss curves to contract towards each other. |
Privacy Ripple Effects from Adding or Removing Personal Information in Language Model Training (2025.findings-acl)
Copied to clipboard
Jaydeep Borkar, Matthew Jagielski, Katherine Lee, Niloofar Mireshghallah, David A. Smith, Christopher A. Choquette-Choo
| Challenge: | PII is a sensitive information that can be removed from large-language model training due to evolving curation techniques, or because it was recently scraped for retraining. |
| Approach: | They characterize a phenomenon where PII that appeared earlier in training becomes extractable at a later step after fine-tuning on other PI I. |
| Outcome: | The authors show that PII memorization is a dynamic property of a model that evolves throughout training pipelines and depends on commonly altered design choices. |
MCTS: A Multi-Reference Chinese Text Simplification Dataset (2024.lrec-main)
Copied to clipboard
Ruining Chong, Luming Lu, Liner Yang, Jinran Nie, Zhenghao Liu, Shuo Wang, Shuhan Zhou, Yaoxin Li, Erhong Yang
| Challenge: | Existing studies on text simplification systems have focused on unsupervised methods due to the limited evaluation data in language and domain. |
| Approach: | They propose a Chinese text simplification dataset that provides a detailed analysis and an annotation process. |
| Outcome: | The proposed dataset evaluates the performance of unsupervised methods and advanced large language models. |
LOHRec: Leveraging Order and Hierarchy in Generative Sequential Recommendation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | generative recommenders focus on maximizing the prediction probability of the next item in the temporal sequence, ignoring diverse potential items. |
| Approach: | They propose a learning framework that leverages order and hierarchy in generative recommendation using quantized identifiers to further explore performance ceiling of lightweight generative recommenders. |
| Outcome: | The proposed learning framework outperforms strong prior baselines across multiple datasets. |
Tailoring Diagnostic Modeling to Individual Learners: Personalized Distractor Generation via MCTS-Guided Reasoning Reconstruction (2026.acl-long)
Copied to clipboard
| Challenge: | Current distractor generation methods produce shared distractors for all students, ignoring individual variations in reasoning, which limits their diagnostic effectiveness. |
| Approach: | They propose a method which tailors distractors to each student’s specific cognitive flaws, inferred from their past question-answering (QA) history. |
| Outcome: | The proposed framework outperforms existing methods in generating plausible distractors and adapts to group-level settings. |
d-TreeRPO: Towards More Reliable Policy Optimization for Diffusion Language Models (2026.acl-long)
Copied to clipboard
Leyi Pan, Shuchang Tao, Yunpeng Zhai, Zheyu Fu, Liancheng Fang, Minghua He, Lingzhe Zhang, Zhaoyang Liu, Bolin Ding, Aiwei Liu, Lijie Wen
| Challenge: | Existing RL methods suffer from reliability bottlenecks due to reward sparsity and intractable computations . d-TreeRPO provides fine-grained and verifiable step-wise reward signals . |
| Approach: | They propose a reliable reinforcement learning framework for diffusion large language models that leverages tree-structured rollouts and bottom-up advantage computation based on verifiable outcome rewards. |
| Outcome: | The proposed framework outperforms baseline models and achieves significant improvements across reasoning benchmarks. |
Perplexity-Aware Data Scaling Law: Perplexity Landscapes Predict Performance for Continual Pre-training (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have impressive capabilities across a wide range of domains, but their generalpurpose pre-training objectives often leave them illsuited for specialized applications such as healthcare. |
| Approach: | They propose a perplexity-aware data scaling law that establishes a predictive relationship between the perplexities of domain-specific data and the test loss. |
| Outcome: | Experiments on medical and general-domain benchmarks show that the proposed scaling law consistently identifies near-optimal training subsets with significantly reduced data consumption. |
G-LoRA: Global-Local Decoupled Low-Rank Adaptation (2026.findings-acl)
Copied to clipboard
| Challenge: | Low-Rank Adaptation (LoRA) improves the fine-tuning efficiency and performance of large language models. |
| Approach: | They propose a low-rank adaptive approach that decomposes update matrix into global and local adapters and assigns them to local and global adapters. |
| Outcome: | The proposed method achieves up to 2.7% accuracy improvement over LoRA and its variants on commonsense reasoning, mathematical reasoning, and code generation. |
Fine-Grained Data Ordering Improves Fine-Tuning for Large Language Models (2026.findings-acl)
Copied to clipboard
Xiaomeng Hu, Yixuan Tang, Haoze Li, Hao Chen, Qi Zhang, Zhanming Shen, Yiming Zhang, Haobo Wang, Junbo Zhao
| Challenge: | Prior work focused on data preprocessing, focusing on filtering and cleaning data . a study aimed to improve fine-grained scheduling of data order in epochs . |
| Approach: | They propose a fine-grained scheduling method of data order in epochs to fill this gap . they define data difficulty based on relevance between data and model . |
| Outcome: | The proposed method improves on pre-training and small-scale fine-tuning experiments 2.4% over baselines. |
Out-of-Context Reasoning in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a lightweight technique trains only new token embeddings on axioms and evaluates them on unseen tasks. |
| Approach: | They propose a lightweight technique that trains only new token embeddings on axioms . they train only new embeddables and evaluate them on unseen tasks . |
| Outcome: | The proposed technique trains only new token embeddings on axioms and evaluates them on unseen tasks. |
Flipping Knowledge Distillation: Leveraging Small Models’ Expertise to Enhance LLMs in Text Matching (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable capabilities in acquiring diverse knowledge, making them highly effective across a wide range of tasks. |
| Approach: | They propose a flipped knowledge distillation paradigm where LLM learns from SLM . they propose to reinterpret LLMs as encoder-decoder models using LoRA . |
| Outcome: | The proposed model has been deployed in an online application environment and validated on financial and healthcare benchmarks and real-world applications. |
EXPO-SQL: Execution-based Clause-level Policy Optimization for Text-to-SQL (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing RL methods assign query-level rewards to all clauses, treating correct and incorrect clauses equally. |
| Approach: | They propose a method which provides fine-grained supervision through clause-level rewards. |
| Outcome: | Experiments on widely-used Text-to-SQL benchmarks show that EXPO-SqL outperforms existing methods by fine-grained clause-level learning. |
Inefficiencies of Meta Agents for Agent Design (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work has automated the design of agentic systems using meta-agents . authors examine three key challenges in a common class of meta-gents. |
| Approach: | They examine how meta-agents learn across iterations and show performance improves with evolutionary approach. |
| Outcome: | The proposed meta-agents perform worse when iterating on multiple agents than human-designed agents. |
DPO Kernels: A Semantically-Aware, Kernel-Enhanced, and Divergence-Rich Paradigm for Direct Preference Optimization (2025.findings-acl)
Copied to clipboard
Amitava Das, Suranjana Trivedy, Danush Khanna, Yaswanth Narsupalli, Basab Ghosh, Rajarshi Roy, Gurpreet Singh, Vinija Jain, Vasu Sharma, Aishwarya Naresh Reganti, Aman Chadha
| Challenge: | Direct Preference Optimization (DPO) is a cornerstone for preference alignment but is constrained by fixed divergence measures and limited feature transformations. |
| Approach: | They propose a new enhancement of Direct Preference Optimization that integrates kernel methods to overcome these challenges. |
| Outcome: | The proposed model improves divergence measures and features by using kernels . the proposed model achieves state-of-the-art generalization in factuality, safety, reasoning, and instruction following . |
DcLM: Output Length Control of Large Language Models via Dynamic Length Markers (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have limited awareness of output length, making it difficult to satisfy precise length requirements. |
| Approach: | They propose a model-agnostic approach that introduces dynamic length markers to guide length-controllable outputs. |
| Outcome: | The proposed method significantly reduces length deviation across multiple datasets. |
ReLook: Vision-Grounded RL with a Multimodal LLM Critic for Agentic Web Coding (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) excel at algorithmic code generation, but front-end development is lacking in visual fidelity and interaction. |
| Approach: | They propose an agentic, vision-grounded reinforcement learning framework that closes a loop by invoking a multimodal LLM as a tool. |
| Outcome: | The proposed framework outperforms baselines in front-end code generation. |
LLM-ForcedAligner: A Non-Autoregressive and Accurate LLM-Based Forced Aligner for Multilingual and Long-Form Speech (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for forcing alignment are language-specific and prone to temporal shifts. |
| Approach: | They propose a slot-filling paradigm that uses time indices to predict slot positions. |
| Outcome: | The proposed method reduces accumulated temporal shifts by 69% compared with prior methods. |
AdaptiveK: Complexity-Driven Sparse Autoencoders for Interpretable Language Model Representations (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to decomposing model activations into interpretable features fail to account for input complexity. |
| Approach: | They propose a framework that dynamically adjusts sparsity levels based on the semantic complexity of each input. |
| Outcome: | The proposed framework outperforms fixed-sparsity approaches on reconstruction fidelity, explained variance, cosine similarity and interpretability metrics while eliminating the burden of extensive hyperparameter tuning. |
Mixture-of-Experts with Intermediate CTC Supervision for Accented Speech Recognition (2026.acl-long)
Copied to clipboard
| Challenge: | Accented speech remains a persistent challenge for automatic speech recognition (ASR) Accent-agnostic approaches improve robustness but struggle with heavily accented or unseen varieties . |
| Approach: | They propose a Mixture-of-Experts architecture with intermediate CTC supervision that promotes expert specialization and generalization. |
| Outcome: | Experiments show that the proposed architecture improves on accented speech . the proposed framework is based on a mixture-of-experts architecture with intermediate supervision . |
What Makes an LLM a Good Optimizer? A Trajectory Analysis of LLM-Guided Evolutionary Search (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent work has demonstrated the promise of orchestrating large language models (LLMs) within evolutionary and agentic optimization systems. |
| Approach: | They present a large-scale study of LLM-guided evolutionary search . they find strong LLMs behave as local refiners, producing frequent improvements . weaker LLM optimizers exhibit large semantic drift, they say . |
| Outcome: | The results highlight the importance of trajectory analysis for understanding and improving LLM-based optimization systems. |
Learning Is Not A Race: Improving Retrieval in Language Models via Equal Learning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Overparametrized models trained on cross-entropy loss can overfit on noise . Fitting some tokens early reduces gradient signals in later iterations . |
| Approach: | They propose to overfit models trained on cross-entropy loss on noise . fitting some tokens early reduces gradient signals in later iterations . |
| Outcome: | The proposed approaches can be applied to large language models with longer contexts or larger embedding sizes. |
Fast and Effective On-Policy Distillation from Reasoning Prefixes (2026.findings-acl)
Copied to clipboard
Dongxu Zhang, Zhichao Yang, Sepehr Janghorbani, Jun Han, Andrew Ressler II, Qian Qian, Gregory D Lyng, Sanjit Singh Batra, Robert E. Tillman
| Challenge: | On-policy distillation (OPD) requires expensive on-the-fly sampling of the student policy during training, which substantially increases training cost. |
| Approach: | They propose to use on-policy distillation to sample trajectories from student model . they propose to terminate the sampling early during distillation . |
| Outcome: | The proposed method matches the performance of full OPD in long reasoning outputs while reducing training FLOP by 2x–40x. |
TituLLMs: A Family of Bangla LLMs with Comprehensive Benchmarking (2025.findings-acl)
Copied to clipboard
Shahriar Kabir Nahin, Rabindra Nath Nandi, Sagor Sarker, Quazi Sarwar Muhtaseem, Md Kowsher, Apu Chandraw Shill, Md Ibrahim, Mehadi Hasan Menon, Tareq Al Muntasir, Firoj Alam
| Challenge: | Existing benchmarking datasets for Bangla LLMs are not available for all languages. |
| Approach: | They present TituLLMs, the first large pretrained Bangla LLMs, available in 1b and 3b parameter sizes. |
| Outcome: | The proposed model outperforms existing models in Bangla, but not always in the first place. |
Improving Preference Alignment of LLM with Inference-Free Self-Refinement (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) develop in-context learning capability through pretraining and instruction tuning. |
| Approach: | Large language models (LLMs) develop in-context learning capability through pretraining and instruction tuning. |
| Outcome: | Experiments show that incorporating IFSR into preference alignment yields performance improvement over 10%. |
Reverse Prompt Engineering: A Zero-Shot, Genetic Algorithm Approach to Language Model Inversion (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods rely on a large number of outputs for training and inference, and they can produce garbled text. |
| Approach: | They propose a training-free framework that reconstructs prompts using only a limited number of text outputs from a language model. |
| Outcome: | The proposed framework generates high-quality prompt recovery and more semantically and functionally aligned with the originals than current state-of-the-art methods. |
Target-Adaptive Consistency Enhanced Prompt-Tuning for Multi-Domain Stance Detection (2024.lrec-main)
Copied to clipboard
| Challenge: | Stance detection is a fundamental task in natural language processing, but it is challenging due to diverse expressions and topics related to the targets from multiple domains. |
| Approach: | They propose a prompt-tuning method that incorporates target knowledge and prior knowledge to construct target-adaptive verbalizers for diverse domains. |
| Outcome: | The proposed method outperforms the state-of-the-art methods on nine stance detection datasets from multiple domains. |
Studying the Role of Input-Neighbor Overlap in Retrieval-Augmented Language Models Training Efficiency (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown that retrieval-augmented language models can perform better than larger models, but the optimal degree of overlap remains unclear. |
| Approach: | They investigate how varying levels of query–context overlap affect model performance . they specifically generate synthetic context through paraphrasing queries . |
| Outcome: | The proposed model can improve test-time perplexity and training time by 40% without compromising performance. |
Dynamic Sampling that Adapts: Self-Aware Iterative Data Persistent Optimization for Mathematical Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Current data selection paradigms rely on static, externally defined metrics, which fail to adapt to the evolving capabilities of models during training. |
| Approach: | They propose a dynamic sampling framework that aligns training data with the model's intrinsic competence by iterating on real-time feedback. |
| Outcome: | Extensive experiments on eight benchmarks show that SAI-DPO outperforms static baselines at most nearly 6 points, achieving state-of-the-art efficiency with significantly less data. |
Embedding-based In-Context Prompt Training for Enhancing LLMs as Text Encoders (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have been widely explored for embedding generation. |
| Approach: | They propose an embedding-based in-context prompt training strategy that leverages in-constext learning to generate high-quality embeddables while reducing computational burden. |
| Outcome: | The proposed method surpasses models trained on publicly available retrieval data and achieves state-of-the-art embedding performance on the MTEB benchmark. |
The Retrieval Bottleneck: Scaling Laws for Reinforcement Learning in RAG (2026.acl-long)
Copied to clipboard
| Challenge: | Retrieval-augmented generation (RAG) has become the dominant paradigm for building knowledge-intensive language systems. |
| Approach: | They propose a sigmoidal scaling law that shows that retrieval quality determines the asymptotic performance ceiling. |
| Outcome: | The proposed model achieves strong performance on knowledge-intensive benchmarks while retaining the predictable scaling long available for pre-training but previously absent in RAG-RL. |
Parameter Importance is Not Static: Evolving Parameter Isolation for Supervised Fine-Tuning (2026.acl-long)
Copied to clipboard
Zekai Lin, Chao Xue, Di Liang, Xingsheng Han, Peiyang Liu, Xianjie Wu, Lei Jiang, Yu Lu, Bob Simons, Shuang Liang, Minlong Peng
| Challenge: | Recent approaches to fine-tuning of large language models suffer from task interference and catastrophic forgetting. |
| Approach: | They propose a fine-tuning framework that adapts isolation decisions based on online estimates of parameter importance. |
| Outcome: | The proposed framework reduces interference and forgetting while releasing outdated parameters to recover plasticity. |
Multilingual Refusal Alignment for Safer Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly used globally, but their safety and alignment can vary unpredictably between languages. |
| Approach: | They propose a multilingual refusal alignment dataset to investigate whether alignment transfers cross-lingually and how language consistency is preserved during training. |
| Outcome: | The proposed model can be trained on multilingual datasets without affecting general performance. |
Leveraging Label Semantics and Entity Description Generation for LLM-based Fine-grained Entity Typing (2026.findings-acl)
Copied to clipboard
| Challenge: | Fine-grained entity typing (FET) aims to assign semantically rich and contextually appropriate types to entity mentions. |
| Approach: | They propose a descriptor-based retrieval-augmented framework that reduces effective label space . they propose to use natural language descriptores as an intermediate semantic representation . |
| Outcome: | The proposed framework outperforms existing methods under noisy supervision. |
Lizard: An Efficient Linearization Framework for Large Language Models (2026.acl-long)
Copied to clipboard
Chien Van Nguyen, Huy Huu Nguyen, Ruiyi Zhang, Hanieh Deilamsalehy, Puneet Mathur, Viet Dac Lai, Haoliang Wang, Jayakumar Subramanian, Ryan A. Rossi, Trung Bui, Nikos Vlassis, Franck Dernoncourt, Thien Huu Nguyen
| Challenge: | Existing linearization frameworks that rely on softmax attention with quadratic time and memory complexity pose significant computational and memory bottlenecks for long-context applications. |
| Approach: | They propose a linearization framework that transforms pretrained Transformer-based Large Language Models into subquadratic architectures that closely approximate softmax attention while preserving model quality. |
| Outcome: | Experiments show that the proposed framework outperforms existing methods by 9.4 - 24.5 points on the 5-shot MMLU benchmark and demonstrating superior associative recall. |
Graph Reasoning Paradigm: Structured and Symbolic Reasoning with Topology-Aware Reinforcement Learning for Large Language Models (2026.acl-long)
Copied to clipboard
Runxuan Liu, Xianhao Ou, Xinyan Ma, Jiyuan Wang, Jiafeng Liang, Jiaqi Li, Tao He, Zheng Chu, Rongchuan Mu, Zekun Wang, Baoxin Wang, Dayong Wu, Ming Liu, Shijin Wang, Guoping Hu, Bing Qin
| Challenge: | Existing methods for long chain-of-thought (LCoT) are coarse-grained, reward hacking, and poor generalization. |
| Approach: | They propose a Long Chain-of-Thought (LCoT) model that integrates reinforcement learning with verifiable rewards with a process-aware verification approach. |
| Outcome: | The proposed model improves reasoning and code generation tasks while reducing the cost of training and performance bottlenecks. |
Enhancing Reinforcement Learning for Radiology Report Generation with Evidence-aware Rewards and Self-correcting Preference Learning (2026.acl-long)
Copied to clipboard
| Challenge: | Recent reinforcement learning approaches have advanced radiology report generation (RRG) however, there are two limitations: report-level rewards offer limited evidence-grounded guidance for clinical faithfulness . |
| Approach: | They propose a method that uses group-wise evidence-aware alignment rewards and self-correcting preference learning to build a reliable, disease-agnostic preference dataset without human supervision. |
| Outcome: | ESC-RL promotes clinically faithful, disease-aligned reward and supports continual self-improvement during training. |
Temporal Precision Matters: Brain-Tuning Speech Language Models with Millisecond-Resolution Neural Signals (2026.acl-long)
Copied to clipboard
| Challenge: | Language models have emerged as powerful tools for predicting human brain activity during language comprehension. |
| Approach: | They propose a technique that leverages electrocorticography’s millisecond precision to train speech language models. |
| Outcome: | The proposed technique improves brain alignment over pretrained and distillation models and produces higher gains in higher-order language regions. |
ReActR: Reasoning through Error-Activated Reflection for LLM Post-Training (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for improving the mathematical abilities of Large Language Models (LLMs) focus disproportionately on scaling correct training samples, overlooking the rich learning signals contained in erroneous reasoning trajectories. |
| Approach: | They propose a framework that enhances reasoning by learning reflective behaviors from erroneous trajectories by using data construction and training. |
| Outcome: | Extensive experiments on three LLMs show that ReActR improves reasoning performance on Llama-3-8B. |
Trajectory2Task: Training Robust Tool-Calling Agents with Synthesized Yet Verifiable Data for Complex User Intents (2026.acl-long)
Copied to clipboard
Ziyi Wang, Yuxuan Lu, Yimeng Zhang, Pei Chen, Ziwei Dong, Jing Huang, Jiri Gesi, Xianfeng Tang, Chen Luo, Qun Liu, Yisi Sang, Hanqing Lu, Manling Li, Jin Lai, Dakuo Wang
| Challenge: | Tool-calling agents are increasingly deployed in real-world customer-facing workflows . but most studies on tool-callers focus on idealized settings with general, fixed, and well-specified tasks. |
| Approach: | They propose a tool-calling agent-based data pipeline that converts trajectories into user-facing tasks with controlled intent adaptations. |
| Outcome: | The proposed pipeline can be used to study tool use under three scenarios. |
ParaSuite: Boosting LLM Reasoning via Paradox Resolution (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for paradox research focus on checking basic logical consistency and not reflective reasoning. |
| Approach: | They propose a pipeline dedicated to paradox research that automates data synthesis, evaluation, and training. |
| Outcome: | The proposed pipeline improves paradoxical and general STEM reasoning. |
Data Efficient RLVR via Off-Policy Influence Guidance (2026.acl-long)
Copied to clipboard
Erle Zhu, Dazhi Jiang, Yuan Wang, Xujun Li, Jiale Cheng, Yuxian Gu, Yilin Niu, Aohan Zeng, Jie Tang, Minlie Huang, Hongning Wang
| Challenge: | Existing data selection methods for RLVR are heuristic-based, lacking theoretical guarantees and generalizability. |
| Approach: | They propose an off-policy influence estimation method that approximates data influence using offline trajectories. |
| Outcome: | The proposed method reduces the computational cost of policy rollouts and improves storage and computation efficiency. |