Papers with in-domain
Two-tiered Encoder-based Hallucination Detection for Retrieval-Augmented Generation in the Wild (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Existing solutions for hallucination detection do not consider latency, train or evaluate on production data. |
| Approach: | They propose to use customer service conversation data to evaluate existing methods . they propose to train small encoder models on a new dataset to outperform existing methods. |
| Outcome: | The proposed model outperforms existing methods and highlights the value of combining small amounts of in-domain data with public datasets. |
T-NER: An All-Round Python Library for Transformer-based Named Entity Recognition (2021.eacl-demos)
Copied to clipboard
| Challenge: | Language model (LM) pretraining has led to consistent improvements in many downstream tasks, including named entity recognition (NER). |
| Approach: | They propose a Python library for NER LM finetuning that facilitates cross-domain and cross-lingual generalization of LMs finetuned on NER. |
| Outcome: | The proposed library outperforms LMs trained on NERs in cross-domain and cross-lingual generalization tests on nine datasets. |
HAD: HAllucination Detection Language Models Based on a Comprehensive Hallucination Taxonomy (2026.acl-industry)
Copied to clipboard
| Challenge: | relying on large language models for information has raised concerns about reliability and accuracy of outputs. |
| Approach: | They propose a hallucination taxonomy with 11 categories for various NLG tasks and propose HAllucination Detection models which integrate hallucinism detection, span-level identification, and correction into a single inference process. |
| Outcome: | The proposed models outperform baselines on HaluEval, FactCHD, and FaithBench, confirming their robustness and versatility. |
Analyzing Biases to Spurious Correlations in Text Classification Tasks (2022.aacl-short)
Copied to clipboard
| Challenge: | Often these systems exceed human performance, but there is a caveat: standard benchmarks often assume that training and evaluation data are drawn independently and identically from the same underlying distribution. |
| Approach: | They propose to exploit spurious correlations in training data to exploit these correlations . they show that even when only ‘stop’ words are available, it is possible to predict the class significantly better than random. |
| Outcome: | The proposed model can predict class significantly better when only ‘stop’ words are available at the input stage, but can degrade the ability of the system to generalize well to out-of-domain data. |
Towards Reliable and Practical Phishing Detection (2025.naacl-industry)
Copied to clipboard
| Challenge: | Existing datasets lack size and diversity, with only 609 voice phishing samples available in Korean and 638 smishing instances available in English. |
| Approach: | They propose to use a Korean dataset to construct a reliable phishing detection system using language models to evaluate the model's in-domain and unseen attack detection performance. |
| Outcome: | The proposed system performs reasonably well in voice and unseen attacks while smishing detection remains challenging. |
SPPD: Self-training with Process Preference Learning Using Dynamic Value Margin (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to improve numerical and logical reasoning of Large Language Models are limited . existing approaches rely on prompt engineering and pretrained knowledge to ensure correctness . |
| Approach: | They propose to train LLMs with process-based reasoning using a dynamic value margin . they use the Bellman optimality equation to derive a value margin for step-level preference optimization . |
| Outcome: | The proposed method is equivalent to on-policy policy gradient methods under constrained reward functions. |
Non-Parametric Domain Adaptation for End-to-End Speech Translation (2022.emnlp-main)
Copied to clipboard
| Challenge: | End-to-end speech translation (E2E-ST) systems have received increasing attention due to its less error propagation, lower latency and fewer parameters. |
| Approach: | They propose a non-parametric method that leverages in-domain text translation corpus to achieve domain adaptation for E2E-ST systems. |
| Outcome: | The proposed method outperforms the existing in-domain fine-tuning strategies on the Europarl-ST benchmark. |
Certified Error Control of Candidate Set Pruning for Two-Stage Relevance Ranking (2022.emnlp-main)
Copied to clipboard
| Challenge: | In information retrieval, candidate set pruning is used to speed up two-stage relevance ranking but lacks accurate error control and empirical guarantees. |
| Approach: | They propose a method that guarantees the test error after pruning is controlled under a user-specified threshold with high probability. |
| Outcome: | The proposed method reduces the average set size from 1000 to 27, increasing reranking speed by about 37 times while keeping MRR@10 greater than a pre-specified value of 0.38 with about 90% empirical coverage. |
Exploring the Effect of Nominal Compound Structure in Scientific Texts on Reading Times of Experts and Novices (2025.acl-srw)
Copied to clipboard
| Challenge: | Using a corpus of eye-tracking data of German native speakers, we find that some compound types are associated with longer reading times. |
| Approach: | They use a corpus containing eye-tracking data of german native speakers reading scientific texts. |
| Outcome: | The authors show that some compound types are associated with longer reading times and that experts may have an advantage while reading in-domain texts, but also while reading out-of-domain. |
Aggretriever: A Simple Approach to Aggregate Textual Representations for Robust Dense Passage Retrieval (2023.tacl-1)
Copied to clipboard
| Challenge: | Pre-trained language models have been successful in knowledge-intensive tasks, but recent research calls into question the robustness of these singlevector models. |
| Approach: | They propose to exploit knowledge in a pre-trained language model for dense passage retrieval by aggregating contextualized token embeddings into a dense vector. |
| Outcome: | The proposed model significantly improves the effectiveness of dense retrieval models on in-domain and zero-shot evaluations without introducing substantial training overhead. |
Generalizing Question Answering System with Pre-trained Language Model Fine-tuning (D19-58)
Copied to clipboard
| Challenge: | Existing methods focus on improving in-domain performance, leaving open the question of how they can generalize to out-of-domain and unseen RC tasks. |
| Approach: | They propose a multi-task learning framework that learns the shared representation across different tasks and builds on a large pre-trained language model and fine-tuned on multiple RC datasets. |
| Outcome: | The proposed framework improves the BERT-Large baseline by 8.39 and 7.22 respectively. |
Contextualizing Argument Quality Assessment with Relevant Knowledge (2024.naacl-short)
Copied to clipboard
| Challenge: | Existing methods for assessing argument quality in isolation analyze their quality in the absence of context, which affects their accuracy and generalizability. |
| Approach: | They propose a method for scoring argument quality based on contextualization via relevant knowledge that leverages large language models to provide feedback, infer hidden assumptions, supply a similar-quality argument, or give a counter-argument. |
| Outcome: | The proposed method outperforms existing methods across multiple metrics in both in-domain and zero-shot setups. |
Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Tasks (2021.naacl-main)
Copied to clipboard
| Challenge: | Cross-encoders perform full-attention over the input pair, while bi-encoding requires substantial training data and fine-tuning over the target task to achieve competitive performance. |
| Approach: | They propose a data augmentation strategy that uses cross-encoders to label larger set of input pairs to augment training data for bi-encoding. |
| Outcome: | The proposed approach improves on multiple tasks and domain adaptation tasks by up to 37 points compared to the original bi-encoder performance. |
How Does Counterfactually Augmented Data Impact Models for Social Computing Constructs? (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have shown that models trained on CAD can learn cues in the dataset which are spuriously correlated with the construct. |
| Approach: | They focus on sentiment, sexism, and hate speech as social constructs to investigate their effects on model performance. |
| Outcome: | The proposed model generalizes better on out-of-domain datasets while relying less on spurious features. |
Training a Utility-based Retriever Through Shared Context Attribution for Retrieval-Augmented Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Utility-based retrieval has emerged as a promising topic for downstream tasks . however, capturing passage utility accurately remains unexplored due to insufficient understanding . |
| Approach: | They propose a framework for training utility-based retrievers in Retrieval-Augmented Language Models . it incorporates multi-task generalization and inter-passage interaction to improve performance . |
| Outcome: | The proposed framework improves performance on ten datasets across different tasks. |
Unsupervised Domain Adaptation for Keyphrase Generation using Citation Contexts (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for keyphrase generation are limited to resource-rich languages. |
| Approach: | They propose to extract silver-standard keyphrases from citation contexts to create synthetic labeled data for domain adaptation. |
| Outcome: | The proposed method produces significant and consistent improvements over baselines across three domains. |
Scaling Back-Translation with Domain Text Generation for Sign Language Gloss Translation (2023.eacl-main)
Copied to clipboard
| Challenge: | Sign language gloss translation aims to translate the sign glosses into spoken language texts, which is challenging due to the scarcity of labeled gloss-text parallel data. |
| Approach: | They propose a back translation technique that generates pseudo-parallel data by translating in-domain spoken language texts into sign glosses. |
| Outcome: | The proposed method outperforms the BT methods on three benchmarks of sign language gloss translation in different languages. |
Multitask Parsing Across Semantic Representations (P18-1)
Copied to clipboard
| Challenge: | UCCA parsing is a test case for multitask learning, with auxiliary tasks AMR, SDP and Universal Dependencies (UD) . Semantic parsers have arguably yet to reach their full potential due to the limited amount of semantically annotated training data. |
| Approach: | They propose a general transition-based parser that can parse UCCA, AMR, SDP and Universal Dependencies (UD) they use a transition-driven learning architecture and a uniform transition-basic learning architecture to train the parsers. |
| Outcome: | The proposed parser improves UCCA, AMR, SDP and Universal Dependencies (UD) parsing over training in English, German and French. |
PACIT: Unlocking the Power of Examples for Better In-Context Instruction Tuning (2024.findings-acl)
Copied to clipboard
| Challenge: | In-context instruction tuning is a method that fine tunes with supervised instruction data to enhance the instruction following ability of large language models. |
| Approach: | They propose an in-context instruction tuning method that incorporates positive and negative examples into the prompt for better performance. |
| Outcome: | The proposed method outperforms baseline instruction tuning methods on in-domain and out-domain tasks up to 9.16 and 3.14 average ROUGE-L scores. |
DiaSynth: Synthetic Dialogue Generation Framework for Low Resource Dialogue Applications (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing research is limited by general or niche datasets that lack sufficient scale for training dialogue systems. |
| Approach: | They propose a synthetic dialogue generation framework that uses Large Language Models and Chain of Thought reasoning to generate dynamic, domain-specific dialogues with simulated personas and diverse conversational features. |
| Outcome: | The proposed framework outperforms existing frameworks on dialogue summarization and quality increases as the size of the LLM increases from 3B to 8B. |
ChrEn: Cherokee-English Machine Translation for Endangered Language Revitalization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Cherokee is a highly endangered Native American language spoken by the Cherokee people . there are only 2,000 fluent first language Cherokee speakers remaining in the world . |
| Approach: | They propose a Cherokee-English parallel dataset to facilitate machine translation between Cherokee and English. |
| Outcome: | The proposed dataset compares Cherokee-English and English-Cherokee machine translation systems . the results show that the datasets are low-resource and low-cost compared to other datasets . |
English-Indonesian Neural Machine Translation for Spoken Language Domains (P19-2)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) is a data-driven method that requires a large amount of data to build a robust model. |
| Approach: | They conduct a study on Neural Machine Translation (NMT) for English-Indonesian and Indonesian-English (ID-EN) they build NMT systems using the Transformer model for both translation directions and implement domain adaptation method to train pre-trained NMT on speech language data. |
| Outcome: | The proposed model can learn formal translation outputs for English-Indonesian and Indonesian-English (ID-EN) given a small dataset of speech-styled language and a larger dataset of less formal language, the proposed model will be useful for learning formality level. |
Penn-Helsinki Parsed Corpus of Early Modern English: First Parsing Results and Analysis (2022.findings-naacl)
Copied to clipboard
| Challenge: | PPCEME has a large set of function tags and is difficult to parse . authors present results for PPceME using a modified version of the Berkeley Neural Parser . |
| Approach: | They propose to use a modified version of the Berkeley Neural Parser to parse PPCEME using function tags. |
| Outcome: | The proposed parser will be used to parse Early English Books Online, a 1.5 billion word corpus. |
CoCoA-MT: A Dataset and Benchmark for Contrastive Controlled MT with Application to Formality (2022.findings-naacl)
Copied to clipboard
| Challenge: | Specific problems arise when translating from English into languages with formality markers, such as “Are you sure?” . Using wrong or inconsistent tone may be perceived as inappropriate or jarring for users of certain cultures and demographics. |
| Approach: | They propose to train formality-controlled models by fine-tuning on labeled contrastive data and a metric to evaluate them. |
| Outcome: | The proposed model achieves high accuracy (82% in-domain and 73% out-of-domain) while maintaining overall quality. |
Test-time Adaptation for Machine Translation Evaluation by Uncertainty Minimization (2023.acl-long)
Copied to clipboard
| Challenge: | evaluators of machine translation systems often use text-based metrics to evaluate performance . however, these metrics lack semantic-level information and exhibit poor correlation with human ratings . authors propose a method to reduce inference bias of neural metrics in out-of-distribution data . |
| Approach: | They propose to reduce inference bias by using uncertainty estimation, test-time adaptation, and inference to reduce model uncertainty. |
| Outcome: | The proposed method reduces model uncertainty and improves correlation performance across models. |
Active Learning by Acquiring Contrastive Examples (2021.emnlp-main)
Copied to clipboard
| Challenge: | Using uncertainty and diversity sampling, active learning acquisition functions select difficult and diverse data points from a pool of unlabeled data. |
| Approach: | They propose an active learning acquisition function that selects contrastive examples from unlabeled data. |
| Outcome: | The proposed approach performs better or equal to the best performing baseline on all tasks, on both in-domain and out-of-domain data. |
Post-Abstention: Towards Reliably Re-Attempting the Abstained Instances in QA (2023.acl-long)
Copied to clipboard
| Challenge: | Despite remarkable progress made in natural language processing, even the state-of-the-art systems often make incorrect predictions. |
| Approach: | They propose to use selective prediction to enable models to abstain from answering when their predictions are likely to be incorrect. |
| Outcome: | The proposed method improves performance on 11 QA datasets and in- and out-of-domain settings. |
LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression (2024.findings-acl)
Copied to clipboard
Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin, Victor Rühle, Yuqing Yang, Chin-Yew Lin, H. Vicky Zhao, Lili Qiu, Dongmei Zhang
| Challenge: | Existing approaches to compress prompts only leverage unidirectional context, causing suboptimal results. |
| Approach: | They propose a task-agnostic prompt compression method that takes tokens from context . they use a Transformer encoder to capture all essential information needed for prompt compression . |
| Outcome: | The proposed method is 3x-6x faster than existing prompt compression methods and faster than baselines. |
VerbAtlas: a Novel Large-Scale Verbal Semantic Resource and Its Application to Semantic Role Labeling (D19-1)
Copied to clipboard
| Challenge: | VerbAtlas is a lexical-semantic resource that combines WordNet synsets into semantically-coherent frames. |
| Approach: | They propose a lexical-semantic resource that brings together WordNet synsets into semantically-coherent frames. |
| Outcome: | The proposed resource brings together all WordNet synsets into semantically-coherent frames. |
In and Out-of-Domain Text Adversarial Robustness via Label Smoothing (2023.acl-short)
Copied to clipboard
| Challenge: | Existing studies show that state-of-the-art NLP models are vulnerable to adversarial attacks . label smoothing has been proven effective in a variety of applications and modalities . |
| Approach: | They propose to use label smoothing to improve adversarial robustness in pre-trained models against various popular attacks. |
| Outcome: | The proposed method significantly improves adversarial robustness in pre-trained models against various popular attacks. |
TSDAE: Using Transformer-based Sequential Denoising Auto-Encoderfor Unsupervised Sentence Embedding Learning (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to learn sentence embeddings require labeled data, but it is expensive. |
| Approach: | They propose an unsupervised method which learns sentence embeddings using unlabeled data . they propose a transformer-based sequence denoising auto-encoder which can be used for training . |
| Outcome: | The proposed method outperforms existing methods on four datasets from heterogeneous domains. |
FactKB: Generalizable Factuality Evaluation using Language Models Enhanced with Factual Knowledge (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing factuality evaluation models are not robust, especially with respect to entity and relation errors in new domains. |
| Approach: | They propose a new approach to factuality evaluation that is generalizable across domains . they propose entities-specific facts, facts extracted from external knowledge bases and facts constructed compositionally through knowledge base walks. |
| Outcome: | The proposed model achieves state-of-the-art on two in-domain news summarization benchmarks and on three out-of domain scientific literature datasets. |
Multilingual Verbalisation of Knowledge Graphs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Most work on Knowledge Graph (KG) verbalisation is monolingual leaving open the question of how to scale KG-to-Text generation to languages with varying amounts of resources. |
| Approach: | They explore how to scale KG-to-Text generation to languages with varying resources . they construct multilingual training data and test data for each language . |
| Outcome: | The proposed approach performs best on all 9 languages, compared with other approaches on low vs high resource languages and on in- v. out-of-domain data. |
How to tackle an emerging topic? Combining strong and weak labels for Covid news NER (2022.aacl-short)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) models can be trained for emerging topics such as medical domain where new topics are constantly evolving out of the scope of existing models and datasets. |
| Approach: | They propose a recipe to combine weak and strong labels to improve Named Entity Recognition (NER) models for emerging topics. |
| Outcome: | The proposed model outperforms methods trained on weak data while combining out-of-domain and in-domain weak label training. |
SCV: Light and Effective Multi-Vector Retrieval with Sequence Compressive Vectors (2025.coling-industry)
Copied to clipboard
| Challenge: | Recent advances in language models (LMs) have driven progress in information retrieval (IR), effectively extracting semantically relevant information. |
| Approach: | They propose a multi-vector retrieval framework with sequence compression vectors and a coarse-to-fine vector search framework that uses span compressive vectors to score documents. |
| Outcome: | The proposed approach achieves the fastest latency compared to other state-of-the-art models and can obtain competitive performance on both in-domain and out-of domain benchmark datasets. |
M2D2: A Massively Multi-Domain Language Modeling Dataset (2022.emnlp-main)
Copied to clipboard
| Challenge: | M2D2 consists of 8.5B tokens and spans 145 domains extracted from Wikipedia and Semantic Scholar. |
| Approach: | They propose to organize 145 domains into 22 groups and use ontologies from Wikipedia and ArXiv to study domain adaptation in language models. |
| Outcome: | The proposed model enables the study of domain adaptation in language models (LMs) it shows that small amounts of fine-grained data can lead to larger in-domain performance gains than weakly relevant data. |
How to Select One Among All ? An Empirical Study Towards the Robustness of Knowledge Distillation in Natural Language Understanding (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Knowledge Distillation (KD) is a model compression algorithm that helps transfer knowledge in a large neural network into a smaller one. |
| Approach: | They propose a framework to assess adversarial robustness of multiple KD algorithms. |
| Outcome: | The proposed algorithm achieves state-of-the-art on the GLUE benchmark and out-of domain generalization and adversarial robustness compared to competitive methods. |
Benchmarking and Mitigating the Impact of Noisy User Prompts in Medical VLMs via Cross-Modal Reflection (2026.eacl-industry)
Copied to clipboard
| Challenge: | Existing medical vision-language models follow user-provided prompts blindly, a new study finds . current models are noisy, causing problems with reliability in real-world interactions . |
| Approach: | They propose a method to evaluate the influence of clinical prompts on medical vision-language models . they use cross-modal reflection chain-of-thought to train the model to produce reasoning paths . |
| Outcome: | The proposed method significantly improves the robustness against noisy prompts . existing Med-VLMs follow user-provided prompts blindly, the authors show . |
Protein2Text: Resampling Mechanism to Translate Protein Sequences into Human-Interpretable Text (2025.naacl-industry)
Copied to clipboard
Ala Jararweh, Oladimeji Macaulay, David Arredondo, Yue Hu, Luis E Tafoya, Kushal Virupakshappa, Avinash Sahu
| Challenge: | Proteins play critical roles in biological systems, yet 99.7% of 227 million known protein sequences remain uncharacterized due to the limitations of experimental methods. |
| Approach: | They propose a multimodal large language model that interprets protein sequences and generates informative text to address open-ended questions about protein functions and attributes. |
| Outcome: | The proposed model outperforms existing models in open-ended question-answering tasks. |
Going beyond research datasets: Novel intent discovery in the industry setting (2023.findings-eacl)
Copied to clipboard
| Challenge: | Novel intent discovery automates grouping of similar messages to identify previously unknown intents. |
| Approach: | They propose to use question-only data to improve the intent discovery pipeline . they propose to utilize conversational structure of real-life datasets for clustering . |
| Outcome: | The proposed method gives 33pp performance boost over state-of-the-art model for question only . it also gives 13pp performance increase over the naive baseline model . |
SELF-EXPERTISE: Knowledge-based Instruction Dataset Augmentation for a Legal Expert Language Model (2024.findings-naacl)
Copied to clipboard
| Challenge: | generating instructions and outputs from LLMs can produce unintentionally inaccurate or misleading information. |
| Approach: | They propose to generate an instruction dataset in the legal domain from a seed dataset by extracting knowledge from the outputs of the seed dataset. |
| Outcome: | The proposed method reduces hallucinations in automatic instruction dataset augmentation. |
Validator-Guided Hard Negative Mining for Masked Language Modeling in Low-Resource Ancient Languages (2026.acl-srw)
Copied to clipboard
| Challenge: | Pre-trained multilingual models lack exposure to low-resource ancient languages such as Sumerian. |
| Approach: | They build a hierarchical validator capturing subword, word, and part-of-speech patterns from 4,545 annotated sequences and use it to filter candidates and identify hard negatives for fine-tuning. |
| Outcome: | The proposed model achieves 18.0% hit@10 accuracy, while the validator alone improves this to 72.8%, while hard negative fine-tuning reaches 78.3%. |
Can Large Language Models Generate High-quality Patent Claims? (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown exceptional performance across various text generation tasks, but remain under-explored in the patent domain, which offers highly structured and precise language. |
| Approach: | They construct a dataset to investigate the performance of current LLMs in patent claim generation. |
| Outcome: | The proposed model outperforms state-of-the-art general LLMs in patent claim generation. |
Supervision versus Demonstration-Based In-Context Learning for Multiword Expression Classification (2026.acl-srw)
Copied to clipboard
| Challenge: | idiomatic light verb constructions (LVCs) are challenging for multiword expression processing . they share the same surface form as fully literal verb–object combinations . |
| Approach: | They frame Turkish LVC detection as a binary classification task . they compare a supervised Turkish encoder baseline to three instruction-tuned LLMs . |
| Outcome: | The proposed method improves Turkish LVC detection on a controlled set with matched negatives and positives. |
REIC: RAG-Enhanced Intent Classification at Scale (2025.emnlp-industry)
Copied to clipboard
Ziji Zhang, Michael Yang, Zhiyu Chen, Yingying Zhuang, Shu-Ting Pi, Qun Liu, Rajashekar Maragoud, Vy Nguyen, Anurag Beniwal
| Challenge: | Accurate intent classification is critical for efficient routing in customer service . however, as companies expand their product lines, intent classification faces scalability challenges . |
| Approach: | They propose a retrieval-augmented generation Enhanced Intent Classification approach which leverages retrieval augmented generation to integrate relevant knowledge into a model. |
| Outcome: | The proposed approach outperforms fine-tuning, zero-shot, and few-shot methods on real-world datasets. |
Improving QA Generalization by Concurrent Modeling of Multiple Biases (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches for debiasing datasets are weaker than current approaches for generalization. |
| Approach: | They propose a framework for analyzing multiple biases in training data to reduce bias weighting. |
| Outcome: | The proposed framework improves generalization on in-domain and out-of-domain datasets by weighting examples based on their strengths and bias strengths. |
Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translation (2023.eacl-main)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) is becoming more accurate, but hallucinations are extremely pathological . previous work focused on artificial settings where the problem is amplified, disregarding some common types of hallucines . |
| Approach: | They propose a method for alleviating hallucinations at test time that significantly reduces the hallucinic rate. |
| Outcome: | The proposed method significantly reduces the hallucinatory rate in a natural setting. |
Improving Unsupervised Out-of-domain detection through Pseudo Labeling and Learning (2023.findings-eacl)
Copied to clipboard
| Challenge: | Unsupervised OOD detection is a task aimed at discriminating whether given samples are from the in-domain (IND) . previous studies adopted the one-class classification approach, assuming that the training samples come from a single domain. |
| Approach: | They propose a framework that leverages latent categorical information to improve representation learning for textual OOD detection. |
| Outcome: | The proposed framework significantly outperforms baseline models on three datasets. |
Direct Speech Translation for Automatic Subtitling (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing models for automatic subtitling generate subtitles in the target language along with their timestamps. |
| Approach: | They propose a direct speech translation model that generates subtitles in the target language along with their timestamps with a single model. |
| Outcome: | The proposed model outperforms a cascade system on 7 language pairs and on new benchmarks. |
Investigating Human Values in Online Communities (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing value frameworks struggle with sample sizes and rely on selfreported surveys to calculate values. |
| Approach: | They propose a method to computationally analyse values on Reddit using in-domain and out-of-domain human annotations to train a value relevance and a polarity classifier. |
| Outcome: | The proposed method can be used to analyse values on reddit using human annotations and human annotation. |
Learning Job Title Representation from Job Description Aggregation Network (2024.findings-acl)
Copied to clipboard
Napat Laosaengpha, Thanit Tativannarat, Chawan Piansaddhayanon, Attapol Rutherford, Ekapol Chuangsuwanich
| Challenge: | Existing methods for learning job title representation neglect the rich content within the job description. |
| Approach: | They propose a framework for learning job titles through their respective job description and utilize a Job Description Aggregator component to handle the lengthy description and bidirectional contrastive loss. |
| Outcome: | The proposed framework outperforms the skill-based approach on in-domain and out-of-domain settings and achieving a superior performance. |
Iterative Dual Domain Adaptation for Neural Machine Translation (D19-1)
Copied to clipboard
| Challenge: | Empirical results on Chinese-English and English-German translation tasks demonstrate the effectiveness of our proposed framework. |
| Approach: | They propose an iterative dual domain adaptation framework for neural machine translation that uses multiple corpora to perform bidirectional translation knowledge transfer. |
| Outcome: | Empirical results on Chinese-English and English-German translation tasks demonstrate the effectiveness of the proposed framework. |
Towards Knowledge-Grounded Counter Narrative Generation for Hate Speech (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to combat online hatred using informed textual responses - called counter narratives - produce generic/repetitive responses and lack grounded and up-to-date evidence such as facts, statistics, or examples. |
| Approach: | They propose to automatically generate counter narratives using an external knowledge repository to provide more informative content to fight online hatred. |
| Outcome: | The proposed pipeline can generate suitable and informative counter narratives in in-domain and cross-domain settings. |
Automated essay scoring with string kernels and word embeddings (P18-2)
Copied to clipboard
| Challenge: | Existing approaches to automatic essay scoring use low-level character n-gram features. |
| Approach: | They propose to combine string kernels and word embeddings for automatic essay scoring. |
| Outcome: | The proposed method outperforms state-of-the-art deep learning methods in Arabic dialect identification and native language identification tasks. |
Large Language Models as Foundations for Next-Gen Dense Retrieval: A Comprehensive Empirical Assessment (2024.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models have limited generalization capabilities and performance challenges. |
| Approach: | They evaluate 15 different backbone LLMs and non-LLMs to evaluate their performance . larger models and extensive pre-training consistently enhance in-domain accuracy and data efficiency . |
| Outcome: | The results show that larger models and extensive pre-training enhance in-domain accuracy and data efficiency. |
OpenDialKG: Explainable Conversational Reasoning with Attention-based Walks over Knowledge Graphs (P19-1)
Copied to clipboard
| Challenge: | Existing models that use a large-scale knowledge graph to create a conversational reasoning model are domain-agnostic and scalable. |
| Approach: | They propose a conversational reasoning model that strategically traverses through a large-scale common fact knowledge graph to introduce engaging and contextually diverse entities and attributes. |
| Outcome: | The proposed model retrieves more natural responses than state-of-the-art models in both in-domain and cross-domain tasks. |
Efficient and Versatile Model for Multilingual Information Retrieval of Islamic Text: Development and Deployment in Real-World Scenarios (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Despite recent advances in multilingual information retrieval, a significant gap remains between research efforts and real-world deployment. |
| Approach: | They propose to use Quranic multilingual corpus to develop an ad-hoc IR system that can satisfy users’ information needs in multiple languages. |
| Outcome: | The proposed model achieves promising results across diverse retrieval scenarios. |
Unsupervised Slot Schema Induction for Task-oriented Dialog (2022.naacl-main)
Copied to clipboard
| Challenge: | Defining task-specific schemas is the first step of building a task-oriented dialog system. |
| Approach: | They propose an unsupervised approach for slot schema induction from unlabeled dialog corpora using in-domain language models and unsupervised parsing structures. |
| Outcome: | The proposed method shows significant performance improvement on multi-domain and SGD datasets. |
Towards Robust Neural Retrieval with Source Domain Synthetic Pre-Finetuning (2022.coling-1)
Copied to clipboard
Revanth Gangi Reddy, Vikas Yadav, Md Arafat Sultan, Martin Franz, Vittorio Castelli, Heng Ji, Avirup Sil
| Challenge: | Existing neural IR systems rely on lexical matching for query-passage alignment, while masked language models use a dual encoder architecture to encode passages and questions into continuous vector representations. |
| Approach: | They propose to enhance the out-of-domain generalization of Dense Passage Retrieval (DPR) through synthetic data augmentation only in the source domain. |
| Outcome: | The proposed model outperforms existing models in in-domain and zero-shot evaluations on Wikipedia-based datasets. |
RedOne 2.0: Rethinking Domain-specific LLM Post-Training in Social Networking Services (2026.acl-industry)
Copied to clipboard
Fei zhao, Chonggang Lu, Haofu Qian, Fangcheng Shi, Zijie Meng, Jianzhao Huang, Zheyong Xie, Shaosheng Cao
| Challenge: | Social networking services (SNS) are critical infrastructure for global interaction . supervised fine-tuning (SFT) can improve in-domain performance, but it often induces a ”seesaw” trade-off with out-of-domain robustness . |
| Approach: | They propose an SNS-oriented LLM with a progressive, RL-prioritized post-training paradigm for fast and stable adaptation. |
| Outcome: | The proposed model improves over the previous 7B model by 2.41 on average . it also yields an 8.74 average gain over its Qwen3-4B base . |
Beyond Guilt: Legal Judgment Prediction with Trichotomous Reasoning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Current legal large language models lack trichotomous reasoning capabilities due to the absence of an appropriate benchmark dataset. |
| Approach: | They propose a benchmark dataset for Legal Judgment Prediction with Innocent Verdicts that incorporates trichotomous dogmatics into zero-shot prompting and fine-tuning. |
| Outcome: | The proposed dataset extends three widely-used legal datasets through LLM-based augmentation and manual verification. |
Cross-Lingual Knowledge Transfer for Clinical Phenotyping (2022.lrec-1)
Copied to clipboard
Jens-Michalis Papaioannou, Paul Grundmann, Betty van Aken, Athanasios Samaras, Ilias Kyparissidis, George Giannakoulas, Felix Gers, Alexander Loeser
| Challenge: | Current models for clinical phenotyping are limited to clinical notes written in English due to the large amount of labeled and unlabeled clinical text resources. |
| Approach: | They propose to use translation-based methods with domain-specific encoders and cross-lingual encoder plus adapters to perform this task for clinics that do not use the English language. |
| Outcome: | The proposed strategies outperform the state-of-the-art models for clinics that do not use the English language and have a small amount of in-domain data available. |
Efficient Hierarchical Domain Adaptation for Pretrained Language Models (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to allow domain adaptation to diverse domains are expensive and require continuing training in-domain. |
| Approach: | They propose a method to permit domain adaptation to many diverse domains using a computationally efficient adapter approach. |
| Outcome: | The proposed method allows domain adaptation to many diverse domains while avoiding negative interference between unrelated domains. |
DAWN-ICL: Strategic Planning of Problem-solving Trajectories for Zero-Shot In-Context Learning (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods to conduct in-context learning without using human-annotated demonstrations are unreliable and lead to error accumulation. |
| Approach: | They propose a method to conduct in-context learning without using human-annotated demonstrations. |
| Outcome: | The proposed method outperforms existing methods using human-annotated demonstrations. |
Leveraging Generative Large Language Models with Visual Instruction and Demonstration Retrieval for Multimodal Sarcasm Detection (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for multimodal sarcasm detection do not fully utilize cross-modal features, limiting their performance on in-domain datasets. |
| Approach: | They propose a multimodal sarcasm detection model with a designed instruction template and a demonstration retrieval module. |
| Outcome: | The proposed model outperforms existing methods on in-domain datasets and achieves state-of-the-art performance. |
SSMBA: Self-Supervised Manifold Based Data Augmentation for Improving Out-of-Domain Robustness (2020.emnlp-main)
Copied to clipboard
| Challenge: | Data augmentation is a common method used to improve out-of-domain (OOD) generalization. |
| Approach: | They propose a data augmentation method that uses corruption and reconstruction functions to move randomly on a manifold to generate training examples. |
| Outcome: | The proposed method outperforms existing methods and baseline models on both in-domain and OOD data and achieves gains of 0.8% on OOD Amazon reviews, 1.8% accuracy on OOO MNLI, and 1.4 BLEU on in- domain IWSLT14 German-English. |
Watch the Neighbors: A Unified K-Nearest Neighbor Contrastive Learning Framework for OOD Intent Discovery (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for finding out-of-domain intents suffer from in-domain overfitting problem . previous methods fail to transfer prior knowledge to downstream clustering . |
| Approach: | They propose a unified K-nearest neighbor contrastive learning framework to discover OOD intents . they propose IND pre-training objective to learn discriminative features while maintaining intra-class diversity . |
| Outcome: | The proposed framework improves on three benchmark datasets. |
The Devil is in the Distributions: Explicit Modeling of Scene Content is Key in Zero-Shot Video Captioning (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for zero-shot video captioning focus on one key aspect of the scene and ignore the rest of the visual input. |
| Approach: | They propose a novel textual prompting strategy for zero-shot video captioning that uses a category-aware retrieval mechanism to promote prompt diversity while ensuring visual relevance. |
| Outcome: | The proposed method outperforms existing methods on in-domain and cross-domain settings. |
Resource Creation Towards Automated Sentiment Analysis in Telugu (a low resource language) and Integrating Multiple Domain Sources to Enhance Sentiment Prediction (L18-1)
Copied to clipboard
| Challenge: | Sentiment Analysis of text is an important task in many applications . but the task becomes challenging when it comes to low resource languages . |
| Approach: | They propose to create a corpus of polarity-based sentiment classifiers in Telugu for different domains like movie reviews, song lyrics, product reviews and book reviews. |
| Outcome: | The proposed model performs well in multiple domains and is compared with the previous models. |
Automatic Data Acquisition for Event Coreference Resolution (2021.eacl-main)
Copied to clipboard
| Challenge: | lexical paraphrases and high precision rules informed by news discourse structure can be used to collect coreferential and non-coreferential event pairs from unlabeled English news articles. |
| Approach: | They propose to use lexical paraphrases and news discourse structure to automatically collect coreferential and non-coreferential event pairs from unlabeled English news articles. |
| Outcome: | The proposed model performs better than the supervised model on evaluation datasets with different event domains and text genres. |
Towards Generalized Open Information Extraction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Open Information Extraction (OpenIE) models are evaluated on in-domain test sets aside from the training corpus, which violates the initial task principle of domain-independence. |
| Approach: | They propose to generalize OpenIE over unseen target domains with different data distributions from source training domains. |
| Outcome: | The proposed method beats the previous methods in both in- and out-of-domain settings by 6.0% in F1 score absolutely. |
Aligning Large and Small Language Models via Chain-of-Thought Reasoning (2024.eacl-long)
Copied to clipboard
| Challenge: | Chain-of-Thought (CoT) prompting empowers Large Language Models to solve complex reasoning tasks in a step-wise manner. |
| Approach: | They propose a method for aligning and transferring reasoning abilities between larger and smaller Language Models by using CoT-Demonstrations. |
| Outcome: | The proposed method outperforms baselines on question-answering and mathematical reasoning benchmarks. |
Domain Knowledge Transferring for Pre-trained Language Model via Calibrated Activation Boundary Distillation (2022.acl-long)
Copied to clipboard
| Challenge: | Pretrained language models are used to boost their performance on downstream tasks . pretraining with in-domain texts requires considerable in- domain data and training resources . |
| Approach: | They propose a domain knowledge transferring framework for pre-trained language models without additional in-domain pretraining. |
| Outcome: | The proposed framework extracts domain knowledge from an existing in-domain pretrained language model and transfers it to other PLMs by applying knowledge distillation. |
DomainSum: A Hierarchical Benchmark for Fine-Grained Domain Shift in Abstractive Text Summarization (2025.findings-naacl)
Copied to clipboard
| Challenge: | Abstractive summarization is a crucial task in natural language processing . current research focuses on summarizing specific types of documents . domain shifts between documents affect summarisation performance . |
| Approach: | They propose a hierarchical benchmark to capture fine-grained domain shifts in abstractive summarization. |
| Outcome: | The proposed benchmark measures the generalization capabilities of pre-trained language models and large language models in in-domain and cross-domain settings. |
MuCPAD: A Multi-Domain Chinese Predicate-Argument Dataset (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent studies show that shallow semantic role labeling (SRL) performance drops under out-of-domain setting. |
| Approach: | They propose to annotate a multi-domain Chinese predicate-argument dataset using a frame-free annotation methodology and strict double annotation for improving data quality. |
| Outcome: | The proposed dataset is compared with a dataset from six different domains. |
An Unsupervised Sentence Embedding Method by Mutual Information Maximization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Sentence BERT is inefficient for sentence-pair tasks as it needs to evaluate combinatorially many sentence pairs which is very time-consuming. |
| Approach: | They propose a lightweight extension on top of BERT and a self-supervised learning objective to derive meaningful sentence embeddings in an unsupervised manner. |
| Outcome: | The proposed method outperforms baselines on common semantic textual similarity tasks and downstream supervised tasks and achieves performance competitive with supervised methods on various tasks. |
Bridging Anaphora Resolution as Question Answering (2020.acl-main)
Copied to clipboard
| Challenge: | Existing studies on bridging anaphora resolution focus on question answering based on context . briding anaphorisms and their antecedents are linked via various lexico-semantic, frame or encyclopedic relations. |
| Approach: | They propose a question answering framework for bridging anaphora resolution . they propose briding anaphorisms and their antecedents are linked via various lexico-semantic, frame or encyclopedic relations. |
| Outcome: | The proposed method generates state-of-the-art results on two bridging corpora. |
Revisiting the Weaknesses of Reinforcement Learning for Neural Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | In neural sequence-to-sequence learning, Reinforcement Learning (RL) has gained popularity due to the suitability of Policy Gradient (PG) methods for the end-to end training paradigm. |
| Approach: | They propose to let the model explore the output space beyond the reference output that is used for standard cross-entropy minimization by reinforcing model outputs according to their quality, effectively increasing the likelihood of higher-quality samples. |
| Outcome: | The proposed model explores the output space beyond the reference output that is used for cross-entropy minimization, increasing the likelihood of higher-quality samples. |
Adaptation of Hierarchical Structured Models for Speech Act Recognition in Asynchronous Conversation (N19-1)
Copied to clipboard
| Challenge: | asynchronous domains lack large labeled datasets to train an effective speech act recognition model. |
| Approach: | They propose methods to leverage abundant unlabeled conversational data and available labeled data from synchronous domains to train an effective SAR model. |
| Outcome: | The proposed method outperforms existing methods when trained on in-domain data only. |
Named Entity Recognition in a Very Homogenous Domain (2023.findings-eacl)
Copied to clipboard
| Challenge: | Developing models that perform well on several domains is important, but domain is vague and can be adapted to a new domain. |
| Approach: | They find that even news articles from the same newspaper in English can be considered different domains. |
| Outcome: | The proposed model performs better on out-of-domain data than on specialized data. |
Self-Refine Instruction-Tuning for Aligning Reasoning in Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to align reasoning abilities between Large Language Models and Smaller Language Model are supervised fine-tuning and preference optimization. |
| Approach: | They propose a method that elicits Smaller Language Models to self-improve their reasoning abilities via preference optimization. |
| Outcome: | The proposed method outperforms Instruction-tuning on commonsense and math reasoning tasks on common and math scenarios. |
DaLC: Domain Adaptation Learning Curve Prediction for Neural Machine Translation (2022.findings-acl)
Copied to clipboard
| Challenge: | Current research in NMT Domain Adaptation rarely provides insights on the amount of data required to perform Domain . |
| Approach: | They propose a Domain adaptation learning curve prediction model that predicts prospective DA performance based on in-domain monolingual samples in the source language. |
| Outcome: | The proposed model predicts prospective DA performance based on in-domain monolingual samples in the source language. |
Unlocking Emergent Modularity in Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing MNNs are explicit, with predefined modular architectures and individual modules expected to implement distinct functions. |
| Approach: | They propose to unlock emergent modularity in language models by fine-tuning them as Mixture-of-Experts (MoEs) EMoE is robust to various configurations and can scale up to Large Language Models . |
| Outcome: | The proposed models can be fine-tuned as Mixture-of-Expert (MoE) counterparts without introducing any extra parameters. |
Generalizing Few-Shot Named Entity Recognizers to Unseen Domains with Type-Related Features (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Few-shot named entity recognition methods struggle with out-of-domain (OOD) examples due to their reliance on manual labeling for the target domain. |
| Approach: | They propose a framework to enable generalization to an unseen target domain with only a few labeled examples. |
| Outcome: | The proposed framework achieves significant performance improvements on in-domain and cross-domain datasets. |
BERT Prescriptions to Avoid Unwanted Headaches: A Comparison of Transformer Architectures for Adverse Drug Event Detection (2021.eacl-main)
Copied to clipboard
| Challenge: | Pretrained transformer-based models are a common choice for identifying drug events from social media texts. |
| Approach: | They propose to compare transformer-based models with in-domain language pretraining to find out which one is better at ADE detection. |
| Outcome: | The proposed models outperform SpanBERT and PubMedBERT on two benchmarks. |
Contrastive Learning of Sociopragmatic Meaning in Social Media (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent progress in representation and contrastive learning in NLP has not considered the class of sociopragmatic meaning (i.e., meaning in interaction within different language communities). |
| Approach: | They propose a framework for learning task-agnostic representations transferable to a wide range of sociopragmatic tasks. |
| Outcome: | The proposed framework outperforms other contrastive learning frameworks for both in-domain and out-of-domain data, across both the general and few-shot settings. |
LatentGate: Low-Latency Semantic Routing via Frozen-Backbone Probing of Small Language Models (2026.acl-industry)
Copied to clipboard
| Challenge: | Prompt-based LLM routers deliver strong semantic reasoning but incur prohibitive latency and cost that scales with agent count. |
| Approach: | They propose a non-generative router that extracts mean-pooled hidden states from a frozen small language model and trains a lightweight linear probe for agent classification. |
| Outcome: | The proposed router achieves 98.8% in-domain and 80.0% OOD accuracy across 5 SLM backbones and 100 enterprise agents. |
AdapterSoup: Weight Averaging to Improve Generalization of Pretrained Language Models (2023.findings-eacl)
Copied to clipboard
| Challenge: | Pretrained language models often need to specialize to specific domains. |
| Approach: | They propose an approach that performs weight-space averaging of adapters trained on different domains. |
| Outcome: | The proposed approach improves performance to new domains without extra training. |
Efficient Cluster-Based k-Nearest-Neighbor Machine Translation (2022.acl-long)
Copied to clipboard
| Challenge: | k-Nearest-Neighbor Machine Translation (kNN-MT) is a non-parametric solution for domain adaptation . previous studies have shown that kNN retrieval is at the expense of high latency . |
| Approach: | They propose to use clustering to improve retrieval efficiency by combining a non-parametric MT with an in-domain feature-based retrieval module. |
| Outcome: | The proposed method reduces translation latency by 57% while maintaining the most useful information of the original datastore. |
A Retrospective Analysis of the Fake News Challenge Stance-Detection Task (C18-1)
Copied to clipboard
Andreas Hanselowski, Avinesh PVS, Benjamin Schiller, Felix Caspelherr, Debanjan Chaudhuri, Christian M. Meyer, Iryna Gurevych
| Challenge: | 2017 Fake News Challenge Stage 1 (FNC-1) shared task addressed a stance classification task as a crucial first step towards detecting fake news. |
| Approach: | They propose a new evaluation metric favoring the majority class, which can be easily classified, and propose stacked LSTM models that perform on par with the best systems, but is superior in predicting minority classes. |
| Outcome: | The proposed evaluation metric favors the majority class, which can be easily classified, and overestimates the true discriminative power of the methods. |
XMAD-Bench: Cross-Domain Multilingual Audio Deepfake Benchmark (2026.findings-eacl)
Copied to clipboard
Ioan-Paul Ciobanu, Andrei-Iulian Hîji, Nicolae Catalin Ristea, Paul Irofti, Cristian Rusu, Radu Tudor Ionescu
| Challenge: | Recent advances in audio generation led to an increasing number of deepfakes . however, these methods are typically tested in an in-domain setup . |
| Approach: | They propose a large-scale cross-domain audio deepfake benchmark comprising 668.8 hours of real and deepfak speech. |
| Outcome: | The proposed benchmark compares audio deepfake detectors with existing methods in the wild . the results show that the proposed methods perform better in different languages than existing methods . |
Multimodal Prompt Learning for Product Title Generation with Extremely Limited Labels (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to generate informative titles for products with limited labels are inadequate for novel products. |
| Approach: | They propose a prompt-based approach to generate attractive titles for novel products . they use multimodal prompts to preserve characteristics and writing styles of novel products. |
| Outcome: | The proposed approach achieves state-of-the-art results on novel product categories with limited labels. |
Laying Anchors: Semantically Priming Numerals in Language Modeling (2024.findings-naacl)
Copied to clipboard
| Challenge: | Numeracy is the comprehension of numbers, and numerals are important for comprehension. |
| Approach: | They propose methods to semantically prime numerals by generating anchors governed by the distribution of numeral in any corpus. |
| Outcome: | The proposed methods perform better on numeracy tasks for both in-domain and out-domain numerals. |
WangchanThaiInstruct: An instruction-following Dataset for Culture-Aware, Multitask, and Multi-domain Evaluation in Thai (2025.emnlp-main)
Copied to clipboard
Peerat Limkonchotiwat, Pume Tuchinda, Lalita Lowphansirikul, Surapon Nonesung, Panuthep Tasawong, Alham Fikri Aji, Can Udomcharoenchaikit, Sarana Nutanong
| Challenge: | Existing benchmarks for large language models rely on translations, missing cultural and domain specificity. |
| Approach: | They present a human-authored dataset for evaluation and instruction tuning in Thai . findings highlight need for culturally and professionally grounded instruction data . |
| Outcome: | a human-authored dataset for evaluation and instruction tuning in Thai outperforms translation-based models . findings highlight need for culturally and professionally grounded instruction data . |
Calibration Meets Explanation: A Simple and Effective Approach for Model Confidence Estimates (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to improve confidence calibration of pre-trained language models are still a mystery. |
| Approach: | They propose a method that leverages model explanations to make models less confident with non-inductive attributions. |
| Outcome: | The proposed method improves confidence calibration in all settings and reduces calibration errors when combined with temperature scaling. |
Measuring Your ASTE Models in The Wild: A Diversified Multi-domain Dataset For Aspect Sentiment Triplet Extraction (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing ASTE datasets are limited in their ability to represent real-world scenarios, hindering progress in this area. |
| Approach: | They propose a new ASTE dataset that is manually annotated to better fit real-world scenarios by providing more diverse and realistic reviews. |
| Outcome: | The proposed dataset is manually annotated to better fit real-world scenarios. |
On the Effects of Transformer Size on In- and Out-of-Domain Calibration (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Large pre-trained transformer language models are notoriously expensive to train . prior work has developed smaller, more compact models to reduce training costs . |
| Approach: | They propose to develop smaller, more compact transformer language models which can be calibrated in-domain . they show that smaller models can achieve competitive calibration compared to larger models . |
| Outcome: | The proposed models achieve competitive calibration and better calibration than larger models on a wide range of tasks. |
Enhancing the generalization for Intent Classification and Out-of-Domain Detection in SLU (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for intent classification are expensive to collect and train . evaluators have shown that the ability to detect out-of-domain utterances is limited . |
| Approach: | They propose to train a model with only IND data while supporting both intent classification and OOD detection. |
| Outcome: | The proposed model improves on existing models and strong baselines on four datasets. |
Event Linking: Grounding Event Mentions to Wikipedia (2023.eacl-main)
Copied to clipboard
| Challenge: | a new task for natural language understanding is called Event Linking . the context where an event is mentioned lacks the details of this event . |
| Approach: | They propose a new task to link an article's event mention to the most appropriate Wikipedia page . they collect a training set from Wikipedia and evaluate two models to test the task . |
| Outcome: | The proposed model is based on a dataset and a real-world news domain . it is expected that the most appropriate Wikipedia page will provide rich knowledge about the mention . |
Lifelong Learning of Topics and Domain-Specific Word Embeddings (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing lifelong topic models focus on indomain text streams in which each chunk only contains documents from a single domain. |
| Approach: | They develop a lifelong collaborative model that uses non-negative matrix factorization to learn topics and domain-specific word embeddings. |
| Outcome: | The proposed model can learn topics and domain-specific word embeddings from a lifelong collaborative model. |
Temporal Adaptation of BERT and Performance on Downstream Document Classification: Insights from Social Media (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Language use differs between domains and even within a domain, language use changes over time. |
| Approach: | They propose to use social media comments to study temporal adaptations in pre-trained language models. |
| Outcome: | The proposed model performs better on past than on future test sets, whereas adapting to domain does not improve performance on the downstream task. |
StyLEx: Explaining Style Using Human Lexical Annotations (2023.eacl-main)
Copied to clipboard
| Challenge: | Large pre-trained language models often learn spurious domain-specific words to make predictions. |
| Approach: | They propose a model that learns from human annotated explanations of stylistic features and jointly predicts them as model explanations. |
| Outcome: | The proposed model can provide human like stylistic lexical explanations without sacrificing performance on in-domain and out-of-domain datasets. |
Overcoming Catastrophic Forgetting During Domain Adaptation of Neural Machine Translation (N19-1)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) performs poorly without large training corpora. |
| Approach: | They propose a machine learning method that retains the majority of general-domain performance lost in continued training without degrading in-domain. |
| Outcome: | The proposed method retains the majority of general-domain performance lost in continued training without degrading in-domain performances. |
HardEval: Focusing on Challenging Tokens to Assess Robustness of NER (2020.lrec-1)
Copied to clipboard
| Challenge: | Named entity recognition (NER) systems are often evaluated on human annotations . a new evaluation method focuses on subsets of tokens that represent specific sources of errors . |
| Approach: | They propose a method that focuses on subsets of tokens that represent specific sources of errors. |
| Outcome: | The proposed evaluation method focuses on subsets of tokens that represent specific sources of errors. |
Confidence-based Ensembling of Perspective-aware Models (2023.emnlp-main)
Copied to clipboard
Silvia Casola, Soda Lo, Valerio Basile, Simona Frenda, Alessandra Cignarella, Viviana Patti, Cristina Bosco
| Challenge: | Human label variability has been a topic of research in the field of NLP recently . Exploiting disagreements in annotations has been shown to offer advantages for accurate modelling and fairer evaluation. |
| Approach: | They propose a highly perspectivist model that exploits disagreements in annotations to capture the subjectivity encoded in the annotation process. |
| Outcome: | The proposed model is validated on irony and hate speech detection scenarios in in-domain and cross-domain settings. |
Generalized but not Robust? Comparing the Effects of Data Modification Methods on Out-of-Domain Generalization and Adversarial Robustness (2022.findings-acl)
Copied to clipboard
| Challenge: | Data modification has been proposed as an effective solution for generalizing to out-of-domain (OOD) inputs. |
| Approach: | They propose to use data modification to generalize to out-of-domain inputs . they also analyze their adversarial robustness using a synthetic dataset . |
| Outcome: | The proposed data modification strategies improve OOD accuracy and AR, but data filtering hurts OOD on other tasks. |
SKIntern: Internalizing Symbolic Knowledge for Distilling Better CoT Capabilities into Small Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have high computational costs and privacy concerns due to their high computational expenses and data privacy. |
| Approach: | They propose a method that empowers SLMs to internalize symbolic knowledge and few-shot examples gradually through a progressive fine-tuning process. |
| Outcome: | The proposed approach outperforms state-of-the-art baselines by over 5% while reducing inference costs by up to 4 across a wide range of SLMs in both in-domain (ID) and out-of domain (OOD) tasks. |
Interpretable Proof Generation via Iterative Backward Reasoning (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing proof generation tasks require reasoning capabilities, but they usually just request for an answer without the reasoning procedure that would make it interpretable. |
| Approach: | They propose an iterative backward reasoning model to solve the proof generation tasks on rule-based Question Answering. |
| Outcome: | The proposed model improves in-domain performance and cross-domain transferability over existing models. |
Adapting Open Domain Fact Extraction and Verification to COVID-FACT through In-Domain Language Modeling (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to verify scientifically false online information are limited by the lack of training data in the scientific domain. |
| Approach: | They propose an in-domain language modeling method for fact extraction and verification systems . they use SCIFACT to extract scientifically false online information . |
| Outcome: | The proposed method improves accuracy 30% on SCIFACT dataset . state-of-the-art model achieves only 46.6% precision, which is hard to be trusted for users. |
Can Automatic Post-Editing Improve NMT? (2020.emnlp-main)
Copied to clipboard
| Challenge: | APE has been successful with statistical machine translation systems but has not been as successful over neural machine translation (NMT) systems. |
| Approach: | They propose to train neural APE models on a corpus of human post-edits of NMT and compile a larger corpus to test their hypothesis. |
| Outcome: | The proposed model can improve a strong in-domain NMT system, challenging the current understanding in the field. |
Learning to Synthesize Data for Semantic Parsing (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for synthesizing data for semantic parsing require handcrafted rules to synthesize new programs or utterance-program pairs. |
| Approach: | They propose to use a (non-neural) PCFG to model the composition of programs and a BART-based translation model to map a program to an utterance to learn a generative model from existing data. |
| Outcome: | The proposed model can be efficiently learned from existing data on benchmarks of GeoQuery and Spider. |
PRODIGy: a PROfile-based DIalogue Generation dataset (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing profiles-based dialogue datasets lack explicit profile representations or are difficult to collect. |
| Approach: | They propose a dataset that brings together multiple profiles for each speaker, and then integrates them together to provide a more comprehensive profile dimension set for generative language models. |
| Outcome: | The PRODIGy dataset provides a more comprehensive profile dimension set for each speaker. |
LegalSearchLM: Rethinking Legal Case Retrieval as Legal Elements Generation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on legal case retrieval have limited results . limited representations and legally irrelevant matches are often used . |
| Approach: | They propose a large-scale Korean LCR benchmark and a retrieval model that performs legal element reasoning over the query case. |
| Outcome: | a new model outperforms baseline models on a Korean LCR benchmark . it performs state-of-the-art on 411 diverse crime types in queries over 1.2M candidate cases . previous studies have shown that the model can generalize to out-of domain cases if it is trained on in-domain data . |
A Simple Yet Strong Domain-Agnostic De-bias Method for Zero-Shot Sentiment Classification (2023.findings-acl)
Copied to clipboard
| Challenge: | a recent study shows that large language models are biased to their pre-training data, leading to poor performance in prompt templates. |
| Approach: | They propose a domain-agnostic data construction method to de-bias a given prompt template . they show that domain-based generic responses are superior to in-domain ground-truth data . |
| Outcome: | The proposed method improves sentiment analysis tasks across domains and domains . it also yields better performance than existing in-domain models . |
Cross-Domain Evaluation of Edge Detection for Biomedical Event Extraction (2020.lrec-1)
Copied to clipboard
| Challenge: | Biomedical event extraction systems are evaluated in-domain and on complete event structures only. |
| Approach: | They present a cross-domain study of edge detection for biomedical event extraction . they analyze differences between five existing gold standard corpora and provide a strong baseline model . |
| Outcome: | The proposed model shows a drop in performance when the baseline is applied on out-of-domain data. |
Monolingual and Cross-Lingual Acceptability Judgments with the Italian CoLA corpus (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Acceptability judgments are the most significant source of data in linguistics . however, there are still many open issues regarding methods for collecting and evaluating them. |
| Approach: | They propose to create a corpus of sentences with acceptability judgments using the same approach and the same steps as the English corpus. |
| Outcome: | The proposed corpus contains almost 10,000 sentences with acceptability judgments. |
A Simple and Robust Approach to Detecting Subject-Verb Agreement Errors (N19-1)
Copied to clipboard
| Challenge: | a recent study shows that neural sequential labelers overfit their training data to detect SVA errors. |
| Approach: | They propose a simple protocol that generates a neural sequential labeler from silver standard data and gold standard data. |
| Outcome: | The proposed method leads to more robust detection of SVA errors on silver standard data and gold standard data. |
ChildTalk: A Multi-Dialect Chinese Child Speech Corpus with Full-Length Child–Caregiver Conversations for Speech Recognition (2026.findings-acl)
Copied to clipboard
| Challenge: | Automatic speech recognition (ASR) for children remains challenging due to developmental variability and the scarcity of high-quality corpora. |
| Approach: | They propose a large-scale Chinese child speech corpus that contains 112.5 hours of speech from 498 children and 500 caregivers. |
| Outcome: | The proposed model improves in-domain and cross-domain performance on children's speech. |
Coarse-to-fine Few-shot Learning for Named Entity Recognition (2023.findings-acl)
Copied to clipboard
Ruotian Ma, Zhang Lin, Xuanting Chen, Xin Zhou, Junzhe Wang, Tao Gui, Qi Zhang, Xiang Gao, Yun Wen Chen
| Challenge: | Existing few-shot NER solutions do not consider sub-class discrimination and various granularity of new classes during coarse training. |
| Approach: | They propose a method that uses a cluster-based prototype loss to learn group-wise discriminative representations of coarse-grained classes and a mixture prototype loss for learning the representations. |
| Outcome: | The proposed method shows superior performance over baseline methods on in-domain and cross-domain settings with various target granularity. |
Unlocking Parameter-Efficient Fine-Tuning for Low-Resource Language Translation (2024.findings-naacl)
Copied to clipboard
| Challenge: | Parameter-efficient fine-tuning (PEFT) methods are important in low-resource language (LRL) Neural Machine Translation (NMT) but their practical effectiveness varies significantly across different languages. |
| Approach: | They evaluated the performance of 8 parameters-efficient fine-tuning methods with 15 architectures using the SacreBLEU score. |
| Outcome: | The Houlsby+Inversion adapter outperforms the baseline architectures in both in-domain and out-domain tests and the Houlson+Inverter achieves the best performance overall. |
ProsocialDialog: A Prosocial Backbone for Conversational Agents (2022.emnlp-main)
Copied to clipboard
Hyunwoo Kim, Youngjae Yu, Liwei Jiang, Ximing Lu, Daniel Khashabi, Gunhee Kim, Yejin Choi, Maarten Sap
| Challenge: | Existing dialogue systems fail to respond properly to potentially unsafe user utterances . existing systems either ignore or passively agree with unsafe content . |
| Approach: | They introduce a dataset to teach conversational agents to respond to problematic content following social norms. |
| Outcome: | The proposed dataset shows that ProsocialDialog generates more socially acceptable dialogues than existing models. |
GenTKG: Generative Forecasting on Temporal Knowledge Graph with Large Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for temporal relational forecasting are limited and require limited training data. |
| Approach: | They propose a retrieval-augmented generation framework that uses temporal logical rule-based retrieval and parameter-efficient instruction tuning to solve temporal knowledge forecasting challenges. |
| Outcome: | The proposed framework outperforms conventional methods in the temporal knowledge graph domain with low computation resources. |
Generalised Unsupervised Domain Adaptation of Neural Machine Translation with Cross-Lingual Data Selection (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on unsupervised domain adaptation of neural machine translation assumes access to monolingual text in either the source or target language in the new domain. |
| Approach: | They propose a method to extract in-domain sentences from a large generic monolingual corpus from 'missing' text. |
| Outcome: | The proposed method outperforms baselines up to +1.5 BLEU score on five diverse domains in three language pairs and a real-world translation scenario. |
Neural Pipeline for Zero-Shot Data-to-Text Generation (2022.acl-long)
Copied to clipboard
| Challenge: | In data-to-text generation, training on in-domain data leads to overfitting and repeating training data noise. |
| Approach: | They propose to train pretrained language models on general-domain text-based operations by transforming single-item descriptions with modules trained on ordering, aggregation, and paragraph compression. |
| Outcome: | The proposed approach enables D2T generation from RDF triples in zero-shot settings. |
LESA: Linguistic Encapsulation and Semantic Amalgamation Based Generalised Claim Detection from Online Content (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing work on claim detection is built on the basis of a 'segregation' of claims across different domains. |
| Approach: | They propose a generalized generalized model that captures syntactic features through part-of-speech and dependency embeddings, as well as contextual features through a fine-tuned language model. |
| Outcome: | The proposed model outperforms baselines on six claim datasets by an average of 3 claim-F1 points and 2 claim-f1 points on the general-domain experiments. |
Domain Adversarial Fine-Tuning as an Effective Regularizer (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing fine-tuning techniques can degrade general-domain representations . however, fine-timing can lead to catastrophic forgetting of knowledge . |
| Approach: | They propose a new regularization technique that complements the task-specific loss used during fine-tuning with an adversarial objective. |
| Outcome: | Empirical results show that AFTER improves performance on various natural language understanding tasks compared to standard fine-tuning. |
TRUST-VL: An Explainable News Assistant for General Multimodal Misinformation Detection (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods focus on a single type of distortion and struggle to generalize to unseen scenarios. |
| Approach: | They propose a vision-language model that combines a question-aware visual amplifier module with a large-scale instruction dataset to support training. |
| Outcome: | The proposed model is able to generalize to multiple distortion types while requiring task-specific skills. |
Zero-Shot Cross-lingual Semantic Parsing (2022.acl-long)
Copied to clipboard
| Challenge: | Recent work in cross-lingual semantic parsing assumes access to high-quality machine translation systems and word alignment tools. |
| Approach: | They propose a multi-task encoder-decoder model to transfer parsing knowledge to additional languages using only English-logical form paired data and in-domain natural language corpora. |
| Outcome: | The proposed model performs significantly above translation-based baselines and competes with the supervised upper-bound. |
Continual Generalized Intent Discovery: Marching Towards Dynamic and Open-world Intent Recognition (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Currently, the generalized intent classification system only considers one stage of OOD learning and requires all IND data for joint training. |
| Approach: | They propose a task that detects OOD intents from dynamic OOD data streams . they propose CGID method that bootstraps new intent discovery through class prototypes . |
| Outcome: | The proposed task can detect out-of-domain (OOD) queries and extend them to the in-domain classifier . it can safely and efficiently detect out of-domain queries and avoid wrong operations . |
UAlign: Leveraging Uncertainty Estimations for Factuality Alignment on Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) often struggle to accurately express factual knowledge, especially in cases where the knowledge boundaries are ambiguous. |
| Approach: | They propose a framework that leverages Uncertainty estimations to represent knowledge boundaries and incorporates these representations into prompts for LLMs to Align with factual knowledge. |
| Outcome: | The proposed framework significantly improves the LLMs’ capacities to confidently answer known questions and refuse unknown questions on both in-domain and out-of-domain tasks. |
Regulatory Compliance through Doc2Doc Information Retrieval: A case study in EU/UK legislation where text similarity has limitations (2021.eacl-main)
Copied to clipboard
| Challenge: | Major scandals in corporate history have urged the need for regulatory compliance, where organizations need to ensure that their controls (processes) comply with relevant laws, regulations, and policies. |
| Approach: | They introduce regulatory information retrieval (REG-IR) an application of document-to-document information retrievals where the query is an entire document making the task more challenging than traditional IR where the queries are short. |
| Outcome: | The proposed approach is more challenging than traditional IR where the query is an entire document making the task more challenging. |
Pruning-then-Expanding Model for Domain Adaptation of Neural Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for domain adaptation suffer from catastrophic forgetting, large domain divergence, and model explosion. |
| Approach: | They propose a method which prunes the model and keeps the important neurons or parameters responsible for both general-domain and in-domain translation. |
| Outcome: | The proposed method improves on different language pairs and domains compared with strong baselines. |
Neural-Symbolic Inference for Robust Autoregressive Graph Parsing via Compositional Uncertainty Quantification (2022.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained models excel at graph semantic parsing with rich annotated data, but generalize poorly to out-of-distribution and long-tail examples. |
| Approach: | They propose a compositionality-aware approach to neural-symbolic inference informed by model confidence to capture different aspects of the graph prediction. |
| Outcome: | The proposed method outperforms state-of-the-art models on an English resource grammar parsing problem on standard in-domain and seven OOD corpora. |
A Data Cartography based MixUp for Pre-trained Language Models (2022.naacl-main)
Copied to clipboard
| Challenge: | Using MixUp, additional samples are generated during training by combining random pairs of training samples and their labels. |
| Approach: | They propose a new MixUp strategy that leverages Training Dynamics and allows more informative samples to be combined for generating new data samples. |
| Outcome: | The proposed method achieves competitive performance using a smaller subset of training data compared with strong baselines and yields lower expected calibration error on the pre-trained language model, BERT, on both in-domain and out-of-domain settings. |
When depth is redundant: Efficient transformer-based speech anti-spoofing (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing anti-spoofing countermeasures exhibit limited generalization to unseen spoof attacks, especially in out-of-domain evaluation settings. |
| Approach: | They propose a training strategy that aligns shallow and intermediate representations with those of the final transformer layer for speech deepfake detection. |
| Outcome: | The proposed model improves robustness to unseen spoofing attacks and enhances out-of-domain generalization over strong baselines. |
Evaluating Parameter Efficient Learning for Generation (2022.emnlp-main)
Copied to clipboard
Peng Xu, Mostofa Patwary, Shrimai Prabhumoye, Virginia Adams, Ryan Prenger, Wei Ping, Nayeon Lee, Mohammad Shoeybi, Bryan Catanzaro
| Challenge: | Parameter efficient learning methods (PERMs) are gaining attention for their ability to adapt to a downstream task. |
| Approach: | They propose to use parameter efficient learning methods to improve model adaptation . they compare in-domain evaluations and generalizations to unseen domains and new datasets . |
| Outcome: | The proposed method outperforms finetuning and PERMs in in-domain evaluations. |
End-to-End AMR Coreference Resolution (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work on AMR focuses on individual sentences, but there is a need for multi-sentence AMRs. |
| Approach: | They propose to use an end-to-end AMR coreference resolution model to generate multi-sentence AMRs. |
| Outcome: | The proposed model reduces error propagation and is more robust for both in- and out-domain situations. |
Learning Cross-Lingual IR from an English Retriever (2022.naacl-main)
Copied to clipboard
| Challenge: | DR.DECR is a cross-lingual information retrieval system trained using multi-stage knowledge distillation (KD) DRDECR demonstrates superior accuracy over direct fine-tuning with labeled CLIR data. |
| Approach: | They propose a cross-lingual information retrieval system with multi-stage knowledge distillation . they teach powerful multilingual representations and CLIR by optimizing two corresponding KD objectives . |
| Outcome: | The proposed system is the best single-model retriever on the XOR-TyDi benchmark . it is based on a multi-stage knowledge distillation process that can be expensive . |
GVdoc - Graph-based Visual DOcument Classification (2023.findings-acl)
Copied to clipboard
| Challenge: | Visual document classifiers have shown impressive performance on in-distribution test sets, but they have a hard time classifying out-of-difference examples. |
| Approach: | They propose a graph-based document classification model that generates a document graph based on its layout and trains a neural network to learn node and graph embeddings. |
| Outcome: | The proposed model outperforms state-of-the-art models on out-of distribution data while maintaining comparable performance on in-distribution data. |
Adversarial Augmentation Policy Search for Domain and Cross-Lingual Generalization in Reading Comprehension (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Reading comprehension models often overfit to nuances of training datasets and fail at adversarial evaluation. |
| Approach: | They propose a method that introduces multiple points of confusion within the context and shows dependence on insertion location of the distractor. |
| Outcome: | The proposed methods improve robustness against adversarial evaluation but weak generalization to the source domain and new domains and languages. |
Unlocking General Long Chain-of-Thought Reasoning Capabilities of Large Language Models via Representation Engineering (2025.acl-long)
Copied to clipboard
| Challenge: | Existing work finds that long CoT reasoning can be efficiently elicited by tuning on only a few examples and can easily transfer to other tasks. |
| Approach: | They propose a representation engineering method to unleash the general long CoT reasoning capabilities of LLMs. |
| Outcome: | The proposed method is effective in in-domain and cross-domain scenarios. |
Improving Cross-task Generalization of Unified Table-to-text Models with Compositional Task Configurations (2023.findings-acl)
Copied to clipboard
Jifan Chen, Yuhao Zhang, Lan Liu, Rui Dong, Xinchi Chen, Patrick Ng, William Yang Wang, Zhiheng Huang
| Challenge: | Existing methods for multitask learning typically use a dataset name as input prefix, which limits the effectiveness of multitask training. |
| Approach: | They propose compositional task configurations, a set of prompts prepended to the encoder to improve cross-task generalization of unified models. |
| Outcome: | The proposed model outperforms the UnifiedSKG baseline by noticeable margins in both in-domain and zero-shot settings. |
Compressing Language Models for Specialized Domains (2026.eacl-long)
Copied to clipboard
| Challenge: | Language models (LMs) excel at tasks across diverse domains, yet require substantial computational resources during inference. |
| Approach: | They propose a calibration method to improve the in-domain performance of compressed LMs in a post-training setting. |
| Outcome: | The proposed method outperforms existing methods on domain-specific tasks while preserving general performance. |
Beyond Under-Alignment: Atomic Preference Enhanced Factuality Tuning for Large Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing work evaluates the factuality of large language models on in-domain (ID) datasets and the factuality on out-of-domain datasets. |
| Approach: | They propose a framework that enhances model’s awareness of factuality at the granularity of individual facts and propose 'Atomic Preference Enhanced Factuality Tuning' this framework enhances the model’ s awareness and accuracy of factual information at the level of individual factual facts. |
| Outcome: | The proposed framework improves model performance by an average of on ID and OOD datasets, which is highly effective. |
Self-Correction Makes LLMs Better Parsers (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have achieved remarkable success across various natural language processing tasks, but they still face challenges in performing fundamental NLP tasks, such as syntactic parsing. |
| Approach: | They propose a method that leverages grammar rules from existing treebanks to guide LLMs in correcting previous errors. |
| Outcome: | The proposed method significantly improves performance on in-domain and cross-domain datasets. |
Non-Parametric Unsupervised Domain Adaptation for Neural Machine Translation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | kNN-MT is a non-parametric method that uses nearest neighbor retrieval to translate out-of-domain sentences, rare words, etc. |
| Approach: | They propose a framework that directly uses in-domain monolingual sentences to build an effective datastore for k-nearest-neighbor retrieval. |
| Outcome: | The proposed framework improves translation accuracy with target-side monolingual data while achieving comparable performance with back-translation. |
BERM: Training the Balanced and Extractable Representation for Matching to Improve Generalization Ability of Dense Retrieval (2023.acl-long)
Copied to clipboard
| Challenge: | Dense retrieval has shown promise in the first-stage retrieval process when trained on in-domain labeled datasets. |
| Approach: | They propose a method to capture matching signal to improve generalization of dense retrieval by capturing matching signal between two texts. |
| Outcome: | The proposed method can be combined with different training methods to improve generalization ability without additional inference overhead and target domain data. |
On the Calibration of Pre-trained Language Models using Mixup Guided by Area Under the Margin and Saliency (2022.acl-long)
Copied to clipboard
| Challenge: | Existing studies have shown that mixing up can improve model calibration on image classification tasks, but little is known about using it on natural language understanding (NLU) tasks. |
| Approach: | They propose a mixup strategy for pre-trained language models that improves model calibration further by using the AUM statistic and saliency map. |
| Outcome: | The proposed mixup improves model calibration on natural language understanding tasks while maintaining competitive accuracy. |
NeuroCounterfactuals: Beyond Minimal-Edit Counterfactuals for Richer Data Augmentation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to produce counterfactuals rely on small perturbations via minimal edits, resulting in simplistic changes. |
| Approach: | They propose a novel approach to produce counterfactuals that allow for larger edits and linguistic diversity while still bearing similarity to the original document. |
| Outcome: | The proposed approach outperforms existing methods for generalizing natural language models under select settings. |
Reports of personal experiences and stories in argumentation: datasets and analysis (2022.acl-long)
Copied to clipboard
| Challenge: | Personal experiences and stories are important in argumentation, but they are not considered in the social sciences. |
| Approach: | They propose to use annotated documents to scale-up the analysis using existing annotations. |
| Outcome: | The proposed classifiers can identify documents containing personal experiences and reports . they can scale up to three domains and show that they perform well across domains. |
Enhancing Chinese Word Segmentation via Pseudo Labels for Practicability (2021.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models alleviate segmentation ambiguity and out-of-vocabulary (OOV) words. |
| Approach: | They propose a semisupervised neural method which distills knowledge from unlabeled data to a student model to improve both in-domain and out-of-domain CWS. |
| Outcome: | The proposed method can keep practicability of the lightweight student model and improve segmentation effectively on downstream Chinese NLP tasks. |
Leveraging Wikipedia article evolution for promotional tone detection (2022.acl-long)
Copied to clipboard
| Challenge: | Detecting biased language is useful for a variety of applications, authors say . a dataset for document-level promotional tone detection is available for WikiEvolve . |
| Approach: | They propose a dataset for document-level promotional tone detection using Wikipedia . they use a gradient reversal framework to encode two versions simultaneously . |
| Outcome: | The proposed dataset improves on in-domain and out-of-domain evaluations. |
Attention Head Masking for Inference Time Content Selection in Abstractive Summarization (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing studies show that multi-heads attentions at the same layer collectively guide the summarization. |
| Approach: | They propose an inference-time attention head masking mechanism that works on encoder-decoder attentions to pinpoint salient content at inference time. |
| Outcome: | The proposed technique outperforms state-of-the-art models on CNN/DailyMail and New York Times datasets and is data-efficient. |
Simple or Complex? Complexity-controllable Question Generation with Soft Templates and Deep Mixture of Experts Model (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on complex questions does not consider controlling complexity of generated questions. |
| Approach: | They propose an end-to-end neural complexity-controllable question generation model that incorporates a mixture of experts as the selector of soft templates to capture question similarity while avoiding the expensive construction of actual templates. |
| Outcome: | The proposed model is superior to state-of-the-art methods in both automatic and manual evaluations on two benchmark QA datasets. |
Addition in Four Movements: Mapping Layer-wise Information Trajectories in LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | arithmetic offers a compact test of whether large language models compute or memorize. |
| Approach: | They study multi-digit addition in LLaMA-3-8B-Instruct using linear probes and the Logit Lens . they find a consistent four-stage, layer-wise ordering of probe-decodable signal types across depth . |
| Outcome: | The proposed method shows that multi-digit addition is linearly decodable with high accuracy across layers. |
Does the Objective Matter? Comparing Training Objectives for Pronoun Resolution (2020.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models have been used to obtain state-of-the-art results on pronoun resolution. |
| Approach: | They propose to use pre-trained language models to train pronoun resolution models . they compare performance and seed-wise stability of four models that represent four objectives . |
| Outcome: | The proposed model performs best in-domain, while the other objectives perform poorly out-of-domain. |
Syllable-based Neural Thai Word Segmentation (2020.coling-main)
Copied to clipboard
| Challenge: | Word segmentation is a challenging pre-processing step for Thai Natural Language Processing due to the lack of explicit word boundaries. |
| Approach: | They propose a neural Thai word segmenter that uses syllable embeddings to capture linguistic constraints and dilated CNN filters to capture the environment of each character. |
| Outcome: | The proposed system outperforms the previous state-of-the-art system on both in-domain and out-domain datasets. |
Pre-train or Annotate? Domain Adaptation with a Constrained Budget (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that pre-training in-domain language models can boost performance when adapting to a new domain. |
| Approach: | They propose to combine annotation and pre-training to maximize performance under budget constraints. |
| Outcome: | The proposed approach is based on the annotation cost of three procedural text datasets and pre-training cost of 3 in-domain language models. |
Discriminative Nearest Neighbor Few-Shot Intent Detection by Transferring Natural Language Inference (2020.emnlp-main)
Copied to clipboard
Jianguo Zhang, Kazuma Hashimoto, Wenhao Liu, Chien-Sheng Wu, Yao Wan, Philip Yu, Richard Socher, Caiming Xiong
| Challenge: | Existing work on few-shot intent classification without OOS has focused on the few-shot intent classification with out-of-scope intents. |
| Approach: | They propose to use BERT-style pairwise encoding to train a binary classifier that estimates the best matched training example for a user input. |
| Outcome: | The proposed approach achieves more stable and accurate in-domain and OOS detection accuracy than RoBERTa-based classifiers and embedding-based nearest neighbor approaches. |
Linguistic representations for fewer-shot relation extraction across domains (2023.acl-long)
Copied to clipboard
| Challenge: | Recent work has demonstrated the positive impact of incorporating linguistic representations as additional context and scaffolds on performance in several NLP tasks. |
| Approach: | They extend previous work to examine whether linguistic representations enhance generalizability . they incorporate syntactic and semantic graphs from off-the-shelf tools into a transformer-based architecture . |
| Outcome: | The proposed approach enhances generalization by providing cross-domain pivots . it also shows that syntactic and semantic graphs exhibit roughly equivalent utility . |
Bernice: A Multilingual Pre-trained Encoder for Twitter (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing language models for Twitter are monolingual, adapted from other domains, or trained on limited amount of in-domain data. |
| Approach: | They propose a multilingual RoBERTa language model that is trained from scratch on 2.5 billion tweets with a custom tweet-focused tokenizer. |
| Outcome: | The proposed model outperforms or matches models trained on monolingual and multilingual tweets on a variety of benchmarks and is more efficient compute- and data-wise to train completely on in-domain data with a specialized domain-specific tokenizer. |
Self-Error-Instruct: Generalizing from Errors for LLMs Mathematical Reasoning (2025.acl-long)
Copied to clipboard
Erxin Yu, Jing Li, Ming Liao, Qi Zhu, Boyang Xue, Minghui Xu, Baojun Wang, Lanqing Hong, Fei Mi, Lifeng Shang
| Challenge: | Existing approaches to learning from errors synthesize training data by extrapolating from isolated bad cases, thereby failing to generalize the extensive patterns inherent within these cases. |
| Approach: | They propose a framework that synthesizes more generalized training data from isolated bad cases by extrapolating from isolated cases. |
| Outcome: | The proposed framework synthesizes more generalized training data to address these model weaknesses. |
WebIE: Faithful and Robust Information Extraction on the Web (2023.acl-long)
Copied to clipboard
| Challenge: | Existing closed IE datasets are built using Wikipedia, but they have limitations when applied to web domains. |
| Approach: | They propose to annotate 25K triples from WebIE through crowdsourcing and introduce mWebIE, a translation of the annotated set in four other languages. |
| Outcome: | The proposed model trains on 1.6M sentences from the English Common Crawl corpus and includes negative examples to better reflect the data on the web. |
Pseudo-Label Enhanced Prototypical Contrastive Learning for Uniformed Intent Discovery (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods focus on transferring in-domain (IND) prior knowledge to out-of-domain data through pre-training and clustering. |
| Approach: | They propose a Pseudo-Label enhanced Prototypical Contrastive Learning model for uniformed intent discovery that integrates supervised and pseudo signals from IND and OOD data. |
| Outcome: | The proposed method has been proven effective in two different settings of discovering new intents. |
Adversarial Self-Supervised Learning for Out-of-Domain Detection (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for detecting out-of-domain (OOD) intents are unsupervised and require extensive labeled data. |
| Approach: | They propose a self-supervised contrastive learning framework to model discriminative semantic features from unlabeled data. |
| Outcome: | The proposed framework outperforms baseline methods on two public benchmark datasets with a statistically significant margin. |
Teaching LLMs to Plan, Not Just Solve: Plan Learning Boosts LLMs Generalization in Reasoning Tasks (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for reinforcement learning (RL) on self-generated data are limited in many domains. |
| Approach: | a new framework combines plan-based search with Step-level Advantage Preference Optimization to optimize plan learning. |
| Outcome: | The proposed framework improves in-domain performance and out-of-domain benchmarks. |
MT3: A Synergistic Multi-Task RL Framework for Specializing MLLMs in Text Image Machine Translation (2026.acl-long)
Copied to clipboard
Zhaopeng Feng, Yupu Liang, Shaosheng Cao, Jiayuan Su, Jiahan Ren, Zhijie Zhou, Wenxuan Huang, Jian Wu, Zuozhu Liu
| Challenge: | Text Image Machine Translation (TIMT) is a critical subfield of machine translation . it requires accurate optical character recognition, robust visual-text reasoning, and high-quality translation a challenge . |
| Approach: | They propose a multi-task optimization framework to specialize MLLMs into expert TIMT models. |
| Outcome: | The proposed model outperforms baselines on the latest in-domain MIT-10M benchmark. |
Leveraging Task Transferability to Meta-learning for Clinical Section Classification with Limited Data (2022.acl-long)
Copied to clipboard
| Challenge: | Existing text classification systems require thousands of in-domain text data to achieve high performance. |
| Approach: | They propose an algorithm to improve task transferability of meta-learning-based text classification by normalizing negative conditional entropy from source task data to boost cross-domain meta- learning accuracy. |
| Outcome: | The proposed method improves section classification accuracy significantly compared to meta-learning algorithms. |
Step-level Value Preference Optimization for Mathematical Reasoning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for generating preference-level annotations do not capture the fine-grained quality of model outputs in multi-step reasoning tasks. |
| Approach: | They propose an algorithm to automatically annotate step-level preferences for multi-step reasoning using Monte Carlo Tree Search. |
| Outcome: | The proposed algorithm achieves state-of-the-art performance on in-domain and out-of domain mathematical reasoning benchmarks. |
Every Token Counts: Generalizing 16M Ultra-Long Context in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | a recent study explores efficient ultra-long context modeling. |
| Approach: | They propose to use Hierarchical Sparse Attention to achieve efficient ultra-long context modeling. |
| Outcome: | The proposed model performs comparable to full-attention baselines on in-domain and out-of-domain tasks. |
KG-Agent: An Efficient Autonomous Agent Framework for Complex Reasoning over Knowledge Graph (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods to design the interaction strategy between large language models and knowledge graphs (KGs) are not effective for large language model (LLM)s to solve complex tasks due to the large volume and structured format of KG data. |
| Approach: | They propose an LLM-based agent framework that enables small LLMs to actively make decisions over knowledge graphs. |
| Outcome: | The proposed framework outperforms existing methods on in-domain and out-domain datasets using 10K samples. |
LM-Searcher: Cross-domain Neural Architecture Search with LLMs via Unified Numerical Encoding (2025.emnlp-main)
Copied to clipboard
Yuxuan Hu, Jihao Liu, Ke Wang, Jinliang Zheng, Weikang Shi, Manyuan Zhang, Qi Dou, Rui Liu, Aojun Zhou, Hongsheng Li
| Challenge: | Recent advances in Large Language Models have opened new avenues for solving complex optimization problems, including Neural Architecture Search (NAS). |
| Approach: | They propose a framework that leverages LLMs for cross-domain neural architecture optimization without extensive domain-specific tuning. |
| Outcome: | The proposed framework achieves competitive performance in both in-domain and out-of-domain tasks. |
Contextualizing Hate Speech Classifiers with Post-hoc Explanation (2020.acl-main)
Copied to clipboard
| Challenge: | Modern text classifiers struggle to learn a model of hate speech that generalizes to real-world applications. |
| Approach: | They propose a method to regularize BERT classifiers to detect bias towards identity terms by providing explanations for group identifiers and allowing models to learn from the context of group identifiers. |
| Outcome: | The proposed method limiting false positives on out-of-domain data while maintaining and improving in-domain performance. |
Evaluating Retrieval for Multi-domain Scientific Publications (2022.lrec-1)
Copied to clipboard
Nancy Ide, Keith Suderman, Jingxuan Tu, Marc Verhagen, Shanan Peters, Ian Ross, John Lawson, Andrew Borg, James Pustejovsky
| Challenge: | a new framework for retrieval and mining of scientific publications is being developed . the AskMe retrieval engine is a bridge between xDD's publication database and the LAPPS Grid suite of NLP tools. |
| Approach: | They evaluate AskMe retrievalengine using BEIR benchmark datasets . they aim to determine when and why certain approaches perform well on in-domain and out-of-domain data. |
| Outcome: | The AskMe retrieval engine performs well on both in-domain and out-of-domain data. |
Implicit Reasoning in Transformers is Reasoning through Shortcuts (2025.findings-acl)
Copied to clipboard
| Challenge: | Language models can perform step-by-step reasoning and achieve high accuracy in both in-domain and out-of-domain tests via implicit reasoning. |
| Approach: | They train GPT-2 from scratch on a curated multi-step mathematical reasoning dataset and conduct analytical experiments to investigate how language models perform implicit reasoning in multi- step tasks. |
| Outcome: | The proposed model performs better on multi-step tasks than the explicit reasoning model. |
UniNL: Aligning Representation Learning with Scoring Function for OOD Detection via Unified Neighborhood Learning (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to detect out-of-domain (OOD) intents ignore alignment between representation learning and scoring function, limiting performance. |
| Approach: | They propose a unified neighborhood learning framework to detect OOD intents . they propose to align representation learning with scoring function . |
| Outcome: | The proposed method is able to detect out-of-domain (OOD) intents from user queries. |
Question Translation Training for Better Multilingual Reasoning (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models have shown compelling performance on reasoning tasks but they tend to perform much worse in languages other than English. |
| Approach: | They propose to train a model to translate reasoning questions into English by fine tuning on X-English parallel question data. |
| Outcome: | The proposed approach improves on LLaMA2-13B on the MGSM and MSVAMP multilingual reasoning benchmarks. |
Understanding Retrieval Robustness for Retrieval-augmented Image Captioning (2024.acl-long)
Copied to clipboard
| Challenge: | Recent retrieval-augmented models for image captioning are not perfect in practice. |
| Approach: | They propose to train a retrieval-augmented captioning model SmallCap by sampling retrieved captions from more diverse sets. |
| Outcome: | The proposed model is sensitive to tokens that appear in the majority of retrieved captions . the proposed model improves both in-domain and cross-domain performance . |
Selective Question Answering under Domain Shift (2020.acl-main)
Copied to clipboard
| Challenge: | Deployed question answering (QA) models need to know when to abstain from answering questions that diverge from their training data. |
| Approach: | They propose a selective question answering under domain shift in which a QA model is tested on a mixture of in-domain and out-of-domain data and must answer (i.e., not abstain on) as many questions as possible. |
| Outcome: | The proposed method answers 56% of questions while maintaining 80% accuracy. |
Improving Few-Shot Domain Transfer for Named Entity Disambiguation with Pattern Exploitation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Named entity disambiguation is a critical subtask of entity linking . a model can be trained on a domain, but it needs to be adapted to the domain . |
| Approach: | They propose to reformulate named entity disambiguation as a masked language modeling problem. |
| Outcome: | The proposed model improves on a mental health news dataset without sacrifices in accuracy. |
Compressive Summarization with Plausibility and Salience Modeling (2020.emnlp-main)
Copied to clipboard
| Challenge: | a new method to learn which compressions to apply is based on syntactic rules for deleting spans . plausibility and salience are the two main criteria for determining which compression to apply . a recent study shows that the plausability model generally selects for grammatical and factual deletions compared to extractive methods . |
| Approach: | They propose to leave the decision about what to delete to two data-driven criteria . they show that plausibility and salience are the most important criteria if a span is deleted . |
| Outcome: | The proposed method achieves strong in-domain results on benchmark datasets and human evaluation shows that plausibility model generally selects for grammatical and factual deletions. |
Cross-lingual neural fuzzy matching for exploiting target-language monolingual corpora in computer-aided translation (2022.emnlp-main)
Copied to clipboard
| Challenge: | CAT tools based on translation memories (TMs) are limited in their use for a number of translation tasks due to the limited availability of in-domain TMs. |
| Approach: | They propose a neural approach to exploit in-domain TMs and in-target-language (TL) monolingual corpora to exploit CAT tools. |
| Outcome: | The proposed approach exploits in-domain TMs and in-target-language (TL) monolingual corpora and increases translation proposals on four language pairs. |
Enhancing Automatic Term Extraction with Large Language Models via Syntactic Retrieval (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have improved IE, but their potential for ATE has not been explored. |
| Approach: | They propose a retrieval-based prompting strategy that selects demonstrations according to syntactic rather than semantic similarity in a few-shot setting. |
| Outcome: | The proposed method improves performance on three specialized ATE benchmarks. |
Layered Insights: Generalizable Analysis of Human Authorial Style by Leveraging All Transformer Layers (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to authorship attribution model only learn from the output layer of pre-trained transformers, ignoring representations learned at other layers. |
| Approach: | They propose a model that leverages the various linguistic representations learned at different layers of pre-trained transformer-based models to model the authorship attribution task more effectively. |
| Outcome: | The proposed model performs better on out-of-domain and in-domain scenarios, while ignoring representations learned at other layers. |
How to Best Use Syntax in Semantic Role Labelling (P19-1)
Copied to clipboard
| Challenge: | Existing studies on integrating external information into NLP tasks focus on word-level shallow features such as POS or chunk tags. |
| Approach: | They propose to integrate syntactic information into a neural ELMo-based SRL sequence labelling model by using a constituency representation as input features. |
| Outcome: | The proposed approach improves performance on the in-domain CoNLL’05 and CoNll’12 benchmarks. |
Class Name Guided Out-of-Scope Intent Classification (2024.findings-emnlp)
Copied to clipboard
Chandan Gautam, Sethupathy Parameswaran, Aditya Kane, Yuan Fang, Savitha Ramasamy, Suresh Sundaram, Sunil Sahu, Xiaoli Li
| Challenge: | SCOOS leverages semantic cues embedded in class labels to improve classification accuracy. |
| Approach: | They propose a method to create a compact feature space around class label semantics . they use a shared latent space between ID features and class names to minimize losses . |
| Outcome: | The proposed method outperforms existing methods for out-of-scope intent detection and ID intent classification. |
Decoupling Pseudo Label Disambiguation and Representation Learning for Generalized Intent Discovery (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for generalized intent discovery lack pseudo label disambiguation and representation learning. |
| Approach: | They propose a prototype learning framework to decouple pseudo label disambiguation and representation learning. |
| Outcome: | The proposed method can decouple pseudo label disambiguation and representation learning. |
Identifying Domain Adjacent Instances for Semantic Parsers (D18-1)
Copied to clipboard
| Challenge: | Semantic parsers map text to logical forms, which can then be used by downstream components to fulfill an action. |
| Approach: | They propose a simple sentence representation that emphasizes unexpected words . they formalize domain-adjacency problem and propose logical form representations . |
| Outcome: | The proposed approach improves the performance of a downstream semantic parser on in-domain and domain-adjacent instances. |
In-context Examples Selection for Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Large-scale generative models can perform a wide range of NLP tasks using in-context learning. |
| Approach: | They aim to understand the properties of good in-context examples for machine translation in both in-domain and out-of-domain settings. |
| Outcome: | The proposed model outperforms a strong kNN-MT baseline in 2 out of 4 out-of-domain datasets. |
Human or Neural Translation? (2020.coling-main)
Copied to clipboard
Shivendra Bhardwaj, David Alfonso Hermelo, Phillippe Langlais, Gabriel Bernier-Colborne, Cyril Goutte, Michel Simard
| Challenge: | a recent study shows that deep neural models have improved machine translation . identifying machine translation is still feasible, but is not yet known. |
| Approach: | They train and apply deep neural models to distinguish between human and machine translations . they use a monolingual and bilingual task to train and train 18 classifiers based on their results . |
| Outcome: | The proposed model improves the ability to distinguish between human and machine translations at the sentence level. |
A Chain-of-Task Framework for Instruction Tuning of LLMs Based on Chinese Grammatical Error Correction (2025.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to address Grammatical Error Correction (GEC) tasks are based on large scale labeled data, which leads to extremely high data annotation costs. |
| Approach: | They propose a Chain-of-Task framework to reduce over-correction in large language models . they propose supervised fine-tuning strategy and an algorithm for automatic dataset annotation . |
| Outcome: | The proposed framework achieves state-of-the-art on both FCGEC (in-domain) and NaCGEC (out-of domain) test sets. |
A Contrastive Cross-Channel Data Augmentation Framework for Aspect-Based Sentiment Analysis (2022.coling-1)
Copied to clipboard
| Challenge: | Aspect-based sentiment analysis is sensitive to multi-aspect challenges, resulting in multiple aspects in a sentence. |
| Approach: | They propose a framework that leverages an in-domain generator to construct more multi-aspect samples . they then boost the robustness of ABSA models via contrastive learning on these generated samples ." |
| Outcome: | The proposed framework outperforms baselines without any augmentations on accuracy and Macro- F1 . the proposed framework can generate more multi-aspect samples and boost the robustness of ABSA models . |
Query2doc: Query Expansion with Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for sparse and dense retrieval have limited success on popular datasets. |
| Approach: | They propose a query expansion approach that generates pseudo-documents by few-shot prompting large language models and then expands the query with generated pseudo-docs. |
| Outcome: | The proposed method boosts the performance of BM25 on ad-hoc IR datasets by 3% to 15% without any model fine-tuning. |
EARA: Improving Biomedical Semantic Textual Similarity with Entity-Aligned Attention and Retrieval Augmentation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to measure semantic similarity between biomedical texts are inefficient due to too many biomedically-related entities. |
| Approach: | They propose an entity-aligned, attention-based and retrieval-augmented PLM that aligns the same type of fine-grained entity information in each sentence pair with an entity alignment matrix with an auxiliary loss. |
| Outcome: | The proposed model can achieve state-of-the-art on both in-domain and out-of domain datasets. |
Variational Pretraining for Semi-supervised Text Classification (P19-1)
Copied to clipboard
| Challenge: | Empirically, we show the relative strength of VAMPIRE against computationally expensive contextual embeddings and other popular semi-supervised baselines under low resource settings. |
| Approach: | They propose a lightweight framework for effective text classification when data and computing resources are limited. |
| Outcome: | The proposed framework is compared with expensive contextual embeddings and semi-supervised baselines under low resource settings. |
Improving domain-specific SMT for low-resourced languages using data from different domains (L18-1)
Copied to clipboard
| Challenge: | Evaluation of domain-specific statistical machine translation system for official government letters . use of pseudo in-domain data showed improvement for both test sets . |
| Approach: | They develop a statistical machine translation system for official government letters . the system is based on a parallel in-domain dataset containing official letters based in Sinhala and Tamil . |
| Outcome: | The proposed system improves on the in-domain data in the domain of official government letters . the evaluations show that the system requires quality data from diverse subject matters and sources to perform better. |
Complex Reasoning over Logical Queries on Commonsense Knowledge Graphs (2024.acl-long)
Copied to clipboard
| Challenge: | Currently, language models struggle to generate commonsense inferences for complex tasks due to data scarcity and the difficulty of reasoning over multiple pieces of information. |
| Approach: | They propose a dataset to generate commonsense inferences from commonsensible data . they use a commonsence knowledge graph to extract and form questions from existing commonseense knowledge graphs. |
| Outcome: | The proposed dataset improves the ability of language models to reason about complex events without expensive human annotations. |
Calibrating Student Models for Emotion-related Tasks (2022.emnlp-main)
Copied to clipboard
| Challenge: | Knowledge distillation is an effective method to transfer knowledge from one network (a.k.a. teacher) to another (as student). |
| Approach: | They propose to use a mixup data augmentation technique to increase the accuracy of the model by providing better training signals to the student models. |
| Outcome: | The proposed method improves the calibration of student models while providing better training signals to the student models using training dynamics. |
CAPSTONE: Curriculum Sampling for Dense Retrieval with Document Expansion (2023.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that dense retrieval models are better at obtaining query-informed representations. |
| Approach: | They propose a dual-encoder approach that computes latent representations of query and document independently, but inference replaces the real query with a generated one. |
| Outcome: | The proposed approach outperforms previous dense retrieval models on in-domain and out-of-domain datasets. |
LLMs-as-Instructors: Learning from Errors Toward Automating Model Improvement (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Using advanced Large Language Models, instructors can improve training of smaller models by analyzing their own model's errors. |
| Approach: | They propose a framework that leverages advanced Large Language Models to enhance training of smaller target models. |
| Outcome: | The proposed framework outperforms ChatGPT on multiple benchmarks and shows that it improves on both in-domain and out-of-domain benchmarks. |
Unleashing LLM Reasoning Capability via Scalable Question Synthesis from Scratch (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods to improve the mathematical reasoning capabilities of Large Language Models (LLMs) are limited due to the proprietary nature of the data. |
| Approach: | They propose a data synthesis method that generates large-scale mathematical reasoning datasets using lightweight 7B-scale models. |
| Outcome: | The proposed method outperforms existing open-source datasets in both in-domain and out-of-domain evaluations and shows improvements in code reasoning tasks. |
Adversarial Removal of Demographic Attributes Revisited (D19-1)
Copied to clipboard
| Challenge: | Several approaches have been proposed to learn classifiers that are invariant (unbiased with respect) to protected attributes. |
| Approach: | They propose to use a diagnostic classifier trained on a held-out subsample to find protected attributes for mention detection at above-chance levels. |
| Outcome: | The proposed classifier generalizes poorly to new in-domain and new domains, suggesting it relies on correlations specific to their particular data sample. |
Assessing Users’ Reputation from Syntactic and Semantic Information in Community Question Answering (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing reputation systems do not take linguistic quality into account in reputation scores estimation. |
| Approach: | They build statistical models that learn reputation from syntactic and semantic structures extracted from their associated answers content. |
| Outcome: | The proposed models show that users’ writing styles play important roles in building reputation points. |
Multitask Learning for Cross-Lingual Transfer of Broad-coverage Semantic Dependencies (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for developing broad-coverage semantic dependency parsers for languages without semantically annotated data are limited to English, Czech and Chinese. |
| Approach: | They propose a multitask learning framework coupled with annotation projection to build broad-coverage semantic dependency parsers for languages without annotated resources. |
| Outcome: | The proposed model improves labeled F1 score on multitask tasks from English to Czech compared to baseline models . |
CITADEL: Conditional Token Interaction via Dynamic Lexical Routing for Efficient and Effective Multi-Vector Retrieval (2023.acl-long)
Copied to clipboard
Minghan Li, Sheng-Chieh Lin, Barlas Oguz, Asish Ghoshal, Jimmy Lin, Yashar Mehdad, Wen-tau Yih, Xilun Chen
| Challenge: | Existing multi-vector retrieval methods are slower and require more space to store indices compared to their single-vektor counterparts. |
| Approach: | They propose a multi-vector retrieval method that uses dynamic lexical routing to route different token vectors to the predicted lexicals. |
| Outcome: | The proposed method achieves state-of-the-art performance on several benchmark datasets while being nearly 40 times faster than the current state-out-of the-art method. |
Coherent or Not? Stressing a Neural Language Model for Discourse Coherence in Multiple Languages (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work on coherence assessment using NLMs focuses on properties acquired from stand-alone sentences, but their ability to model discourse and pragmatic phenomena is still unclear. |
| Approach: | They propose to use a Neural Language Model to assess coherence in multiple languages to compare models' performance and to examine their performance in a cross-language scenario. |
| Outcome: | The proposed model can model coherent and incoherent text in multiple languages and in-domain settings. |
Masked Audio Text Encoders are Effective Multi-Modal Rescorers (2023.findings-acl)
Copied to clipboard
| Challenge: | Masked Language Models (MLMs) have proven to be effective for second-pass rescoring in Automatic Speech Recognition systems. |
| Approach: | They propose a multi-modal masked language model rescorer which integrates acoustic representations into the input space of MLM. |
| Outcome: | The proposed model reduces word error rate (WER) by 4%-16% on in-domain and 3%-7% on out-of-domain datasets over the text-only baseline. |
Unsupervised Domain Clusters in Pretrained Language Models (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods to select domains from large corpus of data are often over-simplistic and vague. |
| Approach: | They propose to use pre-trained language models to learn sentence representations that cluster by domains without supervision. |
| Outcome: | The proposed methods outperform established methods on domain selection and precision and recall with respect to an oracle selection. |
Variational Neural Machine Translation with Normalizing Flows (2020.acl-main)
Copied to clipboard
| Challenge: | Existing frameworks for learning informative latent variables are limited by limitations . existing models rely on strong assumptions on distribution of latent code . |
| Approach: | They propose to apply a variational neural machine translation framework to a Transformer . they propose to introduce a more flexible approximate posterior based on normalizing flows . |
| Outcome: | The proposed framework outperforms baseline models under in-domain and out-of-domain conditions. |
ClusterPrompt: Cluster Semantic Enhanced Prompt Learning for New Intent Discovery (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for identifying new intent categories focus on relations between utterances and clusters, while neglecting the usage of semantics. |
| Approach: | They propose a method that leverages contrastive learning and label semantic alignment to learn meaningful representations of intent clusters. |
| Outcome: | The proposed method outperforms existing methods and suggests meaningful intent labels. |
FIRST: Teach A Reliable Large Language Model Through Efficient Trustworthy Distillation (2024.emnlp-main)
Copied to clipboard
KaShun Shum, Minrui Xu, Jianshu Zhang, Zixin Chen, Shizhe Diao, Hanze Dong, Jipeng Zhang, Muhammad Raza
| Challenge: | Experimental results show that a well-calibrated model is more reliable than a fine-tuned model due to “tuning-induced mis-calibration”. |
| Approach: | They propose a method which utilizes a small portion of teacher’s knowledge to obtain a reliable language model in a cost-efficient way. |
| Outcome: | The proposed method reduces the computational burden by utilizing teacher's knowledge to obtain a reliable language model in a cost-efficient way. |
A Two-Stage Adaptation of Large Language Models for Text Ranking (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in pre-trained language models (PLMs) have significantly improved ranking performance in text ranking tasks. |
| Approach: | They propose a two-stage progressive paradigm to better adapt LLMs to text ranking by conducting continual pre-training on a large weakly-supervised corpus and performing SFT on high-quality data. |
| Outcome: | The proposed approach outperforms previous methods on in- and out-domain scenarios. |
Few-shot Prompting for Pairwise Ranking: An Effective Non-Parametric Retrieval Model (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies show that training examples improve zero-shot performance of supervised ranking models. |
| Approach: | They propose to augment supervised ranking models with pairs of queries and documents to improve their performance. |
| Outcome: | The proposed model outperforms the unsupervised models on in-domain and out-domain retrieval benchmarks. |
Topic-Guided Sampling For Data-Efficient Multi-Domain Stance Detection (2023.acl-long)
Copied to clipboard
| Challenge: | Stance Detection is a task that aims to identify the attitudes of an author towards a target of interest. |
| Approach: | They propose a topic-guided diversity sampling technique and a contrastive objective to improve stance detection using the produced set. |
| Outcome: | The proposed method outperforms the state-of-the-art on 16 datasets with in-domain and out-of domain evaluations and is more generalizable with an averaged 10.2 F1 on out-domain evaluation. |
Expand, Rerank, and Retrieve: Query Reranking for Open-Domain Question Answering (2023.findings-acl)
Copied to clipboard
| Challenge: | Empirically, EAR improves top-5/20 accuracy by 3-8 and 5-10 points . dense retrievers are limited by their inability to perform semantic matching for relevant passages that have low lexical overlap with the query. |
| Approach: | They propose a query expansion and reranking approach for improving passage retrieval with the application to open-domain question answering. |
| Outcome: | Empirically, EAR improves top-5/20 accuracy by 3-8 and 5-10 points when compared to a vanilla query expansion model and a dense retrieval model. |
Identification of Fine-Grained Location Mentions in Crisis Tweets (2022.lrec-1)
Copied to clipboard
| Challenge: | Recent studies have focused on identifying informative tweets by individuals affected by a crisis, without considering their specific types. |
| Approach: | They assemble two tweet crisis datasets and manually annotate them with specific location types to facilitate progress on the fine-grained location identification task. |
| Outcome: | The proposed model performs well in both in-domain and cross-domain settings. |
Task Knowledge Injection via Interpolations and Reinstatement for Large Language Model Generalization (2025.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained large language models have been widely adopted to elicit their superior performance on downstream tasks, but instruction tuning may overfit them to specific task formats, compromising their generalization on unseen tasks. |
| Approach: | They propose to inject latent task adaptation and knowledge reinstatement into large language models to mitigate spurious correlations between inputs and targets. |
| Outcome: | The proposed method improves generalization on in-domain and out-of-domain unseen tasks. |
Revisiting Sparse Retrieval for Few-shot Entity Linking (2023.emnlp-main)
Copied to clipboard
| Challenge: | Entity linking (EL) aims to link ambiguous mentions to their corresponding entities in a knowledge base. |
| Approach: | They propose an ELECTRA-based keyword extractor to denoise the mention context and construct a better query expression. |
| Outcome: | The proposed method outperforms state-of-the-art models on the ZESHEL dataset by a significant margin. |
Context Compression for Auto-regressive Transformers with Sentinel Tokens (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing Transformer-based LLMs have limited performance due to complexity of attention module . key-value cache is the major memory footprint and inference latency problem . |
| Approach: | They propose a plug-and-play approach that incrementally compresses token activation into compact ones . they also profile the benefit of context compression on improving the system throughout . |
| Outcome: | The proposed approach reduces memory footprint and inference latency by compressing tokens into compact ones. |
Referral Augmentation for Zero-Shot Information Retrieval (2024.findings-acl)
Copied to clipboard
| Challenge: | Referral-augmented retrieval improves zero-shot document retrieval in a variety of tasks . prior work shows sparse models struggle to reconcile with dense models . |
| Approach: | They propose a technique that concatenates document indices with referrals from other documents that cite or link to the given document. |
| Outcome: | The proposed technique outperforms generative text expansion techniques on structured tasks and improves on ACL paper retrieval. |
Small Agent Can Also Rock! Empowering Small Language Models as Hallucination Detector (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on hallucination detection rely heavily on closed-source LLMs such as GPT-4. |
| Approach: | They propose an LLM-based agent framework called HaluAgent that integrates LLMs, multi-functional toolbox and a memory mechanism for hallucination detection. |
| Outcome: | The proposed framework integrates the LLM, multi-functional toolbox, and can detect hallucinations on Chinese and English datasets. |
Few-Shot Named Entity Recognition: An Empirical Baseline Study (2021.emnlp-main)
Copied to clipboard
Jiaxin Huang, Chunyuan Li, Krishan Subudhi, Damien Jose, Shobana Balakrishnan, Weizhu Chen, Baolin Peng, Jianfeng Gao, Jiawei Han
| Challenge: | Existing methods to build named entity recognition systems with limited labeled data are lacking. |
| Approach: | They propose three orthogonal schemes to build named entity recognition systems when labeled data is limited. |
| Outcome: | The proposed NER systems outperform existing methods on few-shot and training-free settings. |
RetroLLM: Empowering Large Language Models to Retrieve Fine-grained Evidence within Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods rely on separate retrievers to fetch top-k text chunks for generating evidence, and they lack joint optimization. |
| Approach: | They propose a framework that integrates retrieval and generation into a single, auto-regressive process, enabling LLMs to directly generate fine-grained evidence from the corpus with constrained decoding. |
| Outcome: | Extensive experiments on five open-domain QA datasets demonstrate the proposed framework’s superior performance across both in-domain and out-of-domain tasks. |
Can Post-Training Transform LLMs into Causal Reasoners? (2026.findings-acl)
Copied to clipboard
| Challenge: | Causal inference is a core component of human cognition and requires decision-makers to distinguish between causation and association. |
| Approach: | They propose a dataset comprising seven core causal tasks for training and five diverse test sets and evaluate five different post-training approaches. |
| Outcome: | The proposed model achieves 93.5% accuracy on the CaLM benchmark, compared to 55.4% by OpenAI o3. |
ReLiK: Retrieve and LinK, Fast and Accurate Entity Linking and Relation Extraction on an Academic Budget (2024.findings-acl)
Copied to clipboard
| Challenge: | Entity Linking and Relation Extraction (EL) are fundamental tasks in Natural Language Processing. |
| Approach: | They propose a Retriever-Reader architecture for Entity Linking and Relation Extraction . they propose an input representation that incorporates the candidate entities alongside the text . |
| Outcome: | The proposed architecture achieves state-of-the-art in in- and out-of domain benchmarks while using academic budget training and with 40x inference speed compared to competitors. |
Iterative Nearest Neighbour Machine Translation for Unsupervised Domain Adaptation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for supervised domain adaptation of machine translation focus on fine-tuning, which is non-extensible. |
| Approach: | They propose to perform unsupervised domain adaptation in a non-parametric manner by using in-domain monolingual data and performing nearest neighbour inference on both forward and backward directions. |
| Outcome: | The proposed method significantly improves the in-domain translation performance and achieves state-of-the-art results among non-parametric methods. |
Towards the Necessity for Debiasing Natural Language Inference Datasets (2020.lrec-1)
Copied to clipboard
| Challenge: | Delexicalization of datasets can improve natural language inference performance . a dataset with a delexicalized version of the FEVER dataset is used for natural language learning . |
| Approach: | They propose two techniques for delexicalization that modify annotated datasets to control the importance of lexical entities. |
| Outcome: | The proposed methods maintain performance in-domain and improve performance in some out-of-domain settings. |
TRIAL: Token Relations and Importance Aware Late-interaction for Accurate Text Retrieval (2025.emnlp-main)
Copied to clipboard
| Challenge: | Late-interaction based multi-vector retrieval systems rely on a naive summation of token-level similarity scores . this leads to inaccurate relevance estimation due to tokenization of semantic units and the influence of low-content words. |
| Approach: | They propose a late-interaction-based multi-vector retrieval system that uses token relations and token importance in relevance scoring. |
| Outcome: | Extensive tests show that TRIAL achieves state-of-the-art accuracy compared to existing methods. |
Optimizing Question Semantic Space for Dynamic Retrieval-Augmented Multi-hop Question Answering (2025.acl-long)
Copied to clipboard
| Challenge: | Existing retrieval-augmented generation methods are insufficient for multi-hop question answering . however, they tend to generate hallucinations due to semantic mismatching . |
| Approach: | They propose to optimize question semantic space for dynamic retrieval-augmented multi-hop question answering by optimizing the semantic embeddings. |
| Outcome: | The proposed method outperforms existing RAG methods in both in- and out-of-domain settings. |
Generate, Discriminate, Evolve: Enhancing Context Faithfulness via Fine-Grained Sentence-Level Self-Evolution (2025.findings-acl)
Copied to clipboard
Kun Li, Tianhua Zhang, Yunxiang Li, Hongyin Luo, Abdalla Mohamed Salama Sayed Moustafa, Xixin Wu, James R. Glass, Helen M. Meng
| Challenge: | Existing methods to improve context faithfulness in large language models are either inadequate or overlook the potential for self-improvement. |
| Approach: | They propose a framework that enhances context faithfulness through fine-grained sentence-level optimization. |
| Outcome: | Experiments on ASQA and ConFiQA datasets show that GenDiE surpasses baselines in faithfulness and correctness and exhibits robust performance for domain adaptation. |
What Makes a Good Order of Examples in In-Context Learning (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) demonstrate impressive few-shot learning capabilities via in-context learning (ICL). |
| Approach: | They propose to use unlabeled data to evaluate order performance . they propose to filter out subsets of orders with label fairness and select the most influential order for each test instance. |
| Outcome: | The proposed method is superior over strong baselines and validates generalizability across settings. |
Tokenization Consistency Matters for Generative Models on Extractive NLP Tasks (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained sequence-to-sequence (seq2sequ) models have been widely used to solve extractive tasks, where parts of the input are extracted to form the desired output. |
| Approach: | They propose a simple fix to tokenization inconsistency that damages extractive nature of generative models by causing performance drop and hallucination. |
| Outcome: | The proposed model performs better in both in-domain and out-of-domain datasets with a notable average of +1.7 F1 gain when a BART model is trained on SQuAD and evaluated on 8 QA datasets. |
Check-COVID: Fact-Checking COVID-19 News Claims with Scientific Evidence (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing fact-checking benchmarks require systems to verify claims from everyday text against evidence from scientific journal articles. |
| Approach: | They propose a benchmark system that checks claims from news against scientific journal articles and veracity labels. |
| Outcome: | The new benchmark achieves F1 scores of 76.99 and 69.90 on both a fact-checking specific system and GPT-3.5, respectively. |
Nearest Neighbor Machine Translation is Meta-Optimizer on Output Projection Layer (2023.emnlp-main)
Copied to clipboard
| Challenge: | Nearest Neighbor Machine Translation (kNN-MT) is a powerful domain adaptation tool . the reasons for its success have not been thoroughly investigated . |
| Approach: | They propose to integrate pre-trained Neural Machine Translation models with token-level retrieval . they propose to implicitly execute gradient descent on the output projection layer of NMT . |
| Outcome: | The proposed approach outperforms model fine-tuning on in-domain tests while achieving better performance on out-of-domain sets. |
Representation Projection Invariance Mitigates Representation Collapse (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Fine-tuning contextualized representations by pre-trained models can lead to representation degradation, which can result in instability, sub-optimal performance, and weak generalization. |
| Approach: | They propose a regularization method to maintain the information content of representations and reduce representation collapse during fine-tuning. |
| Outcome: | The proposed method outperforms baselines on most tasks and improves out-of-distribution performance. |
Fairness in Automatic Speech Recognition Isn’t a One-Size-Fits-All (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained speech models like Whisper exhibit inconsistent group-level performance that varies across domains. |
| Approach: | They fine-tune a Whisper model on the Fair-Speech corpus using basic fine- tuning, demographic rebalancing, gender-swapped data augmentation and a novel contrastive learning objective. |
| Outcome: | The proposed method achieves stable, cross-domain fairness improvements without changes to the training data distribution and with minimal accuracy trade-offs. |
IOPO: Empowering LLMs with Complex Instruction Following via Input-Output Preference Optimization (2025.acl-long)
Copied to clipboard
| Challenge: | Existing algorithms to improve the ability of LLMs to follow complex instructions are lacking. |
| Approach: | They propose a benchmark to improve the ability to follow complex instructions by using a IOPO alignment method to take input and output preference into consideration. |
| Outcome: | The proposed algorithm shows 8.15%, 2.18% improvements on in-domain data and 5.91%, 2.83% on out-of-domain datasets compared to SFT and DPO respectively. |
S2R: Teaching LLMs to Self-verify and Self-correct via Reinforcement Learning (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to incentivize LLMs’ deep thinking abilities require large-scale data or significant training efforts. |
| Approach: | They introduce an efficient framework that enhances LLM reasoning by teaching models to self-verify and self-correct during inference. |
| Outcome: | The proposed framework outperforms models trained on long-CoT distilled data with 3.1k initialization samples and achieves an accuracy improvement of 51.0% to 81.6%. |
Tracing and Dissecting How LLMs Recall Factual Knowledge for Real World Questions (2025.acl-long)
Copied to clipboard
| Challenge: | Recent advances in large language models have shown promising ability to perform commonsense reasoning. |
| Approach: | They propose a two-dimensional analysis framework that incorporates token back-tracing and token decoding to uncover how LLMs conduct factual knowledge recall. |
| Outcome: | The proposed framework shows that LLMs lack relevant knowledge but struggle to select the most accurate information based on context during the retrieval and rerank phase. |
RAt: Injecting Implicit Bias for Text-To-Image Prompt Refinement Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Text-to-image prompt refinement (T2I-Refine) aims to rephrase or extend an input prompt with more descriptive details that can be leveraged to generate images with higher quality. |
| Approach: | They develop an adversarial prompt attacking framework that implicitly attacks input prompts with intentional adversarials to generate images with higher quality. |
| Outcome: | The proposed framework can implicitly attack input prompts with implicit concept biases to generate images with higher quality and explicit visual bias towards the target group. |
When Claims Evolve: Evaluating and Enhancing the Robustness of Embedding Models Against Misinformation Edits (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing claims-matching systems that use sentence embedding models are not robust to edits as users interact with claims online. |
| Approach: | They propose a perturbation framework that generates valid and natural claim variations and evaluate different mitigation approaches to improve their findings. |
| Outcome: | The proposed framework evaluates embedding models in a multi-stage retrieval pipeline and identifies the effectiveness of mitigation approaches. |
Task Oriented In-Domain Data Augmentation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for large language models suffer from two major issues: in-domain data are scarce compared with general domain-agnostic data. |
| Approach: | They propose a task-oriented in-domain data augmentation framework that uses in- domain data selection and task-orientated synthetic passage generation to adapt LLMs to two domains: advertisement and math. |
| Outcome: | The proposed framework improves LLM performance by 8% in the advertisement domain and 7.5% in the math domain. |
Diversification Catalyzes Language Models’ Instruction Generalization To Unseen Semantics (2025.findings-acl)
Copied to clipboard
| Challenge: | Instruction-tuned language models excel in knowledge, reasoning, and instruction-following . however, the factors enabling generalization to unseen instructions remain underexplored . |
| Approach: | They propose to model instruction-following as a computational process and design controlled experiments inspired by the Turing-complete Markov algorithm to disentangle its dynamics. |
| Outcome: | The proposed model outperforms scaling up data volumes in generalist models by combining in-domain and diverse out-of-domain tasks. |
Connecting the Knowledge Dots: Retrieval-augmented Knowledge Connection for Commonsense Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies show that large language models exhibit a limited understanding of commonsense reasoning due to the necessity of implicit knowledge that is rarely expressed in text. |
| Approach: | They propose a retrieval-augmented knowledge connection framework that transforms indirectly relevant documents into a direct explanation to answer a given question. |
| Outcome: | The proposed framework outperforms state-of-the-art (SOTA) benchmarks and achieves +2.0% and +4.6% average accuracy on in-domain (ID) and out-of domain (OOD) benchmark. |
\mathtt{GeLLM^3O}: Generalizing Large Language Models for Multi-property Molecule Optimization (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have remarkable out-of-domain generalizability to novel optimization tasks. |
| Approach: | They propose a series of instruction-tuned LLMs for molecule optimization that outperform state-of-the-art instruction-based LLM models. |
| Outcome: | mathttMuMOInstruct outperforms state-of-the-art LLMs on 5 in-domain and 5 out-of domain tasks. |
FairFlow: Mitigating Dataset Biases through Undecided Learning for Natural Language Understanding (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing debiasing frameworks can detect known dataset biases and spurious correlations in data. |
| Approach: | They propose a framework that learns to be undecided in its predictions for data samples . they propose 'contrary' objective that learn debiased and robust representations from biased views . |
| Outcome: | The proposed framework outperforms existing methods against out-of-domain and hard test samples without compromising performance. |
Full-Step-DPO: Self-Supervised Preference Optimization with Step-wise Rewards for Mathematical Reasoning (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to improve long-chain mathematical reasoning focus on the first erroneous step, but ignore all other steps and rely heavily on external signals. |
| Approach: | They propose a DPO framework that leverages step-wise rewards from the entire reasoning chain instead of optimizing only the first erroneous step. |
| Outcome: | The proposed framework improves on in-domain and out-of-domain mathematical reasoning benchmarks. |
Sebastian, Basti, Wastl?! Recognizing Named Entities in Bavarian Dialectal Data (2024.lrec-main)
Copied to clipboard
Siyao Peng, Zihang Sun, Huangyan Shan, Marie Kolm, Verena Blaschke, Ekaterina Artemova, Barbara Plank
| Challenge: | Named Entity Recognition (NER) is a fundamental task to extract key information from texts, but annotated resources are scarce for dialects. |
| Approach: | They present the first dialectal NER dataset for German, BarNER, with 161K tokens annotated on Bavarian Wikipedia articles and tweets. |
| Outcome: | The proposed dataset improves on bar-wiki and moderately on bartweet with training first on Bavarian . |
Iterative Repair with Weak Verifiers for Few-shot Transfer in KBQA with Unanswerability (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing models for KBQA with unanswerable questions are inadequate for real-world applications. |
| Approach: | They propose a task of few-shot transfer for KBQA with unanswerable questions that extends FuSIC-KBQA to include feedback for unanswered questions. |
| Outcome: | The proposed model outperforms suitable adaptations of multiple LLM-based and supervised SoTA models on the task while establishing a new performance for answerable few-shot transfer as well. |
Answer When Needed, Forget When Not: Language Models Pretend to Forget via In-Context Knowledge Unlearning (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly required to selectively unlearn specific information. |
| Approach: | They propose a method which fine-tunes pre-trained LLMs to enable prompt unlearning of target knowledge within the context while preserving unrelated information. |
| Outcome: | The proposed method achieves up to 95% forget accuracy while retaining 80% of unrelated knowledge, significantly outperforming baselines in both in-domain and out-of-domain scenarios. |
Do Generalisation Results Generalise? (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies evaluating generalisation performance on large language models focuses on a single out-of-distribution dataset . |
| Approach: | They examine whether OOD generalisation results generalise across multiple OOD testsets throughout a finetuning run and then evaluate the partial correlation of results . |
| Outcome: | The proposed model achieves high scores on multiple OOD testsets, regressing out in-domain performance. |
VCORE: Variance-Controlled Optimization-based Reweighting for Chain-of-Thought Supervision (2026.acl-long)
Copied to clipboard
| Challenge: | Empirical evaluations demonstrate that VCORE achieves the strongest overall average performance, with especially clear gains on lower-capacity models. |
| Approach: | They propose a framework that reformulates supervision as a constrained optimization problem. |
| Outcome: | Empirical evaluations show that VCORE achieves the strongest overall average performance, with especially clear gains on lower-capacity models. |
Chronos: Learning Temporal Dynamics of Reasoning Chains for Test-Time Scaling (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for testing time scales treat reasoning traces or tokens equally, ignoring substantial variations in trajectory quality and localized logical failures. |
| Approach: | They propose a chronological reasoning scorer that models each trajectory as a time series. |
| Outcome: | The proposed method achieves relative improvements of 34.21% over Pass@128 and 22.70% over Maj@135 on HMMT25, highlighting its effectiveness. |
MrGuard: A Multilingual Reasoning Guardrail for Universal LLM Safety (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are susceptible to adversarial attacks such as jailbreaking, which can elicit harmful or unsafe behaviors. |
| Approach: | They propose a multilingual guardrail with reasoning for prompt classification that integrates culturally and linguistically nuanced variants and supervised fine-tuning. |
| Outcome: | The proposed guardrail outperforms baselines across in-domain and out-of-domain languages by more than 15%. |
Translation with Thought: Difficulty-Adaptive Reasoning via Reinforcement Learning for Multi-Domain Machine Translation (2026.acl-long)
Copied to clipboard
| Challenge: | Multi-domain machine translation (MDMT) is a unique challenge due to varying levels of linguistic complexity across domains. |
| Approach: | They propose a resource-rational framework that learns to modulate inference between intuitive and deliberate reasoning. |
| Outcome: | Evaluated on 15 benchmarks spanning in-domain and out-of-domain settings, as well as 3 seen and 59 unseen languages, with ablations across three backbone models, TwT-7B and Twt-14B outperform much larger SOTA reasoning models in translation quality, while reducing token usage by 32–60%. |
Multi-view-guided Passage Reranking with Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing models rely on autoregressive generation and sliding window strategies to rank passages, which incur heavy computational overhead as the number of passages increases. |
| Approach: | They propose a non-generative LLM-based reranking method that encodes query-passage information into diverse view embeddings without being influenced by external biases. |
| Outcome: | The proposed model matches the performance of much larger 7B-scale fine-tuned models while achieving a 100x reduction in inference latency. |
More is not always better? Enhancing Many-Shot In-Context Learning with Differentiated and Reweighting Objectives (2025.acl-long)
Copied to clipboard
Xiaoqing Zhang, Ang Lv, Yuhan Liu, Flood Sung, Wei Liu, Jian Luan, Shuo Shang, Xiuying Chen, Rui Yan
| Challenge: | Large language models excel at few-shot in-context learning but performance plateaus as ICL demonstrations increase from a few to many. |
| Approach: | They propose a novel optimization method that optimizes the negative log-likelihood objective and reweights the model to achieve many-shot performance. |
| Outcome: | The proposed method achieves significant performance improvements across a large-scale dataset. |
CPC-GRPO: Answer-Free Reinforcement Learning with Cross-Prompt Consensus Rewards (2026.findings-acl)
Copied to clipboard
| Challenge: | Reinforcement learning with verifiable rewards is a popular post-training tool for large language models, but relies on a ground-truth answer or external verifier, which limits applicability and increases cost. |
| Approach: | They propose an answer-free training objective that derives rewards solely from the model’s own probabilities by exploiting prompt paraphrases as multiple semantic views of the same intent. |
| Outcome: | The proposed objective derives rewards solely from the model’s own probabilities by exploiting prompt paraphrases as multiple semantic views of the same intent. |
Prior Prompt Engineering for Reinforcement Fine-Tuning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have focused on algorithms, reward shaping, and data curation, but prior prompt engineering is understudied. |
| Approach: | They investigate prior prompt engineering (pPE) in reinforcement fine-tuning . they translate five representative iPE strategies into corresponding pPE approaches . |
| Outcome: | The proposed approaches outperform iPE-prompted models on in-domain and out-of-domain benchmarks. |
SPARKLE: A Structured and Plug-and-play Agentic Retrieval Policy for Adaptive RAG Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for integrating external knowledge rely on frozen large language models without explicit supervision or require costly LLM finetuning. |
| Approach: | They propose a structured and plug-and-play agentic retrieval policy with an additional proxy model to control the retrieval process. |
| Outcome: | Experiments on three in-domain and four out-of-domain QA benchmarks show that SPARKLE outperforms state-of the-art adaptive RAG models, achieving average improvements of 9.17% and 2.85%, respectively. |
A Head to Predict and a Head to Question: Pre-trained Uncertainty Quantification Heads for Hallucination Detection in LLM Outputs (2025.emnlp-main)
Copied to clipboard
Artem Shelmanov, Ekaterina Fadeeva, Akim Tsvigun, Ivan Tsvigun, Zhuohan Xie, Igor Kiselev, Nico Daheim, Caiqi Zhang, Artem Vazhentsev, Mrinmaya Sachan, Preslav Nakov, Timothy Baldwin
| Challenge: | Uncertainty quantification (UQ) is a framework for assessing the reliability of model outputs. |
| Approach: | They introduce pre-trained UQ heads for LLMs that are highly robust and generalized to languages they were not explicitly trained on. |
| Outcome: | The pre-trained heads significantly improve their ability to capture uncertainty compared to unsupervised methods. |
On-policy Reinforcement Fine-tuning with Offline reward for Multi-step Embodied Planning (2026.acl-long)
Copied to clipboard
| Challenge: | Embodied planning requires agents to make coherent multi-step decisions based on dynamic visual observations and verbal goals. |
| Approach: | They propose an On-policy Reinforcement fine-tuning framework with offline rewards for Embodied Task Planning that preserves generalization benefits of RFT while addressing costly interaction and sparse rewards. |
| Outcome: | The proposed framework outperforms closed-source and online-RL methods on EmbodiedBench, a recent benchmark for interactive embodied tasks. |
GerAV: Towards New Heights in German Authorship Verification using Fine-Tuned LLMs on a New Benchmark (2026.findings-acl)
Copied to clipboard
| Challenge: | Authorship verification (AV) is a task of determining whether two texts were written by the same author. |
| Approach: | They propose a benchmark for German AV comprising over 400k labeled text pairs. |
| Outcome: | The proposed model outperforms baselines and state-of-the-art models by 0.09 and surpasses GPT-5 in a zero-shot setting by 0.08. |
A Lightweight Explainable Guardrail for Prompt Safety (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods to detect unsafe prompts are based on reinforcement learning with human feedback (RLHF) or direct preference optimization (DPO) . however, these methods lack explainability and are costly to implement. |
| Approach: | They propose a lightweight explainable guardrail method to detect unsafe prompts using a multi-task learning architecture and a novel strategy to counteract confirmation biases. |
| Outcome: | The proposed method obtains equivalent or better performance than the state-of-the-art for both prompt classification and explainability on three datasets. |
CANDICE: Agentic Causal Disentanglement with Class Conditional Knowledge Integration for Long Tailed Domain Generalization (2026.findings-acl)
Copied to clipboard
| Challenge: | Domain generalization and long-tailed (LT) learning models face two challenges . domain invariance often suppresses class-discriminative signals essential for long-tail recognition. |
| Approach: | They propose a framework that disentangles domain-invariant and class-discriminative features . they evaluate 10 diverse medical imaging datasets spanning four modalities . |
| Outcome: | The proposed framework achieves an average performance improvement of 10.3% across multi-domain and in-domain long-tailed tasks while preserving minority class performance. |