Papers with heuristics
Reducing Confusion in Active Learning for Part-Of-Speech Tagging (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing algorithms for annotating parts of speech are not optimal for all languages. |
| Approach: | They propose to use a data selection algorithm to select useful training samples to minimize annotation cost. |
| Outcome: | The proposed strategy outperforms existing strategies on six typologically diverse languages. |
The emergence of number and syntax units in LSTM language models (N19-1)
Copied to clipboard
| Challenge: | a recent study shows that LSTMs can capture syntax-sensitive generalizations such as long-distance number agreement. |
| Approach: | They investigate the inner mechanics of number tracking in LSTMs at the single neuron level . they find that long-distance number information is largely managed by two "number units" importantly, the behaviour of these units is partially controlled by other units to track syntactic structure . |
| Outcome: | The proposed model is based on a language model with a long-distance number agreement task. |
Coreference Resolution without Span Representations (2021.acl-short)
Copied to clipboard
| Challenge: | Pretraining has reduced many complex task-specific NLP models to simple lightweight layers. |
| Approach: | They propose a lightweight end-to-end coreference model that removes the dependency on span representations, handcrafted features, pruning heuristics, and more. |
| Outcome: | The proposed model performs competitively with the current standard model, while being simpler and more efficient. |
Controlling Japanese Honorifics in English-to-Japanese Neural Machine Translation (D19-52)
Copied to clipboard
| Challenge: | In the Japanese language different levels of honorific speech are used to convey respect, deference, humility, formality and social distance. |
| Approach: | They propose a method for controlling the level of formality of Japanese output . they use heuristics to identify honorific verb forms to classify Japanese sentences . |
| Outcome: | The proposed model can produce Japanese translations in different honorific speech styles for the same English input sentence. |
Controlled Data Generation via Insertion Operations for NLU (2022.naacl-industry)
Copied to clipboard
| Challenge: | a new approach to annotate live traffic is emerging to be cost-effective and efficient . manual data annotation is expensive and not preferred for meeting customer privacy expectations . |
| Approach: | They propose a targeted synthetic data generation technique by inserting tokens into a given semantic signature. |
| Outcome: | The proposed approach achieves the same accuracy as training with all available data on a voice assistant dataset. |
Is a Document Educational or Just Wikipedia-Style? — Pitfalls of Classifier-Based Quality Filtering (2026.acl-short)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are pre-trained on massive data corpora, and the quality of these corporales is one of the main factors in achieving stateof-the-art performance. |
| Approach: | They propose to use Wikipedia-style reformatting to alter a model's quality assessment and enable low-quality content to surpass filtering thresholds. |
| Outcome: | The proposed model would reverse filtering decision for approximately 7% of evaluated documents, thereby admitting content into the pre-training corpus that would otherwise have been excluded. |
Visio-Linguistic Brain Encoding (2022.coling-1)
Copied to clipboard
| Challenge: | Existing studies have failed to explore co-attentive multi-modal modeling for visual and text reasoning. |
| Approach: | They propose to use image and multi-modal Transformers to reconstruct fMRI brain activity . they use two popular datasets to study visual and text reasoning . |
| Outcome: | The proposed model outperforms existing models on two popular datasets . the results raise the question whether visual processing is affected implicitly by linguistic processing . |
skweak: Weak Supervision Made Easy for NLP (2021.acl-demo)
Copied to clipboard
| Challenge: | skweak is a Python-based toolkit for NLP developers to use weak supervision . labelled data remains a scarce resource in many practical NLP scenarios . |
| Approach: | They present a Python-based toolkit that allows NLP developers to use weak supervision . skweak is designed to facilitate the use of weak supervision for NLP tasks . |
| Outcome: | skweak is a Python-based toolkit that facilitates weak supervision . the toolkit provides a simple interface to apply labels to a large corpus of text data . |
Neural Label Search for Zero-Shot Multi-Lingual Extractive Summarization (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods to translate sentences to other languages using heuristics are challenging. |
| Approach: | They propose a model that learns hierarchical weights for different sets of labels and applies them to other languages to translate them. |
| Outcome: | The proposed model can translate English datasets to other languages and obtain different sets of labels again using heuristics. |
Dynamic Data Selection for Curriculum Learning via Ability Estimation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing curriculum learning methods rely on heuristics to estimate difficulty of data . a major drawback is that they ignore competency of the model during training . |
| Approach: | They propose replacing difficulty heuristics with learned difficulty parameters . they propose a strategy that probes model ability at each training epoch . |
| Outcome: | The proposed strategy outperforms heuristic-based learning models on the GLUE classification tasks. |
On-the-fly Denoising for Data Augmentation in Natural Language Understanding (2024.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods to improve data augmentation performance may introduce noisy data that impairs training. |
| Approach: | They propose an on-the-fly denoising technique that learns from soft augmented labels provided by an organic teacher model trained on the cleaner original dataset. |
| Outcome: | The proposed method improves on text classification and question-answering tasks on general augmentation techniques and prevents overfitting on noisy labels. |
Data Programming for Learning Discourse Structure (P19-1)
Copied to clipboard
| Challenge: | Discourse structures are a relational semantic structure that convey causal, topical, argumentative relations or more generally coherence relations. |
| Approach: | They propose to use Snorkel to label training data using expert-composed heuristics and transform them into probability distributions of the class labels given to training candidates. |
| Outcome: | The proposed paradigm can be used for difficult tasks such as that of discourse attachment. |
Read, Listen, and See: Leveraging Multimodal Information Helps Chinese Spell Checking (2021.findings-acl)
Copied to clipboard
| Challenge: | Chinese Spell Checking (CSC) aims to detect and correct erroneous characters for usergenerated text in Chinese. |
| Approach: | They propose a Chinese spell checker that leverages multimodal Chinese characters' information to predict the correct output. |
| Outcome: | The proposed model outperforms strong baselines on the SIGHAN benchmarks by a large margin. |
Align-to-Distill: Trainable Attention Alignment for Knowledge Distillation in Neural Machine Translation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing knowledge distillation approaches to NMT often rely on heuristics when deciding which teacher layers to distill from. |
| Approach: | They propose an approach to align student attention heads with their teacher counterparts by heuristics to solve a feature mapping problem. |
| Outcome: | The proposed strategy shows gains of +3.61 and +0.63 BLEU points for WMT-2022 DeDsb and WMT-2014 EnDe compared to baselines. |
Investigating the Performance of Transformer-Based NLI Models on Presuppositional Inferences (2022.coling-1)
Copied to clipboard
| Challenge: | Presuppositions are assumptions that are taken for granted by an utterance. |
| Approach: | They propose to use heuristics to create alternative "contrastive" test cases . they also analyze samples from ImpPres datasets to better understand their predictions . |
| Outcome: | The proposed model performs better on the ImpPres dataset than on the other datasets. |
Investigating UD Treebanks via Dataset Difficulty Measures (2023.eacl-main)
Copied to clipboard
| Challenge: | Treebanks annotated with Universal Dependencies (UD) are currently available for over 100 languages and are only partially reflected in parser evaluations via accuracy metrics like LAS. |
| Approach: | They propose to use dataset cartography, V-information, and minimum description length to analyze UD treebanks using three accuracy-free methods to provide insights about them. |
| Outcome: | The proposed methods provide insights about UD treebanks that would remain undetected if only LAS was considered. |
EvolKV: Evolutionary KV Cache Compression for LLM Inference (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing key-value (KV) cache compression methods ignore interplays between layer-specific feature patterns and task performance. |
| Approach: | They propose an adaptive framework for layer-wise, task-driven KV cache compression that optimizes memory efficiency and task performance. |
| Outcome: | EvolKV outperforms baseline methods on long-context tasks and surpasses heuristics by 7 percentage points on GSM8K. |
AugESC: Dialogue Augmentation with Large Language Models for Emotional Support Conversation (2023.findings-acl)
Copied to clipboard
| Challenge: | Crowdsourced dialogue corpora are limited in scale and topic coverage due to the expensive cost of data curation. |
| Approach: | They construct an augmented dataset for the emotional support conversation task using large language models for dialogue augmentation. |
| Outcome: | The proposed approach outperforms baselines of dialogue augmentation and improves the model's generalization ability to open-domain topics. |
Which Melbourne? Augmenting Geocoding with Maps (P18-1)
Copied to clipboard
| Challenge: | Existing methods to associate geographic information in text with coordinates are limited by lexical features and cartesian coordinates. |
| Approach: | They propose a geocoder that exploits implicit lexical clues to associate coordinates with text . they propose encoding of geographic metadata to generate two distinct views of the same text. |
| Outcome: | The proposed method improves state-of-the-art results on three datasets and an open-source dataset for disease outbreaks and epidemics. |
ComFact: A Benchmark for Linking Contextual Commonsense Knowledge (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to retrieve facts from commonsense knowledge graphs are imprecise, requiring heuristics that ignore contexts and ambiguity . a novel benchmark, ComFact, contains 293k in-context relevance annotations for commonsensense triplets . |
| Approach: | They propose a task of commonsense fact linking where models are given contexts and trained to identify situationally-relevant commonsensical knowledge from KGs. |
| Outcome: | The proposed benchmark shows that heuristic fact linking approaches are imprecise . however, the models still significantly underperform humans in the commonsense augmentation task . |
RouteLMT: Learned Sample Routing for Hybrid LLM Translation Deployment (2026.acl-industry)
Copied to clipboard
Yingfeng Luo, Hongyu Liu, DingYang Lin, Kaiyan Chang, Chenglong Wang, Bei Li, Quan Du, Tong Xiao, JingBo Zhu
| Challenge: | Existing routing strategies rely on heuristics, external predictors, or absolute quality estimation to capture whether the large model provides a worthwhile improvement over the small one. |
| Approach: | They propose a budget allocation problem for routing large model to large model . they propose heuristics, external predictors, or absolute quality estimation to determine the optimal signal for budgeted decisions. |
| Outcome: | The proposed model outperforms heuristics, quality/difficulty estimation baselines and achieves a superior quality–budget Pareto frontier. |
Analyzing LLM Instruction Optimization for Tabular Fact Verification (2026.findings-eacl)
Copied to clipboard
Xiaotang Du, Giwon Hong, Wai-Chung Kwan, Rohit Saxena, Ivan Titov, Pasquale Minervini, Emily Allaway
| Challenge: | evaluating instruction optimization for tabular fact verification is a key challenge for reliable NLP systems. |
| Approach: | They compare instruction optimization for tabular fact verification with a framework based on DSPy . they find that instruction optimization consistently improves verification accuracy . |
| Outcome: | The proposed method improves verification accuracy across four benchmarks and three model families. |
LazyReview: A Dataset for Uncovering Lazy Thinking in NLP Peer Reviews (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models struggle to detect lazy thinking in a zero-shot setting, but instruction-based fine-tuning significantly boosts performance by 10-20 performance points. |
| Approach: | They propose to use LazyReview to train junior reviewers in the community to detect lazy thinking in peer-review sentences annotated with fine-grained lazy thinking categories. |
| Outcome: | The proposed dataset shows that LLMs struggle to detect lazy thinking instances in a zero-shot setting, while instruction-based fine-tuning significantly boosts performance by 10-20 performance points. |
Diffusion Language Model Inference with Monte Carlo Tree Search (2026.findings-eacl)
Copied to clipboard
Zheng Huang, Kiran Ramnath, Yueyan Chen, Aosong Feng, Sangmin Woo, Balasubramaniam Srinivasan, Zhichao Xu, Kang Zhou, Shuai Wang, Haibo Ding, Lin Lee Cheong
| Challenge: | Existing methods for inference use heuristics to determine which positions to unmask and which tokens to commit . MEDAL is an inference-time scaling framework that integrates Monte Carlo Tree SEarch initialization for Diffusion Language Model inference. |
| Approach: | They propose a framework that integrates Monte Carlo Tree SEarch initialization for Diffusion Language Model inference. |
| Outcome: | The proposed framework achieves 22.0% improvement over existing inference strategies across multiple benchmarks. |
Probabilistic, Structure-Aware Algorithms for Improved Variety, Accuracy, and Coverage of AMR Alignments (2021.acl-long)
Copied to clipboard
| Challenge: | Current algorithms for AMR parsing suffer from limited coverage and less-than-ideal accuracy . a new algorithm for AML uses unsupervised learning and heuristics to align components of AMR graphs to spans in English sentences . |
| Approach: | They propose algorithms for aligning components of Abstract Meaning Representation graphs to spans in English sentences. |
| Outcome: | The proposed approach covers a wider variety of AMR substructures than previously considered . it achieves higher coverage of nodes and edges, and does so with higher accuracy. |
WN-Salience: A Corpus of News Articles with Entity Salience Annotations (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing work on entity salience does not distinguish between salient and non-salient entities. |
| Approach: | They propose a dataset to measure entity salience using WikiNews dataset . WN-Salience is built on top of Wikinews, a Wikimedia project . |
| Outcome: | The proposed dataset can be used to benchmark tasks such as entity salience detection and salient entity linking. |
RW-KD: Sample-wise Loss Terms Re-Weighting for Knowledge Distillation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Knowledge Distillation (KD) is used to compress the pre-training and task-specific fine-tuning phases of large neural language models. |
| Approach: | They propose a sample-wise loss weighting method that re-weights the two losses for each sample. |
| Outcome: | The proposed method outperforms existing methods on 7 datasets of the GLUE benchmark. |
An LLM Feature-based Framework for Dialogue Constructiveness Assessment (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on dialogue constructiveness assessment focus on analysing conversational factors that influence individuals to take specific actions, win debates, change their perspectives or broaden their open-mindedness. |
| Approach: | They propose an LLM feature-based framework for dialogue constructiveness assessment that combines the strengths of feature- and neural approaches while mitigating their downsides. |
| Outcome: | The proposed framework outperforms standard feature-based models and neural models on three dialogue constructiveness datasets. |
Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference (P19-1)
Copied to clipboard
| Challenge: | lexical overlap heuristics are effective for frequent example types but break down in more challenging cases. |
| Approach: | They propose to use a set of examples to test whether a sentence entails another . they propose to adopt three fallible syntactic heuristics for statistical NLI models . |
| Outcome: | The proposed model performs poorly on the HANS dataset, suggesting they have adopted these heuristics. |
A Surprisingly Effective Fix for Deep Latent Variable Modeling of Text (D19-1)
Copied to clipboard
| Challenge: | Variational Autoencoders are powerful language models and effective representation learning frameworks. |
| Approach: | They propose a fix for posterior collapse which improves held-out likelihood, reconstruction and latent representation learning . |
| Outcome: | The proposed fix significantly improves held-out likelihood, reconstruction, and latent representation learning compared with previous state-of-the-art methods. |
Opening the Romance Verbal Inflection Dataset 2.0: A CLDF lexicon (2020.lrec-1)
Copied to clipboard
| Challenge: | lexicon provides verbal paradigm forms in broad IPA phonemic notation for 74 varieties . most resources used to study language evolution computationally rely on multilingual contemporary information . |
| Approach: | They propose a multilingual lexicon of Romance inflection covering 74 varieties . they annotate verbal paradigm forms in broad IPA phonemic notation and organize paradigm cells to reflect cognacy . |
| Outcome: | The lexicon provides verbal paradigm forms in broad IPA phonemic notation for 74 varieties. |
Learning to Detect Noisy Labels Using Model-Based Features (2022.findings-emnlp)
Copied to clipboard
Zhihao Wang, Zongyu Lin, Junjie Wen, Xianxin Chen, Peiqi Liu, Guidong Zheng, Yujun Chen, Zhilin Yang
| Challenge: | Existing approaches to reduce label noise rely on heuristics and sample losses. |
| Approach: | They propose a method that transfers the noise distribution to a clean set and trains a model to distinguish noisy labels from clean ones using model-based features. |
| Outcome: | Empirically, the proposed approach improves over strong baselines on a wide range of tasks including text classification and speech recognition. |
Connecting Attributions and QA Model Behavior on Realistic Counterfactuals (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent research in interpretability of neural models has yielded numerous token attribution techniques, but it is hard to evaluate whether these explanations are faithful. |
| Approach: | They propose to use pairwise attributions to connect outputs to high-level model behavior to examine how well different attribution techniques align with this assumption on realistic counterfactuals in the case of reading comprehension (RC). |
| Outcome: | The proposed methods are better suited to RC than token-level attributions across different RC settings, and the best performance comes from a modification that was proposed to an existing pairwise attribution method. |
MoR: Better Handling Diverse Queries with a Mixture of Sparse, Dense, and Human Retrievers (2025.emnlp-main)
Copied to clipboard
| Challenge: | Different retrievers offer distinct, often complementary signals, but they are not optimal for all queries. |
| Approach: | They propose a zero-shot, weighted combination of heterogeneous retrievers . they validate this intuition by incorporating specialized non-oracle human information sources . |
| Outcome: | Experiments show that a mixture of heterogeneous retrievers outperforms each retriever and larger models by +10.8% and +3.9% on average. |
Pyramid-BERT: Reducing Complexity via Successive Core-set based Token Selection (2022.acl-long)
Copied to clipboard
| Challenge: | Existing models that use heuristics to shorten sequence lengths are computationally prohibitive. |
| Approach: | They propose a new method to shorten sequence lengths by transforming tokens through encoders and a core-set based token selection method that avoids expensive pre-training and fine tuning. |
| Outcome: | The proposed model outperforms existing models on GLUE benchmarks and Long Range Arena datasets and demonstrates that it is cost-effective and space-efficient. |
Examining False Positives under Inference Scaling for Mathematical Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in language models have led to significant improvements in mathematical reasoning across benchmarks. |
| Approach: | They analyze the prevalence of false positives in language models by using heuristic evaluation methods . they find that false positive models produce correct final answers but with flawed deduction paths . |
| Outcome: | The proposed model performance improvements are based on the proposed model and its evaluation metrics. |
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling (2022.emnlp-main)
Copied to clipboard
| Challenge: | Abstractive summarization models often generate inconsistent summaries containing factual errors or fabricated content. |
| Approach: | They propose to generate representative examples of non-factual summaries through infilling language models and train a robust fact-correction model to post-edit them to improve factual consistency. |
| Outcome: | The proposed model outperforms previous methods in correcting factual errors on two popular summarization datasets. |
Transforming Wikipedia into a Large-Scale Fine-Grained Entity Type Corpus (L18-1)
Copied to clipboard
| Challenge: | et al. (2017): WiFiNE annotated with fine-grained entity types . lack of a well-established training corpus makes it difficult to manually annotate the amount of data needed for training. |
| Approach: | They propose an English corpus annotated with fine-grained entity types based on Wikipedia . they use heuristics to build a large, high quality, annotating corpus using 2 manually annotized benchmarks . |
| Outcome: | The proposed system outperforms the existing systems with two datasets and gains a 2.8 macro F1 score. |
Guidelines as Environments: A World Model Approach to Rule Following (2026.acl-long)
Copied to clipboard
| Challenge: | Existing models for guideline-following are a poor fit for ambiguous, text-defined constraints. |
| Approach: | They propose a Rule-Grounded Causal World Model that builds an explicit state space from guideline text itself. |
| Outcome: | Experiments show that the proposed model can be used to model rule execution with an explicit state space from the guideline text itself. |
Token-Level Metrics for Detecting Incorrect Gold Annotations in Named Entity Recognition (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Annotated datasets for supervised learning often contain incorrect labels, i.e. label noise. |
| Approach: | They compare popular sample metrics for detecting incorrect annotations in named entity recognition (NER) they find that training dynamics metrics work the best overall, and they detect errors that the model has not yet memorized . |
| Outcome: | The proposed measures reduce label noise across noise types by detecting errors in trained models. |
Data Whisperer: Efficient Data Selection for Task-Specific LLM Fine-Tuning via Few-Shot In-Context Learning (2025.acl-long)
Copied to clipboard
Shaobo Wang, Xiangqi Jin, Ziming Wang, Jize Wang, Jiajun Zhang, Kaixin Li, Zichen Wen, Zhong Li, Conghui He, Xuming Hu, Linfeng Zhang
| Challenge: | Using fine-tuning on task-specific data is essential for large language models to be effective in specialized tasks. |
| Approach: | They propose a method that leverages few-shot in-context learning with the model to be fine-tuned. |
| Outcome: | The proposed method outperforms existing methods with a 3.1-point improvement and a 7.4 speedup on the Llama-3-8B-Instruct model using just 10% of the dataset. |
Local Normalization Distortion and the Thermodynamic Formalism of Decoding Strategies for Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing decoding strategies for language models are based on heuristics and are difficult to improve. |
| Approach: | They propose a theory of decoding strategies for language models by expressing popular decoding algorithms as equilibrium states and stating objective functions they optimize. |
| Outcome: | The proposed method is based on ergodic theory and is difficult to improve . it shows that local normalization distortion is a fundamental defect of decoding strategies . |
Stability Implies Redundancy: Delta Attention Selective Halting for Efficient Long-Context Prefilling (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods to reduce sequence length rely on heuristics that break compatibility with hardware-efficient kernels like FlashAttention. |
| Approach: | They propose a method that selectively halts stabilized tokens by monitoring layer-wise update dynamics of the self-attention mechanism. |
| Outcome: | The proposed method can reduce prefill complexity while preserving model accuracy and hardware efficiency. |
Do Transformers Grok Succinct Algorithms? Mechanistic Evidence for Counting Circuits (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent studies suggest that Transformers are inherently succinct, capable of representing recursive algorithms like binary counting over exponential state spaces. |
| Approach: | They propose to bridge this gap by testing the Succinctness Hypothesis using mechanistic interpretability on a large-scale computation task. |
| Outcome: | The proposed model can represent recursive algorithms over exponential state spaces . the proposed model is able to generalize perfectly, whereas massive LSTM baselines fail completely. |
SOBR: A Corpus for Stylometry, Obfuscation, and Bias on Reddit (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing corpora are limited in scope and can be used to collect data on author attributes. |
| Approach: | They propose to use subreddits, flairs, and self-reports as distant labels for author attributes (age, gender, nationality, personality, and political leaning) . |
| Outcome: | The proposed method could be used to infer author attributes from public posts despite their discreetness and anonymity . |
A Theory of Response Sampling in LLMs: Part Descriptive and Part Prescriptive (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly utilized in autonomous decision-making, where they sample options from vast action spaces. |
| Approach: | They propose to use heuristics to sample LLMs to identify a prescriptive component and a descriptive component to represent a concept. |
| Outcome: | The proposed model is compared with human decision-making models in public health, economic trends and other real-world domains to show that it is biased. |