Papers with classifier
Double Trouble: How to not Explain a Text Classifier’s Decisions Using Counterfactuals Synthesized by Masked Language Models? (2022.aacl-main)
Copied to clipboard
| Challenge: | Input Marginalization (IM) is a method that takes the prediction difference between before-and-after an input feature (here, a token) is removed as its attribution. |
| Approach: | They propose to use a BERT-based method to replace a token with a feature to give more plausible counterfactuals. |
| Outcome: | The proposed method is effective, but the Deletion-BERT metric is biased towards IM, and the results are not convincing. |
Train, Sort, Explain: Learning to Diagnose Translation Models (N19-4)
Copied to clipboard
| Challenge: | Evaluating translation models is a trade-off between effort and detail. |
| Approach: | They propose to use a neural text classifier to automatically expose systematic differences between human and machine translations to human experts. |
| Outcome: | The proposed method exposes systematic differences between human and machine translations to human experts. |
Mapping (Dis-)Information Flow about the MH17 Plane Crash (D19-50)
Copied to clipboard
| Challenge: | Digital media enables fast sharing of information, but also disinformation . studies on the spread of disinformation on social media focused on small, manually annotated datasets or used proxys for data annotation. |
| Approach: | They propose to use text classifiers to label Twitter content related to the MH17 crash to improve annotation accuracy. |
| Outcome: | The proposed classifier improves over a hashtag-based baseline, but still remains a challenge in labelling pro-Russian and pro-Ukrainian content with high precision. |
Experiments with ad hoc ambiguous abbreviation expansion (D19-62)
Copied to clipboard
| Challenge: | ad hoc abbreviations are difficult to interpret for patients and nonspecialists. |
| Approach: | They propose to use morphologically annotated medical notes to expand ad hoc abbreviations without using additional domain resources. |
| Outcome: | The proposed methods outperform the previously proposed methods on Polish data but can be used for other languages. |
Learning Word Embeddings for Data Sparse and Sentiment Rich Data Sets (N18-4)
Copied to clipboard
| Challenge: | Existing word embeddings for sentiment analysis are limited in domain specific applications . generic word embeds are poor initialization for tasks on domain specific data sets. |
| Approach: | They propose to use word embeddings adapted for domain specific data sets in sentiment classification applications. |
| Outcome: | The proposed algorithms learn word embeddings on sparse and sentiment rich data sets. |
Unsupervised Question Answering for Fact-Checking (D19-66)
Copied to clipboard
| Challenge: | Recent Deep Learning (DL) models have achieved human-level accuracy on natural language tasks such as question-answering, natural language inference, and textual entailment. |
| Approach: | They propose an unsupervised question-answering based approach for a similar task, fact-checking. |
| Outcome: | The proposed approach achieves label accuracy of 80.2% on the development set and 80.25% on the test set. |
Using Classifier Features to Determine Language Transfer on Morphemes (N18-4)
Copied to clipboard
| Challenge: | Using native English data, we identify an English learner’s native language background based solely on the learner's English writing samples. |
| Approach: | They perform a Native Language Identification task where they identify an English learner’s native language background based only on the learner's English writing samples. |
| Outcome: | The proposed task is connected to a position in second language acquisition research that holds all learners acquire English grammatical morphemes in the same order, regardless of native language background. |
Unraveling the Search Space of Abusive Language in Wikipedia with Dynamic Lexicon Acquisition (D19-50)
Copied to clipboard
| Challenge: | Existing methods to detect abusive language only train one classifier for the whole variety of offending . a new method can support a moderator with explicit unraveled explanations for why something was flagged as abusive . |
| Approach: | a new method is proposed to distinguish explicitly abusive cases from the more "shadowed" ones . the researchers extend a lexicon of abusive terms to include new obfuscations of abusive words . |
| Outcome: | a new method can distinguish explicitly abusive cases from the more "shadowed" ones . the method can support a moderator with explicit unraveled explanations for why something was flagged as abusive . |
A Grounded Preference Model for LLM Alignment (2024.findings-acl)
Copied to clipboard
Tahira Naseem, Guangxuan Xu, Sarathkrishna Swaminathan, Asaf Yehudai, Subhajit Chaudhury, Radu Florian, Ramón Astudillo, Asim Munawar
| Challenge: | Large Language Models (LLMs) suffer from factual inconsistency and hallucination despite recent advances . training a preference model requires substantial human annotation, which is expensive and labor-intensive. |
| Approach: | They propose to generate synthetic grounded preference data and train a Grounded Preference Model to assess the overall quality of grounded responses. |
| Outcome: | The proposed model can generate much better grounded responses as judged by GPT4 and achieves the TRUE faithfulness Benchmark. |
A Challenge Dataset and Effective Models for Conversational Stance Detection (2024.lrec-main)
Copied to clipboard
| Challenge: | stance detection studies focus on evaluating stances within individual instances, hindering progress of conversational stance analysis. |
| Approach: | They propose a multi-turn conversation stance detection dataset that encompasses multiple targets for conversational stance detector. |
| Outcome: | The proposed dataset encompasses multiple targets for conversational stance detection. |
Predicting Algorithm Classes for Programming Word Problems (D19-55)
Copied to clipboard
| Challenge: | Using a text classification problem, we map programming word problems to relevant classes of algorithms. |
| Approach: | They propose to map programming word problems to relevant classes of algorithms by using a text classification problem as a classification task. |
| Outcome: | The proposed algorithm class prediction is 9 percent lower than a human on the task. |
Automatic identification of writers’ intentions: Comparing different methods for predicting relationship goals in online dating profile texts (D19-55)
Copied to clipboard
| Challenge: | lexicon-based text analysis methods such as LIWC have been criticized by computational linguists for their lack of adaptability, but they have not been systematically compared with either human evaluations or machine learning approaches. |
| Approach: | They used a corpus of online dating profile texts to compare LIWC, machine learning, and a human baseline to assess their effectiveness on a relationship goal classification task. |
| Outcome: | The proposed methods were compared with a corpus of online dating profile texts and a human baseline. |
Through the Lens of Split Vote: Exploring Disagreement, Difficulty and Calibration in Legal Case Outcome Classification (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for NLP calibration ignore inherent human label variation (HLV) split votes are a problem in high-stakes domains such as legal and medical decisions . |
| Approach: | They present a case outcome classification dataset with judges' vote distributions and build a taxonomy of disagreement with SV-specific subcategories. |
| Outcome: | The proposed model is compared against a judge vote distribution and assesses the alignment of perceived difficulty between models and humans. |
Information-Theoretic Probing with Minimum Description Length (2020.emnlp-main)
Copied to clipboard
| Challenge: | Despite widespread adoption of probes, differences in their accuracy fail to adequately reflect differences in representations. |
| Approach: | They propose an alternative to the standard probes, information-theoretic probing with minimum description length (MDL). |
| Outcome: | The proposed method agrees in results and is more informative and stable than the standard probes. |
Dia-Lingle: A Gamified Interface for Dialectal Data Collection (2025.acl-demo)
Copied to clipboard
| Challenge: | Dialects suffer from the scarcity of textual resources and are largely spoken rather than written. |
| Approach: | They propose a gamified interface that combines active learning with gamification to enhance the dialect corpus. |
| Outcome: | The proposed interface demonstrates high levels of user satisfaction while requiring minimal effort. |
Label Sleuth: From Unlabeled Text to a Classifier in a Few Hours (2022.emnlp-demos)
Copied to clipboard
Eyal Shnarch, Alon Halfon, Ariel Gera, Marina Danilevsky, Yannis Katsis, Leshem Choshen, Martin Santillan Cooper, Dina Epelboim, Zheng Zhang, Dakuo Wang
| Challenge: | Label Sleuth is an open source system for labeling and creating text classifiers which does not require coding skills nor machine learning knowledge. |
| Approach: | *Label Sleuth* is an open source system for labeling and creating text classifiers which does not require coding skills nor machine learning knowledge. |
| Outcome: | *Label Sleuth* is an open source system for labeling and creating text classifiers. |
A Weakly Supervised Classifier and Dataset of White Supremacist Language (2023.acl-short)
Copied to clipboard
| Challenge: | Existing studies on white supremacist language have focused on specific hateful ideologies, but little attention has been given to specific hate speech. |
| Approach: | They propose a weakly supervised classifier for detecting white supremacist language . they use large datasets of white supremacy domains paired with neutral and anti-racist data from similar domains to train the classifiers. |
| Outcome: | The proposed classifiers outperform previous studies on white supremacist classification on unseen datasets and find strong generalization performance for models with weakly annotated data. |
Neural Text Sanitization with Explicit Measures of Privacy Risk (2022.aacl-main)
Copied to clipboard
| Challenge: | Personal data, also known as Personally Identifiable Information (PII), often abound in text documents. |
| Approach: | They propose a method for text sanitization that uses a neural entity recognizer to detect and classify potential personal identifiers and a classifier trained on labelled data to determine which entities to mask . |
| Outcome: | The proposed approach masks all personal identifiers and thereby conceals the identity of the individuals mentioned in the document. |
SensorLLM: Aligning Large Language Models with Motion Sensors for Human Activity Recognition (2025.emnlp-main)
Copied to clipboard
| Challenge: | SensorLLM is a timeseries classification framework that can perform human activity recognition tasks. |
| Approach: | They propose a framework that enables Large Language Models to perform human activity recognition from sensor time-series data. |
| Outcome: | The proposed framework can perform human activity recognition (HAR) tasks with human inputs. |
PEMV: Improving Spatial Distribution for Emotion Recognition in Conversations Using Proximal Emotion Mean Vectors (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing research focuses on the analysis of contextual structure in dialogue and the interactions between different emotions. |
| Approach: | They propose a method that generates Proximal Emotion Mean Vectors (PEMVs) based on emotion feature queues to optimize the spatial representation of text features. |
| Outcome: | The proposed method achieves state-of-the-art performance on three widely used benchmark datasets. |
A Cross-Topic Method for Supervised Relevance Classification (D19-55)
Copied to clipboard
| Challenge: | Existing approaches to relevance classification are limited by annotated data and lack of relevance for each topic. |
| Approach: | They propose a cross-topic relevance embedding aggregation methodology that can expand the range of training data and apply what has been learned from source topics to a target topic. |
| Outcome: | The proposed method can capture common features within small amount of annotated data and improve performance compared with baselines. |
A Framework to Generate High-Quality Datapoints for Multiple Novel Intent Detection (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to detect novel intents have been tested in the last decade. |
| Approach: | They propose a framework to detect multiple novel intents with budgeted human annotation cost. |
| Outcome: | The proposed framework outperforms baseline methods in terms of accuracy and F1-score on a set of benchmark datasets. |
Triple-Hybrid Energy-based Model Makes Better Calibrated Natural Language Understanding Models (2023.eacl-main)
Copied to clipboard
| Challenge: | In-distribution (ID) miscalibration and out-of-difference (OOD) detection are main concerns for pre-trained language models. |
| Approach: | They propose a triple-hybrid EBM which combines the benefits of classifier, conditional generative model and marginal generative models altogether. |
| Outcome: | The proposed model outperforms previous methods in terms of ID calibration and OOD detection by a large margin while maintaining competitive accuracy. |
A dataset for identifying actionable feedback in collaborative software development (P18-2)
Copied to clipboard
Benjamin S. Meyers, Nuthan Munaiah, Emily Prud’hommeaux, Andrew Meneely, Josephine Wolff, Cecilia Ovesdotter Alm, Pradeep Murukannaiah
| Challenge: | a dataset of code reviews for the Google Chromium project analyzed linguistic features of code review feedback that elicited responsive actions from coworkers. |
| Approach: | They analyze code reviews for Google Chromium and extract linguistic features that elicit responsive responses from coworkers. |
| Outcome: | The proposed dataset shows that using NLP can be useful in code reviews . it also shows that it can be used to improve code reviews in a collaborative environment . |
Non-Parametric Domain Adaptation for End-to-End Speech Translation (2022.emnlp-main)
Copied to clipboard
| Challenge: | End-to-end speech translation (E2E-ST) systems have received increasing attention due to its less error propagation, lower latency and fewer parameters. |
| Approach: | They propose a non-parametric method that leverages in-domain text translation corpus to achieve domain adaptation for E2E-ST systems. |
| Outcome: | The proposed method outperforms the existing in-domain fine-tuning strategies on the Europarl-ST benchmark. |
LanguageNet: Learning to Find Sense Relevant Example Sentences (C18-2)
Copied to clipboard
| Challenge: | LanguageNet is a system that can help second language learners to search for different meanings and usages of a word . the polysemy of words, namely words with more than one sense, is one of the major challenges for ESOL learners . |
| Approach: | They propose a system which can help second language learners to search for different meanings of a word. |
| Outcome: | The proposed system can help second language learners to search for different meanings and usages of a word. |
Assessing In-context Learning and Fine-tuning for Topic Classification of German Web Data (2024.acl-srw)
Copied to clipboard
| Challenge: | Using a few hundred annotated data points per topic, we detect content related to three German policies in a database of scraped webpages. |
| Approach: | They propose to use annotated data to train a binary classification task to detect topic-related content in a scraped database of webpages. |
| Outcome: | The proposed model detects content related to three German policies in a scraped database of scrapes of webpages using a few hundred annotated data points per topic. |
Exploring Interpretability in Event Extraction: Multitask Learning of a Neural Event Classifier and an Explanation Decoder (2020.acl-srw)
Copied to clipboard
| Challenge: | EE is a key requirement for machine learning in many domains, e.g., legal, medical, finance. |
| Approach: | They propose an interpretable approach for event extraction that jointly trains a classifier and a rule decoder for event processing. |
| Outcome: | The proposed approach can be used for semi-supervised learning and its performance improves when trained on automatically-labeled data generated by a rule-based system. |
Is It Navajo? Accurate Language Detection for Endangered Athabaskan Languages (2025.naacl-short)
Copied to clipboard
| Challenge: | Endangered languages are significantly underrepresented in contemporary language technologies, exacerbating the challenges of their preservation and revitalization. |
| Approach: | They propose a random forest classifier trained on Navajo and 20 erroneously suggested languages by Google's Language Identification tool. |
| Outcome: | The proposed classifier achieves near-perfect accuracy across other Athabaskan languages suggesting its potential for broader application. |
Zero-shot Learning of Classifiers from Natural Language Quantification (P18-1)
Copied to clipboard
| Challenge: | Existing methods to learn concepts from natural language are limited or no labeled examples. |
| Approach: | They propose a framework through which a set of explanations of a concept can be used to learn a classifier without access to any labeled examples. |
| Outcome: | The proposed framework outperforms previous approaches for learning with limited data and is comparable with fully supervised classifiers trained from a small number of labeled examples. |
AraDIC: Arabic Document Classification Using Image-Based Character Embeddings and Class-Balanced Loss (2020.acl-srw)
Copied to clipboard
| Challenge: | a novel image-based character embedding framework is used for text classification in Arabic . classical methods require morphological analysis, word segmentation, and hand-crafted feature engineering. |
| Approach: | They propose a novel end-to-end Arabic document classification framework, Arabic document image-based classifier, inspired by image-basic character embeddings. |
| Outcome: | The proposed framework improves on modern standard Arabic, colloquial Arabic, and Classical Arabic. |
SNaC: Coherence Error Detection for Narrative Summarization (2022.emnlp-main)
Copied to clipboard
| Challenge: | SNaC framework is used to evaluate long summaries, but it fails to identify gaps in coherence . nallapati and colleagues have developed a framework for fine-grained annotations of long summarizations . |
| Approach: | They propose a narrative coherence evaluation framework for fine-grained annotations of long summaries that can be used to evaluate coherent narratives. |
| Outcome: | The proposed framework can support future work in document summarization and coherence evaluation, the authors show . |
Contextualized Weak Supervision for Text Classification (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for weakly supervised text classification generate pseudo-labels in a context-free manner, thus, the ambiguous, context-dependent nature of human language has been long overlooked. |
| Approach: | They propose a framework that provides contextualized weak supervision for text classification . they leverage contextualized representations of word occurrences and seed word information . |
| Outcome: | The proposed framework provides contextualized weak supervision for text classification . it leverages representations of word occurrences and seed word information to differentiate interpretations . the proposed framework also disambiguates initial seed words, making it fully contextualized . |
Less is More: Mitigate Spurious Correlations for Open-Domain Dialogue Response Generation Models by Causal Discovery (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing models suffer from spurious correlations and generate irrelevant and generic responses. |
| Approach: | They propose a model-agnostic method for training and inference using a conditional independence classifier that overcomes data sparsity. |
| Outcome: | The proposed method outperforms the baseline models in relevance, informativeness, and fluency. |
Simple Large-scale Relation Extraction from Unstructured Text (L18-1)
Copied to clipboard
| Challenge: | Knowledge-based question answering relies on the availability of facts, most of which cannot be found in structured sources. |
| Approach: | They propose a method for creating distant (weak) supervision labels for training a large-scale RE system by decoupling the model architecture from the feature design of a state-of-the-art neural network system. |
| Outcome: | The proposed method performs on par with the state-of-the-art model with similar features at 75x reduction in training time. |
Debiasing Made State-of-the-art: Revisiting the Simple Seed-based Weak Supervision for Text Classification (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in weakly supervised text classification focus on designing sophisticated methods to turn high-level human heuristics into quality pseudo-labels. |
| Approach: | They propose to use a seed matching-based method to generate quality pseudo-labels by deleting the seed words present in the matched input text. |
| Outcome: | The proposed method can be improved significantly by deleting the seed words in the matched input text with a high deletion ratio. |
WikiRef: Wikilinks as a route to recommending appropriate references for scientific Wikipedia pages (C18-1)
Copied to clipboard
| Challenge: | Existing methods to enhance Wikipedia's reference section are not effective. |
| Approach: | They propose a two-step approach that leverages the wikilinks present in a scientific Wikipedia target page and recommends highly relevant references to be included in that target page appropriately and automatically borrowed from the reference section of the wikipedia links. |
| Outcome: | The proposed approach achieves a notably good performance on two datasets from Computer Science and Physics. |
Extreme Fine-tuning: A Novel and Fast Fine-tuning Approach for Text Classification (2024.eacl-short)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning pre-trained models require massive computational resources and time. |
| Approach: | They propose a novel approach for fine-tuning a pre-trained model using backpropagation and an iterative extreme learning machine for training a classifier. |
| Outcome: | The proposed approach outperforms state-of-the-art approaches in training-time measurement and performance with comparable model performance. |
Finding a Needle in the Adversarial Haystack: A Targeted Paraphrasing Approach For Uncovering Edge Cases with Minimal Distribution Distortion (2024.eacl-long)
Copied to clipboard
| Challenge: | Adversarial attacks against Language models (LMs) are a significant concern. |
| Approach: | They propose an approach to automatically learn a policy to generate challenging examples that improve the model’s performance. |
| Outcome: | The proposed approach outperforms baselines and exhibits generalizability across classifiers and datasets. |
AMRFact: Enhancing Summarization Factuality Evaluation with AMR-Driven Negative Samples Generation (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for evaluating factual consistency of abstractive summarization lack coherence or error-type coverage. |
| Approach: | They propose a framework that generates perturbed summaries using Abstract Meaning Representations (AMRs) they use a selection module NegFilter to ensure the quality of the generated negative examples . |
| Outcome: | The proposed framework outperforms existing systems on the AggreFact-SOTA benchmark and provides high error-type coverage. |
Vector of Locally-Aggregated Word Embeddings (VLAWE): A Novel Document-level Representation (N19-1)
Copied to clipboard
| Challenge: | a novel word embedding representation for text documents is proposed . the method is based on the Vector of Locally-Aggregated Descriptors used for image representation . |
| Approach: | They propose a novel representation for text documents based on aggregating word embedding vectors into document embeddables. |
| Outcome: | The proposed representation improves on the Movie Review data set and is 10% better than the state-of-the-art representation. |
WHOSe Heritage: Classification of UNESCO World Heritage Statements of "Outstanding Universal Value” with Soft Labels (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Evaluating and justifying Outstanding Universal Value (OUV) is essential for each site inscribed in the WHL . manual annotation of heritage values and attributes from multi-source textual data is knowledge-demanding and time-consuming. |
| Approach: | They propose to use NLP to build a classifier on a dataset containing Statements of OUV. |
| Outcome: | The proposed model can reach 94.3% accuracy on a dataset containing Statements of OUV . the study is promising to be further developed and applied in heritage research and practice. |
Prompt-Tuned Muti-Task Taxonomic Transformer (PTMTTaxoFormer) (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Existing methods for Hierarchical Text Classification (HTC) are expensive and require explicit injection of the hierarchy, verbalizers, and/or prompt engineering. |
| Approach: | They propose a hierarchical text classification system that uses a single classifier to predict one or more topics using differentiable prompts and labels that are learnt through backpropagation. |
| Outcome: | The proposed model outperforms existing models on several benchmarks that span a range of topics consistently. |
Data Selection for Fine-tuning Large Language Models Using Transferred Shapley Values (2023.acl-srw)
Copied to clipboard
| Challenge: | Large language models (LMs) have been shown to be highly effective for identifying harmful training instances, but dataset size and model complexity constraints limit the ability to apply Shapley-based data valuation to fine-tuning large pre-trained language models. |
| Approach: | They propose an algorithm that aggregates Shapley values from subsets for valuation of entire training set and a value transfer method that leverages value information extracted from a simple classifier trained using representations from the target language model. |
| Outcome: | The proposed method outperforms existing methods on benchmark datasets and can filter fine-tuning data to increase language model performance compared to training with the full fine-uning dataset. |
Where do LLMs Encode the Knowledge to Assess the Ambiguity? (2025.coling-industry)
Copied to clipboard
| Challenge: | False sizing of large language models can generate unreliable responses . |
| Approach: | They propose a method to train large language models without ambiguity labels . |
| Outcome: | The proposed method detects ambiguous input prompts better than representations from the final layer. |
AdapterFusion: Non-Destructive Task Composition for Transfer Learning (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for incorporating knowledge from multiple tasks suffer from catastrophic forgetting and difficulties in dataset balancing. |
| Approach: | They propose an algorithm that extracts and combine adapters in a knowledge composition step. |
| Outcome: | The proposed class outperforms traditional methods such as full fine-tuning and multi-task learning on 16 diverse NLU tasks. |
Balanced Adversarial Training: Balancing Tradeoffs between Fickleness and Obstinacy in NLP Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Traditional adversarial examples involve finding a small perturbation that does not change an input’s true label but confuses the classifier into outputting a different prediction. |
| Approach: | They propose to use contrastive learning to increase model robustness against fickle adversarial examples by reducing the vulnerability of adversarials to fickle ones. |
| Outcome: | The proposed method improves model robustness against fickle and obstinate adversarial examples. |
Thesis Proposal: An Explainable Multimodal Framework for Detecting Harmful Content in Code-Switched Children’s Media (2026.acl-srw)
Copied to clipboard
| Challenge: | Current content moderation systems fail to protect children from harmful content, especially in under-resourced, code-switched settings. |
| Approach: | They propose to integrate a fine-tuned classifier with an LLM-powered module that synthesizes the classifier’s internal evidential signals to generate faithful, human-readable rationales for each decision. |
| Outcome: | The proposed framework integrates a fine-tuned classifier for accurate, scalable detection with an LLM-powered module that synthesizes the classifier’s internal evidential signals to generate faithful, human-readable rationales for each decision. |
VertAttack: Taking Advantage of Text Classifiers’ Horizontal Vision (2024.naacl-long)
Copied to clipboard
| Challenge: | Current SOTA classifiers are subject to problems like bias and are vulnerable to adversarial attacks. |
| Approach: | They propose an attack to mimic a classifier's character based attack and thenrewrite those words vertically. |
| Outcome: | The proposed attack can drop the accuracy of 4 different transformer models on 5 datasets and preserve meaning. |
R&R: Metric-guided Adversarial Sentence Generation (2022.findings-aacl)
Copied to clipboard
| Challenge: | Existing methods prioritize misclassification by maximizing each perturbation’s effectiveness at misleading a text classifier. |
| Approach: | They propose a rewrite and rollback framework for adversarial attack that optimizes a critique score which combines fluency, similarity, and misclassification metrics. |
| Outcome: | The proposed framework outperforms current state-of-the-art in attack success rate by +16.2%, +12.8%, and +14.0% on the classifiers respectively. |
Lost in Space: Probing Fine-grained Spatial Understanding in Vision and Language Resamplers (2024.naacl-short)
Copied to clipboard
| Challenge: | Recent approaches for developing vision and language models leverage existing vision and a language expert and try to learn a mapping between them. |
| Approach: | They propose to use a resampler module to create a ‘visual prompt’ which is then fed to the large language models (LLM) using a textual prompt. |
| Outcome: | The proposed method has been shown to be effective across coarse-grained tasks like image captioning and visual question answering, but more fine-grounded tasks that require spatial understanding have not been thoroughly examined. |
Coarse2Fine: Fine-grained Text Classification on Coarsely-grained Annotated Data (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing text classification methods focus on a fixed label set, but many real-world applications require extending to new fine-grained classes as the number of samples per label increases. |
| Approach: | They propose a problem called coarse-to-fine grained classification that leverages label surface names as the only human guidance. |
| Outcome: | The proposed method outperforms existing methods on two real-world datasets. |
Structured Discourse Representation for Factual Consistency Verification (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to verify factual consistency of text capture a performance gap compared with sentence-level entailment. |
| Approach: | They propose a method that combines structured discourse information extraction with a classifier, FDSpotter, for factual consistency verification. |
| Outcome: | The proposed method achieves competitive performance on two tasks: data-to-text generation and text summarisation. |
Enhancing Local Feature Extraction with Global Representation for Neural Text Classification (D19-1)
Copied to clipboard
| Challenge: | Existing methods for text classification learn long dependency by deeply stacking or hybrid modeling. |
| Approach: | They propose a global-based local feature extraction architecture with global information incorporated into the local feature extractor. |
| Outcome: | The proposed architecture outperforms the previous best models on eight benchmark datasets. |
Generate, Annotate, and Learn: NLP with Synthetic Text (2022.tacl-1)
Copied to clipboard
| Challenge: | Existing methods to generate unlabeled text are difficult to find. |
| Approach: | They propose a general framework called "generate, annotate, and learn" to take advantage of synthetic text within knowledge distillation, self-training, and few-shot learning applications. |
| Outcome: | The proposed framework achieves state-of-the-art knowledge distillation results for 6-layer transformers on the GLUE leaderboard. |
An Edit-centric Approach for Wikipedia Article Quality Assessment (D19-55)
Copied to clipboard
| Challenge: | Existing methods to assess article quality in Wikipedia are based on a document-level approach, but considering the whole document could have negative repercussions. |
| Approach: | They propose an edit-centric approach to assess Wikipedia article quality using an edit classifier and a generative module that provides an estimation and translates it into natural language. |
| Outcome: | The proposed model is cost-effective and can be implemented in Wikipedia. |
Improving the TENOR of Labeling: Re-evaluating Topic Models for Content Analysis (2024.eacl-long)
Copied to clipboard
Zongxia Li, Andrew Mao, Daniel Stephens, Pranav Goel, Emily Walpole, Alden Dima, Juan Fung, Jordan Boyd-Graber
| Challenge: | Existing evaluation metrics such as coherence and coherency are inadequate for neural topic models. |
| Approach: | They conduct the first evaluation of neural, supervised and classical topic models in an interactive task-based setting. |
| Outcome: | The proposed model performs better on cluster evaluation metrics and human evaluations than classical models on real-world tasks. |
Neural Event Semantics for Grounded Language Understanding (2021.tacl-1)
Copied to clipboard
| Challenge: | a new framework for compositional grounded language understanding is proposed . NES is trainable end-to-end by gradient descent with minimal supervision. |
| Approach: | They propose a conjunctivist framework for compositional grounded language understanding . they use words as classifiers that compose to form a sentence meaning by multiplying output scores . |
| Outcome: | The proposed framework improves on compositional grounded language tasks. |
Discrete Cosine Transform as Universal Sentence Encoder (2021.acl-short)
Copied to clipboard
| Challenge: | Modern sentence encoders capture underlying linguistic characteristics of words . Discrete Cosine Transform (DCT) is an efficient alternative to averaging . |
| Approach: | They propose to use a Discrete Cosine Transform to generate universal sentence representations in different languages. |
| Outcome: | The proposed model captures the underlying syntactic characteristics of a given text without compromising practical efficiency. |
On Large Language Models’ Hallucination with Regard to Known Facts (2024.naacl-long)
Copied to clipboard
Che Jiang, Biqing Qi, Xiangyu Hong, Dayuan Fu, Yang Cheng, Fandong Meng, Mo Yu, Bowen Zhou, Jie Zhou
| Challenge: | Large language models are successful in answering factoid questions but are also prone to hallucination. |
| Approach: | They propose self-reporting to the model when faced with such limitations. |
| Outcome: | The proposed classifier can detect hallucinations with an 88% success rate and can be used to answer factoid questions with correct answer knowledge. |
“Nice Try, Kiddo”: Investigating Ad Hominems in Dialogue Responses (2021.naacl-main)
Copied to clipboard
| Challenge: | Ad hominem attacks target a person's character instead of the position the person is maintaining. |
| Approach: | They propose to use salient n-gram similarity as a soft constraint to reduce the amount of ad hominems generated in Twitter conversations. |
| Outcome: | The proposed method reduces the amount of ad hominems generated in human and dialogue system responses to English Twitter posts by using salient n-gram similarity as a soft constraint. |
Cross-Lingual Transfer from Related Languages: Treating Low-Resource Maltese as Multilingual Code-Switching (2024.eacl-long)
Copied to clipboard
| Challenge: | Multilingual models exhibit impressive cross-lingual transfer capabilities on unseen languages, but performance is impacted when there is a script disparity with the languages used in the model’s pre-training data. |
| Approach: | They propose a novel method to align a resource-rich language's script with a target language and train a classifier that can make informed decisions regarding the appropriate processing of each token. |
| Outcome: | The proposed model can be used to transfer a language's scripts across multiple languages, but it is suboptimal for mixed languages, where only a subset benefits while the rest is impeded. |
Fine-Grained Temporal Orientation and its Relationship with Psycho-Demographic Correlates (N18-1)
Copied to clipboard
| Challenge: | Temporal orientation refers to an individual’s tendency to connect to the psychological concepts of past, present or future and affects personality, motivation, emotion, decision making and stress coping processes. |
| Approach: | They propose to use a minimally supervised method to classify tweets in one of three temporal categories, past, present, and future, and a deep bi-directional long-term memory (BLSTM) to measure correlation between sentiment view of temporal orientation and different psycho-demographic factors. |
| Outcome: | The proposed method achieves 78.27% accuracy on a manually created test set. |
On the Impact of Temporal Representations on Metaphor Detection (2022.lrec-1)
Copied to clipboard
| Challenge: | State-of-the-art approaches for metaphor detection compare their literal - or core - meaning and their contextual meaning using neural networks. |
| Approach: | They propose to use temporal and static word embeddings to account for different representations of literal meanings to examine metaphor detection tasks. |
| Outcome: | The proposed method outperforms static methods but may provide representations of the core meaning of the metaphor too close to their contextual meaning, causing confusion. |
Estimating User Communication Styles for Spoken Dialogue Systems (2020.lrec-1)
Copied to clipboard
| Challenge: | a neural network estimation system for spoken dialogues can be used to estimate the communication style of a user's interaction, but this is rarely implemented in a live system. |
| Approach: | They propose a neural network approach to estimate the communication style of spoken interaction, namely elaborateness and directness. |
| Outcome: | The proposed method can estimate the elaborateness and directness of spoken interaction and improve the results with additional linguistic features. |
Understanding and Mitigating Spurious Correlations in Text Classification with Neighborhood Analysis (2024.findings-eacl)
Copied to clipboard
| Challenge: | Recent research has revealed that machine learning models have a tendency to leverage spurious correlations that exist in the training set but may not hold true in general circumstances. |
| Approach: | They propose a metric to detect spurious tokens and a family of regularization methods to mitigate spurious correlations in text classification. |
| Outcome: | The proposed method prevents spurious clusters and significantly improves the robustness of classifiers without auxiliary data. |
The Change that Matters in Discourse Parsing: Estimating the Impact of Domain Shift on Parser Error (2022.findings-acl)
Copied to clipboard
| Challenge: | Discourse analysis is very low on texts outside of the training distribution’s coverage, diminishing the practical utility of existing models. |
| Approach: | They propose to use a distribution shift statistic to estimate the error-gap of a discourse model and to use it to estimate it. |
| Outcome: | The proposed model can be estimated via distribution shift but does not correlate with change in the observed error of a classifier (i.e. error-gap). |
The Internal State of an LLM Knows When It’s Lying (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown exceptional performance in various tasks, but one of their main drawbacks is generating inaccurate or false information with a confident tone. |
| Approach: | They propose to train a classifier that outputs the probability that a statement is truthful based on the hidden layer activations of the LLM as it reads or generates the statement. |
| Outcome: | The proposed classifier achieves an average of 71% to 83% accuracy labeling which sentences are true versus false, depending on the LLM base model. |
Balanced Accuracy: The Right Metric for Evaluating LLM Judges - Explained through Youden’s J statistic (2026.eacl-industry)
Copied to clipboard
| Challenge: | False refusals and task pass rates are key to reliable evaluation of large language models. |
| Approach: | They propose a principled best practice for evaluating judges based on a golden set of judge-quality metrics. |
| Outcome: | The proposed method improves the quality of judge-quality metrics on a golden set. |
A Classification-Guided Approach for Adversarial Attacks against Neural Machine Translation (2024.eacl-long)
Copied to clipboard
| Challenge: | Extensive research has been devoted to adversarial attacks against NMT models . perturbations of inputs can mislead the target model, resulting in incorrect outputs . |
| Approach: | They propose an adversarial attack framework that alters the class of output translations of an NMT model and a classifier to craft adversarials whose translations belong to a different class . |
| Outcome: | The proposed approach has a more substantial effect on the translation by altering the overall meaning, which leads to a different class determined by an oracle classifier. |
Logic Matters in Lightweight Hallucination Classification for RAG System (2026.acl-long)
Copied to clipboard
| Challenge: | Existing hallucination detection frameworks for RAGs lack robustness and performance . a compact model may lose track of precise information in retrieved segments or misinterpret a document's entailment score. |
| Approach: | They propose a lightweight, modular framework for hallucination detection in RAG systems . they capture logical relationships among retrieved documents within the vector space . |
| Outcome: | The proposed framework improves hallucination detection in RAG systems without complex architectures or pre-training on datasets. |
Why Generate When You Can Discriminate? A Novel Technique for Text Classification using Language Models (2024.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for text classification using autoregressive language models are limited . authors propose a novel technique for text classification using autoreregressives . |
| Approach: | They propose a two-step technique for text classification using autoregressive language models . they use a set of perplexity and log-likelihood based numeric features to elicit a text instance . |
| Outcome: | The proposed technique eliminates parameter updates in LMs and does not limit training examples . it is evaluated across 5 datasets and compares with multiple competent baselines . |
Generation, Distillation and Evaluation of Motivational Interviewing-Style Reflections with a Foundational Language Model (2024.eacl-long)
Copied to clipboard
| Challenge: | Motivational Interviewing (MI) is a counselling technique used to guide people towards behaviour change. |
| Approach: | They propose a method for distilling reflections from a foundational language model into smaller models that can be owned and controlled. |
| Outcome: | The proposed method achieves 100% success rate on hold-out test set and 90% on the GPT-2 XL. |
On Assigning Product and Software Codes to Customer Service Requests with Large Language Models (2025.emnlp-industry)
Copied to clipboard
| Challenge: | In a technology company, quality of customer service is a crucial asset. |
| Approach: | They propose to use Large Language Models to assign product names and software version labels to customer Service Requests (SRs) they frame assignment as multiple-choice question answering task instead of conventional prompts . |
| Outcome: | The proposed model can identify product names and software versions when they are mentioned with over 90% accuracy while cutting LLM costs by 40-60% on average. |
Improving Unsupervised Out-of-domain detection through Pseudo Labeling and Learning (2023.findings-eacl)
Copied to clipboard
| Challenge: | Unsupervised OOD detection is a task aimed at discriminating whether given samples are from the in-domain (IND) . previous studies adopted the one-class classification approach, assuming that the training samples come from a single domain. |
| Approach: | They propose a framework that leverages latent categorical information to improve representation learning for textual OOD detection. |
| Outcome: | The proposed framework significantly outperforms baseline models on three datasets. |
Cross-lingual Text Classification with Heterogeneous Graph Neural Network (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods for cross-lingual text classification only consider factors beyond semantic similarity, causing performance degradation between some language pairs. |
| Approach: | They propose a method to incorporate heterogeneous information within and across languages for cross-lingual text classification using graph convolutional networks. |
| Outcome: | The proposed method significantly outperforms state-of-the-art models on all tasks and achieves consistent performance gain over baselines in low-resource settings. |
The Art of Abstention: Selective Prediction and Error Regularization for Natural Language Processing (2021.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models have improved the state-of-the-art results on many NLP applications. |
| Approach: | They propose a simple error regularization trick that improves confidence estimation without substantially increasing the computation budget. |
| Outcome: | The proposed regularization improves confidence estimation without increasing computation budget. |
Classifier-Augmented Generation for Structured Workflow Prediction (2025.emnlp-industry)
Copied to clipboard
| Challenge: | a new system translates natural language descriptions into executable workflows . configuring stages and their properties is time consuming and requires deep tool knowledge. |
| Approach: | They propose a system that translates natural language descriptions into executable workflows . it uses a Classifier-Augmented Generation approach that combines utterance decomposition with a classifier and stage-specific prompting to produce accurate stage predictions. |
| Outcome: | The proposed system outperforms existing models and reduces token usage by 60%. |
Bicleaner AI: Bicleaner Goes Neural (2022.lrec-1)
Copied to clipboard
| Challenge: | a new version of Bicleaner detects noisy sentences in parallel corpora . the tool is based on pre-trained transformer-based language models fine-tuned on a binary classification task. |
| Approach: | They propose to use Bicleaner AI to detect noisy sentences in parallel corpora . they use pre-trained transformer-based language models fine-tuned on a binary classification task . |
| Outcome: | The proposed tool improves translation quality and reduces manual cleaning steps. |
Data Augmentation for Hypernymy Detection (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for supervised inference have limited quality training data. |
| Approach: | They propose two techniques which generate new training examples from existing ones . they combine linguistic principles of hypernym transitivity and intersective modifier-noun composition . |
| Outcome: | The proposed techniques generate new training examples from existing datasets. |
Reinforcement Learning for Adversarial Query Generation to Enhance Relevance in Cold-Start Product Search (2025.acl-industry)
Copied to clipboard
| Challenge: | Existing methods do not incorporate feedback from the query relevance model, limiting their ability to generate queries that enhance product retrieval. |
| Approach: | They propose an adversarial reinforcement learning framework that exposes weaknesses in query classification models by creating synthetic queries that augment the classifier's training set. |
| Outcome: | The proposed framework improves query generation performance on public datasets and on proprietary datasets. |
Identifying Well-formed Natural Language Questions (D18-1)
Copied to clipboard
| Challenge: | Understanding search queries is a hard problem as it involves dealing with “word salad” text. |
| Approach: | They propose to identify a well-formed natural language question from a dataset of 25,100 questions. |
| Outcome: | The proposed classifier improves the performance of neural sequence-to-sequence models for generating questions for reading comprehension. |
Ghostbuster: Detecting Text Ghostwritten by Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Ghostbuster is a system that passes documents through weaker language models, runs a structured search over possible combinations of their features, and trains a classifier on the selected features. |
| Approach: | They propose a method that passes documents through weaker language models, runs a structured search over possible combinations of their features, and trains a classifier on the selected features. |
| Outcome: | The proposed method outperforms existing detectors and a new baseline on student essays, creative writing, and news articles. |
Labeled Anchors and a Scalable, Transparent, and Interactive Classifier (D18-1)
Copied to clipboard
| Challenge: | Labeled Anchors is an interactive and supervised topic model based on the anchor words algorithm . |
| Approach: | They propose an interactive supervised topic model based on the anchor words algorithm . they propose a classifier which requires no training beyond topic inference . |
| Outcome: | The proposed model is human-interpretable and fast, and can be interactive. |
Complicate Then Simplify: A Novel Way to Explore Pre-trained Models for Text Classification (2022.coling-1)
Copied to clipboard
| Challenge: | Existing frameworks for text classification employing pre-trained models are constrained by the difficulty of the task. |
| Approach: | They propose a framework which implements a two-stage training strategy to fully exploit the knowledge in pre-trained models. |
| Outcome: | The proposed framework outperforms state-of-the-art classification models on six text classification corpora. |
VEIL: Vetting Extracted Image Labels from In-the-Wild Captions for Weakly-Supervised Object Detection (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing methods to “vet” labels from noisy captions for weakly-supervised object detection are limited for object detection. |
| Approach: | They propose a technique to “vet” labels extracted from noisy captions and use them for weakly-supervised object detection without any bounding boxes. |
| Outcome: | The proposed method improves WSOD without label vetting by 30% on five datasets. |
Robust Text Classifier on Test-Time Budgets (D19-1)
Copied to clipboard
| Challenge: | Recent advances in deep neural networks (DNNs) achieve high accuracy on many text classification tasks. |
| Approach: | They propose a generic framework for learning a robust text classification model . they use a data aggregation method to train the classifier on a large corpus of text . |
| Outcome: | The proposed framework achieves consistent speedup with little degradation in accuracy on four benchmark text classification tasks. |
Factorising Meaning and Form for Intent-Preserving Paraphrasing (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for paraphrasing questions retain original intent but use a different surface form. |
| Approach: | They propose a method for generating paraphrases of English questions that retain original intent but use a different surface form. |
| Outcome: | The proposed method generates paraphrases with better tradeoff between semantic preservation and novelty compared to previous methods. |
Handling Ambiguity in Emotion: From Out-of-Domain Detection to Distribution Estimation (2024.acl-long)
Copied to clipboard
| Challenge: | Experimental results show that incorporating utterances without majority-agreed labels into an additional class reduces the classification performance of the other emotion classes. |
| Approach: | They propose to combine utterances without majority-agreed labels into an additional class . they propose to quantify uncertainty in emotion classification using evidential deep learning . |
| Outcome: | The proposed method retains classification accuracy while effectively detects ambiguous emotion expressions. |
Context-specific Language Modeling for Human Trafficking Detection from Online Advertisements (P19-1)
Copied to clipboard
Saeideh Shahrokh Esfahani, Michael J. Cafarella, Maziyar Baran Pouyan, Gregory DeAngelo, Elena Eneva, Andy E. Fano
| Challenge: | Human trafficking is a worldwide crisis. |
| Approach: | They propose a method to detect trafficking ads on online sites using natural language processing using a pre-trained textual language model. |
| Outcome: | The proposed classifier significantly outperforms any single feature set alone. |
Cross-Target Stance Classification with Self-Attention Networks (P18-2)
Copied to clipboard
| Challenge: | In stance classification, the target on which the stance is made defines the boundary of the task, and a classifier is usually trained for prediction on the same target. |
| Approach: | They propose a neural model that can generalize classifiers between different targets by finding useful information shared between relevant targets. |
| Outcome: | The proposed model can generalize between relevant targets and find useful information shared between relevant target domains which improves generalization in certain scenarios. |
ASTE-Transformer: Modelling Dependencies in Aspect-Sentiment Triplet Extraction (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Aspect-Sentiment Triplet Extraction (ASTE) is a recent task in aspect-based sentiment analysis. |
| Approach: | They propose a task of aspect-based sentiment analysis that extracts triples from sentences . they propose three transformer-inspired layers to enable modelling of dependencies . |
| Outcome: | The proposed method achieves higher performance in terms of F1 measure than other methods studied on popular benchmarks. |
Balancing via Generation for Multi-Class Text Classification Improvement (2020.findings-emnlp)
Copied to clipboard
| Challenge: | balancing is a known technique for improving classification performance . balancy is based on a balancing policy and a text generation mechanism . |
| Approach: | They propose a balancing-via-generation framework that augments a dataset for more balanced distribution by using a text generation mechanism. |
| Outcome: | The proposed framework can augment a dataset for more balanced distribution while under-sampling others. |
PEEB: Part-based Image Classifiers with an Explainable and Editable Language Bottleneck (2024.findings-naacl)
Copied to clipboard
| Challenge: | CLIP-based classifiers rely on the prompt containing a class name that is known to the text encoder and perform poorly on new classes or the classes whose names rarely appear on the Internet. |
| Approach: | They propose to use a set of text descriptors to express a class name into a textual descriptable and match the embeddings of the detected parts to their textual ones to compute a logit score. |
| Outcome: | The proposed classifier outperforms CLIP-based classifiers on zero-shot and supervised learning settings by 88.80% and 92.20% accuracy on CUB-200 and Stanford Dogs-120. |
Identifying Speakers and Addressees in Dialogues Extracted from Literary Fiction (L18-1)
Copied to clipboard
Adam Ek, Mats Wirén, Robert Östling, Kristina N. Björkenstam, Gintarė Grigonytė, Sofia Gustafson Capková
| Challenge: | Using a sequence labeling approach, it is possible to identify speakers and addressees in dialogues extracted from literary fiction using a small amount of training data. |
| Approach: | They propose to use a sequence labeling approach applied to a given set of characters to identify speakers and addressees in dialogues extracted from literary fiction. |
| Outcome: | The proposed method allows for enriched search facilities and construction of social networks from the corpora. |
Unsupervised Information Extraction: Regularizing Discriminative Approaches with Relation Distribution Losses (P19-1)
Copied to clipboard
| Challenge: | Existing unsupervised relation extraction models are either generative or discriminative . however, they are hard to train without supervision and are unstable . |
| Approach: | They propose a skewness loss and distribution distance loss to improve the performance of discriminative based models. |
| Outcome: | The proposed models surpass current state-of-the-art on three different datasets. |
On the Creation of a Corpus for Coherence Evaluation of Discursive Units (2020.lrec-1)
Copied to clipboard
| Challenge: | Most corpora for textual coherence evaluation are composed of randomly shuffled sentences that focus on sentence ordering. |
| Approach: | They propose to use a variety of corruption strategies to build a corpus of incoherent pairs of sentences by swapping their discourse connective or a discourse argument. |
| Outcome: | The proposed corpus is constructed from discourse argument pairs from the Penn Discourse Tree Bank and is compared with existing corpus models. |
Improving the results of string kernels in sentiment analysis and Arabic dialect identification by adapting them to your test set (D18-1)
Copied to clipboard
| Challenge: | Recent studies have demonstrated remarkable performance in text classification tasks such as Arabic dialect identification. |
| Approach: | They propose two approaches to improve string kernels' accuracy in Arabic and English . first approach interprets pairwise string kernel similarities between training and test sets as features . second approach adapts to training set and adds test samples for another round of training . |
| Outcome: | The proposed methods improve English polarity classification and Arabic dialect identification. |
Minimally-Supervised Relation Induction from Pre-trained Language Model (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to induce relation in NLP depend heavily on word embeddings. |
| Approach: | They propose a method to induce relation with BERT under minimal supervision . they first extract proper templates from corpus and then use BERT attention weights to represent the pseudo-sentences. |
| Outcome: | The proposed method achieves state-of-the-art in relation induction tasks on Google Analogy Test Sets, Bigger Analogy test set (BATS) and DiffVec. |
Teamwork Is Not Always Good: An Empirical Study of Classifier Drift in Class-incremental Information Extraction (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for learning incrementally do not address the problem of class-incremental learning. |
| Approach: | They propose a framework that can continuously learn new classes from a data stream without forgetting previously learned classes. |
| Outcome: | The proposed framework shows significant improvement over the state-of-the-art frameworks with up to 44.7% absolute F-score gain. |
MProto: Multi-Prototype Network with Denoised Optimal Transport for Distantly Supervised Named Entity Recognition (2023.emnlp-main)
Copied to clipboard
| Challenge: | Distantly supervised named entity recognition (DS-NER) aims to locate entity mentions and classify their types with knowledge bases or gazetteers and unlabeled corpus. |
| Approach: | They propose a noise-robust prototype network named MProto for a DS-NER task . they propose an optimal transport algorithm to mitigate the noise from incomplete labeling . |
| Outcome: | The proposed network achieves state-of-the-art on several DS-NER benchmarks. |
Encoders Help You Disambiguate Word Senses in Neural Machine Translation (D19-1)
Copied to clipboard
| Challenge: | Neural machine translation models can perform word sense disambiguation (WSD) however, it is unclear which component dominates the process of disambiguating words. |
| Approach: | They evaluate hidden states and investigate distributions of self-attention in NMT encoders and decoders to disambiguate word senses. |
| Outcome: | The proposed model outperforms encoder hidden states on large datasets . the model outpersforms decoders on large data sets . |
DRAFT: Dense Retrieval Augmented Few-shot Topic classifier Framework (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for few-shot topic classification are limited due to the volume of information pouring in from the Internet . a new framework is proposed to train a classifier for few shot topics . |
| Approach: | They propose a framework to train a classifier for few-shot topic classification using a customized dataset and a dense retriever model. |
| Outcome: | The proposed framework shows superior performance on few-shot topic classification tasks compared to baselines that use in-context learning . |
D2U: Distance-to-Uniform Learning for Out-of-Scope Detection (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for out-of-scope (OOS) detection use classifier confidence score, but model cannot infer correctly. |
| Approach: | They propose a zero-shot post-processing step that exploits the classification confidence score and the shape of the entire output distribution. |
| Outcome: | The proposed method improves performance when there is no OOS training data and learning procedure when OOS data is available. |
Correct after Answer: Enhancing Multi-Span Question Answering with Post-Processing Method (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work focuses on designing specific methods or applying heuristic strategies to encourage models to predict more correct predictions. |
| Approach: | They propose a framework that uses a post-processing strategy to handle incorrect predictions. |
| Outcome: | The proposed framework significantly improves the Exact Match scores on multiple MSQA datasets. |
Transfer Learning for Text Classification via Model Risk Analysis (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to text classification are limited by distribution drift and misprediction risk. |
| Approach: | They propose a model risk analysis approach to adapt a pre-trained DNN model to a new dataset given only a small set of representative data. |
| Outcome: | The proposed model performs considerably better than existing approaches on real datasets. |
A Multi- versus a Single-classifier Approach for the Identification of Modality in the Portuguese Language (L18-1)
Copied to clipboard
| Challenge: | Comparative study of two different approaches to build an automatic classification system for Modality values in the Portuguese language. |
| Approach: | They propose to use a single multi-class classifier with the full Portuguese language dataset that includes eleven modal verbs and a weighted average approach to build different classifiers for each verb. |
| Outcome: | The proposed system is based on a Portuguese language dataset with 11 modal verbs and two different classifiers, one for each verb. |
Habibi - a multi Dialect multi National Arabic Song Lyrics Corpus (2020.lrec-1)
Copied to clipboard
| Challenge: | Unlike western music, Arabic songs are poorly classified and the majority of the songs available online are classified under Modern Arabic Pop genre or what is now known as Franco-Arabic . |
| Approach: | They introduce Habibi the first Arabic Song Lyrics corpus for singers from 18 different Arabic countries. |
| Outcome: | The proposed corpus contains more than 30,000 Arabic song lyrics in 6 Arabic dialects for singers from 18 different arab countries. |
AutoCAD: Automatically Generate Counterfactuals for Mitigating Shortcut Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for generating counterfactuals rely on human efforts or task-specific designs. |
| Approach: | They propose to use a fully automatic and task-agnostic CAD generation framework to generate diverse counterfactuals. |
| Outcome: | The proposed framework outperforms human-in-the-loop and task-specific CAD methods on multiple out-of-domain and challenge benchmarks. |
Ad Lingua: Text Classification Improves Symbolism Prediction in Image Advertisements (2020.coling-main)
Copied to clipboard
Andrey Savchenko, Anton Alekseev, Sejeong Kwon, Elena Tutubalina, Evgeny Myasnikov, Sergey Nikolenko
| Challenge: | a recent study shows that image-based symbols are insufficient for symbolism prediction in visual advertising . a new method is proposed to help understand image advertisements . |
| Approach: | They propose a multimodal image-based classifier and object detection classifier for symbols . they propose 'symbolic' annotation tasks to help users understand ads' |
| Outcome: | The proposed system establishes state-of-the-art in symbolism prediction. |
Sentiment Analysis: It’s Complicated! (N18-1)
Copied to clipboard
Kian Kenyon-Dean, Eisha Ahmed, Scott Fujimoto, Jeremy Georges-Filteau, Christopher Glasz, Barleen Kaur, Auguste Lalande, Shruti Bhanderi, Robert Belfer, Nirmal Kanagasabai, Roman Sarrazingendron, Rohit Verma, Derek Ruths
| Challenge: | a dataset of over 7,000 tweets annotated with 5x coverage is used for sentiment analysis . a "complicated" class of sentiment is used to categorize text based on a predefined notion of sentiment . |
| Approach: | They propose to use a "complicated" class of sentiment to categorize tweets . they build a publicly available tweet sentiment analysis dataset . |
| Outcome: | The proposed classifiers perform better over a new publicly available TSA dataset . the classifier performance is compared with existing methods and improves on existing ones . |
Implicit Discourse Relation Classification For Nigerian Pidgin (2025.coling-main)
Copied to clipboard
| Challenge: | Existing discourse parsing tools are not available for Nigerian Pidgin (NP) this task requires supervised training and requires prompting. |
| Approach: | They propose to use implicit discourse relation classification (IDRC) for Nigerian Pidgin, which requires supervised training. |
| Outcome: | The proposed framework outperforms baseline and NP IDR classifiers in f1 scores. |
Training Classifiers with Natural Language Explanations (P18-1)
Copied to clipboard
| Challenge: | a semantic parser converts explanations into programmatic labeling functions . a standard protocol for obtaining a labeled dataset provides only one bit of information per example . |
| Approach: | They propose a framework where an annotator provides an explanation for each labeling decision . they use a semantic parser to convert these explanations into programmatic labeling functions . |
| Outcome: | The proposed framework trains classifiers faster by providing explanations instead of labels . the proposed framework is based on a rule-based semantic parser . |
Label-Driven Denoising Framework for Multi-Label Few-Shot Aspect Category Detection (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for ACD use label information of aspect categories to detect aspect categories . but, they still suffer from noise problems due to lack of supervised data . |
| Approach: | They propose a Label-Driven Denoising Framework to alleviate noise problems for ACD subtask . they use the label information of each aspect to generate a better prototype . |
| Outcome: | The proposed framework improves the performance of the multi-label few-shot Aspect Category Detection task. |
MetaSLRCL: A Self-Adaptive Learning Rate and Curriculum Learning Based Framework for Few-Shot Text Classification (2022.coling-1)
Copied to clipboard
| Challenge: | Existing few-shot text classification methods lack labeled data in many scenarios. |
| Approach: | They propose a meta learning framework that obtains different learning rates for different tasks and neural network layers to enable the meta learner to quickly adapt to new training data. |
| Outcome: | The proposed framework can obtain different learning rates for different tasks and neural network layers so as to enable the meta learner to quickly adapt to new tasks. |
Learning to Recognize Dialect Features (2021.naacl-main)
Copied to clipboard
| Challenge: | linguistics do not characterize dialects as simple categories, but as collections of correlated features. |
| Approach: | They propose two multitask learning approaches based on pretrained transformers to detect dialect features in speech and text. |
| Outcome: | The proposed models learn to recognize many features with high accuracy on 22 dialect features of Indian English. |
Improving Graph-based Sentence Ordering with Iteratively Predicted Pairwise Orderings (2021.emnlp-main)
Copied to clipboard
Shaopeng Lai, Ante Wang, Fandong Meng, Jie Zhou, Yubin Ge, Jiali Zeng, Junfeng Yao, Degen Huang, Jinsong Su
| Challenge: | Existing sentence ordering models can be classified into pairwise ordering models and set-to-sequence models. |
| Approach: | They propose a novel sentence ordering framework which introduces two classifiers to make better use of pairwise orderings for graph-based sentence ordering. |
| Outcome: | The proposed model achieves state-of-the-art performance on five commonly-used datasets. |
Data Expansion Using WordNet-based Semantic Expansion and Word Disambiguation for Cyberbullying Detection (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing methods to identify cyberbullying from text are limited due to the complexity of the content and the lack of labeled large-scale corpus. |
| Approach: | They propose a data augmentation-based approach that could enhance the automatic detection of cyberbullying in social media texts. |
| Outcome: | The proposed approach overcomes limitations of social media posts with word sense disambiguation and synonymy relation . results show that the proposed approach improves on the existing classifiers with and without data augmentation. |
On the Curious Case of l2 norm of Sense Embeddings (2022.findings-emnlp)
Copied to clipboard
| Challenge: | l2 norm of sense embeddings encodes information related to frequency of that sense in the training corpus . l2-normal feature is useful for word-in-context (WiC) and word sense disambiguation (WSD) |
| Approach: | They propose to include the l2 norm of a sense embedding as a feature in a classifier to improve word sense learning methods that use static sense embeds. |
| Outcome: | The l2 norm of sense embeddings is a surprisingly effective feature for word sense related tasks such as word-in-context (WiC) and word sense disambiguation (WSD). |
Exploratory Neural Relation Classification for Domain Knowledge Acquisition (C18-1)
Copied to clipboard
| Challenge: | Existing methods for relation classification are limited and lack of low-frequency relations in specific domains. |
| Approach: | They propose a method to learn a classifier on pre-defined relations and discover new relations expressed in texts. |
| Outcome: | The proposed method can classify entities into a finite set of relations and discover relations with high precision and recall. |
Validating Automatic Evaluation of Controllable Counterspeech Generation: Rankings Matter More Than Scores (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing methods for evaluating attributes of counterspeech are limited and the validity of such evaluations is questionable when the classifiers themselves have only modest performance. |
| Approach: | They examine the automatic evaluation of counterspeech attributes using a multi-attribute counterseech dataset containing 2,728 samples. |
| Outcome: | The proposed model can be trusted by classifier validation, and it can rank models with confidence. |
Predicting Desirable Revisions of Evidence and Reasoning in Argumentative Writing (2023.findings-eacl)
Copied to clipboard
| Challenge: | Using the essay context of the revision and feedback from students prior to the revision, we identify desirable and undesirable revisions. |
| Approach: | They propose to use the essay context of the revision and the feedback students received before the revision to improve classifier performance. |
| Outcome: | The proposed models improve over baseline models, while models utilizing context improve over the baseline models. |
Don’t sweat the small stuff, classify the rest: Sample Shielding to protect text classifiers against adversarial attacks (2022.naacl-main)
Copied to clipboard
| Challenge: | Current text classifiers are subject to adversarial attacks from adversaries, typically executed using machine learning methods. |
| Approach: | They propose a novel and intuitive defense strategy called Sample Shielding that is attacker and classifier agnostic and does not require reconfiguration of the classifier or external resources. |
| Outcome: | The proposed defense is attacker and classifier agnostic and does not require reconfiguration of the classifier or external resources and is simple to implement. |
Learning with Latent Language (N18-1)
Copied to clipboard
| Challenge: | Using the space of natural language strings as a parameter space is an effective way to capture natural task structure. |
| Approach: | They propose to use natural language as a parameter space for few-shot learning problems including classification, transduction and policy search. |
| Outcome: | The proposed model outperforms models with a linguistic parameterization on image classification, text editing, and reinforcement learning. |
Elote, Choclo and Mazorca: on the Varieties of Spanish (2024.naacl-long)
Copied to clipboard
| Challenge: | Spanish is the official language in 20 countries and the second most-spoken native language . available corpora treat it as one monolithic language, damping prediction power . |
| Approach: | They compile and curate datasets in different varieties of Spanish around the world at an unprecedented scale and create the CEREAL corpus. |
| Outcome: | The results show that Spanish is a multilingual language with a wide range of cultural and cultural influences. |
On Controlling Fallback Responses for Grounded Dialogue Generation (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing knowledge grounded dialogue frameworks assume that the user intention is always answerable. |
| Approach: | They propose a framework that automatically generates a control token with the generator to bias the succeeding response towards informativeness for answerable contexts and fallback for unanswerable context. |
| Outcome: | The proposed framework incorporates fallback responses to respond to unanswerable contexts in an informative manner while retaining informativeness for answerable context. |
P-Stance: A Large Dataset for Stance Detection in Political Domain (2021.findings-acl)
Copied to clipboard
| Challenge: | stance detection is a method to determine whether a text author is in favor of, against or neutral toward a specific target. |
| Approach: | They propose to use a large stance detection dataset in the political domain to detect stances on twitter. |
| Outcome: | The proposed model achieves a macro-average F1-score of 80.53% and can be used to improve cross-domain stance detection. |
Long-form evaluation of model editing (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing evaluations of model editing only use the ‘next few tokens’ completions after a prompt. |
| Approach: | They propose a new evaluation protocol that measures the efficacy and impact of model editing in long-form generative settings by using a machine-rated survey and a classifier which correlates well with human ratings. |
| Outcome: | The proposed evaluation protocol has little relationship with short-form metrics despite being designed to extend efficacy, generalization, locality, and portability into a long-form setting. |
SEP-MLDC: A Simple and Effective Paradigm for Multi-Label Document Classification (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods focus on optimizing document features, overlooking the potential of high-quality label features to enhance classification performance. |
| Approach: | They propose a multi-label document classification paradigm that utilizes large language models to expand the label content and generate pseudo-samples for the tail categories. |
| Outcome: | The proposed method significantly outperforms state-of-the-art models. |
Deep Subjecthood: Higher-Order Grammatical Features in Multilingual BERT (2021.eacl-main)
Copied to clipboard
| Challenge: | a recent study has shown that multilingual BERT encodes sentences in structurally meaningful ways. |
| Approach: | They analyze how morphosyntactic alignment manifests across embedding spaces of languages . they train classifiers to recover subjecthood of mBERT embedds in transitive sentences . |
| Outcome: | The proposed model encodes a high-order grammatical feature of morphosyntactic alignment across languages . the results show that the classifier distributions reflect the morphological alignment of their training languages based on the results . |
Have a Cake and Eat it Too: Assessing Discriminating Performance of an Intelligibility Index Obtained from a Reduced Sample Size (2020.lrec-1)
Copied to clipboard
| Challenge: | Random vs. phonetically motivated reduction of linguistic material in speech disordered populations has a higher ranking ability than when the sample size is reduced based on a phonetic criterium. |
| Approach: | They investigate random vs. phonetically motivated reduction of linguistic material in speech disordered populations and their impact on the discrimination classifier. |
| Outcome: | The proposed method is reliable and efficient, and it is of significant interest to clinicians and patients. |
Red Teaming Language Models with Language Models (2022.emnlp-main)
Copied to clipboard
Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, Geoffrey Irving
| Challenge: | Prior work has found that language models (LMs) can harm users in hard-to-predict ways, and human annotation is expensive, limiting the number and diversity of test cases. |
| Approach: | They propose to generate test inputs using an LM itself, and use a classifier to detect harmful behavior on test input. |
| Outcome: | The proposed approach detects tens of thousands of offensive responses in a 280B parameter LM chatbot. |
Deep Natural Language Feature Learning for Interpretable Prediction (2023.emnlp-main)
Copied to clipboard
| Challenge: | Using a small transformer language model, we can break down a complex task into a set of intermediary easier sub-tasks. |
| Approach: | They propose a method to break down a main task into a set of intermediary easier sub-tasks, which are formulated in natural language as binary questions related to the final target task. |
| Outcome: | The proposed method breaks down a complex task into a set of easier sub-tasks, which are formulated in natural language as binary questions related to the final target task. |
TBD3: A Thresholding-Based Dynamic Depression Detection from Social Media for Low-Resource Users (2022.lrec-1)
Copied to clipboard
| Challenge: | Social media are heavily used by many users to share their mental health concerns and diagnoses. |
| Approach: | They propose a dynamic thresholding technique that adjusts the classifier’s sensitivity as a function of the number of posts a user has. |
| Outcome: | The proposed method reduces the margin between users with many and few posts, on average, by 45% across all methods and increases overall performance, onaverage, by 33%. |
Cross-Domain Sentiment Classification with Target Domain Specific Information (P18-1)
Copied to clipboard
| Challenge: | Existing methods for sentiment classification focus on learning domain-invariant representations . few of them pay attention to domain-specific information, which should also be informative. |
| Approach: | They propose a method to extract domain specific and invariant representations and train a classifier on each of them. |
| Outcome: | The proposed model can achieve better performance than state-of-the-art methods. |
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection (2022.acl-long)
Copied to clipboard
| Challenge: | Toxic language detection systems often falsely flag text that contains minority group mentions as toxic . this over-reliance on spurious correlations also causes systems to struggle with detecting implicitly toxic language. |
| Approach: | They develop a machine-generated dataset of toxic and benign statements about 13 minority groups that generates subtly toxic and harmless text with a massive pretrained language model. |
| Outcome: | The proposed method can detect toxic and benign statements on a large scale . it can also detect hate speech on 94.5% of the toxic examples . |
Multi-label Few/Zero-shot Learning with Knowledge Aggregated from Multiple Label Graphs (2020.emnlp-main)
Copied to clipboard
| Challenge: | Few/zero-shot learning is a big challenge of many classification tasks, where a classifier is required to recognise instances of classes that have very few or even no training samples. |
| Approach: | They propose a multi-graph aggregation model that fuses knowledge from multiple label graphs encoding different semantic label relationships to improve multi-label zero/few-shot document classification. |
| Outcome: | The proposed model improves on two large clinical datasets and the EU legislation dataset on few/zero-shot labels. |
CLiMP: A Benchmark for Chinese Language Model Evaluation (2021.eacl-main)
Copied to clipboard
| Challenge: | Linguistically informed analyses of language models (LMs) contribute to understanding and improvement of such models. |
| Approach: | They introduce a corpus of Chinese linguistic minimal pairs (CLiMP) to investigate what knowledge Chinese LMs acquire. |
| Outcome: | The proposed corpus of Chinese linguistic minimal pairs (CLiMP) covers 9 major Chinese linguist phenomena. |
Joint Learning of Hyperbolic Label Embeddings for Hierarchical Multi-label Classification (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for hierarchical multi-label classification do not assume label hierarchy exists. |
| Approach: | They propose to jointly learn the classifier parameters as well as the label embeddings . they propose to use hyperbolic embeddables to gain better generalisation over the labels . |
| Outcome: | The proposed method achieves state-of-the-art generalization on benchmarks and is more accurate than existing methods. |
Upstream Mitigation Is Not All You Need: Testing the Bias Transfer Hypothesis in Pre-Trained Language Models (2022.acl-long)
Copied to clipboard
| Challenge: | Large language models and other massively pre-trained "foundation" models can easily adapt to a wide variety of downstream tasks in a process called finetuning. |
| Approach: | They propose to use the bias transfer hypothesis to reduce social biases internalized by large language models during pre-training into harmful task-specific behavior after fine-tuning. |
| Outcome: | The bias transfer hypothesis is the theory that social biases internalized by large language models during pre-training transfer into harmful task-specific behavior after fine-tuning. |
Label-Free Distant Supervision for Relation Extraction via Knowledge Graph Embedding (D18-1)
Copied to clipboard
| Challenge: | Existing methods to generate large scale labeled data for relation extraction produce noisy relation labels when there are multiple relationships between entities. |
| Approach: | They propose a method which assumes that a pair of entities appears in a Knowledge Graph and trains a relation classifier. |
| Outcome: | The proposed method performs well in the current distant supervision dataset. |
Classifying Sluice Occurrences in Dialogue (L18-1)
Copied to clipboard
| Challenge: | Ellipsis is an important challenge for natural language processing systems, says a new paper . previous work on ellipsis focused on news data, but sluicing presents a challenge for dialogue systems . |
| Approach: | They describe a corpus of 4100 sluice occurrences from the NYTimes Gigaword corpus . they build a classifier model to automatically classify slujce . |
| Outcome: | The proposed corpus contains 4100 sluice occurrences, with an accuracy of 67% . the work will support empirical research into slujcing in dialogue systems . |
How Do We Answer Complex Questions: Discourse Structure of Long-form Answers (2022.acl-long)
Copied to clipboard
| Challenge: | Recent work explored long-form answers, where answers are free-form texts consisting of multiple sentences. |
| Approach: | They develop an ontology of six sentence-level functional roles for long-form answers . they annotate 3.9k sentences in 640 answer paragraphs and train a strong classifier . |
| Outcome: | The proposed model-generated answers agree less with model-driven answers than human-written answers. |
Energy-based Unknown Intent Detection with Data Manipulation (2021.findings-acl)
Copied to clipboard
| Challenge: | Unknown intent detection aims to identify the out-of-distribution (OOD) utterance whose intent has never appeared in the training set. |
| Approach: | They propose a framework to generate high-quality OOD utterances with importance weighTs (GOT) their framework is fine-tuned to detect out-of-distribution utterrances . |
| Outcome: | The proposed framework can achieve state-of-the-art results on two benchmark datasets. |
Adversarial Domain Adaptation for Machine Reading Comprehension (D19-1)
Copied to clipboard
| Challenge: | Existing models for machine reading comprehension rely on large amounts of human-annotated in-domain data. |
| Approach: | They propose an unsupervised domain adaptation framework for Machine Reading Comprehension where the source domain has a large amount of labeled data, while only unlabeled passages are available in the target domain. |
| Outcome: | The proposed framework can be generalizable to different MRC models and datasets and can be extended to semi-supervised learning. |
Adversarial Category Alignment Network for Cross-domain Sentiment Classification (N19-1)
Copied to clipboard
| Challenge: | Existing methods for cross-domain sentiment classification focus on aligning marginal distribution without taking category-specific decision boundaries into consideration. |
| Approach: | They propose an adversarial category alignment network to enhance category consistency . experimental results show the proposed method can achieve state-of-the-art performance . |
| Outcome: | The proposed method achieves state-of-the-art performance and produces more discriminative features on benchmark datasets. |
Contextual Text Style Transfer (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for text style transfer are limited by the lack of parallel data. |
| Approach: | They propose a task to translate a sentence into a desired style with its surrounding context taken into account. |
| Outcome: | The proposed model outperforms state-of-the-art methods across style accuracy, content preservation and contextual consistency metrics. |
Generalised Unsupervised Domain Adaptation of Neural Machine Translation with Cross-Lingual Data Selection (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on unsupervised domain adaptation of neural machine translation assumes access to monolingual text in either the source or target language in the new domain. |
| Approach: | They propose a method to extract in-domain sentences from a large generic monolingual corpus from 'missing' text. |
| Outcome: | The proposed method outperforms baselines up to +1.5 BLEU score on five diverse domains in three language pairs and a real-world translation scenario. |
Randomized Smoothing with Masked Inference for Adversarially Robust Text Classifications (2023.acl-long)
Copied to clipboard
| Challenge: | Large-scale pre-trained language models are brittle against specifically crafted adversarial examples, leading to increasing interest in probing the adversariality of NLP systems. |
| Approach: | They propose a two-stage framework that combines randomized smoothing and masked inference to improve the adversarial robustness of NLP systems. |
| Outcome: | The proposed framework improves adversarial robustness by 2 to 3 times over existing state-of-the-art methods on benchmark datasets. |
Interpretable Neural Predictions with Differentiable Binary Variables (P19-1)
Copied to clipboard
| Challenge: | Neural networks are bringing incredible performance gains on text classification tasks, but they also require interpretability. |
| Approach: | They propose a latent model that selects a rationale and a classifier that learns from the words in the rationale alone. |
| Outcome: | The proposed model can predict expected value of penalties without REINFORCE and can be directly optimised towards a pre-specified text selection rate. |
Task-Aware Representation of Sentences for Generic Text Classification (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to text classification use a transformer architecture with a linear layer on top. |
| Approach: | They propose a transformer-based approach that outputs a class distribution for a given prediction problem. |
| Outcome: | The proposed model outperforms existing approaches on small training data and can learn to predict new classes even with no training examples. |
Effective Inter-Clause Modeling for End-to-End Emotion-Cause Pair Extraction (2020.acl-main)
Copied to clipboard
| Challenge: | Emotion-cause pair extraction aims to extract all emotion clauses coupled with their cause clauses from a given document. |
| Approach: | They propose a one-step neural approach which emphasizes inter-clause modeling to perform end-to-end extraction. |
| Outcome: | The proposed method outperforms existing methods in the extraction of emotion-cause pairs . it emphasizes inter-clause modeling to perform end-to-end extraction . |
Continual Generalized Intent Discovery: Marching Towards Dynamic and Open-world Intent Recognition (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Currently, the generalized intent classification system only considers one stage of OOD learning and requires all IND data for joint training. |
| Approach: | They propose a task that detects OOD intents from dynamic OOD data streams . they propose CGID method that bootstraps new intent discovery through class prototypes . |
| Outcome: | The proposed task can detect out-of-domain (OOD) queries and extend them to the in-domain classifier . it can safely and efficiently detect out of-domain queries and avoid wrong operations . |
Simple Algorithms For Sentiment Analysis On Sentiment Rich, Data Poor Domains. (C18-1)
Copied to clipboard
| Challenge: | Standard word embedding algorithms learn vector representations from large corpora of text documents in unsupervised fashion. |
| Approach: | They propose an algorithm that learns word embeddings jointly with a classifier . their algorithm leverages document label information to learn vector representations of words . |
| Outcome: | The proposed algorithm has superior performance on domains with limited data compared to other methods. |
Improving Distantly-supervised Entity Typing with Compact Latent Space Clustering (N19-1)
Copied to clipboard
| Challenge: | Existing studies have addressed this problem with partial-label loss, but they suffer from confirmation bias, which means the classifier fit a pseudo data distribution given by itself. |
| Approach: | They propose to regularize distantly supervised models with Compact Latent Space Clustering to bypass this problem and effectively utilize noisy data yet. |
| Outcome: | The proposed model outperforms state-of-the-art models on standard benchmarks on fine-grained entity typing (FET) by a significant margin. |
CASA: Causality-driven Argument Sufficiency Assessment (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods to assess the sufficiency of arguments are laborious and inconsistent due to subjective criteria. |
| Approach: | They propose a causality-driven argument sufficiency assessment framework that uses the probability of sufficience to estimate the probability that a premise event would lead to a conclusion when both premise and conclusion events are absent. |
| Outcome: | The proposed framework identifies insufficient arguments and improves them in a writing aid application. |
Rethinking Word Similarity: Semantic Similarity through Classification Confusion (2025.naacl-long)
Copied to clipboard
| Challenge: | Word similarity measures cannot capture context-dependent, asymmetrical, polysemous nature of semantic similarity. |
| Approach: | They propose a new measure of similarity that reframes semantic similarity in terms of feature-based classification confusion. |
| Outcome: | The proposed model is comparable to cosine similarity in matching human similarity judgments across several datasets and can measure similarity using predetermined features of interest. |
Sampling Better Negatives for Distantly Supervised Named Entity Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing supervised named entity recognition approaches rely on human annotations. |
| Approach: | They propose a method to select negative samples with high similarities with positive samples . they propose to use automatically labeled training data instead of human annotations . |
| Outcome: | The proposed method achieves consistent performance improvements on four distantly supervised NER datasets. |
Communicative-Function-Based Sentence Classification for Construction of an Academic Formulaic Expression Database (2021.eacl-main)
Copied to clipboard
| Challenge: | Formulaic expressions (FEs) are used in scientific papers. |
| Approach: | They propose to use top–down approach to assign CF labels to sentences and then extract FEs from them using a CF-labelled sentence dataset. |
| Outcome: | The proposed method can be used to build FE databases of disciplines that are different from the training data. |
Interpretable Research Replication Prediction via Variational Contextual Consistency Sentence Masking (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for predicting research replication are insufficient especially for long research papers. |
| Approach: | They propose to build an interpretable neural model which can provide sentence-level explanations and apply weakly supervised approach to leverage large corpus of unlabeled datasets. |
| Outcome: | The proposed model can provide sentence-level explanations and leverage large unlabeled datasets to boost interpretability and improve prediction performance. |
ThoughtProbe: Classifier-Guided LLM Thought Space Exploration via Probing Representations (2025.emnlp-main)
Copied to clipboard
| Challenge: | Unlike previous works that manipulate representations to steer LLM generation, ThoughtProbe harnesses them as discriminative signals to guide the tree-structured response space exploration. |
| Approach: | They propose a tree-structured inference-time framework that leverages the hidden reasoning features of Large Language Models to improve their reasoning performance. |
| Outcome: | The proposed framework improves reasoning performance across multiple arithmetic reasoning benchmarks and covers valid reasoning chains and identifies optimal answers. |
UOREX: Towards Uncertainty-Aware Open Relation Extraction (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for relation extraction are limited by their inability to accurately self-assess their performance. |
| Approach: | They propose an approach that effectively models a part of the epistemic uncertainty within OpenRE by preventing overconfident errors. |
| Outcome: | The proposed approach improves OpenRE reliability by preventing overconfident errors. |
On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmark (2022.findings-acl)
Copied to clipboard
Hao Sun, Guangxuan Xu, Jiawen Deng, Jiale Cheng, Chujie Zheng, Hao Zhou, Nanyun Peng, Xiaoyan Zhu, Minlie Huang
| Challenge: | Dialogue safety problems severely limit the real-world deployment of generative conversational models. |
| Approach: | They propose a taxonomy for dialogue safety specifically designed to capture unsafe behaviors in human-bot dialogue settings. |
| Outcome: | The proposed taxonomy captures unsafe behaviors in human-bot dialogue settings with rich context-sensitive unsafe examples. |
OpenCeres: When Open Information Extraction Meets the Semi-Structured Web (N19-1)
Copied to clipboard
| Challenge: | Open Information Extraction (OpenIE) is a problem of extracting triples from natural language text whose predicate relations are not aligned to any pre-defined ontology. |
| Approach: | They propose an open-source method to extract triples from semi-structured websites . they use a semi-supervised label propagation technique to create training data for relations . |
| Outcome: | The proposed method extracts over 2 million triples from 31 websites in the movie vertical. |
On the Robustness of Language Encoders against Grammatical Errors (2020.acl-main)
Copied to clipboard
| Challenge: | Pre-trained language encoders are effective in facilitating downstream natural language processing tasks, but they often assume training and test corpora are clean and it is unclear how the models behave when confronted with noisy input. |
| Approach: | They conduct adversarial attacks to simulate grammatical errors on clean text data. |
| Outcome: | The proposed model performs better when confronted with natural grammatical errors than when faced with noisy input. |
Large Language Models Share Representations of Latent Grammatical Concepts Across Typologically Diverse Languages (2025.naacl-long)
Copied to clipboard
| Challenge: | In the brains of human bilinguals, syntax processing may occur in similar regions for their first and second language, depending on factors like when the second language was learned and language proficiency. |
| Approach: | They propose to use sparse autoencoders to train Llama-3-8B and Aya-23-8B models to train multilingual models that share morphsyntactic representations of grammatical concepts. |
| Outcome: | The proposed model can predict plural verbs in different languages by activating the same plural feature. |
Domain- and Task-Adaptation for VaccinChatNL, a Dutch COVID-19 FAQ Answering Corpus and Classification Model (2022.coling-1)
Copied to clipboard
| Challenge: | VaccinChatNL is the first FAQ chatbot with over 12k user queries . it can be used to find the representative question that matches a user's request . |
| Approach: | They build a FAQ chatbot from 50 question-answer pairs and annotate user questions with appropriate or new answer classes. |
| Outcome: | The VaccinChatNL is the first publicly available Dutch FAQ answering corpus with large groups of human-paraphrased questions. |
Examining and Adapting Time for Multilingual Classification via Mixture of Temporal Experts (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing classification models only consider the temporal variations of existing data . current models focus on English corpora, leaving time as domains unexplored . |
| Approach: | They propose a framework to generalize classifiers over time on four languages, English, Danish, French, and German. |
| Outcome: | The proposed framework can generalize classifiers over time on four languages, English, Danish, French, and German. |
FastClass: A Time-Efficient Approach to Weakly-Supervised Text Classification (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent research shows keyword-driven methods can achieve state-of-the-art performance on various tasks. |
| Approach: | They propose an efficient weakly-supervised text classification approach using unlabeled data . they use dense text representation to retrieve class-relevant documents from unlabed corpus . |
| Outcome: | The proposed weakly-supervised classification method outperforms keyword-driven models on a wide range of classification tasks. |
Cyberbullying Classifiers are Sensitive to Model-Agnostic Perturbations (2022.lrec-1)
Copied to clipboard
| Challenge: | toxicity classifiers rely on lexical cues, so creative language use can be detrimental to utility of current corpora and state-of-the-art models. |
| Approach: | They propose to use model-agnostic adversarial behavior to enhance toxic content classification models. |
| Outcome: | The proposed model-agnostic adversarial behavior and augmentation for cyberbullying detection are robust against word-level perturbations at a slight trade-off in overall task performance. |
Grey-box Adversarial Attack And Defence For Sentiment Classification (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent advances in deep neural networks have created applications for a range of different domains. |
| Approach: | They propose a grey-box adversarial attack and defence framework for sentiment classification . they show that the framework produces an improved classifier that is robust in defending . |
| Outcome: | The proposed framework produces an improved classifier that is robust in defending against multiple adversarial attacking methods. |
Integrating Tree Structures and Graph Structures with Neural Networks to Classify Discussion Discourse Acts (C18-1)
Copied to clipboard
| Challenge: | Existing models that analyze textual contents and discussion structures require understanding of textual content and discussion structure. |
| Approach: | They propose a model that integrates discussion structures with neural networks to classify discourse acts. |
| Outcome: | The proposed model improves accuracy and FB1 score by 1.5% compared to the previous best model. |
An Analysis of Dialogue Act Sequence Similarity Across Multiple Domains (2022.lrec-1)
Copied to clipboard
| Challenge: | a recent study shows that many machine learning models perform poorly when exposed to domain shifts due to contextual differences. |
| Approach: | They analyze dialogue act sequences from related domains to predict performance degradation . they find that when dialogue acts sequences are dissimilar they lie further away in embedding space . |
| Outcome: | The proposed model can be trained even when the datasets are corrupted with noise. |
TaxoClass: Hierarchical Multi-Label Text Classification Using Only Class Names (2021.naacl-main)
Copied to clipboard
| Challenge: | Hierarchical multi-label text classification (HMTC) aims to assign each text document to a set of relevant classes from a taxonomy. |
| Approach: | They propose to conduct HMTC based on only class surface names as supervision signals to mimic human experts. |
| Outcome: | The proposed framework outperforms the best existing method by 25% on two challenging datasets. |
MuTox: Universal MUltilingual Audio-based TOXicity Dataset and Zero-shot Detector (2024.findings-acl)
Copied to clipboard
Marta Costa-jussà, Mariano Meglioli, Pierre Andrews, David Dale, Prangthip Hansanti, Elahe Kalbassi, Alexandre Mourachko, Christophe Ropers, Carleigh Wood
| Challenge: | Existing studies on text-based toxicity detection for other languages are limited, especially for languages other than English. |
| Approach: | They propose a multilingual audio-based toxicity classifier which covers 14 different linguistic families and a dataset of 20,000 audio utterances for English and Spanish. |
| Outcome: | The new classifier improves F1-Score by an average of 100% when compared to existing wordlist-based classifiers. |
Delta-training: Simple Semi-Supervised Text Classification using Pretrained Word Embeddings (D19-1)
Copied to clipboard
| Challenge: | Pretrained word embeddings outperforms classifiers with randomly initialized word embeds, a new method is proposed for semi-supervised text classification. |
| Approach: | They propose a method that uses pretrained word embeddings to predict text classification . they use unlabeled data to build a classifier, and use early-stopping to improve performance . |
| Outcome: | The proposed method outperforms self-training and co-training frameworks on unlabeled data. |
Is Probing All You Need? Indicator Tasks as an Alternative to Probing Embedding Spaces (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing probing tasks are designed to evaluate the information existing in representations by training a simple classification model. |
| Approach: | They propose to use indicators to query embedding spaces for the existence of certain properties to determine whether a property exists in an embeddable space. |
| Outcome: | The proposed indicators provide a more accurate picture of the information captured and removed compared to probes. |
Towards end-2-end learning for predicting behavior codes from spoken utterances in psychotherapy conversations (2020.acl-main)
Copied to clipboard
| Challenge: | Xu and Sarikaya, 2014) proposes a framework for predicting utterance level labels directly from speech features. |
| Approach: | They propose a framework for predicting utterance level labels directly from speech features using a pretrained Speech-2-Vector encoder as bottleneck. |
| Outcome: | The proposed model outperforms state-of-the-art approaches which use transcribed text for the task of predicting psychotherapy-relevant behavior codes. |
TWAG: A Topic-Guided Wikipedia Abstract Generator (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models view Wikipedia abstract as plain text, ignoring that it is a description of a certain entity and can be decomposed into different topics. |
| Approach: | They propose a model that guides Wikipedia abstract generation with topical information. |
| Outcome: | The proposed model outperforms baselines and is capable of generating comprehensive abstracts. |
Corpus-based Identification of Verbs Participating in Verb Alternations Using Classification and Manual Annotation (2020.coling-main)
Copied to clipboard
| Challenge: | Verb alternations allow verbs to appear in a set of syntactically different constructions whose associated semantic frames are systematically related. |
| Approach: | They use ENCOW and VerbNet data to train classifiers to predict the instrument subject alternation and the causative-inchoative alternation . they use count-based and vector-based features as well as perplexity-based language model features to reflect each alternation’s felicity by simulating it. |
| Outcome: | The proposed approach reduces the required annotation effort by only presenting annotators with the highest-scoring candidates from the previous classification. |
A Formidable Ability: Detecting Adjectival Extremeness with DSMs (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on distributional semantic models capture abstract semantic properties across domains . abstract properties can form the basis for abstract semantic classes . |
| Approach: | They propose to use distributional semantic models to capture cross-domain properties . they use extremeness to model emergence of intensifier meaning in adverbs . |
| Outcome: | The proposed model can capture extremeness and intensifier meaning in adverbs. |
Unsupervised Label Refinement Improves Dataless Text Classification (2021.findings-acl)
Copied to clipboard
| Challenge: | Prior work has shown that dataless classifiers can classify text into categories without using annotated training data. |
| Approach: | They propose a clustering-based approach that assigns a score to any document paired with a label description. |
| Outcome: | The proposed method improves classification performance across multiple datasets and makes it more robust to label descriptions. |
Dynamic Long Context Reasoning over Compressed Memory via End-to-End Reinforcement Learning (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models face quadratic computational costs, information forgetting, and context fragmentation . current methodologies diverge into three paradigms, each facing inherent trade-offs between Equal contribution. |
| Approach: | They propose a framework that enables efficient long-context inference via chunk-wise compression and selective memory recall. |
| Outcome: | The proposed framework reduces peak GPU memory usage and speeds up inference on multi-hop reasoning benchmarks. |
Adversarial and Domain-Aware BERT for Cross-Domain Sentiment Analysis (2020.acl-main)
Copied to clipboard
| Challenge: | Cross-domain sentiment classification requires large amounts of labeled data. |
| Approach: | They propose to apply a pre-training language model BERT on unsupervised domain adaptation . they propose to distill domain-specific features in a self-supervised way . |
| Outcome: | The proposed model outperforms state-of-the-art methods on Amazon dataset . it can be applied to the unsupervised domain adaptation task without domain awareness . |
Word Complexity Estimation for Japanese Lexical Simplification (2020.lrec-1)
Copied to clipboard
| Challenge: | Experimental results show that the proposed method achieves the highest performance of Japanese lexical simplification. |
| Approach: | They propose a large-scale word complexity lexicon, a synonym lexicone and a toolkit for developing and benchmarking Japanese lexical simplification systems. |
| Outcome: | The proposed method achieves the highest performance of Japanese lexical simplification. |
Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent Large Language Models (LLMs) generate factually incorrect answers based on their parametric memory. |
| Approach: | They propose a retrieval-augmented large language model that can dynamically select the most suitable strategy based on query complexity. |
| Outcome: | The proposed approach improves the performance of QA systems on open-domain QA datasets. |
Exploring Self-Identified Counseling Expertise in Online Support Forums (2021.findings-acl)
Copied to clipboard
Allison Lahnala, Yuntian Zhao, Charles Welch, Jonathan K. Kummerfeld, Lawrence C An, Kenneth Resnicow, Rada Mihalcea, Verónica Pérez-Rosas
| Challenge: | Increasing number of people engage in online health forums, making it important to understand the quality of the advice they receive. |
| Approach: | They examine the role of expertise in responses to help-seeking posts . they find that a classifier can distinguish between peer and self-identified mental health professionals' interactions . |
| Outcome: | The findings show that experts' language use differs between groups, and that their comments engage the support-seeker further. |
Using English Baits to Catch Serbian Multi-Word Terminology (L18-1)
Copied to clipboard
| Challenge: | a new method for bilingual terminology extraction is proposed for a source language and a target language. |
| Approach: | They propose to use a bilingual terminology extraction approach for a source language and a target language to extract the terminology for sri lanka. |
| Outcome: | The proposed method extracts terminology for a source language and a target language from it. |
Enriching Epidemiological Thematic Features For Disease Surveillance Corpora Classification (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing disease surveillance systems use indicators to monitor official sources and unofficial sources. |
| Approach: | They propose a way to perform epidemiological document classification by enriching thematic features . they use a pre-trained biomedical language model with a novel approach . |
| Outcome: | The proposed method improves the classifier's ability to avoid false positive alerts on disease surveillance systems. |
NonFactS: NonFactual Summary Generation for Factuality Evaluation in Document Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained abstractive summarization models generate fluent summaries that are inconsistent with context document and contain nonfactual information. |
| Approach: | They propose a data generation model that synthesizes nonfactual summaries using human annotations. |
| Outcome: | The proposed model can generate nonfactual summaries and generalize to out-of-domain documents. |
DirectProbe: Studying Representations without Classifiers (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches for probing opaque representations often use training classifiers and use the accuracy, mutual information, or complexity as a proxy for the representation’s goodness. |
| Approach: | They propose a heuristic that directly studies the geometry of a representation by building upon the notion of 'version space' they argue that doing so can be unreliable because different representations may need different classifiers . |
| Outcome: | Experiments with linguistic tasks and contextualized embeddings show that even without training classifiers, DirectProbe can shine lights on how an embeddable space represents labels and anticipate the classifier performance for the representation. |
Learning to Faithfully Rationalize by Construction (2020.acl-main)
Copied to clipboard
| Challenge: | Neural models dominate NLP but it remains difficult to know why they make specific predictions for sequential text inputs. |
| Approach: | They propose a model to produce faithful rationales for neural text classification by defining independent snippet extraction and prediction modules. |
| Outcome: | The proposed model produces faithful explanations even when the model is complex and complex. |
Probing Task-Oriented Dialogue Representation from Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Using pre-trained language models, we find out which model has the most informative representation for task-oriented dialogue tasks. |
| Approach: | They propose a supervised classifier probe and unsupervised mutual information probe to investigate the mutual dependence between a real clustering and a representation clustering. |
| Outcome: | The proposed model is a supervised classifier probe and unsupervised mutual information probe. |
A Generalized Approach to Protest Event Detection in German Local News (2022.lrec-1)
Copied to clipboard
| Challenge: | Social scientists conduct protest event analysis to learn about developments and trends of the forms, scale and hot topics of political protests. |
| Approach: | They propose to use a German language resource to analyze newspaper articles on protest events . they train and evaluate transformer-based text classifiers to automatically detect relevant newspaper articles . |
| Outcome: | The proposed method achieves a binary F1-score of 93.3 %, but does not generalize well to other datasets. |
NaturalAdversaries: Can Naturalistic Adversaries Be as Effective as Artificial Adversaries? (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work has explored adversarial example generation for natural language understanding tasks, but these examples are often unrealistic and diverge from the real-world data distributions. |
| Approach: | They propose a framework for adversarial example generation that is effective at fooling a given classifier and a generative model based on the key tokens from the first stage. |
| Outcome: | The proposed framework generalizes across domains and offers insights for future research on improving robustness of neural text classification models. |
Hidden State Variability of Pretrained Language Models Can Guide Computation Reduction for Transfer Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to transfer a pretrained language model include fine-tuning all the parameters in the language model and adapting all its subsets. |
| Approach: | They propose to select layers based on the variability of their hidden states given a task-specific corpus. |
| Outcome: | The proposed model reduces the computational cost of transfer learning methods without sacrificing performance. |
What’s in a Name? Reducing Bias in Bios without Access to Protected Attributes (N19-1)
Copied to clipboard
Alexey Romanov, Maria De-Arteaga, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, Anna Rumshisky, Adam Kalai
| Challenge: | Existing methods for mitigating bias in machine learning systems rely on access to protected attributes such as race, gender, or age. |
| Approach: | They propose a method for discouraging correlation between predicted probability of an individual’s true occupation and a word embedding of their name. |
| Outcome: | The proposed method reduces race and gender biases, with almost no reduction in the classifier’s overall true positive rate. |
Topics to Avoid: Demoting Latent Confounds in Text Classification (D19-1)
Copied to clipboard
| Challenge: | Despite impressive performance on many text classification tasks, deep neural networks tend to learn frequent superficial patterns that are specific to the training data and do not always generalize well. |
| Approach: | They propose a method that represents latent topical confounds and a model which “unlearns” confounding features by predicting both the label of the input text and the confound. |
| Outcome: | The proposed model generalizes better and learns features indicative of the writing style rather than the content. |
Can Explanations Be Useful for Calibrating Black Box Models? (2022.acl-long)
Copied to clipboard
| Challenge: | Existing models are often used as black boxes to adapt to new domains, but there is no single recipe for making them work. |
| Approach: | They propose to use black box models to improve their performance on new domains by leveraging explanations of their behavior. |
| Outcome: | The proposed method improves model generalization performance on two tasks using explanations. |
Reweighting Strategy Based on Synthetic Data Identification for Sentence Similarity (2022.coling-1)
Copied to clipboard
| Challenge: | obtaining large amounts of human-annotated datasets to train a sentence embedding model is difficult and expensive. |
| Approach: | They propose to train a classifier that identifies machine-written sentences and then use it to train an embedding model on synthetic data. |
| Outcome: | The proposed method outperforms baselines on four real-world datasets and generalizes well. |
Weakly Supervised Subevent Knowledge Acquisition (2020.emnlp-main)
Copied to clipboard
| Challenge: | Subevents elaborate an event and exist in event descriptions. |
| Approach: | They propose a weakly supervised approach to extract subevent relation tuples from text . they then use the initial seed subeven pairs to train a contextual classifier . |
| Outcome: | The proposed method is high quality and covers a wide range of event types. |
Understanding User Resistance Strategies in Persuasive Conversations (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Persuasive dialog systems have various usages, such as donation persuation and physical exercise persulasion. |
| Approach: | They adopt a preliminary framework on persuasion resistance in psychology and build a fine-grained resistance strategy annotation scheme to analyze the persuitee's resistance strategies. |
| Outcome: | The proposed system can understand and address user resistance strategies appropriately. |
LLMEmbed: Rethinking Lightweight LLM’s Genuine Function in Text Classification (2024.acl-long)
Copied to clipboard
| Challenge: | Recent attempts to improve text classification performance are based on heuristic Chain-of-Thought (CoT) LLMEmbed is a simple and effective transfer learning strategy that can be used to improve the performance of large language models. |
| Approach: | They propose a simple transfer learning strategy to improve text classification using heuristic Chain-of-Thought. |
| Outcome: | The proposed method achieves strong performance on publicly available datasets while using low training overhead. |
Dense-Caption Matching and Frame-Selection Gating for Temporal Localization in VideoQA (2020.acl-main)
Copied to clipboard
| Challenge: | Recent years have witnessed a paradigm shift in the way we get our information, and a lot of it. |
| Approach: | They propose a video question answering model which integrates multi-modal input sources and finds temporally relevant information to answer questions. |
| Outcome: | The proposed model outperforms the state-of-the-art on a TVQA dataset. |
Explanation Regularisation through the Lens of Attributions (2025.coling-main)
Copied to clipboard
| Challenge: | Explanation regularisation (ER) is a method to guide text classifiers to form their predictions relying on tokens that humans consider plausible. |
| Approach: | They introduce an auxiliary explanation loss to measure how well an input attribution technique's output agrees with human-annotated rationales. |
| Outcome: | The proposed model improves classification performance in out-of-domain (OOD) settings by relying on tokens humans consider plausible. |
Embarrassingly Simple Performance Prediction for Abductive Natural Language Inference (2022.naacl-main)
Copied to clipboard
| Challenge: | a method for learning an NLI model is time-consuming and resource-intensive, but it can save time and resources. |
| Approach: | They propose a method for predicting model performance without fine-tuning it . they compare sentence embeddings with cosine similarity to classifiers . |
| Outcome: | The proposed method can save time and resources by comparing pre-trained models to real-world datasets. |
Learning to Learn and Predict: A Meta-Learning Approach for Multi-Label Classification (D19-1)
Copied to clipboard
| Challenge: | Existing models for multi-label classification ignore complexity and dependencies among labels . Experimental results show that our method can obtain more accurate multi-lab classification results. |
| Approach: | They propose a meta-learning method to capture complex label dependencies . they use a Meta-learner to jointly learn the training policies and prediction policies for different labels. |
| Outcome: | The proposed method can capture complex label dependencies on fine-grained entity typing and text classification tasks. |
Human Rationales as Attribution Priors for Explainable Stance Detection (2021.emnlp-main)
Copied to clipboard
| Challenge: | In this work, we present a method for imparting human-like rationalization to a stance detection model using crowdsourced annotations on a small fraction of the training data. |
| Approach: | They propose a method for imparting human-like rationalization to a stance detection model using crowdsourced annotations on a small fraction of the training data. |
| Outcome: | The proposed method improves the reasoning of a state-of-the-art classifier in a data-scarce setting at no cost in predictive performance. |
Classifier Probes May Just Learn from Linear Context Features (2020.coling-main)
Copied to clipboard
| Challenge: | Current probing methods can help to better estimate the complexity of learning, but not build a foundation for speculations about the nature of the linguistic structure encoded in the learned representations. |
| Approach: | They propose to use token embeddings to test whether probing tasks contain linguistic structure . they argue that current probing methods do not provide enough information to support this hypothesis . |
| Outcome: | The proposed method can be scrutinized and proves that representations encode linguistic structure even without additional linguistic structures. |
Text Counterfactuals via Latent Optimization and Shapley-Guided Search (2021.emnlp-main)
Copied to clipboard
| Challenge: | Using latent optimization and Shapley values, we generate a set of minimal modifications to the text to change the classifier's prediction. |
| Approach: | They propose to generate a counterfactual by making minimal modifications to the text to change the model's prediction. |
| Outcome: | The proposed approach achieves favorable performance compared to white-box and black-box baselines using human and automatic evaluations. |
Affective Event Classification with Discourse-enhanced Self-training (2020.emnlp-main)
Copied to clipboard
| Challenge: | Prior work on recognizing affective events focused on producing lexical resources of verbs or event phrases with corresponding affective polarity values. |
| Approach: | They propose a BERT-based model for affective event classification and a discourse-enhanced self-training method that iteratively improves the classifier with unlabeled data. |
| Outcome: | The proposed model outperforms existing models with unlabeled data and improves recall and precision. |
DiffuseDef: Improved Robustness to Adversarial Attacks via Iterative Denoising (2025.acl-long)
Copied to clipboard
| Challenge: | Existing adversarial defense methods for natural language processing still pose challenges to adversarials. |
| Approach: | They propose a novel adversarial defense method that incorporates a diffusion layer as a denoiser between the encoder and the classifier. |
| Outcome: | The proposed method improves over existing adversarial defense methods and achieves state-of-the-art performance against black-box and white-box adversarials. |
Exploiting Commonsense Knowledge about Objects for Visual Activity Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing tasks that aim to identify the objects in an image are object detection and image classification, but recent work has focused on more comprehensive image under- standing tasks. |
| Approach: | They propose to incorporate commonsense knowledge about physical objects into a transformer-based model that is trained to predict the actionverb for visual activity recognition. |
| Outcome: | The proposed model incorporates prototypical function knowledge about physical objects to predict the actionverb for visual activity recognition. |
Gradient-guided Attention Map Editing: Towards Efficient Contextual Hallucination Mitigation (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) often experience “contextual hallucination” where they prioritize self-generated content over input context, leading to a disregard for pertinent details. |
| Approach: | They propose a method that dynamically adjusts attention maps to enhance contextual relevance by using a trained classifier to identify attention maps likely to induce hallucinations. |
| Outcome: | The proposed approach reduces hallucinations across open-source models on summarization and open-book QA tasks. |
Efficient k-Nearest-Neighbor Machine Translation with Dynamic Retrieval (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing models for non-parametric domain adaptation lack kNN retrieval at each timestep, leading to substantial time overhead. |
| Approach: | They propose a kNN-MT-based model that uses a domain-specific translation knowledge store to interpolate the prediction distribution of the model. |
| Outcome: | The proposed model significantly extends kNN-MT with dynamic retrieval on widely-used datasets. |
Towards Formality-Aware Neural Machine Translation by Leveraging Context Information (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Formality is one of the most important linguistic properties to determine the naturalness of translation. |
| Approach: | They propose a method to explicitly inform neural machine translation models by pinpointing key informative tokens using a formality classifier. |
| Outcome: | The proposed method improves translation quality and conforms to the appropriate syntax. |
Identifying Cognates in English-Dutch and French-Dutch by means of Orthographic Information and Cross-lingual Word Embeddings (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing methods to identify cognate pairs in English-Dutch and French-Dutsch combine orthographic information with cross-lingual word embeddings. |
| Approach: | They combine traditional orthographic information with cross-lingual word embeddings to identify cognate pairs in English-Dutch and French-Dutsch. |
| Outcome: | The proposed classifier achieves good results on the basis of orthographic information but improves by including semantic information in the form of cross-lingual word embeddings. |
Whose Preferences? Differences in Fairness Preferences and Their Impact on the Fairness of AI Utilizing Human Feedback (2024.acl-long)
Copied to clipboard
| Challenge: | a growing body of work on learning from human feedback to align various aspects of machine learning systems with human values and preferences is focusing on the setting of fairness in content moderation. |
| Approach: | They propose to use human feedback to determine how two comments should be treated in content moderation to learn about human values and preferences. |
| Outcome: | The proposed approach is promising, as human preferences can often not be A: Some ladies like smaller men. B: Some men like smaller guys. Figure 1 shows that the proposed approach performs better for demographic intersections than a single classifier that gives equal weight to each annotation. |
Multi-Label Intent Detection via Contrastive Task Specialization of Sentence Encoders (2022.emnlp-main)
Copied to clipboard
Ivan Vulić, Iñigo Casanueva, Georgios Spithourakis, Avishek Mondal, Tsung-Hsien Wen, Paweł Budzianowski
| Challenge: | a novel framework for task-oriented dialog ToD systems is proposed . a task of recognizing the user's intent or goal from their utterance is a crucial component of any TOD system. |
| Approach: | They propose to transform general-purpose sentences into task-specialized SEs by contrastive fine-tuning on annotated multi-label data. |
| Outcome: | The proposed framework yields effective mID models with large gains over non-specialized models across a spectrum of different m ID datasets. |
HebID: Detecting Social Identities in Hebrew-language Political Text (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing NLP datasets focus on coarse-grained identity categories . existing datasets are mostly English-centric and focus on fine-grain categories based on cultural contexts. |
| Approach: | They introduce the first multilabel Hebrew corpus for social identity detection . they use Hebrew-tuned encoders alongside 2B-9B-parameter decoders . |
| Outcome: | The proposed classifier is based on a national public survey and uses Hebrew-tuned encoders to analyze political discourse and political speeches. |
Detecting Erroneously Recognized Handwritten Byzantine Text (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Handwritten text recognition (HTR) produces textual output that contains errors, which are much higher than recognised printed text. |
| Approach: | They investigate the properties of handwritten texts that lead post-correction systems to this adversarial behaviour in Byzantine Greek. |
| Outcome: | The proposed model achieves an average precision score of 95% in Byzantine Greek and 97% in modern and ancient Greek. |
IDTraffickers: An Authorship Attribution Dataset to link and connect Potential Human-Trafficking Operations on Text Escort Advertisements (2023.emnlp-main)
Copied to clipboard
| Challenge: | a significant number of human trafficking cases are associated with online advertisements . identification of HT vendors is challenging for law enforcement agencies . |
| Approach: | IDTraffickers uses 87,595 text ads and 5,244 vendor labels to link HT vendors . a macro-F1 score is achieved in a closed-set classification environment . |
| Outcome: | IDTraffickers is a dataset that enables verification and identification of HT vendors . the model achieves a macro-F1 score in a closed-set classification environment . |
Bold Claims or Self-Doubt? Factuality Hallucination Type Detection via Belief State (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies focus on detecting the presence of hallucinations but lack a systematic classification approach, which hinders deeper exploration of their characteristics. |
| Approach: | They propose a method to categorize hallucinations into two types: Overconfident and Unaware . |
| Outcome: | The proposed method categorizes factuality hallucination into two types: Overconfident and Unaware Hallucinations. |
Class Name Guided Out-of-Scope Intent Classification (2024.findings-emnlp)
Copied to clipboard
Chandan Gautam, Sethupathy Parameswaran, Aditya Kane, Yuan Fang, Savitha Ramasamy, Suresh Sundaram, Sunil Sahu, Xiaoli Li
| Challenge: | SCOOS leverages semantic cues embedded in class labels to improve classification accuracy. |
| Approach: | They propose a method to create a compact feature space around class label semantics . they use a shared latent space between ID features and class names to minimize losses . |
| Outcome: | The proposed method outperforms existing methods for out-of-scope intent detection and ID intent classification. |
Weakly Supervised Cross-lingual Semantic Relation Classification via Knowledge Distillation (D19-1)
Copied to clipboard
| Challenge: | Existing work characterizes differences in meaning between words across languages using semantic relations . however, because of translation ambiguity, semantic relations are not always preserved by translation. |
| Approach: | They propose a cross-lingual relation classifier trained only with English examples and a bilingual dictionary to account for translation ambiguity when transferring knowledge from English to cross-linguistic settings. |
| Outcome: | The proposed model outperforms baselines that rely on bilingual embeddings or dictionaries for cross-lingual transfer and approaches fully supervised systems on English tasks. |
Enhancing Low-Resource LLMs Classification with PEFT and Synthetic Data (2024.lrec-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) operating in 0-shot or few-shot settings achieve competitive results in Text Classification tasks. |
| Approach: | They propose to make Large Language Models (LLMs) operating in 0-shot or few-shot settings as efficient as 0- shot text classifiers by leveraging a small number of samples. |
| Outcome: | The proposed model is able to perform better on multiple datasets than existing models on 0-shot or few-shot settings. |
Constructing a Lexicon of Relational Nouns (L18-1)
Copied to clipboard
| Challenge: | Existing systems for extracting relations expressed using nouns do not exist for relational noun. |
| Approach: | They contribute a lexicon of 6,224 labeled nouns which includes 1,446 relational noun. |
| Outcome: | The proposed classifier achieves 70.4% F1 on held out nouns among the most common 2,500 word types in Gigaword. |
Building a Knowledge Graph from Natural Language Definitions for Interpretable Text Entailment Recognition (L18-1)
Copied to clipboard
| Challenge: | a conceptual model for dictionary definitions is used to construct a knowledge graph from natural language definitions. |
| Approach: | They propose a method for automatically building a graph world knowledge base from natural language definitions. |
| Outcome: | The proposed method was used in an interpretable text entailment recognition approach. |
LexicalAT: Lexical-Based Adversarial Reinforcement Training for Robust Sentiment Classification (D19-1)
Copied to clipboard
| Challenge: | Existing text classification models are fragile and sensitive to simple perturbations. |
| Approach: | They propose a generator-classifier adversarial training approach to improve classification models . they use a large-scale lexical knowledge base to generate attacking examples . |
| Outcome: | The proposed approach outperforms strong baselines and reduces test errors on neural networks. |
Rethinking Attribute Representation and Injection for Sentiment Classification (D19-1)
Copied to clipboard
| Challenge: | Existing models that use text attributes to improve sentiment classification use text as a categorical feature. |
| Approach: | They propose to represent attributes as chunk-wise importance weight matrices and consider four locations to inject attributes. |
| Outcome: | The proposed method outperforms the state-of-the-art and outperformed previous models. |
Diversified Multiple Instance Learning for Document-Level Multi-Aspect Sentiment Classification (2020.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that D-MILN outperforms recent weakly-supervised baselines . document-level multi-aspect sentiment classification requires a lot of manual aspect-level annotations - which is time-consuming and laborious . |
| Approach: | They propose a novel Diversified Multiple Instance Learning Network to achieve DMSC with only document-level weak supervision. |
| Outcome: | The proposed method outperforms weakly-supervised baselines on TripAdvisor and BeerAdvocate datasets. |
Probing for Semantic Classes: Diagnosing the Meaning Content of Word Embeddings (P19-1)
Copied to clipboard
| Challenge: | Empirical analysis of word embeddings of ambiguous words is limited by the small size of manually annotated resources and by the fact that word senses are treated as unrelated individual concepts. |
| Approach: | They present a large dataset based on manual Wikipedia annotations and word senses, where word sense from different words are related by semantic classes. |
| Outcome: | The proposed method can predict whether a word is single-sense or multi-sensor, if the sense is frequent, and it can predict rare senses. |
Human or Neural Translation? (2020.coling-main)
Copied to clipboard
Shivendra Bhardwaj, David Alfonso Hermelo, Phillippe Langlais, Gabriel Bernier-Colborne, Cyril Goutte, Michel Simard
| Challenge: | a recent study shows that deep neural models have improved machine translation . identifying machine translation is still feasible, but is not yet known. |
| Approach: | They train and apply deep neural models to distinguish between human and machine translations . they use a monolingual and bilingual task to train and train 18 classifiers based on their results . |
| Outcome: | The proposed model improves the ability to distinguish between human and machine translations at the sentence level. |
Domain Classification-based Source-specific Term Penalization for Domain Adaptation in Hate-speech Detection (2022.coling-1)
Copied to clipboard
| Challenge: | Existing approaches for hate-speech detection exhibit poor performance in out-of-domain settings due to overemphasizing source-specific information that negatively impacts its domain invariance. |
| Approach: | They propose a domain adaptation approach that automatically extracts and penalizes source-specific terms using a classifier. |
| Outcome: | The proposed approach improves cross-domain evaluation on indomain held-out instances while preserving high performance on out-of-domain settings. |
Classifying Implant-Bearing Patients via their Medical Histories: a Pre-Study on Swedish EMRs with Semi-Supervised GanBERT (2022.lrec-1)
Copied to clipboard
Benjamin Danielsson, Marina Santini, Peter Lundberg, Yosef Al-Abasse, Arne Jonsson, Emma Eneling, Magnus Stridsman
| Challenge: | Identifying the presence of implants in certain patients is important for radiologists because some implants are not compatible with MRI scanning. |
| Approach: | They compare the performance of two BERT-based text classifiers whose task is to classify patients as having or not having implant(s) in their body. |
| Outcome: | The proposed classifiers outperform fully-supervised classifier models on annotated data. |
LBC: Language-Based-Classifier for Out-Of-Variable Generalization (2025.naacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have great success in natural language processing tasks such as response generation, but their performance on tabular data tasks has been limited due to their inferior performance compared to traditional machine learning models (TMLs). |
| Approach: | They propose a Language-Based-Classifier (LBC) that maximizes the benefits of LLMs to outperform TMLs on OOV tasks. |
| Outcome: | The proposed model outperforms TMLs on OOV tasks by using three key methods. |
Adversarial Learning of Privacy-Preserving Text Representations for De-Identification of Medical Records (P19-1)
Copied to clipboard
| Challenge: | De-identification is the task of detecting protected health information (PHI) in medical text. |
| Approach: | They propose to create shareable representations of medical text that contain no PHI and can be shared between organizations to create unified datasets for training de-identification models. |
| Outcome: | The proposed representation allows training a simple LSTM-CRF model to an F1 score of 97.4%. |
RedHerring Attack: Testing the Reliability of Attack Detection (2025.emnlp-main)
Copied to clipboard
| Challenge: | In response to adversarial text attacks, attack detection models have been proposed and shown to successfully identify text modified by adversaries. |
| Approach: | They propose to modify a text to cause the detection model to predict an attack while keeping the classifier correct. |
| Outcome: | The proposed model drops detection accuracy between 20 - 71 points while maintaining (or improving) classifier accuracy. |
One Document, Many Revisions: A Dataset for Classification and Description of Edit Intents (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing methods to understand revisions have failed to provide a deeper understanding of the nature of these edits. |
| Approach: | They propose to use a Wikipedia revision history dataset to train a classifier that achieves a 90% accuracy in identifying edit intent and a distantly-supervised model that generates . |
| Outcome: | The proposed model achieves 90% accuracy in identifying edit intent and a best score of 28 ROUGE. |
FlipDA: Effective and Robust Data Augmentation for Few-Shot Learning (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for text data augmentation are limited to simple tasks and weak baselines. |
| Approach: | They propose a data augmentation method FlipDA that uses a generative model and a classifier to generate label-flipped data. |
| Outcome: | The proposed method improves many tasks while not negatively affecting the others. |
Examining the Limitations of Computational Rumor Detection Models Trained on Static Datasets (2024.lrec-main)
Copied to clipboard
| Challenge: | Past research has indicated that content-based rumor detection models perform less effectively on unseen rumors. |
| Approach: | They propose to use data split strategies to minimize the effects of temporal concept drift in static datasets during the training of rumor detection methods. |
| Outcome: | The proposed model over-relys on the information derived from the rumors’ source post and overlooks the significant role that contextual information can play. |
Modeling What-to-ask and How-to-ask for Answer-unaware Conversational Question Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods to generate conversational question are naive and do not account for the answer span. |
| Approach: | They propose a framework for generating a conversational question from a context. |
| Outcome: | The proposed framework achieves state-of-the-art in two different settings compared to existing models . it uses a sentence as the rationale and extracts the answer span from it . |
Learning Robust Models for e-Commerce Product Search (2020.acl-main)
Copied to clipboard
| Challenge: | Existing models that understand search intent are difficult to learn due to lack of labeled datasets. |
| Approach: | They develop a deep, end-to-end model that learns to effectively classify mismatches . they introduce a latent variable into the cross-entropy loss that alternates between real and generated samples . |
| Outcome: | The proposed model achieves a relative gain of over 26% in F-score and 17% in Area Under PR curve on live search traffic in multiple countries. |
Enhancing a Lexicon of Polarity Shifters through the Supervised Classification of Shifting Directions (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing polarity shifter lexica only specify when a word can cause shifting, but do not specify when this is limited to a single shifting direction. |
| Approach: | They propose a classifier that determines the shifting direction of polarity shifters by using resource-driven features and data-driven feature. |
| Outcome: | The proposed classifier enhances the largest available polarity shifter lexicon. |
Incorporating Priors with Feature Attribution on Text Classification (P19-1)
Copied to clipboard
| Challenge: | Feature attribution methods are used to help users interpret complex models. |
| Approach: | They propose a feature attribution method that integrates feature attributed features into the objective function to allow machine learning practitioners to incorporate priors in model building. |
| Outcome: | The proposed method reduces undesired model biases without a tradeoff on the original task and improves classifier performance in scarce data setting. |
Extracting Event Temporal Relations via Hyperbolic Geometry (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent neural approaches to event temporal relation extraction map events to embeddings in the Euclidean space and train a classifier to detect temporal relations between event pairs. |
| Approach: | They propose to embed events into hyperbolic spaces to model hierarchical structures . they propose to use hyperbolical embeddings to directly infer event relations . |
| Outcome: | The proposed architecture is based on two approaches to encode events and their temporal relations in hyperbolic spaces. |
Combining Spans into Entities: A Neural Two-Stage Approach for Recognizing Discontiguous Entities (D19-1)
Copied to clipboard
| Challenge: | Named entity recognition (NER) aims at identifying shallow semantic elements in text. |
| Approach: | They propose a neural two-stage approach to recognizing discontiguous and overlapping entities by decomposing the problem into two subtasks. |
| Outcome: | The proposed model achieves state-of-the-art in a standard dataset even without external features. |
Iterative Dual-Model Alignment for Story Evaluation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing evaluators of large language models are static and lack the ability to refine their reasoning through interaction. |
| Approach: | They propose an Alpha–Beta Learning framework that trains two complementary 8B models: an Alpha classifier that assesses pairwise story engagement, and a Beta generator that produces structured, rubric-guided comparative explanations. |
| Outcome: | The proposed framework outperforms strong single-model baselines on human-annotated story-pair datasets in both accuracy and explanation quality across multiple iterative rounds. |
ClarQ: A large-scale and diverse dataset for Clarification Question Generation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing datasets hinder development of large-scale models capable of generating and utilising clarification questions. |
| Approach: | They propose a bootstrapping framework that utilises a neural network architecture to classify clarification questions based on post-comment tuples extracted from stackexchange. |
| Outcome: | The proposed framework aims to increase the accuracy of the classifier and increase recall of clarification questions by applying it to question-answering tasks. |
Contrastive Novelty-Augmented Learning: Anticipating Outliers with Large Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for classification are overly confident on unseen examples . despite recent advances in NLP, some categories of distribution shift still pose serious challenges. |
| Approach: | They propose a method that generates OOD examples representative of novel classes and trains to decrease confidence on them. |
| Outcome: | The proposed method improves classifiers' ability to detect and abstain on novel class examples over previous methods by 2.3% and 5.5% over previous approaches. |
Context-based Virtual Adversarial Training for Text Classification with Noisy Labels (2022.lrec-1)
Copied to clipboard
| Challenge: | Recent studies show that deep neural networks can memorize noisy labels with limited training time. |
| Approach: | They propose a virtual adversarial training method to prevent a classifier from overfitting to noisy labels. |
| Outcome: | The proposed method performs the adversarial training in the context rather than the inputs. |
Knowing Before Saying: LLM Representations Encode Information About Chain-of-Thought Success Before Completion (2025.findings-acl)
Copied to clipboard
| Challenge: | Using later reasoning steps does not always improve classification, suggesting LLMs encode key information early. |
| Approach: | They propose a method to predict the success of a zero-shot Chain-of-Thought process by using LLM representations that are based on initial steps representations. |
| Outcome: | The proposed method performs well even before a single token is generated, suggesting that crucial information about the reasoning process is already present in the initial steps representations. |
FACTCHECKMATE: Preemptively Detecting and Mitigating Hallucinations in LMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Language models (LMs) hallucinate. |
| Approach: | They introduce a classifier that predicts whether LMs hallucinate based on model’s hidden states before decoding begins. |
| Outcome: | The proposed model preemptively detects hallucinations by learning a classifier that predicts whether the LM will hallucinate . if a hallucinomy is detected, FactCheckmate intervenes by adjusting the model’s hidden states to produce more factual outputs. |
Rationalizing Transformer Predictions via End-To-End Differentiable Self-Training (2024.emnlp-main)
Copied to clipboard
| Challenge: | Neural networks are increasingly prevalent across a wide range of applications, driving significant advancements in fields such as natural language processing, computer vision, and beyond. |
| Approach: | They propose an end-to-end differentiable training paradigm for stable training of a rationalized transformer classifier. |
| Outcome: | The proposed model is capable of classifying a sample and scoring input tokens without any explicit supervision and produces class-wise rationales without instabilities. |
From Text to Source: Results in Detecting Large Language Model-Generated Content (2024.lrec-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) generate human-like text, but have ethical and misuse concerns. |
| Approach: | They evaluate whether a classifier trained to distinguish between source and target LLMs can detect text from an LLM without further training. |
| Outcome: | The proposed method detects text from target LLMs without further training. |
Don’t Augment, Rewrite? Assessing Abusive Language Detection with Synthetic Data (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing datasets for abusive language detection and content moderation are limited by regulatory bodies and social media platforms. |
| Approach: | They propose to replace existing datasets in English with synthetic data by rewriting original texts with an instruction-based generative model. |
| Outcome: | The proposed model improves performance in cross-dataset training. |
META: Metadata-Empowered Weak Supervision for Text Classification (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for weakly supervised text classification use text data alone to generate pseudo-labels . strong label indicators exist in metadata and it has been long overlooked due to challenges . |
| Approach: | They propose a framework that leverages metadata as an additional source of weak supervision by combining text data and metadata into a text-rich network. |
| Outcome: | The proposed framework exploits metadata as an additional source of weak supervision. |
Generating Hard-Negative Out-of-Scope Data with ChatGPT for Intent Classification (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies have not studied the performance of intent classifiers against hard-negative out-of-scope utterances. |
| Approach: | They propose to generate hard-negative OOS data using ChatGPT and evaluate them against three benchmark intent classifiers. |
| Outcome: | The proposed method improves classifiers' robustness against hard-negative out-of-scope utterances and general OOS data. |
Enriching a Lexicon of Discourse Connectives with Corpus-based Data (L18-1)
Copied to clipboard
| Challenge: | Existing annotation efforts for multiple languages have focused on discourse connectives, but we have limited it to the class of connectives marking contrast and the additional relations such connectives might convey. |
| Approach: | They enrich a lexicon of italian COnnectives with real corpus data for connectives marking contrast relations in text. |
| Outcome: | The proposed resource is a valuable tool for linguistic analyses of discourse relations and the training of a classifier for NLP applications. |
Translationese as a Language in “Multilingual” NMT (2020.acl-main)
Copied to clipboard
| Challenge: | Recent work examines the impact of translationese in machine translation evaluation using the WMT evaluation campaign. |
| Approach: | They propose to use a sentence-level classifier to distinguish translationese from original target text to generate a machine translation model that can produce more natural outputs at test time. |
| Outcome: | The proposed model produces more natural outputs at test time, yielding gains in human evaluation scores on accuracy and fluency. |
Language Identification for Austronesian Languages (2022.lrec-1)
Copied to clipboard
| Challenge: | This paper provides language identification models for low- and under-resourced languages in the Pacific region with a focus on previously unavailable Austronesian languages. |
| Approach: | They compare a classifier based on skip-gram embeddings with other methods . they then increase the number of non-Austronesian languages to 800 to evaluate their performance . |
| Outcome: | The proposed model improves on the previous methods for low- and under-resourced languages in the Pacific region. |
Discourse Context Predictability Effects in Hindi Word Order (2022.emnlp-main)
Copied to clipboard
| Challenge: | Prior work has shown that information status, dependency length, and syntactic surprisal influence word order preferences, but the role of discourse predictability is underexplored in the literature. |
| Approach: | They propose to use Hindi-Urdu Treebank corpus to build a classifier to predict which sentences actually occurred in the corpus against artificially generated distractors. |
| Outcome: | The proposed classifier predicts which sentences occur in the Hindi-Urdu Treebank corpus against artificial distractors. |
SELFOOD: Self-Supervised Out-Of-Distribution Detection via Learning to Rank (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for supervised OOD detection require expensive manual annotation of in-distribution and OOD samples. |
| Approach: | They propose a self-supervised OOD detection method that requires only in-distribution samples as supervision. |
| Outcome: | Experiments with multiple classifiers on coarse- and fine-grained datasets show the proposed method performs well in both coarse-and fine-grid settings. |
How to Do Politics with Words: Investigating Speech Acts in Parliamentary Debates (2024.lrec-main)
Copied to clipboard
| Challenge: | a new perspective on framing through the lens of speech acts investigates how politicians make use of different pragmatic speech act functions in political debates. |
| Approach: | They propose a new framework for framing through the lens of speech acts and an annotation scheme for political debates. |
| Outcome: | The proposed framework can predict speech acts with an avg. F1 of around 82.0% . the proposed framework is based on a dataset of German parliamentary debates . |
SLICER: Sliced Fine-Tuning for Low-Resource Cross-Lingual Transfer for Named Entity Recognition (2022.emnlp-main)
Copied to clipboard
| Challenge: | Large multilingual models fail to successfully transfer to low-resource languages for zero-shot cross-lingual transfer . sliced fine-tuning for named entity recognition (SLICER) forces stronger token contextualization in the Transformer. |
| Approach: | They propose a simple yet highly effective approach for improving zero-shot cross-lingual transfer for named entity recognition to low-resource languages. |
| Outcome: | The proposed approach reduces decontextualization of token representations and classifiers . it yields consistent transfer gains for low-resource languages, the authors show . |
PIEClass: Weakly-Supervised Text Classification with Prompting and Noise-Robust Iterative Ensemble Training (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for text classification use label names of target classes as the only supervision. |
| Approach: | They propose a method that uses keyword-based keyword matching to generate pseudo labels . they propose 'pieclass' module that iteratively trains classifiers and updates pseudo labels. |
| Outcome: | The proposed method achieves better performance than existing strong baselines on seven benchmark datasets and similar performance to fully-supervised classifiers on sentiment classification tasks. |
Generative Adversarial Training with Perturbed Token Detection for Model Robustness (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing adversarial training methods use discrete tokens to deceive models . current approaches use embeddings, whereas actual text-based training uses discrete text tokens. |
| Approach: | They propose a framework that integrates gradient-based learning, adversarial example generation and perturbed token detection to enhance adversariarial robustness. |
| Outcome: | The proposed framework surpasses the state-of-the-art results of ChatGPT by 10% in average accuracy. |
Investigating the Saliency of Sentiment Expressions in Aspect-Based Sentiment Analysis (2023.findings-acl)
Copied to clipboard
| Challenge: | In a set of masking experiments, we examine the extent to which the tokens identified as salient by LIME and a gradient-based method are being used by the classifier. |
| Approach: | They use a BERT BASE model to mask the sentiments of an English dataset and find that both methods produce faithful rationales. |
| Outcome: | The proposed classifier outperforms both the gradient-based and black-box saliency methods on the SemEval 2016 english dataset. |
Theory of Mind in Freely-Told Children’s Narratives: A Classification Approach (2023.findings-acl)
Copied to clipboard
| Challenge: | Language and Theory of Mind (ToM) competences are often studied with younger children and standardized tests, but as both are social competences, data and methods with higher ecological validity are critical. |
| Approach: | They leveraged a corpus of 442 freely-told stories by Dutch children aged 4-12 to study language and ToM with NLP-tools. |
| Outcome: | The proposed method is robust relative to the complexity of the task for humans and is consistent with previous studies. |
LLM as a metric critic for low resource relation identification (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies show that small language models (SLMs) overfit in low resource situations . however, the gap between pre-training and fine-tuning leads to performance decay . |
| Approach: | They propose to combine large language models and LLM for relation identification by co-evolution . they propose to use a masked language model prompt to generate a relation identification task . |
| Outcome: | The proposed model can handle low resource relation identification tasks with minimal overfitting . the proposed model provides essential background knowledge to assist training process . |
Finding the SWEET Spot: Analysis and Improvement of Adaptive Inference in Low Resource Settings (2023.acl-long)
Copied to clipboard
| Challenge: | Pre-trained Transformer-based language models such as BERT, DeBERTa, and GPT3 have become the go-to tool in NLP. |
| Approach: | They propose an Early-Exit fine-tuning method that assigns each classifier its own set of unique model weights, not updated by other classifiers. |
| Outcome: | The proposed method outperforms Early-Exit and Multi-Model at fast speeds while maintaining comparable scores to Early- Exit at slow speeds. |
MaNtLE: Model-agnostic Natural Language Explainer (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent research suggests that practitioners prefer examining language explanations that explain sub-groups of examples. |
| Approach: | They propose a model-agnostic natural language explainer that generates faithful explanations of classifier rationale for structured classification tasks. |
| Outcome: | The proposed model-agnostic natural language explainer generates faithful explanations of classifier rationale for structured classification tasks. |
GitHub Typo Corpus: A Large-Scale Multilingual Dataset of Misspellings and Grammatical Errors (2020.lrec-1)
Copied to clipboard
| Challenge: | Lack of large-scale datasets has been a major hindrance to the development of NLP tasks such as spelling correction and grammatical error correction. |
| Approach: | They propose to make GitHub Typo Corpus a multilingual dataset of misspellings and grammatical errors available for use in NLP. |
| Outcome: | The proposed dataset contains more than 350k edits and 65M characters in more than 15 languages. |
HalluMeasure: Fine-grained Hallucination Measurement Using Chain-of-Thought Reasoning (2024.emnlp-main)
Copied to clipboard
Shayan Ali Akbar, Md Mosharaf Hossain, Tess Wood, Si-Chi Chin, Erica Salinas, Victor Alvarez, Erwin Cornejo
| Challenge: | HalluMeasure is a new LLM-based hallucination detection mechanism that decomposes an LLM response into atomic claims and evaluates each claim against the provided reference context. |
| Approach: | They propose a new LLM-based hallucination detection mechanism that decomposes an LLM response into atomic claims and evaluates each atomic claim against the provided reference context. |
| Outcome: | The proposed model can detect 3 major categories of hallucinations and 10 more specific subtypes which help to identify reasons behind the hallucinian errors. |
Softmax Tree: An Accurate, Fast Classifier When the Number of Classes Is Large (2021.emnlp-main)
Copied to clipboard
| Challenge: | Classification problems with thousands or more classes occur in NLP, for example language models or document classification. |
| Approach: | a new algorithm uses a binary tree with sparse hyperplanes and small softmax classifiers at the leaves to predict the top class. |
| Outcome: | The proposed model is faster at inference because the input follows a single path to a leaf and the softmax classifier operates on a small subset of the classes. |
The Enemy from Within: A Study of Political Delegitimization Discourse in Israeli Political Speech (2025.emnlp-main)
Copied to clipboard
| Challenge: | a new computational model for political delegitimization discourse is proposed for analysis of democratic discourse . we identify the importance of PDD as a powerful tool in political competition . |
| Approach: | They propose a computational classification pipeline for political delegitimization discourse . they annotate a Hebrew-language corpus of 10,410 sentences from parliamentary speeches, facebook posts and leading news outlets . |
| Outcome: | The proposed model achieves an F1 of 0.74 for binary detection and a macro-F1 of 0.6 for classification of delegitimization characteristics. |
Language-Agnostic Bias Detection in Language Models with Bias Probing (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained language models (PLMs) contain strong social biases, which are difficult to quantify because current methods focusing on fill-the-mask objectives are sensitive to slight changes in input. |
| Approach: | They propose a bias probing technique called LABDet to evaluate social bias in pretrained language models with a language-agnostic method. |
| Outcome: | The proposed method “surfaces” nationality bias by training a classifier on top of a frozen PLM on non-nationality sentiment detection. |
Less, but Better: Efficient Multilingual Expansion for LLMs via Layer-wise Mixture-of-Experts (2025.acl-long)
Copied to clipboard
| Challenge: | Existing large language models (LLMs) have remarkable ability in high-resource languages, but their performance in multilingual scenarios is still limited. |
| Approach: | They propose a layer-wise expert allocation algorithm to determine the appropriate number of new experts for each layer. |
| Outcome: | The proposed method outperforms the previous state-of-the-art baseline with 60% fewer experts in the single-expansion setting and 33.3% fewer in the lifelong-expanding setting. |
Analyzing Transformers in Embedding Space (2023.acl-long)
Copied to clipboard
| Challenge: | Understanding Transformer-based models has attracted significant attention . a zero-pass approach is feasible for some parameters, and for two-layer attention networks . |
| Approach: | They propose a theoretical framework where parameters of a trained Transformer are interpreted by projecting them into the embedding space. |
| Outcome: | The proposed framework shows that pre-trained and fine-tuned models can be interpreted in embedding space. |
kNN-CM: A Non-parametric Inference-Phase Adaptation of Parametric Text Classifiers (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on text-discriminating properties of semi-parametric models have not been done on non-parameter models. |
| Approach: | They propose an inference-phase approach that incorporates a neighborhood search into a model to enhance the capacity of a pre-trained parametric text classifier. |
| Outcome: | The proposed model improves performance on eight SuperGLUE tasks, three adversarial natural language inference datasets, 11 question-answering (QA) datasets and two sentiment classification datasets. |
Exploring Space Efficiency in a Tree-based Linear Model for Extreme Multi-label Classification (2024.emnlp-main)
Copied to clipboard
| Challenge: | Extreme multi-label classification (XMC) aims to identify relevant subsets from numerous labels. |
| Approach: | They propose to store a tree model under the assumption of sparse data under the condition that some features may be unused when training binary classifiers in a trees method. |
| Outcome: | The proposed method can save 10% of the size of the standard one-vs-rest method for multi-label classification. |
Mapping the Past: Geographically Linking an Early 20th Century Swedish Encyclopedia with Wikidata (2024.lrec-main)
Copied to clipboard
| Challenge: | The Nordisk Familjebok encyclopedia is one of the most extensive encyclopepedias in the world. |
| Approach: | They propose to extract location entries from the Nordisk Familjebok encyclopedia from the early 20th century, focusing on the second edition called Uggleupplagan. |
| Outcome: | The extracted locations were found to be located within Sweden, Germany, and the United Kingdom. |
Beyond the Score: Uncertainty-Calibrated LLMs for Automated Essay Assessment (2025.emnlp-main)
Copied to clipboard
| Challenge: | Automated Essay Scoring (AES) systems attain near–human agreement on some public benchmarks, but real-world adoption is limited. |
| Approach: | They propose a distribution-free wrapper that equips any classifier with set-valued outputs enjoying formal coverage guarantees. |
| Outcome: | The proposed model achieves coverage targets while keeping prediction sets compact. |
How Can Synthetic Data Improve Multilingual Language Model Pretraining? A Data Quality Perspective (2026.acl-long)
Copied to clipboard
| Challenge: | Low-resource languages are a long-tail problem for multilingual LLMs due to limited high-quality training data. |
| Approach: | They propose a method that translates high-quality, knowledge-rich English data into low-resource languages . they propose SynRank, which leverages synthetic data as positive samples to train a classifier . |
| Outcome: | The proposed method matches handcrafted rule-based filtering by human experts and significantly improves knowledge-intensive tasks with less data. |
Euphemistic Abuse – A New Dataset and Classification Experiments for Implicitly Abusive Language (2023.emnlp-main)
Copied to clipboard
| Challenge: | Currently, only explicit abuse can be reliably detected due to the increasing amount of abusive language on the Web. |
| Approach: | They propose a crowdsourced dataset that can detect euphemistic abuse by paraphrasing simple explicit utterances. |
| Outcome: | The proposed classifier augments training data with automatically-generated GPT-3 completions. |
Debiasing Online Preference Learning via Preference Feature Preservation (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent preference learning frameworks for large language models (LLMs) simplify human preferences with binary pairwise comparisons and scalar rewards. |
| Approach: | They propose a preference feature preservation framework that preserves the distribution of human preference features and maps them throughout the online preference learning process. |
| Outcome: | The proposed framework maintains the distribution of human preference features and utilizing such rich signals throughout the online preference learning process. |
SelectLLM: Query-Aware Efficient Selection Algorithm for Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing large language models struggle with complex tasks such as factually-grounded reasoning and planning due to inherent training biases, model size constraints, and the quality or diversity of pre-training datasets. |
| Approach: | They propose a novel algorithm to select the most suitable LLMs from a large pool and use it to efficiently generalize and perform tasks. |
| Outcome: | The proposed model outperforms existing ensemble-based baselines and achieves competitive performance with similarly sized top-performing LLMs while maintaining efficiency. |
Matina: A Culturally-Aligned Persian Language Model Using Multiple LoRA Experts (2025.findings-acl)
Copied to clipboard
Sara Bourbour Hosseinbeigi, MohammadAli SeifKashani, Javad Seraj, Fatemeh Taherinezhad, Ali Nafisi, Fatemeh Nadi, Iman Barati, Hosein Hasani, Mostafa Amiri, Mostafa Masoudi
| Challenge: | Existing Large language models fail to accurately model underrepresented languages and cultures, limiting their applicability and acceptance. |
| Approach: | They develop a Persian-focused multi-expert model that incorporates Iranian cultural values and linguistic structures. |
| Outcome: | The proposed model outperforms baseline models in task performance and user satisfaction. |
A Classifier of Word-Level Variants in Witnesses of Biblical Hebrew Manuscripts (2025.findings-acl)
Copied to clipboard
| Challenge: | a strong classifier (F1 value of 0.80) is trained to predict the category of difference between word pairs as present in collated (aligned) pairs of witnesses. |
| Approach: | The project is based on the relationship between available witnesses of biblical Hebrew and a strong classifier (F1 value of 0.80) is trained to predict the category of difference between word pairs as present in collated pairs of witnesses. |
| Outcome: | The proposed model is non-neural and uses part-of-speech tags, hand-crafted rules per category and synthetically derived data. |
Parsing Headed Constituencies (2024.lrec-main)
Copied to clipboard
| Challenge: | Using constituency and dependency trees, syntactic representations are preferred for tasks such as nominal phrase extraction and identification of terminology. |
| Approach: | They propose a parsing technique that generates headed constituency trees which combine information typically contained in constituency and dependency trees. |
| Outcome: | The proposed method generates headed constituency trees with discontinuities and can generate constituency tree with discontinuity. |
FREE: Fast and Robust Vision Language Models with Early Exits (2025.findings-acl)
Copied to clipboard
| Challenge: | Vision-Language Models (VLMs) have shown remarkable performance improvements in Vision-language tasks, but their large size poses challenges for real-world applications. |
| Approach: | They propose an adversarial approach to train exit classifiers in Vision-Language Models that uses a transformer layer and a classifier to perform input-adaptive inference. |
| Outcome: | The proposed approach speeds up inference speed with minimal drop in performance by 1.51 while retaining comparable performance. |
What Do Transformers Know about Government? (2024.lrec-main)
Copied to clipboard
Jue Hou, Anisia Katinskaia, Lari Kotilainen, Sathianpong Trangcasanchai, Anh-Duc Vu, Roman Yangarber
| Challenge: | Currently, data is lacking for the research community working on grammatical constructions, and government in particular. |
| Approach: | They use transformer language models to study how government relations are encoded . they use morphologically rich languages to train a classifier capable of discovering new types of government . |
| Outcome: | The proposed classifiers can learn new types of government, the authors show . they find that the classifier can learn government relations in two languages . |
XVD: Cross-Vocabulary Differentiable Training for Generative Adversarial Attacks (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing approaches to create adversarial examples using tokens are not sufficient to ensure other desirable properties such as similarity to non-adversarial examples, linguistic fluency, and so forth. |
| Approach: | They propose a method which leverages a set of pretrained language models to promote similarity to non-adversarial examples, linguistic fluency, and so forth. |
| Outcome: | The proposed approach outperforms existing methods and is competitive with token-based approaches. |
microCLIP: Unsupervised CLIP Adaptation via Coarse-Fine Token Fusion for Fine-Grained Image Classification (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing UA methods for fine-grained image classification rely on coarse-grain visual tokens, which misses fine spatial details. |
| Approach: | They propose a label-free self-training framework that adapts visual features and LLMderived text prototypes using fine-grained cues. |
| Outcome: | The proposed framework improves alignment between finegrained visual regions and rich textual descriptions while updating only layer norms and a tiny head. |
AI Argues Differently: Distinct Argumentative and Linguistic Patterns of LLMs in Persuasive Contexts (2025.emnlp-main)
Copied to clipboard
| Challenge: | Distinguishing LLM-generated text from human-written is a key challenge for safe and ethical NLP, especially in high-stake settings such as persuasive online discourse. |
| Approach: | They propose to use general-purpose linguistic features and domain-specific features related to argument quality to compare human- and LLM-authored arguments. |
| Outcome: | The proposed framework compares arguments by humans and three LLMs using two easily-interpretable feature sets. |