Papers with DNNs
SyGNS: A Systematic Generalization Testbed Based on Natural Language Semantics (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing models have limitations to generalize to diverse semantic phenomena, and it is unclear whether they can capture compositional meanings. |
| Approach: | They propose a systematic generalization testbed based on Natural language semantics to map natural language sentences to multiple meaning representations. |
| Outcome: | The proposed model can generalize to unseen combinations of quantifiers, negations, and modifiers, but not to the others. |
On the Calibration and Uncertainty of Neural Learning to Rank Models for Conversational Search (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing methods to rank documents in decreasing order of their probability of relevance are not well calibrated and have several sources of uncertainty. |
| Approach: | They propose to calibrate deterministic neural rankers for conversational search problems . they then use two techniques to model the uncertainty of neural ranker's uncertainty . |
| Outcome: | The proposed rankers output a predictive distribution of relevance as opposed to point estimates. |
Fine-mixing: Mitigating Backdoors in Fine-tuned Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for defending NLP models against backdoors have ignored the clean weights of PLMs. |
| Approach: | They exploit pre-trained weights to mitigate backdoors in fine-tuned NLP models . they use a fine-mixing technique and an Embedding Purification technique to do the same . |
| Outcome: | The proposed method outperforms baseline mitigation methods on three single-sentence sentiment classification tasks and two sentence-pair classification tasks. |
Evaluating neural network explanation methods using hybrid documents and morphosyntactic agreement (P18-1)
Copied to clipboard
| Challenge: | a number of post hoc explanation methods for deep neural networks have been proposed . due to the complexity of the DNNs they explain, these methods are necessarily approximations and come with their own sources of error. |
| Approach: | They propose two evaluation paradigms that cover two important classes of NLP problems . they propose LIMSSE, LRP and DeepLIFT as the most effective explanation methods . |
| Outcome: | The proposed methods are most effective for explaining deep neural networks in NLP . the proposed methods can explain complex models without manual annotation . |
“Yes, My LoRD.” Guiding Language Model Extraction with Locality Reinforced Distillation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for model extraction attacks on large language models are inadequate . existing methods neglect the inconsistency between training tasks and LLM alignment . |
| Approach: | They propose a model extraction algorithm that uses a policy-gradient-style training task to guide the crafting of preference for the local model. |
| Outcome: | The proposed algorithm reduces query complexity while mitigating watermark protection . it can extract various state-of-the-art commercial LLMs while minimizing query complexity . |
Meta Self-Refinement for Robust Learning with Weak Supervision (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent methods leverage self-training to build noise-resistant models . however, the teacher trained under weak supervision may have fitted a substantial amount of noise and therefore produce incorrect pseudo-labels. |
| Approach: | They propose a framework that encourages teacher to refine its pseudo-labels to effectively combat label noise from weak supervision. |
| Outcome: | The proposed framework outperforms state-of-the-art methods by 11.4% in accuracy and 9.26% in F1 score on eight NLP benchmarks. |
Domain Adaptation with Adversarial Training and Graph Embeddings (P18-1)
Copied to clipboard
| Challenge: | Existing models for deep neural networks can handle data distributions between source and target domains, but they must deal with data distribution drifts. |
| Approach: | They propose a model that leverages unlabeled and labeled data from a related domain to deal with distribution drifts. |
| Outcome: | The proposed model improves over baselines on two real-world disaster datasets. |
A Deep Generative Distance-Based Classifier for Out-of-Domain Detection with Mahalanobis Space (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for detecting out-of-domain (OOD) intents rely on manually labeled samples . a strong generative distance-based classifier can detect OOD samples in task-oriented dialog systems . |
| Approach: | They propose a generative distance-based classifier to detect out-of-domain (OOD) intents . they use Gaussian discriminant analysis to avoid over-confidence problems . |
| Outcome: | The proposed method outperforms baseline methods on four benchmark datasets. |
Characterizing the Impacts of Instances on Robustness (2023.findings-acl)
Copied to clipboard
Rui Zheng, Zhiheng Xi, Qin Liu, Wenbin Lai, Tao Gui, Qi Zhang, Xuanjing Huang, Jin Ma, Ying Shan, Weifeng Ge
| Challenge: | Existing defense approaches focus on developing new model structures or training algorithms, but they do little to tap the potential of training instances. |
| Approach: | They propose a method that can distinguish between robust and non-robust instances according to the model’s sensitivity to perturbations on individual instances during training. |
| Outcome: | The proposed method can distinguish between robust and non-robust instances according to the model’s sensitivity to perturbations on individual instances during training. |
Towards an argumentative content search engine using weak supervision (C18-1)
Copied to clipboard
| Challenge: | Existing work focused on detecting claims within a small set of documents . however, pinpointing relevant claims within massive unstructured corpora, received little attention. |
| Approach: | They propose to use a weak signal to develop a query for claim–sentence detection using a large text corpus. |
| Outcome: | The proposed system outperforms previous results in terms of precision and coverage. |
Measuring and Mitigating Local Instability in Deep Neural Networks (2023.findings-acl)
Copied to clipboard
| Challenge: | Uncertain details like random initialization can change the outputs of a trained system with potentially disastrous consequences. |
| Approach: | They propose a model stability problem by studying how the predictions of a deep neural network change as a consequence of stochasticity in the training process. |
| Outcome: | The proposed method outperforms data-agnostic methods and is 90% cheaper than the gold standard. |
Deep Dominance - How to Properly Compare Deep Neural Models (P19-1)
Copied to clipboard
| Challenge: | Existing methods for comparing DNNs on unseen data are not suitable for this task. |
| Approach: | They propose to adapt a test for the Almost Stochastic Dominance relation between two distributions to the problem by comparing their performance on unseen data. |
| Outcome: | The proposed method meets all criteria while previously proposed methods fail to do so. |
Asymmetric feature interaction for interpreting model predictions (2023.findings-acl)
Copied to clipboard
| Challenge: | Prior work on feature interaction attribution studies focus on asymmetric interaction that only explains the additional influence of a set of words in combination, which fails to capture asymmetry influence that contributes to model prediction. |
| Approach: | They propose an asymmetric feature interaction attribution explanation model that explores asymmetry higher-order feature interactions in the inference of deep neural NLP models. |
| Outcome: | The proposed model outperforms state-of-the-art models on two sentiment classification datasets. |
Generating Natural Language Adversarial Examples (D18-1)
Copied to clipboard
| Challenge: | Recent research has shown that deep neural networks are vulnerable to adversarial examples, perturbations to correctly classified examples which can cause the model to misclassify. |
| Approach: | They propose to generate adversarial examples that fool well-trained sentiment analysis and textual entailment models by using a black-box population-based optimization algorithm. |
| Outcome: | The proposed model is able to fool well-trained sentiment analysis and textual entailment models with success rates of 97% and 70%, respectively. |
BadActs: A Universal Backdoor Defense in the Activation Space (2024.findings-acl)
Copied to clipboard
| Challenge: | Backdoor attacks pose an increasingly severe security threat to Deep Neural Networks . existing methods focused on the word space are ineffective against feature-space triggers - a recent study has shown . |
| Approach: | They propose a backdoor defense that purifies backdoor samples in the activation space . they aim to eliminate backdoor triggers while preserving the integrity of clean data . |
| Outcome: | The proposed method achieves state-of-the-art against backdoor attacks on clean data. |
CapEEN: Image Captioning with Early Exits and Knowledge Distillation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Early Exit (EE) strategies can be used to enhance their efficiency, but their adaptation presents challenges in image captioning as it requires varying levels of semantic information for accurate predictions. |
| Approach: | They propose a framework to improve the performance of EE strategies by knowledge distillation . they use a variant A-CapEEN to adapt thresholds on the fly to account for drifts . |
| Outcome: | The proposed framework gains speedup of 1.77 while maintaining competitive performance compared to the final layer. |
DROWN: Towards Tighter LiRPA-based Robustness Certification (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for certifying the robustness of deep neural networks suffer from precision or scalability issues. |
| Approach: | They propose a method to certify the robustness of deep neural networks . they propose to use two pairs of linear bounds to refine pre-activation bounds . |
| Outcome: | The proposed method achieves higher certified robustness than the baseline on CNNs and 4.68 times larger certified radii than the Transformers. |
“Low-Resource” Text Classification: A Parameter-Free Classification Method with Compressors (2023.findings-acl)
Copied to clipboard
| Challenge: | Text classification is one of the most fundamental tasks in natural language processing (NLP), but deep neural networks are data-hungry and expensive to train. |
| Approach: | They propose a non-parametric alternative to DNNs that uses a compressor like gzip and a k-nearest-neighbor classifier to achieve competitive results. |
| Outcome: | The proposed method outperforms BERT on all five OOD datasets and outperformed other methods on the few-shot setting. |
Learning Latent Parameters without Human Response Patterns: Item Response Theory with Artificial Crowds (D19-1)
Copied to clipboard
| Challenge: | Incorporating Item Response Theory (IRT) into NLP tasks can provide valuable information about model performance and behavior. |
| Approach: | They propose to use IRT models generated from artificial crowds of DNNs to learn IRT. |
| Outcome: | The proposed model learning method outperforms baseline methods for two NLP tasks. |
Joint Multitask Learning for Community Question Answering Using Task-Specific Embeddings (D18-1)
Copied to clipboard
| Challenge: | Stack-Overflow, Quora, and Yahoo! Answers forums are not moderated, which results in noisy and redundant content. |
| Approach: | They use deep neural networks to learn meaningful task-specific embeddings . they incorporate the embeddables into a conditional random field model . |
| Outcome: | The proposed task improves significantly across evaluation metrics. |
Explain Yourself! Leveraging Language Models for Commonsense Reasoning (P19-1)
Copied to clipboard
| Challenge: | Empirical results indicate that we can effectively leverage language models for commonsense reasoning. |
| Approach: | They propose to use commonsense auto-generated explanations to train language models to generate explanations that can be used during training and inference in a commonsensense Auto-Generated Explanation framework. |
| Outcome: | Empirical results show that the proposed framework improves on the commonsenseQA task by 10%. |
Understanding Deep Learning Performance through an Examination of Test Set Difficulty: A Psychometric Case Study (D18-1)
Copied to clipboard
| Challenge: | Existing methods to evaluate deep learning models that are not considered for test set accuracy are difficult to interpret. |
| Approach: | They examine the impact of a test set question’s difficulty to determine if there is a relationship between difficulty and performance. |
| Outcome: | The proposed model can learn examples of varying difficulty at different rates if it does well on hard examples and poor on easy items because a dataset is all easy, but has "solved" anything? |
Creating New Language and Voice Components for the Updated MaryTTS Text-to-Speech Synthesis Platform (L18-1)
Copied to clipboard
| Challenge: | a reboot of the MaryTTS system became unavoidable due to the number of people who have contributed to its development over the years. |
| Approach: | They propose a workflow to create components for the MaryTTS text-to-speech synthesis platform. |
| Outcome: | The proposed workflow is compatible with the updated MaryTTS architecture, enabling new features and state-of-the-art paradigms such as synthesis based on deep neural networks (DNNs). |
Neural Automated Essay Scoring Incorporating Handcrafted Features (2020.coling-main)
Copied to clipboard
| Challenge: | Automated essay scoring (AES) relies on handcrafted features, but recent studies have proposed a hybrid method that integrates handcrafted essay-level features into a DNN-AES model. |
| Approach: | They propose a method that integrates handcrafted features into a DNN-AES model. |
| Outcome: | The proposed method significantly improves the accuracy of existing methods. |
Local Interpretation of Transformer Based on Linear Decomposition (2023.acl-long)
Copied to clipboard
| Challenge: | Existing work on local explanation generation attempts to understand model dynamics on word-level or phraselevel by assigning importance scores on input features. |
| Approach: | They propose to interpret neural networks by linear decomposition by a Transformer model on a single input and a linear decomposing of the output to generate local explanations. |
| Outcome: | The proposed method achieves competitive performance in sentiment classification and machine translation, and fidelity of explanation. |
A Geometry-Inspired Attack for Generating Natural Language Adversarial Examples (2020.coling-main)
Copied to clipboard
| Challenge: | Existing techniques to generate adversarial examples for natural language are limited . previous research on adversarials focused on images, but this is not the case with natural language. |
| Approach: | They propose a geometry-inspired attack for generating natural language adversarial examples . they use deep neural networks to iteratively approximate the decision boundary . |
| Outcome: | The proposed attack fools natural language models with high success rates while replacing a few words. |
Learning with Noisy Labels for Sentence-level Sentiment Classification (D19-1)
Copied to clipboard
| Challenge: | Existing research on learning with noisy labels dates back to the 1980s, but it is still vibrant today. |
| Approach: | They propose a novel DNN model called NetAb to deal with noisy labels during training and train the networks using their respective loss functions in mutual reinforcement. |
| Outcome: | The proposed model can fit training data with noisy labels and predict clean labels. |
RAP: Robustness-Aware Perturbations for Defending against Backdoor Attacks on NLP Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Backdoor attacks are a serious threat to the safety of reusing deep neural networks (DNNs). |
| Approach: | They propose an efficient online defense mechanism based on robustness-aware perturbations to distinguish poisoned and clean samples to defend against backdoor attacks on natural language processing models. |
| Outcome: | The proposed method achieves better defending performance and lower computational costs than existing defense methods. |
Context-based Virtual Adversarial Training for Text Classification with Noisy Labels (2022.lrec-1)
Copied to clipboard
| Challenge: | Recent studies show that deep neural networks can memorize noisy labels with limited training time. |
| Approach: | They propose a virtual adversarial training method to prevent a classifier from overfitting to noisy labels. |
| Outcome: | The proposed method performs the adversarial training in the context rather than the inputs. |
Detecting Adversarial Samples through Sharpness of Loss Landscape (2023.findings-acl)
Copied to clipboard
Rui Zheng, Shihan Dou, Yuhao Zhou, Qin Liu, Tao Gui, Qi Zhang, Zhongyu Wei, Xuanjing Huang, Menghan Zhang
| Challenge: | Existing studies have shown that adversarial samples are more vulnerable than normal ones to textual adversarials. |
| Approach: | They propose a simple and effective sharpness-based detector that can distinguish adversarial samples by maximizing the loss increment within the region where the inference sample is located. |
| Outcome: | The proposed method outperforms previous detection methods by large margins on three text classification tasks. |
ONION: A Simple and Effective Defense Against Textual Backdoor Attacks (2021.emnlp-main)
Copied to clipboard
| Challenge: | Backdoor attacks can manipulate the output of deep neural networks and possess high insidiousness. |
| Approach: | They propose a textual backdoor defense based on outlier word detection that can handle all the textual attacks. |
| Outcome: | The proposed method can handle all the textual backdoor attack situations. |
Connectivity Patterns are Task Embeddings (2023.findings-acl)
Copied to clipboard
Zhiheng Xi, Rui Zheng, Yuansen Zhang, Xuanjing Huang, Zhongyu Wei, Minlong Peng, Mingming Sun, Qi Zhang, Tao Gui
| Challenge: | Existing methods for predicting inter-task transferability are sparse and task-specific. |
| Approach: | They propose a method that uses connectivity patterns of neurons as a unique identifier associated with a task. |
| Outcome: | The proposed method outperforms baselines in predicting inter-task transferability across data regimes and transfer settings while keeping high efficiency in computation and storage. |
Are Data Augmentation Methods in Named Entity Recognition Applicable for Uncertainty Estimation? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a key task in NLP to find mentions of named entities and classify them into predefined categories. |
| Approach: | They investigated the impact of data augmentation on confidence calibration and uncertainty estimation in Named Entity Recognition (NER) tasks. |
| Outcome: | The data augmentation improves calibration and uncertainty in cross-genre and cross-lingual setting, especially in-domain setting. |
Selective Knowledge Distillation: Fusing LLM Semantic Strengths with DNN Efficiency for Binary Code Similarity Detection (2026.acl-long)
Copied to clipboard
| Challenge: | BinSKD is a binary code similarity detection technique that can be used in bug detection, patch analysis, and malware detection. |
| Approach: | They propose to leverage an LLM-based BCSD method as the teacher model and transfer its knowledge of high-level program semantics to various DNN-based student models. |
| Outcome: | The proposed method yields Recall@1 improvements of 14.5%–91.2% for DNN-based BCSD methods and enables HermesSim to match the teacher’s performance with orders-of-magnitude efficiency. |
Subspace Defense: Discarding Adversarial Perturbations by Learning a Subspace for Clean Signals (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing models that extract discrete inputs into fixed-length representations are vulnerable to adversarial attacks that place perturbations on clean inputs to fool DNNs. |
| Approach: | They propose to inspect the subspaces of sample features through spectral analysis to better understand adversarial attacks. |
| Outcome: | The proposed strategy enables the model to inherently suppress adversaries, which boosts model robustness and motivates new directions of effective adversarial defense. |