A Mutual Information Maximization Approach for the Spurious Solution Problem in Weakly Supervised Question Answering (2021.acl-long)
Copied to clipboard
| Challenge: | Weakly supervised question answering usually has only final answers as supervision signals while correct solutions are not provided. |
| Approach: | They propose to explicitly exploit the semantic correlations between question-answer pairs and predicted answers by maximizing mutual information between question and answer pairs. |
| Outcome: | The proposed method significantly outperforms previous learning methods in terms of task performance and is more effective in training models to produce correct solutions. |
Similar Papers
Enforcing Consistency in Weakly Supervised Semantic Parsing (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods for training semantic parsers from only (utterance, denotation) supervision are challenging. |
| Approach: | They propose to use consistency between output programs for related inputs to reduce the impact of spurious programs. |
| Outcome: | The proposed formalisms improve model performance even without consistency-based training. |
QAInfomax: Learning Robust Question Answering System by Mutual Information Maximization (D19-1)
Copied to clipboard
| Challenge: | Existing models are not good at distinguishing distractor sentences which look related but do not answer the question. |
| Approach: | They propose a method to regularize question answering models by maximizing mutual information among passages, questions, and answers. |
| Outcome: | The proposed model achieves state-of-the-art on the Adversarial-SQuAD dataset. |
Reducing Spurious Correlations for Answer Selection by Feature Decorrelation and Language Debiasing (2022.coling-1)
Copied to clipboard
| Challenge: | Existing deep neural models rely on spurious correlations between prediction labels and input features, which in general suffer from robustness and generalization. |
| Approach: | They propose a feature decorrelation module to remove feature dependencies and reduce spurious correlations by learning a weight for each instance at the training phase. |
| Outcome: | The proposed method improves the robustness of the neural ANswer selection models from the sample and feature perspectives. |
A Discrete Hard EM Approach for Weakly Supervised Question Answering (D19-1)
Copied to clipboard
| Challenge: | Existing work on question answering tasks only provide weak supervision for how the answer should be computed . weak supervision is attractive because it is relatively easy to gather, allowing for large datasets . but weak supervision complicates learning because there are many different spurious ways to derive the correct answer. |
| Approach: | They propose a method to convert question answering tasks into discrete latent variable learning problems with a precomputed set of possible solutions that contains one correct option. |
| Outcome: | The proposed approach outperforms previous methods on six QA tasks and achieves state-of-the-art on five of them. |
Iterative Search for Weakly Supervised Semantic Parsing (N19-1)
Copied to clipboard
| Challenge: | Recent work has focused on training semantic parsers via weak supervision from denotations alone. |
| Approach: | They propose an iterative training algorithm that alternates between searching for consistent logical forms and maximizing the marginal likelihood of the retrieved ones. |
| Outcome: | The proposed algorithm outperforms the previous best systems on WikiTableQuestions and Cornell Natural Language Visual Reasoning (NLVR) iteratively train models that provide guidance to subsequent models to search for logical forms of increasing complexity, thus dealing with spuriousness. |
An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models (2020.tacl-1)
Copied to clipboard
| Challenge: | Recent work shows that pre-trained language models perform poorly on challenging datasets where spurious correlations do not hold. |
| Approach: | They propose to use multi-task learning to improve generalization from minority examples . they propose to combine MTL with auxiliary tasks to improve performance . |
| Outcome: | The proposed model generalizes from minority examples without hurting in-distribution performance. |
Pointwise Mutual Information as a Performance Gauge for Retrieval-Augmented Generation (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods to improve language models' performance do not exploit this phenomenon . |
| Approach: | They propose to use contextual information to select and construct prompts that improve model performance. |
| Outcome: | The proposed methods show that the mutual information between a context and a question is an effective gauge for language model performance. |
Identifying and Mitigating Spurious Correlations for Improving Robustness in NLP Models (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing work identifies task-specific shortcuts via human priors or error analyses, which requires extensive expertise and efforts. |
| Approach: | They propose to automatically identify spurious correlations in NLP models at scale by using existing interpretability methods to extract tokens that significantly affect model’s decision process. |
| Outcome: | The proposed method can identify spurious correlations in NLP models at scale and mitigate these leads to more robust models in multiple applications. |
Weakly Supervised Semantic Parsing with Execution-based Spurious Program Filtering (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to train a semantic parser from weak supervision focus on exploiting similarities between examples based on domain-specific knowledge. |
| Approach: | They propose a domain-agnostic filtering mechanism based on program execution results to identify and filter out programs with significantly different semantics from the other programs. |
| Outcome: | The proposed method improves the performance of existing weakly-supervised parsers by incorporating a majority vote on the program search results. |
Easy to Decide, Hard to Agree: Reducing Disagreements Between Saliency Methods (2023.findings-acl)
Copied to clipboard
| Challenge: | A popular approach to unveiling the black box of neural NLP models is to leverage saliency methods, which assign scalar importance scores to each input component. |
| Approach: | They propose to use saliency methods to evaluate whether an explanation is faithful and argue that Pearson-r is a better-suited alternative to rank correlation. |
| Outcome: | The proposed methods exhibit weak rank correlations even when applied to the same model instance and advocated for alternative diagnostic methods. |