Papers with i.i.d
Practical Obstacles to Deploying Active Learning (D19-1)
Copied to clipboard
| Challenge: | Active learning (AL) is a widely-used training strategy for maximizing predictive performance subject to a fixed annotation budget. |
| Approach: | They propose to use active learning to optimize predictive performance . they find that current approaches do not generalize reliably across models and tasks . |
| Outcome: | The proposed approach outperforms training on i.i.d. datasets on supervised learning tasks. |
SUBS: Subtree Substitution for Compositional Semantic Parsing (2022.naacl-main)
Copied to clipboard
| Challenge: | Semantic parsing models fail at compositional generalization due to lack of reasoning ability. |
| Approach: | They propose to use subtree substitution for compositional data augmentation to increase the number of subtreas with similar semantic functions as exchangeable. |
| Outcome: | The proposed method improves performance on Scan and GeoQuery, and new SOTA on compositional split of GeoQuery. |
How to Enhance Causal Discrimination of Utterances: A Case on Affective Reasoning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing models excel at capturing semantic correlations within utterance embeddings but fail to determine specific causal relationships. |
| Approach: | They propose to incorporate i.i.d. noise terms into conversation process to build a structural causal model . they propose to use unstructured conversation data to facilitate deep learning . |
| Outcome: | The proposed approach can be implemented in unstructured conversation data and a synthetic dataset that includes i.i.d. noise. |
On Graph-based Reentrancy-free Semantic Parsing (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing graph-based approaches for semantic parsing fail on compositional generalization tasks. |
| Approach: | They propose a graph-based approach for semantic parsing that solves two problems . they propose two algorithms based on constraint smoothing and conditional gradient to approximate these problems. |
| Outcome: | The proposed graph-based approach delivers state-of-the-art results on GeoQuery, Scan, and Clevr . |
Cross-functional Analysis of Generalization in Behavioral Learning (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing evaluation paradigms for behavioral learning use correlations in training data, but they ignore important model properties such as fairness. |
| Approach: | They propose an analysis method for evaluating behavioral learning considering generalization across dimensions of different granularity levels. |
| Outcome: | The proposed method optimizes behavior-specific loss functions and evaluates models on several partitions of the behavioral test suite controlled to leave out specific phenomena. |
MUTANT: A Training Paradigm for Out-of-Distribution Generalization in Visual Question Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | Availability of large-scale datasets has enabled statistical machine learning in vision and language understanding. |
| Approach: | They propose a training paradigm that exposes models to perceptually similar mutations of input . they show a 10.57% improvement in the VQA-CP challenge . |
| Outcome: | The proposed training paradigm improves on the visual question answering challenge with 10.57% accuracy. |
Towards Generalizable and Robust Text-to-SQL Parsing (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Text-to-SQL parsers must be generalizable and robust against input perturbations. |
| Approach: | They propose a novel framework to learn text-to-SQL parsing in stages to improve parser's ability to acquire general SQL knowledge instead of capturing spurious patterns. |
| Outcome: | The proposed framework achieves state-of-the-art performance on the Spider, SParC, and CoSQL datasets. |
Leveraging Code to Improve In-Context Learning for Semantic Parsing (2024.naacl-long)
Copied to clipboard
| Challenge: | In-context learning is an attractive approach for semantic parsing, but learning to parse to rare domain-specific languages from a few demonstrations is challenging. |
| Approach: | They propose to use Python instead of DSLs to augment prompts with a structured domain description. |
| Outcome: | The proposed approach improves accuracy and generalization across three datasets. |
Deep Bayesian Active Learning for Natural Language Processing: Results of a Large-Scale Empirical Study (D18-1)
Copied to clipboard
| Challenge: | Existing studies on Active Learning (AL) for natural language processing have limited data requirements. |
| Approach: | They propose a Bayesian active learning approach that reduces deep learning's data dependence by comparing models and acquisition functions. |
| Outcome: | The proposed approach outperforms i.i.d. baselines and is more efficient than other approaches. |
GraphQ IR: Unifying the Semantic Parsing of Graph Query Languages with One Intermediate Representation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to neural semantic parsing are limited by the semantic gap between natural and formal languages. |
| Approach: | They propose a unified intermediate representation for graph query languages, named GraphQ IR, which has a natural-language-like expression that bridges the semantic gap and formally defined syntax that maintains the graph structure. |
| Outcome: | The proposed representation can convert user queries into graphQ IR, which can later be losslessly compiled into various downstream graph query languages. |
RNG-KBQA: Generation Augmented Iterative Ranking for Knowledge Base Question Answering (2022.acl-long)
Copied to clipboard
| Challenge: | Existing KBQA approaches struggle with generalization of unseen KB schema items . Rank-and-generate approach solves coverage issue with strong generalization . |
| Approach: | They propose a Rank-and-Generate approach for KBQA that uses a generation model to generalize to unseen KB schema items. |
| Outcome: | The proposed approach outperforms the prior state-of-the-art on GrailQA and WebQSP datasets. |
LLM-Evolve: Evaluation for LLM’s Evolving Capability on Benchmarks (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing benchmarks for large language models evaluate LLMs on i.i.d. tasks, overlooking their ability to learn iteratively from past experiences. |
| Approach: | They propose a framework which extends established benchmarks to sequential problem-solving settings and provides feedback after each round to build a demonstration memory that the models can query in future tasks. |
| Outcome: | The proposed framework can improve performance of LLMs by learning from past interactions and improve models' performance over time. |
Latent Attention Denoising: A Training-Free Energy-Based Framework for Mitigating Hallucinations in Vision-Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing models for large vision-Language Models lack reliable visual hallucination . despite advances in multimodal perception, they are still limited in real-world applications . |
| Approach: | They propose a framework that recasts attention calibration as a one-step score-based denoising process. |
| Outcome: | The proposed framework achieves superior performance on generative and discriminative tasks while maintaining efficiency comparable to standard decoding. |