| Challenge: | Using uncertainty and diversity sampling, active learning acquisition functions select difficult and diverse data points from a pool of unlabeled data. |
| Approach: | They propose an active learning acquisition function that selects contrastive examples from unlabeled data. |
| Outcome: | The proposed approach performs better or equal to the best performing baseline on all tasks, on both in-domain and out-of-domain data. |
Similar Papers
Active Learning Principles for In-Context Learning with Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | In-context learning has significantly enhanced predictive performance in few-shot learning settings. |
| Approach: | They propose to use pool-based Active Learning to identify the most informative demonstrations for few-shot learning over a single iteration to identify best demonstrations. |
| Outcome: | The proposed model outperforms all other methods, including random sampling, in the analysis of 24 classification and multi-choice tasks. |
Investigating Multi-source Active Learning for Natural Language Inference (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent studies often assume that training and test data are drawn from the same distribution. |
| Approach: | They propose to apply active learning to unlabelled data pools to test for learning and generalisation. |
| Outcome: | The proposed strategies outperform random selection and outperformed hard-to-learn data on the task of natural language inference. |
Contrastive Data and Learning for Natural Language Processing (2022.naacl-tutorials)
Copied to clipboard
| Challenge: | Current NLP models heavily rely on effective representation learning algorithms. |
| Approach: | This tutorial introduces contrastive learning and provides an introduction to the techniques. |
| Outcome: | This tutorial provides an introduction to the fundamentals of contrastive learning approaches and the theory behind them. |
ALLSH: Active Learning Guided by Local Sensitivity and Hardness (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies show that labeling in crowdsourcing annotations is not an annotation artifact but rather a core linguistic phenomenon. |
| Approach: | They propose to retrieve unlabeled data with a local sensitivity and hardness-aware acquisition function. |
| Outcome: | The proposed method achieves consistent gains over the commonly used active learning strategies in various classification tasks. |
Active Learning for New Domains in Natural Language Understanding (N19-2)
Copied to clipboard
| Challenge: | Existing approaches to improve the accuracy of new domains are lacking annotated live utterances. |
| Approach: | They propose an algorithm called Majority-CRF that uses an ensemble of classification models to guide the selection of relevant utterances and a sequence labeling model to prioritize informative examples. |
| Outcome: | The proposed algorithm achieves 6.6%-9% error rate reduction and statistically significant improvements on six new domains. |
Learning with Contrastive Examples for Data-to-Text Generation (2020.coling-main)
Copied to clipboard
Yui Uehara, Tatsuya Ishigaki, Kasumi Aoki, Hiroshi Noji, Keiichi Goshima, Ichiro Kobayashi, Hiroya Takamura, Yusuke Miyao
| Challenge: | Existing models for data-to-text generation generate fluent but sometimes incorrect sentences . Existing studies show that using contrastive examples improves the ability of generating sentences with better lexical choice without degrading the fluency. |
| Approach: | They propose to use models trained on incorrect sentences and learning methods that exploit contrastive examples to reduce such errors. |
| Outcome: | The proposed models generate fluent sentences but often have problematic ones in terms of correctness. |
ALICE: Active Learning with Contrastive Natural Language Explanations (2020.emnlp-main)
Copied to clipboard
| Challenge: | Annotating a large dataset with annotations is costly and infeasible. |
| Approach: | They propose an expert-in-the-loop training framework that utilizes contrastive natural language explanations to improve data efficiency in learning. |
| Outcome: | The proposed framework outperforms baseline models trained with 40-100% more training data on bird species classification and social relationship classification tasks. |
Annotator-Centric Active Learning for Subjective NLP Tasks (2024.emnlp-main)
Copied to clipboard
| Challenge: | Annotator-centric active learning addresses the high costs of collecting human annotations by strategically annotating the most informative samples. |
| Approach: | They propose annotator-centric active learning which incorporates an annotation strategy following data sampling to approximate the full diversity of human judgments. |
| Outcome: | The proposed approach improves data efficiency and performs well in annotator-centric evaluations. |
Deep Bayesian Active Learning for Natural Language Processing: Results of a Large-Scale Empirical Study (D18-1)
Copied to clipboard
| Challenge: | Existing studies on Active Learning (AL) for natural language processing have limited data requirements. |
| Approach: | They propose a Bayesian active learning approach that reduces deep learning's data dependence by comparing models and acquisition functions. |
| Outcome: | The proposed approach outperforms i.i.d. baselines and is more efficient than other approaches. |
Contrastive Novelty-Augmented Learning: Anticipating Outliers with Large Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for classification are overly confident on unseen examples . despite recent advances in NLP, some categories of distribution shift still pose serious challenges. |
| Approach: | They propose a method that generates OOD examples representative of novel classes and trains to decrease confidence on them. |
| Outcome: | The proposed method improves classifiers' ability to detect and abstain on novel class examples over previous methods by 2.3% and 5.5% over previous approaches. |