| Challenge: | Experimental results show that heuristic-based active learning methods are limited when the data distribution of the underlying learning problems vary. |
| Approach: | They propose a method that learns an AL "policy" using "imitation learning" they use an efficient "algorithmic expert" which provides the policy learner with good actions in the encountered AL situations. |
| Outcome: | The proposed method is more effective than previous methods on two tasks . labeled data is rare while unlabelled data is abundant . |
Similar Papers
Learning How to Active Learn by Dreaming (P19-1)
Copied to clipboard
| Challenge: | Recent active learning methods are limited when the data distribution of learning problems vary. |
| Approach: | They propose a wake-and-dream-based active learning method that learns the AL policy directly on the target domain of interest by using wake and dream cycles. |
| Outcome: | The proposed method improves on cross-domain and cross-lingual tasks. |
Active Imitation Learning with Noisy Guidance (2020.acl-main)
Copied to clipboard
| Challenge: | Structured prediction methods learn models to map inputs to complex outputs with internal dependencies. |
| Approach: | They propose an algorithm that mimics an expert's choice at any queried state . they apply LEAQI to three sequence labelling tasks to reduce query costs . |
| Outcome: | The proposed algorithm shows better accuracies over a passive approach. |
Learning a Policy for Opportunistic Active Learning (D18-1)
Copied to clipboard
| Challenge: | Prior work has shown that opportunistic active learning can be used to improve grounding of natural language descriptions in interactive object retrieval tasks. |
| Approach: | They propose to use active learning to constrain possible queries during interactions to improve grounding of natural language descriptions in an interactive object retrieval task. |
| Outcome: | The proposed policy trades off task completion with model improvement that would benefit future tasks while lowering the cost of annotation without sacrificing model performance. |
Optimizing Annotation Effort Using Active Learning Strategies: A Sentiment Analysis Case Study in Persian (2020.lrec-1)
Copied to clipboard
Seyed Arad Ashrafi Asli, Behnam Sabeti, Zahra Majdabadi, Preni Golazizian, Reza Fahmi, Omid Momenzadeh
| Challenge: | Existing deep learning approaches require huge amounts of data to be trained properly. |
| Approach: | They propose to use Persian as a model to choose the samples for annotation instead of labeling the whole dataset. |
| Outcome: | The proposed models achieve the baseline performance with a significantly lower amount of labeled data. |
Practical Obstacles to Deploying Active Learning (D19-1)
Copied to clipboard
| Challenge: | Active learning (AL) is a widely-used training strategy for maximizing predictive performance subject to a fixed annotation budget. |
| Approach: | They propose to use active learning to optimize predictive performance . they find that current approaches do not generalize reliably across models and tasks . |
| Outcome: | The proposed approach outperforms training on i.i.d. datasets on supervised learning tasks. |
Cartography Active Learning (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to label data are limited in their notion of informativeness, due to post-training model uncertainty and batch diversity. |
| Approach: | They propose a new Active Learning algorithm that exploits the behavior of the model on individual instances during training as a proxy to find the most informative instances for labeling. |
| Outcome: | The proposed method is competitive to other common AL methods, showing that training dynamics derived from small seed data can be successfully used for AL. |
Active2 Learning: Actively reducing redundancies in Active Learning methods for Sequence Tagging and Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to deep learning for NLP require large amounts of labeled data. |
| Approach: | They propose an approach that iteratively selects a small number of examples for expert annotation based on their estimated utility in training the model. |
| Outcome: | The proposed approach reduces the data requirements of state-of-the-art AL strategies by 3-25% on multiple NLP tasks while achieving the same performance with virtually no additional computation overhead. |
Subsequence Based Deep Active Learning for Named Entity Recognition (2021.acl-long)
Copied to clipboard
| Challenge: | Active Learning (AL) has been successfully applied to Deep Learning to drastically reduce the amount of data required to achieve high performance. |
| Approach: | They propose to query subsequences within sentences and propagate their labels to other sentences. |
| Outcome: | The proposed approach achieves high performance on OntoNotes 5.0 and CoNLL 2003 with only 13% of training data and 27% of the training data. |
Deep Bayesian Active Learning for Natural Language Processing: Results of a Large-Scale Empirical Study (D18-1)
Copied to clipboard
| Challenge: | Existing studies on Active Learning (AL) for natural language processing have limited data requirements. |
| Approach: | They propose a Bayesian active learning approach that reduces deep learning's data dependence by comparing models and acquisition functions. |
| Outcome: | The proposed approach outperforms i.i.d. baselines and is more efficient than other approaches. |
Active Learning for BERT: An Empirical Study (2020.emnlp-main)
Copied to clipboard
Liat Ein-Dor, Alon Halfon, Ariel Gera, Eyal Shnarch, Lena Dankin, Leshem Choshen, Marina Danilevsky, Ranit Aharonov, Yoav Katz, Noam Slonim
| Challenge: | Existing approaches to deal with data scarcity are active learning (AL) and pre-trained models are not being considered. |
| Approach: | They propose to use active learning techniques to cope with data scarcity in binary text classification scenarios where the annotation budget is very small and the data is often skewed. |
| Outcome: | The proposed methods improve BERT performance in binary text classification scenarios where the annotation budget is very small and the data is often skewed. |