Data-efficient Active Learning for Structured Prediction with Partial Annotation and Self-Training (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Structured prediction is a fundamental problem in NLP, wherein the label space consists of complex structured outputs with groups of interdependent variables. |
| Approach: | They propose a partial annotation approach that selects only the most informative sub-structures for annotation and a method that incorporates the current model's automatic predictions as pseudo-labels for un-annotated sub-structurals. |
| Outcome: | The proposed approach reduces annotation cost over strong full annotation baselines under a fair comparison scheme that takes reading time into consideration. |
Similar Papers
Self-Training for Sample-Efficient Active Learning for Text Classification with Pre-Trained Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to train models without labeled data are lacking in supervised tasks . a lack of labeles is the main obstacle to real-world applications . |
| Approach: | They propose a semi-supervised approach that uses a model to obtain pseudo-labels for unlabeled data. |
| Outcome: | The proposed method outperforms the reproduced methods on four text classification benchmarks. |
Reinforced Active Learning for Low-Resource, Domain-Specific, Multi-Label Text Classification (2023.findings-acl)
Copied to clipboard
| Challenge: | Modern text classification systems achieve excellent accuracy across tasks and corpora. |
| Approach: | They propose a Reinforcement Learning policy that uses many different aspects of the data and task to select the most informative unlabeled subset dynamically over the course of the AL procedure. |
| Outcome: | The proposed framework outperforms baselines on four complex multi-class, multi-label text classification datasets. |
Partial Or Complete, That’s The Question (N19-1)
Copied to clipboard
| Challenge: | Existing annotation schemes aim at acquiring completely annotated structures, but partial annotations can be costly and hinder learning. |
| Approach: | They propose a method to find out that learning from partial structures can sometimes outperform learning from complete ones. |
| Outcome: | The proposed method outperforms existing methods in three different structured learning tasks. |
Leveraging the Structure of Pre-trained Embeddings to Minimize Annotation Effort (2024.naacl-long)
Copied to clipboard
| Challenge: | Current approaches for text classification are based on fine-tuning the representations computed by large language models. |
| Approach: | They propose to exploit structural properties of pre-trained embeddings to spread information . they use a semisupervised strategy to train models with minimal annotation effort . |
| Outcome: | The proposed method outperforms self-training and random walk labels on different datasets. |
Adapting Coreference Resolution Models through Active Learning (2022.acl-long)
Copied to clipboard
| Challenge: | Neural coreference resolution models trained on one dataset may not transfer to new, low-resource domains. |
| Approach: | They investigate how to actively label coreference by sampling a small subset of data for annotators to label. |
| Outcome: | The proposed model can be more realistic when labeling spans within the same document than when annotating spans across documents. |
Selective Labeling: How to Radically Lower Data-Labeling Costs for Document Extraction Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | a key bottleneck in developing automatic extraction models for visually rich documents is the cost of acquiring labeled documents. |
| Approach: | They propose selective labeling to provide "yes/no" labels for candidate extractions predicted by a model trained on partially labeled documents. |
| Outcome: | The proposed method reduces the cost of acquiring labeled data by 10 with a negligible loss in accuracy. |
Towards Computationally Feasible Deep Active Learning (2022.findings-naacl)
Copied to clipboard
Akim Tsvigun, Artem Shelmanov, Gleb Kuzmin, Leonid Sanochkin, Daniil Larionov, Gleb Gusev, Manvel Avetisian, Leonid Zhukov
| Challenge: | Active learning (AL) is a technique for reducing the amount of annotation required for training machine learning models. |
| Approach: | They propose two techniques that reduce the amount of time required for AL . they use pseudo-labeling and distilled models to train a successor model . |
| Outcome: | The proposed algorithm reduces the time and computational overhead required to train an acquisition model and estimate uncertainty on instances in the unlabeled pool. |
Reassessing Active Learning Adoption in Contemporary NLP: A Community Survey (2026.eacl-long)
Copied to clipboard
| Challenge: | a longstanding strategy to reduce annotation costs is active learning . data annotation is expected to remain important and active learning to stay relevant . |
| Approach: | They conduct an online survey to assess the perceived relevance of data annotation and active learning . they propose a strategy to reduce annotation costs using active learning, an iterative process . |
| Outcome: | The proposed strategies reduce setup complexity and uncertainty cost while maintaining model performance. |
Investigating Active Learning Sampling Strategies for Extreme Multi Label Text Classification (2022.lrec-1)
Copied to clipboard
| Challenge: | Large scale, multi-label text datasets with high numbers of different classes are expensive to annotate due to domain experts taking a lot of time working through all the classes. |
| Approach: | They propose to build classifiers on multi-label text datasets using Active Learning to reduce labeling effort. |
| Outcome: | The proposed classifiers can be used to reduce labeling effort on multi-label datasets. |
Empowering Active Learning to Jointly Optimize System and User Demands (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to active learning maximize the system performance by sampling unlabeled instances for annotation that yield the most efficient training. |
| Approach: | They propose an active learning approach that integrates active learning with an end-user application to optimize the user's training and receiving useful instances. |
| Outcome: | The proposed approach satisfies both objectives when alternative methods lead to many unsuitable exercises for end users. |