Low-resource Interactive Active Labeling for Fine-tuning Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing active learning methods for fine-tuning language models are underperforming in low-resource, interactive labeling setting. |
| Approach: | They propose a novel active learning method that employs a hybrid sampling strategy to minimize labeling cost and acquisition latency while providing a framework for adapting to dataset diversity. |
| Outcome: | The proposed method reduces labeling cost and acquisition latency while providing a framework for adapting to dataset diversity via user guidance. |
Similar Papers
Parameter-Efficient Language Model Tuning with Active Learning in Low-Resource Settings (2023.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) have ignited a surge in demand for effective fine-tuning techniques . data labeling is notoriously time-consuming and expensive, hindering the development of sizable labeled datasets . |
| Approach: | They propose to use active learning to reduce labeling costs by minimizing label complexity . they find PEFT adapter modules have significant potential in low-resource settings . |
| Outcome: | The proposed model outperforms FFT in low-resource settings and shows that it yields more stable representations of early and middle layers than FFT. |
On the Importance of Effectively Adapting Pretrained Language Models for Active Learning (2022.acl-short)
Copied to clipboard
| Challenge: | Recent active learning approaches in NLP use off-the-shelf pretrained language models (LMs) . a poor training strategy can be catastrophic for AL, authors argue . |
| Approach: | They propose to first adapt the pretrained LM to the target task and then use it for AL. |
| Outcome: | The proposed approach provides substantial data efficiency improvements compared to the standard fine-tuning approach. |
Fine-tuning BERT for Low-Resource Natural Language Understanding via Active Learning (2020.coling-main)
Copied to clipboard
| Challenge: | Recent work has explored the suitability of pre-trained language models in low resource settings with less than 1,000 training data points. |
| Approach: | They propose to use pool-based active learning to speed up training while keeping the cost of labeling new data constant. |
| Outcome: | The proposed model can be fine-tuned to optimize for low-resource settings while keeping the cost of labeling constant. |
Rethinking Full Finetuning from Pretraining Checkpoints in Active Learning for African Languages (2025.acl-srw)
Copied to clipboard
| Challenge: | Existing approaches to improve model performance are finetuning on all acquired data after each round, which is computationally expensive in multilingual and low-resource settings. |
| Approach: | They evaluate continual finetuning (CF) against full finetuned (FA) across 28 African languages using MasakhaNEWS and SIB-200. |
| Outcome: | The proposed approach outperforms full finetuning (FA) in 28 African languages, achieving up to 35% reductions in GPU memory, FLOPs, and training time. |
From Selection to Generation: A Survey of LLM-based Active Learning (2025.acl-long)
Copied to clipboard
Yu Xia, Subhojyoti Mukherjee, Zhouhang Xie, Junda Wu, Xintong Li, Ryan Aponte, Hanjia Lyu, Joe Barrow, Hongjie Chen, Franck Dernoncourt, Branislav Kveton, Tong Yu, Ruiyi Zhang, Jiuxiang Gu, Nesreen K. Ahmed, Yu Wang, Xiang Chen, Hanieh Deilamsalehy, Sungchul Kim, Zhengmian Hu, Yue Zhao, Nedim Lipka, Seunghyun Yoon, Ting-Hao Kenneth Huang, Zichao Wang, Puneet Mathur, Soumyabrata Pal, Koyel Mukherjee, Zhehao Zhang, Namyong Park, Thien Huu Nguyen, Jiebo Luo, Ryan A. Rossi, Julian McAuley
| Challenge: | Large Language Models (LLMs) have been used for selection and training of data for active learning. |
| Approach: | They propose an intuitive taxonomy that categorizes LLM-based active learning techniques and discuss the transformative roles they can play in the active learning loop. |
| Outcome: | The proposed model can generate entirely new data instances and provide more cost-effective annotations with fewer labeled data instances. |
Noise-Robust Fine-Tuning of Pretrained Language Models via External Guidance (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained Language Models (PLMs) are advanced but data labels are noisy due to the complex annotation process. |
| Approach: | They propose a framework for fine-tuning PLMs using noisy labels that incorporates guidance from Large Language Models like ChatGPT. |
| Outcome: | Experiments on synthetic and real-world noisy datasets show that the proposed framework outperforms the state-of-the-art framework. |
A Little Annotation does a Lot of Good: A Study in Bootstrapping Low-resource Named Entity Recognizers (D19-1)
Copied to clipboard
| Challenge: | Named entity recognition models rely on large amounts of labeled data, making them challenging to extend to new, lower-resource languages. |
| Approach: | They propose a method for bootstrapping named entity recognition models in under-resourced languages . they use cross-lingual transfer learning and targeted annotation of only uncertain entities . |
| Outcome: | The proposed method achieves competitive accuracy with just one-tenth of training data. |
Semi-supervised Fine-tuning for Large Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing LLMs require labeled data, which can be costly in real-world applications. |
| Approach: | They propose a framework that can fully exploit labeled and unlabeled data for LLM fine-tuning . they conducted experiments using GPT-4o-mini and Llama-3.1 on seven general or domain-specific datasets . |
| Outcome: | The proposed framework can fully exploit labeled and unlabeled data for LLM alignment from a propagate-and-select manner. |
Cold-start Active Learning through Self-supervised Language Modeling (2020.emnlp-main)
Copied to clipboard
| Challenge: | Labeling data is a fundamental bottleneck in machine learning due to annotation cost and time. |
| Approach: | They propose a strategy that uses the pre-training loss to find examples that surprise the model and minimize labeling costs. |
| Outcome: | The proposed approach reduces labeling costs and costs by using pre-trained language models. |
AcTune: Uncertainty-Based Active Self-Training for Active Fine-Tuning of Pretrained Language Models (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning pre-trained language models ignore the potential of unlabeled data. |
| Approach: | They propose a framework that allows users to unleash the power of unlabeled data via self-training. |
| Outcome: | The proposed framework outperforms active learning and self-training baselines and improves the label efficiency of PLM fine-tuning by 56.2% on average. |