| Challenge: | Active learning (AL) is a special family of machine learning algorithms designed to reduce labeling costs and improve accuracy. |
| Approach: | They developed an open-source annotation system for NLP tasks equipped with features to make AL effective in real-world annotation projects. |
| Outcome: | ALANNO is an open-source annotation system for NLP tasks equipped with features to make AL effective in real-world annotation projects. |
Similar Papers
ALToolbox: A Set of Tools for Active Learning Annotation of Natural Language Texts (2022.emnlp-demos)
Copied to clipboard
Akim Tsvigun, Leonid Sanochkin, Daniil Larionov, Gleb Kuzmin, Artem Vazhentsev, Ivan Lazichny, Nikita Khromov, Danil Kireev, Aleksandr Rubashevskii, Olga Shahmatova, Dmitry V. Dylov, Igor Galitskiy, Artem Shelmanov
| Challenge: | Currently, the framework supports text classification, sequence tagging, and seq2seq tasks. |
| Approach: | They propose an open-source framework for active learning annotation in natural language processing that provides an easy-to-deploy GUI annotation tool directly in the Jupyter IDE. |
| Outcome: | The proposed framework reduces computational overhead and duration of AL iterations and increases annotated data reusability. |
A Survey of Active Learning for Natural Language Processing (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing literature surveys on active learning for NLP are too specific or too general, covering deep active learning. |
| Approach: | They propose to use active learning to improve model learning and annotation cost for NLP problems. |
| Outcome: | The proposed approach is based on a large dataset of data-driven machine learning models. |
Reassessing Active Learning Adoption in Contemporary NLP: A Community Survey (2026.eacl-long)
Copied to clipboard
| Challenge: | a longstanding strategy to reduce annotation costs is active learning . data annotation is expected to remain important and active learning to stay relevant . |
| Approach: | They conduct an online survey to assess the perceived relevance of data annotation and active learning . they propose a strategy to reduce annotation costs using active learning, an iterative process . |
| Outcome: | The proposed strategies reduce setup complexity and uncertainty cost while maintaining model performance. |
Practical, Efficient, and Customizable Active Learning for Named Entity Recognition in the Digital Humanities (N19-1)
Copied to clipboard
Alexander Erdmann, David Joseph Wrisley, Benjamin Allen, Christopher Brown, Sophie Cohen-Bodénès, Micha Elsner, Yukun Feng, Brian Joseph, Béatrice Joyeux-Prunel, Marie-Catherine de Marneffe
| Challenge: | Scholars in interdisciplinary fields like the Digital Humanities are increasingly interested in semantic annotation of specialized corpora. |
| Approach: | They propose an active learning solution for named entity recognition that maximizes a custom model’s improvement per additional unit of manual annotation. |
| Outcome: | The proposed model reduces required annotation by 20-60% and outperforms a competitive active learning baseline. |
Active Learning for BERT: An Empirical Study (2020.emnlp-main)
Copied to clipboard
Liat Ein-Dor, Alon Halfon, Ariel Gera, Eyal Shnarch, Lena Dankin, Leshem Choshen, Marina Danilevsky, Ranit Aharonov, Yoav Katz, Noam Slonim
| Challenge: | Existing approaches to deal with data scarcity are active learning (AL) and pre-trained models are not being considered. |
| Approach: | They propose to use active learning techniques to cope with data scarcity in binary text classification scenarios where the annotation budget is very small and the data is often skewed. |
| Outcome: | The proposed methods improve BERT performance in binary text classification scenarios where the annotation budget is very small and the data is often skewed. |
Beyond Labels: Empowering Human Annotators with Natural Language Explanations through a Novel Active-Learning Architecture (2023.findings-emnlp)
Copied to clipboard
Bingsheng Yao, Ishan Jindal, Lucian Popa, Yannis Katsis, Sayan Ghosh, Lihong He, Yuxuan Lu, Shashank Srivastava, Yunyao Li, James Hendler, Dakuo Wang
| Challenge: | Existing low-resource learning techniques focus on label annotation while neglecting the natural language explanation of a data point. |
| Approach: | They propose a novel architecture that leverages an explanation-generation model to produce explanations guided by human explanations and a prediction model that utilizes generated explanations toward prediction faithfully. |
| Outcome: | The proposed architecture produces explanations guided by human explanations, a prediction model that utilizes generated explanations toward prediction faithfully, and a data diversity-based AL sampling strategy that benefits from the explanation annotations. |
LLMaAA: Making Large Language Models as Active Annotators (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing supervised learning methods in natural language processing require large amounts of data. |
| Approach: | They propose an active learning loop that takes LLMs as annotators and puts them into an active loop to determine what to annotate efficiently. |
| Outcome: | The proposed model outperforms existing models with few-shot performance in two NLP tasks. |
From Selection to Generation: A Survey of LLM-based Active Learning (2025.acl-long)
Copied to clipboard
Yu Xia, Subhojyoti Mukherjee, Zhouhang Xie, Junda Wu, Xintong Li, Ryan Aponte, Hanjia Lyu, Joe Barrow, Hongjie Chen, Franck Dernoncourt, Branislav Kveton, Tong Yu, Ruiyi Zhang, Jiuxiang Gu, Nesreen K. Ahmed, Yu Wang, Xiang Chen, Hanieh Deilamsalehy, Sungchul Kim, Zhengmian Hu, Yue Zhao, Nedim Lipka, Seunghyun Yoon, Ting-Hao Kenneth Huang, Zichao Wang, Puneet Mathur, Soumyabrata Pal, Koyel Mukherjee, Zhehao Zhang, Namyong Park, Thien Huu Nguyen, Jiebo Luo, Ryan A. Rossi, Julian McAuley
| Challenge: | Large Language Models (LLMs) have been used for selection and training of data for active learning. |
| Approach: | They propose an intuitive taxonomy that categorizes LLM-based active learning techniques and discuss the transformative roles they can play in the active learning loop. |
| Outcome: | The proposed model can generate entirely new data instances and provide more cost-effective annotations with fewer labeled data instances. |
FreeAL: Towards Human-Free Active Learning in the Era of Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Modern machine learning models require a huge collection of precisely labeled data, which can be labor-intensive and time-consuming. |
| Approach: | They propose a collaborative learning framework that interactively distills and filters the task-specific knowledge from LLMs. |
| Outcome: | The proposed framework improves zero-shot performance on eight benchmark datasets without human supervision. |
Annotator-Centric Active Learning for Subjective NLP Tasks (2024.emnlp-main)
Copied to clipboard
| Challenge: | Annotator-centric active learning addresses the high costs of collecting human annotations by strategically annotating the most informative samples. |
| Approach: | They propose annotator-centric active learning which incorporates an annotation strategy following data sampling to approximate the full diversity of human judgments. |
| Outcome: | The proposed approach improves data efficiency and performs well in annotator-centric evaluations. |