Beyond Labels: Empowering Human Annotators with Natural Language Explanations through a Novel Active-Learning Architecture

Challenge:	Existing low-resource learning techniques focus on label annotation while neglecting the natural language explanation of a data point.
Approach:	They propose a novel architecture that leverages an explanation-generation model to produce explanations guided by human explanations and a prediction model that utilizes generated explanations toward prediction faithfully.
Outcome:	The proposed architecture produces explanations guided by human explanations, a prediction model that utilizes generated explanations toward prediction faithfully, and a data diversity-based AL sampling strategy that benefits from the explanation annotations.

Similar Papers

Overview

Challenge

Approach

Outcome

XAL: EXplainable Active Learning Makes Classifiers Better Low-resource Learners (2024.naacl-long)

Copied to clipboard

Yun Luo, Zhen Yang, Fandong Meng, Yingjie Li, Fang Guo, Qinglin Qi, Jie Zhou, Yue Zhang

Responsible & Trustworthy NLP

Challenge:	Existing methods for active learning rely on model uncertainty or disagreement to pick unlabeled data, leading to over-confidence in superficial patterns and lack of exploration.
Approach:	They propose to use a bi-directional encoder and a uni-directional decoder to generate and score an explanation for low-resource text classification.
Outcome:	The proposed model improves on 9 strong baselines on six datasets and can generate explanations for its predictions.

ALANNO: An Active Learning Annotation System for Mortals (2023.eacl-demo)

Copied to clipboard

Josip Jukić, Fran Jelenić, Miroslav Bićanić, Jan Snajder

Responsible & Trustworthy NLP

Challenge:	Active learning (AL) is a special family of machine learning algorithms designed to reduce labeling costs and improve accuracy.
Approach:	They developed an open-source annotation system for NLP tasks equipped with features to make AL effective in real-world annotation projects.
Outcome:	ALANNO is an open-source annotation system for NLP tasks equipped with features to make AL effective in real-world annotation projects.

From Selection to Generation: A Survey of LLM-based Active Learning (2025.acl-long)

Copied to clipboard

Yu Xia, Subhojyoti Mukherjee, Zhouhang Xie, Junda Wu, Xintong Li, Ryan Aponte, Hanjia Lyu, Joe Barrow, Hongjie Chen, Franck Dernoncourt, Branislav Kveton, Tong Yu, Ruiyi Zhang, Jiuxiang Gu, Nesreen K. Ahmed, Yu Wang, Xiang Chen, Hanieh Deilamsalehy, Sungchul Kim, Zhengmian Hu, Yue Zhao, Nedim Lipka, Seunghyun Yoon, Ting-Hao Kenneth Huang, Zichao Wang, Puneet Mathur, Soumyabrata Pal, Koyel Mukherjee, Zhehao Zhang, Namyong Park, Thien Huu Nguyen, Jiebo Luo, Ryan A. Rossi, Julian McAuley

Responsible & Trustworthy NLP

Challenge:	Large Language Models (LLMs) have been used for selection and training of data for active learning.
Approach:	They propose an intuitive taxonomy that categorizes LLM-based active learning techniques and discuss the transformative roles they can play in the active learning loop.
Outcome:	The proposed model can generate entirely new data instances and provide more cost-effective annotations with fewer labeled data instances.

Label and Explanation Variation in LLM-Based Annotation: a Case Study in Natural Language Inference (2026.acl-long)

Copied to clipboard

Artur Kulmizev, Erika Lombart, Patrick Watrin, Marie-Catherine de Marneffe

Explainability & Interpretability in NLP

Challenge:	Large language models (LLMs) have shown considerable promise for annotation purposes, but questions remain about their ability to capture human label variation (HLV) label variation is genuine disagreement between annotators observed across NLP tasks.
Approach:	They investigate how label and explanation variation manifests within and across LLMs with respect to the Natural Language Inference task.
Outcome:	The proposed models generate label distributions similar to humans but exhibit distinct, idiosyncratic judgments and disagreement patterns.

Active Learning for Natural Language Generation (2023.emnlp-main)

Copied to clipboard

Yotam Perlitz, Ariel Gera, Michal Shmueli-Scheuer, Dafna Sheinwald, Noam Slonim, Liat Ein-Dor

Information Extraction & Text Mining

Challenge:	Existing approaches to NLG are limited by the lack of annotated data.
Approach:	They propose to use active learning to reduce the cost of manual annotation to improve annotation efficiency by selecting the most informative examples to label.
Outcome:	The proposed approach surpasses baseline of random example selection in some cases but not in others.

A Rose by Any Other Name: LLM-Generated Explanations Are Good Proxies for Human Explanations to Collect Label Distributions on NLI (2025.findings-acl)

Copied to clipboard

Beiduo Chen, Siyao Peng, Anna Korhonen, Barbara Plank

Explainability & Interpretability in NLP

Challenge:	Recent research has shown that explanations provide valuable information for understanding human label variation (HLV) Large language models (LLMs) can approximate HJD from a few human-provided label-explanation pairs, but collecting explanations for every label is still time-consuming.
Approach:	They propose to use Large Language Models (LLMs) as annotators to generate model explanations for a few given human labels.
Outcome:	The proposed models can generate human-provided explanations from human labels, but they are still time-consuming.

LLMaAA: Making Large Language Models as Active Annotators (2023.findings-emnlp)

Copied to clipboard

Ruoyu Zhang, Yanzeng Li, Yongliang Ma, Ming Zhou, Lei Zou

Language Models

Challenge:	Existing supervised learning methods in natural language processing require large amounts of data.
Approach:	They propose an active learning loop that takes LLMs as annotators and puts them into an active loop to determine what to annotate efficiently.
Outcome:	The proposed model outperforms existing models with few-shot performance in two NLP tasks.

Active2 Learning: Actively reducing redundancies in Active Learning methods for Sequence Tagging and Machine Translation (2021.naacl-main)

Copied to clipboard

Rishi Hazra, Parag Dutta, Shubham Gupta, Mohammed Abdul Qaathir, Ambedkar Dukkipati

Machine Translation

Challenge:	Existing approaches to deep learning for NLP require large amounts of labeled data.
Approach:	They propose an approach that iteratively selects a small number of examples for expert annotation based on their estimated utility in training the model.
Outcome:	The proposed approach reduces the data requirements of state-of-the-art AL strategies by 3-25% on multiple NLP tasks while achieving the same performance with virtually no additional computation overhead.

Explain Yourself! Leveraging Language Models for Commonsense Reasoning (P19-1)

Copied to clipboard

Nazneen Fatema Rajani, Bryan McCann, Caiming Xiong, Richard Socher

Reasoning

Challenge:	Empirical results indicate that we can effectively leverage language models for commonsense reasoning.
Approach:	They propose to use commonsense auto-generated explanations to train language models to generate explanations that can be used during training and inference in a commonsensense Auto-Generated Explanation framework.
Outcome:	Empirical results show that the proposed framework improves on the commonsenseQA task by 10%.

Active Learning for New Domains in Natural Language Understanding (N19-2)

Copied to clipboard

Stanislav Peshterliev, John Kearney, Abhyuday Jagannatha, Imre Kiss, Spyros Matsoukas

Low-Resource NLP

Challenge:	Existing approaches to improve the accuracy of new domains are lacking annotated live utterances.
Approach:	They propose an algorithm called Majority-CRF that uses an ensemble of classification models to guide the selection of relevant utterances and a sequence labeling model to prioritize informative examples.
Outcome:	The proposed algorithm achieves 6.6%-9% error rate reduction and statistically significant improvements on six new domains.