QUEST: Efficient Extreme Multi-Label Text Classification with Large Language Models on Commodity Hardware (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Extreme multi-label text classification (EMTC) involves predicting multiple labels from a vast pool of candidates based on a user’s textual query. |
| Approach: | They propose a Quantized and Efficient Learning with Sampling Technique that uses a hash sampling module to reduce the data volume to one-fourth of its original size. |
| Outcome: | Extensive experiments show that QUEST outperforms existing methods while requiring fewer computational resources. |
Similar Papers
Enhancing Extreme Multi-Label Text Classification: Addressing Challenges in Model, Data, and Evaluation (2023.emnlp-industry)
Copied to clipboard
Dan Li, Zi Long Zhu, Janneke van de Loo, Agnes Masip Gomez, Vikrant Yadav, Georgios Tsatsaronis, Zubair Afzal
| Challenge: | Existing approaches to extreme multi-label text classification face inherent challenges in terms of model, data, and evaluation. |
| Approach: | They propose a label ranking model as an alternative to the conventional SciBERT-based classification model and an active learning-based pipeline that addresses the data scarcity of new labels during the update of a classification system. |
| Outcome: | The proposed model enables efficient handling of large-scale labels and accommodates new labels. |
Large Language Model as a Teacher for Zero-shot Tagging at Extreme Scales (2025.coling-main)
Copied to clipboard
| Challenge: | Extreme Zero-shot XMC uses lightweight bi-encoders to identify pseudo labels . state-of-the-art methods rely on suboptimal labels for training . |
| Approach: | They propose a framework that uses a lightweight bi-encoder to identify high-quality pseudo labels during training while employing a lightweight bi-coder for efficient inference. |
| Outcome: | The proposed framework achieves superior performance and efficiency over existing methods. |
Investigating Active Learning Sampling Strategies for Extreme Multi Label Text Classification (2022.lrec-1)
Copied to clipboard
| Challenge: | Large scale, multi-label text datasets with high numbers of different classes are expensive to annotate due to domain experts taking a lot of time working through all the classes. |
| Approach: | They propose to build classifiers on multi-label text datasets using Active Learning to reduce labeling effort. |
| Outcome: | The proposed classifiers can be used to reduce labeling effort on multi-label datasets. |
Long-tailed Extreme Multi-label Text Classification by the Retrieval of Generated Pseudo Label Descriptions (2023.findings-eacl)
Copied to clipboard
| Challenge: | Extreme Multi-label text classification (XMTC) is a tough challenge due to the sheer size of the label spaces and the severe data scarcity problem associated with the long tail of rare labels in highly skewed distributions. |
| Approach: | They propose to use a trained bag-of-words classifier to generate pseudo label descriptions from a training bag- of-word classifier. |
| Outcome: | The proposed approach outperforms the existing models in the tail label prediction problem and achieves state-of-the-art (SOTA) performance on XMTC benchmark datasets. |
Prototypical Extreme Multi-label Classification with a Dynamic Margin Loss (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent work in XMC addresses this problem using deep encoders that project text descriptions to an embedding space suitable for recovering the closest labels. |
| Approach: | They propose a method that uses a shallow transformer encoder to combine text-based embeddings, label centroids and learnable free vectors to improve XMC efficiency. |
| Outcome: | The proposed method achieves state-of-the-art in several public benchmarks of different sizes and domains while keeping the model efficient. |
Extreme Zero-Shot Learning for Extreme Text Classification (2022.naacl-main)
Copied to clipboard
| Challenge: | Experimental results show that MACLR achieves superior performance compared to other baseline methods. |
| Approach: | They propose to pre-train Transformer-based encoders with self-supervised contrastive losses to learn the semantic embeddings of instances and labels with raw text. |
| Outcome: | The proposed method improves on the EZ-XMC model with a limited number of ground-truth positive pairs. |
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies report that prompt-based direct classification eliminates the need for fine-tuning but lacks data and inference scalability. |
| Approach: | They propose a data augmentation technique that leverages large-scale language models to generate real text samples from a mixture of real samples. |
| Outcome: | The proposed method outperforms existing methods on diverse classification tasks. |
Can Large Language Models Serve as Effective Classifiers for Hierarchical Multi-Label Classification of Scientific Documents at Industrial Scale? (2025.coling-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated great potential in complex tasks such as multi-label classification, but the vast number of labels can exceed LLMs’ input limits. |
| Approach: | They propose a method that integrates large language models with dense retrieval techniques to overcome these challenges. |
| Outcome: | The proposed methods avoid frequent retraining by leveraging zero-shot and few-shot learning for real-time label assignment. |
XMC-Agent : Dynamic Navigation over Scalable Hierarchical Index for Incremental Extreme Multi-label Classification (2024.findings-acl)
Copied to clipboard
Yanjiang Liu, Tianyun Zhong, Yaojie Lu, Hongyu Lin, Ben He, Shuheng Zhou, Huijia Zhu, Weiqiang Wang, Zhongyi Liu, Xianpei Han, Le Sun
| Challenge: | Existing methods for XMC struggle with the growing set of labels due to their static label assumptions, and embedding-based methods struggle with complex mapping relationships due to late interaction paradigm. |
| Approach: | They propose a large language model (LLM) powered agent framework for extreme multi-label classification, XMC-Agent, which can effectively learn, manage and predict the extremely large and dynamically increasing set of labels. |
| Outcome: | The proposed framework can learn, manage and predict the extremely large and dynamically growing set of labels and achieves state-of-the-art performance on three standard datasets. |
From Text Segmentation to Enhanced Representation Learning: A Novel Approach to Multi-Label Classification for Long Texts (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models rely on pre-trained language models, which have a maximum input sequence length of 512 tokens, and therefore have 'input length limitation'. |
| Approach: | They propose a text segmentation algorithm which guarantees to produce the optimal segmentation to address the issue of input length limitation caused by PLMs. |
| Outcome: | The proposed method improves both text and label representations on MLTC datasets, unraveling the intricate correlations between texts and labels. |