| Challenge: | Large-Scale multi-label text classification is a task of assigning to each document all the relevant labels from a large set, typically containing thousands of labels (classes). |
| Approach: | They propose to use a dataset of 57k English EU legislative documents annotated with 4.3k EUROVOC labels for LMTC, few-shot learning and contextual embeddings. |
| Outcome: | The proposed dataset is suitable for LMTC, few- and zero-shot learning and bypasses the maximum text length limit. |
Similar Papers
An Empirical Study on Large-Scale Multi-Label Text Classification Including Few and Zero-Shot Labels (2020.emnlp-main)
Copied to clipboard
Ilias Chalkidis, Manos Fergadiotis, Sotiris Kotitsas, Prodromos Malakasiotis, Nikolaos Aletras, Ion Androutsopoulos
| Challenge: | Large-scale Multi-label Text Classification (LMTC) is a type of classification that assigns labels to a large set of labels. |
| Approach: | They propose to use probabilistic label trees to improve frequent, few and zero-shot learning . they propose to combine a new state-of-the-art method with pre-trained Transformers . |
| Outcome: | The proposed models outperform existing models on frequent, few and zero-shot learning on three datasets from different domains. |
Meta-LMTC: Meta-Learning for Large-Scale Multi-Label Text Classification (2021.emnlp-main)
Copied to clipboard
| Challenge: | Large-scale multi-label text classification tasks often face long-tailed label distributions, where many labels have few or even no training instances. |
| Approach: | They propose a meta-learning approach that incorporates the objective of adapting to new low-resource tasks into the meta-Learning phase. |
| Outcome: | The proposed approach achieves state-of-the-art against strong baselines and can still enhance powerful BERTlike models. |
Can Large Language Models Serve as Effective Classifiers for Hierarchical Multi-Label Classification of Scientific Documents at Industrial Scale? (2025.coling-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated great potential in complex tasks such as multi-label classification, but the vast number of labels can exceed LLMs’ input limits. |
| Approach: | They propose a method that integrates large language models with dense retrieval techniques to overcome these challenges. |
| Outcome: | The proposed methods avoid frequent retraining by leveraging zero-shot and few-shot learning for real-time label assignment. |
MultiEURLEX - A multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfer (2021.emnlp-main)
Copied to clipboard
| Challenge: | MULTI-EURLEX is a dataset for topic classification of EU legal documents . fine-tuning a multilingually pretrained model in a single source language leads to catastrophic forgetting of multilingual knowledge and poor zero-shot transfer to other languages. |
| Approach: | They propose to use the dataset as a testbed for zero-shot cross-lingual transfer to exploit annotated training documents in one language to classify documents in another language. |
| Outcome: | The proposed model can be used to classify EU legal documents in other languages without a single source language and retain multilingual knowledge. |
CoPHE: A Count-Preserving Hierarchical Evaluation Metric in Large-Scale Multi-Label Text Classification (2021.emnlp-main)
Copied to clipboard
| Challenge: | Large-Scale Multi-Label Text Classification (LMTC) tasks with hierarchical label spaces include automatic assignment of ICD-9 codes to discharge summaries. |
| Approach: | They propose a set of metrics for hierarchical evaluation using the depth of the ontology to evaluate the predictions of neural LMTC models. |
| Outcome: | The proposed metrics compare with previous evaluations on prior art models for ICD-9 coding in MIMIC-III and propose further avenues of research involving the proposed representation. |
From Text Segmentation to Enhanced Representation Learning: A Novel Approach to Multi-Label Classification for Long Texts (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models rely on pre-trained language models, which have a maximum input sequence length of 512 tokens, and therefore have 'input length limitation'. |
| Approach: | They propose a text segmentation algorithm which guarantees to produce the optimal segmentation to address the issue of input length limitation caused by PLMs. |
| Outcome: | The proposed method improves both text and label representations on MLTC datasets, unraveling the intricate correlations between texts and labels. |
Large Language Model as a Teacher for Zero-shot Tagging at Extreme Scales (2025.coling-main)
Copied to clipboard
| Challenge: | Extreme Zero-shot XMC uses lightweight bi-encoders to identify pseudo labels . state-of-the-art methods rely on suboptimal labels for training . |
| Approach: | They propose a framework that uses a lightweight bi-encoder to identify high-quality pseudo labels during training while employing a lightweight bi-coder for efficient inference. |
| Outcome: | The proposed framework achieves superior performance and efficiency over existing methods. |
mOSCAR: A Large-scale Multilingual and Multimodal Document-level Corpus (2025.findings-acl)
Copied to clipboard
Matthieu Futeral, Armel Randy Zebaze, Pedro Ortiz Suarez, Julien Abadji, Rémi Lacroix, Cordelia Schmid, Rachel Bawden, Benoît Sagot
| Challenge: | Existing studies show that multimodal large language models can learn from text-image data. |
| Approach: | They propose to train multimodal large language models on large amounts of text-image data . they also show a boost in few-shot learning performance across various multilingual tasks . |
| Outcome: | The proposed dataset is not public and is only in English . it is the first large-scale multilingual and multimodal document corpus crawled from the web. |
Large Language Models Do Multi-Label Classification Differently (2025.emnlp-main)
Copied to clipboard
| Challenge: | Multi-label classification is prevalent in real-world settings, but the behavior of Large Language Models (LLMs) in this setting is understudied. |
| Approach: | They propose to use initial probability distributions to analyze output distributions of LLMs at each label generation step to find out how LLM models perform multi-label classification. |
| Outcome: | The proposed methods improve alignment and predictive performance over existing methods. |
Structural Contrastive Representation Learning for Zero-shot Multi-label Text Classification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches for zero-shot multi-label text classification struggle with accuracy and poor training efficiency. |
| Approach: | They propose a structural contrastive representation learning approach that uses randomized text segmentation to generate high-quality contrastive pairs. |
| Outcome: | The proposed approach improves accuracy and speed up training time on publicly available datasets. |