Uncertainty-aware generative models for inferring document class prevalence (D18-1)
Copied to clipboard
| Challenge: | Existing methods for inferring the relative frequency of classes of unlabeled examples are imperfect. |
| Approach: | They propose a generative probabilistic modeling approach to prevalence estimation . they back out an implicit individual-level likelihood function to conduct fast inference . |
| Outcome: | The proposed method provides better confidence interval coverage than an alternative and is significantly more robust to shifts in the class prior between training and testing. |
Similar Papers
Estimating Confidence of Predictions of Individual Classifiers and TheirEnsembles for the Genre Classification Task (2022.lrec-1)
Copied to clipboard
| Challenge: | Genre identification is a kind of non-topic text classification. genre is defined as a functional space. |
| Approach: | They propose to use SOTA to identify genres in non-topic texts . genres are functional and cannot be expressed just by some keywords . |
| Outcome: | The proposed models show that they perform better than their individual models in large datasets. |
Learning under Label Proportions for Text Classification (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Using a set of data samples with ground truth labels, learning from label proportions is infeasible in many cases. |
| Approach: | They propose a setup where the data is provided in an aggregate form called bags and only the proportion of samples belonging to each class as the ground truth. |
| Outcome: | The proposed model achieves better results than baselines in almost 87% of the configurations. |
Improved Multi-label Classification under Temporal Concept Drift: Rethinking Group-Robust Algorithms in a Label-Wise Setting (2022.findings-acl)
Copied to clipboard
| Challenge: | Large-scale multi-label document classification presents interesting challenges due to the large label space and two-tiered skewed label distributions. |
| Approach: | They evaluate several group-robust optimization algorithms proposed to mitigate temporal concept drift and class imbalance in document classification. |
| Outcome: | The proposed algorithms outperform sampling-based approaches to class imbalance and concept drift and lead to much better performance on minority classes. |
Word-Level Uncertainty Estimation for Black-Box Text Classifiers using RNNs (2020.coling-main)
Copied to clipboard
| Challenge: | Neural Networks are not interpretable, since they provide no information about why particular decisions were made. |
| Approach: | They propose to decompose and visualize uncertainty of text classifiers at the level of words to provide detailed explanations of uncertainties. |
| Outcome: | The proposed approach decomposes and visualizes uncertainty of text classifiers at the level of words and enables a deeper understanding of unreliable model behaviours. |
Exploring Predictive Uncertainty and Calibration in NLP: A Study on the Impact of Method & Data Scarcity (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Using low-resource languages, we assess the quality of uncertainty estimates from a wide array of approaches, but with more data. |
| Approach: | They train models on sub-sampled datasets in three different languages to assess the confidence of a neural classifier. |
| Outcome: | The proposed models train on sub-sampled datasets in three different languages and show that the quality of uncertainty estimates suffers with more data. |
Efficient, Uncertainty-based Moderation of Neural Networks Text Classifiers (2022.findings-acl)
Copied to clipboard
| Challenge: | A series of benchmarking experiments based on three different datasets and three state-of-the-art classifiers show that our framework can improve the classification F1-scores by 5.1 to 11.2% (up to approx. 98 to 99%) |
| Approach: | They propose a semi-automated approach that passes unconfident, probably incorrect classifications to human moderators to minimize the workload. |
| Outcome: | The proposed approach can improve the classification F1-scores by 5.1 to 11.2% (up to approx. 98 to 99%) while reducing the moderation load up to 73.3% compared to a random moderation. |
Uncertainty Estimation and Reduction of Pre-trained Models for Text Regression (2022.tacl-1)
Copied to clipboard
| Challenge: | State-of-the-art classification and regression models are often not well calibrated and can be inaccurate. |
| Approach: | They quantify calibration of pre- trained language models for text regression . they apply uncertainty estimates to augment training data in low-resource domains . |
| Outcome: | The proposed model calibrations improve performance and generalizability in low-resource settings. |
Latent-Variable Generative Models for Data-Efficient Text Classification (D19-1)
Copied to clipboard
| Challenge: | Generative classifiers offer potential advantages over discriminative classifications, including data efficiency and zero-shot learning. |
| Approach: | They introduce discrete latent variables into generative story to improve classifiers' performance . they empirically characterize performance of their models on six text classification datasets . |
| Outcome: | The proposed model outperforms discriminative and generative classifiers on six text classification datasets. |
Learning from Emptiness: De-biasing Listwise Rerankers with Content-Agnostic Probability Calibration (2026.acl-short)
Copied to clipboard
| Challenge: | Existing methods for listwise reranking exhibit intrinsic position bias . existing methods are constrained by an inherent trade-off between efficiency and flexibility . |
| Approach: | They propose a training-free framework that mechanically decouples positional bias from ranking decisions. |
| Outcome: | a training-free framework decouples position bias from ranking decisions . evaluations show it outperforms training-based methods and outperformed expensive methods . |
Contextual Relevance and Adaptive Sampling for LLM-Based Document Reranking (2026.acl-long)
Copied to clipboard
| Challenge: | identifying relevant documents for Reasoning-intensive queries remains a challenge . large language models have shown strong performance in zero-shot document reranking . |
| Approach: | They propose a reranking algorithm that estimates contextual relevance by aggregating LLMs' relevance judgments across batches. |
| Outcome: | The proposed algorithm improves nDCG@10 over retrieval and reranking baselines by 15% and 6–21% respectively. |