Papers with BADGE
BADGE: Speeding Up BERT Inference after Deployment via Block-wise Bypasses and Divergence-based Early Exiting (2023.acl-industry)
Copied to clipboard
| Challenge: | Recent years have witnessed the rise of many pre-trained language models (PLMs) such as GPT (Radford et al., 2019) and XLNet (Yang e.t al, 2019). |
| Approach: | They propose a framework which consists of two off-the-shelf methods for improving PLMs’ early exiting. |
| Outcome: | The proposed method can reduce the average latency of pre-trained language models and work with other inference speed-up methods like model pruning. |
CAL-Log: Cost-Aware Active Learning with Logarithmic Cognitive Effort Modeling and Online Adaptation to Human Annotation Behavior (2026.acl-srw)
Copied to clipboard
| Challenge: | Standard uncertainty sampling assumes that annotating a 500-word document requires the same effort as a 50-word tweet, leading to suboptimal resource allocation when documents vary in length. |
| Approach: | They propose a cost-aware AL variant using logarithmic cost modeling where C(x) is the predicted annotation time for document x and L(x), is its token length. |
| Outcome: | Experiments on ten text classification benchmarks show a 3.3 speedup over BADGE and 3.9 over Entropy sampling to reach F1=0.80, with large effect sizes. |
Denoising Labeled Data for Comment Moderation Using Active Learning (2024.lrec-main)
Copied to clipboard
| Challenge: | Large contextualized language models (LLMs) are becoming ubiquitous in natural language processing due to their performance and adaptability to diverse tasks. |
| Approach: | They propose to use active learning methods to denoise textual data for model training by sampling the most informative examples with noisy labels with active learning. |
| Outcome: | The proposed method reduces the cost of reannotation by reducing noise in noisy examples. |