Generative-to-Discriminative Test-Time Adaptation via Manifold-Aware Diffusion and Bayesian Distillation (2026.findings-acl)
Copied to clipboard
Boyun Zhang, Zequn Xie, Fangming Feng, Zihan Zhang, Yongbo He, Chuxin Wang, Sihang Cai, Tao Jin, Qifei Zhang
| Challenge: | Existing discriminative approaches suffer from "confident but wrong" failure mode, blindly adapting to OOD noise leading to error accumulation. |
| Approach: | They propose a TTA framework that harmonizes the robustness of generative diffusion models with the efficiency of discriminative regression networks via Bayesian Diffusion Distillation (BDD). |
| Outcome: | The proposed framework reduces MAE from 0.6872 to 0.5673 and boosts binary accuracy by 5.81 percentage points (reaching 57.33%) it also reduces the MAE of the MOSI to SIMS shift and achieves an 11.18-point gain over the baseline. |
Similar Papers
STAF: Pushing the Boundaries of Test-Time Adaptation towards Practical Noise Scenarios (2024.lrec-main)
Copied to clipboard
| Challenge: | Pre-trained language models have demonstrated superior performance on NLP tasks . however, when the training domain and testing domain are taken from different distributions, the deployed model often violates this assumption. |
| Approach: | They propose a Stable Test-time Adaptation Framework to stabilize the adaptation process. |
| Outcome: | The proposed framework boosts model robustness to noise distribution shifts while minimizing error accumulation and catastrophic forgetting. |
Preconditioned Test-Time Adaptation for Out-of-Distribution Debiasing in Narrative Generation (2026.acl-long)
Copied to clipboard
| Challenge: | Debiased large language models excel at handling known or low-bias prompts, but fail on unfamiliar and high-biased prompts. |
| Approach: | They propose a debiasing framework that detects high-bias prompts and triggers context-aware LoRA updates only when a bias-risk score exceeds a threshold. |
| Outcome: | The proposed framework reduces toxicity/bias score with significantly lower latency than standard optimization methods. |
CDAˆ2: Counterfactual Diffusion Augmentation for Cross-Domain Adaptation in Low-Resource Sentiment Analysis (2025.coling-main)
Copied to clipboard
| Challenge: | Domain adaptation is widely employed in cross-domain sentiment analysis, but concerns have been raised regarding their robustness and sensitivity to data distribution shift. |
| Approach: | They propose a framework CDA2 for cross-domain adaptation in low-resource sentiment analysis which employs counterfactual diffusion augmentation. |
| Outcome: | The proposed framework generates high-quality counterfactual target samples and achieves state-of-the-art performance on benchmark datasets. |
Structured Confidence–Guided Online Adaptation for LLM-based Multi-Label Classification (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) enable zero-shot and few-shot multi-label text classification . but most approaches perform static inference and degrade under streaming test data . |
| Approach: | They propose a structured confidence-guided online adaptation framework for LLM-based multi-label generation without parameter updates. |
| Outcome: | The proposed framework improves Micro-F1 and Macro-F1, with the largest gains on long-tail labels. |
From AR to Diffusion: Efficiently Adapting Large Language Models with Strictly Causal and Elastic Horizons (2026.acl-long)
Copied to clipboard
| Challenge: | Autoregressive (AR) models rely on bidirectional attention, creating a structural mismatch with pre-trained Autoregression models. |
| Approach: | They propose a framework that efficiently adapts autoregressive (AR) models to the diffusion paradigm. |
| Outcome: | The proposed framework reduces training costs by orders of magnitude while maintaining state-of-the-art performance. |
Training-Free Test-Time Contrastive Learning for Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing training-free alternatives to training-based models are static or depend on external guidance. |
| Approach: | They propose a training-free adaptation framework that enables a frozen LLM to improve online by distilling supervision from its own inference experiences. |
| Outcome: | The proposed framework outperforms existing test-time adaptation methods under online evaluation. |
SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation (2026.findings-acl)
Copied to clipboard
Shuang Cheng, Yihan Bian, Dawei Liu, Yuhua Jiang, Yihao Liu, Linfeng Zhang, Qian Yao, Zhongbo Tian, Wenhai Wang, Qipeng Guo, Kai Chen, Biqing Qi, Bowen Zhou
| Challenge: | Autoregressive (AR) language models are a dominant paradigm in the field of parallelism and non-causal modeling. |
| Approach: | They propose a blockwise discrete diffusion model that preserves AR-compatible serving while enabling parallel intra-block generation. |
| Outcome: | The proposed model achieves theoretical speedups over 5 and wall-clock speedup of 2.3 on H200 GPUs in latency-critical regimes. |
Modular Domain Adaptation (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing models for sentiment analysis and hate speech detection are difficult to account for domain shift without access to source data. |
| Approach: | They propose to treat domain adaptation as a modular process that involves separate model producers and model consumers . they demonstrate that they can independently cooperate to facilitate more accurate measurements of text . |
| Outcome: | The proposed methods improve out-of-domain accuracy on four multi-domain text classification datasets. |
Adapt-and-Distill: Developing Small, Fast and Effective Pretrained Language Models for Domains (2021.findings-acl)
Copied to clipboard
| Challenge: | Large pre-trained models suffer from domain shift and are not optimal for specific domains. |
| Approach: | They propose a general approach to developing small, fast and effective pretrained models for specific domains by adapting off-the-shelf general pretrained model and performing task-agnostic knowledge distillation in target domains. |
| Outcome: | The proposed approach achieves better performance over the BERT BASE model in domain-specific tasks while 3.3 smaller and 5.1 faster than the BRT BASE. |
Learning from LLM Agents: In-Context Generative Models for Text Casing in E-Commerce Ads (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing NER-based transformer models are expensive and lack contextual dependencies, making them less reliable when handling unseen or ad-specific terms, e.g., brand names. |
| Approach: | They propose a two-stage approach to casing correction in e-commerce ad content that leverages Chain-of-Actions to enforce content policies while accurately handling ads-specific terms. |
| Outcome: | The proposed model outperforms existing NER-based models and achieves near-LLM performance at a fraction of the cost. |