Papers with task-agnostic
Adversarial Text Normalization (2022.naacl-industry)
Copied to clipboard
| Challenge: | Text-based adversarial attacks are becoming more commonplace and accessible to general internet users. |
| Approach: | They propose a method that restores baseline performance on attacked content with low computational overhead. |
| Outcome: | The proposed method restores baseline performance on attacked content with low computational overhead. |
TransPrompt: Towards an Automatic Transferable Prompting Framework for Few-shot Text Classification (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies show that prompts improve performance of large pre-trained language models for few-shot text classification. |
| Approach: | They propose a prompt-based framework for few-shot learning that captures cross-task transferable knowledge and uses two de-biasing techniques to make it more task-agnostic and unbiased . |
| Outcome: | The proposed framework outperforms strong baselines over multiple NLP tasks and datasets. |
DISCO: Distilling Counterfactuals with Large Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | high-quality counterfactual data is scarce for most tasks and not easily generated at scale. |
| Approach: | They propose a method for automatically generating high-quality counterfactual data at scale . they use a large general language model to generate phrasal perturbations and filter them . |
| Outcome: | The proposed method is task-agnostic and can be applied to the task of natural language inference. |
Domain-Agnostic Adapter Architecture for Deception Detection: Extensive Evaluations with the DIFrauD Benchmark (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing research focuses predominantly on specific fields, which results in the need for clarity on linguistic markers associated with deception. |
| Approach: | They propose a domain-independent fraud detection benchmark with 100,000 honest and misleading statements in seven domains and a parameter-efficient finetuning adapter to improve tuning methods. |
| Outcome: | The proposed adapter outperforms all competition on the DIFrauD benchmark and is able to predict the performance of the proposed model. |
Train No Evil: Selective Masking for Task-Guided Pre-Training (2020.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models can't capture domain-specific and task-specific patterns because of the task-agnostic pre-training stage. |
| Approach: | They propose a task-guided pre-training stage with selective masking between general pre-train and fine-tuning to learn domain-specific patterns. |
| Outcome: | The proposed method can achieve comparable or even better performance with less than 50% of computation cost. |
SSR: Utilizing Simplified Stance Reasoning Process for Robust Stance Detection (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods for stance detection are task-agnostic, which fail to utilize task knowledge to better discriminate between genuine and bias features. |
| Approach: | They propose to incorporate stance reasoning process as task knowledge to aid in learning genuine features without using targets. |
| Outcome: | The proposed model achieves better performance than previous task-agnostic debiasing methods on new test sets. |
A Unified Agentic Framework for Evaluating Conditional Image Generation (2025.acl-long)
Copied to clipboard
Jifang Wang, Yangxue Yangxue, Longyue Wang, Zhenran Xu, Yiyu Wang, Yaowei Wang, Weihua Luo, Kaifu Zhang, Baotian Hu, Min Zhang
| Challenge: | Conditional image generation is a popular and personalization-oriented task, but there are challenges in developing task-agnostic, reliable, and explainable evaluation metrics. |
| Approach: | They propose a unified agentic framework for comprehensive evaluation of conditional image generation tasks. |
| Outcome: | The proposed framework achieves a high correlation with human assessments on seven prominent image generation tasks. |
To BERT or Not to BERT: Comparing Task-specific and Task-agnostic Semi-Supervised Approaches for Sequence Tagging (2020.emnlp-main)
Copied to clipboard
Kasturi Bhattacharjee, Miguel Ballesteros, Rishita Anubhai, Smaranda Muresan, Jie Ma, Faisal Ladhak, Yaser Al-Onaizan
| Challenge: | Using large amounts of unlabeled data to improve performance has become the foundation for many natural language processing tasks. |
| Approach: | They propose a task-specific semi-supervised approach that uses unlabeled data in a more task-agnostic manner. |
| Outcome: | The proposed approach achieves similar performance to BERT on a set of sequence tagging tasks with less financial and environmental impact. |
Meta Distant Transfer Learning for Pre-trained Language Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Notable PLMs are available for text classification tasks, but performance of PLM on downstream tasks may be limited by the availability of training set. |
| Approach: | They propose a meta-learning framework to learn the transferable knowledge across tasks using PLMs. |
| Outcome: | The proposed framework outperforms baselines on seven datasets and is task-agnostic and unbiased. |
Dropping Experts, Recombining Neurons: Retraining-Free Pruning for Sparse Mixture-of-Experts LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Sparse Mixture-of-Experts (SMoE) architectures require loading all expert parameters . previous work focused on expert pruning and merging but focused on neuron-level structure . |
| Approach: | They propose a task-agnostic framework for expert pruning and reconstruction . it prunes redundant experts using router statistics, then decomposes them into neuron-level expert segments . |
| Outcome: | The proposed framework reduces the number of experts and memory usage, making it easier to deploy. |
Cross-Modal Attribute Insertions for Assessing the Robustness of Vision-and-Language Learning (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to model multimodal data do not leverage cross-modal information . augmenting input text using cross-module attribute insertions results in poor performance . |
| Approach: | They propose a multimodal deep learning approach that adds visual attributes to inputs to enhance model robustness. |
| Outcome: | The proposed approach is modular, controllable, and task-agnostic. |
ORTicket: Let One Robust BERT Ticket Transfer across Different Tasks (2024.lrec-main)
Copied to clipboard
| Challenge: | Pretrained language models are susceptible to subtle perturbations and require multiple adversarial training during fine-tuning to improve their robustness. |
| Approach: | They propose a novel adversarial defense method ORTicket that fine-tunes a model for downstream tasks. |
| Outcome: | The proposed method achieves comparable robustness to other defense methods while maintaining the efficiency of fine-tuning. |
Powering Verifiable Learning via Automated Evolutionary Data Synthesis (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to building generalizable verifiable data are task-specific and lack a principled, universal evaluator of verifikatability. |
| Approach: | They propose a task-agnostic, strategy-guided, executably-checkable data synthesis framework that synthesizes problems, diverse candidate solutions and verification artifacts from a single source. |
| Outcome: | The proposed framework synthesizes problems, candidates, and verification artifacts from human-annotated and strategy-induced checks and iteratively discovers strategies. |
Task-agnostic Distillation of Encoder-Decoder Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing distillation methods that focus on encoder-only LMs fail to handle the distillation of encoder decoder LM. |
| Approach: | They propose a method that finetunes pretrained language models (LMs) they propose 'MiniEnD' that allows for task-agnostic distillation of LMs. |
| Outcome: | The proposed distillation method is generally effective and competitive compared to other alternatives. |
From Local to Global: Revisiting Structured Pruning Paradigms for Large Language Models (2026.acl-long)
Copied to clipboard
Ziyan Wang, Enmao Diao, Qi Le, Pu Wang, Minwoo Lee, Shu-ping Yeh, Evgeny Stupachenko, Hao Feng, Li Yang
| Challenge: | Structured pruning is a practical approach to deploying large language models (LLMs) but it fails to capitalize on modest task-specific calibration signals, causing limited downstream gains. |
| Approach: | They propose a method that removes attention heads and MLP channels using loss-based important scores . they use perplexity for language modeling and a margin-based objective for decision-style tasks . |
| Outcome: | The proposed method lowers perplexity and improves accuracy at higher sparsity . it also stabilizes accuracy and mitigates perxity collapse without fine-tuning . |