Partitioned Gradient Matching-based Data Subset Selection for Compute-Efficient Robust ASR Training

Challenge:	Existing DSS algorithms for RNN-T have a high cost and performance degradation.
Approach:	They propose a distributable DSS algorithm for RNN-T that can be used to train a subset of training data.
Outcome:	The proposed algorithm achieves between 3x to 6x speedup with only a small accuracy degradation even in settings where the training data is corrupted with noise.

Similar Papers

Overview

Challenge

Approach

Outcome

Investigating data partitioning strategies for crosslinguistic low-resource ASR evaluation (2023.eacl-main)

Copied to clipboard

Zoey Liu, Justin Spence, Emily Prud’hommeaux

Multimodality

Challenge:	Automatic speech recognition data sets include a single pre-defined test set consisting of one or more speakers whose speech never appears in the training set.
Approach:	They propose to use hold-speaker(s)-out partitioning to partition data for five languages . utterance duration and intensity are more predictive factors of variability .
Outcome:	The proposed method can produce results that do not reflect model performance on unseen data or speakers.

Reinforced Training Data Selection for Domain Adaptation (P19-1)

Copied to clipboard

Miaofeng Liu, Yan Song, Hongbin Zou, Tong Zhang

Low-Resource NLP

Challenge:	Existing approaches to learn domains with massive data are not easy to implement and require a predefined threshold.
Approach:	They propose a framework that searches for training instances relevant to the target domain and learns better representations for them.
Outcome:	The proposed framework is effective in data selection and representation, but generalized to accommodate different NLP tasks.

Copied to clipboard

Yang Zhao, Li Du, Xiao Ding, Yangou Ouyang, Hepeng Wang, Kai Xiong, Jinglong Gao, Zhouhao Sun, Dongliang Xu, Qing Yang, Dongchen Li, Bing Qin, Ting Liu

Responsible & Trustworthy NLP

Challenge:	Existing methods for selecting training data from general datasets fail to account for the joint distribution of instructions, resulting in inefficient learning and suboptimal knowledge transfer.
Approach:	They propose a method that constructs a mixed gradient-based instruction graph to capture the joint distribution and interdependencies among instructions.
Outcome:	The proposed method outperforms existing methods on domain adaptation tasks and in complex, data-scarce scenarios.

DITTO: Data-efficient and Fair Targeted Subset Selection for ASR Accent Adaptation (2023.acl-long)

Copied to clipboard

Suraj Kothawade, Anmol Mekala, D.Chandra Sekhara Hetha Havya, Mayank Kothyari, Rishabh Iyer, Ganesh Ramakrishnan, Preethi Jyothi

Multimodality

Challenge:	State-of-the-art automatic speech recognition systems exhibit disparate performance on varying speech accents.
Approach:	They propose to use submodular mutual information to find the most informative set of utterances matching a target accent within a fixed budget.
Outcome:	The proposed model is 3-5 times more label-efficient on the Indic-TTS and L2 datasets than other methods.

Robust Text Classifier on Test-Time Budgets (D19-1)

Copied to clipboard

Md Rizwan Parvez, Tolga Bolukbasi, Kai-Wei Chang, Venkatesh Saligrama

Information Extraction & Text Mining

Challenge:	Recent advances in deep neural networks (DNNs) achieve high accuracy on many text classification tasks.
Approach:	They propose a generic framework for learning a robust text classification model . they use a data aggregation method to train the classifier on a large corpus of text .
Outcome:	The proposed framework achieves consistent speedup with little degradation in accuracy on four benchmark text classification tasks.

Stratified Selective Sampling for Instruction Tuning with Dedicated Scoring Strategy (2025.findings-emnlp)

Copied to clipboard

Paramita Mirza, Lucas Weber, Fabian Küch

Responsible & Trustworthy NLP

Challenge:	Recent work shows that post-training datasets can be substantially downsampled without noticeably deteriorating performance.
Approach:	They propose a method that efficiently bins data into groups and scores difficulty using specialized models.
Outcome:	The proposed method can be efficient and universally applied to post-training datasets.

Pruning Unsafe Tickets: A Resource-Efficient Framework for Safer and More Robust LLMs (2026.acl-long)

Copied to clipboard

Wai Man Si, Mingjie Li, Michael Backes, Yang Zhang

Responsible & Trustworthy NLP

Challenge:	Empirical evaluations on ML models show substantial reductions in unsafe generations and improved robustness against jailbreak attacks.
Approach:	They propose a resource-efficient pruning framework that directly identifies unsafe behaviors while preserving model utility.
Outcome:	The proposed framework reduces unsafe generations and improves robustness against jailbreak attacks with minimal utility loss.

Efficient Contrastive Learning via Novel Data Augmentation and Curriculum Learning (2021.emnlp-main)

Copied to clipboard

Seonghyeon Ye, Jiseon Kim, Alice Oh

Responsible & Trustworthy NLP

Challenge:	Recent studies describe how to apply contrastive learning to the language domain but it is difficult to apply data augmentation methods directly to language modeling.
Approach:	They propose a memory-efficient continual pretraining method that applies contrastive learning with novel data augmentation and curriculum learning.
Outcome:	The proposed method outperforms baseline models on sentence-level tasks with only 70% of memory compared to the baseline model.

SELECting over Tokens: Curating Pre-training Data at Scale via Token Classification (2026.acl-long)

Copied to clipboard

Xin Tong, Weidong Zhang, Jiaang Li, Haibin Chen, Shilei Liu, Langming Liu, Kangtao Lv, Yujin Yuan, Wenbo Su, Bo Zheng

Information Retrieval

Challenge:	Existing pipelines rely on expert-crafted heuristic rules, which lack content-aware, fine-grained noise detection.
Approach:	They propose a framework that reframes data refinement as a highly efficient token classification task.
Outcome:	The proposed framework outperforms existing pipelines on benchmarks and is 2.5x faster at inference.

ClusterUCB: Efficient Gradient-Based Data Selection for Targeted Fine-Tuning of LLMs (2025.findings-emnlp)

Copied to clipboard

Zige Wang, Qi Zhu, Fei Mi, Minghui Xu, Ruochun Jin, Wenjing Yang

Language Models

Challenge:	Gradient-based data influence approximation is not feasible in practice.
Approach:	They propose a gradient-based data selection framework with clustering and a modified Upper Confidence Bound algorithm to solve this problem.
Outcome:	The proposed framework can achieve comparable results to the original gradient-based data selection methods while reducing computational consumption.