Dynamic Data Selection for Curriculum Learning via Ability Estimation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing curriculum learning methods rely on heuristics to estimate difficulty of data . a major drawback is that they ignore competency of the model during training . |
| Approach: | They propose replacing difficulty heuristics with learned difficulty parameters . they propose a strategy that probes model ability at each training epoch . |
| Outcome: | The proposed strategy outperforms heuristic-based learning models on the GLUE classification tasks. |
Similar Papers
Dynamic Augmentation Data Selection for Few-shot Text Classification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Data augmentation is a popular method for fine-tuning pre-trained language models to increase model robustness and performance. |
| Approach: | They propose a dynamic data selection method to select effective augmentation data from different augmentation sources according to the model’s learning stage by identifying a set of augmentation samples that optimally facilitates the learning process of the most current model. |
| Outcome: | The proposed method outperforms strong baselines on a variety of sentence classification tasks. |
HuCurl: Human-induced Curriculum Discovery (2023.acl-long)
Copied to clipboard
| Challenge: | Existing curriculum learning frameworks can be used to discover effective curricula for NLP tasks based on prior knowledge about sample difficulty. |
| Approach: | They propose a framework for curriculum learning based on prior knowledge about sample difficulty. |
| Outcome: | The proposed framework outperforms existing curriculum learning approaches on several NLP tasks and can prune and weight samples for better learning. |
An Effective Incorporating Heterogeneous Knowledge Curriculum Learning for Sequence Labeling (2025.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to enhance sequence labeling models require data heterogeneity and additional modules. |
| Approach: | They propose a dual-stage curriculum learning framework specifically designed for sequence labeling tasks. |
| Outcome: | The proposed model improves training and accelerates training, mitigating the slow training issue of complex models. |
Curriculum Learning for Graph Neural Networks: A Multiview Competence-based Approach (2023.acl-long)
Copied to clipboard
| Challenge: | Existing curriculum learning approaches often employ a single criterion of difficulty in their training paradigms. |
| Approach: | They propose a new approach that builds on graph complexity formalisms and model competence during training. |
| Outcome: | The proposed approach improves learning efficiency on real-world link prediction and node classification tasks. |
Beyond Random Sampling: Efficient Language Model Pretraining via Curriculum Learning (2026.eacl-long)
Copied to clipboard
| Challenge: | Curriculum learning has improved efficiency across machine learning domains, but remains underexplored for language model pretraining. |
| Approach: | They present a systematic investigation of curriculum learning in LLM pretraining . they use vanilla curriculum learning, pacing-based sampling, and interleaved curricula . |
| Outcome: | The proposed framework accelerates convergence in early and mid-training phases, reducing training steps by 18-45% to reach baseline performance. |
Teaching According to Talents! Instruction Tuning LLMs with Competence-Aware Curriculum Learning (2025.findings-emnlp)
Copied to clipboard
Yangning Li, Tingwei Lu, Yinghui Li, Yankai Chen, Wei-Chieh Huang, Wenhao Jiang, Hui Wang, Hai-Tao Zheng, Philip S. Yu
| Challenge: | Efficient instruction tuning aims to enhance the ultimate performance of large language models (LLMs) current methods suffer from the curriculum rigidity, resulting in a fixed and potentially sub-optimal learning trajectory. |
| Approach: | a framework for efficient instruction tuning is proposed to address the issue of curriculum rigidity . current methods rely on static heuristic difficulty metrics and fail to adapt to evolving capabilities . |
| Outcome: | Efficient instruction tuning aims to enhance the ultimate performance of large language models . current methods suffer from the curriculum rigidity, resulting in a fixed learning trajectory . |
In-sample Curriculum Learning by Sequence Completion for Natural Language Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing work on curriculum learning rely on task-specific expertise and cannot generalize to different tasks. |
| Approach: | They propose to do in-sample curriculum learning for natural language generation tasks using human-crafted rules and a numeric score for each sample based on domain expertise to rank the model. |
| Outcome: | The proposed learning strategy generalizes well to different tasks and achieves significant improvements over baselines. |
Generic and Trend-aware Curriculum Learning for Relation Extraction (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing curriculum learning approaches for relation extraction are lacking in text graphs. |
| Approach: | They propose a generic and trend-aware curriculum learning approach that integrates textual and structural information in text graphs for relation extraction between entities. |
| Outcome: | The proposed model shows improvement over state-of-the-art methods across several datasets. |
Dynamic Curriculum Learning for Low-Resource Neural Machine Translation (2020.coling-main)
Copied to clipboard
| Challenge: | Recent work on neural machine translation (NMT) has demonstrated impressive performance improvements and became the de-facto standard. |
| Approach: | They propose a dynamic curriculum learning method to reorder training samples in training using a Transformer-based system. |
| Outcome: | The proposed method outperforms baselines on three low-resource machine translation benchmarks and different sized data of WMT’16 En-De. |
Reinforcement Learning based Curriculum Optimization for Neural Machine Translation (N19-1)
Copied to clipboard
| Challenge: | a heterogeneous training dataset can vary in characteristics such as domain, translation quality, and degree of difficulty. |
| Approach: | They propose to use reinforcement learning to learn an optimal curriculum for NMT training . they find it can beat uniform baselines and hand-designed, state-of-the-art curricula . |
| Outcome: | The proposed approach beats baselines and hand-designed curricula on English-to-French datasets. |