Curriculum Consistency Learning for Conditional Sentence Generation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Consistency learning (CL) has proven to be a valuable technique for improving the robustness of conditional sentence generation models. |
| Approach: | They propose a strategy that guides models to learn consistency in alignment with their current capacity to differentiate between features. |
| Outcome: | The proposed strategy delivers +2.0 accuracy point improvement compared with vanilla IT and +0.7 COMET scores over traditional CL methods in MT tasks. |
Similar Papers
Improving the Robustness of Large Language Models via Consistency Alignment (2024.lrec-main)
Copied to clipboard
Yukun Zhao, Lingyong Yan, Weiwei Sun, Guoliang Xing, Shuaiqiang Wang, Chong Meng, Zhicong Cheng, Zhaochun Ren, Dawei Yin
| Challenge: | Large language models have shown tremendous success in following user instructions and generating helpful responses, but their robustness is still far from optimal. |
| Approach: | They propose a two-stage training framework that helps a model generalize on following instructions via similar instruction augmentations. |
| Outcome: | The proposed training framework improves diversity and aligns the model with human expectations by differentiating subtle differences in similar responses. |
Keeping Consistency of Sentence Generation and Document Classification with Multi-Task Learning (D19-1)
Copied to clipboard
| Challenge: | Existing automated generation of articles' characteristics is inconsistent if they are generated individually. |
| Approach: | They propose a multi-task learning model with a shared encoder and multiple decoders for each task. |
| Outcome: | The proposed model generates more consistent headlines, key phrases and categories . it outperforms baseline model on the ROUGE scores and generates fluent headlines . |
TrueTeacher: Learning Factual Consistency Evaluation with Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for evaluating factual consistency are limited in their effectiveness. |
| Approach: | They propose a method for generating synthetic data by annotating diverse model-generated summaries using a LLM. |
| Outcome: | The proposed method outperforms state-of-the-art models and the LLM teacher on TRUE benchmarks. |
In-sample Curriculum Learning by Sequence Completion for Natural Language Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing work on curriculum learning rely on task-specific expertise and cannot generalize to different tasks. |
| Approach: | They propose to do in-sample curriculum learning for natural language generation tasks using human-crafted rules and a numeric score for each sample based on domain expertise to rank the model. |
| Outcome: | The proposed learning strategy generalizes well to different tasks and achieves significant improvements over baselines. |
Token-wise Curriculum Learning for Neural Machine Translation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing curriculum learning approaches to Neural Machine Translation (NMT) require sampling sufficient amounts of “easy” samples from training data at the early stage of training. |
| Approach: | They propose a token-wise curriculum learning approach that creates sufficient amounts of easy samples from training data. |
| Outcome: | The proposed approach outperforms baselines on five language pairs on low-resource languages. |
Competence-based Curriculum Learning for Multilingual Machine Translation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing multilingual machine translation models face an imbalance problem due to the different learning competencies of different languages. |
| Approach: | They propose Competence-based Curriculum Learning for Multilingual Machine Translation, named CCL-M, to help schedule the high resource languages and low resource languages. |
| Outcome: | The proposed approach achieves a steady and significant performance gain compared to the previous state-of-the-art approach on the TED talks dataset. |
Consistent CCG Parsing over Multiple Sentences for Improved Logical Reasoning (N18-2)
Copied to clipboard
| Challenge: | Existing methods to recognize textual entailment use a CCG parser to process sentences . failing to recognize the similar syntactic structure results in inconsistent argument structures . |
| Approach: | They propose to extend existing CCG parsers to parse sentences consistently . they use an inter-sentence modeling with Markov Random Fields to achieve this . |
| Outcome: | The proposed method improves on English and Japanese languages. |
Your Pretrained Model Tells the Difficulty Itself: A Self-Adaptive Curriculum Learning Paradigm for Natural Language Understanding (2025.acl-srw)
Copied to clipboard
| Challenge: | Existing curriculum learning approaches rely on manually defined difficulty metrics which may not accurately reflect the model’s own perspective. |
| Approach: | They propose a self-adaptive curriculum learning paradigm that prioritizes fine-tuning examples based on difficulty scores predicted by pre-trained language models (PLMs) they evaluate four datasets covering binary and multi-class classification tasks. |
| Outcome: | The proposed model leads to faster convergence and improved performance compared to standard random sampling. |
Curriculum Learning for Domain Adaptation in Neural Machine Translation (N19-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) performance drops when domains do not match and in-domain training data is scarce. |
| Approach: | They propose a curriculum learning approach to adapt generic neural machine translation models to a specific domain. |
| Outcome: | The proposed approach outperforms unadapted and adapted baselines in two domains and two language pairs. |
Be Consistent! Improving Procedural Text Comprehension using Label Consistency (N19-1)
Copied to clipboard
| Challenge: | Existing systems for procedural text comprehension still struggle with this task . evaluative work shows that consistent predictions from multiple entities can improve performance . |
| Approach: | They propose a framework that leverages label consistency during training to improve prediction performance. |
| Outcome: | The proposed framework significantly improves prediction performance over previous state-of-the-art systems on a standard benchmark dataset for procedural text, ProPara. |