Data Selection Curriculum for Abstractive Text Summarization (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Abstractive Text Summarization (ATS) models are commonly trained using large-scale data that is randomly shuffled. |
| Approach: | They propose a data selection curriculum scoring system that measures the learning difficulty of an ATS model and expected performance on an instance. |
| Outcome: | The proposed system surpasses baselines on CNN/DailyMail dataset, utilizing 20% of available instances. |
Similar Papers
Active Learning for Abstractive Text Summarization via LLM-Determined Curriculum and Certainty Gain Maximization (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Abstractive text summarization (ATS) requires laborious data annotation and time-consuming model training. |
| Approach: | They propose a novel active learning framework that asks large language models to rate difficulty of instances and then uses certainty gain maximization to select instances with a distribution that aligns well with the overall distribution. |
| Outcome: | The proposed framework improves stability, effectiveness, and efficiency of abstractive text summarization backbones. |
Mitigating Data Scarceness through Data Synthesis, Augmentation and Curriculum for Abstractive Summarization (2021.findings-emnlp)
Copied to clipboard
| Challenge: | a new study explores data manipulation techniques for improving abstractive summarization models without the need for any additional data. |
| Approach: | They propose a method of data synthesis with paraphrasing, data augmentation with sample mixing and curriculum learning with new difficulty metrics based on specificity and abstractiveness. |
| Outcome: | The proposed techniques improve abstractive summarization models without additional data . the proposed techniques can be applied in isolation and when combined . |
SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization (2021.acl-short)
Copied to clipboard
| Challenge: | Experimental results show that SimCLS can improve existing top-performing models by a large margin. |
| Approach: | They propose a framework for abstractive summarization that is conceptually simple and empirically powerful. |
| Outcome: | The proposed framework improves the performance of top-performing models by a large margin against existing top-scoring systems. |
Quantifying Appropriateness of Summarization Data for Curriculum Learning (2021.eacl-main)
Copied to clipboard
| Challenge: | Summarization datasets are noisy, and summaries often do not reflect what is written in the source texts. |
| Approach: | They propose a method of curriculum learning to train summarization models from noisy data. |
| Outcome: | The proposed method improves the performance of pretrained and non-pretrained models on human evaluation. |
Does the Order of Training Samples Matter? Improving Neural Data-to-Text Generation with Curriculum Learning (2021.eacl-main)
Copied to clipboard
| Challenge: | Recent advances in data-to-text generation have been focused on curriculum learning, which is a process of presenting training data in a specific order, starting from easy examples and moving on to more difficult ones, as the learner becomes more competent. |
| Approach: | They propose to use a curriculum learning process to change the order of training samples in a model based on the model's competence to improve model performance and convergence speed. |
| Outcome: | The proposed model shows faster convergence speed and reduced training time by 38.7% and performance by 4.84 BLEU. |
Your Pretrained Model Tells the Difficulty Itself: A Self-Adaptive Curriculum Learning Paradigm for Natural Language Understanding (2025.acl-srw)
Copied to clipboard
| Challenge: | Existing curriculum learning approaches rely on manually defined difficulty metrics which may not accurately reflect the model’s own perspective. |
| Approach: | They propose a self-adaptive curriculum learning paradigm that prioritizes fine-tuning examples based on difficulty scores predicted by pre-trained language models (PLMs) they evaluate four datasets covering binary and multi-class classification tasks. |
| Outcome: | The proposed model leads to faster convergence and improved performance compared to standard random sampling. |
Active Learning for Abstractive Text Summarization (2022.findings-emnlp)
Copied to clipboard
Akim Tsvigun, Ivan Lysenko, Danila Sedashov, Ivan Lazichny, Eldar Damirov, Vladimir Karlov, Artemy Belousov, Leonid Sanochkin, Maxim Panov, Alexander Panchenko, Mikhail Burtsev, Artem Shelmanov
| Challenge: | Abstractive text summarization (ATS) requires a long document and short summaries. |
| Approach: | They propose a query strategy for AL in abstractive text summarization that uses uncertainty estimation to reduce model performance. |
| Outcome: | The proposed query strategy improves ROUGE and consistency scores for annotated datasets . it also increases the performance of the model, compared to passive annotation. |
Data Selection Curriculum for Neural Machine Translation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Neural Machine Translation models are typically trained on heterogeneous data that are concatenated and randomly shuffled. |
| Approach: | They propose a two-stage curriculum training framework where a NMT model is fine-tuned on subsets of data, selected by deterministic scoring and online scoring. |
| Outcome: | The proposed framework improves on six language pairs comprising low- and high-resource languages and shows up to +2.2 BLEU improvement and faster convergence. |
Make The Most of Prior Data: A Solution for Interactive Text Summarization with Preference Feedback (2022.findings-naacl)
Copied to clipboard
Duy-Hung Nguyen, Nguyen Viet Dung Nghiem, Bao-Sinh Nguyen, Dung Tien Tien Le, Shahab Sabahi, Minh-Tien Nguyen, Hung Le
| Challenge: | a framework to train summarization models with preference feedback is proposed . human-in-the-loop (HITL) allows humans to actively participate in supervising AI systems . |
| Approach: | They propose a framework to train summarization models with preference feedback interactively. |
| Outcome: | The proposed framework improves ROUGE scores and sample-efficiency in active, few-shot and online settings. |
Objective Function Learning to Match Human Judgements for Optimization-Based Summarization (N18-2)
Copied to clipboard
| Challenge: | In previous work on summarization, the objective function is based on ad-hoc assumptions about which quality aspects of a summary are relevant. |
| Approach: | They learn a summary-level scoring function including human judgments as supervision and automatically generated data as regularization. |
| Outcome: | The proposed method performs well across automatic and manual evaluations. |