Effects of diversity incentives on sample diversity and downstream model performance in LLM-based text augmentation (2024.acl-long)
Copied to clipboard
| Challenge: | generative large language models (LLMs) have found their application in data augmentation tasks, where small numbers of text samples are paraphrased and then used to fine-tune downstream models. |
| Approach: | They propose to use taboo words, hints by previous outlier solutions, and chaining on previous outliest solutions to augment text datasets as part of instructions to LLMs augmenting text dataset. |
| Outcome: | The proposed methods increase diversity of generated texts, but performance is highest with hints. |
Similar Papers
Diversity-oriented Data Augmentation with Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing data augmentation methods focus on increasing sample numbers while neglecting sample distribution diversity, which can lead to model overfitting. |
| Approach: | They propose a data augmentation framework that focuses on sample distribution diversity and trains a large language model as a diverse paraphraser. |
| Outcome: | The proposed framework achieves an average performance gain of 10.52% surpassing the runner-up baseline with more than three percentage points. |
LLMs vs Established Text Augmentation Techniques for Classification: When do the Benefits Outweight the Costs? (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent studies have compared LLM-based augmentations with established methods, but the results are contradictory. |
| Approach: | They compare the performance of LLM-based augmentation methods with established ones . they found that LLMs are worthy of deployment only when very small number of seeds is used . |
| Outcome: | The proposed methods are worthy of deployment only when very small number of seeds is used. |
Use Random Selection for Now: Investigation of Few-Shot Selection Strategies in LLM-based Text Augmentation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | generative large language models are increasingly used for data augmentation tasks . text samples are mostly selected randomly and a comprehensive overview of other sample selection strategies is lacking. |
| Approach: | They compare random sample selection strategies and random sample sampling strategies to evaluate their effects in a low-resource setting. |
| Outcome: | The proposed model performance improvements are compared with other sample selection strategies. |
Increasing Diversity While Maintaining Accuracy: Text Data Generation with Large Language Models and Human Interventions (2023.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) can be used to generate text data for training and evaluating other models. |
| Approach: | They propose to use logit suppression and temperature sampling to diversify text generation but at the cost of data accuracy. |
| Outcome: | The proposed approach can increase diversity but at the cost of data accuracy. |
Synthetic Eggs in Many Baskets: The Impact of Synthetic Data Diversity on LLM Fine-Tuning (2026.findings-acl)
Copied to clipboard
| Challenge: | Increasing demand for training data is causing language models to be trained on synthetic data, a new study finds . fine-tuning models on synthetic datasets reduces self-preference bias . |
| Approach: | They investigate the impact of diversity of synthetic data on fine-tuned large language models. |
| Outcome: | The proposed model can mitigate distribution collapse, maintain diversity of output distribution, and reduce self-preference bias. |
Scaling Data Diversity for Fine-Tuning Language Models in Human Alignment (2024.lrec-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) can reveal toxic or offensive content inadvertently or intentionally. |
| Approach: | They propose to control the diversity of both sides according to the number of samples for fine-tuning, which can directly reflect their impact. |
| Outcome: | The proposed approach improves the performance of large language models after fine-tuning. |
Empowering Large Language Models for Textual Data Augmentation (2024.findings-acl)
Copied to clipboard
| Challenge: | True. True. False |
| Approach: | False slants are proposed to generate a large pool of augmentation instructions and select the most suitable task-informed instructions. |
| Outcome: | False omissions: the proposed approach consistently generates augmented data with better quality compared to non-LLM and LLM-based data augmentation methods. |
Understanding and Mitigating Bias Inheritance in LLM-based Data Augmentation on Downstream Tasks (2026.acl-long)
Copied to clipboard
Miaomiao Li, Hao Chen, Yang Wang, Tingyuan Zhu, Weijia Zhang, Kaijie Zhu, Kam-Fai Wong, Jindong Wang
| Challenge: | Generating synthetic datasets via large language models (LLMs) has emerged as promising approach to improve LLM performance. |
| Approach: | They propose three mitigation strategies to mitigate bias inheritance in LLMs by analyzing real and LLM-augmented data. |
| Outcome: | The proposed methods can work differently on different tasks and biases. |
Cross-Lingual Transfer of Debiasing and Detoxification in Multilingual LLMs: An Extensive Investigation (2025.findings-acl)
Copied to clipboard
| Challenge: | Prior work has shown that finetuning on specialized datasets can mitigate this behavior, and doing so in English can transfer to other languages. |
| Approach: | They propose to fine tune generative large language models to provide safe responses to harmful user input and to use direct preference optimization to mitigate toxicity. |
| Outcome: | The proposed models show that finetuning on specialized datasets reduces biases but also produces fluent and diverse text in non-English languages. |
ROBBIE: Robust Bias Evaluation of Large Generative Language Models (2023.emnlp-main)
Copied to clipboard
David Esiobu, Xiaoqing Tan, Saghar Hosseini, Megan Ung, Yuchen Zhang, Jude Fernandes, Jane Dwivedi-Yu, Eleonora Presani, Adina Williams, Eric Smith
| Challenge: | generative large language models (LLMs) are becoming more performant and prevalent . we need tools to measure and improve their fairness, authors say . |
| Approach: | They propose to compare 6 different prompt-based bias and toxicity metrics across 12 demographic axes and 5 families of generative large language models. |
| Outcome: | The proposed model can be tested on more datasets to better characterize and mitigate biases . the study compared 6 prompt-based bias and toxicity metrics across 12 demographic axes and 5 families of generative large language models. |