Generating Artificial Texts as Substitution or Complement of Training Data (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing approaches to generate text for supervised learning tasks use transformers to generate learning data. |
| Approach: | They propose to use transformers to generate supervised learning data for supervised machine learning tasks and propose to train a neural language model trained on the original training texts. |
| Outcome: | The proposed models can be used in a certain extend but require pre-processing to significantly improve performance. |
Similar Papers
A Benchmark Corpus for the Detection of Automatically Generated Text in Academic Publications (2022.lrec-1)
Copied to clipboard
| Challenge: | Automated text generation has achieved performance levels that make the generated text almost indistinguishable from those written by humans. |
| Approach: | They propose to use a completely synthetic dataset and a partial text substitution dataset to evaluate the quality of the generated research content. |
| Outcome: | The proposed datasets compare the generated texts to aligned original texts using fluency metrics such as BLEU and ROUGE. |
Neural Data-to-Text Generation with LM-based Text Augmentation (2021.eacl-main)
Copied to clipboard
| Challenge: | Neural data-to-text generation is a difficult task for many new applications because of a lack of training data. |
| Approach: | They propose a few-shot approach that augments the data available for training by generating new text samples based on replacing specific values by alternative ones from the same category and pairing the new text with data samples. |
| Outcome: | The proposed approach outperforms fully supervised sequence-to-sequence models with less than 10% of the training set on both datasets. |
Benefits of Data Augmentation for NMT-based Text Normalization of User-Generated Content (D19-55)
Copied to clipboard
| Challenge: | Social media texts are considered important language resources for several NLP tasks, but their use of non-standard words makes it difficult to process and analyze UGC. |
| Approach: | They propose to use a Neural Machine Translation approach to normalize lexical variants to their canonical forms to overcome performance drop in UGC. |
| Outcome: | The proposed approach overcomes a data bottleneck in Dutch, a low-resource language. |
Controlled Text Generation for Data Augmentation in Intelligent Artificial Agents (D19-56)
Copied to clipboard
| Challenge: | Data availability is a bottleneck during early stages of development of new capabilities for intelligent artificial agents. |
| Approach: | They propose to use conditional variational auto-encoders to augment training data of a popular commercial artificial agent with a small set of phrase templates to generate new semantically similar phrases. |
| Outcome: | The proposed approach outperforms the previous controlled text generation techniques with limited data and significantly outperformed the previous methods. |
Data Augmentation for Text Generation Without Any Augmented Data (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for data augmentation need to define or choose proper data mapping functions to create augmented samples. |
| Approach: | They propose to use data mapping functions to augment text samples without using specific mapping functions. |
| Outcome: | The proposed approach can approximate or even surpass popular data augmentation methods on two text generation tasks with a convergence rate guarantee. |
Understanding the Influence of Synthetic Data for Text Embedders (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in general purpose text embedders have been driven by training on synthetic training data. |
| Approach: | They propose to use GPT-4 to produce high quality synthetic data that expands existing training datasets for embeddings to new tasks. |
| Outcome: | The proposed dataset is high quality and leads to consistent improvements in performance. |
Rethinking Data Augmentation in Text-to-text Paradigm (2022.coling-1)
Copied to clipboard
| Challenge: | Existing approaches to augment training data are limited or marginal, or even diminishing or adverse especially given original training corpus is relatively sufficient or the backbone classifiers are PLM based. |
| Approach: | They propose to integrate text-to-text language models and construct a new two-phase framework for augmentation using two novel schemes. |
| Outcome: | The proposed framework synthesizes new samples benefiting from the knowledge learned from pre-trained language models on two public classification datasets and shows remarkable gains. |
On Evaluation Protocols for Data Augmentation in a Limited Data Scenario (2025.coling-main)
Copied to clipboard
| Challenge: | Textual data augmentation (DA) is a prolific field of study where novel techniques to create artificial data are regularly proposed. |
| Approach: | They propose to use textual data augmentation (DA) to generate new sentences for text classification in a limited data setting. |
| Outcome: | The proposed methods perform better on small data settings and on large datasets, but they are not as effective on large data sets. |
Faithful Low-Resource Data-to-Text Generation through Cycle Training (2023.acl-long)
Copied to clipboard
| Challenge: | Methods to generate text from structured data have advanced significantly in recent years, but can fail to produce output faithful to the input data, especially on out-of-domain data. |
| Approach: | They evaluate the effectiveness of cycle training by using two models which are inverses of each other to generate text from structured data and one which generates the structured data from natural language text. |
| Outcome: | The proposed approach achieves nearly the same performance as fully supervised approaches on the WebNLG, E2E, WTQ, and WSQL datasets. |
As Good as New. How to Successfully Recycle English GPT-2 to Make Models for Other Languages (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing pre-trained language models are limited in their ability to train for English, which is a problem for many other languages. |
| Approach: | They propose to adapt existing generative language models to new languages by retraining lexical embeddings without tuning the Transformer layers. |
| Outcome: | The proposed method achieves lexical embeddings for Italian and Dutch that are aligned with the original English lexicals. |