Unsupervised Text Style Transfer with Padded Masked Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for style transfer are difficult to obtain and require substantial amounts of parallel training examples to work well. |
| Approach: | They propose an unsupervised method for style transfer that uses masked language models to find the text spans where the two models disagree the most in terms of likelihood. |
| Outcome: | The proposed method performs competitively in a fully unsupervised setting and improves accuracy in low-resource settings by over 10 percentage points when pre-training on silver training data generated by Masker. |
Similar Papers
Text Style Transferring via Adversarial Masking and Styled Filling (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for text style transfer suffer from two challenges: the word masking procedure may mistakenly remove unexpected words and the selected words in the word filling procedure lack diversity and semantic consistency. |
| Approach: | They propose a style transfer model with adversarial masking and styled filling techniques to solve these challenges. |
| Outcome: | The proposed model performs well on two benchmark text style transfer data sets. |
TextSETTR: Few-Shot Text Style Extraction and Tunable Targeted Restyling (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for text style transfer require style-labeled training data, but use only labeled data at inference time. |
| Approach: | They propose a method that uses readily-available unlabeled text to train style transfer . they use a style vector to condition a decoder to perform style transfer using unlabelled text . |
| Outcome: | The proposed method is competitive on sentiment transfer, even compared to models trained fully on labeled data. |
Text Style Transfer for Bias Mitigation using Masked Language Modeling (2022.naacl-srw)
Copied to clipboard
| Challenge: | Various research findings have concluded that biased textual data has significant effects on target demographic groups. |
| Approach: | They propose a text-style transfer model that can be trained on non-parallel data and be used to automatically mitigate bias in textual data. |
| Outcome: | The proposed model improves on limitations of existing methods while maintaining good style transfer accuracy. |
Text Detoxification using Large Pre-trained Neural Models (2021.emnlp-main)
Copied to clipboard
David Dale, Anton Voronov, Daryna Dementieva, Varvara Logacheva, Olga Kozlova, Nikita Semenov, Alexander Panchenko
| Challenge: | Existing studies on text detoxification cast this task as style transfer . text detox requires better preservation of the original meaning, authors argue . |
| Approach: | They propose two unsupervised methods for eliminating toxicity in text . they use a paraphraser guided by style-trained language models to keep the text content . |
| Outcome: | The proposed methods yield new SOTA results. |
How Positive Are You: Text Style Transfer using Adaptive Style Embedding (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches for unsupervised text style transfer are disentanglement between content and style. |
| Approach: | They propose to separate a model with a sentence reconstruction module and a style module to improve model architecture. |
| Outcome: | The proposed method improves style transfer performance and content preservation . the proposed method can be used to modify a sentence with a specified style attribute . |
Unsupervised Subtitle Segmentation with Masked Language Models (2023.acl-short)
Copied to clipboard
| Challenge: | Existing methods to improve subtitle segmentation are based on character counting and linguistically correct segmentation. |
| Approach: | They propose a method where subtitle breaks are predicted according to likelihood of punctuation . their approach is highly portable across languages and domains . |
| Outcome: | The proposed method obtained competitive results in terms of segmentation accuracy across metrics while also fully preserving the original text and complying with length constraints. |
Exploring Unsupervised Pretraining Objectives for Machine Translation (2021.findings-acl)
Copied to clipboard
| Challenge: | Unsupervised cross-lingual pretraining has significantly reduced the need for large parallel data. |
| Approach: | They compare unsupervised cross-lingual pretraining with masking and reconstructing inputs in the decoder to produce real sentences. |
| Outcome: | The proposed methods produce inputs resembling real (full) sentences, by reordering and replacing words based on their context. |
Fighting Offensive Language on Social Media with Unsupervised Text Style Transfer (P18-2)
Copied to clipboard
| Challenge: | Existing methods to tackle the problem of offensive language in social media are based on machine learning. |
| Approach: | They propose a method for training encoder-decoders using non-parallel data . they use a collaborative classifier, attention and the cycle consistency loss . |
| Outcome: | The proposed method outperforms state-of-the-art text style transfer systems on Twitter and Reddit . it produces reliable non-offensive transferred sentences, the authors show . |
Prefix-Tuning Based Unsupervised Text Style Transfer (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Unsupervised text style transfer is an important task in computer vision and natural language processing. |
| Approach: | They propose a method that uses pre-trained large language models to train a generative model that can alter the style of the input sentence without using any parallel data. |
| Outcome: | The proposed method outperforms the state-of-the-art methods on well-known datasets. |
Mask-Predict: Parallel Decoding of Conditional Masked Language Models (D19-1)
Copied to clipboard
| Challenge: | a masked language model is used to train a model to predict subsets of mangled words . a parallel decoding algorithm can be used to generate translations in a constant number of iterations. |
| Approach: | They propose a model and a parallel decoding algorithm which train a machine to predict any subset of target words . they introduce conditional masked language models (CMLMs) which are trained with a mangled language model objective . |
| Outcome: | The proposed model improves state-of-the-art performance levels for non-autoregressive and parallel decoding models by over 4 BLEU on average. |