| Challenge: | Experimentally, we find that the proposed models consistently outperform models that encapsulate single-style or average-style language generation capabilities. |
| Approach: | They propose a family of model architectures capable of capturing both generic language characteristics via shared model parameters, as well as particular style characteristics via private model parameters. |
| Outcome: | The proposed models outperform models that encapsulate single-style or average-style language generation capabilities. |
Similar Papers
TextSETTR: Few-Shot Text Style Extraction and Tunable Targeted Restyling (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for text style transfer require style-labeled training data, but use only labeled data at inference time. |
| Approach: | They propose a method that uses readily-available unlabeled text to train style transfer . they use a style vector to condition a decoder to perform style transfer using unlabelled text . |
| Outcome: | The proposed method is competitive on sentiment transfer, even compared to models trained fully on labeled data. |
Style Vectors for Steering Generative Large Language Models (2024.findings-eacl)
Copied to clipboard
Kai Konen, Sophie Jentzsch, Diaoulé Diallo, Peer Schütt, Oliver Bensch, Roxanne El Baff, Dominik Opitz, Tobias Hecking
| Challenge: | Large language models (LLMs) can be trained on vast corpora and can generate text in a nuanced and parameterisable way. |
| Approach: | They propose to add style vectors to the activations of hidden layers during text generation to steer output towards specific styles. |
| Outcome: | The proposed approach differs from prompt engineering in that it can be nuanced and parameterisable. |
Prefix-Tuning Based Unsupervised Text Style Transfer (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Unsupervised text style transfer is an important task in computer vision and natural language processing. |
| Approach: | They propose a method that uses pre-trained large language models to train a generative model that can alter the style of the input sentence without using any parallel data. |
| Outcome: | The proposed method outperforms the state-of-the-art methods on well-known datasets. |
Towards Modeling the Style of Translators in Neural Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | a key ingredient of neural machine translation is the use of large datasets with different but consistent translation styles . however, the models do not capture the variety of translators' styles from the data . a recent study shows that style-augmented models can capture the style variations of translator . |
| Approach: | They propose to augment a neural machine translation model with translator information . they use TED talk datasets to model and control translator-related stylistic variations . |
| Outcome: | The proposed models capture the style variations of translators and generate translations with different styles on new data. |
Meta-Tuning LLMs to Leverage Lexical Knowledge for Generalizable Language Style Understanding (2024.acl-long)
Copied to clipboard
| Challenge: | Existing large language models struggle to capture some language styles without fine-tuning. |
| Approach: | They propose to meta-trained LLMs based on representative lexicons to recognize new styles they have not been fine-tuned on. |
| Outcome: | The proposed method improves zero-shot transfer across styles on 13 established and 63 novel tasks generated with LLMs. |
Text Generation with Exemplar-based Adaptive Decoding (N19-1)
Copied to clipboard
| Challenge: | Empirical results show that the proposed model achieves strong performance and outperforms comparable baselines. |
| Approach: | They propose a conditioned text generation model that uses a template-based approach to generate content from input text. |
| Outcome: | The proposed model outperforms baselines on abstractive text summarization and data-to-text generation. |
MPCoder: Multi-user Personalized Code Generator with Explicit and Implicit Style Representation Learning (2024.acl-long)
Copied to clipboard
| Challenge: | Existing Large Language Models are usually generalized with large programming corpus, therefore the generated code is difficult to adapt to personalized and/or customized requests. |
| Approach: | They propose a method to use Large Language Models to generate personalized code for multiple users. |
| Outcome: | The proposed model can generate personalized code for multiple users . it can be used to improve code generation and reduce maintenance costs. |
Towards Robust and Semantically Organised Latent Representations for Unsupervised Text Style Transfer (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent studies show that auto-encoders perform language generation, smooth sentence interpolation, and style transfer over unseen attributes using unlabelled datasets in a zero-shot manner. |
| Approach: | They propose a discrete token-based perturbation approach to map "similar" sentences close by in latent space. |
| Outcome: | The proposed model can generate and perform language generation, style transfer and sentence interpolation tasks on unlabelled datasets in a zero-shot manner. |
Multilingual Machine Translation: Closing the Gap between Shared and Language-specific Encoder-Decoders (2021.eacl-main)
Copied to clipboard
| Challenge: | State-of-the-art multilingual machine translation relies on a universal encoder-decoder, which requires retraining the entire system to add new languages. |
| Approach: | They propose an encoder-decoder approach that can be extended to new languages by learning their corresponding modules. |
| Outcome: | The proposed approach outperforms the universal encoder-decoder by 3.28 BLEU points on average while allowing to add new languages without retraining the rest of the modules. |
How Positive Are You: Text Style Transfer using Adaptive Style Embedding (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches for unsupervised text style transfer are disentanglement between content and style. |
| Approach: | They propose to separate a model with a sentence reconstruction module and a style module to improve model architecture. |
| Outcome: | The proposed method improves style transfer performance and content preservation . the proposed method can be used to modify a sentence with a specified style attribute . |