Towards Actual (Not Operational) Textual Style Transfer Auto-Evaluation (D19-55)
Copied to clipboard
| Challenge: | elucidates the dangerous current state of style transfer auto-evaluation research. |
| Approach: | They propose ways to aggregate the three metrics into one evaluator. |
| Outcome: | The proposed method could be used to aggregate the three metrics into one evaluator. |
Similar Papers
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer (D19-56)
Copied to clipboard
| Challenge: | Existing methods for textual transfer with no parallel corpora are insufficient to evaluate textual paraphrases with modified attributes or properties. |
| Approach: | They propose to add a metric for post-transfer classification accuracy and a method to combine them into a single overall score. |
| Outcome: | The proposed metrics correlate well with human judgments, at both the sentence-level and system-level. |
Evaluating the Evaluation Metrics for Style Transfer: A Case Study in Multilingual Formality Transfer (2021.emnlp-main)
Copied to clipboard
| Challenge: | a lack of standardized and reliable methods for automatic evaluation hinders ST . prior work has employed as many as nine different automatic systems to rate formality alone . |
| Approach: | They evaluate automatic metrics on the oft-researched task of formality style transfer . they outline best practices for automatic evaluation in (formality) style transfer and identify models that correlate well with human judgments. |
| Outcome: | The proposed models correlate well with human judgments and are robust across languages. |
Evaluating Text Style Transfer Evaluation: Are There Any Reliable Metrics? (2025.naacl-srw)
Copied to clipboard
| Challenge: | Text style transfer (TST) is a multidimensional task requiring the assessment of style transfer accuracy, content preservation, and naturalness. |
| Approach: | They propose to use text style transfer metrics to evaluate outputs of text editors . they also investigate the potential of large language models as tools for TST evaluation . |
| Outcome: | The proposed methods provide better insights than existing metrics, the authors show . their meta-evaluation through correlation with hu-man judgments shows they are effective . |
Reformulating Unsupervised Style Transfer as Paraphrase Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing systems for style transfer warp the input’s meaning through attribute transfer, which changes semantic properties such as sentiment. |
| Approach: | They propose a method for fine-tuning pretrained language models on automatically generated paraphrase data to improve the efficiency of style transfer. |
| Outcome: | The proposed method outperforms state-of-the-art style transfer systems on human and automatic evaluations and proposes fixed variants. |
Dear Sir or Madam, May I Introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer (N18-1)
Copied to clipboard
| Challenge: | a lack of training and evaluation datasets, benchmarks and automatic metrics has blocked progress in this field. |
| Approach: | They propose to use a grammarly's Yahoo Answers Formality corpus to create the largest corpus for a particular style . they also propose to apply machine translation metrics to the task . |
| Outcome: | The proposed model can be used to train and evaluate a text in a particular style . the proposed model is based on the existing model and can be applied to other tasks . |
Style Transfer for Texts: Retrain, Report Errors, Compare with Rewrites (D19-1)
Copied to clipboard
| Challenge: | Currently, standard methods for style transfer have several significant problems. |
| Approach: | They propose to take BLEU between input and human-written reformulations into consideration for benchmarks. |
| Outcome: | The proposed architectures outperform state-of-the-art in style transfer metric on human-written reformulations and take BLEU between input and output into consideration for benchmarks. |
Evaluating Style Transfer for Text (N19-1)
Copied to clipboard
| Challenge: | Existing studies on style transfer for text are lacking a standard set of evaluation practices. |
| Approach: | They propose a set of metrics for automated evaluation that are more strongly correlated with human judgment and show tradeoffs between aspects of interest. |
| Outcome: | The proposed models exhibit tradeoffs between aspects of interest and human judgment, demonstrating the importance of evaluating them at specific points of their tradeoff plots. |
An Empirical Study on Multi-Task Learning for Text Style Transfer and Paraphrase Generation (2020.coling-industry)
Copied to clipboard
Pawel Bujnowski, Kseniia Ryzhova, Hyungtak Choi, Katarzyna Witkowska, Jaroslaw Piersa, Tymoteusz Krumholc, Katarzyna Beksa
| Challenge: | a limited amount of style data is needed for text style transfer, but there are no convincing methods for evaluating them. |
| Approach: | They propose an efficient method for neutral-to-style transformation using the transformer framework. |
| Outcome: | The proposed method can train neutral-to-style transformation models using large paraphrases and a small style transfer corpus. |
Mind the Style Gap: Meta-Evaluation of Style and Attribute Transfer Metrics (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) make it easy to rewrite a text in any style, but they are not straightforward when evaluating content preservation. |
| Approach: | They propose a large meta-evaluation of metrics for evaluating style and attribute transfer, focusing on content preservation. |
| Outcome: | The proposed method achieves higher alignment with human judgements than prompting a model of a similar size as an autorater. |
Style Transfer Through Back-Translation (P18-1)
Copied to clipboard
| Challenge: | a new method for automatic style transfer is proposed to preserve the meaning of the text while reducing stylistic properties. |
| Approach: | They propose a method for automatic style transfer that uses latent representations of the input sentence to preserve meaning while reducing stylistic properties. |
| Outcome: | The proposed method improves on sentiment, gender and political slant styles on three different styles. |