A large-scale computational study of content preservation measures for text style transfer and paraphrase generation (2022.acl-srw)
Copied to clipboard
| Challenge: | Text style transfer and paraphrases generation are growing areas of NLP . many researchers still use BLEU-like measures to evaluate content preservation . |
| Approach: | They compare 57 different measures based on different principles on 19 annotated datasets . they find that measures relying on cross-encoder models outperform alternative approaches . |
| Outcome: | The proposed methods outperform traditional methods on 19 datasets. |
Similar Papers
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer (D19-56)
Copied to clipboard
| Challenge: | Existing methods for textual transfer with no parallel corpora are insufficient to evaluate textual paraphrases with modified attributes or properties. |
| Approach: | They propose to add a metric for post-transfer classification accuracy and a method to combine them into a single overall score. |
| Outcome: | The proposed metrics correlate well with human judgments, at both the sentence-level and system-level. |
Towards Actual (Not Operational) Textual Style Transfer Auto-Evaluation (D19-55)
Copied to clipboard
| Challenge: | elucidates the dangerous current state of style transfer auto-evaluation research. |
| Approach: | They propose ways to aggregate the three metrics into one evaluator. |
| Outcome: | The proposed method could be used to aggregate the three metrics into one evaluator. |
Evaluating Text Style Transfer Evaluation: Are There Any Reliable Metrics? (2025.naacl-srw)
Copied to clipboard
| Challenge: | Text style transfer (TST) is a multidimensional task requiring the assessment of style transfer accuracy, content preservation, and naturalness. |
| Approach: | They propose to use text style transfer metrics to evaluate outputs of text editors . they also investigate the potential of large language models as tools for TST evaluation . |
| Outcome: | The proposed methods provide better insights than existing metrics, the authors show . their meta-evaluation through correlation with hu-man judgments shows they are effective . |
Reformulating Unsupervised Style Transfer as Paraphrase Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing systems for style transfer warp the input’s meaning through attribute transfer, which changes semantic properties such as sentiment. |
| Approach: | They propose a method for fine-tuning pretrained language models on automatically generated paraphrase data to improve the efficiency of style transfer. |
| Outcome: | The proposed method outperforms state-of-the-art style transfer systems on human and automatic evaluations and proposes fixed variants. |
An Empirical Study on Multi-Task Learning for Text Style Transfer and Paraphrase Generation (2020.coling-industry)
Copied to clipboard
Pawel Bujnowski, Kseniia Ryzhova, Hyungtak Choi, Katarzyna Witkowska, Jaroslaw Piersa, Tymoteusz Krumholc, Katarzyna Beksa
| Challenge: | a limited amount of style data is needed for text style transfer, but there are no convincing methods for evaluating them. |
| Approach: | They propose an efficient method for neutral-to-style transformation using the transformer framework. |
| Outcome: | The proposed method can train neutral-to-style transformation models using large paraphrases and a small style transfer corpus. |
On Learning Text Style Transfer with Direct Rewards (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for text style transfer lack parallel corpora, which makes it impossible to train supervised models. |
| Approach: | They propose to use semantic similarity metrics to explicitly assess the preservation of content between system outputs and inputs. |
| Outcome: | The proposed methods provide significant gains in automatic and human evaluation over strong baselines. |
Mind the Style Gap: Meta-Evaluation of Style and Attribute Transfer Metrics (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) make it easy to rewrite a text in any style, but they are not straightforward when evaluating content preservation. |
| Approach: | They propose a large meta-evaluation of metrics for evaluating style and attribute transfer, focusing on content preservation. |
| Outcome: | The proposed method achieves higher alignment with human judgements than prompting a model of a similar size as an autorater. |
Paraphrase Types for Generation and Detection (2023.emnlp-main)
Copied to clipboard
| Challenge: | Current approaches to paraphrase generation and detection ignore the intricate linguistic properties of language. |
| Approach: | They propose two tasks to consider specific linguistic perturbations at particular text positions. |
| Outcome: | The proposed tasks address the shortcoming of ignoring the linguistic properties of language. |
Reference-guided Style-Consistent Content Transfer (2024.lrec-main)
Copied to clipboard
| Challenge: | Text style transfer involves changing the style of a text while preserving its original style. |
| Approach: | They propose a task of style-consistent content transfer which involves modifying a text’s content based on a provided reference statement while preserving its original style. |
| Outcome: | The proposed approach meets three important conditions: reference faithfulness, style adherence, and coherence. |
SC2: Towards Enhancing Content Preservation and Style Consistency in Long Text Style Transfer (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for short TST are difficult to implement and can cause content degradation. |
| Approach: | They propose a method to vary the style polarity of text while preserving semantic content. |
| Outcome: | The proposed method improves over baselines and is highly efficient. |