Adapting Sentence-level Automatic Metrics for Document-level Simplification Evaluation (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing studies on text simplification have focused on sentence simplification, but these metrics often underperform on longer texts. |
| Approach: | They propose to adapt existing sentence-level metrics for paragraph- or document-level simplification by incorporating a new approach to the evaluation of text simplification metrics. |
| Outcome: | The proposed approach outperforms existing sentence-level metrics in terms of correlation with human judgment and the sensitivity and robustness of various metrics to different types of errors produced by existing systems. |
Similar Papers
Document-Level Text Simplification: Dataset, Criteria and Baseline (2021.emnlp-main)
Copied to clipboard
| Challenge: | Text simplification is a valuable technique, but research on it is limited. |
| Approach: | They propose a document-level simplification task using Wikipedia dumps as a dataset and propose an automatic evaluation metric called D-SARI. |
| Outcome: | The proposed metric is more suitable for document-level simplification task. |
Meta-Evaluation of Sentence Simplification Metrics (2024.lrec-main)
Copied to clipboard
| Challenge: | Automatic Text Simplification (ATS) is a major natural language processing task that aims to help people understand complex text. |
| Approach: | They propose to use a human-annotated dataset to study automatic text simplification models to determine which metrics to use when evaluating new models. |
| Outcome: | The proposed models reconstruct the text into a simpler format by deletion, substitution, addition or splitting, while preserving the original meaning and correct grammar. |
Context-Aware Document Simplification (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work on document simplification has focused on sentence-level inputs but fails to preserve the discourse structure. |
| Approach: | They explore various systems that use document context within the simplification process . they investigate the performance and efficiency tradeoffs of system variants . |
| Outcome: | The proposed approach achieves state-of-the-art even when not relying on plan-guidance. |
Document-Level Planning for Text Simplification (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing work on text simplification is limited to sentence-level inputs . attempts to iteratively apply these approaches fail to preserve discourse structure of document . |
| Approach: | They propose a simplification plan that labels each sentence in the input document while considering both its context and internal structure. |
| Outcome: | The proposed model outperforms baselines on two simplification benchmarks and when used to guide document-level simplification models. |
LENS: A Learnable Evaluation Metric for Text Simplification (2023.acl-long)
Copied to clipboard
| Challenge: | Existing metrics for text simplification are based on unitary or outdated models, making them unsuitable for this approach. |
| Approach: | They present a learnable evaluation metric for text simplification using language models . they also introduce a human evaluation framework that rates simplifications from several models a list-wise manner . |
| Outcome: | The proposed model correlates much better with human judgment than existing metrics. |
Simplicity Level Estimate (SLE): A Learned Reference-Less Metric for Sentence Simplification (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluation metrics conflate simplicity with correlated attributes such as fluency or meaning preservation. |
| Approach: | They propose a new learning evaluation metric that focuses on simplicity outperforming most existing metrics in terms of correlation with human judgements. |
| Outcome: | The proposed metric outperforms most existing metrics in terms of correlation with human judgements. |
Data-Driven Text Simplification (C18-3)
Copied to clipboard
| Challenge: | Automatic text simplification is the process of transforming a complex text into an equivalent version which would be easier to read or understand by automatic natural language processors. |
| Approach: | This tutorial provides an overview of automatic text simplification, which is the process of transforming a complex text into an equivalent version. |
| Outcome: | The aim of this paper is to provide a comprehensive overview of past and current research on automatic text simplification. |
SIMSUM: Document-level Text Simplification via Simultaneous Summarization (2023.acl-long)
Copied to clipboard
| Challenge: | Document-level text simplification is a specific type of simplification which involves simplifying documents consisting of several sentences by rewriting them into fewer or more sentences. |
| Approach: | They propose a new two-stage framework SIMSUM for automated document-level text simplification which uses explicit summarization and simplification models and guides the generation using the main keywords of a source text. |
| Outcome: | The proposed model outperforms baseline models on two document-level simplification datasets, namely D-Wikipedia and Wiki-Doc. |
EASSE: Easier Automatic Sentence Simplification Evaluation (D19-3)
Copied to clipboard
| Challenge: | EASSE provides access to a broad range of evaluation resources including standard automatic metrics, word-level accuracy scores and reference-independent quality estimation features. |
| Approach: | They propose to provide a Python package that provides access to automatic evaluation and comparison of Sentence Simplification (SS) systems. |
| Outcome: | The proposed tool allows comparison and understanding of the performance of Sentence Simplification (SS) systems. |
Linguistic Corpus Annotation for Automatic Text Simplification Evaluation (2022.emnlp-main)
Copied to clipboard
Rémi Cardon, Adrien Bibal, Rodrigo Wilkens, David Alfter, Magali Norré, Adeline Müller, Watrin Patrick, Thomas François
| Challenge: | Evaluating automatic text simplification systems is a difficult task that is performed either by automatic metrics or user-based evaluations. |
| Approach: | They propose to use annotations of the ASSET corpus to analyze SARI’s behavior and to re-evaluate existing ATS systems. |
| Outcome: | The proposed methods can be used to analyze SARI’s behavior and to re-evaluate existing ATS systems. |