Linguistic Corpus Annotation for Automatic Text Simplification Evaluation (2022.emnlp-main)
Copied to clipboard
Rémi Cardon, Adrien Bibal, Rodrigo Wilkens, David Alfter, Magali Norré, Adeline Müller, Watrin Patrick, Thomas François
| Challenge: | Evaluating automatic text simplification systems is a difficult task that is performed either by automatic metrics or user-based evaluations. |
| Approach: | They propose to use annotations of the ASSET corpus to analyze SARI’s behavior and to re-evaluate existing ATS systems. |
| Outcome: | The proposed methods can be used to analyze SARI’s behavior and to re-evaluate existing ATS systems. |
Similar Papers
Adapting Sentence-level Automatic Metrics for Document-level Simplification Evaluation (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing studies on text simplification have focused on sentence simplification, but these metrics often underperform on longer texts. |
| Approach: | They propose to adapt existing sentence-level metrics for paragraph- or document-level simplification by incorporating a new approach to the evaluation of text simplification metrics. |
| Outcome: | The proposed approach outperforms existing sentence-level metrics in terms of correlation with human judgment and the sensitivity and robustness of various metrics to different types of errors produced by existing systems. |
Investigating Text Simplification Evaluation (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that parallel TS corpora contain inaccurate simplifications and incorrect alignments. |
| Approach: | They propose to improve the distribution of parallel text simplification corpora to build more robust TS models. |
| Outcome: | The proposed models can be improved by improving the distribution of TS datasets. |
Meta-Evaluation of Sentence Simplification Metrics (2024.lrec-main)
Copied to clipboard
| Challenge: | Automatic Text Simplification (ATS) is a major natural language processing task that aims to help people understand complex text. |
| Approach: | They propose to use a human-annotated dataset to study automatic text simplification models to determine which metrics to use when evaluating new models. |
| Outcome: | The proposed models reconstruct the text into a simpler format by deletion, substitution, addition or splitting, while preserving the original meaning and correct grammar. |
Automatic Text Simplification for Social Good: Progress and Challenges (2021.findings-acl)
Copied to clipboard
| Challenge: | ATS has been promoted as a natural language processing task since the 1990s . but since 2010, the field has been focusing on building complex end-to-end neural architectures based on ATS . |
| Approach: | They propose to use automated text simplification (ATS) to make texts more accessible to people with disabilities . they argue that lack of high-quality TS datasets and standardized evaluation procedures are barriers . |
| Outcome: | The proposed neural ATS systems are based on a new set of TS datasets and a standardized evaluation procedure. |
ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformations (2020.acl-main)
Copied to clipboard
| Challenge: | Existing models for sentence simplification are focused on a single transformation, such as lexical paraphrasing or splitting. |
| Approach: | They propose a dataset for assessing sentence simplification in English using a crowdsourced multi-reference corpus. |
| Outcome: | The proposed dataset shows that it captures characteristics of simplicity better than other datasets. |
DETECT: Determining Ease and Textual Clarity of German Text Simplifications (2026.eacl-long)
Copied to clipboard
| Challenge: | Current evaluation of German automatic text simplification relies on general-purpose metrics such as SARI, BLEU, and BERTScore. |
| Approach: | They propose a German-specific metric that holistically evaluates ATS quality across all three dimensions of simplicity, meaning preservation, and fluency. |
| Outcome: | The proposed metric achieves higher correlations with human judgments than widely used ATS metrics. |
Document-Level Text Simplification: Dataset, Criteria and Baseline (2021.emnlp-main)
Copied to clipboard
| Challenge: | Text simplification is a valuable technique, but research on it is limited. |
| Approach: | They propose a document-level simplification task using Wikipedia dumps as a dataset and propose an automatic evaluation metric called D-SARI. |
| Outcome: | The proposed metric is more suitable for document-level simplification task. |
Semantic Structural Evaluation for Text Simplification (N18-1)
Copied to clipboard
| Challenge: | Current measures for evaluating text simplification systems focus on lexical aspects, neglecting its structural aspects. |
| Approach: | They propose to use a reference-less automatic evaluation procedure to assess simplification quality by decomposing the input based on its semantic structure and comparing it to the output. |
| Outcome: | The proposed measure has a significant correlation with human judgments and is highly comparable with existing measures. |
SimplifyUR: Unsupervised Lexical Text Simplification for Urdu (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing methods for text simplification for Urdu rely on manual lexicons and simplified corpora, but are not applicable to the language. |
| Approach: | They propose an unsupervised method for automatic text simplification for Urdu using word embeddings and morphological features. |
| Outcome: | The proposed method achieves BLEU score of 80.15 and SARI score of 42.02 on simple text generated on simplified corpora and human evaluations for correctness, grammaticality, meaning-preservation and simplicity. |
HECTOR: A Hybrid TExt SimplifiCation TOol for Raw Texts in French (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing systems for automatic text simplification (ATS) focus on lexical and syntactic transformations, but there is no end-to-end system for French. |
| Approach: | They propose to use word embeddings for lexical simplification and rule-based strategies for syntax and discourse adaptations to improve the complexity of texts. |
| Outcome: | The proposed system performs at lexical, syntactic and discourse levels according to automatic and humanevaluations. |