Refined Evaluation for End-to-End Grammatical Error Correction Using an Alignment-Based Approach (2025.coling-main)
Copied to clipboard
| Challenge: | errant is a new evaluation tool that can be used to evaluate end-to-end grammatical error correction systems. |
| Approach: | They propose a method to assess end-to-end grammatical error correction systems using alignment-based alignment methods that reproduce and improve results from existing evaluation tools. |
| Outcome: | The proposed method reproduces and improves results from existing evaluation tools, such as errant, even when applied to raw text input. |
Similar Papers
Grammatical Error Correction: Are We There Yet? (2022.coling-1)
Copied to clipboard
| Challenge: | grammatical error correction (GEC) systems outperform humans on the CoNLL-2014 test set, but there are still classes of errors that they fail to correct. |
| Approach: | They found that state-of-the-art GEC systems outperform humans by a wide margin on the CoNLL-2014 test set . however, they found that there are still classes of errors that they fail to correct . |
| Outcome: | The F0.5 evaluation metric outperforms the CoNLL-2014 test set, but there are still classes of errors that they fail to correct. |
Is this the end of the gold standard? A straightforward reference-less grammatical error correction metric (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluations of grammatical error correction systems use reference-based metrics, but they are limited because of multiple correct outputs. |
| Approach: | They propose a system that uses commonly available tools to evaluate grammatical error correction (GEC) systems. |
| Outcome: | The proposed system solves the issues related to the use of a reference and does not need another annotated dataset for fine-tuning. |
Evaluation of Really Good Grammatical Error Correction (2024.lrec-main)
Copied to clipboard
| Challenge: | emergence of large language models has highlighted the shortcomings of evaluation methods . evaluators often use grammatical error correction (GEC) to correct language errors at multiple levels . |
| Approach: | They perform a comprehensive evaluation of various GEC systems using Swedish learner texts . they suggest using human post-editing to analyze amount of change required to reach native-level human performance . |
| Outcome: | The proposed evaluations outperform existing methods for grammatical error correction in Swedish . the results highlight the shortcomings of existing evaluation methods . |
UnifiedGEC: Integrating Grammatical Error Correction Approaches for Multi-languages with a Unified Framework (2025.coling-demos)
Copied to clipboard
| Challenge: | Existing tools for GEC have been developed to support research on grammatical errors, but there is no comprehensive evaluation on these models. |
| Approach: | They propose an open-source framework for Grammatical Error Correction that integrates 5 widely-used GEC models and compares their performance on 7 datasets in different languages. |
| Outcome: | The proposed framework compares 5 widely-used models on 7 datasets in different languages. |
GPT-3.5 for Grammatical Error Correction (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent work shows that GPT-3.5 struggles with several error types, including punctuation mistakes, tense errors, syntactic dependencies between words, and lexical compatibility at the sentence level. |
| Approach: | They evaluate GPT-3.5 for grammatical error correction in multiple languages . they use it to re-rank correction hypotheses generated by other GEC models . |
| Outcome: | The proposed model performs well in English and Russian, but struggles with errors in other languages. |
Grammatical Error Correction in Low Error Density Domains: A New Benchmark and Analyses (2020.emnlp-main)
Copied to clipboard
| Challenge: | CWEB is a new benchmark for grammatical error correction (GEC) systems . website data contains far fewer grammamatical errors than learner essays . |
| Approach: | They propose to broaden the target domain of grammatical error correction (GEC) systems . website data contains far fewer grammamatical errors than learner essays . |
| Outcome: | The proposed model can't rely on a strong internal language model in low error density domains. |
IMPARA-GED: Grammatical Error Detection is Boosting Reference-free Grammatical Error Quality Estimator (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing reference-free automatic grammatical error correction methods do not correlate with human evaluation. |
| Approach: | They propose a reference-free automatic grammatical error correction evaluation method with enhanced gramma-ed capabilities. |
| Outcome: | The proposed method achieves highest correlation with human evaluations on a meta-evaluation dataset. |
gec-metrics: A Unified Library for Grammatical Error Correction Evaluation (2025.acl-demo)
Copied to clipboard
| Challenge: | a library for using and developing grammatical error correction (GEC) evaluation metrics is released under the MIT license . |
| Approach: | They propose a library for using and developing grammatical error correction (GEC) evaluation metrics through a unified interface. |
| Outcome: | The proposed method is based on a unified evaluation framework with a strong focus on API usage and extensible. |
LET: Leveraging Error Type Information for Grammatical Error Correction (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for grammatical error correction (GEC) are mainly divided into detection-based and end-to-end generative models. |
| Approach: | They propose an end-to-end framework which Leverages Error Type (LET) information in the generation process to introduce more convincing error type information. |
| Outcome: | The proposed framework outperforms existing methods on various datasets by a clear margin. |
A Simple Recipe for Multilingual Grammatical Error Correction (2021.acl-short)
Copied to clipboard
| Challenge: | Modern approaches view the task of Grammatical Error Correction (GEC) as monolingual text-to-text rewriting and employ encoderdecoder neural architectures. |
| Approach: | They propose a language-agnostic method to generate a large number of synthetic examples and use large-scale multilingual language models to train state-of-the-art GEC models. |
| Outcome: | The proposed method surpasses state-of-the-art results on GEC benchmarks in English, Czech, German and Russian. |