RobustGEC: Robust Grammatical Error Correction Against Subtle Context Perturbation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Grammatical Error Correction (GEC) systems perform well in academic benchmarks, but in practical applications they may not correct errors when users perform irrelevant modifications. |
| Approach: | They propose a benchmark to evaluate the context robustness of Grammatical Error Correction systems. |
| Outcome: | The proposed method improves the accuracy of errors corrected by human annotations. |
Similar Papers
COCOGEC: Counterfactual Generation for Robust Grammatical Error Correction (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing GEC models fail to understand error patterns in varying contexts . a framework that generates copies of training instances with error-irrelevant contexts altered is proposed . |
| Approach: | They propose a framework that generates copies of training instances with error-irrelevant contexts altered. |
| Outcome: | The proposed framework outperforms baselines on the simulated tasks and outperformed existing models. |
GPT-3.5 for Grammatical Error Correction (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent work shows that GPT-3.5 struggles with several error types, including punctuation mistakes, tense errors, syntactic dependencies between words, and lexical compatibility at the sentence level. |
| Approach: | They evaluate GPT-3.5 for grammatical error correction in multiple languages . they use it to re-rank correction hypotheses generated by other GEC models . |
| Outcome: | The proposed model performs well in English and Russian, but struggles with errors in other languages. |
Grammatical Error Correction: Are We There Yet? (2022.coling-1)
Copied to clipboard
| Challenge: | grammatical error correction (GEC) systems outperform humans on the CoNLL-2014 test set, but there are still classes of errors that they fail to correct. |
| Approach: | They found that state-of-the-art GEC systems outperform humans by a wide margin on the CoNLL-2014 test set . however, they found that there are still classes of errors that they fail to correct . |
| Outcome: | The F0.5 evaluation metric outperforms the CoNLL-2014 test set, but there are still classes of errors that they fail to correct. |
How Good (really) are Grammatical Error Correction Systems? (2021.eacl-main)
Copied to clipboard
| Challenge: | Standard evaluations of Grammatical Error Correction systems use a fixed reference text generated relative to the original text. |
| Approach: | They propose to use a gold reference text to evaluate Grammatical Error Correction systems that is generated relative to the original text and is independent of the system output. |
| Outcome: | The proposed evaluations show that the system performs 20-40 points better than standard evaluations. |
Personalizing Grammatical Error Correction: Adaptation to Proficiency Level and L1 (D19-55)
Copied to clipboard
| Challenge: | Grammar error correction systems have become ubiquitous in a variety of software applications, but little is known about how to efficiently personalize them to the user’s characteristics, such as proficiency level and first language. |
| Approach: | They propose to adapt a general purpose neural GEC system to the proficiency level and the first language of a writer, using only a few thousand annotated sentences. |
| Outcome: | The proposed system improves on adapting to proficiency level and first language . the results are the broadest of its kind, covering five proficiency levels and twelve different languages. |
Grammatical Error Correction in Low Error Density Domains: A New Benchmark and Analyses (2020.emnlp-main)
Copied to clipboard
| Challenge: | CWEB is a new benchmark for grammatical error correction (GEC) systems . website data contains far fewer grammamatical errors than learner essays . |
| Approach: | They propose to broaden the target domain of grammatical error correction (GEC) systems . website data contains far fewer grammamatical errors than learner essays . |
| Outcome: | The proposed model can't rely on a strong internal language model in low error density domains. |
Evaluation of Really Good Grammatical Error Correction (2024.lrec-main)
Copied to clipboard
| Challenge: | emergence of large language models has highlighted the shortcomings of evaluation methods . evaluators often use grammatical error correction (GEC) to correct language errors at multiple levels . |
| Approach: | They perform a comprehensive evaluation of various GEC systems using Swedish learner texts . they suggest using human post-editing to analyze amount of change required to reach native-level human performance . |
| Outcome: | The proposed evaluations outperform existing methods for grammatical error correction in Swedish . the results highlight the shortcomings of existing evaluation methods . |
A Crash Course in Automatic Grammatical Error Correction (2020.coling-tutorials)
Copied to clipboard
| Challenge: | Grammatical Error Correction (GEC) is the task of automatically detecting and correcting all types of errors in written text. |
| Approach: | tutorial aims to introduce participants to the field of Grammatical Error Correction . aim is to examine the development of neural-based GEC systems . |
| Outcome: | the tutorial aims to introduce participants to the current state of the art in the field of Grammatical Error Correction (GEC) |
FCGEC: Fine-Grained Corpus for Chinese Grammatical Error Correction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | grammatical error correction (GEC) is a complex task that requires high-quality data from native speakers. |
| Approach: | They propose a human-annotated corpus to detect, identify and correct grammatical errors in Chinese examinations. |
| Outcome: | The proposed model outperforms other models in low-resource settings, but there is a significant gap between the models and humans that encourages future models to bridge it. |
Beyond Static Synthetic Noise: Assessing the Robustness of Large Language Models to Natural Context Variation in the Real World (2026.findings-acl)
Copied to clipboard
| Challenge: | Current robustness evaluation methods rely on static synthetic perturbations to stress-test models. |
| Approach: | They propose a framework for automatically evaluating QA models under naturally occurring textual perturbations by replacing context passages with revised Wikipedia edit histories. |
| Outcome: | The proposed framework replaces context passages with revised Wikipedia edit histories to improve model performance. |