Papers with IMPARA
IMPARA: Impact-Based Metric for GEC Using Parallel Data (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods for automatic evaluation of grammatical error correction require multiple reference sentences or manual scores. |
| Approach: | They propose an Impact-based Metric for GEC using PARAllel data, IMPARA . IMPRA computes correction impacts computed by parallel data comprising pairs of grammatical/ungrammatically-spaced sentences. |
| Outcome: | The proposed method can perform evaluations that fit different domains and correction styles. |
IMPARA-GED: Grammatical Error Detection is Boosting Reference-free Grammatical Error Quality Estimator (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing reference-free automatic grammatical error correction methods do not correlate with human evaluation. |
| Approach: | They propose a reference-free automatic grammatical error correction evaluation method with enhanced gramma-ed capabilities. |
| Outcome: | The proposed method achieves highest correlation with human evaluations on a meta-evaluation dataset. |
Reliability Crisis of Reference-free Metrics for Grammatical Error Correction (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Reference-free evaluation metrics for grammatical error correction have high correlation with human judgments, but they are not designed to evaluate adversarial systems that aim to obtain unjustifiably high scores. |
| Approach: | They propose adversarial attack strategies for four reference-free metrics . they propose SOME, Scribendi, IMPARA, and LLM-based metrics based on these metrics a . |
| Outcome: | The proposed attacks outperform the current state-of-the-art for four reference-free metrics . |