Papers with GLEU
A Reassessment of Reference-Based Grammatical Error Correction Metrics (C18-1)
Copied to clipboard
| Challenge: | Existing studies on the correlation of GEC metrics with human judgments were inconclusive . a recent study found that GLEU produces counter-intuitive scores in common test examples . |
| Approach: | They propose to use GLEU to evaluate grammatical error correction (GEC) systems . they also use statistical significance tests to assess their agreement with human judgments . |
| Outcome: | The proposed metrics show no significant advantage over MaxMatch (GLEU) the results contradict previous studies that claim GLEU superior . |
Bidirectional Transformer Reranker for Grammatical Error Correction (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained seq2seq models suffer from a prediction bias due to their unidirectional decoding. |
| Approach: | They propose a bidirectional Transformer reranker that re-estimates the probability of each candidate sentence generated by pre-trained seq2seq models. |
| Outcome: | The proposed model improves on the original model and gives a 59.52 GLEU score on the JFLEG corpus. |
Using Context in Neural Machine Translation Training Objectives (2020.acl-main)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) training is based on document-level metrics, not sentence-level BLEU. |
| Approach: | They propose to merge document-level metrics with batch-level documents to improve NMT training. |
| Outcome: | The proposed training is more robust for document-level metrics than sequence MRT and maximum-likelihood training. |
Is Crowdsourcing Breaking Your Bank? Cost-Effective Fine-Tuning of Pre-trained Language Models with Proximal Policy Optimization (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods to improve language models require manual ranking and annotators. |
| Approach: | They propose a self-supervised text ranking approach for applying Proximal-Policy-Optimization to fine-tune language models while eliminating the need for human annotators. |
| Outcome: | The proposed method significantly outperforms baselines regarding BLEU, GLEU, and METEOR scores on three tasks and is consistent with humans. |
IndiGEC: Multilingual Grammar Error Correction for Low-Resource Indian Languages (2025.emnlp-main)
Copied to clipboard
| Challenge: | despite growing interest in GEC, most research has focused on English due to the lack of benchmark datasets for low-resource lan-guages. |
| Approach: | They propose a new approach to generate high-quality synthetic data for GEC using monolingual corpora. |
| Outcome: | The proposed framework outperforms other monolingual methods in English, Hindi, Bengali, Marathi, and Tamil. |