Ensembling and Knowledge Distilling of Large Sequence Taggers for Grammatical Error Correction (2022.acl-long)
Copied to clipboard
| Challenge: | Currently, machine translation (MT) is the mainstream approach for GEC. |
| Approach: | They propose to ensemble Transformer-based encoders by majority votes on span-level edits . their best ensemble achieves a new SOTA result even without pre-training on synthetic datasets - "Troy-Blogs" and "Try-1BW". |
| Outcome: | The proposed model achieves a new SOTA result even without pre-training on synthetic datasets. |
Similar Papers
An Extended Sequence Tagging Vocabulary for Grammatical Error Correction (2023.findings-eacl)
Copied to clipboard
| Challenge: | Current sequence-to-sequence and sequence-tagging approaches treat GEC as a machine-translation problem. |
| Approach: | They propose to introduce specialised tags for spelling correction and morphological inflection using the SymSpell and LemmInflect algorithms. |
| Outcome: | The proposed approach outperforms existing methods on the BEA benchmark. |
Distillation of encoder-decoder transformers for sequence labelling (2023.findings-eacl)
Copied to clipboard
| Challenge: | despite the strong trend in NLP to explore the use of large language models, there is still limited work evaluating prompting and decoding mechanisms for SL tasks. |
| Approach: | They propose a hallucination-free framework for sequence tagging that is especially suited for distillation. |
| Outcome: | The proposed framework performs well across multiple sequence labelling datasets and in a few-shot learning scenario. |
GEC-DePenD: Non-Autoregressive Grammatical Error Correction with Decoupled Permutation and Decoding (2023.acl-long)
Copied to clipboard
| Challenge: | grammatical error correction is an important NLP task that is usually solved with autoregressive sequence-to-sequence models. |
| Approach: | They propose a non-autoregressive approach to grammatical error correction that decouples a permutation network and a decoder network that fills in specific tokens. |
| Outcome: | The proposed approach improves over previously known non-autoregressive methods and reaches the level of autoregressive approaches that do not use language-specific synthetic data generation methods. |
Grammatical Error Correction via Sequence Tagging for Russian (2025.acl-srw)
Copied to clipboard
| Challenge: | Several types of models have been suggested for grammatical error correction . despite being successful, the difference between GEC and machine translation is not taken into account . |
| Approach: | They propose a modified sequence tagging architecture for the Russian language to be used for grammatical error correction. |
| Outcome: | The proposed model outperforms previous approaches on two Russian GEC benchmarks while achieving competitive performance on RULEC-GEC. |
Corpora Generation for Grammatical Error Correction (N19-1)
Copied to clipboard
| Challenge: | Grammatical Error Correction (GEC) is a computational task that requires large amounts of data to solve. |
| Approach: | They propose two approaches to generate large parallel datasets for GEC using publicly available Wikipedia edit histories using minimal filtration heuristics and round-trip translation through bridge languages. |
| Outcome: | The proposed methods yield similar sized parallel corpora with around 4B tokens and are far ahead of the state-of-the-art on the CoNLL ‘14 benchmark and the JFLEG task. |
Encode, Tag, Realize: High-Precision Text Editing (D19-1)
Copied to clipboard
| Challenge: | Neural sequence-to-sequence models provide a powerful framework for learning to translate source texts into target texts. |
| Approach: | They propose a sequence tagging approach that casts text generation as a text editing task. |
| Outcome: | The proposed model outperforms strong seq2seq models on sentence fusion, sentence splitting, abstractive summarization, and grammar correction tasks and achieves state-of-the-art performance. |
Better, Faster, Stronger Sequence Tagging Constituent Parsers (N19-1)
Copied to clipboard
| Challenge: | Existing efforts to speed up constituent parsing have focused on chart-based or shift-reduce parsers. |
| Approach: | They propose to use auxiliary losses and sentence-level fine-tuning to mitigate greedy decoding issues. |
| Outcome: | The proposed model surpasses the performance of sequence tagging constituent parsers on the English and Chinese Penn Treebank datasets and reduces their parsing time even further. |
InstructGEC: Enhancing Unsupervised Grammatical Error Correction with Instruction Tuning (2025.coling-main)
Copied to clipboard
| Challenge: | Recent studies have proposed methods of generating synthetic data for unsupervised GEC . however, the cost of such methods is high and the quality of the data is poor . |
| Approach: | They propose a method to generate synthetic data automatically for unsupervised GEC . they use a masking strategy to mask an erroneous sentence and the instruction consistently . |
| Outcome: | The proposed method outperforms state-of-the-art unsupervised methods on English and Chinese GEC datasets. |
Data Weighted Training Strategies for Grammatical Error Correction (2020.tacl-1)
Copied to clipboard
| Challenge: | Recent advances in the task of Grammatical Error Correction (GEC) have been driven by addressing data sparsity, both through new methods for generating large and noisy pretraining data and through the publication of small and higher-quality finetuning data in the BEA-2019 shared task. |
| Approach: | They propose to incorporate delta-log-perplexity, a type of example scoring, into a training schedule for Grammatical Error Correction (GEC) they perform experiments that shed light on the function and applicability of delta- log-perplicity. |
| Outcome: | The proposed methods incorporate delta-log-perplexity, a type of example scoring, into a training schedule for the task. |
Focal Training and Tagger Decouple for Grammatical Error Correction (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for Grammatical Error Correction (GEC) are categorized into sequence-to-sequence approaches, tagging-based approaches, and hybrid approaches. |
| Approach: | They propose to decouple error detection layer from label tagging layer and to down-weight label imbalance and tabbing entanglement loss using Focal Loss. |
| Outcome: | The proposed methods are effective over three latest Chinese Grammatical Error Correction datasets. |