Stronger Baselines for Grammatical Error Correction Using a Pretrained Encoder-Decoder Model (2020.aacl-main)
Copied to clipboard
| Challenge: | Strong pretraining approaches for grammatical error correction require extensive use of a pseudo-parallel corpus. |
| Approach: | They propose to use bidirectional and auto-regressive transformers as a generic pretrained encoder-decoder model for grammatical error correction (GEC) they find that monolingual and multilingual BART models achieve high performance in GEC, with one of the results being comparable to the current strong results in English GEC. |
| Outcome: | The proposed model achieves comparable results to the current strong results in English GEC. |
Similar Papers
Efficient Grammatical Error Correction Via Multi-Task Training and Optimized Training Schedule (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent research has focused on using synthetic data for grammatical error correction . lack of annotated training data hinders progress in the field . |
| Approach: | They propose auxiliary tasks that exploit alignment between original and corrected sentences . they propose a sequence-to-sequence problem and perform multi-task training . |
| Outcome: | The proposed auxiliary tasks outperform the best models with a BART-based model on 11B parameters. |
Encoder-Decoder Models Can Benefit from Pre-trained Masked Language Models in Grammatical Error Correction (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for incorporating a masked language model into an EncDec model have potential drawbacks when applied to GEC. |
| Approach: | They propose to incorporate a pre-trained masked language model (MLM) into an encoder-decoder model for grammatical error correction. |
| Outcome: | The proposed method achieves state-of-the-art on BEA-2019 and CoNLL-2014 benchmarks. |
Chinese Grammatical Correction Using BERT-based Pre-trained Model (2020.aacl-main)
Copied to clipboard
| Challenge: | Recent studies have shown that pre-trained models improve performance on downstream tasks. |
| Approach: | They propose to incorporate a pre-trained model into an encoder-decoder model to improve the performance of Chinese grammatical error correction tasks. |
| Outcome: | The proposed method improves the performance of Chinese grammatical error correction tasks. |
No Error Left Behind: Multilingual Grammatical Error Correction with Pre-trained Translation Models (2024.eacl-long)
Copied to clipboard
| Challenge: | Grammatical Error Correction (GEC) research has primarily focused on English with little coverage for other languages. |
| Approach: | They propose a multilingual machine translation model that can be fine-tuned to improve error correction out-of-the-box. |
| Outcome: | The proposed model outperforms similar-sized MT5 models and competes favourably with larger models. |
Grammatical Error Correction Using Pseudo Learner Corpus Considering Learner’s Error Tendency (2020.acl-srw)
Copied to clipboard
| Challenge: | Recent studies have focused on improving the performance of grammatical error correction (GEC) tasks using pseudo data. |
| Approach: | They propose to extract sentences similar to those written by language learners and generate pseudo errors by considering error types that learners often make. |
| Outcome: | The proposed model significantly improves the performance of the Russian GEC task compared with other models using pseudo data. |
Revisiting Grammatical Error Correction Evaluation and Beyond (2022.emnlp-main)
Copied to clipboard
| Challenge: | Pretraining-based (PT) evaluation metrics are not effective for training grammatical error correction systems. |
| Approach: | They propose a pretraining-based GEC evaluation metric which only uses PT-based metrics to score the corrected parts of the system. |
| Outcome: | The proposed evaluation metric outperforms existing methods on a CoNLL14 evaluation task. |
Type-Driven Multi-Turn Corrections for Grammatical Error Correction (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing studies focus on data augmentation to combat exposure bias . but data augmented models lack the ability to recognize the procedure of gradual corrections . |
| Approach: | They propose a type-driven multi-turn corrections approach that uses multiple training instances to train dominant models. |
| Outcome: | The proposed model achieves state-of-the-art single-model performance on English GEC benchmarks. |
Comparison of Grammatical Error Correction Using Back-Translation Models (2021.naacl-srw)
Copied to clipboard
| Challenge: | Currently, a mainstream approach to generate pseudo data is back-translation (BT). |
| Approach: | They propose to use back-translation to generate pseudo data that contains grammatical and ungrammatically produced sentences. |
| Outcome: | The proposed methods improve or interpolate the performance of each error type compared with a single BT model with different seeds. |
Data Weighted Training Strategies for Grammatical Error Correction (2020.tacl-1)
Copied to clipboard
| Challenge: | Recent advances in the task of Grammatical Error Correction (GEC) have been driven by addressing data sparsity, both through new methods for generating large and noisy pretraining data and through the publication of small and higher-quality finetuning data in the BEA-2019 shared task. |
| Approach: | They propose to incorporate delta-log-perplexity, a type of example scoring, into a training schedule for Grammatical Error Correction (GEC) they perform experiments that shed light on the function and applicability of delta- log-perplicity. |
| Outcome: | The proposed methods incorporate delta-log-perplexity, a type of example scoring, into a training schedule for the task. |
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension (2020.acl-main)
Copied to clipboard
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, Luke Zettlemoyer
| Challenge: | Recent work has shown gains by improving the distribution of masked tokens and the order in which mucked tokens are predicted. |
| Approach: | They propose a denoising autoencoder for pretraining sequence-to-sequence models that corrupts text with an arbitrary noising function and learns a model to reconstruct the original text. |
| Outcome: | The proposed model outperforms RoBERTa on GLUE and SQUAD and provides a 1.1 BLEU increase over a back-translation system for machine translation. |