Papers by Xiuyu Wu
Position Offset Label Prediction for Grammatical Error Correction (2022.coling-1)
Copied to clipboard
| Challenge: | Experimental results show that our proposed POL-Pc framework improves baseline models and yields consistent performance gain over various data augmentation methods. |
| Approach: | They propose a position offset label prediction subtask to integrate correction editing operations into a unified framework. |
| Outcome: | The proposed model outperforms baseline models on Chinese, English and Japanese datasets by a wide margin. |
LLoCO: Learning Long Contexts Offline (2024.emnlp-main)
Copied to clipboard
Sijun Tan, Xiuyu Li, Shishir G Patil, Ziyang Wu, Tianjun Zhang, Kurt Keutzer, Joseph Gonzalez, Raluca Popa
| Challenge: | Large language models are still unable to handle long contexts due to the quadratic computational and memory overhead of the self-attention mechanism and the substantial KV cache sizes during generation. |
| Approach: | They propose a method to learn contexts offline through context compression and in-domain parameter-efficient finetuning with LoRA. |
| Outcome: | The proposed model outperforms in-context learning while using 30 fewer tokens during inference and significantly reduces the cost of long document question answering. |
From Spelling to Grammar: A New Framework for Chinese Grammatical Error Correction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Chinese Grammatical Error Correction (CGEC) aims to generate correct sentences from erroneous sequences. |
| Approach: | They propose a zero-shot approach for spelling error correction that is simple but effective . they propose auxiliary task to predict POS sequence of target sentence . |
| Outcome: | The proposed framework achieves 42.11 F-0.5 on the English GEC dataset outperforms the previous state-of-the-art by a wide margin of 1.30 points. |
Are Pre-trained Language Models Useful for Model Ensemble in Chinese Grammatical Error Correction? (2023.acl-short)
Copied to clipboard
| Challenge: | Model ensemble has been in widespread use for Grammatical Error Correction (GEC), boosting model performance. |
| Approach: | They propose to use model ensembles computed by pre-trained language models to improve model performance. |
| Outcome: | The proposed ensembles do not improve but get worse after the PLM-based ensemble. |
An Error-Guided Correction Model for Chinese Spelling Error Correction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing neural network approaches have achieved great progress on Chinese spelling correction, but there is still room for improvement. |
| Approach: | They propose an error-guided correction model that uses pre-trained BERT models to detect errors and integrate the error confusion set into the model. |
| Outcome: | The proposed model outperforms state-of-the-art models on widely used benchmarks and achieves superior performance on both quality and computation speed. |