Papers by Jinliang Lu
Exploiting Curriculum Learning in Unsupervised Neural Machine Translation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Experimental results show that the proposed method achieves consistent improvements with faster convergence speed. |
| Approach: | They propose a curriculum learning method to gradually utilize pseudo bi-texts based on their quality from multiple granularities. |
| Outcome: | The proposed method achieves consistent improvements with faster convergence speed on WMT 14 En-Fr, WMT14 En-De, and LDC En-Zh translation tasks. |
X-Instruction: Aligning Language Model in Low-resource Languages with Self-curated Cross-lingual Instructions (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models respond well in high-resource languages but struggle in low-resourced languages. |
| Approach: | They propose a method to construct cross-lingual instruction following samples with instruction in English and response in low-resource languages. |
| Outcome: | The proposed method builds a large-scale cross-lingual instruction tuning dataset on 10 languages. |
Take a Closer Look at Multilinguality! Improve Multilingual Pre-Training Using Monolingual Corpora Only (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have demonstrated remarkable cross-lingual capability of pre-trained language models . however, semantic alignments may be the reason behind such capability but remain under-explored. |
| Approach: | They propose token-level and semantic-level code-switched masked language modeling to improve cross-lingual interactions over mono-mPLMs without parallel sentences. |
| Outcome: | The proposed method outperforms mono-mPLMs on natural language understanding and unsupervised machine translation tasks. |
Bridging the Gap between Different Vocabularies for LLM Ensemble (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods to combine large language models with complete outputs have limited effectiveness . lexical gaps between different LLMs hinder dynamic correction and enhancement . |
| Approach: | They propose a method to ensemble large language models via Vocabulary Alignment (EVA) they learn mappings between vocabularies of different LLMs with overlapping tokens . |
| Outcome: | The proposed method bridges the lexical gap among various LLMs, enabling meticulous ensemble at each generation step. |
Improving Unsupervised Neural Machine Translation via Training Data Self-Correction (2024.lrec-main)
Copied to clipboard
| Challenge: | Unsupervised neural machine translation models can generate mistakes during training . however, the quality of pseudo-parallel sentences cannot be guaranteed . |
| Approach: | They propose a method to improve the quality of pseudo-parallel sentences . they use token-level translations to correct mis-translated tokens . |
| Outcome: | Empirical results show that the proposed method outperforms baselines on widely used datasets. |