A Novel Paradigm Boosting Translation Capabilities of Large Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies on LLMs focused on supervised fine-tuning but their effectiveness has been limited. |
| Approach: | They propose a paradigm consisting of three stages: Secondary Pre-training using extensive monolingual data, Continual Pre- training with interlinear text format documents, and Leveraging source-language consistent instruction for supervised fine-tuning. |
| Outcome: | The proposed approach surpasses previous work and achieves superior performance compared to models such as NLLB-54B(CITATION) and GPT3.5-text-davinci-003. |
Similar Papers
Contextual Refinement of Translations: Large Language Models for Sentence and Document-Level Post-Editing (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models have demonstrated considerable success in various natural language processing tasks, but their performance in NMT tasks is still underexplored. |
| Approach: | They propose to use LLMs as automatic post-editors rather than direct translators to improve BLEU and COMET performance. |
| Outcome: | The proposed approach improves BLEU but COMET performance compared to in-context learning. |
LLaMAX: Scaling Linguistic Horizons of LLM by Enhancing Translation Capabilities Beyond 100 Languages (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) exhibit exceptional translation capabilities in high-resource language tasks, yet their effectiveness in low-resourced languages is suboptimal. |
| Approach: | They conduct extensive multilingual continual pre-training on the LLaMA series models and develop LLiMAX for translation support across more than 100 languages. |
| Outcome: | The proposed model achieves higher translation performance than existing open-source models and performs on-par with specialized translation model on the Flores-101 benchmark. |
Multilingual Machine Translation with Open Large Language Models at Practical Scale: An Empirical Study (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown continuously improving multilingual capabilities. |
| Approach: | They evaluate the ability of open LLMs to handle multilingual machine translation tasks using a parallel-first monolingual-second data mixing strategy. |
| Outcome: | The proposed model outperforms state-of-the-art models and achieves competitive performance with Google Translate and GPT-4-turbo. |
A Paradigm Shift: The Future of Machine Translation Lies with Large Language Models (2024.lrec-main)
Copied to clipboard
Chenyang Lyu, Zefeng Du, Jitao Xu, Yitao Duan, Minghao Wu, Teresa Lynn, Alham Fikri Aji, Derek F. Wong, Longyue Wang
| Challenge: | Large Language Models (LLMs) are introducing a new phase in machine translation . despite advances in MT, there are still many challenges to overcome . |
| Approach: | They propose to highlight several new directions for MT that are influenced by Large Language Models like GPT-4 and ChatGPT. |
| Outcome: | The proposed models offer vast linguistic understandings and bring innovative methodologies, such as prompt-based techniques, that have the potential to further elevate MT. |
Fine-Tuning Large Language Models to Translate: Will a Touch of Noisy Data in Misaligned Languages Suffice? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Traditionally, success in multilingual machine translation depends on large volume, diverse directions, and high quality of training data. |
| Approach: | They revisit the importance of large language models for translation by fine-tuning on 32 parallel sentences. |
| Outcome: | The proposed model can be fine-tuned on as few as 32 parallel sentences . however, the choice of direction is critical to avoid misinterpretation, the authors say . |
Benchmarking and Improving Long-Text Translation with Large Language Models (2024.findings-acl)
Copied to clipboard
Longyue Wang, Zefeng Du, Wenxiang Jiao, Chenyang Lyu, Jianhui Pang, Leyang Cui, Kaiqiang Song, Derek Wong, Shuming Shi, Zhaopeng Tu
| Challenge: | Recent studies have illuminated the promising capabilities of large language models (LLMs) in handling long texts. |
| Approach: | They construct a benchmark dataset specifically designed for the finetuning and evaluation of large language models (LLMs) they compare LLMs with MT models and find they exhibit shortcomings in long-text domains . |
| Outcome: | The proposed model performs better in long-text translation, and its performance diminishes as document size increases. |
Getting More from Less: Large Language Models are Good Spontaneous Multilingual Learners (2024.emnlp-main)
Copied to clipboard
Shimao Zhang, Changjiang Gao, Wenhao Zhu, Jiajun Chen, Xin Huang, Xue Han, Junlan Feng, Chao Deng, Shujian Huang
| Challenge: | Large Language Models (LLMs) have shown impressive language capabilities, but most of them have very unbalanced performance across different languages. |
| Approach: | They propose to use question translation data to enhance LLMs' multilingual capabilities by using mechanistic interpretability methods. |
| Outcome: | The proposed method improves multilingual alignment even with unannotated answers in English and a wide range of languages even with instruction-tuned LLMs. |
The Fine-Tuning Paradox: Boosting Translation Quality Without Sacrificing LLM Abilities (2024.acl-long)
Copied to clipboard
| Challenge: | Recent studies have shown that fine-tuning large language models improves their translations, but it is unclear what is the impact on desirable LLM behaviors that are not present in neural machine translation models. |
| Approach: | They perform an extensive translation evaluation on LLaMA and Falcon models with model size ranging from 7 billion up to 65 billion parameters. |
| Outcome: | The proposed model produces less literal translations after fine-tuning on parallel data. |
A Preference-driven Paradigm for Enhanced Translation with Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent research shows that large language models (LLMs) can achieve remarkable translation performance through supervised fine-tuning (SFT) however, SFT simply instructs the model to imitate reference translations token by token, making it vulnerable to the noise present in the data. |
| Approach: | They propose a preference-based approach to supervised fine-tuning that trains the model to imitate reference translations token by token, making it vulnerable to noise. |
| Outcome: | The proposed approach overcomes the plateau associated with imitation-based SFT and is more resilient in the absence of gold translations. |
Extending LLMs to New Languages: A Case Study of Llama and Persian Adaptation (2025.coling-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are mainly trained on English data and struggle with low-resource languages. |
| Approach: | They propose to add a new language to Llama to improve classification accuracy for Persian tasks by aligning representations through bilingual pretraining and instruction datasets. |
| Outcome: | The proposed model performs on generation and classification tasks with no adverse impact and sometimes even improvements on English tasks. |