The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Extending state-of-the-art language models to low-resource languages requires addressing what we call the low-Resource double bind. |
| Approach: | They propose a low-resource double bind to refer to the co-occurrence of data limitations and compute resource constraints. |
| Outcome: | The proposed model improves performance on frequent sentences but disparates on infrequent ones. |
Similar Papers
What Do Compressed Multilingual Machine Translation Models Forget? (2022.findings-emnlp)
Copied to clipboard
Alireza Mohammadshahi, Vassilina Nikoulina, Alexandre Berard, Caroline Brun, James Henderson, Laurent Besacier
| Challenge: | Recent studies show that pre-trained models achieve state-of-the-art results in NLP tasks but their size makes it more challenging to apply them in resource-constrained environments. |
| Approach: | They assess the impact of compression methods on multilingual Neural Machine Translation models for various language groups, gender, and semantic biases. |
| Outcome: | The proposed compression methods improve models on different benchmarks for language groups, gender, and semantic biases. |
Optimizing Transformer for Low-Resource Neural Machine Translation (2020.coling-main)
Copied to clipboard
| Challenge: | Language pairs with limited amounts of parallel data remain a challenge for neural machine translation. |
| Approach: | They propose to optimize a Transformer model for low-resource conditions to improve translation quality by 7.3 BLEU points compared to the default settings. |
| Outcome: | The proposed model improves translation quality up to 7.3 BLEU points compared to the default settings on the IWSLT14 training data compared with the Transformer model. |
Small Data, Big Impact: Leveraging Minimal Data for Effective Machine Translation (2023.acl-long)
Copied to clipboard
Jean Maillard, Cynthia Gao, Elahe Kalbassi, Kaushik Ram Sadagopan, Vedanuj Goswami, Philipp Koehn, Angela Fan, Francisco Guzman
| Challenge: | Existing datasets are not economical to create large-scale datasets, but for low-resource languages, a few thousand professionally translated sentence pairs can be useful. |
| Approach: | They propose to use a dataset to train machine translation models on pre-existing and synthetic data to augment them with millions of sentences through backtranslation. |
| Outcome: | The proposed model can cover hundreds of languages with high quality training data even when smaller but lower quality datasets are used. |
A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios (2021.naacl-main)
Copied to clipboard
| Challenge: | a growing body of work is focused on improving performance in low-resource settings . a goal of this study is to explain how these methods differ in their requirements . |
| Approach: | They propose to analyze data-lean scenarios across different dimensions of data availability to understand which approaches are effective in a specific low-resource setting. |
| Outcome: | The proposed methods enable learning when training data is sparse. |
Revisiting Low-Resource Neural Machine Translation: A Case Study (P19-1)
Copied to clipboard
| Challenge: | Recent research has shown that neural machine translation models are highly data-inefficient and underperform phrase-based statistical machine translation (PBSMT) in low-resource settings. |
| Approach: | They propose to use auxiliary data to train low-resource neural machine translation systems without auxiliary monolingual or multilingual data. |
| Outcome: | The proposed methods outperform PBSMT and other statistical machine translation models in Korean–English with minimal data. |
Generalized Data Augmentation for Low-Resource Translation (P19-1)
Copied to clipboard
| Challenge: | Low-resource language pairs with a lack of parallel data pose challenges for machine translation . data augmentation using monolingual data is an effective way to alleviate the problem . |
| Approach: | They propose a general framework for data augmentation for low-resource machine translation using monolingual data and a related high-resourced language. |
| Outcome: | The proposed method improves translation quality by 1.5 to 8 BLEU points under extreme low-resource settings compared to baselines. |
Translation or Recitation? Calibrating Evaluation Scores for Machine Translation of Extremely Low-Resource Languages (2026.acl-short)
Copied to clipboard
| Challenge: | Existing studies show that performance across low-resource settings is variable, resulting in a significant barrier for the MT community. |
| Approach: | They propose to use FRED Difficulty Metrics to contextualize reported performance across different language pairs to determine whether breakthroughs reported in other contexts are artifacts of benchmark collection. |
| Outcome: | The proposed metrics explain a significant portion of result variability rather than model capability. |
Rethinking Network Pruning – under the Pre-train and Fine-tune Paradigm (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing pruning results on benchmark transformers, such as BERT, are not as remarkable as those of convolutional neural networks. |
| Approach: | They propose to apply a knowledge-aware pruning process to transformer-based pre-trained language models to reduce model size and model weight. |
| Outcome: | The proposed pruning method outperforms the leading competitors with a 20-times weight/FLOPs compression and neglectable loss in prediction accuracy. |
CoDa: Constrained Generation based Data Augmentation for Low-Resource NLP (2024.findings-naacl)
Copied to clipboard
| Challenge: | a low-resource dataset is limited in training data, so generating task-specific data is challenging. |
| Approach: | They propose a data augmentation technique that prompts off-the-shelf instruction-following Large Language Models to generate augmentations. |
| Outcome: | The proposed technique outperforms baselines on 11 datasets spanning 3 tasks and 3 low-resource settings. |
Leveraging Loanword Constraints for Improving Machine Translation in a Low-Resource Multilingual Context (2025.emnlp-main)
Copied to clipboard
| Challenge: | a recent study addresses the challenge of adapting loanwords during the translation process in low-resource languages. |
| Approach: | They propose a method that augments source sentences with loanword constraints . they then integrate loanwords as external linguistic knowledge into machine translation systems . |
| Outcome: | The proposed approach improves translation quality and handling loanword adaptation correctly in target languages. |