Visual Pivoting Unsupervised Multimodal Machine Translation in Low-Resource Distant Language Pairs (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies show that neural MT achieves much worse translation quality than statistical MT with a small number of corpora. |
| Approach: | They propose a visual pivoting method for alignment between distant language pairs . they first construct a dataset and then apply it to pre-training and fine-tuning . |
| Outcome: | The proposed method outperforms baselines on DLPs and close language pairs. |
Similar Papers
Scene Graph as Pivoting: Inference-time Image-free Unsupervised Multimodal Machine Translation with Visual Scene Hallucination (2023.acl-long)
Copied to clipboard
| Challenge: | Current neural machine translation (NMT) relies on parallel sentences, which obstructs the development of NMT for minor languages. |
| Approach: | They propose an unsupervised multimodal machine translation setup where the model is trained with source-text image pairs and tested with only source- text inputs. |
| Outcome: | The proposed model outperforms the baseline model on the task and setup, helping yield translations with better completeness, relevance and fluency without relying on paired images. |
Unsupervised Multimodal Neural Machine Translation with Pseudo Visual Pivoting (2020.acl-main)
Copied to clipboard
| Challenge: | Unsupervised machine translation (MT) has recently achieved impressive results with monolingual corpora. |
| Approach: | They propose to utilize visual content for disambiguation and promoting latent space alignment in unsupervised machine translation by using multimodal back-translation and pseudo visual pivoting. |
| Outcome: | The proposed model improves over state-of-the-art methods and generalizes well when images are not available at the testing time. |
Unsupervised Pivot Translation for Distant Languages (P19-1)
Copied to clipboard
| Challenge: | Unsupervised neural machine translation (NMT) is a popular method for transferring information between languages. |
| Approach: | They propose an unsupervised pivot translation method which translates a language to a distant language through multiple hops. |
| Outcome: | The proposed method improves translation on 20 languages and 294 distant languages on 20 different languages and language pairs. |
Pivot-based Transfer Learning for Neural Machine Translation between Non-English Languages (D19-1)
Copied to clipboard
| Challenge: | Using parallel corpora, we train a single, direct NMT model for non-English language pairs. |
| Approach: | They propose three ways to increase the relation among source, pivot, and target languages in pre-training . they use additional adapter component to smoothly connect pre-trained encoder and decoder . |
| Outcome: | The proposed methods outperform multilingual models up to +2.6% BLEU in WMT 2019 French-German and German-Czech tasks. |
AlignMMBench: Evaluating Chinese Multimodal Alignment in Large Vision-Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks focus on basic abilities using nonverbal methods, such as yes-no and multiple-choice questions. |
| Approach: | They propose a benchmark that provides more nuanced evaluations of alignment capabilities for large Vision-Language Models (VLMs) they use a rule-calibrated evaluator that exceeds GPT-4's evaluation ability and a “alignment score” to assess the robustness and stability of models across diverse prompts. |
| Outcome: | The proposed benchmark covers 13 tasks across three categories and includes both single-turn and multi-turn dialogue scenarios. |
Multilingual-To-Multimodal (M2M): Unlocking New Languages with Monolingual Text (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing multimodal models rely on machine translation, but performance drops for other languages due to limited multilingual multimodal resources. |
| Approach: | They propose a lightweight alignment method that learns only a few linear layers using English text alone to map multilingual text embeddings into multimodal space. |
| Outcome: | M2M achieves strong zero-shot transfer on XTD Text-to-Image retrieval in English and spanish . it learns only a few linear layers to map multilingual text embeddings into multimodal space . |
Neural Machine Translation between Low-Resource Languages with Synthetic Pivoting (2024.lrec-main)
Copied to clipboard
| Challenge: | Pivot-based neural machine translation systems overcome data scarcity by including a high-resource pivot language in the process of translating between low-resourced languages. |
| Approach: | They propose a novel approach to pivot-based translation in which pivot sentences are generated synthetically from both the source and target languages. |
| Outcome: | The proposed approach improves pivot-based systems translating between low-resource Southern African languages by up to 5.6 BLEU points after fine-tuning. |
Unsupervised Cross-Lingual Representation Learning (P19-4)
Copied to clipboard
| Challenge: | a comprehensive survey of cutting-edge weakly-supervised and unsupervised cross-lingual word representations is presented . |
| Approach: | This tutorial provides a comprehensive survey of recent work on weakly-supervised and unsupervised cross-lingual word representations. |
| Outcome: | This tutorial provides a comprehensive survey of cutting-edge weakly-supervised and unsupervised word representations. |
Single-to-mix Modality Alignment with Multimodal Large Language Model for Document Image Machine Translation (2025.acl-long)
Copied to clipboard
| Challenge: | Document Image Machine Translation (DIMT) faces generalization challenges due to limited training data and the complex interplay between visual and textual information. |
| Approach: | They propose a single-to-mix Modality alignment framework leveraging Multimodal Large Language Models (MLLMs) this framework aligns an imageonly encoder with multimodal representations of an MLLM pre-trained on large-scale document image datasets. |
| Outcome: | The proposed framework improves translation quality in cross-domain generalization and challenging document image scenarios. |
NiuTrans.LMT: Toward Inclusive and Scalable Multilingual Machine Translation with LLMs (2026.acl-long)
Copied to clipboard
Yingfeng Luo, Ziqiang Xu, Yuxuan Ouyang, MuRun Yang, DingYang Lin, Kaiyan Chang, Tong Zheng, Bei Li, Peinan Feng, Quan Du, Tong Xiao, JingBo Zhu
| Challenge: | Large language models have significantly advanced Multilingual Machine Translation (MMT) yet scaling to many languages while maintaining robust performance across directions remains challenging. |
| Approach: | They propose a strategy to reduce the number of translations in one direction . they propose auxiliary parallel sentences to promote cross-lingual transfer . |
| Outcome: | The proposed model performs on par with or better than substantially larger baselines. |