Parameter-Efficient Cross-lingual Transfer of Vision and Language Models via Translation-based Alignment

Challenge:	Existing cross-lingual transfer methods that use labeled data and linguistic resources would consume excessive resources for a large number of languages.
Approach:	They propose a parameter-efficient cross-lingual transfer learning framework that utilizes a translation-based alignment method to mitigate multilingual disparities.
Outcome:	The proposed framework reduces disparities among languages and improves cross-lingual transfer results in low-resource scenarios while keeping and fine-tuning only a small number of parameters.

Similar Papers

Overview

Challenge

Approach

Outcome

A Simple and Effective Method to Improve Zero-Shot Cross-Lingual Transfer Learning (2022.coling-1)

Copied to clipboard

Kunbo Ding, Weijie Liu, Yuejian Fang, Weiquan Mao, Zhe Zhao, Tao Zhu, Haoyan Liu, Rong Tian, Yiren Chen

Semantic Text Processing

Challenge:	Existing zero-shot cross-lingual transfer methods rely on parallel corpora or bilingual dictionaries . however, its effect is limited by the gap between embedding clusters of different languages .
Approach:	They propose Embedding-Push, Attention-Pull, and Robust targets to transfer English embeddings to virtual multilingual embedders without semantic loss.
Outcome:	Experimental results show that the proposed method outperforms existing methods on cross-lingual tasks and can achieve a better multilingual alignment.

Efficiently Aligned Cross-Lingual Transfer Learning for Conversational Tasks using Prompt-Tuning (2024.findings-eacl)

Copied to clipboard

Lifu Tu, Jin Qu, Semih Yavuz, Shafiq Joty, Wenhao Liu, Caiming Xiong, Yingbo Zhou

Responsible & Trustworthy NLP

Challenge:	Cross-lingual transfer of language models trained on high-resource languages such as English has been limited due to the high cost of obtaining non-English conversational data.
Approach:	They introduce a parallel and large-scale multilingual conversation dataset that is used for cross-lingual alignment pretraining by translating the English-only Schema-Guided Dialogue dataset into 105 other languages.
Outcome:	The proposed model performs well on slot-filling and intent classification tasks, and is able to perform well in other languages.

Multilingual Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer of Vision-Language Models (2021.naacl-main)

Copied to clipboard

Po-Yao Huang, Mandela Patrick, Junjie Hu, Graham Neubig, Florian Metze, Alexander Hauptmann

Language Models

Challenge:	a new study examines zero-shot cross-lingual transfer of vision-language models . we study multilingual text-to-video search in non-English languages without annotations .
Approach:	They propose a Transformer-based model that learns contextual multilingual multimodal embeddings . they propose 'zero-shot cross-lingual transfer' to improve multilingual search .
Outcome:	The proposed model outperforms baselines on multilingual text-to-video search and multilingual image search on VTT and VATEX.

Optimal Transport Posterior Alignment for Cross-lingual Semantic Parsing (2023.tacl-1)

Copied to clipboard

Tom Sherborne, Tom Hosking, Mirella Lapata

Semantic Text Processing

Challenge:	Existing work on cross-lingual semantic parsing has focused on English . a few-shot approach to parse from natural languages is comparatively unexplored .
Approach:	They propose a method that minimizes cross-lingual divergence between probabilistic latent variables by Optimal Transport.
Outcome:	The proposed method improves performance even without parallel input translations on two datasets.

Cross-Lingual Representation Alignment Through Contrastive Image-Caption Tuning (2025.acl-short)

Copied to clipboard

Nathaniel Krasner, Nicholas Lanuzo, Antonios Anastasopoulos

Multimodality

Challenge:	Multilingual alignment of sentence representations has mostly required bitexts to bridge the gap between languages.
Approach:	They propose to use image captions to implicitly align text representations between languages to make them usable for cross-lingual Natural Language Understanding (NLU) and bitext retrieval.
Outcome:	The proposed approach is usable for cross-lingual Natural Language Understanding (NLU) and bitext retrieval.

HyperPELT: Unified Parameter-Efficient Language Model Tuning for Both Language and Vision-and-Language Tasks (2023.findings-acl)

Copied to clipboard

Zhengkun Zhang, Wenya Guo, Xiaojun Meng, Yasheng Wang, Yadao Wang, Xin Jiang, Qun Liu, Zhenglu Yang

Language Models

Challenge:	Pretraining and fine-tuning are the dominant paradigms in natural language processing.
Approach:	They propose a parameter-efficient multitask learning framework that takes trainable hyper-embeddings and visual modality as input and outputs weights for different modules in a pretrained language model.
Outcome:	The proposed framework adds fewer trainable parameters in multi-task learning while achieving superior performances and transfer ability compared to state-of-the-art methods.

Multilingual Pixel Representations for Translation and Effective Cross-lingual Transfer (2023.emnlp-main)

Copied to clipboard

Elizabeth Salesky, Neha Verma, Philipp Koehn, Matt Post

Multilinguality

Challenge:	Recent work shows that pixel representations can be finetuned across scripts without vocabulary extensions, adapters, or transliteration.
Approach:	They propose to use pixel representations to train multilingual machine translation models . they explore parameter sharing within and across scripts to better understand where they lead to positive transfer .
Outcome:	The proposed model improves on two multilingual datasets with different language coverage compared to subword embeddings . the proposed model can be finetuned cross-lingually or to unseen scripts, and is more data-efficient than other alternatives such as vocabulary expansion .

Cross-Align: Modeling Deep Cross-lingual Interactions for Word Alignment (2022.emnlp-main)

Copied to clipboard

Siyu Lai, Zhen Yang, Fandong Meng, Yufeng Chen, Jinan Xu, Jie Zhou

Multilinguality

Challenge:	Existing word alignment models capture few interactions between input sentence pairs, which severely degrades the word alignment quality.
Approach:	They propose to model deep interactions between input and target sentences using a two-stage training framework to train the model.
Outcome:	The proposed model achieves the state-of-the-art (SOTA) performance on four out of five language pairs.

Cross-lingual and Multilingual CLIP (2022.lrec-1)

Copied to clipboard

Fredrik Carlsson, Philipp Eisen, Faton Rekathati, Magnus Sahlgren

Multilinguality

Challenge:	OpenAI released CLIP, a model that relates the textual and visual domains with unprecedented accuracy.
Approach:	They propose to use cross-lingual teacher learning to re-train an English textual encoder using a large dataset of images and captions.
Outcome:	The proposed method outperforms baselines on multilingual image-text retrieval while retaining low cost.

Unifying Cross-Lingual Transfer across Scenarios of Resource Scarcity (2023.emnlp-main)

Copied to clipboard

Alan Ansell, Marinela Parović, Ivan Vulić, Anna Korhonen, Edoardo Ponti

Multilinguality

Challenge:	Existing approaches to deal with resource scarcity have not been developed to deal effectively with the problem.
Approach:	They propose to use a set of tools to harness data from one or more high-resource "source" languages to compensate for a shortage of data in low-resourced "target" languages.
Outcome:	The proposed technique can be easily adapted to unseen languages, extending the range of the proposed technique and translation-based transfer more broadly.