Cross-lingual Matryoshka Representation Learning across Speech and Text (2026.findings-acl)
Copied to clipboard
| Challenge: | Speakers of under-represented languages face language barriers and modality barriers . we train a bilingual speech-text embedding model for French-Wolof . |
| Approach: | They train a bilingual speech-text Matryoshka embedding model that enables efficient retrieval of French text from Wolof speech queries. |
| Outcome: | The proposed model can retrieve French text from Wolof speech queries without expensive ASR-translation pipelines. |
Similar Papers
Matryoshka-Adaptor: Unsupervised and Supervised Tuning for Smaller Embedding Dimensions (2024.emnlp-main)
Copied to clipboard
| Challenge: | Embeddings from Large Language Models (LLMs) have emerged as critical components in information retrieval applications. |
| Approach: | They propose a tuning framework for the customization of LLM embeddings. |
| Outcome: | The proposed framework reduces embedding dimensions while maintaining comparable performance levels. |
SMEC:Rethinking Matryoshka Representation Learning for Retrieval Embedding Compression (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models generate high-dimensional embeddings that capture rich semantic and syntactic information. |
| Approach: | They propose a training framework to reduce dimensionality and complexity of large language models. |
| Outcome: | Experiments on image, text, and multimodal datasets show that the proposed training framework reduces dimensionality while maintaining performance. |
Leveraging a Bilingual Dictionary to Learn Wolastoqey Word Representations (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing word embeddings for lowresource languages require large corpora of running text to learn high quality representations. |
| Approach: | They leverage a bilingual dictionary to learn Wolastoqey word embeddings by encoding their corresponding English definitions into vector representations using pretrained English word and sequence representation models. |
| Outcome: | The proposed model outperforms baseline models without language-specific training or fine-tuning. |
Unsupervised Cross-lingual Representation Learning at Scale (2020.acl-main)
Copied to clipboard
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, Veselin Stoyanov
| Challenge: | Pretraining multilingual language models at scale leads to performance gains for cross-lingual transfer tasks. |
| Approach: | They present a transformer-based multilingual masked language model pre-trained on 100 languages . they show that pretraining multilingual models at scale leads to significant performance gains . |
| Outcome: | The proposed model outperforms multilingual BERT (mBERT) on cross-lingual benchmarks. |
MIPIC: Matryoshka Representation Learning via Self-Distilled Intra-Relational and Progressive Information Chaining (2026.findings-acl)
Copied to clipboard
Phung Gia Huy, Hai An Vu, Minh-Phuc Truong, Thang Duc Tran, Linh Ngo Van, Thanh Hong Nguyen, Trung Le
| Challenge: | Existing approaches to train dense representations require explicit coordination of how information is arranged across embedding dimensionality and model depth. |
| Approach: | They propose a framework that trains Matryoshka representations using self-distilled intra-relational alignment and Progressive information chaining. |
| Outcome: | The proposed framework produces coherent and compact Matryoshka representations with significant performance advantages under low-dimensional models. |
Unsupervised Cross-Lingual Representation Learning (P19-4)
Copied to clipboard
| Challenge: | a comprehensive survey of cutting-edge weakly-supervised and unsupervised cross-lingual word representations is presented . |
| Approach: | This tutorial provides a comprehensive survey of recent work on weakly-supervised and unsupervised cross-lingual word representations. |
| Outcome: | This tutorial provides a comprehensive survey of cutting-edge weakly-supervised and unsupervised word representations. |
Language Embeddings for Typology and Cross-lingual Transfer Learning (2021.acl-long)
Copied to clipboard
| Challenge: | Recent efforts to leverage multilingual datasets highlight potential of multilingual models that can perform well across various languages. |
| Approach: | They propose to generate language representations that capture relationships among languages and evaluate them using WALS and two extrinsic tasks. |
| Outcome: | The proposed model can be leveraged in cross-lingual tasks without parallel data . the proposed model is based on the World Atlas of Language Structures (WALS) and two extrinsic tasks . |
Can Textual Unlearning Solve Cross-Modality Safety Alignment? (2024.findings-emnlp)
Copied to clipboard
Trishna Chakraborty, Erfan Shayegani, Zikui Cai, Nael Abu-Ghazaleh, M. Salman Asif, Yue Dong, Amit Roy-Chowdhury, Chengyu Song
| Challenge: | integrating new modalities into large language models creates new attack surface . existing safety training techniques like SFT and RLHF are not feasible in multi-modal settings . |
| Approach: | They explore whether unlearning in the textual domain can be effective for cross-modality safety alignment. |
| Outcome: | The proposed approach reduces the Attack Success Rate (ASR) to less than 8% and preserves the utility. |
Give your Text Representation Models some Love: the Case for Basque (2020.lrec-1)
Copied to clipboard
Rodrigo Agerri, Iñaki San Vicente, Jon Ander Campos, Ander Barrena, Xabier Saralegi, Aitor Soroa, Eneko Agirre
| Challenge: | Word embeddings and pre-trained language models are expensive to train and are often used by small companies and research groups to build their own. |
| Approach: | They propose to use word embeddings and pre-trained language models to build rich representations of text and improve NLP tasks. |
| Outcome: | The proposed models perform better than publicly available versions in downstream NLP tasks for Basque. |
Unsupervised Cross-lingual Transfer of Word Embedding Spaces (D18-1)
Copied to clipboard
| Challenge: | Existing methods for cross-lingual word mapping require cross-linguistic supervision, but this is not available for many low resource languages. |
| Approach: | They propose an unsupervised method that learns transformation functions over corresponding word embedding spaces using a distributed distributional matching algorithm. |
| Outcome: | The proposed method performs better on bilingual lexicon induction and cross-lingual word similarity prediction datasets than other supervised and unsupervised methods. |