Papers by Rémi Lebret
Aligning Multilingual Word Embeddings for Cross-Modal Retrieval Task (D19-66)
Copied to clipboard
| Challenge: | Existing methods to learn multimodal multilingual embeddings for text and image retrieval tasks are limited to English. |
| Approach: | They propose a new approach to learn multimodal multilingual embeddings for matching images and captions in two languages by combing two existing objective functions and adapting alignment between existing languages. |
| Outcome: | The proposed model achieves state-of-the-art in retrieval and caption-caption tasks while adapting existing language alignments. |
Stop Pre-Training: Adapt Visual-Language Models to Unseen Languages (2023.acl-short)
Copied to clipboard
| Challenge: | Existing studies have shown that the pre-training in English does not transfer well to other languages in a zero-shot setting. |
| Approach: | They propose a simple yet efficient approach to adapt VLP to unseen languages using MPLM. |
| Outcome: | The proposed approach outperforms state-of-the-art models without large parallel corpora across three tasks. |
Revisiting Offline Compression: Going Beyond Factorization-based Methods for Transformer Language Models (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent transformer language models achieve outstanding results on many downstream tasks, but their enormous size often makes them impractical on memory-constrained devices. |
| Approach: | They propose an offline compression approach that reduces the complexity of the model by enabling collaboration between modules. |
| Outcome: | The proposed approach outperforms commonly used factorization-based offline compression methods on various NLP tasks. |
Aligning Multilingual Word Embeddings for Cross-Modal Retrieval Task (D19-64)
Copied to clipboard
| Challenge: | Existing methods to learn multimodal multilingual embeddings for text and image retrieval tasks are limited to English. |
| Approach: | They propose a new approach to learn multimodal multilingual embeddings for matching images and captions in two languages by combing two existing objective functions and adapting alignment between existing languages. |
| Outcome: | The proposed model achieves state-of-the-art in retrieval and caption-caption tasks while adapting existing language alignments. |
Discovering Language-neutral Sub-networks in Multilingual Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | a recent study shows that multilingual pre-trained language models transfer well on cross-lingual downstream tasks. |
| Approach: | They conceptualize language neutrality as a function of overlap between language-encoding sub-networks of multilingual models. |
| Outcome: | The proposed model performs well on cross-lingual tasks despite being pre-trained on multiple languages . |
WikiMixQA: A Multimodal Benchmark for Question Answering over Tables and Charts (2025.findings-acl)
Copied to clipboard
Negar Foroutan, Angelika Romanou, Matin Ansaripour, Julian Martin Eisenschlos, Karl Aberer, Rémi Lebret
| Challenge: | Documents are fundamental to preserving and disseminating information, often incorporating complex layouts, tables, and charts that pose significant challenges for automatic document understanding (DU). |
| Approach: | They propose a benchmark for evaluating cross-modal reasoning over tables and charts extracted from 4,000 Wikipedia pages . they evaluate 12 vision-language models that achieve 70% accuracy when provided with direct context . |
| Outcome: | The proposed benchmark evaluates models with high accuracy over tables and charts extracted from 4,000 Wikipedia pages . proprietary models achieve 70% accuracy when provided with direct context, but open-source models perform worse when retrieval from long documents is required. |