Preserving Multilingual Quality While Tuning Query Encoder on English Only (2025.naacl-short)
Copied to clipboard
| Challenge: | Xiong et al., 2020, 2021b) and Dong eet., 2022) . Switching from one query encoder to another is easily feasible . |
| Approach: | They propose a general tuning technique that can be used to modify query representations for specific types of queries or domains while keeping precomputed and stored documents intact. |
| Outcome: | The proposed model preserves multilingual qualities and improves embedding qualities on different datasets. |
Similar Papers
Recipes for Adapting Pre-trained Monolingual and Multilingual Models to Machine Translation (2021.eacl-main)
Copied to clipboard
| Challenge: | Recent advances in machine translation (MT) have improved performance on low-resource language pairs. |
| Approach: | They propose to freeze most BART parameters and add new ones to fine-tune a model trained on MT. |
| Outcome: | The proposed model outperforms naive fine-tuning on Vietnamese to English on a training set for Vietnamese to Vietnamese . the proposed model is able to fine- tune on smaller datasets while still maintaining the same model performance. |
When Does Mixing Help? Analyzing Query Embedding Interpolation in Multilingual Dense Retrieval (2026.acl-long)
Copied to clipboard
| Challenge: | Existing multilingual retrievers are largely untested for mixed language queries . sensitivity of dense retrievers to mixed language querying remains poorly understood . |
| Approach: | They propose to use mixed queries as an interpolation of monolingual embeddings to evaluate retrieval performance. |
| Outcome: | The proposed model outperforms the best monolingual endpoint in 88/105 cases. |
Monolingual or Multilingual Instruction Tuning: Which Makes a Better Alpaca (2024.findings-eacl)
Copied to clipboard
| Challenge: | Foundational large language models (LLMs) can be instruction-tuned to perform open-domain question answering, facilitating applications like chat assistants. |
| Approach: | They employ a dataset and machine translations of it to form multilingual data and use it to tune LLMs. |
| Outcome: | The proposed model is on par or better than a model for each language, and multilingual tuning with downsampled data is as powerful and robust. |
Dissecting Fine-Tuning Unlearning in Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning-based unlearning are ineffective at completely erasing model-embedded knowledge, but their true effectiveness remains unclear. |
| Approach: | They propose to use activation patching and parameter restoration experiments to examine the limitations of fine-tuning-based unlearning methods for erasing harmful, sensitive, or copyrighted information within large language models. |
| Outcome: | The proposed methods alter the model’s knowledge retrieval process rather than genuinely erasing the problematic knowledge embedded in the model parameters. |
Empowering Dual-Encoder with Query Generator for Cross-Lingual Dense Retrieval (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to distill knowledge from cross-encoder re-ranker to dual-encoding retriever are lacking in the cross-lingual setting. |
| Approach: | They propose to use a query generator as the teacher in the cross-lingual setting to distill knowledge to a dual-encoder retrieval model. |
| Outcome: | The proposed method outperforms state-of-the-art methods on two benchmark datasets. |
Revisiting Query Variation Robustness of Transformer Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Despite their proficiency with natural language, transformer-based large language models are not robust to query variations such as typos and paraphrases. |
| Approach: | They extend their findings to include more recent large language models . they find that instruct-LLMs are more robust to query variations . |
| Outcome: | The proposed model can be prompted for robustness by a set of instruction-tuned LLMs. |
Investigating Multilingual Instruction-Tuning: Do Polyglot Models Demand for Multilingual Instructions? (2024.emnlp-main)
Copied to clipboard
Alexander Weber, Klaudia Thellmann, Jan Ebert, Nicolas Flores-Herr, Jens Lehmann, Michael Fromm, Mehdi Ali
| Challenge: | a study of multilingual pre-trained LLMs on parallel instruction-tuning benchmarks shows that instruction-following models can be used across languages by up to 9.9%. |
| Approach: | They conduct an extensive study of the performance of multilingual pre-trained LLMs instruction-tuned on parallel instruction-uning datasets. |
| Outcome: | The proposed model improves cross-lingual instruction following capabilities by 9.9% on a large and mid-sized LLM on parallel instruction-tuning datasets. |
Fine-Tuned Machine Translation Metrics Struggle in Unseen Domains (2024.acl-short)
Copied to clipboard
| Challenge: | a new dataset examines whether fine-tuned metrics are robust to domain shifts between training and inference. |
| Approach: | They use an annotated multidimensional quality metrics dataset to examine whether they are robust to domain shifts between training and inference. |
| Outcome: | The proposed metrics exhibit a substantial performance drop in the unseen domain scenario compared to metrics that rely on the surface form and pre-trained metrics that are not fine-tuned on MT quality judgments. |
Multilingual Instruction Tuning With Just a Pinch of Multilinguality (2024.findings-acl)
Copied to clipboard
| Challenge: | Using multilingual instruction tuning, large language models can be used to follow instructions in multiple languages . a multilingual model can be tuned on a wide range of languages, yet most datasets are limited to English . |
| Approach: | They investigate how multilinguality during instruction tuning affects instruction-following across languages . they find that only 40 multilingual examples improve multilingual instruction- follow . |
| Outcome: | The results show that multilingual models perform better on multilingual mixtures compared to monolingual models . the results suggest that building multilingual instruction-tuned models can be done with only 2-4 languages . |
Fine-tuning Encoders for Improved Monolingual and Zero-shot Polylingual Neural Topic Modeling (2021.naacl-main)
Copied to clipboard
| Challenge: | Topic models can augment or replace bag-of-words inputs with pre-trained transformer-based word prediction models. |
| Approach: | They propose several methods for fine-tuning encoders to improve both monolingual and zero-shot polylingual topic modeling. |
| Outcome: | The proposed methods improve both monolingual and zero-shot polylingual topic modeling. |