Papers by Kelly Marchisio
Mini-Model Adaptation: Efficiently Extending Pretrained Models to New Languages via Aligned Shallow Training (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to pretrain Masked Language Models (MLMs) are expensive and require a full forward and backward pass over the entire model. |
| Approach: | They propose to learn a shallow mini-model from a fraction of a large model's parameters and plug it into a larger model for rapid cross-lingual transfer. |
| Outcome: | Experiments on XNLI, MLQA and PAWS-X show that mini-model adaptation matches the standard approach using up to 2.3x less compute on average. |
RLHF Can Speak Many Languages: Unlocking Multilingual Preference Optimization for LLMs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Preference optimization is a widely adopted post-training technique to align large language models with human preferences. |
| Approach: | They propose a method for generating multilingual feedback data to balance data coverage. |
| Outcome: | The proposed method achieves 54.4% win-rate against current state-of-the-art multilingual LLM in its parameter class and 69.5% win- rate or higher against widely used models like Gemma, Mistral and Llama 3. |
Bilingual Lexicon Induction for Low-Resource Languages using Graph Matching via Optimal Transport (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing literature on bilingual lexicon induction fails in low-resource scenarios . a language dataset is considered low- resource based on its own embedding space . |
| Approach: | They propose a graph-matching method that improves bilingual lexicon induction performance across 40 language pairs using optimal transport. |
| Outcome: | The proposed method is especially strong with low amounts of supervision. |
On Systematic Style Differences between Unsupervised and Supervised MT and an Application for High-Resource Machine Translation (2022.naacl-main)
Copied to clipboard
| Challenge: | Modern unsupervised machine translation systems reach reasonable translation quality under clean and controlled data conditions. |
| Approach: | They compare unsupervised and supervised machine translation systems of similar quality . they combine the benefits of both methods into a single system . |
| Outcome: | The proposed system improves adequacy and fluency as measured by human evaluators. |
MAPS: A Multilingual Benchmark for Agent Performance and Security (2026.findings-eacl)
Copied to clipboard
Omer Hofman, Jonathan Brokman, Oren Rachmil, Shamik Bose, Vikas Pahuja, Toshiya Shimizu, Trisha Starostina, Kelly Marchisio, Seraphina Goldfarb-Tarrant, Roman Vainshtein
| Challenge: | Existing benchmarks do not provide a comprehensive, multi-domain, security-aware evaluation of multilingual agentic AI systems. |
| Approach: | They propose a multilingual benchmark suite to evaluate agentic AI systems across languages and tasks. |
| Outcome: | The proposed framework evaluates agentic AI systems across languages and tasks. |
The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Sparse attention is a promising strategy to extend long-context capabilities in LLMs . but its efficiency–accuracy trade-offs remain unclear due to the lack of comprehensive evaluation . |
| Approach: | They evaluate sparse attention methods across multiple model families and sizes . they find larger sparser models outperform smaller dense ones at equivalent cost . |
| Outcome: | The proposed methods outperform smaller sparse models at equivalent cost and improve the Pareto frontier. |
How Does Quantization Affect Multilingual LLMs? (2024.findings-emnlp)
Copied to clipboard
Kelly Marchisio, Saurabh Dash, Hongyu Chen, Dennis Aumiller, Ahmet Üstün, Sara Hooker, Sebastian Ruder
| Challenge: | Quantization is widely used to improve inference speed and deployment of large language models. |
| Approach: | They conduct a thorough analysis of quantized multilingual LLMs . they find language disparately affected by quantization, non-Latin script languages worst . authors urge consideration of multilingual performance as evaluation criterion for efficient models . |
| Outcome: | The results show that quantization has harmful effects on human evaluation . language performance is disparately affected by quantization, the authors say . |
AL-QASIDA: Analyzing LLM Quality and Accuracy Systematically in Dialectal Arabic (2025.findings-acl)
Copied to clipboard
| Challenge: | Dialectal Arabic (DA) varieties are under-served by language technologies, particularly large language models (LLMs). |
| Approach: | They propose a framework that comprehensively assesses LLMs’ DA modeling capabilities across four dimensions: fidelity, understanding, quality, and diglossia. |
| Outcome: | The proposed framework assesses LLMs’ DA modeling capabilities across four dimensions: fidelity, understanding, quality, and diglossia. |
IsoVec: Controlling the Relative Isomorphism of Word Embedding Spaces (2022.emnlp-main)
Copied to clipboard
| Challenge: | a faulty cross-lingual mapping technique results in non-isomorphic word embedding spaces . we address the root-cause of this problem by increasing the relative isomorphism of word embedsing spaces. |
| Approach: | They address the root-cause of faulty cross-lingual mapping by incorporating global measures of isomorphism into the skipgram loss function. |
| Outcome: | The proposed method improves bilingual lexicon induction under domain mismatch and with training algorithm dissimilarities. |
An Analysis of Euclidean vs. Graph-Based Framing for Bilingual Lexicon Induction from Word Embedding Spaces (2021.findings-emnlp)
Copied to clipboard
Kelly Marchisio, Youngser Park, Ali Saad-Eldin, Anton Alyakin, Kevin Duh, Carey Priebe, Philipp Koehn
| Challenge: | Existing work in bilingual lexicon induction views word embeddings as vectors in Euclidean space. |
| Approach: | They propose to use word embeddings as nodes in a weighted graph to examine a node’s graph neighborhood without assuming a linear transform. |
| Outcome: | The proposed approaches are compared under different data conditions and show that they complement each other when combined. |
Understanding and Mitigating Language Confusion in LLMs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Llama Instruct and Mistral models exhibit high degrees of language confusion and even the strongest models fail to consistently respond in the correct language. |
| Approach: | They develop a language confusion benchmark to evaluate LLMs' inability to consistently generate text in a user’s desired language. |
| Outcome: | The proposed model fails to consistently respond in the correct language, despite being prone to high temperatures and complex prompts. |
Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation (2025.acl-long)
Copied to clipboard
Shivalika Singh, Angelika Romanou, Clémentine Fourrier, David Ifeoluwa Adelani, Jian Gang Ngui, Daniel Vila-Suero, Peerat Limkonchotiwat, Kelly Marchisio, Wei Qi Leong, Yosephine Susanto, Raymond Ng, Shayne Longpre, Sebastian Ruder, Wei-Yin Ko, Antoine Bosselut, Alice Oh, Andre Martins, Leshem Choshen, Daphne Ippolito, Enzo Ferrante, Marzieh Fadaee, Beyza Ermis, Sara Hooker
| Challenge: | Reliable multilingual evaluation is difficult and culturally appropriate evaluation is even harder to achieve. |
| Approach: | They propose a multilingual evaluation framework that aims to mitigate these biases by improving translations and annotation practices. |
| Outcome: | The proposed framework improves translation quality and cultural coverage and is culturally sensitive and culturally agnostic. |