Papers by Kelly Chen
Mini-Model Adaptation: Efficiently Extending Pretrained Models to New Languages via Aligned Shallow Training (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to pretrain Masked Language Models (MLMs) are expensive and require a full forward and backward pass over the entire model. |
| Approach: | They propose to learn a shallow mini-model from a fraction of a large model's parameters and plug it into a larger model for rapid cross-lingual transfer. |
| Outcome: | Experiments on XNLI, MLQA and PAWS-X show that mini-model adaptation matches the standard approach using up to 2.3x less compute on average. |
Spoken Document Retrieval for an Unwritten Language: A Case Study on Gormati (2025.findings-emnlp)
Copied to clipboard
Sanjay Booshanam, Kelly Chen, Ondrej Klejch, Thomas Reitmaier, Dani Kalarikalayil Raju, Electra Wallington, Nina Markl, Jennifer Pearson, Matt Jones, Simon Robinson, Peter Bell
| Challenge: | Speakers of unwritten languages have the potential to benefit from speech-based automatic information retrieval systems. |
| Approach: | They propose a speech embedding technique that facilitates a zero-shot speech-based automatic information retrieval system for unwritten languages. |
| Outcome: | The proposed method achieves a Top 5 retrieval rate of 87.9% on a corpus of Gormati, an unwritten language, that was collected in partnership with an agrarian Banjara community in Maharashtra State, India. |
How Does Quantization Affect Multilingual LLMs? (2024.findings-emnlp)
Copied to clipboard
Kelly Marchisio, Saurabh Dash, Hongyu Chen, Dennis Aumiller, Ahmet Üstün, Sara Hooker, Sebastian Ruder
| Challenge: | Quantization is widely used to improve inference speed and deployment of large language models. |
| Approach: | They conduct a thorough analysis of quantized multilingual LLMs . they find language disparately affected by quantization, non-Latin script languages worst . authors urge consideration of multilingual performance as evaluation criterion for efficient models . |
| Outcome: | The results show that quantization has harmful effects on human evaluation . language performance is disparately affected by quantization, the authors say . |
Identity-Robust Language Model Generation via Content Integrity Preservation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies show that Large Language Model outputs vary across sociodemographic attributes . this causes disparities in factual accuracy, utility, and safety, even for objective questions . |
| Approach: | They propose a lightweight framework for identity-robust generation that neutralizes non-critical identity information while preserving semantically essential attributes. |
| Outcome: | The proposed framework reduces identity-dependent generation bias by 66.3% over vanilla prompting and outperforms existing prompt-based defenses. |