Papers by Philipp Koehn

The FLORES Evaluation Datasets for Low-Resource Machine Translation: Nepali–English and Sinhala–English (D19-1)

Challenge:	Recent pruning methods remove redundant parameters according to parameter sensitivity, a gradient-based measure reflecting the contribution of the parameters.
Approach:	They propose a general task-agnostic method to balance parameter sensitivity and a novel adaptive learning method to control strength of intra-distillation loss for faster convergence.
Outcome:	The proposed method can reduce redundant parameters by over 80% without obvious performance degradation.

Copied to clipboard

Francisco Guzmán, Peng-Jen Chen, Myle Ott, Juan Pino, Guillaume Lample, Philipp Koehn, Vishrav Chaudhary, Marc’Aurelio Ranzato

Learn and Unlearn: Addressing Misinformation in Multilingual LLMs (2025.emnlp-main)

Challenge:	a vast majority of language pairs in the world are considered low-resource because they have little parallel data available.
Approach:	They propose to use a dataset to evaluate methods trained on low-resource language pairs . they report baseline performance using supervised, weakly supervised and semi-supervised settings .
Outcome:	The proposed evaluation datasets show that current state-of-the-art methods perform poorly on this benchmark, posing a challenge to the research community working on low-resource MT.

Copied to clipboard

TaiMing Lu, Philipp Koehn

Responsible & Trustworthy NLP

Challenge:	Existing methods to unlearning large language models (LLMs) focus on English data, but they ignore multilingual contexts and can produce misleading, offensive, or otherwise fake content.
Approach:	They investigate the propagation of information in multilingual large language models and evaluate unlearning methods to address harmful content in multi-lingual contexts.
Outcome:	The proposed methods can effectively eliminate harmful content for all languages by addressing both English and the original language of the harmful data.

CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairs (2020.emnlp-main)

Copied to clipboard

Ahmed El-Kishky, Vishrav Chaudhary, Francisco Guzmán, Philipp Koehn

Alternative Input Signals Ease Transfer in Multilingual Machine Translation (2022.acl-long)

Challenge:	Cross-lingual document alignment aims to identify pairs of documents in two distinct languages that are of comparable content or translations of each other.
Approach:	They exploit the signals embedded in URLs to label web documents at scale with an average precision of 94.5% across different language pairs.
Outcome:	The proposed method can label documents at 94.5% across languages with high precision . the proposed method is useful for low-resource languages with limited resources .

Copied to clipboard

Simeng Sun, Angela Fan, James Cross, Vishrav Chaudhary, Chau Tran, Philipp Koehn, Francisco Guzmán

Recovering document annotations for sentence-level bitext (2024.findings-acl)

Challenge:	Recent work in multilingual machine translation (MMT) has focused on the potential of positive transfer between languages.
Approach:	They propose to augment training data with alternative signals that unify different writing systems, such as phonetic, romanized, and transliterated input.
Outcome:	The proposed model outperforms strong ensemble baselines on Indic and Turkic languages by 1.3 BLEU points on both languages.

Copied to clipboard

Rachel Wicks, Matt Post, Philipp Koehn

De-Mixing Sentiment from Code-Mixed Text (P19-2)

Challenge:	In machine translation, historical models were incapable of handling longer contexts, so the lack of document-level datasets was less noticeable.
Approach:	They propose a document-level filtering technique that discards document- level metadata.
Outcome:	The proposed method improves translation without degradation of sentence-level translation.

Copied to clipboard

Yash Kumar Lal, Vaibhav Kumar, Mrinal Dhar, Manish Shrivastava, Philipp Koehn

Context and Copying in Neural Machine Translation (D18-1)

Challenge:	Code-mixing is the phenomenon of mixing the vocabulary and syntax of multiple languages in the same sentence.
Approach:	They propose a hybrid architecture for the task of Sentiment Analysis of English-Hindi code-mixed data using CNNs to generate subword representations for the sentences.
Outcome:	The proposed architecture achieves 83.54% accuracy and 0.827 F1 score on a benchmark dataset.

Copied to clipboard

Rebecca Knowles, Philipp Koehn

Where are you from? Geolocating Speech and Applications to Language Identification (2024.naacl-long)

Challenge:	Neural machine translation systems with subword vocabularies can translate or copy unknown words . we examine the influence of context and subword features on copying behavior .
Approach:	They show that neural machine translation systems with subword vocabularies can translate unknown words . they also learn to copy words based on context and features of the words themselves .
Outcome:	The proposed model outperforms phrase-based statistical machine translation systems on translation of unknown words.

Copied to clipboard

Patrick Foley, Matthew Wiesner, Bismarck Odoom, Leibny Paola Garcia Perera, Kenton Murray, Philipp Koehn

Information Extraction & Text Mining

Challenge:	Language identification (LID) is a critical component in many modern multilingual speech technologies.
Approach:	They propose to use radio broadcasts with known origin to train regression models . they also propose to explore using geolocation as a proxy task for LID .
Outcome:	The proposed model outperforms pretrained models on the FLEURS benchmark and on the VoxLingua benchmark.

HABLex: Human Annotated Bilingual Lexicons for Experiments in Machine Translation (D19-1)

Copied to clipboard

Brian Thompson, Rebecca Knowles, Xuan Zhang, Huda Khayrallah, Kevin Duh, Philipp Koehn

Bilingual Lexicon Induction for Low-Resource Languages using Graph Matching via Optimal Transport (2022.emnlp-main)

Challenge:	Existing methods to incorporate bilingual lexicons into statistical machine translation are unclear how to do so in the neural framework.
Approach:	They present a dataset to test methods for bilingual lexicon integration into neural machine translation using human generated alignments of words and phrases in three language pairs.
Outcome:	The proposed method improves on baselines and improves training to address overfitting.

Copied to clipboard

Kelly Marchisio, Ali Saad-Eldin, Kevin Duh, Carey Priebe, Philipp Koehn

Structured Data in NLP

Challenge:	Existing literature on bilingual lexicon induction fails in low-resource scenarios . a language dataset is considered low- resource based on its own embedding space .
Approach:	They propose a graph-matching method that improves bilingual lexicon induction performance across 40 language pairs using optimal transport.
Outcome:	The proposed method is especially strong with low amounts of supervision.

HiMATE: A Hierarchical Multi-Agent Framework for Machine Translation Evaluation (2025.findings-emnlp)

Copied to clipboard

Shijie Zhang, Renhao Li, Songsheng Wang, Philipp Koehn, Min Yang, Derek F. Wong

Adapting High-resource NMT Models to Translate Low-resource Related Languages without Parallel Data (2021.acl-long)

Challenge:	Existing LLM-based evaluation methods fail to accurately identify error spans and assess their severity.
Approach:	They propose a Hierarchical Multi-Agent Framework for Machine Translation Evaluation based on the MQM error typology and a hierarchical multi-agent system enabling granular evaluation of subtype errors.
Outcome:	The proposed framework outperforms baselines in error span detection and severity assessment.

Copied to clipboard

Wei-Jen Ko, Ahmed El-Kishky, Adithya Renduchintala, Vishrav Chaudhary, Naman Goyal, Francisco Guzmán, Pascale Fung, Philipp Koehn, Mona Diab

Spelling-Aware Construction of Macaronic Texts for Teaching Foreign-Language Vocabulary (D19-1)

Challenge:	linguistic overlap between low-resource languages and high-resourced languages is a major obstacle for training high-quality machine translation systems.
Approach:	They exploit linguistic overlap to facilitate translation to and from low-resource languages . they use monolingual data and parallel data in related high-resourced languages based on their method .
Outcome:	The proposed method significantly improves translation into low-resource language compared to baselines on 7 languages from three different language families.

Copied to clipboard

Adithya Renduchintala, Philipp Koehn, Jason Eisner

Learn To Remember: Transformer with Recurrent Memory for Document-Level Machine Translation (2022.findings-naacl)

Challenge:	a machine foreign-language teacher replaces word tokens with glosses in a foreign language to ease the human reader into understanding the L2 vocabulary.
Approach:	They propose a machine foreign-language teacher that modifies text by replacing word tokens with glosses in a foreign language to ease the human reader into understanding the L2 .
Outcome:	The proposed model can learn representations for novel words and is a proxy for word guessing and learning ability of real human students.

Copied to clipboard

Yukun Feng, Feng Li, Ziang Song, Boyuan Zheng, Philipp Koehn

Statistical Power and Translationese in Machine Translation Evaluation (2020.emnlp-main)

Challenge:	Recent studies have shown that the effective use of contextual information between sentences can achieve better performance in document-level machine translation.
Approach:	They propose a recurrent memory unit to the Transformer to support the information exchange between the sentence and previous context.
Outcome:	The proposed model outperforms the previous work on TED and News by 0.91 s-BLEU and 1.49 d-BLUE on average.

Copied to clipboard

Yvette Graham, Barry Haddow, Philipp Koehn

Data Selection Curriculum for Neural Machine Translation (2022.findings-emnlp)

Challenge:	a recent paper argues that translationese has been used to describe features of translated text . a translationed text can be more explicit than the original source, authors say . authors recommend reverse-created test data be omitted from future evaluations .
Approach:	They propose to omit translationese from future machine translation evaluations . they also re-evaluate a past evaluation claiming human-parity of MT .
Outcome:	The proposed analysis shows that translationese does not affect machine translation evaluations.

Copied to clipboard

Tasnim Mohiuddin, Philipp Koehn, Vishrav Chaudhary, James Cross, Shruti Bhosale, Shafiq Joty

Overcoming Catastrophic Forgetting During Domain Adaptation of Neural Machine Translation (N19-1)

Challenge:	Neural Machine Translation models are typically trained on heterogeneous data that are concatenated and randomly shuffled.
Approach:	They propose a two-stage curriculum training framework where a NMT model is fine-tuned on subsets of data, selected by deterministic scoring and online scoring.
Outcome:	The proposed framework improves on six language pairs comprising low- and high-resource languages and shows up to +2.2 BLEU improvement and faster convergence.

Copied to clipboard

Brian Thompson, Jeremy Gwinnup, Huda Khayrallah, Kevin Duh, Philipp Koehn

Seeing is Believing: Emotion-Aware Audio-Visual Language Modeling for Expressive Speech Generation (2025.findings-emnlp)

Challenge:	Neural Machine Translation (NMT) performs poorly without large training corpora.
Approach:	They propose a machine learning method that retains the majority of general-domain performance lost in continued training without degrading in-domain.
Outcome:	The proposed method retains the majority of general-domain performance lost in continued training without degrading in-domain performances.

Copied to clipboard

Weiting Tan, Jiachen Lian, Hirofumi Inaguma, Paden Tomasello, Philipp Koehn, Xutai Ma

Multimodality

Challenge:	AVLM integrates full-face visual cues into a pre-trained expressive speech model.
Approach:	They propose an Audio-Visual Language Model (AVLM) for expressive speech generation by integrating full-face visual cues into a pre-trained expressive speech model.
Outcome:	The proposed model incorporates full-face visual cues into a pre-trained expressive speech model.

Evaluating Saliency Methods for Neural Language Models (2021.naacl-main)

Copied to clipboard

Shuoyang Ding, Philipp Koehn

Simulated multiple reference training improves low-resource machine translation (2020.emnlp-main)

Challenge:	a general complaint of neural network models is that their internal decision mechanisms are hard to understand.
Approach:	They evaluate the quality of prediction interpretations from two perspectives: plausibility and faithfulness.
Outcome:	The evaluation of saliency methods on neural language models shows they can be trusted . the methods can be used to interpret the same prediction, but they disagree on interpretations .

Copied to clipboard

Huda Khayrallah, Brian Thompson, Matt Post, Philipp Koehn

Text Generation

Challenge:	Existing valid translations for a given sentence are limited by a single reference translation, causing data sparsity in low-resource settings.
Approach:	They propose a method that approximates the full space of possible translations by sampling a paraphrase of the reference sentence from a MT model and training it to predict the paraphraser’s distribution over possible tokens.
Outcome:	The proposed method improves in low-resource settings and is complementary to back-translation.

Exploiting Sentence Order in Document Alignment (2020.emnlp-main)

Copied to clipboard

Brian Thompson, Philipp Koehn

Multilingual Pixel Representations for Translation and Effective Cross-lingual Transfer (2023.emnlp-main)

Challenge:	a document alignment method that exploits sentence order information is beneficial even when the end goal is sentence-level bitext.
Approach:	They propose a document alignment method that incorporates sentence order information in both candidate generation and candidate re-scoring.
Outcome:	The proposed method outperforms the most recent document alignment method on Sinhala–English documents.

Copied to clipboard

Elizabeth Salesky, Neha Verma, Philipp Koehn, Matt Post

SimulMT to SimulST: Adapting Simultaneous Text Translation to End-to-End Simultaneous Speech Translation (2020.aacl-main)

Challenge:	Recent work shows that pixel representations can be finetuned across scripts without vocabulary extensions, adapters, or transliteration.
Approach:	They propose to use pixel representations to train multilingual machine translation models . they explore parameter sharing within and across scripts to better understand where they lead to positive transfer .
Outcome:	The proposed model improves on two multilingual datasets with different language coverage compared to subword embeddings . the proposed model can be finetuned cross-lingually or to unseen scripts, and is more data-efficient than other alternatives such as vocabulary expansion .

Copied to clipboard

Xutai Ma, Juan Pino, Philipp Koehn

Multimodality

Challenge:	Using end-to-end Simultaneous text translation, we adapt wait-k and monotonic multihead attention to end- to-end simultaneous speech translation.
Approach:	They propose to combine a fixed and flexible pre-decision module with fixed and flexibility policies to adapt simultaneous text translation methods such as wait-k and monotonic multihead attention to end-to-end simultaneous speech translation.
Outcome:	The proposed method can generate translations with maximum quality and minimal latency, targeting video caption translations and real-time language interpreter.

IsoVec: Controlling the Relative Isomorphism of Word Embedding Spaces (2022.emnlp-main)

Copied to clipboard

Kelly Marchisio, Neha Verma, Kevin Duh, Philipp Koehn

Representation Learning

Challenge:	a faulty cross-lingual mapping technique results in non-isomorphic word embedding spaces . we address the root-cause of this problem by increasing the relative isomorphism of word embedsing spaces.
Approach:	They address the root-cause of faulty cross-lingual mapping by incorporating global measures of isomorphism into the skipgram loss function.
Outcome:	The proposed method improves bilingual lexicon induction under domain mismatch and with training algorithm dissimilarities.

Levenshtein Training for Word-level Quality Estimation (2021.emnlp-main)

Copied to clipboard

Shuoyang Ding, Marcin Junczys-Dowmunt, Matt Post, Philipp Koehn

An Analysis of Euclidean vs. Graph-Based Framing for Bilingual Lexicon Induction from Word Embedding Spaces (2021.findings-emnlp)

Challenge:	a novel scheme to perform word-level quality estimation is proposed for word-based quality estimation . authors propose a two-stage transfer learning procedure on augmented and human data . a Levenshtein Transformer can learn to post-edit without explicit supervision.
Approach:	They propose a novel scheme to use a Levenshtein Transformer to perform word-level quality estimation.
Outcome:	The proposed method performs better under data-constrained and unconstrained conditions than existing methods.

Copied to clipboard

Kelly Marchisio, Youngser Park, Ali Saad-Eldin, Anton Alyakin, Kevin Duh, Carey Priebe, Philipp Koehn

Narrowing the Gap between Zero- and Few-shot Machine Translation by Matching Styles (2024.findings-naacl)

Challenge:	Existing work in bilingual lexicon induction views word embeddings as vectors in Euclidean space.
Approach:	They propose to use word embeddings as nodes in a weighted graph to examine a node’s graph neighborhood without assuming a linear transform.
Outcome:	The proposed approaches are compared under different data conditions and show that they complement each other when combined.

Copied to clipboard

Weiting Tan, Haoran Xu, Lingfeng Shen, Shuyue Stella Li, Kenton Murray, Philipp Koehn, Benjamin Van Durme, Yunmo Chen

Speech Vecalign: an Embedding-based Method for Aligning Parallel Speech Documents (2025.emnlp-main)

Challenge:	Recent work shows that large language models can generalize to machine translation using zero-shot examples with in-context learning.
Approach:	They investigate the factors contributing to this gap by matching the writing styles of the target corpus.
Outcome:	The proposed methods can be enhanced without the need for parallel demonstration examples.

Copied to clipboard

Chutong Meng, Philipp Koehn

Multimodality

Challenge:	Speech Vecalign is a parallel speech document alignment method that monotonically aligns speech segment embeddings and does not depend on text transcriptions.
Approach:	They propose a parallel speech document alignment method that monotonically aligns speech segment embeddings and does not depend on text transcriptions.
Outcome:	The proposed method outperforms SpeechMatrix models on 3,000 hours of unlabeled speech documents and produces longer speech-to-speech alignments.

Multilingual Representation Distillation with Contrastive Learning (2023.eacl-main)

Copied to clipboard

Weiting Tan, Kevin Heffernan, Holger Schwenk, Philipp Koehn

Toward the Limitation of Code-Switching in Cross-Lingual Transfer (2022.emnlp-main)

Challenge:	Contextual representations from large pretrained language models encode semantic information from two or more languages.
Approach:	They integrate contrastive learning into multilingual representation distillation and use it for quality estimation of parallel sentences.
Outcome:	The proposed model outperforms existing models with similarity searches and filtering tasks across low-resource languages.

Copied to clipboard

Yukun Feng, Feng Li, Philipp Koehn

Vecalign: Improved Sentence Alignment in Linear Time and Space (D19-1)

Challenge:	Recent studies have shown the success of multilingual pretrained models for cross-lingual knowledge transfer.
Approach:	They propose to make code-switched sentences replace tokens from multiple languages so they are grammatically consistent . they also consider the similarity between context and the switched tokens to ensure that the newly substituted sentences are grammatically consistent - a limitation that could affect inference .
Outcome:	The proposed method outperforms the mBERT and original code-switching method on cross-lingual POS and Named-Entity-Recognition tasks on 30+ languages.

Copied to clipboard

Brian Thompson, Philipp Koehn

XLEnt: Mining a Large Cross-lingual Entity Dataset with Lexical-Semantic-Phonetic Word Alignment (2021.emnlp-main)

Challenge:	Sentence-aligned bitext is used to train nearly all machine translation systems.
Approach:	They propose a bilingual sentence alignment method which is linear in time and space with respect to the number of sentences being aligned.
Outcome:	The proposed method outperforms the existing method by 5 F1 points on a German–French test set and improves downstream MT quality by 1.7 and 1.6 BLEU in Sinhala-English and Nepali-English, respectively.

Copied to clipboard

Ahmed El-Kishky, Adithya Renduchintala, James Cross, Francisco Guzmán, Philipp Koehn

The Language Barrier: Dissecting Safety Challenges of LLMs in Multilingual Contexts (2024.findings-acl)

Challenge:	Existing approaches to generate named entity lexica for lower-resource languages are under performing.
Approach:	They propose a technique to automatically mine cross-lingual named-entity lexica from mined web data.
Outcome:	The proposed technique outperforms baselines at extracting cross-lingual entity pairs and mines 164 million entity pairs from 120 different languages aligned with English.

Copied to clipboard

Lingfeng Shen, Weiting Tan, Sihao Chen, Yunmo Chen, Jingyu Zhang, Haoran Xu, Boyuan Zheng, Philipp Koehn, Daniel Khashabi

Small Data, Big Impact: Leveraging Minimal Data for Effective Machine Translation (2023.acl-long)

Challenge:	Recent studies show that malicious prompt instructions could solicit objectionable content from LLMs.
Approach:	They compare how state-of-the-art LLMs respond to malicious prompts in different languages . they find that LLM's generate unsafe responses more often when a prompt is written in a lower-resource language .
Outcome:	The proposed model can generate unsafe responses more often when a malicious prompt is written in a lower-resource language, and less irrelevant responses when written in lower-source languages.

Copied to clipboard

Jean Maillard, Cynthia Gao, Elahe Kalbassi, Kaushik Ram Sadagopan, Vedanuj Goswami, Philipp Koehn, Angela Fan, Francisco Guzman