Papers by Benjamin Chang
Correlations between Multilingual Language Model Geometry and Crosslingual Transfer Performance (2024.lrec-main)
Copied to clipboard
| Challenge: | Pre-trained multilingual language models represent multiple languages in a single vector space, a feature hypothesized to enable impressive crosslingual transfer capabilities. |
| Approach: | They propose to use a multilingual representation space that sorts axes based on their language-separability to determine whether geometric distances between languages correlate with crosslingual transfer performance. |
| Outcome: | The proposed measures do not generalize well across models, layers, and tasks. |
PaperMage: A Unified Toolkit for Processing, Representing, and Manipulating Visually-Rich Scientific Documents (2023.emnlp-demo)
Copied to clipboard
Kyle Lo, Zejiang Shen, Benjamin Newman, Joseph Chang, Russell Authur, Erin Bransom, Stefan Candra, Yoganand Chandrasekhar, Regan Huff, Bailey Kuehl, Amanpreet Singh, Chris Wilhelm, Angele Zamarron, Marti A. Hearst, Daniel Weld, Doug Downey, Luca Soldaini
| Challenge: | Existing tools for working with scientific documents are limited and documents are often in difficult-to-use PDF formats. |
| Approach: | They propose an open-source Python toolkit for analyzing and processing visually-rich scientific documents. |
| Outcome: | PaperMage provides turn-key recipes for common scientific document processing use-cases. |
Putting words into the system’s mouth: A targeted attack on neural machine translation using monolingual data poisoning (2021.findings-acl)
Copied to clipboard
Jun Wang, Chang Xu, Francisco Guzmán, Ahmed El-Kishky, Yuqing Tang, Benjamin Rubinstein, Trevor Cohn
| Challenge: | Neural machine translation systems are known to be vulnerable to adversarial test inputs, however, they are also vulnerable to training attacks. |
| Approach: | They propose a poisoning attack in which a malicious adversary inserts a small poisoned sample of monolingual text into a training set of a system trained using back-translation. |
| Outcome: | The proposed attack is based on two methods that can be used to craft poisoned examples. |
FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructions (2025.naacl-long)
Copied to clipboard
Orion Weller, Benjamin Chang, Sean MacAvaney, Kyle Lo, Arman Cohan, Benjamin Van Durme, Dawn Lawrie, Luca Soldaini
| Challenge: | Modern language models (LMs) are capable of following long and complex instructions that enable a large and diverse set of user requests. |
| Approach: | They propose a dataset that contains an instruction evaluation benchmark and a training set to help IR models learn to follow instructions. |
| Outcome: | The proposed model improves after fine-tuning on a training set and rigorous instruction evaluation benchmark. |
Closing the Spatial Execution Gap in Digital Whiteboards via Verifiable Reinforcement Learning (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models suffer from a fundamental Spatial Execution Gap, failing to translate visual semantics into precise, schema-valid coordinate operations in interactive environments. |
| Approach: | They propose a pipeline that leverages Group Relative Policy Optimization to enforce a strict Identify-Reason-Verify protocol and train on execution-verifiable rewards. |
| Outcome: | The proposed pipeline outperforms a state-of-the-art frontier model by 16.75% in operation accuracy. |
The Geometry of Multilingual Language Model Representations (2022.emnlp-main)
Copied to clipboard
| Challenge: | XLM-R models encode language-sensitive information in each language, allowing them to extract features for downstream tasks and cross-lingual transfer learning. |
| Approach: | They evaluate how multilingual language models maintain a shared multilingual representation space while still encoding language-sensitive information in each language. |
| Outcome: | The proposed model can extract features for downstream tasks and cross-lingual transfer learning. |
ArxivDIGESTables: Synthesizing Scientific Literature into Tables using Language Models (2024.emnlp-main)
Copied to clipboard
Benjamin Newman, Yoonjoo Lee, Aakanksha Naik, Pao Siangliulue, Raymond Fok, Juho Kim, Daniel Weld, Joseph Chee Chang, Kyle Lo
| Challenge: | Using language models (LMs) can generate literature review tables by decomposing it into separate schema and value generation steps. |
| Approach: | They propose a framework that leverages language models to perform literature review table generation by decomposing it into separate schema and value generation steps. |
| Outcome: | The proposed framework decomposes the task into two sub-tasks: schema generation and value generation. |
Mitigating Data Poisoning in Text Classification with Differential Privacy (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Data poisoning attacks can plant a backdoor in a model by injecting poisoned examples into training data, causing the model to misclassify test instances which include a specific pattern. |
| Approach: | They propose a generic defence mechanism that makes training robust to poisoning attacks by smoothing the gradient from each training example. |
| Outcome: | The proposed method is highly effective in mitigating, or even eliminating, poisoning attacks on text classification, with only a small cost in predictive accuracy. |
A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translation (2022.naacl-main)
Copied to clipboard
David Adelani, Jesujoba Alabi, Angela Fan, Julia Kreutzer, Xiaoyu Shen, Machel Reid, Dana Ruiter, Dietrich Klakow, Peter Nabende, Ernie Chang, Tajuddeen Gwadabe, Freshia Sackey, Bonaventure F. P. Dossou, Chris Emezue, Colin Leong, Michael Beukman, Shamsuddeen Muhammad, Guyo Jarso, Oreen Yousuf, Andre Niyongabo Rubungo, Gilles Hacheme, Eric Peter Wairagala, Muhammad Umair Nasir, Benjamin Ajibade, Tunde Ajayi, Yvonne Gitau, Jade Abbott, Mohamed Ahmed, Millicent Ochieng, Anuoluwapo Aremu, Perez Ogayo, Jonathan Mukiibi, Fatoumata Ouoba Kabore, Godson Kalipe, Derguene Mbaye, Allahsera Auguste Tapo, Victoire Memdjokam Koagne, Edwin Munkoh-Buabeng, Valencia Wagner, Idris Abdulmumin, Ayodele Awokoya, Happy Buzaaba, Blessing Sibanda, Andiswa Bukula, Sam Manthalu
| Challenge: | Low-resource languages are left out of large-scale pretraining datasets . authors explore how to leverage existing pre-trained models to create low-resourced translation systems for 16 African languages. |
| Approach: | They investigate how large-scale pre-trained models can be used to create low-resource translation systems for 16 African languages. |
| Outcome: | The proposed models can translate between hundreds of languages even though there is little parallel data available for training. |
As Easy as 1, 2, 3: Behavioural Testing of NMT Systems for Numerical Translation (2021.findings-acl)
Copied to clipboard
| Challenge: | Mistranslated numbers can cause financial loss or medical misinformation. |
| Approach: | They propose a method to assess the robustness of neural machine translation systems to numerical text via behavioural testing. |
| Outcome: | The proposed method systematically assesses four fundamental capabilities of neural machine translation systems in translation numbers by virtue of a variety of test cases. |