Papers by Mike Zhou
CommonLID: Re-evaluating State-of-the-Art Language Identification Performance on Web Data (2026.acl-long)
Copied to clipboard
Pedro Ortiz Suarez, Laurie Burchell, Catherine Arnett, Rafael Mosquera, Sara Hincapié Monsalve, Thom Vaughan, Damian Stewart, Malte Ostendorff, Idris Abdulmumin, Vukosi Marivate, Shamsuddeen Hassan Muhammad, Atnafu Lambebo Tonja, Hend Al-Khalifa, Nadia Ghezaiel Hammouda, Verrah Akinyi Otiende, Tack Hwa Wong, Jakhongir Saydaliev, Melika Nobakhtian, Muhammad Ravi Shulthan Habibi, Chalamalasetti Kranti, Carol Muchemi, Khang Nguyen, Faisal Muhammad Adam, Luis Frentzen Salim, Reem Alqifari, Cynthia Jayne Amol, Joseph Marvin Imperial, Ilker Kesen, Ahmad Mustafid, Pavel Stepachev, Leshem Choshen, David Anugraha, Hamada Nayel, Seid Muhie Yimam, Vallerie Alexandra Putra, My Chiffon Nguyen, Azmine Toushik Wasi, Gouthami Vadithya, Rob Van Der Goot, Lanwenn ar C’horr, Karan Dua, Andrew Yates, Mithil Bangera, Yeshil Bangera, Hitesh Laxmichand Patel, Shu Okabe, Fenal Ashokbhai Ilasariya, Dmitry Gaynullin, Genta Indra Winata, Yiyuan Li, Juan Pablo Martínez, Amit Agarwal, Ikhlasul Akmal Hanif, Raia Abu Ahmad, Esther Adenuga, Filbert Aurelian Tjiaranata, Weerayut Buaphet, Michael Anugraha, Sowmya Vajjala, Benjamin L Rice, Azril Hafizi Amirudin, Jesujoba Oluwadara Alabi, Srikant Panda, Yassine Toughrai, Bruhan Kyomuhendo, Daniel Ruffinelli, null Akshata, Manuel Goulão, Ej Zhou, Ingrid Gabriela Franco Ramirez, Cristina Aggazzotti, Konstantin Dobler, Jun Kevin, Quentin Pagès, Nicholas Andrews, Nuhu Ibrahim, Mattes Ruckdeschel, Amr Keleg, Mike Zhang, Casper Rufaro Muziri, Saron Samuel, Sotaro Takeshita, Kun Kerdthaisong, Luca Foppiano, Rasul Dent, Tommaso Green, Ahmad Mustapha Wali, Kamohelo Makaaka, Vicky Feliren, Inshirah Idris, Hande Celikkanat, Abdulhamid Abubakar, Jean Maillard, Benoît Sagot, Thibault Clérice, Kenton Murray, Sarah K. K. Luger
| Challenge: | Language identification (LID) is a fundamental step in curating multilingual corpora. |
| Approach: | They introduce CommonLID, a community-driven, human-annotated LID benchmark for the web domain, covering 109 languages. |
| Outcome: | The proposed benchmark covers 109 languages and shows that existing evaluations overestimate accuracy for many languages in the web domain. |
Can Transformers Learn n-gram Language Models? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing work has tested transformers' ability to represent formal languages, but language models are not classifiers of strings but rather distributions over them. |
| Approach: | They relate transformers' ability to learn random n-gram language models to ngram language model (LM) they find add- smoothing outperforms transformers on the former, while transformers perform better on the latter . |
| Outcome: | The proposed models outperform classical methods designed to learn n-gram LMs, while transformers perform better on the latter. |
AutoReply: Detecting Nonsense in Dialogue with Discriminative Replies (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for dialogue detection make many errors in their own messages . a dataset of long dialogues richly grounded in the game state contains many errors . |
| Approach: | They propose to use an annotated dialogue dataset to generate automatic responses for dialogue models. |
| Outcome: | The proposed model outperforms handcrafted replies and performs on par with supervised learning approaches. |
Byte Latent Transformer: Patches Scale Better Than Tokens (2025.acl-long)
Copied to clipboard
Artidoro Pagnoni, Ramakanth Pasunuru, Pedro Rodriguez, John Nguyen, Benjamin Muller, Margaret Li, Chunting Zhou, Lili Yu, Jason E Weston, Luke Zettlemoyer, Gargi Ghosh, Mike Lewis, Ari Holtzman, Srini Iyer
| Challenge: | Existing large language models (LLMs) are trained on bytes, except for tokenization, which groups bytes into a static set of tokens. |
| Approach: | They propose a new byte-level LLM architecture that encodes bytes into dynamically sized patches, which serve as the primary units of computation. |
| Outcome: | The proposed architecture matches tokenization-based models with improvements in inference efficiency and robustness. |
In-context Examples Selection for Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Large-scale generative models can perform a wide range of NLP tasks using in-context learning. |
| Approach: | They aim to understand the properties of good in-context examples for machine translation in both in-domain and out-of-domain settings. |
| Outcome: | The proposed model outperforms a strong kNN-MT baseline in 2 out of 4 out-of-domain datasets. |
Program-of-Thought Reveals LLM Abstraction Ceilings (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large language models exhibit reasoning ability when supervised with chain-of-thought (CoT) traces. |
| Approach: | They evaluate large language models with CoT traces and fine-tune them with Program-of-Thought supervision. |
| Outcome: | The proposed model performance degrades sharply under numeric perturbations under isomorphic variants. |