Papers by Ivan Lee
MULTITuDE: Large-Scale Multilingual Machine-Generated Text Detection Benchmark (2023.emnlp-main)
Copied to clipboard
Dominik Macko, Robert Moro, Adaku Uchendu, Jason Lucas, Michiharu Yamashita, Matúš Pikuliak, Ivan Srba, Thai Le, Dongwon Lee, Jakub Simko, Maria Bielikova
| Challenge: | MULTITuDE benchmarks lack authentic and machine-generated text in languages other than English . defining characteristic of new generation of LLMs is increased quality of text . |
| Approach: | They propose a benchmarking dataset for multilingual machine-generated text detection that compares detectors with authentic and machine-generated texts in 11 languages. |
| Outcome: | The proposed dataset compares detectors with zero-shot and fine-tuned detectors in 11 languages. |
Align-SLM: Textless Spoken Language Models with Reinforcement Learning from AI Feedback (2025.acl-long)
Copied to clipboard
Guan-Ting Lin, Prashanth Gurunath Shivakumar, Aditya Gourav, Yile Gu, Ankur Gandhe, Hung-yi Lee, Ivan Bulyko
| Challenge: | Textless Spoken Language Models lag behind text-based Large Language Model (LLM) in semantic coherence and relevance. |
| Approach: | They propose a framework that leverages preference optimization inspired by Reinforcement Learning with Human Feedback to enhance the semantic understanding of SLMs. |
| Outcome: | The proposed framework achieves state-of-the-art performance of SLMs for most benchmarks . it leverages preference optimization inspired by Reinforcement Learning with Human Feedback . |
Masked Measurement Prediction: Learning to Jointly Predict Quantities and Units from Textual Context (2022.findings-naacl)
Copied to clipboard
| Challenge: | Current benchmarks do not evaluate numeracy of pretraining language models on measurements. |
| Approach: | They propose a new task where a model learns to reconstruct a number with its associated unit given masked text. |
| Outcome: | The proposed model significantly underperforms pre-trained model with baselines and ablations. |
A Ship of Theseus: Curious Cases of Paraphrasing in LLM-Generated Texts (2024.acl-long)
Copied to clipboard
Nafis Irtiza Tripto, Saranya Venkatraman, Dominik Macko, Robert Moro, Ivan Srba, Adaku Uchendu, Thai Le, Dongwon Lee
| Challenge: | Using a computational approach, we discover that diminishing performance in text classification models is closely associated with the extent of deviation from the original author’s style. |
| Approach: | They propose to use large language models to determine whether a text retains original authorship when it undergoes numerous paraphrasing iterations. |
| Outcome: | The results suggest that authorship should be task-dependent . |
Optimizing Hidden Markov Language Models: An Empirical Study of Reparameterization and Initialization Techniques (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent work on scaling-up HMMs to perform competitively as language models has indicated that this challenge only increases with larger hidden state sizes. |
| Approach: | They propose two strategies that use neural reparameterization and neural initialization to enhance HMM optimization. |
| Outcome: | The proposed techniques work well for scaled HMM language modeling, and linear reparameterizations can be as effective as non-linear ones, and the strategies are complementary. |
HeLo: Learning-Free Lookahead Decoding for Conversation Infilling (2022.findings-emnlp)
Copied to clipboard
| Challenge: | We propose a scalable decoding strategy for conversation infilling . large pretrained language models are effective solutions to many popular natural language generation tasks such as machine translation and conversational dialogue. |
| Approach: | They propose a heuristic guided lookahead decoding strategy for conversation infilling which leverages a greedy lookalike phase before committing to any token. |
| Outcome: | The proposed strategy outperforms baselines when evaluated with automatic and human evaluation metrics, which, we argue, are appropriate for the task. |
SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languages (2024.emnlp-main)
Copied to clipboard
Holy Lovenia, Rahmad Mahendra, Salsabil Akbar, Lester James Miranda, Jennifer Santoso, Elyanah Aco, Akhdan Fadhilah, Jonibek Mansurov, Joseph Marvin Imperial, Onno Kampman, Joel Moniz, Muhammad Habibi, Frederikus Hudi, Jann Montalan, Ryan Hadiwijaya, Joanito Lopo, William Nixon, Börje Karlsson, James Jaya, Ryandito Diandaru, Yuze Gao, Patrick Irawan, Bin Wang, Jan Christian Blaise Cruz, Chenxi Whitehouse, Ivan Parmonangan, Maria Khelli, Wenyu Zhang, Lucky Susanto, Reynard Ryanda, Sonny Hermawan, Dan Velasco, Muhammad Kautsar, Willy Hendria, Yasmin Moslem, Noah Flynn, Muhammad Adilazuarda, Haochen Li, Johanes Lee, R. Damanhuri, Shuo Sun, Muhammad Qorib, Amirbek Djanibekov, Wei Qi Leong, Quyet V. Do, Niklas Muennighoff, Tanrada Pansuwan, Ilham Firdausi Putra, Yan Xu, Tai Chia, Ayu Purwarianti, Sebastian Ruder, William Tjhi, Peerat Limkonchotiwat, Alham Aji, Sedrick Keh, Genta Winata, Ruochen Zhang, Fajri Koto, Zheng Xin Yong, Samuel Cahyawijaya
| Challenge: | Southeast Asia (SEA) is home to over 1,300 indigenous languages and 671 million people . prevailing AI models suffer from a significant lack of representation of texts, images, and audio datasets from SEA . |
| Approach: | They propose to provide a resource center that provides standardized corpora in nearly 1,000 SEA languages across three modalities. |
| Outcome: | a new benchmark assesses the quality of AI models on 36 SEA languages across 13 tasks . the results highlight the importance of SEA as a culturally diverse region . |
Authorship Obfuscation in Multilingual Machine-Generated Text Detection (2024.findings-emnlp)
Copied to clipboard
Dominik Macko, Robert Moro, Adaku Uchendu, Ivan Srba, Jason Lucas, Michiharu Yamashita, Nafis Irtiza Tripto, Dongwon Lee, Jakub Simko, Maria Bielikova
| Challenge: | Recent advances in Language Modeling have birthed Large Language Models (LLMs), which exhibit significant improvements, including the ability to generate texts easily misconstrued as humanwritten. |
| Approach: | They compare authorship obfuscation methods against machine-generated text (MGT) in 11 languages and analyze their performance against 37 well-known AO methods. |
| Outcome: | The proposed methods can cause evasion of detection in all languages, with homoglyph attacks particularly successful. |