Papers by Adam Lee
LLMs as ASP Programmers: Self-Correction Enables Task-Agnostic Nonmonotonic Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent large language models struggle with high computational costs and logical inconsistencies . a framework that translates natural language into Answer Set Programming (ASP) is developed . |
| Approach: | They propose a framework that translates natural language into Answer Set Programming (ASP) stable model semantics allow LLMs to express default rules and exceptions, they show . |
| Outcome: | The proposed framework outperforms existing methods on nonmonotonic reasoning tasks without any per-task engineering and applies uniformly across reasoning tasks. |
fairseq Sˆ2: A Scalable and Integrable Speech Synthesis Toolkit (2021.emnlp-demo)
Copied to clipboard
| Challenge: | Speech synthesis is the task of generating speech waveforms with desired characteristics, including but not limited to textual content, speaker identity, and speaking styles. |
| Approach: | They propose a fairseq extension for speech synthesis that implements autoregressive and non-AR text-to-speech models and their multi-speaker variants. |
| Outcome: | The proposed extension can train autoregressive and non-AR models and their multi-speaker variants with less curated data and has automatic metrics to facilitate faster iteration and analysis. |
Coupling Large Language Models with Logic Programming for Robust and General Reasoning from Text (2023.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are robust and general, but their reasoning ability is not at a level to compete with the best models trained for specific natural language reasoning problems. |
| Approach: | They propose to use large language models as a few-shot semantic parser to convert natural language sentences into a logical form that serves as input for answer set programs. |
| Outcome: | The proposed model can handle multiple question-answering tasks without requiring retraining for each new task. |
Text-Free Prosody-Aware Generative Spoken Language Modeling (2022.acl-long)
Copied to clipboard
Eugene Kharitonov, Ann Lee, Adam Polyak, Yossi Adi, Jade Copet, Kushal Lakhotia, Tu Anh Nguyen, Morgane Riviere, Abdelrahman Mohamed, Emmanuel Dupoux, Wei-Ning Hsu
| Challenge: | Experimental results show that generative spoken language models (LMs) are natural unsupervised multitask learners. |
| Approach: | They propose a prosody-aware generative spoken language model that uses discovered units to generate natural, meaningful, and coherent speech. |
| Outcome: | The proposed model can generate natural, meaningful, and coherent speech given a spoken prompt. |
CapWAP: Image Captioning with a Purpose (2020.emnlp-main)
Copied to clipboard
| Challenge: | a traditional image captioning task uses generic reference captions to provide textual information about images. |
| Approach: | They propose a task that uses question-answer pairs to provide visual information instead of generic reference captions. |
| Outcome: | The proposed captioning with a purpose task can be tailored to meet user needs . question-answer pairs are used as a source of supervision for learning visual information needs a new task is proposed . |
Direct Speech-to-Speech Translation With Discrete Units (2022.acl-long)
Copied to clipboard
Ann Lee, Peng-Jen Chen, Changhan Wang, Jiatao Gu, Sravya Popuri, Xutai Ma, Adam Polyak, Yossi Adi, Qing He, Yun Tang, Juan Pino, Wei-Ning Hsu
| Challenge: | Existing direct speech-to-speech translation models rely on text generation as an intermediate step. |
| Approach: | They propose a direct speech-to-speech translation model that translates speech from one language to another without relying on intermediate text generation. |
| Outcome: | The proposed model produces 6.7 BLEUs in the Fisher Spanish-English dataset when trained without any text transcripts and with text supervision. |
WorldCuisines: A Massive-Scale Benchmark for Multilingual and Multicultural Visual Question Answering on Global Cuisines (2025.naacl-long)
Copied to clipboard
Genta Indra Winata, Frederikus Hudi, Patrick Amadeus Irawan, David Anugraha, Rifki Afina Putri, Wang Yutong, Adam Nohejl, Ubaidillah Ariq Prathama, Nedjma Ousidhoum, Afifa Amriani, Anar Rzayev, Anirban Das, Ashmari Pramodya, Aulia Adila, Bryan Wilie, Candy Olivia Mawalim, Cheng Ching Lam, Daud Abolade, Emmanuele Chersoni, Enrico Santus, Fariz Ikhwantri, Garry Kuwanto, Hanyang Zhao, Haryo Akbarianto Wibowo, Holy Lovenia, Jan Christian Blaise Cruz, Jan Wira Gotama Putra, Junho Myung, Lucky Susanto, Maria Angelica Riera Machin, Marina Zhukova, Michael Anugraha, Muhammad Farid Adilazuarda, Natasha Christabelle Santosa, Peerat Limkonchotiwat, Raj Dabre, Rio Alexander Audino, Samuel Cahyawijaya, Shi-Xiong Zhang, Stephanie Yulia Salim, Yi Zhou, Yinxuan Gui, David Ifeoluwa Adelani, En-Shiun Annie Lee, Shogo Okada, Ayu Purwarianti, Alham Fikri Aji, Taro Watanabe, Derry Tanti Wijaya, Alice Oh, Chong-Wah Ngo
| Challenge: | Vision Language Models struggle with cultural-specific knowledge, especially in languages other than English and in underrepresented cultural contexts. |
| Approach: | They propose a visual question answering (VQA) dataset with text-image pairs across 30 languages and dialects and a training dataset. |
| Outcome: | The proposed model performs better with correct location context, but struggles with adversarial contexts and predicting specific regional cuisines and languages. |
How Does DPO Reduce Toxicity? A Mechanistic Neuron-Level Analysis (2025.emnlp-main)
Copied to clipboard
| Challenge: | Direct Preference Optimization (DPO) is a popular choice of safety fine-tuning algorithms, but prior explanations of its effects only account for dampened toxic neurons in the MLP layers. |
| Approach: | They analysed four language models and found that toxic neurons only account for 2.5% to 24% of DPO’s effects across models. |
| Outcome: | The proposed method outperforms DPO in reducing toxicity while preserving perplexity, without requiring any weight updates. |
A Pretrainer’s Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity (2024.naacl-long)
Copied to clipboard
Shayne Longpre, Gregory Yauney, Emily Reif, Katherine Lee, Adam Roberts, Barret Zoph, Denny Zhou, Jason Wei, Kevin Robinson, David Mimno, Daphne Ippolito
| Challenge: | a large number of pretraining data design practices are under-documented, authors say . authors: strong performance of modern language models depends on selfsupervised pretraining . |
| Approach: | They propose to pretrain models on data curated at different collection times . they find temporal shift between evaluation data and pretraining data leads to performance degradation . |
| Outcome: | The results validate, quantify, and expose many undocumented intuitions about text pretraining . authors say this practice has outperformed other models in the field . |
Towards Efficient Visual-Language Alignment of the Q-Former for Visual Reasoning Tasks (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained large language models can be fine-tuned with instruction tuning to align the model responses with human intentions. |
| Approach: | They investigate the effectiveness of parameter efficient fine-tuning (PEFT) of the Q-Former with visual reasoning benchmarks ScienceQA and IconQA. |
| Outcome: | The proposed model achieves comparable performance to full fine-tuning using under 2% of the trainable parameters. |
MERLIN: Multimodal Embedding Refinement via LLM-based Iterative Navigation for Text-Video Retrieval-Rerank Pipeline (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Recent advances in text-video retrieval neglect the crucial user perspective, leading to discrepancies between user queries and content retrieved. |
| Approach: | They propose a novel, training-free pipeline that leverages Large Language Models for iterative feedback learning. |
| Outcome: | Experimental results show that MERLIN significantly outperforms existing systems in video retrieval. |