Papers by Andrew Li
Probabilistic Embedding of Knowledge Graphs with Box Lattice Measures (P18-1)
Copied to clipboard
| Challenge: | Structured embeddings based on regions, densities, and orderings have gained popularity for their inductive bias towards the essential asymmetries inherent in problems such as image captioning. |
| Approach: | They propose a box lattice and accompanying probability measure to capture negative correlations over arbitrary concepts. |
| Outcome: | The proposed model can capture anti-correlation and even disjoint concepts while learning from and predicting calibrated uncertainty. |
Grammatical Error Correction for Code-Switched Sentences by Learners of English (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing grammar error correction systems have been trained on monolingual data and not developed for CSW text. |
| Approach: | They propose a method of generating synthetic CSW GEC datasets by translating different spans of text within existing GEC corpora and investigate different methods of selecting these spans based on CSW ratio, switch-point factor and linguistic constraints. |
| Outcome: | The proposed model achieves an average increase of 1.57 F0.5 across 3 CSW test sets (English-Chinese, English-Korean and English-Japanese) without affecting the model’s performance on a monolingual dataset. |
Learning Personalized Alignment for Evaluating Open-ended Text Generation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Traditional evaluation metrics rely heavily on lexical similarity with human-written references, showing poor correlation with human judgments and failing to account for alignment with the diversity of human preferences. |
| Approach: | They propose an interpretable evaluation framework that evaluates alignment with specific human preferences by providing detailed comments and fine-grained scoring. |
| Outcome: | The proposed framework outperforms GPT-4 in Kendall correlation and accuracy with zero-shot reviewers. |
Unifying Inference-Time Planning Language Generation (2026.findings-acl)
Copied to clipboard
Prabhu Prakash Kagitha, Bo Sun, Ishan Desai, Andrew Zhu, Cassie Huang, Manling Li, Ziyang Li, Li Zhang
| Challenge: | Large language models (LLMs) are used to generate a formal representation of a plan in a planning language. |
| Approach: | They propose a unifying organizational framework based on intermediate representations to unify the inference-time LLM-as-formalizer methodology for classical planning. |
| Outcome: | The proposed framework subsumes most existing work and proposes new ones that involve syntactically similar but high-resource intermediate languages. |
CommonLID: Re-evaluating State-of-the-Art Language Identification Performance on Web Data (2026.acl-long)
Copied to clipboard
Pedro Ortiz Suarez, Laurie Burchell, Catherine Arnett, Rafael Mosquera, Sara Hincapié Monsalve, Thom Vaughan, Damian Stewart, Malte Ostendorff, Idris Abdulmumin, Vukosi Marivate, Shamsuddeen Hassan Muhammad, Atnafu Lambebo Tonja, Hend Al-Khalifa, Nadia Ghezaiel Hammouda, Verrah Akinyi Otiende, Tack Hwa Wong, Jakhongir Saydaliev, Melika Nobakhtian, Muhammad Ravi Shulthan Habibi, Chalamalasetti Kranti, Carol Muchemi, Khang Nguyen, Faisal Muhammad Adam, Luis Frentzen Salim, Reem Alqifari, Cynthia Jayne Amol, Joseph Marvin Imperial, Ilker Kesen, Ahmad Mustafid, Pavel Stepachev, Leshem Choshen, David Anugraha, Hamada Nayel, Seid Muhie Yimam, Vallerie Alexandra Putra, My Chiffon Nguyen, Azmine Toushik Wasi, Gouthami Vadithya, Rob Van Der Goot, Lanwenn ar C’horr, Karan Dua, Andrew Yates, Mithil Bangera, Yeshil Bangera, Hitesh Laxmichand Patel, Shu Okabe, Fenal Ashokbhai Ilasariya, Dmitry Gaynullin, Genta Indra Winata, Yiyuan Li, Juan Pablo Martínez, Amit Agarwal, Ikhlasul Akmal Hanif, Raia Abu Ahmad, Esther Adenuga, Filbert Aurelian Tjiaranata, Weerayut Buaphet, Michael Anugraha, Sowmya Vajjala, Benjamin L Rice, Azril Hafizi Amirudin, Jesujoba Oluwadara Alabi, Srikant Panda, Yassine Toughrai, Bruhan Kyomuhendo, Daniel Ruffinelli, null Akshata, Manuel Goulão, Ej Zhou, Ingrid Gabriela Franco Ramirez, Cristina Aggazzotti, Konstantin Dobler, Jun Kevin, Quentin Pagès, Nicholas Andrews, Nuhu Ibrahim, Mattes Ruckdeschel, Amr Keleg, Mike Zhang, Casper Rufaro Muziri, Saron Samuel, Sotaro Takeshita, Kun Kerdthaisong, Luca Foppiano, Rasul Dent, Tommaso Green, Ahmad Mustapha Wali, Kamohelo Makaaka, Vicky Feliren, Inshirah Idris, Hande Celikkanat, Abdulhamid Abubakar, Jean Maillard, Benoît Sagot, Thibault Clérice, Kenton Murray, Sarah K. K. Luger
| Challenge: | Language identification (LID) is a fundamental step in curating multilingual corpora. |
| Approach: | They introduce CommonLID, a community-driven, human-annotated LID benchmark for the web domain, covering 109 languages. |
| Outcome: | The proposed benchmark covers 109 languages and shows that existing evaluations overestimate accuracy for many languages in the web domain. |
Making Large Language Models Efficient Dense Retrievers (2026.acl-long)
Copied to clipboard
| Challenge: | Recent studies have shown that fine-tuning large language models for dense retrieval yields strong performance, but their substantial parameter counts make them computationally inefficient. |
| Approach: | They propose a framework for developing efficient retrievers that performs coarse-to-fine compression through a coarse-grained coarse-tuning strategy. |
| Outcome: | The proposed framework reduces model size and inference cost while preserving performance of full-size models. |
NPRF: A Neural Pseudo Relevance Feedback Framework for Ad-hoc Information Retrieval (D18-1)
Copied to clipboard
| Challenge: | Existing neural IR models do not have a mechanism for treating expansion terms differently from the original query terms, making it difficult to combine them with existing PRF approaches. |
| Approach: | They propose an end-to-end neural PRF framework that can be used with existing neural IR models by embedding different neural models as building blocks. |
| Outcome: | Extensive experiments on two standard test collections confirm the effectiveness of the proposed framework in improving the performance of two state-of-the-art neural IR models. |
ClinicalTrialsHub: Bridging Registries and Literature for Comprehensive Clinical Trial Access (2026.eacl-demo)
Copied to clipboard
Jiwoo Park, Ruoqi Liu, Avani Jagdale, Andrew Srisuwananukorn, Jing Zhao, Lang Li, Ping Zhang, Sachin Kumar
| Challenge: | ClinicalTrialsHub consolidates clinical trial data from ClinicalTrial.gov and augments it by extracting and structuring trial-relevant information from PubMed. |
| Approach: | They propose a search-focused platform that consolidates PubMed data and extracts structured trial information. |
| Outcome: | ClinicalTrialsHub increases access to structured clinical trial data by 83.8% compared to ClinicalTrial.gov alone. |
Eeyore: Realistic Depression Simulation via Expert-in-the-Loop Supervised and Preference Optimization (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have been explored for mental healthcare training and therapy client simulation, but they fail to authentically capture diverse client traits and psychological conditions. |
| Approach: | They propose an 8B model optimized for realistic depression simulation with expert input at every stage. |
| Outcome: | The model outperforms GPT-4o in linguistic authenticity and profile adherence. |
Every Answer Matters: Evaluating Commonsense with Probabilistic Measures (2024.acl-long)
Copied to clipboard
Qi Cheng, Michael Boratko, Pranay Kumar Yelugam, Tim O’Gorman, Nalini Singh, Andrew McCallum, Xiang Li
| Challenge: | Existing commonsense evaluations are often posed as multiple-choice questions, allowing models to exploit systematic biases. |
| Approach: | They propose a generative task that evaluates common sense via multiple open-ended generations and a method that strongly correlates with human judgments. |
| Outcome: | The proposed method outperforms strong language model baselines on a dataset of human and machine common sense. |
Automated Structured Radiology Report Generation (2025.acl-long)
Copied to clipboard
Jean-Benoit Delbrouck, Justin Xu, Johannes Moll, Alois Thomas, Zhihong Chen, Sophie Ostmeier, Asfandyar Azhar, Kelvin Zhenghao Li, Andrew Johnston, Christian Bluethgen, Eduardo Pontes Reis, Mohamed S Muneer, Maya Varma, Curtis Langlotz
| Challenge: | Existing models struggle to produce consistent, clinically meaningful reports and standard evaluation metrics fail to capture the nuances of radiological interpretation. |
| Approach: | They propose to reformulate free-text radiology reports into a standardized format, ensuring clarity, consistency, and structured clinical reporting. |
| Outcome: | The proposed task reformulates free-text radiology reports into a standardized format, ensuring clarity, consistency, and structured clinical reporting. |
Probabilistic Box Embeddings for Uncertain Knowledge Graph Reasoning (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to generalize knowledge bases model triple-level uncertainty . Existing models only model triple level uncertainty, and reasoning results lack global consistency. |
| Approach: | They propose a method to embed knowledge graphs with calibrated probabilistic semantics . they model each entity as a box and relations between two entities as affine transforms based on affinity transforms. |
| Outcome: | Experiments show that the proposed method outperforms baseline methods on confidence prediction and fact ranking. |
Word2Box: Capturing Set-Theoretic Semantics of Words using Box Embeddings (2022.acl-long)
Copied to clipboard
Shib Dasgupta, Michael Boratko, Siddhartha Mishra, Shriya Atmakuri, Dhruvesh Patel, Xiang Li, Andrew McCallum
| Challenge: | Word2Box provides a set-theoretic training objective for learning word representations . word representation is not natural, all senses and contexts, levels of abstraction, variants and modifications which the word may represent are forced to be captured by mat t is nunc. |
| Approach: | They propose a fuzzy-set interpretation of box embeddings and learn box representations of words using a set-theoretic training objective. |
| Outcome: | The proposed model improves word similarity tasks on less common words. |
ChatHF: Collecting Rich Human Feedback from Real-time Conversations (2024.emnlp-demo)
Copied to clipboard
| Challenge: | We present an interactive framework for chatbot evaluation that integrates configurable annotation within a chat interface. |
| Approach: | They propose an interactive framework for chatbot evaluation that integrates configurable annotation within a chat interface. |
| Outcome: | The proposed framework supports fine-grained error detection and human evaluation at the same time. |
ProtoQA: A Question Answering Dataset for Prototypical Common-Sense Reasoning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing question answering datasets for common sense reasoning are lacking for prototypical situations. |
| Approach: | They propose a question answering dataset for training and evaluating common sense reasoning capabilities of artificial intelligence systems in such prototypical situations. |
| Outcome: | The proposed model outperforms existing models on all evaluation metrics with a meaningful gap. |
To the Globe (TTG): Towards Language-Driven Guaranteed Travel Planning (2024.emnlp-demo)
Copied to clipboard
Da Ju, Song Jiang, Andrew Cohen, Aaron Foss, Sasha Mitts, Arman Zharmagambetov, Brandon Amos, Xian Li, Justine Kao, Maryam Fazel-Zarandi, Yuandong Tian
| Challenge: | a new system that takes natural language requests from users generates and trains optimal travel plans . a user can provide instructions and an agent provides optimal solutions . the system takes 5seconds to reply to the user request with guaranteed itineraries . |
| Approach: | They propose a real-time demo system that takes natural language requests from users . it translates requests to symbolic form and produces optimal travel itineraries with LLM . |
| Outcome: | The proposed system produces optimal travel itineraries with mixed integer linear programming solvers. |
Improving the TENOR of Labeling: Re-evaluating Topic Models for Content Analysis (2024.eacl-long)
Copied to clipboard
Zongxia Li, Andrew Mao, Daniel Stephens, Pranav Goel, Emily Walpole, Alden Dima, Juan Fung, Jordan Boyd-Graber
| Challenge: | Existing evaluation metrics such as coherence and coherency are inadequate for neural topic models. |
| Approach: | They conduct the first evaluation of neural, supervised and classical topic models in an interactive task-based setting. |
| Outcome: | The proposed model performs better on cluster evaluation metrics and human evaluations than classical models on real-world tasks. |
DQ-BART: Efficient Sequence-to-Sequence Model via Joint Distillation and Quantization (2022.acl-short)
Copied to clipboard
Zheng Li, Zijian Wang, Ming Tan, Ramesh Nallapati, Parminder Bhatia, Andrew Arnold, Bing Xiang, Dan Roth
| Challenge: | Empirical analyses show that pre-trained sequence-to-sequence models can achieve a 16.5x model footprint compression ratio with little performance drop relative to full-precision counterparts. |
| Approach: | They propose to distill and quantize pre-trained sequence-to-sequence models to reduce memory and latency requirements. |
| Outcome: | Empirical results show that the proposed model achieves 16.5x model footprint compression ratio with little performance drop relative to full-precision counterparts on multiple summarization and QA datasets. |
BrowseComp-Plus: A Fair and Disentangled Evaluation Benchmark for Deep Search Agents (2026.acl-long)
Copied to clipboard
Zijian Chen, Xueguang Ma, Shengyao Zhuang, Ping Nie, Kai Zou, Sahel Sharifymoghaddam, Andrew Liu, Joshua Green, Kshama Patel, Ruoxi Meng, Mingyi Su, Yanxi Li, Haoran Hong, Xinyu Shi, Xuye Liu, Hosna Oyarhoseini, Nandan Thakur, Crystina Zhang, Luyu Gao, Wenhu Chen, Jimmy Lin
| Challenge: | Existing benchmarks for deep search agents rely on blackbox web search APIs . dynamic and opaque web APIs hinder reproducibility and fair comparisons - authors . |
| Approach: | They propose a benchmark that employs a fixed corpus for controlled retrieval for deep search agents. |
| Outcome: | The new benchmark shows that agents that combine large language models with retrieval tools excel at complex, reasoning-intensive queries. |
AirDialogue: An Environment for Goal-Oriented Dialogue Research (D18-1)
Copied to clipboard
| Challenge: | Recent advances in dialogue generation have inspired a number of studies on dialogue systems . however, current datasets are limited in size and the environment for training agents is relatively unsophisticated. |
| Approach: | They propose to use a context-generator to generate travel and flight restrictions to train agents. |
| Outcome: | The proposed model achieves a score of 0.17 while humans can reach 0.91 . the proposed model is based on a large dataset that contains 301,427 goal-oriented conversations . |
Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora (2022.naacl-main)
Copied to clipboard
| Challenge: | Pretrained language models are typically learned over a large, static corpus and fine-tuned for various downstream tasks. |
| Approach: | They propose to continuously update a pretrained language model to adapt to emerging data and to keep track of the model's performance. |
| Outcome: | The proposed model can adapt to new corpora while retaining knowledge in earlier domains. |
Tree-of-Quote Prompting Improves Factuality and Attribution in Multi-Hop and Medical Reasoning (2025.emnlp-main)
Copied to clipboard
Justin Xu, Yiming Li, Zizheng Zhang, Augustine Yui Hei Luk, Mayank Jobanputra, Samarth Oza, Ashley Murray, Meghana Reddy Kasula, Andrew Parker, David W Eyre
| Challenge: | Large language models (LLMs) produce fluent but factually incorrect outputs, a phenomenon commonly referred to as hallucination. |
| Approach: | They propose a Tree-of-Quote framework that decomposes complex questions into subquestions and generates quotes to support each step without retrieval. |
| Outcome: | Experiments on StrategyQA, 2WikiMultiHopQA, MuSiQue, MoreHopQ, and MedQA show that ToQ improves factuality and attribution over baselines. |