Papers by Michael R. Lyu
C2LEVA: Toward Comprehensive and Contamination-Free Language Model Evaluation (2025.findings-acl)
Copied to clipboard
Yanyang Li, Wong Tin Long, Cheung To Hung, Jianqiao Zhao, Duo Zheng, Liu Ka Wai, Michael R. Lyu, Liwei Wang
| Challenge: | Recent advances in large language models (LLMs) have shown significant promise, yet their evaluation raises concerns regarding data contamination due to the lack of access to proprietary training data. |
| Approach: | They propose a bilingual benchmark that offers a holistic evaluation and systematic contamination prevention. |
| Outcome: | The proposed evaluations of 15 open-source and proprietary models show that they are reliable and free of data contamination. |
Topic-Aware Neural Keyphrase Generation for Social Media Language (P19-1)
Copied to clipboard
| Challenge: | Existing methods to extract words from source posts to form keyphrases do not exploit latent topics. |
| Approach: | They propose a sequence-to-sequence-based neural keyphrase generation framework . it allows absent keyphrases to be created, and it allows joint modeling of latent topic representations . |
| Outcome: | The proposed model outperforms extraction and generation models without exploiting latent topics. |
Where Fact Ends and Fairness Begins: Redefining AI Bias Evaluation through Cognitive Biases (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing benchmarks conflate factual correctness and normative fairness . a model may generate responses that are factually accurate but socially unfair . |
| Approach: | They propose a benchmark to examine the boundary between fact and fair . they draw on representativeness bias, attribution bias and ingroup–outgroup bias to explain why models often misalign fact and faireness. |
| Outcome: | The proposed model is based on ten frontier models and is available on github . it is compared with a standard model that generates people of color in Nazi-era uniforms . |
Understanding Secret Leakage Risks in Code LLMs: A Tokenization Perspective (2026.findings-acl)
Copied to clipboard
| Challenge: | Code Large Language Models (CLLMs) are reshaping how software is built, maintained, and evolved. |
| Approach: | They propose to use BPE tokenization to inadvertently leak code secrets . they propose to mitigate the gibberish bias by using a newer tokenizer . |
| Outcome: | The proposed model is based on a novel method that can be used to detect and mitigate gibberish bias in CLLMs. |
Learning to Reason from Feedback at Test-Time (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to utilizing feedback are expensive and lack the time to perform iterative interactions with the environment. |
| Approach: | They propose a novel paradigm that formulates feedback utilization as an optimization problem at test time and a learnable test-time optimizer to effectively exploit feedback. |
| Outcome: | The proposed paradigm improves scalability and performance on two large language models across four reasoning datasets. |
UniDebugger: Hierarchical Multi-Agent Framework for Unified Software Debugging (2025.emnlp-main)
Copied to clipboard
Cheryl Lee, Chunqiu Steven Xia, Longji Yang, Jen-tse Huang, Zhouruixing Zhu, Lingming Zhang, Michael R. Lyu
| Challenge: | Existing LLMs focus on isolated steps and struggle with complex bugs. |
| Approach: | They propose a framework for unified debugging through multi-agent synergy . it mimics the entire cognitive processes of developers with each agent specialized as a particular component of this process . |
| Outcome: | The proposed framework outperforms state-of-the-art methods on repo-level benchmarks. |
Identifying the Achilles’ Heel: An Iterative Method for Uncovering Factual Errors in Large Language Models (2026.findings-acl)
Copied to clipboard
Wenxuan Wang, Yuk-Kit Chan, Zixuan Ling, Shi Juluan, Youliang Yuan, Jen-tse Huang, Yifei Zhang, Wenxiang Jiao, Zhaopeng Tu, Michael R. Lyu
| Challenge: | Current methods for evaluating LLMs’ veracity are limited by the need for extensive human labor, test data contamination, or limited scope, hindering efficient and effective exposure of errors. |
| Approach: | They propose a framework that extracts fact triplets to generate diverse question types using rule-based natural language processing techniques. |
| Outcome: | The proposed framework can trigger factual errors in up to 55% of questions in large LLMs while maintaining coverage of questions. |
Learning to Ask: When LLM Agents Meet Unclear Instruction (2025.emnlp-main)
Copied to clipboard
Wenxuan Wang, Shi Juluan, Zixuan Ling, Yuk-Kit Chan, Chaozheng Wang, Cheryl Lee, Youliang Yuan, Jen-tse Huang, Wenxiang Jiao, Michael R. Lyu
| Challenge: | Despite their impressive capabilities, LLMs struggle with complex computations and delivering accurate, timely information. |
| Approach: | They propose a framework that prompts LLM agents to ask questions when they encounter obstacles due to unclear instructions and an automated evaluation tool called ToolEvaluator. |
| Outcome: | The proposed framework outperforms existing frameworks for tool learning in the Noisy ToolBench. |
Information Aggregation for Multi-Head Attention with Routing-by-Agreement (N19-1)
Copied to clipboard
| Challenge: | Existing studies focus on extracting informative or distinct partial-representations from different subspaces, while few studies have paid attention to the aggregation of the extracted partial-Representations. |
| Approach: | They propose to use a routing-by-agreement algorithm to improve multi-head attention by iteratively updating the proportion of how much a part should be assigned to a whole based on agreement between parts and wholes. |
| Outcome: | The proposed algorithm improves the information aggregation for multi-head attention over the standard linear transformation on linguistic probing and machine translation tasks. |
Interconnected Question Generation with Coreference Alignment and Conversation Flow Modeling (P19-1)
Copied to clipboard
| Challenge: | Extensive experiments show that our system outperforms several baselines and can generate highly conversational questions. |
| Approach: | They propose a neural model that generates interconnected questions in question-answering style conversations. |
| Outcome: | The proposed model outperforms baselines and can generate highly conversational questions. |
Multi-Head Attention with Disagreement Regularization (D18-1)
Copied to clipboard
| Challenge: | Existing methods to encourage diversity among multi-head attention are limited. |
| Approach: | They propose a disagreement regularization term to encourage diversity among attention heads . they validated their approach on EnglishGerman and ChineseEnglish translation tasks . |
| Outcome: | The proposed approach improves translation performance across language pairs on English-German and Chinese-English translation tasks. |
From Laboratory to Real-World Applications: Benchmarking Agentic Code Reasoning at the Repository Level (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for repository-level reasoning are inconsistent . repoReason is a white-box diagnostic benchmark centered on abductive assertion verification . |
| Approach: | They propose a white-box diagnostic benchmark centered on abductive assertion verification. |
| Outcome: | The proposed framework eliminates memorization while maintaining authentic logical depth . it also regenerates ground-truth states and quantifyes reasoning via three orthogonal metrics . |
Topic Memory Networks for Short Text Classification (D18-1)
Copied to clipboard
| Challenge: | Existing classification models for short texts are weak due to data sparsity . |
| Approach: | They propose topic memory networks for short text classification with a novel topic memory mechanism to encode latent topic representations indicative of class labels. |
| Outcome: | The proposed model outperforms state-of-the-art models on short text classification, while generating coherent topics. |
Microblog Hashtag Generation via Encoding Conversation Contexts (N19-1)
Copied to clipboard
| Challenge: | Automated hashtag annotation plays an important role in content understanding for microblog posts. |
| Approach: | They propose to annotate hashtags with a novel sequence generation framework via viewing the hashtag as a short sequence of words. |
| Outcome: | The proposed model outperforms existing models on two large-scale datasets . it can generate rare and even unseen hashtags, which is not possible with existing models . |
Inference-Time Scaling of Verification: Self-Evolving Deep Research Agents via Test-Time Rubric-Guided Verification (2026.findings-acl)
Copied to clipboard
Yuxuan Wan, Tianqing Fang, Zaitang LI, Yintong Huo, Wenxuan Wang, Haitao Mi, Dong Yu, Michael R. Lyu
| Challenge: | Recent advances in Deep Research Agents (DRAs) are transforming automated knowledge discovery and problem-solving. |
| Approach: | They propose an inference-time scaling of verification wherein an agent self-improves at test time by evaluating its generated answers. |
| Outcome: | The proposed model outperforms vanilla agent-as-judge and LLM judge baselines by 12%–48% in meta-evaluation F1 score. |
SlideCoder: Layout-aware RAG-enhanced Hierarchical Slide Generation from Design (2025.emnlp-main)
Copied to clipboard
Wenxin Tang, Jingyu Xiao, Wenxuan Jiang, Xi Xiao, Yuhang Wang, Xuxin Tang, Qing Li, Yuehe Ma, Junliang Liu, Shisong Tang, Michael R. Lyu
| Challenge: | Existing natural language-based LLM generation methods struggle to capture visual and structural nuances of slide designs. |
| Approach: | They propose a layout-aware framework for generating editable slides from reference images . they propose python code that translates NL instructions into Python code to construct each slide . |
| Outcome: | The proposed framework outperforms state-of-the-art models by up to 40.5 points . it also outperformed open-source models with improved reverse-engineered data. |
Asclepius: A Spectrum Evaluation Benchmark for Medical Multi-Modal Large Language Models (2025.acl-long)
Copied to clipboard
Jie Liu, Wenxuan Wang, Su Yihang, Jingyuan Huang, Yudi Zhang, Cheng-Yi Li, Wenting Chen, Xiaohan Xing, Kao-Jung Chang, Linlin Shen, Michael R. Lyu
| Challenge: | Medical Multi-Modal Large Language Models (Med-MLLMs) are a promising new form of artificial general intelligence due to their ability to tackle complex tasks. |
| Approach: | They propose a new benchmark that comprehensively assesses medical multi-modal large language models in terms of distinct medical specialties and different diagnostic capacities. |
| Outcome: | The proposed model covers 15 medical specialties and different diagnostic capacities, and excludes overlap with existing VQA dataset. |
HiGRU: Hierarchical Gated Recurrent Units for Utterance-Level Emotion Recognition (N19-1)
Copied to clipboard
| Challenge: | Using textual features, our proposed HiGRU models achieve at least 8.7%, 7.5%, 6.0% improvement over the state-of-the-art methods on each dataset. |
| Approach: | They propose a hierarchical gated recurrent unit framework to model word-level inputs and an upper-level GRU to capture contexts of utterance-level embeddings. |
| Outcome: | The proposed framework achieves 8.7%, 7.5%, 6.0% improvement over state-of-the-art methods on three datasets. |
Improving Question Generation With to the Point Context (D19-1)
Copied to clipboard
| Challenge: | Existing sequence-to-sequence neural models may not be able to identify answer-relevant context words for question generation. |
| Approach: | They propose to model the unstructured sentence and the structured answer-relevant relation for question generation by combining to the point context and unstructure. |
| Outcome: | Experiments show that the proposed model improves on the unstructured sentence and the structured answer-relevant relation. |