Papers by Lingyao Li
NPHardEval: Dynamic Benchmark on Reasoning Ability of Large Language Models via Complexity Classes (2024.acl-long)
Copied to clipboard
| Challenge: | Complex reasoning ability is one of the most important features of Large Language Models. |
| Approach: | They propose a new benchmark that measures the reasoning ability of Large Language Models . it contains 900 algorithmic questions belonging to the NP-Hard complexity class . |
| Outcome: | The proposed benchmark contains 900 questions belonging to the NP-Hard complexity class and is updated on a monthly basis. |
Disentangling Logic: The Role of Context in Large Language Model Reasoning Capabilities (2025.findings-acl)
Copied to clipboard
Wenyue Hua, Kaijie Zhu, Lingyao Li, Lizhou Fan, Mingyu Jin, Shuhang Lin, Haochen Xue, Zelong Li, Jindong Wang, Yongfeng Zhang
| Challenge: | Using large language models, large language model models can be used to evaluate reasoning abilities in context-rich scenarios. |
| Approach: | They construct datasets for both propositional logic and abductive logic reasoning with four difficulty levels across 12 distinct domains based on Wikipedia categorization and those with purely abstract variables. |
| Outcome: | The proposed model can be used to benchmark LLMs in real-world scenarios, but not in context-rich scenarios. |
Data Interpreter: An LLM Agent for Data Science (2025.findings-acl)
Copied to clipboard
Sirui Hong, Yizhang Lin, Bang Liu, Bangbang Liu, Binhao Wu, Ceyao Zhang, Danyang Li, Jiaqi Chen, Jiayi Zhang, Jinlin Wang, Li Zhang, Lingyao Zhang, Min Yang, Mingchen Zhuge, Taicheng Guo, Tuo Zhou, Wei Tao, Robert Tang, Xiangtao Lu, Xiawu Zheng, Xinbing Liang, Yaying Fei, Yuheng Cheng, Yongxin Ni, Zhibin Gou, Zongze Xu, Yuyu Luo, Chenglin Wu
| Challenge: | Large Language Models (LLMs) excel in various domains but face challenges when applied to data science workflows due to their complex, multi-stage nature. |
| Approach: | They propose a hierarchical graph-based agent that represents complexity and a progressive strategy for step-by-step verification, refinement, and consistent context management. |
| Outcome: | The proposed agent surpasses state-of-the-art baselines on the MATH dataset and performs better on InfiAgent-DABench. |
Multi-Agent Comedy Club: Investigating Community Discussion Effects on LLM Humor Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on the use of multi-turn interaction and feedback for LLM writing focus on prompts and localized feedback. |
| Approach: | They build a controlled multi-agent sandbox that instantiates a small standup comedy community and allows it to manipu-late whether public reception is generated, logged, and fed back into later rounds. |
| Outcome: | The proposed model improves craft/clarity and social response with occasional increases in aggressive humor. |
BattleAgent: Multi-modal Dynamic Emulation on Historical Battles to Complement Historical Analysis (2024.emnlp-demo)
Copied to clipboard
Shuhang Lin, Wenyue Hua, Lingyao Li, Che-Jui Chang, Lizhou Fan, Jianchao Ji, Hang Hua, Mingyu Jin, Jiebo Luo, Yongfeng Zhang
| Challenge: | Recent advances in large language models have demonstrated impressive reasoning capabilities, indicating their potential to serve as the foundation for agents. |
| Approach: | They propose a detailed emulation system that combines large vision-language model and multi-agent system to emulate dynamic interactions between multiple agents over a period of time. |
| Outcome: | The proposed system combines large vision-language model and multi-agent system to emulate dynamic interactions between agents and their environments over a period of time. |
ADO: Automatic Data Optimization for Inputs in LLM Prompts (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent research has focused on refining instruction components and augmenting input data with in-context examples, but this study explores the potential benefits of optimizing the input data itself. |
| Approach: | They propose a content engineering and structural reformulation strategy to optimize input data within prompts to improve performance of Large Language Models. |
| Outcome: | The proposed approach improves performance of Large Language Models (LLMs) in various tasks, offering a promising avenue for future research in prompt engineering. |
LLMs as World Models: Data-Driven and Human-Centered Pre-Event Simulation for Disaster Impact Assessment (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) show promise in simulating complex scenarios. |
| Approach: | They examine multiple LLMs to proactively estimate perceived earthquake impacts using multimodal datasets and multimodal imagery. |
| Outcome: | The framework generates Modified Mercalli Intensity (MMI) predictions at zip code and county scales using multimodal datasets. |
Invisible Prompts, Visible Threats: Malicious Font Injection in External Resources for Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly equipped with capabilities of real-time web search and integrated with protocols like the Model Context Protocol (MCP). |
| Approach: | They investigate the vulnerability of Large Language Models to hidden adversarial prompts . they evaluate two critical attack scenarios: malicious content relay and sensitive data leakage . |
| Outcome: | The proposed extension could introduce new security vulnerabilities. |