Papers by Keyu Wang
Can Large Language Models Understand DL-Lite Ontologies? An Empirical Study (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown remarkable proficiency in understanding textual data and revolutionizing the field of natural language processing. |
| Approach: | They empirically analyze LLMs' capability of understanding Description Logic (DL) ontologies covering 6 representative tasks from syntactic and semantic aspects. |
| Outcome: | The proposed model can understand formal syntax and model-theoretic semantics of concepts and roles, but struggle with understanding TBox NI transitivity and handling ontologies with large ABoxes. |
Fraud-R1 : A Multi-Round Benchmark for Assessing the Robustness of LLM Against Augmented Fraud and Phishing Inducements (2025.findings-acl)
Copied to clipboard
Shu Yang, Shenzhe Zhu, Zeyu Wu, Keyu Wang, Junchi Yao, Junchao Wu, Lijie Hu, Mengdi Li, Derek F. Wong, Di Wang
| Challenge: | Existing fraud detection benchmarks focus on single-turn classification tasks, failing to capture dynamic nature of real-world fraud attempts. |
| Approach: | They propose a bilingual benchmark to assess LLMs' ability to resist fraud and phishing attacks across five key fraud categories: Fraudulent Services, Impersonation, Phishing Scams, Fake Job Postings, and Online Relationships. |
| Outcome: | The proposed model improves in role-play settings and in e-commerce and recommendation systems. |
Benchmarking and Enabling Efficient Chinese Medical Retrieval via Asymmetric Encoders (2026.acl-long)
Copied to clipboard
| Challenge: | Effective medical text retrieval requires high accuracy and low latency. |
| Approach: | They propose a benchmark for medical text retrieval in Chinese using a symmetric architecture . CARE is a lightweight encoder with an LLM-based encoder for offline document encoding . |
| Outcome: | The proposed benchmark surpasses state-of-the-art symmetric models on CMedTEB . it matches high retrieval quality without increasing latency, and it performs well on a single GPU . |
Generative Input: Towards Next-Generation Input Methods Paradigm (2024.findings-acl)
Copied to clipboard
| Challenge: | generative models have been used for various NLP tasks but their application in the field of input methods remains under-explored. |
| Approach: | They propose a novel Generative Input paradigm that uses prompts to handle all input scenarios and other intelligent auxiliary input functions, optimizing the model with user feedback. |
| Outcome: | The proposed paradigm achieves state-of-the-art in the Full-mode Key-sequence to Characters task and surpasses GPT-4 in the other input methods. |
AgencyBench: Benchmarking the Frontiers of Autonomous Agents in 1M-Token Real-World Contexts (2026.acl-long)
Copied to clipboard
Keyu Li, Junhao Shi, Yang Xiao, Mohan Jiang, Jie Sun, Yunze Wu, Dayuan Fu, Shijie Xia, Xiaojie Cai, Tianze Xu, Weiye Si, Wenjie Li, Dequan Wang, Pengfei Liu
| Challenge: | Existing benchmarks focus on single agentic capability, failing to capture long-horizon real-world scenarios. |
| Approach: | They propose a benchmark that evaluates 6 agentic capabilities across 32 real-world scenarios. |
| Outcome: | Experiments show that closed-source models outperform open-source model (48.4% vs 32.1%) integrating models with advanced scaffolds to form autonomous agents is a paradigm shift. |