Papers by Keyu Li
Can Large Language Models Understand DL-Lite Ontologies? An Empirical Study (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown remarkable proficiency in understanding textual data and revolutionizing the field of natural language processing. |
| Approach: | They empirically analyze LLMs' capability of understanding Description Logic (DL) ontologies covering 6 representative tasks from syntactic and semantic aspects. |
| Outcome: | The proposed model can understand formal syntax and model-theoretic semantics of concepts and roles, but struggle with understanding TBox NI transitivity and handling ontologies with large ABoxes. |
Embedding-based In-Context Prompt Training for Enhancing LLMs as Text Encoders (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have been widely explored for embedding generation. |
| Approach: | They propose an embedding-based in-context prompt training strategy that leverages in-constext learning to generate high-quality embeddables while reducing computational burden. |
| Outcome: | The proposed method surpasses models trained on publicly available retrieval data and achieves state-of-the-art embedding performance on the MTEB benchmark. |
Fraud-R1 : A Multi-Round Benchmark for Assessing the Robustness of LLM Against Augmented Fraud and Phishing Inducements (2025.findings-acl)
Copied to clipboard
Shu Yang, Shenzhe Zhu, Zeyu Wu, Keyu Wang, Junchi Yao, Junchao Wu, Lijie Hu, Mengdi Li, Derek F. Wong, Di Wang
| Challenge: | Existing fraud detection benchmarks focus on single-turn classification tasks, failing to capture dynamic nature of real-world fraud attempts. |
| Approach: | They propose a bilingual benchmark to assess LLMs' ability to resist fraud and phishing attacks across five key fraud categories: Fraudulent Services, Impersonation, Phishing Scams, Fake Job Postings, and Online Relationships. |
| Outcome: | The proposed model improves in role-play settings and in e-commerce and recommendation systems. |
CogniBench: A Legal-inspired Framework and Dataset for Assessing Cognitive Faithfulness of Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks focus on “factual statements” that rephrase source materials, but ignore “cognitive statements” . evaluating and detecting "faithfulness hallucinations" remains challenging . |
| Approach: | They propose a framework to assess faithfulness of cognitive statements and introduce a dataset to scale easily across models. |
| Outcome: | The proposed framework assesses faithfulness of cognitive statements and scales easily across models. |
Benchmarking and Enabling Efficient Chinese Medical Retrieval via Asymmetric Encoders (2026.acl-long)
Copied to clipboard
| Challenge: | Effective medical text retrieval requires high accuracy and low latency. |
| Approach: | They propose a benchmark for medical text retrieval in Chinese using a symmetric architecture . CARE is a lightweight encoder with an LLM-based encoder for offline document encoding . |
| Outcome: | The proposed benchmark surpasses state-of-the-art symmetric models on CMedTEB . it matches high retrieval quality without increasing latency, and it performs well on a single GPU . |
SGCD: Subtask-Guided Causal-Debiasing Framework for Robust Cross-Utterance Sentiment Quadruple Extraction in Dialogues (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new framework for sentiment analysis in dialogues addresses cross-utterance elements and focus biases . SGCD framework employs multi-granularity attention paths to enhance cross-interaction matching . |
| Approach: | a framework is developed to help analyze sentiments in multi-turn dialogues . it leverages subtask-specific features to guide learning of token-level features . |
| Outcome: | The proposed framework outperforms state-of-the-art methods in analyzing conversational data . cross-utterance elements and focus bias are challenges, authors say . |
AgencyBench: Benchmarking the Frontiers of Autonomous Agents in 1M-Token Real-World Contexts (2026.acl-long)
Copied to clipboard
Keyu Li, Junhao Shi, Yang Xiao, Mohan Jiang, Jie Sun, Yunze Wu, Dayuan Fu, Shijie Xia, Xiaojie Cai, Tianze Xu, Weiye Si, Wenjie Li, Dequan Wang, Pengfei Liu
| Challenge: | Existing benchmarks focus on single agentic capability, failing to capture long-horizon real-world scenarios. |
| Approach: | They propose a benchmark that evaluates 6 agentic capabilities across 32 real-world scenarios. |
| Outcome: | Experiments show that closed-source models outperform open-source model (48.4% vs 32.1%) integrating models with advanced scaffolds to form autonomous agents is a paradigm shift. |