Papers by Zhiyong Lu
Benchmarking Retrieval-Augmented Generation for Medicine (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have state-of-the-art performance on a wide range of medical question answering tasks, but they still face challenges with hallucinations and outdated knowledge. |
| Approach: | They propose a benchmark to evaluate medical RAG systems using large-scale experiments with over 1.8 trillion prompt tokens. |
| Outcome: | The proposed benchmark improves accuracy of six different LLMs by up to 18% over chain-of-thought prompting. |
CogBench: Benchmarking Cognitive Alignment of Large Language Models in Educational Question Answering (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) possess strong capabilities in language understanding and generation, as well as remarkable problem-solving abilities. |
| Approach: | They propose a benchmark to assess the cognitive alignment capabilities of large language models in educational QA. |
| Outcome: | The proposed evaluation benchmark assesses the cognitive alignment capabilities of large language models in educational QA. |
Personalized neural language models for real-world query auto completion (N18-3)
Copied to clipboard
| Challenge: | Existing popularity-based methods for query auto completion (QAC) are ineffective in predicting unseen queries. |
| Approach: | They propose to use real-world data to build an end-to-end system that can predict unseen queries by integrating user information. |
| Outcome: | The proposed methods improve on two separate datasets while increasing diversity while scalability. |
MedCite: Can Language Models Generate Verifiable Text for Medicine? (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing LLM-based medical question answering systems lack citation generation and evaluation capabilities, raising concerns about their adoption in practice. |
| Approach: | They propose a framework that facilitates the design and evaluation of LLM citations for medical tasks and a retrieval-citation method that generates high-quality citation. |
| Outcome: | The proposed method achieves superior citation precision and recall improvements compared to strong baseline methods and correlates well with annotation results from professional experts. |