Papers by Yingqi Qu
An Empirical Study of LLM-as-a-Judge for LLM Evaluation: Fine-tuned Judge Model is not a General Substitute for GPT-4 (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have fine-tuned judge models based on open-source LLMs to evaluate the quality of other LLM. |
| Approach: | They propose to use open-source LLMs to evaluate Large Language Models (LLMs) their empirical results show that the models underperform GPT-4 in several dimensions . |
| Outcome: | The proposed models outperform GPT-4 on several dimensions including generalizability, fairness and adaptability. |
DuReader-Retrieval: A Large-scale Chinese Benchmark for Passage Retrieval from Web Search Engine (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing datasets for non-English passage retrieval are lacking in quality and accuracy. |
| Approach: | They present a large-scale Chinese dataset for passage retrieval . they reduce false negatives by manually annotating results pooled from multiple retrievers . |
| Outcome: | The proposed dataset reduces false negatives in development and testing sets and removes similar training queries. |
PAIR: Leveraging Passage-Centric Similarity Relation for Improving Dense Passage Retrieval (2021.findings-acl)
Copied to clipboard
Ruiyang Ren, Shangwen Lv, Yingqi Qu, Jing Liu, Wayne Xin Zhao, QiaoQiao She, Hua Wu, Haifeng Wang, Ji-Rong Wen
| Challenge: | Recent studies only consider query-centric similarity relation when learning the dual-encoder retriever. |
| Approach: | They propose a query-centric and PAssage-centric approach to capture more comprehensive similarity relations for dense passage retrieval. |
| Outcome: | The proposed approach significantly outperforms existing models on both MSMARCO and Natural Questions datasets. |
Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentation (2025.coling-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown impressive prowess in solving a wide range of tasks with world knowledge, but it remains unclear how well they perceive their factual knowledge boundaries. |
| Approach: | They propose to use a retrieval augmentation approach to enhance LLMs' awareness of factual knowledge boundaries to analyze factual and factual information in open-domain question answering (QA) |
| Outcome: | The proposed method improves LLMs’ QA and judgemental capabilities by integrating supporting documents with the questions. |
RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies show that passage retrieval and passage reranking are important for achieving mutual improvement. |
| Approach: | They propose a unified listwise training approach for passage retrieval and passage reranking that incorporates a retrieval procedure and a hybrid data augmentation strategy. |
| Outcome: | The proposed approach improves on both MSMARCO and Natural Questions datasets. |
A Thorough Examination on Zero-shot Dense Retrieval (2023.findings-emnlp)
Copied to clipboard
Ruiyang Ren, Yingqi Qu, Jing Liu, Xin Zhao, Qifei Wu, Yuchen Ding, Hua Wu, Haifeng Wang, Ji-Rong Wen
| Challenge: | Recent advances in dense retrieval (DR) models have been shown to be not as competitive as traditional sparse retrieval models in a zero-shot retrieval setting. |
| Approach: | They propose to examine the zero-shot capability of DR models by analyzing key factors related to source training set and potential bias from target dataset. |
| Outcome: | The proposed model is not as competitive as sparse retrieval models in a zero-shot retrieval setting. |
Self-Evaluation of Large Language Model based on Glass-box Features (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing evaluation methods rely on external evaluators, focusing on training and prompting strategies, but model-aware glass-box features are overlooked. |
| Approach: | They propose to use model-aware glass-box features to evaluate an LLM's output. |
| Outcome: | The proposed model-aware features are reliable quality indicators for self-evaluation on public benchmarks. |
BASES: Large-scale Web Search User Simulation with Large Language Model based Agents (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing research on web search rely on real-user experiments, which can be costly to scale up. |
| Approach: | They propose a user simulation framework with LLM-based agents that can generate unique user profiles at scale. |
| Outcome: | The proposed framework can generate unique user profiles at scale, leading to diverse search behaviors. |
RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering (2021.naacl-main)
Copied to clipboard
Yingqi Qu, Yuchen Ding, Jing Liu, Kai Liu, Ruiyang Ren, Wayne Xin Zhao, Daxiang Dong, Hua Wu, Haifeng Wang
| Challenge: | Open-domain question answering uses dense passage retrieval to find answers . however, it is difficult to effectively train a dual-encoder due to discrepancy between training and inference . |
| Approach: | They propose an optimized training approach to improve dense passage retrieval using RocketQA . they propose cross-batch negatives, denoised hard negatives and data augmentation . |
| Outcome: | The proposed approach outperforms state-of-the-art models on both MSMARCO and Natural Questions. |