Papers by Peng Lan
Hybrid Hierarchical Retrieval for Open-Domain Question Answering (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work shows that dense hierarchical retrieval (DHR) can outperform dense passage retrieval. |
| Approach: | They propose a framework that applies sparse, dense and a combination of them to document and passage retrieval. |
| Outcome: | The proposed framework can outperform dense hierarchical retrieval (DHR) and sparse retrievers (BM25) on open-domain question answering (ODQA) datasets with an average improvement of 4.69% on recall@100 over DHR. |
You Only Read Once (YORO): Learning to Internalize Database Knowledge for Text-to-SQL (2025.naacl-long)
Copied to clipboard
Hideo Kobayashi, Wuwei Lan, Peng Shi, Shuaichen Chang, Jiang Guo, Henghui Zhu, Zhiguo Wang, Patrick Ng
| Challenge: | Existing text-to-SQL systems encode the same schema for every question, resulting in unnecessary high inference cost and missing crucial database knowledge. |
| Approach: | They propose a paradigm that directly internalizes database knowledge into the parametric knowledge of a text-to-SQL model during training and eliminates the need for schema encoding during inference. |
| Outcome: | The proposed paradigm significantly reduces the input token length by 66%-98% and outperforms traditional systems on three benchmarks. |
Unifying Discrete and Continuous Representations for Unsupervised Paraphrase Generation (2023.emnlp-main)
Copied to clipboard
Mingfeng Xue, Dayiheng Liu, Wenqiang Lei, Jie Fu, Jian Lan, Mei Li, Baosong Yang, Jun Xie, Yidan Zhang, Dezhong Peng, Jiancheng Lv
| Challenge: | Existing unsupervised paraphrase generation methods require large-scale, manually annotated paraphrase datasets, which are labor-intensive to build. |
| Approach: | They propose a self-supervised pseudo-data construction method that generates diverse pseudo-paraphrases in distinct surface structures for a given sentence. |
| Outcome: | The proposed method generates diverse pseudo-paraphrases in distinct surface structures for a given sentence. |
RobustQA: Benchmarking the Robustness of Domain Adaptation for Open-Domain Question Answering (2023.findings-acl)
Copied to clipboard
Rujun Han, Peng Qi, Yuhao Zhang, Lan Liu, Juliette Burger, William Yang Wang, Zhiheng Huang, Bing Xiang, Dan Roth
| Challenge: | Existing ODQA datasets consist mainly of Wikipedia corpus, and are insufficient to study models’ generalizability across diverse domains. |
| Approach: | They propose a benchmark to evaluate ODQA's domain robustness using Wikipedia corpus . they annotate QA pairs in retrieval datasets with rigorous quality control . |
| Outcome: | The proposed benchmark improves model performance on annotated QA pairs in retrieval datasets with rigorous quality control. |
PATIMT-Bench: A Multi-Scenario Benchmark for Position-Aware Text Image Machine Translation in Large Vision-Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Current TIMT studies focus on providing translations for all text within an image, neglecting to provide bounding boxes and covering limited scenarios. |
| Approach: | They extend traditional TIMT into position-aware TIMt to support fine-grained translation . they introduce an Adaptive Image OCR Refinement Pipeline to refine results . |
| Outcome: | The proposed model supports fine-grained and layout-preserving translation . the experimental data highlight the scalability and generalizability of the model. |
Learning from Failures: Error Notebook-guided Secure Code Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing security code generation methods rely on abstract security knowledge, resulting in suboptimal security. |
| Approach: | They propose a framework that integrates a Security Error Notebook and a Function Erro Notebook to provide concrete, actionable guidance to LLMs. |
| Outcome: | The proposed framework achieves a substantial leap in SP@1 metric, with GPT-4o-mini performance improving from 60.21% to 66.7% on CodeGuard+. |
RAG-QA Arena: Evaluating Domain Robustness for Long-form Retrieval Augmented Question Answering (2024.emnlp-main)
Copied to clipboard
Rujun Han, Yuhao Zhang, Peng Qi, Yumo Xu, Jenyuan Wang, Lan Liu, William Yang Wang, Bonan Min, Vittorio Castelli
| Challenge: | Existing datasets for question answering based on retrieval augmented generation (RAG-QA) are either constructed using a single source corpus or consist of short extractive answers, which fall short of evaluating large language model (LLM) based RAG-QA systems on cross-domain generalization. |
| Approach: | They propose a dataset that integrates short extractive answers from multiple documents into a single coherent narrative. |
| Outcome: | The proposed dataset integrates short extractive answers from multiple documents into a single coherent narrative, covering 26K queries and large corpora across seven different domains. |
ROSE: Robust Selective Fine-tuning for Pre-trained Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have highlighted the lack of adversarial robustness in pre-trained models. |
| Approach: | They propose a fine-tuning approach that conducts selective updates when adapting pre-trained models to downstream tasks. |
| Outcome: | The proposed approach improves adversarial robustness on downstream tasks . it eliminates spurious updates, leading to flatter and wider optima than the conventional method . |
Multimodal Transformers are Hierarchical Modal-wise Heterogeneous Graphs (2025.acl-long)
Copied to clipboard
| Challenge: | Multimodal Sentiment Analysis (MSA) is a rapidly developing field that integrates multimodal information to recognize sentiments. |
| Approach: | They propose a multimodal fusion model that integrates multimodal information to recognize sentiments using multimodal transformers. |
| Outcome: | The proposed model achieves significantly higher performance than MulTs and the existing model is robust. |
Tokenization Consistency Matters for Generative Models on Extractive NLP Tasks (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained sequence-to-sequence (seq2sequ) models have been widely used to solve extractive tasks, where parts of the input are extracted to form the desired output. |
| Approach: | They propose a simple fix to tokenization inconsistency that damages extractive nature of generative models by causing performance drop and hallucination. |
| Outcome: | The proposed model performs better in both in-domain and out-of-domain datasets with a notable average of +1.7 F1 gain when a BART model is trained on SQuAD and evaluated on 8 QA datasets. |
CiteEval: Principle-Driven Citation Evaluation for Source Attribution (2025.acl-long)
Copied to clipboard
Yumo Xu, Peng Qi, Jifan Chen, Kunlun Liu, Rujun Han, Lan Liu, Bonan Min, Vittorio Castelli, Arshit Gupta, Zhiguo Wang
| Challenge: | Current evaluation frameworks rely on NLI to assess binary or ternary support from cited sources, which is suboptimal for citation evaluation. |
| Approach: | They propose a citation evaluation framework based on fine-grained citation ratings within a broad context and construct a multi-domain benchmark with high-quality human annotations. |
| Outcome: | The proposed framework provides a high-quality human annotation benchmark and a suite of model-based metrics that exhibit strong correlation with human judgments. |