Papers by Nan Zhao

Large Language Models Can Learn Representation in Natural Language (2024.findings-acl)

Challenge:	Existing studies focus on adapting either the retriever or the reader, but this approach is more focused on adaptation of the query itself.
Approach:	They propose a new framework for retrieval-augmented Large Language Models . they propose rewrite-retrieve-read instead of retrieve-then-read .
Outcome:	The proposed framework improves performance on downstream tasks, open-domain QA and multiple-choice QA.

Copied to clipboard

Yiduo Guo, Yaobo Liang, Dongyan Zhao, Nan Duan

OpenRT: An Open-source Framework for Reasoning Over Tabular Data (2023.acl-demo)

Challenge:	Large Language Models (LLMs) are unable to complete complex tasks involving multiple entities, such as tool APIs.
Approach:	They propose a method which uses natural language representations to refine entity descriptions for improved retrieval and LLM utilization.
Outcome:	The proposed method improves GPT-4's performance on classification tasks and API call tasks.

Copied to clipboard

Yilun Zhao, Boyu Mi, Zhenting Qi, Linyong Nan, Minghao Guo, Arman Cohan, Dragomir Radev

JDDC 2.1: A Multimodal Chinese Dialogue Dataset with Joint Tasks of Query Rewriting, Response Generation, Discourse Parsing, and Summarization (2022.emnlp-main)

Challenge:	Existing table pre-training methods are benchmarked on a limited number of datasets with varying configurations, resulting in a lack of unified, standardized, fair, and comprehensive comparison between methods.
Approach:	They propose to use OpenRT to reproduce existing table pre-training models and develop new models quickly.
Outcome:	The proposed framework reproduces existing table pre-training models and compares them against four question answering, one fact checking, and one faithful text generation datasets.

Copied to clipboard

Nan Zhao, Haoran Li, Youzheng Wu, Xiaodong He

Mitigating the Alignment Tax of RLHF (2024.emnlp-main)

Challenge:	e-commerce users express their needs using text, images, or videos . but detailed information provided by images is limited, and customer service systems cannot understand the intent of users without the input text.
Approach:	They construct a large-scale multimodal multi-turn dialogue dataset from a mainstream Chinese E-commerce platform . the dataset contains about 246K dialogue sessions, 3M utterances, and 507K images .
Outcome:	The proposed dataset contains 246K dialogue sessions, 3M utterances, 507K images . it also includes product knowledge bases and image category annotations .

Copied to clipboard

Yong Lin, Hangyu Lin, Wei Xiong, Shizhe Diao, Jianmeng Liu, Jipeng Zhang, Rui Pan, Haoxiang Wang, Wenbin Hu, Hanning Zhang, Hanze Dong, Renjie Pi, Han Zhao, Nan Jiang, Heng Ji, Yuan Yao, Tong Zhang

OpenT2T: An Open-Source Toolkit for Table-to-Text Generation (2024.emnlp-demo)

Challenge:	Large Language Models (LLMs) acquire a wide range of abilities during pre-training, but aligning LLMs under Reinforcement Learning with Human Feedback (RLHF) can lead to forgetting pretrained abilities, which is also known as the alignment tax.
Approach:	They propose to use a model averaging technique to find the most powerful alignment-forging Pareto front among RLHF algorithms.
Outcome:	The proposed method achieves the strongest alignment-forging Pareto front among competing methods.

Copied to clipboard

Haowei Zhang, Shengyun Si, Yilun Zhao, Lujing Xie, Zhijian Xu, Lyuhao Chen, Linyong Nan, Pengcheng Wang, Xiangru Tang, Arman Cohan

MentalGLM Series: Explainable Large Language Models for Mental Health Analysis on Chinese Social Media (2025.emnlp-main)

Challenge:	Existing methods for table-to-text generation are limited and benchmarked on a limited number of datasets.
Approach:	They propose to use open-source tools to reproduce existing large language models for performance comparison and expedite the development of new models.
Outcome:	The proposed toolkit compares existing large language models on 9 table-to-text generation datasets and maintains a leaderboard to provide insights for future work.

Copied to clipboard

Wei Zhai, Nan Bai, Qing Zhao, Jianqiang Li, Fan Wang, Hongzhi Qi, Meng Jiang, Xiaoqin Wang, Bing Xiang Yang, Guanghui Fu

Responsible & Trustworthy NLP

Challenge:	Social media is a key platform for emotional expression, yet deep learning lacks flexibility and interpretability.
Approach:	They propose to use Chinese social media to train interpretable mental health instruction datasets to test models' ability to explain their decisions.
Outcome:	The proposed models outperform deep learning and LLMs on three mental health downstream tasks and demonstrate their potential for clinical applications.

Learning to Plan by Updating Natural Language (2024.findings-emnlp)

Copied to clipboard

Yiduo Guo, Yaobo Liang, Chenfei Wu, Wenshan Wu, Dongyan Zhao, Nan Duan

Analyzing and Reducing the Performance Gap in Cross-Lingual Transfer with Fine-tuning Slow and Fast (2023.acl-long)

Challenge:	Large Language Models (LLMs) have shown remarkable performance in basic natural language tasks.
Approach:	They propose a method that iteratively updates the task plan with new steps and behavioral instructions to guide LLMs to generate the correct solutions step by step.
Outcome:	The proposed method improves performance on five reasoning type tasks and can be directly applied to other LLMs.

Copied to clipboard

Yiduo Guo, Yaobo Liang, Dongyan Zhao, Bing Liu, Nan Duan

ReasTAP: Injecting Table Reasoning Skills During Pre-training via Synthetic Reasoning Examples (2022.emnlp-main)

Challenge:	Existing research shows that a multilingual pre-trained language model fine-tuned with one (source) language performs well on downstream tasks for non-source languages . However, there is a clear performance gap between the source and non-sourced languages - this gap can be reduced by reducing forgetting.
Approach:	They propose a method to fine-tune a multilingual pre-trained language model fine- tuned with one (source) language and four training policies to address the performance gap.
Outcome:	The proposed method outperforms baselines on the XNLI dataset by a clear margin.

Copied to clipboard

Yilun Zhao, Linyong Nan, Zhenting Qi, Rui Zhang, Dragomir Radev

PPTC-R benchmark: Towards Evaluating the Robustness of Large Language Models for PowerPoint Task Completion (2024.findings-emnlp)

Challenge:	Existing models with table-specific architectures and pre-training methods perform well on understanding table structures but lack table reasoning skills.
Approach:	They propose to pre-train tables with table reasoning skills without complex architectures . they define 7 table reasoning skill, and then pre-teach them to generate tables .
Outcome:	The proposed model improves on four tasks and is available on github.

Copied to clipboard

Zekai Zhang, Yiduo Guo, Yaobo Liang, Dongyan Zhao, Nan Duan

Robustness in NLP

Challenge:	Large Language Models (LLMs) are increasingly used for task completion in real-world situations.
Approach:	They propose a PowerPoint Task Completion-Robustness (PPTC-R) benchmark to measure LLMs’ robustness to the user PPT task instruction and software version (Powerpoint).
Outcome:	The proposed benchmark compares 3 closed-source and 4 open-source LLMs to the PowerPoint task instruction and software version (Powerpoint) .

Beyond Boundaries: Learning a Universal Entity Taxonomy across Datasets and Languages for Open Named Entity Recognition (2025.coling-main)

Copied to clipboard

Yuming Yang, Wantong Zhao, Caishuang Huang, Junjie Ye, Xiao Wang, Huiyuan Zheng, Yang Nan, Yuran Wang, Xueying Xu, Kaixin Huang, Yunke Zhang, Tao Gui, Qi Zhang, Xuanjing Huang

Information Extraction & Text Mining

Challenge:	Current Large Language Models struggle with complex entity taxonomies in open domains and lack NER capabilities.
Approach:	They propose a dataset to guide LLMs' generalization in Open NER under a universal entity taxonomy.
Outcome:	The proposed model outperforms GPT-4 in 3 out-of-domain benchmarks across 15 datasets and 6 languages.

PromISe: Releasing the Capabilities of LLMs with Prompt Introspective Search (2024.lrec-main)

Copied to clipboard

Minzheng Wang, Nan Xu, Jiahao Zhao, Yin Luo, Wenji Mao

R2D2: Robust Data-to-Text with Replacement Detection (2022.emnlp-main)

Challenge:	Existing evaluation benchmarks for large language models use uniform manual prompts, resulting in underestimation of performance.
Approach:	They propose a prompt introspective search framework that integrates self-introspect and self-refine to unlock the capabilities of LLMs.
Outcome:	The proposed framework significantly boosts the performance of 12 well-known LLMs compared to baseline methods.

Copied to clipboard

Linyong Nan, Lorenzo Jaime Flores, Yilun Zhao, Yixin Liu, Luke Benson, Weijin Zou, Dragomir Radev

CMMLU: Measuring massive multitask language understanding in Chinese (2024.findings-acl)

Challenge:	Existing methods to mitigate unfaithful text generation are inadequate . data-to-text generation requires a structured input format .
Approach:	They propose a training framework that addresses unfaithful Data-to-Text generation by training a system as a generator and faithfulness discriminator with additional replacement detection and unlikelihood learning tasks.
Outcome:	The proposed training framework improves FeTaQA, LogicNLG, and ToTTo fidelity on D2T systems.

Copied to clipboard

Haonan Li, Yixuan Zhang, Fajri Koto, Yifei Yang, Hai Zhao, Yeyun Gong, Nan Duan, Timothy Baldwin

Fair Abstractive Summarization of Diverse Perspectives (2024.naacl-long)

Challenge:	Existing large language models struggle to achieve an accuracy of even 60%, which is the pass mark for Chinese exams.
Approach:	They propose to use CMMLU to evaluate Chinese multilingual and Chinese LLMs in a comprehensive benchmark that covers various subjects and settings.
Outcome:	The proposed benchmark covers natural sciences, social sciences, engineering, and the humanities and aims to improve on existing models.

Copied to clipboard

Yusen Zhang, Nan Zhang, Yixin Liu, Alexander Fabbri, Junru Liu, Ryo Kamoi, Xiaoxin Lu, Caiming Xiong, Jieyu Zhao, Dragomir Radev, Kathleen McKeown, Rui Zhang

Pruning as a Domain-specific LLM Extractor (2024.findings-naacl)

Challenge:	Existing work on summarization metrics and large language models has not explored fair abstractive summarizing.
Approach:	They propose four reference-free automatic metrics to measure the differences between target and source perspectives.
Outcome:	The proposed methods alleviate fair abstractive summarization on user-generated data.

Copied to clipboard

Nan Zhang, Yanchi Liu, Xujiang Zhao, Wei Cheng, Runxue Bao, Rui Zhang, Prasenjit Mitra, Haifeng Chen

Challenge:	Large Language Models (LLMs) have exhibited remarkable proficiency across a wide array of NLP tasks.
Approach:	They propose a method for pruning large language models using general or task-specific weights to extract a compressed, task-agnostic LLM.
Outcome:	The proposed method extracts a compressed, domain-specific, and task- agnostic LLM by identifying LLM weights that are pivotal for general capabilities, like linguistic capability and multi-task solving, and domain- specific knowledge.

SimANS: Simple Ambiguous Negatives Sampling for Dense Text Retrieval (2022.emnlp-industry)

Copied to clipboard

Kun Zhou, Yeyun Gong, Xiao Liu, Wayne Xin Zhao, Yelong Shen, Anlei Dong, Jingwen Lu, Rangan Majumder, Ji-rong Wen, Nan Duan

Information Retrieval

Challenge:	Existing methods for sapping negatives from large document pool suffer from the uninformative or false negative problem.
Approach:	They propose a method to sample negatives from a large document pool using a new sampling probability distribution.
Outcome:	The proposed method can be used to sample more ambiguous negatives on four public and one industry datasets.

DUET: Joint Exploration of User–Item Profiles in Recommendation System (2026.findings-acl)

Copied to clipboard

Yue Chen, Yifei Sun, Lu Wang, Fangkai Yang, Pu Zhao, Minjie Hong, Yifei Dong, Minghua He, Nan Hu, Jianjin Zhang, Zhiwei Dai, Yuefeng Zhan, Weihao Han, Hao Sun, Qingwei Lin, Weiwei Deng, Feng Sun, Qi Zhang, Saravan Rajmohan, Dongmei Zhang

Challenge:	Existing LLMs are opaque and difficult to interpret, resulting in limited interpretability.
Approach:	They propose an interaction-aware profile generator that jointly produces user and item profiles conditioned on both user history and item evidence.
Outcome:	The proposed model outperforms baselines on three real-world datasets.

A Contrastive Framework for Learning Sentence Representations from Pairwise and Triple-wise Perspective in Angular Space (2022.acl-long)

Copied to clipboard

Yuhao Zhang, Hongji Zhu, Yongliang Wang, Nan Xu, Xiaobo Li, Binqiang Zhao

RobuT: A Systematic Study of Table QA Robustness Against Human-Annotated Adversarial Perturbations (2023.acl-long)

Challenge:	Existing methods for learning sentence representations focus on constitution of positive and negative representation pairs and do not focus on training objective.
Approach:	They propose a new method to learn sentence representations using BERT-like pre-trained models . they use a pairwise discriminating power and a model to model the entailment relation of triplet sentences .
Outcome:	The proposed method outperforms the previous state-of-the-art on diverse sentence related tasks.

Copied to clipboard

Yilun Zhao, Chen Zhao, Linyong Nan, Zhenting Qi, Wenlin Zhang, Xiangru Tang, Boyu Mi, Dragomir Radev

OMG-QA: Building Open-Domain Multi-Modal Generative Question Answering Systems (2024.emnlp-industry)

Challenge:	Existing Table QA models are vulnerable to task-specific perturbations, such as replacing key question entities or shuffling table columns.
Approach:	They propose to use large language models to generate adversarial examples to enhance training, which significantly improves the robustness of Table QA models.
Outcome:	The proposed model significantly improves on existing Table QA models against human-annotated adversarial perturbations.

Copied to clipboard

Linyong Nan, Weining Fang, Aylin Rasteh, Pouya Lahabi, Weijin Zou, Yilun Zhao, Arman Cohan

Natural Language Interfaces

Challenge:	Existing approaches to QA require multiple modalities and a broad pool of information sources to generate coherent answers.
Approach:	They propose a new resource to evaluate the effectiveness of question answering systems that perform retrieval augmented generation in scenarios that demand reasoning on multi-modal, multi-document contexts.
Outcome:	The proposed method evaluates question answering systems that perform retrieval augmented generation (RAG) in open-domain questions . it requires systems to navigate diverse modalities and a broad pool of information sources, making it uniquely challenging.

Streaming Hallucination Detection in Long Chain-of-Thought Reasoning (2026.findings-acl)

Copied to clipboard

Haolang Lu, Minghui Pan, Ripeng LI, Guoshun Nan, Jialin Zhuang, Zijie Zhao, Zhongxiang Sun, Kun Wang, Yang Liu

Reasoning

Challenge:	Long chain-of-thought reasoning improves performance of large language models, yet hallucinations in such settings often emerge subtly and propagate across reasoning steps.
Approach:	They propose to treat step-level hallucination judgments as local observations and introduce a cumulative prefix-level signal that tracks the global evolution of the reasoning state over the entire trajectory.
Outcome:	The proposed method enables streaming hallucination detection in long CoT reasoning, providing real-time, interpretable evidence.

PASTA: Table-Operations Aware Fact Verification via Sentence-Table Cloze Pre-training (2022.emnlp-main)

Copied to clipboard

Zihui Gu, Ju Fan, Nan Tang, Preslav Nakov, Xiaoman Zhao, Xiaoyong Du

Ethical NLP

Challenge:	Table-based fact verification has attracted a lot of attention recently due to the lack of datasets that can be used to pre-train language models to be aware of common table operations.
Approach:	They propose a table-based fact verification tool that pre-trains language models to be aware of common table operations such as aggregating a column or comparing tuples.
Outcome:	The proposed method outperforms previous methods on two table-based fact verification datasets TabFact and SEM-TAB- FACTS.

Investigating Table-to-Text Generation Capabilities of Large Language Models in Real-World Information Seeking Scenarios (2023.emnlp-industry)

Copied to clipboard

Yilun Zhao, Haowei Zhang, Shengyun Si, Linyong Nan, Xiangru Tang, Arman Cohan

On Evaluating the Integration of Reasoning and Action in LLM Agents with Database Question Answering (2024.findings-naacl)

Challenge:	Existing table-to-text generation techniques that transform complex tabular data into comprehensible narratives are lacking in real-world applications.
Approach:	They investigate the table-to-text capabilities of different LLMs using four datasets within two real-world information seeking scenarios.
Outcome:	The proposed models can generate table-to-text data in two real-world information seeking scenarios and perform better than existing models.

Copied to clipboard

Linyong Nan, Ellen Zhang, Weijin Zou, Yilun Zhao, Wenfei Zhou, Arman Cohan

Natural Language Interfaces

Challenge:	a new study evaluates how Large Language Models interact with a SQL interpreter . the model is limited in context and is stochastic, making it less suited for tasks requiring high precision and extensive computations.
Approach:	They propose and evaluate two interaction strategies to evaluate how LLMs interact with a SQL interpreter.
Outcome:	The proposed framework improves the accuracy and reliability of the evaluations.

LoFT: Enhancing Faithfulness and Diversity for Table-to-Text Generation via Logic Form Control (2023.eacl-main)

Copied to clipboard

Yilun Zhao, Zhenting Qi, Linyong Nan, Lorenzo Jaime Flores, Dragomir Radev

PPTC Benchmark: Evaluating Large Language Models for PowerPoint Task Completion (2024.findings-acl)

Challenge:	Existing models for LT2T generation focus on surface-level realizations without much logical inference.
Approach:	They propose a model that uses logic forms as fact verifiers and content planners to control LT2T generation.
Outcome:	Experimental results show that the proposed model addresses unfaithfulness and diversity issues simultaneously.

Copied to clipboard

Yiduo Guo, Zekai Zhang, Yaobo Liang, Dongyan Zhao, Nan Duan

Little Giants: Synthesizing High-Quality Embedding Data at Scale (2025.naacl-long)

Challenge:	Recent evaluations of Large Language Models (LLMs) focus on their zero-shot/few-shot capabilities for basic natural language tasks and their ability to translate instructions into tool APIs.
Approach:	They propose a PowerPoint Task Completion benchmark to assess LLMs’ ability to create and edit PPT files based on user instructions.
Outcome:	The proposed system outperforms open-source and closed LLMs with 75.1% accuracy in single-turn dialogue testing but only achieves 6% session accuracy.

Copied to clipboard

Haonan Chen, Liang Wang, Nan Yang, Yutao Zhu, Ziliang Zhao, Furu Wei, Zhicheng Dou

Enhancing Text-to-SQL Capabilities of Large Language Models: A Study on Prompt Design Strategies (2023.findings-emnlp)

Challenge:	Synthetic data generation is an increasingly popular way of training models without the need for large, manually labeled datasets.
Approach:	They propose a framework that aligns open-source small models to efficiently generate large-scale embedding data.
Outcome:	The proposed framework outperforms state-of-the-art embedding models by using only 1/10 of the GPT API calls.

Copied to clipboard

Linyong Nan, Yilun Zhao, Weijin Zou, Narutatsu Ri, Jaesung Tae, Ellen Zhang, Arman Cohan, Dragomir Radev

Programming Languages in NLP

Challenge:	In-context learning (ICL) is a new approach to natural language processing tasks that rely on large language models to make predictions based on context . recent studies have shown that neural symbolic design is the preferred choice for question answering systems because of its limited working memory and unreliable long-term memory.
Approach:	They propose to extend in-context learning to question answering tasks that utilize structured knowledge sources and to explore various prompt design strategies for employing LLMs.
Outcome:	The proposed approach outperforms the state-of-the-art system by 2.5 points and the best fine-tuned system by 5.1 points on the Spider dataset.

mmE5: Improving Multimodal Multilingual Embeddings via High-quality Synthetic Data (2025.findings-acl)

Copied to clipboard

Haonan Chen, Liang Wang, Nan Yang, Yutao Zhu, Ziliang Zhao, Furu Wei, Zhicheng Dou

Neural Document Summarization by Jointly Learning to Score and Select Sentences (P18-1)

Challenge:	Multimodal embedding models encode multimedia inputs into latent vector representations.
Approach:	They propose to synthesize multimodal multilingual data using a multimodal large language model . they identify three criteria for high-quality synthetic multimodal data .
Outcome:	The proposed model outperforms existing models on the MMEB Benchmark and the XTD benchmark.

Copied to clipboard

Qingyu Zhou, Nan Yang, Furu Wei, Shaohan Huang, Ming Zhou, Tiejun Zhao

Information Extraction & Text Mining

Challenge:	Sentence scoring and sentence selection are two main steps in extractive document summarization systems.
Approach:	They propose an end-to-end neural network framework for extractive document summarization by jointly learning to score and select sentences.
Outcome:	The proposed framework outperforms the state-of-the-art summarization models on the CNN/Daily Mail dataset.

Learning to Collaborate for Question Answering and Asking (N18-1)

Copied to clipboard

Duyu Tang, Nan Duan, Zhao Yan, Zhirui Zhang, Yibo Sun, Shujie Liu, Yuanhua Lv, Ming Zhou

Natural Language Interfaces

Challenge:	Question answering (QA) and question generation (QG) are closely related tasks.
Approach:	They propose a training algorithm that generalizes both Generative Adversarial Network and Generating Domain-Adaptive Nets under the question answering scenario.
Outcome:	The proposed training algorithm generalizes both Generative Adversarial Network (GAN) and Generating Domain-Adaptive Nets (GDAN) under the question answering scenario.

Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation (2023.acl-long)

Copied to clipboard

Yixin Liu, Alex Fabbri, Pengfei Liu, Yilun Zhao, Linyong Nan, Ruilin Han, Simeng Han, Shafiq Joty, Chien-Sheng Wu, Caiming Xiong, Dragomir Radev

Robustness in NLP

Challenge:	Existing studies for summarization evaluation exhibit low inter-annotator agreement or lack scale.
Approach:	They propose a modified summarization salience protocol based on fine-grained semantic units and a robust summarizing evaluation benchmark.
Outcome:	The proposed protocol is based on fine-grained semantic units and allows for high inter-annotator agreement.

QTSumm: Query-Focused Summarization over Tabular Data (2023.emnlp-main)

Copied to clipboard

Yilun Zhao, Zhenting Qi, Linyong Nan, Boyu Mi, Yixin Liu, Weijin Zou, Simeng Han, Ruizhe Chen, Xiangru Tang, Yumo Xu, Dragomir Radev, Arman Cohan

TED-TTS: Training-Free Intra-Utterance Emotion and Duration Control for Text-to-Speech Synthesis (2026.acl-long)

Challenge:	Existing text generation systems that can provide accurate table summaries can facilitate more efficient access to relevant data insights.
Approach:	They propose a query-focused task where text generation models have to perform human-like reasoning and analysis over the given table to generate a tailored table summary.
Outcome:	The proposed method improves existing baselines on table-to-text generation and large language models by concatenating generated facts to the model input.

Copied to clipboard

Qifan Liang, Yuansen Liu, Ruixin Wei, Nan Lu, Junchuan Zhao, Ye Wang