Papers by Baolin Peng

SocAoG: Incremental Graph Parsing for Social Relation Inference in Dialogues (2021.acl-long)

Challenge:	Existing solutions for math reasoning tasks use semantic parsing or AST decoding, but performance can degrade dramatically even with slight changes to the questions.
Approach:	They propose three calibration methods based on self-consistency for math reasoning tasks.
Outcome:	The proposed methods bridge model confidence and accuracy better than existing methods based on p(True) or logit.

Copied to clipboard

Liang Qiu, Yuan Liang, Yizhou Zhao, Pan Lu, Baolin Peng, Zhou Yu, Ying Nian Wu, Song-Chun Zhu

Structured Data in NLP

Challenge:	Existing studies focus on identifying entities' relations from the semantics of dialogues-they utilize either the attention mechanism or a refined token graph to locate informative words.
Approach:	They propose a sequential structure prediction task to incrementally parse SocAoG for dynamic inference upon any incoming utterance.
Outcome:	Empirical results show that the proposed model infers social relations more accurately than the state-of-the-art methods.

#HowYouTagTweets: Learning User Hashtagging Preferences via Personalized Topic Attention (2021.emnlp-main)

Copied to clipboard

Yuji Zhang, Yubo Zhang, Chunpu Xu, Jing Li, Ziyan Jiang, Baolin Peng

Tagging

Challenge:	Existing methods based on latent topics cannot capture user interests and thus can't be used to predict how likely a user will post with a hashtag.
Approach:	They propose a personalized topic attention model that captures salient contents to personalize hashtag contexts by predicting how likely a user will post with a hashtag.
Outcome:	The proposed model significantly outperforms the state-of-the-art recommendation approach without exploiting latent topics.

Z-Code++: A Pre-trained Language Model Optimized for Abstractive Summarization (2023.acl-long)

Copied to clipboard

Pengcheng He, Baolin Peng, Song Wang, Yang Liu, Ruochen Xu, Hany Hassan, Yu Shi, Chenguang Zhu, Wayne Xiong, Michael Zeng, Jianfeng Gao, Xuedong Huang

Sub-Sentence Encoder: Contrastive Learning of Propositional Semantic Representations (2024.naacl-long)

Challenge:	Z-Code++ is a pre-trained language model optimized for abstractive text summarization.
Approach:	They propose a pre-trained language model optimized for abstractive text summarization that uses a two-phase pre-training technique to improve model's performance.
Outcome:	The proposed model outperforms the competing models on low-resource summarization tasks in zero-shot and few-shot settings.

Copied to clipboard

Sihao Chen, Hongming Zhang, Tong Chen, Ben Zhou, Wenhao Yu, Dian Yu, Baolin Peng, Hongwei Wang, Dan Roth, Dong Yu

Semantic Text Processing

Challenge:	Sentence embeddings are typically learned to recognize the semantic relation between two text inputs.
Approach:	They introduce a contrastively-learned contextual embedding model for fine-grained semantic representation of text.
Outcome:	The proposed model is able to produce contextual embeddings corresponding to different atomic propositions, i.e. semantic equivalence between propositions across different text sequences.

Self-Alignment for Factuality: Mitigating Hallucinations in LLMs via Self-Evaluation (2024.acl-long)

Copied to clipboard

Xiaoying Zhang, Baolin Peng, Ye Tian, Jingyan Zhou, Lifeng Jin, Linfeng Song, Haitao Mi, Helen Meng

Interactive Text Generation (2023.emnlp-main)

Challenge:	Existing approaches to addressing factual inaccuracies require high-quality human factuality annotations to mitigate these hallucinations.
Approach:	They propose to leverage the self-evaluation capability of an LLM to provide training signals that steer the model towards factuality.
Outcome:	The proposed approach significantly improves factual accuracy over LLMs across three key knowledge-intensive tasks on TruthfulQA and BioGEN.

Copied to clipboard

Felix Faltings, Michel Galley, Kianté Brantley, Baolin Peng, Weixin Cai, Yizhe Zhang, Jianfeng Gao, Bill Dolan

Multimodality

Challenge:	Advances in generative modeling have made it possible to automatically generate high-quality texts, code, and images, but they can be unsatisfactory in many respects.
Approach:	They propose a task that allows training generation models interactively without the costs of involving real users.
Outcome:	The proposed model trains with Imitation Learning without the cost of involving real users and is superior to non-interactive models.

Few-Shot Named Entity Recognition: An Empirical Baseline Study (2021.emnlp-main)

Copied to clipboard

Jiaxin Huang, Chunyuan Li, Krishan Subudhi, Damien Jose, Shobana Balakrishnan, Weizhu Chen, Baolin Peng, Jianfeng Gao, Jiawei Han

Learning Efficient Dialogue Policy from Demonstrations through Shaping (2020.acl-main)

Challenge:	Existing methods to build named entity recognition systems with limited labeled data are lacking.
Approach:	They propose three orthogonal schemes to build named entity recognition systems when labeled data is limited.
Outcome:	The proposed NER systems outperform existing methods on few-shot and training-free settings.

Copied to clipboard

Huimin Wang, Baolin Peng, Kam-Fai Wong

RADDLE: An Evaluation Benchmark and Analysis Platform for Robust Task-oriented Dialog Systems (2021.acl-long)

Challenge:	Using reinforcement learning to learn dialogue policy requires a large volume of interactions with users.
Approach:	They propose a task-oriented dialogue agent that efficiently learns dialogue policy from demonstrations . they use an imitation model to distill knowledge from demonstration and reward shaping .
Outcome:	The proposed agent efficiently learns dialogue policy from demonstrations through policy shaping and reward shaping.

Copied to clipboard

Baolin Peng, Chunyuan Li, Zhu Zhang, Chenguang Zhu, Jinchao Li, Jianfeng Gao

Robustness in NLP

Challenge:	Existing task-oriented dialog systems are less than satisfactory in robustness evaluation . existing systems are weak in robustity evaluation based on pre-training and fine-tuning .
Approach:	They propose to use a set of training examples to evaluate model generalization ability . they propose to include tasks with limited training data to favor models with strong generalization abilities .
Outcome:	The proposed model generalizes well with limited training data and is robust to user input across domains.

Entropy Guided Extrapolative Decoding to Improve Factuality in Large Language Models (2025.coling-main)

Copied to clipboard

Souvik Das, Lifeng Jin, Linfeng Song, Haitao Mi, Baolin Peng, Dong Yu

Deep Dyna-Q: Integrating Planning for Task-Completion Dialogue Policy Learning (P18-1)

Challenge:	Large language models (LLMs) exhibit impressive natural language capabilities but suffer from hallucination – generating content that does not align with realworld facts.
Approach:	They propose to extrapolate critical token probabilities beyond the last layer to improve decoding by manipulating the predicted distributions at inference time.
Outcome:	The proposed methods surpass state-of-the-art on multiple datasets by large margins.

Copied to clipboard

Baolin Peng, Xiujun Li, Jianfeng Gao, Jingjing Liu, Kam-Fai Wong

Optimus: Organizing Sentences via Pre-trained Modeling of a Latent Space (2020.emnlp-main)

Challenge:	Training a task-completion dialogue agent via reinforcement learning (RL) is costly because it requires many interactions with real users.
Approach:	They propose a framework that integrates planning for task-completion dialogue policy learning into a dialogue agent using a world model to mimic real user response and generate simulated experience.
Outcome:	The proposed framework integrates planning for task-completion dialogue policy learning with real user interaction and simulated user behavior.

Copied to clipboard

Chunyuan Li, Xiang Gao, Yuan Li, Baolin Peng, Xiujun Li, Yizhe Zhang, Jianfeng Gao

Semantic Text Processing

Challenge:	Existing models for language understanding and understanding can be trained to provide contextualized representations of words based on text data.
Approach:	They propose a large-scale language VAE model Optimus that is pre-trained on large text corpus and fine-tuned for various language generation and understanding tasks.
Outcome:	The proposed model achieves new state-of-the-art on VAE language modeling benchmarks.

Conversation Learner - A Machine Teaching Tool for Building Dialog Managers for Task-Oriented Dialog Systems (2020.acl-demos)

Copied to clipboard

Swadheen Shukla, Lars Liden, Shahin Shayandeh, Eslam Kamal, Jinchao Li, Matt Mazzola, Thomas Park, Baolin Peng, Jianfeng Gao

Evaluating the Instruction-Following Robustness of Large Language Models to Prompt Injection (2024.emnlp-main)

Challenge:	a wide variety of tasks have created a need for flexible task-oriented dialog systems . dialog flows are intuitively interpretable but lack the flexibility needed to handle complex dialogs .
Approach:	They propose a machine teaching tool for building dialog managers using familiar tools . they convert the dialog flow into a parametric model and use user-system dialog logs as training data .
Outcome:	The proposed tool combines the best of both approaches to build dialog managers . it converts the dialog flow into a parametric model and improves it over time .

Copied to clipboard

Zekun Li, Baolin Peng, Pengcheng He, Xifeng Yan

Responsible & Trustworthy NLP

Challenge:	Large Language Models (LLMs) have demonstrated exceptional proficiency in instruction-following, making them increasingly integral to various applications.
Approach:	They establish a benchmark to evaluate the robustness of instruction-following LLMs against prompt injection attacks, assessing their ability to discern which instructions to follow and which to disregard.
Outcome:	The proposed model is overly sensitive to prompt injection attacks, focusing on the latter part of the prompt without fully understanding the context.

Grounded Keys-to-Text Generation: Towards Factual Open-Ended Generation (2022.findings-emnlp)

Copied to clipboard

Faeze Brahman, Baolin Peng, Michel Galley, Sudha Rao, Bill Dolan, Snigdha Chaturvedi, Jianfeng Gao

Text Generation

Challenge:	Large pre-trained language models have enabled open-ended generation frameworks to tackle a variety of tasks beyond data-to-text generation.
Approach:	They propose a new task to generate a factual description about an entity given guiding keys and grounding passages using a dataset.
Outcome:	The proposed model improves factual correctness and recall significantly compared to previous models.

Improving LLM Generations via Fine-Grained Self-Endorsement (2024.findings-acl)

Copied to clipboard

Ante Wang, Linfeng Song, Baolin Peng, Lifeng Jin, Ye Tian, Haitao Mi, Jinsong Su, Dong Yu

Few-shot Natural Language Generation for Task-Oriented Dialog (2020.findings-emnlp)

Challenge:	Recent large language models (LLMs) have demonstrated remarkable capabilities but can still fail frequently on knowledge-intensive tasks.
Approach:	They propose a self-endorsement framework that leverages fine-grained fact-level comparisons across multiple sampled responses.
Outcome:	The proposed framework can improve factuality of generations with simple prompts across scales of LLMs.

Copied to clipboard

Baolin Peng, Chenguang Zhu, Chunyuan Li, Xiujun Li, Jinchao Li, Michael Zeng, Jianfeng Gao

Task-oriented Dialogue System for Automatic Diagnosis (P18-2)

Challenge:	Existing methods for NLG depend on heavily annotated data, which is infeasible for new domains.
Approach:	They propose a system that converts a dialog act into a response in natural language . they propose 'nuclear language generation' to simulate a few-shot learning setting .
Outcome:	The proposed model outperforms existing methods on a large set of annotated datasets.

Copied to clipboard

Zhongyu Wei, Qianlong Liu, Baolin Peng, Huaixiao Tou, Ting Chen, Xuanjing Huang, Kam-fai Wong, Xiangying Dai

Natural Language Interfaces

Challenge:	Existing methods to identify phenotypes using electronic health records (EHRs) are expensive and difficult to transfer models from one disease to another.
Approach:	They propose a task-oriented dialogue system framework to make diagnosis for patients automatically, which can converse with patients to collect additional symptoms beyond their self-reports.
Outcome:	The proposed system can collect additional symptoms from conversation and improve disease identification accuracy.

LEDGER: Scaling Agentic Document Editing with Dependency-aware Graph Retrieval (2026.findings-acl)

Copied to clipboard

Hang Wang, Utkarsh Garg, Reza Davari, Huitian Jiao, Hao Cheng, Baolin Peng, Si-Qing Chen, Tao Ge

Green & Sustainable NLP

Challenge:	Document editing requires full-context awareness of dependencies, but processing entire documents for each edit incurs prohibitive token costs and latency.
Approach:	a framework that constructs lightweight dependency graphs captures semantic relationships and structural hierarchies across document elements is proposed for agentic document editing . a scaLing agentic agentic framework is based on a dependency graph framework that captures dependencies and refactors function dependencies.
Outcome:	a new framework achieves 76 consistency versus 56 baseline while reducing token usage by 85 . the framework is based on a framework that captures semantic relationships and structural hierarchies across document elements . it can be used to improve document consistency, but it also reduces token costs and latency .

ConvLab-3: A Flexible Dialogue System Toolkit Based on a Unified Data Format (2023.emnlp-demo)

Copied to clipboard

Qi Zhu, Christian Geishauser, Hsien-chin Lin, Carel van Niekerk, Baolin Peng, Zheng Zhang, Shutong Feng, Michael Heck, Nurul Lubis, Dazhen Wan, Xiaochen Zhu, Jianfeng Gao, Milica Gasic, Minlie Huang

Robustness in NLP

Challenge:	Existing tools for building TOD systems often lack a user-friendly interface . a toolkit with advanced, easily integrable modules is needed to bridge this gap .
Approach:	They propose a multifaceted dialogue system toolkit that integrates diverse datasets and models with a streamlined training process and in-depth evaluation tools.
Outcome:	The proposed toolkit combines RL and transfer learning to support the rapid development and evaluation of robust dialogue policies.

Self-Tuning: Instructing LLMs to Effectively Acquire New Knowledge through Self-Teaching (2025.findings-acl)

Copied to clipboard

Xiaoying Zhang, Baolin Peng, Ye Tian, Jingyan Zhou, Yipeng Zhang, Haitao Mi, Helen M. Meng

SimulatorArena: Are User Simulators Reliable Proxies for Multi-Turn Evaluation of AI Assistants? (2025.emnlp-main)

Challenge:	Existing approaches to keeping large language models current involve continued pre-training on new documents.
Approach:	They propose a learning framework that augments documents with knowledge-intensive tasks created in a self-supervised manner, focusing on memorization, comprehension, and self-reflection.
Outcome:	The proposed learning framework improves an LLM’s ability to acquire new knowledge from unseen raw documents through self-teaching.

Copied to clipboard

Yao Dou, Michel Galley, Baolin Peng, Chris Kedzie, Weixin Cai, Alan Ritter, Chris Quirk, Wei Xu, Jianfeng Gao

Self-Checker: Plug-and-Play Modules for Fact-Checking with Large Language Models (2024.findings-naacl)

Challenge:	Large language models (LLMs) are increasingly used in interactive applications, and human evaluation remains the gold standard for assessing their performance in multi-turn conversations.
Approach:	They propose to use large language models to simulate users for automatic assistant evaluation.
Outcome:	The proposed model outperforms human evaluations on two interactive tasks and achieves Spearman’s of 0.7 on both tasks.

Copied to clipboard

Miaoran Li, Baolin Peng, Michel Galley, Jianfeng Gao, Zhu Zhang

DIONYSUS: A Pre-trained Model for Low-Resource Dialogue Summarization (2023.acl-long)

Challenge:	Existing methods for fact-checking text generated by large language models are expensive and time-consuming.
Approach:	They propose a plug-and-play framework that harnesses large language models for efficient fact-checking in a few-shot manner.
Outcome:	The proposed framework is compared with state-of-the-art models and shows that it can be used to speed up fact-checking in a few-shot manner.

Copied to clipboard

Yu Li, Baolin Peng, Pengcheng He, Michel Galley, Zhou Yu, Jianfeng Gao

Teaching Language Models to Self-Improve through Interactive Demonstrations (2024.naacl-long)

Challenge:	Existing methods for summarizing dialogues lack in taking into account the structure of dialogues and rely heavily on labeled data.
Approach:	They propose a pre-trained encoder-decoder model for summarizing dialogues in any new domain.
Outcome:	The proposed model outperforms existing methods on six datasets and shows ROUGE scores in zero-shot and few-shot settings.

Copied to clipboard

Xiao Yu, Baolin Peng, Michel Galley, Jianfeng Gao, Zhou Yu

SGP-TOD: Building Task Bots Effortlessly via Schema-Guided LLM Prompting (2023.findings-emnlp)

Challenge:	Large language models (LLMs) have been shown to improve performance on downstream tasks by prompting them to analyze and revise their outputs.
Approach:	They propose a training algorithm that prompts large language models to analyze and revise their own outputs and uses this feedback to train the small model.
Outcome:	The proposed approach improves LLaMA-7B's performance on math and reasoning tasks by up to 7.13%.

Copied to clipboard

Xiaoying Zhang, Baolin Peng, Kun Li, Jingyan Zhou, Helen Meng

SynthAgent: Adapting Web Agents with Synthetic Supervision (2026.acl-long)

Challenge:	Experimental results show that SGP-TOD provides state-of-the-art zero-shot performance . prevailing approach for creating task bots is to fine-tune pre-trained language models .
Approach:	They propose a Schema-Guided Prompting for building Task-Oriented Dialog systems . they use predefined task schema and dialog policy to instruct fixed LLMs to generate appropriate responses .
Outcome:	The proposed system outperforms few-shot approaches on multiwoz, RADDLE, and STAR datasets.

Copied to clipboard

Zhaoyang Wang, Yiming Liang, Xuchao Zhang, Qianhui Wu, Siwei Han, Anson Bastos, Rujia Wang, Chetan Bansal, Baolin Peng, Jianfeng Gao, Saravan Rajmohan, Huaxiu Yao

ConvLab-2: An Open-Source Toolkit for Building, Evaluating, and Diagnosing Dialogue Systems (2020.acl-demos)

Challenge:	Existing studies have focused on synthetic supervision but have encountered data quality issues.
Approach:	They propose a fully synthetic supervision framework that aims at improving data quality via dual refinement of both tasks and trajectories.
Outcome:	The proposed framework outperforms existing methods on standardized benchmarks and shows promising results on a standardized test.

Copied to clipboard

Qi Zhu, Zheng Zhang, Yan Fang, Xiang Li, Ryuichi Takanobu, Jinchao Li, Baolin Peng, Jianfeng Gao, Xiaoyan Zhu, Minlie Huang

Natural Language Interfaces

Challenge:	ConvLab-2 inherits Convlab's framework but integrates more powerful dialogue models and supports more datasets.
Approach:	They present ConvLab-2, an open-source toolkit that enables researchers to build task-oriented dialogue systems with state-of-the-art models and perform an end-to-end evaluation.
Outcome:	The new tool inherits ConvLab's framework and extends it by integrating many recently proposed state-of-the-art dialogue models.

Soloist: Building Task Bots at Scale with Transfer Learning and Machine Teaching (2021.tacl-1)

Copied to clipboard

Baolin Peng, Chunyuan Li, Jinchao Li, Shahin Shayandeh, Lars Liden, Jianfeng Gao

Guided Dialogue Policy Learning without Adversarial Learning in the Loop (2020.findings-emnlp)

Challenge:	Existing methods for building task-oriented dialog systems are limited to a few tasks and domains.
Approach:	They propose a method that uses transfer learning and machine teaching to build task bots at scale.
Outcome:	The proposed method outperforms existing methods on well-studied task-oriented dialog benchmarks on well studied tasks.

Copied to clipboard

Ziming Li, Sungjin Lee, Baolin Peng, Jinchao Li, Julia Kiseleva, Maarten de Rijke, Shahin Shayandeh, Jianfeng Gao

Robustness in NLP

Challenge:	Reinforcement learning methods suffer from sparse and unstable reward signals . alternating training of dialogue agent and reward model can get stuck in local optima .
Approach:	They propose to decompose adversarial training into two steps to improve dialogue policy learning.
Outcome:	The proposed method achieves remarkable task success rate using both on-policy and off-poly reinforcement learning methods.

ConvLab: Multi-Domain End-to-End Dialog System Platform (P19-3)

Copied to clipboard

Sungjin Lee, Qi Zhu, Ryuichi Takanobu, Zheng Zhang, Yaoqin Zhang, Xiang Li, Jinchao Li, Baolin Peng, Xiujun Li, Minlie Huang, Jianfeng Gao

Engage the Public: Poll Question Generation for Social Media Posts (2021.acl-long)

Challenge:	ConvLab is an open-source multi-domain end-to-end dialog system platform . it allows researchers to quickly set up experiments with reusable components and compare a large set of different approaches in common environments.
Approach:	They propose to use an open-source multi-domain end-to-end dialog system platform to train and evaluate dialog bots in common environments.
Outcome:	The proposed system enables researchers to quickly set up experiments with reusable components and compare a large set of different approaches in common environments.

Copied to clipboard

Zexin Lu, Keyang Ding, Yuji Zhang, Jing Li, Baolin Peng, Lemao Liu