Papers by Xiaodan Liang

CLOMO: Counterfactual Logical Modification with Large Language Models (2024.acl-long)

Challenge:	Existing methods of fine-tuning vision-language navigation models require extra human-labeled data and lack self-exploration capabilities in environments.
Approach:	They propose a method that can self-explore environments without human labeling . they use a large-scale cross-modal pretrained model to build an in-domain dataset .
Outcome:	The proposed model can self-explore environments without human labeling without human supervision and generates structured instructions without human intervention.

Copied to clipboard

Yinya Huang, Ruixin Hong, Hongming Zhang, Wei Shao, Zhicheng Yang, Dong Yu, Changshui Zhang, Xiaodan Liang, Linqi Song

VisDiaHalBench: A Visual Dialogue Benchmark For Diagnosing Hallucination in Large Vision-Language Models (2024.acl-long)

Challenge:	Existing studies on evaluating model reasoning are limited in both form and content.
Approach:	They propose a task to cultivate counterfactual thought processes within large language models and an evaluation metric to evaluate their natural language output instead of modeling the task as a multiple-choice problem.
Outcome:	The proposed evaluation metric aligns well with human preference.

Copied to clipboard

Qingxing Cao, Junhao Cheng, Xiaodan Liang, Liang Lin

Unbiased Math Word Problems Benchmark for Mitigating Solving Bias (2022.findings-naacl)

Challenge:	Despite the significant success of large vision-language models, some studies have revealed that LVLMs suffer from the hallucination problem when given long-term misleading textual history.
Approach:	They propose a visual dialogue hallucination evaluation benchmark VisDiaHalBench to investigate the halluciation problem of large vision-language models when given long-term misleading textual history.
Outcome:	The proposed benchmark consists of samples with five-turn questions about an edited image and its original version.

Copied to clipboard

Zhicheng Yang, Jinghui Qin, Jiaqi Chen, Xiaodan Liang

Numerical Reasoning

Challenge:	Existing solvers with data bias and learning bias only learn shallow heuristics rather than deep semantics for understanding problems.
Approach:	They propose a MWP dataset named UnbiasedMWP which is constructed by varying the grounded expressions in collected data and annotating them manually.
Outcome:	The proposed dataset has significantly fewer biases than its original data and other datasets, posing a promising benchmark for fairly evaluating the solvers’ reasoning skills rather than matching nearest neighbors.

Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation (P19-3)

Copied to clipboard

Zhiting Hu, Haoran Shi, Bowen Tan, Wentao Wang, Zichao Yang, Tiancheng Zhao, Junxian He, Lianhui Qin, Di Wang, Xuezhe Ma, Zhengzhong Liu, Xiaodan Liang, Wanrong Zhu, Devendra Sachan, Eric Xing

Text Generation

Challenge:	Texar is an open-source text generation toolkit that supports a broad set of text generation tasks.
Approach:	They introduce Texar, an open-source text generation toolkit that supports text generation tasks.
Outcome:	Texar supports machine translation, summarization, dialog, content manipulation, and more.

GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoning (2021.findings-acl)

Copied to clipboard

Jiaqi Chen, Jianheng Tang, Jinghui Qin, Xiaodan Liang, Lingbo Liu, Eric Xing, Liang Lin

Target-Guided Open-Domain Conversation (P19-1)

Challenge:	Existing methods to solve geometric problems are dependent on handcraft rules and limited on small-scale datasets.
Approach:	They propose a Geometric Question Answering dataset with 5,010 geometric problems with corresponding annotated programs to illustrate the solving process.
Outcome:	The proposed method is significantly lower than human performance on the proposed dataset than on a publicly available dataset.

Copied to clipboard

Jianheng Tang, Tiancheng Zhao, Chenyan Xiong, Xiaodan Liang, Eric Xing, Zhiting Hu

Dialogue Systems & Conversational Agents

Challenge:	a new study aims to improve opendomain chat systems by integrating goals and strategy into the system.
Approach:	They propose a structured approach that introduces coarse-grained keywords to control intended content of system responses and attains smooth conversation transition through turn-level supervised learning.
Outcome:	The proposed system produces meaningful and effective conversations significantly better than other approaches.

DialogGen: Multi-modal Interactive Dialogue System with Multi-turn Text-Image Generation (2025.findings-naacl)

Copied to clipboard

Minbin Huang, Yanxin Long, Xinchi Deng, Ruihang Chu, Jiangfeng Xiong, Xiaodan Liang, Hong Cheng, Qinglin Lu, Wei Liu

TRIGO: Benchmarking Formal Mathematical Proof Reduction for Generative Language Models (2023.emnlp-main)

Challenge:	Text-to-image (T2I) generation models have advanced in recent years, but effective interaction with these models is challenging for average users due to the need for specialized prompt engineering knowledge and the inability to perform multi-turn image generation.
Approach:	They propose to use off-the-shelf MLLMs and T2I models to build a multi-modal interactive dialogue system (MIDS) that can generate correct output modalities and coherence of output images.
Outcome:	The proposed pipeline can generate correct output modalities and coherent multi-modal outputs compared with other state-of-the-art models.

Copied to clipboard

Jing Xiong, Jianhao Shen, Ye Yuan, Haiming Wang, Yichun Yin, Zhengying Liu, Lin Li, Zhijiang Guo, Qingxing Cao, Yinya Huang, Chuanyang Zheng, Xiaodan Liang, Ming Zhang, Qun Liu

DAGN: Discourse-Aware Graph Network for Logical Reasoning (2021.naacl-main)

Challenge:	Automated theorem proving (ATP) benchmarks focus on symbolic inference but rarely involve understanding complex number combination reasoning.
Approach:	They propose a benchmark that requires a model to reduce a trigonometric expression with step-by-step proof and evaluates a generative LM’s reasoning ability on formulas and ability to manipulate, group, and factor number terms.
Outcome:	The proposed benchmark evaluates a generative LM’s reasoning ability on formulas and ability to manipulate, group, and factor number terms.

Copied to clipboard

Yinya Huang, Meng Fang, Yu Cao, Liwei Wang, Xiaodan Liang

Structured Data in NLP

Challenge:	Recent QA with logical reasoning questions requires passage-level relations among the sentences.
Approach:	They propose a discourse-aware graph network that aggregates passage-level clues for QA by using discourse-based information.
Outcome:	The proposed model achieves competitive results on two logical reasoning QA datasets.

Structured Preference Optimization for Vision-Language Long-Horizon Task Planning (2025.emnlp-main)

Copied to clipboard

Xiwen Liang, Min Lin, Weiqi Ruan, Rongtao Xu, Yuecheng Liu, Jiaqi Chen, Bingqian Lin, Yuzheng Zhuang, Xiaodan Liang

Improving Multi-turn Emotional Support Dialogue Generation with Lookahead Strategy Planning (2022.emnlp-main)

Challenge:	Existing vision-language planning methods struggle with long-horizon reasoning in dynamic environments due to the difficulty of training models to generate high-quality reasoning processes.
Approach:	They propose a framework that enhances reasoning and action selection for long-horizon task planning through structured evaluation and optimized training.
Outcome:	The proposed framework outperforms existing methods on short-horizon tasks but struggles with long-horizon reasoning in dynamic environments.

Copied to clipboard

Yi Cheng, Wenge Liu, Wenjie Li, Jiashuo Wang, Ruihui Zhao, Bang Liu, Xiaodan Liang, Yefeng Zheng

Dialogue Systems & Conversational Agents

Challenge:	Existing research on building ES conversation systems only considered single-turn interactions with users, which is over-simplified and has limited support for multi-turn systems.
Approach:	They propose a multi-turn ES conversation system that uses lookahead heuristics to estimate future user feedback after using particular strategies.
Outcome:	The proposed system significantly outperforms baselines in both dialogue generation and strategy planning.

Towards Quantifiable Dialogue Coherence Evaluation (2021.acl-long)

Copied to clipboard

Zheng Ye, Liucun Lu, Lishan Huang, Liang Lin, Xiaodan Liang

Dialogue Systems & Conversational Agents

Challenge:	Existing automatic dialogue coherence evaluation metrics are expensive and high-latency, which cannot meet the requirements of a dialogue system.
Approach:	They propose a framework to train a quantifiable dialogue coherence metric that can reflect actual human rating standards.
Outcome:	Experimental results show that the model trained by QuantiDCE presents stronger correlations with human judgements than the other state-of-the-art metrics.

Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoning (2021.acl-long)

Copied to clipboard

Pan Lu, Ran Gong, Shibiao Jiang, Liang Qiu, Siyuan Huang, Xiaodan Liang, Song-Chun Zhu

Responsible & Trustworthy NLP

Challenge:	Existing methods for solving geometric problems are either small in scale or not publicly available.
Approach:	They propose a large-scale benchmark for geometric problem solving using formal language and symbolic reasoning.
Outcome:	The proposed approach parses geometry problems into formal language and performs symbolic reasoning step by step.

AlignedCoT: Prompting Large Language Models via Native-Speaking Demonstrations (2024.findings-emnlp)

Copied to clipboard

Zhicheng Yang, Yinya Huang, Jing Xiong, Liang Feng, Xiaodan Liang, Yiwei Wang, Jing Tang

UniGeo: Unifying Geometry Logical Reasoning via Reformulating Mathematical Expression (2022.emnlp-main)

Challenge:	Existing LLMs are delicate and elusive in prompt words and styles.
Approach:	They propose an LLM-acquainted prompting technique that includes proficient "native-speaking" they propose to use in-context learning to prompt LLMs to perform high-performance reasoning .
Outcome:	The proposed technique achieves step-wise prompts in zero-shot scenarios while maintaining the prompt quality.

Copied to clipboard

Jiaqi Chen, Tong Li, Jinghui Qin, Pan Lu, Liang Lin, Chongyu Chen, Xiaodan Liang

MapGPT: Map-Guided Prompting with Adaptive Path Planning for Vision-and-Language Navigation (2024.acl-long)

Challenge:	Existing work on geometry problem solving treats calculation and proving as two specific tasks hindering a deep model to unify reasoning ability on multiple math tasks.
Approach:	They propose a large-scale Unified Geometry problem benchmark to unify geometry on multiple math tasks.
Outcome:	The proposed framework outperforms the existing model with 5.6% and 3.2% accuracies on calculation and proving problems.

Copied to clipboard

Jiaqi Chen, Bingqian Lin, Ran Xu, Zhenhua Chai, Xiaodan Liang, Kwan-Yee Wong

Neural-Symbolic Solver for Math Word Problems with Auxiliary Tasks (2021.acl-long)

Challenge:	Embodied agents equipped with GPT as their brains have extraordinary decision-making and generalization abilities across various tasks.
Approach:	They propose a map-based agent that introduces an online linguistic-formed map to encourage global exploration.
Outcome:	The proposed agent achieves state-of-the-art zero-shot performance on R2R and REVERIE simultaneously.

Copied to clipboard

Jinghui Qin, Xiaodan Liang, Yining Hong, Jianheng Tang, Liang Lin

RelCLIP: Adapting Language-Image Pretraining for Visual Relationship Detection via Relational Contrastive Learning (2022.emnlp-main)

Challenge:	Existing solutions for math word problems lack explicit integration of math symbolic constraints, leading to unexplainable and unreasonable predictions.
Approach:	They propose a novel mathematical model that explicitly incorporates symbolic constraints by auxiliary tasks to enforce different symbolic reasoning.
Outcome:	The proposed solver incorporates symbolic constraints by auxiliary tasks to enforce different symbolic reasoning.

Copied to clipboard

Yi Zhu, Zhaoqing Zhu, Bingqian Lin, Xiaodan Liang, Feng Zhao, Jianzhuang Liu

Don’t Take It Literally: An Edit-Invariant Sequence Loss for Text Generation (2022.naacl-main)

Challenge:	Existing visual relationship detection models only use numeric ids of relation labels for training, but ignore semantic correlation between labels.
Approach:	They propose a visual Relationship prediction framework that transfers natural language knowledge from Contrastive Language-Image Pre-training models to enhance the relationship prediction.
Outcome:	The proposed framework improves visual relationship prediction by matching semantic correlations with relation triplets.

Copied to clipboard

Guangyi Liu, Zichao Yang, Tianhua Tao, Xiaodan Liang, Junwei Bao, Zhen Li, Xiaodong He, Shuguang Cui, Zhiting Hu

Text Generation

Challenge:	Neural text generation models are typically trained by maximizing log-likelihood with the sequence cross entropy (CE) loss.
Approach:	They propose an Edit-Invariant Sequence Loss method which computes the matching loss of a target sequence with all n-grams in the generated sequence.
Outcome:	The proposed method outperforms the common CE loss and strong baselines on a wide range of tasks.

ATG: Benchmarking Automated Theorem Generation for Generative Language Models (2024.findings-naacl)

Copied to clipboard

Xiaohan Lin, Qingxing Cao, Yinya Huang, Zhicheng Yang, Zhengying Liu, Zhenguo Li, Xiaodan Liang

CorNav: Autonomous Agent with Self-Corrected Planning for Zero-Shot Vision-and-Language Navigation (2024.findings-acl)

Challenge:	Existing generative language models (LMs) can generate new or reusable theorems, but their ability to generate new theorels is under-explored.
Approach:	They propose to use Metamath library to generate new theorems that can be saved as reusable knowledge for future theoretical proving.
Outcome:	The proposed benchmark evaluates whether an agent can generate valuable (and possibly brand new) theorems that are applicable for downstream theoretic proving as reusable knowledge.

Copied to clipboard

Xiwen Liang, Liang Ma, Shanshan Guo, Jianhua Han, Hang Xu, Shikui Ma, Xiaodan Liang

DT-Solver: Automated Theorem Proving with Dynamic-Tree Sampling Guided by Proof-level Value Function (2023.acl-long)

Challenge:	Existing vision-and-language navigation methods do not incorporate environmental feedback into their decision-making processes.
Approach:	They propose a framework that incorporates environmental feedback into decision-making and a 3D simulator that renders realistic scenarios using Unreal Engine 5.
Outcome:	The proposed framework outperforms existing vision-and-language navigation methods in a zero-shot multi-task setting by 28.1% on average.

Copied to clipboard

Haiming Wang, Ye Yuan, Zhengying Liu, Jianhao Shen, Yichun Yin, Jing Xiong, Enze Xie, Han Shi, Yujun Li, Lin Li, Jian Yin, Zhenguo Li, Xiaodan Liang

Semantically-Aligned Universal Tree-Structured Solver for Math Word Problems (2020.emnlp-main)

Challenge:	Recent advances in neural theorem-proving resort to large language models and tree searches.
Approach:	They propose a Dynamic-Tree Driven Theorem Solver to accommodate general theoremes by guiding the search procedure with state confidence and proof-level values.
Outcome:	The proposed method outperforms state-of-the-art methods on two popular theorem-proving datasets with a 6.65% improvement on average in terms of success rate.

Copied to clipboard

Jinghui Qin, Lihui Lin, Xiaodan Liang, Rumin Zhang, Liang Lin

Numerical Reasoning

Challenge:	Existing models focus on one-unknown linear MWPs.
Approach:	They propose a universal expression tree-structured solver that integrates multiple expression trees underlying a MWP into a single expression tree.
Outcome:	The proposed method outperforms state-of-the-art models on a MWPs dataset and generates a universal expression tree explicitly by deciding which symbol to generate .

GRADE: Automatic Graph-Enhanced Coherence Metric for Evaluating Open-Domain Dialogue Systems (2020.emnlp-main)

Copied to clipboard

Lishan Huang, Zheng Ye, Jinghui Qin, Liang Lin, Xiaodan Liang

Natural Language Interfaces

Challenge:	Existing evaluation metrics only consider surface features or utterance-level semantics, without explicitly considering the fine-grained topic transition dynamics of dialogue flows.
Approach:	They propose a graph-enhanced evaluation metric GRADE to evaluate dialogue coherence . GRADE incorporates utterance-level contextualized representations and fine-grained topic-level graph representations to improve communication logic.
Outcome:	The proposed evaluation metric outperforms state-of-the-art metrics on measuring diverse dialogue models in terms of Pearson and Spearman correlations with human judgments.

Data-to-Text Generation with Style Imitation (2020.findings-emnlp)

Copied to clipboard

Shuai Lin, Wentao Wang, Zichao Yang, Xiaodan Liang, Frank F. Xu, Eric Xing, Zhiting Hu

Text Generation

Challenge:	Recent approaches to data-to-text generation focus on improving content fidelity, but lack explicit control over writing styles.
Approach:	They propose a way to control writing styles by using existing sentences as "soft" templates . they conduct experiments in restaurants and sports domains to test their approach .
Outcome:	The proposed approach achieves stronger performance than a range of comparison methods.

A Data-Centric Framework for Composable NLP Workflows (2020.emnlp-demos)

Copied to clipboard

Zhengzhong Liu, Guanxiong Ding, Avinash Bukkittu, Mansi Gupta, Pengzhi Gao, Atif Ahmed, Shikun Zhang, Xin Gao, Swapnil Singhavi, Linwei Li, Wei Wei, Zecong Hu, Haoran Shi, Xiaodan Liang, Teruko Mitamura, Eric Xing, Zhiting Hu

Challenge:	Empirical natural language processing (NLP) systems involve interoperation among multiple components . a wealth of NLP toolkits exist ( 4), such as spaCy, DKPro, CoreNLP.
Approach:	They propose a unified open-source framework that supports fast development of NLP workflows . framework includes processors for NLP tasks, visualization, and annotation .
Outcome:	The framework offers processors for NLP tasks, visualization, and annotation, and is extensible . it is delivered through two modularized yet integratable open-source projects, Forte and Stave .

Getting More Juice Out of Your Data: Hard Pair Refinement Enhances Visual-Language Models Without Extra Data (2025.naacl-long)

Copied to clipboard

Haonan Wang, Minbin Huang, Runhui Huang, Lanqing Hong, Hang Xu, Tianyang Hu, Xiaodan Liang, Zhenguo Li, Hong Cheng, Kenji Kawaguchi

EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation (2021.findings-emnlp)

Challenge:	Contrastive Language-Image Pre-training (CLIP) is a standard for cross-modal image-text representation learning.
Approach:	They propose a framework that enhances pre-trained CLIP models by exploiting challenging text-image pairs within existing datasets.
Outcome:	The proposed framework improves CLIP models by exploiting text-image pairs in training.

Copied to clipboard

Chenhe Dong, Guangrun Wang, Hang Xu, Jiefeng Peng, Xiaozhe Ren, Xiaodan Liang

ORMind: A Cognitive-Inspired End-to-End Reasoning Framework for Operations Research (2025.acl-industry)

Challenge:	Pre-trained language models have shown remarkable results on various NLP tasks.
Approach:	They propose to improve the feed-forward network (FFN) in BERT with a higher computational cost than improving the multi-head attention (MHA).
Outcome:	The proposed model is 6.9 smaller and 4.4 faster than BERTBASE and has competitive performances on GLUE and SQuAD Benchmarks.

Copied to clipboard

Zhiyuan Wang, Bokui Chen, Yinya Huang, Qingxing Cao, Ming He, Jianping Fan, Xiaodan Liang

LogicSolver: Towards Interpretable Math Word Problem Solving with Logical Prompt-enhanced Learning (2022.findings-emnlp)

Challenge:	Large Language Models (LLMs) have shown promising results in various domains, but their practical application in industry-relevant operations research presents significant challenges and opportunities.
Approach:	They propose a cognitive-inspired framework that enhances optimization through counterfactual reasoning . they use a workflow that transforms requirements into mathematical models and executable solver code .
Outcome:	Experiments show that ORMind outperforms existing methods in the NL4Opt dataset and ComplexOR dataset.

Copied to clipboard

Zhicheng Yang, Jinghui Qin, Jiaqi Chen, Liang Lin, Xiaodan Liang

Responsible & Trustworthy NLP

Challenge:	Recent advances in MWP solving are uninterpretable due to shallow heuristics . a new approach to solve automatic word problem solvers requires a solver to predict expression tree and corresponding linguistic logic formulas simultaneously.
Approach:	They propose to annotate interpretable logical formulas based on algebraic knowledge as the grounded linguistic logic of each solution equation.
Outcome:	The proposed approach improves interpretability of a MWP solver by using logical prompts and interpretation generation.

Wav-BERT: Cooperative Acoustic and Linguistic Representation Learning for Low-Resource Speech Recognition (2021.findings-emnlp)

Copied to clipboard

Guolin Zheng, Yubei Xiao, Ke Gong, Pan Zhou, Xiaodan Liang, Liang Lin