Papers by Cheng Wen

CORAL: Benchmarking Multi-turn Conversational Retrieval-Augmented Generation (2025.findings-naacl)

Challenge:	Recent research has focused on pushing weight-only quantization to extremely low-bit due to numerical representation limitations.
Approach:	They propose a vector-based quantization approach that pushes LLMs to extremely low-bit . they propose scalar-based weight quantization that reduces memory requirements and optimizes storage costs .
Outcome:	The proposed method reduces model quantization perplexity by 0.01-0.34 on LLaMA-2, 0.38-0.68 on mistral-7B, 4.41-7.34, on llaMA-3 on QA tasks on average.

Copied to clipboard

Yiruo Cheng, Kelong Mao, Ziliang Zhao, Guanting Dong, Hongjin Qian, Yongkang Wu, Tetsuya Sakai, Ji-Rong Wen, Zhicheng Dou

Natural Language Interfaces

Challenge:	Existing research focuses on single-turn RAG, leaving a gap in addressing multi-turn conversations . a new benchmark is designed to assess RAG systems in realistic multi-turned conversations based on Wikipedia .
Approach:	They propose a large-scale benchmark to assess RAG systems in multi-turn contexts . CORAL includes diverse information-seeking conversations automatically derived from Wikipedia . authors propose unified framework to standardize various conversational RAG methods .
Outcome:	The proposed framework supports three core tasks of conversational RAG: passage retrieval, response generation, and citation labeling.

SDiaReward: Modeling and Benchmarking Spoken Dialogue Rewards with Modality and Colloquialness (2026.acl-long)

Copied to clipboard

Jingyu Lu, Yuhan Wang, Fan Zhuo, Xize Cheng, Changhao Pan, Xueyi Pu, Yifu Chen, Chenyuhao Wen, Tianle Liang, Zhou Zhao

Speech & Audio in NLP

Challenge:	SDiaReward is an end-to-end spoken dialogue system that integrates paralinguistic nuances and spontaneous nature of human conversation.
Approach:	They propose an end-to-end multi-turn reward model trained on SDiaReward-Dataset . it is a collection of episode-level preference pairs targeting modality and colloquiality gaps .
Outcome:	The proposed model outperforms general-purpose audio LLMs in episode-level evaluation.

Formally Specifying the Intended Behavior of the Program: LLM-Driven Neuro-Symbolic Program Specification Synthesis (2026.acl-demo)

Copied to clipboard

Cheng Wen, Hu Junjie, YiKun Hu, Jie Su, Bin Yu, Dugang Liu, Zhiwu Xu, Weidi Sun, Shengchao Qin, Cong Tian

AlignBench: Benchmarking Chinese Alignment of Large Language Models (2024.acl-long)

Challenge:	Formal verification typically requires developers to write detailed formal specifications . a formal verification system that generates candidate specifications is costly and error-prone .
Approach:	They propose an LLM-driven neuro-symbolic demonstration system that reframes specification writing as constrained structured synthesis.
Outcome:	The proposed system reduces hallucinations and produces proof-ready annotations.

Copied to clipboard

Xiao Liu, Xuanyu Lei, Shengyuan Wang, Yue Huang, Andrew Feng, Bosi Wen, Jiale Cheng, Pei Ke, Yifan Xu, Weng Lam Tam, Xiaohan Zhang, Lichao Sun, Xiaotao Gu, Hongning Wang, Jing Zhang, Minlie Huang, Yuxiao Dong, Jie Tang

Evaluating Implicit Bias in Large Language Models by Attacking From a Psychometric Perspective (2025.findings-acl)

Challenge:	Effective evaluation of alignment for emerging Chinese LLMs is still significantly lacking, calling for real-scenario grounded, open-ended, challenging and automatic evaluations tailored for alignment.
Approach:	They propose a multi-dimensional benchmark for evaluating LLMs’ alignment in Chinese with 8 main categories, 683 real-scenario rooted queries and corresponding human verified references.
Outcome:	The benchmark uses a human-in-the-loop data curation pipeline, 683 real-scenario rooted queries and human verified references.

Copied to clipboard

Yuchen Wen, Keping Bi, Wei Chen, Jiafeng Guo, Xueqi Cheng

CRSLab: An Open-Source Toolkit for Building Conversational Recommender System (2021.acl-demo)

Challenge:	Existing studies have shown that large language models (LLMs) can elicit implicit biases that hurt certain demographics without explicit harmful words.
Approach:	They propose three attack approaches to elicit agreements to biased viewpoints from LLMs from a psychometric perspective and built two benchmarks to compare them.
Outcome:	The proposed methods elicit agreements to biased viewpoints more effectively than baselines.

Copied to clipboard

Kun Zhou, Xiaolei Wang, Yuanhang Zhou, Chenzhan Shang, Yuan Cheng, Wayne Xin Zhao, Yaliang Li, Ji-Rong Wen

Natural Language Interfaces

Challenge:	Existing studies on conversational recommender systems lack a unified and standardized implementation or comparison.
Approach:	They propose to use a unified framework and highly-decoupled modules to develop CRSs.
Outcome:	The proposed framework collects 6 commonly used human-annotated CRS datasets and implements 19 models that include advanced techniques such as graph neural networks and pre-training models.

Logic-Thinker: Teaching Large Language Models to Think more Logically. (2025.findings-emnlp)

Copied to clipboard

Chengyao Wen, Qiang Cheng, Shaofei Wang, Zhizhen Liu, Deng Zhao, Lei Liang

Training Language Model to Critique for Better Refinement (2025.findings-acl)

Challenge:	Recent Large Reasoning Models (LRMs) have demonstrated the ability to generate long chains of thought (LongCoT) LongCoT still faces challenges such as redundancy and logical incoherence.
Approach:	They propose a neural-symbolic reasoning framework that generates chains of thought . they propose Logic-Thinker, which transforms symbolic solvers into chains of thoughts .
Outcome:	The proposed framework outperforms models fine-tuned with ThinkerCoT on logic reasoning tasks.

Copied to clipboard

Tianshu Yu, Chao Xiang, Mingchuan Yang, Pei Ke, Bosi Wen, Cunxiang Wang, Jiale Cheng, Li Zhang, Xinyu Mu, Chuxiong Sun, Minlie Huang

Original Content Is All You Need! an Empirical Study on Leveraging Answer Summary for WikiHowQA Answer Selection Task (2022.coling-1)

Challenge:	Large language models (LLMs) have remarkable evaluation and critique capabilities, providing insightful feedback and identifying flaws in various tasks.
Approach:	They propose a framework to train critic models using refinement signals to generate feedback loops where critiques guide the model in refining its responses.
Outcome:	The proposed framework outperforms traditional methods and open-source models in terms of critique quality and refinement outcomes.

Copied to clipboard

Liang Wen, Juan Li, Houfeng Wang, Yingwei Luo, Xiaolin Wang, Xiaodong Zhang, Zhicong Cheng, Dawei Yin

DR-HM: Distill-then-Reinforce Training with Cognition-Aware Data Synthesis for Harmful Meme Detection (2026.findings-acl)

Challenge:	Existing answer selection approaches for community question answering lack additional answer summaries due to redundancy and lengthiness issues of crowdsourced answers.
Approach:	They constructed a dataset which contains a corresponding reference summary for each original lengthy answer.
Outcome:	The proposed model improves the performance of a question and candidate answer on a WikiHowQA dataset.

Copied to clipboard

Zihan Cheng, Jianxiang Ma, Xiaocui Yang, Peidong Wang, Wen Zhang, Shi Feng, Daling Wang, Yifei Zhang, Mingfu Zhang

Responsible & Trustworthy NLP

Challenge:	Current methods for harmful meme detection lack the knowledge required to identify such hate . current methods lack the ability to identify cultural stereotypes and visual metaphors .
Approach:	They propose a framework that decomposes meme analysis into a human-inspired reasoning process . they propose DR-HM to transfer knowledge from closed-source models while mitigating biases .
Outcome:	The proposed framework outperforms existing methods on three benchmark datasets.

OmniFlatten: An End-to-end GPT Model for Seamless Voice Conversation (2025.acl-long)

Copied to clipboard

Qinglin Zhang, Luyao Cheng, Chong Deng, Qian Chen, Wen Wang, Siqi Zheng, Jiaqing Liu, Hai Yu, Chao-Hong Tan, Zhihao Du, ShiLiang Zhang

Dialogue Systems & Conversational Agents

Challenge:	Full-duplex spoken dialogue systems allow simultaneous bidirectional communication . low latency and natural interactions in full-duplice systems remains a challenge .
Approach:	They propose a multi-stage post-training scheme that adapts a text large language model into a speech-text dialogue LLM.
Outcome:	The proposed model can model human conversation behaviors with low latency and natural interactions with low delay.

Learning Intrinsic Dimension via Information Bottleneck for Explainable Aspect-based Sentiment Analysis (2024.lrec-main)

Copied to clipboard

Zhenxiao Cheng, Jie Zhou, Wen Wu, Qin Chen, Liang He

The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Models (2024.acl-long)

Challenge:	Gradient-based explanation methods are increasingly used to interpret neural models in natural language processing (NLP) however, in the context of Aspect-based Sentiment Analysis, only specific dimensions are pertinent.
Approach:	They propose a Gradient-based explanation framework that leverages an information bottleneck to refine word embeddings into a concise intrinsic dimension, maintaining essential features and omitting unrelated information.
Outcome:	The proposed framework improves both the models’ performance and explanations’ clarity by identifying sentiment-aware features.

Copied to clipboard

Junyi Li, Jie Chen, Ruiyang Ren, Xiaoxue Cheng, Xin Zhao, Jian-Yun Nie, Ji-Rong Wen

Integrating Audio, Visual, and Semantic Information for Enhanced Multimodal Speaker Diarization on Multi-party Conversation (2025.acl-long)

Challenge:	a growing number of researchers are studying the hallucination issue in large language models.
Approach:	They propose a hallucination detection benchmark and a method to detect hallucines in LLMs.
Outcome:	The proposed method detects hallucinations and mitigates them using different training stages.

Copied to clipboard

Luyao Cheng, Hui Wang, Chong Deng, Siqi Zheng, Yafeng Chen, Rongjie Huang, Qinglin Zhang, Qian Chen, Xihao Li, Wen Wang

ControlSpeech: Towards Simultaneous and Independent Zero-shot Speaker Cloning and Zero-shot Language Style Control (2025.acl-long)

Challenge:	Mainstream speaker diarization systems rely only on acoustic information, making it challenging in complex aural environments.
Approach:	They propose a multimodal approach that integrates audio, visual, and semantic cues to enhance speaker diarization.
Outcome:	The proposed approach outperforms state-of-the-art methods on multi-party conversations . it integrates audio-visual-semantic cues into the clustering process for acoustic speaker embeddings .

Copied to clipboard

Shengpeng Ji, Qian Chen, Wen Wang, Jialong Zuo, Minghui Fang, Ziyue Jiang, Hai Huang, Zehan Wang, Xize Cheng, Siqi Zheng, Zhou Zhao

Robustness Testing of Language Understanding in Task-Oriented Dialog (2021.acl-long)

Challenge:	Prior zero-shot TTS models only mimic the speaker’s voice without further control and adjustment capabilities while prior controllable TTS systems cannot perform speaker-specific voice generation.
Approach:	They propose a style control module that captures codec representations corresponding to timbre, content, and style in a discrete decoupling codec space.
Outcome:	The proposed system can fully clone the speaker's voice and perform speech-specific adjustment and control functions.

Copied to clipboard

Jiexi Liu, Ryuichi Takanobu, Jiaxin Wen, Dazhen Wan, Hongguang Li, Weiran Nie, Cheng Li, Wei Peng, Minlie Huang

Robustness in NLP

Challenge:	a lack of systematic studies on the robustness of language understanding models in task-oriented dialog systems is limiting . authors propose a model-agnostic toolkit LAUG to approximate natural language perturbations .
Approach:	They propose a model-agnostic toolkit LAUG to approximate natural language perturbations for testing the robustness of language understanding models in task-oriented dialog systems.
Outcome:	The proposed toolkit reveals critical robustness issues in state-of-the-art models.

Visual Named Entity Linking: A New Dataset and A Baseline (2022.findings-emnlp)

Copied to clipboard

Wen Sun, Yixing Fan, Jiafeng Guo, Ruqing Zhang, Xueqi Cheng

Visual Data in NLP

Challenge:	Existing tasks in Visual Entity Linking (VEL) rely on textual data to complement multi-modal linking or only link objects with general entities.
Approach:	They propose a task to link regions of images with corresponding entities in Knowledge Bases . they propose three sub-tasks, based on a human-annotated visual person dataset .
Outcome:	The proposed task is based on a human-annotated visual person linking dataset . the proposed sub-tasks are validated on the WIKIPerson dataset based upon the proposed methods .

HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models (2023.emnlp-main)

Copied to clipboard

Junyi Li, Xiaoxue Cheng, Xin Zhao, Jian-Yun Nie, Ji-Rong Wen

Natural Language Interfaces

Challenge:	Existing large language models (LLMs) are prone to generate hallucinations . a recent study shows that LLMs are able to generate content that conflicts with the source or cannot be verified by factual knowledge.
Approach:	They propose a framework to evaluate the performance of large language models (LLMs) they propose to use a sample of generated and human-annotated hallucinated samples to evaluate their performance .
Outcome:	The proposed framework generates and annotates hallucinated samples from ChatGPT . the results show that existing LLMs face great challenges in recognizing hallucines .

CritiqueLLM: Towards an Informative Critique Generation Model for Evaluation of Large Language Model Generation (2024.acl-long)

Copied to clipboard

Pei Ke, Bosi Wen, Andrew Feng, Xiao Liu, Xuanyu Lei, Jiale Cheng, Shengyuan Wang, Aohan Zeng, Yuxiao Dong, Hongning Wang, Jie Tang, Minlie Huang

Cross-Utterance Conditioned VAE for Non-Autoregressive Text-to-Speech (2022.acl-long)

Challenge:	Existing models for NLP evaluations lack the ability to generate informative critiques in pointwise grading and pairwise comparison especially without references.
Approach:	They propose a method which can acquire pointwise grading critiques with pseudo references and revise these critiques via multi-path prompting to obtain informative evaluation data in different tasks and settings.
Outcome:	The proposed method outperforms all open-source models and even GPT-4 in system-level correlations of pointwise grading.

Copied to clipboard

Yang Li, Cheng Yu, Guangzhi Sun, Hua Jiang, Fanglei Sun, Weiqin Zu, Ying Wen, Yang Yang, Jun Wang

LLMBox: A Comprehensive Library for Large Language Models (2024.acl-demos)

Challenge:	Experimental results show that the proposed model improves naturalness and prosody diversity with clear margins.
Approach:	They propose a cross-utterance conditional VAE to estimate posterior probability distribution of latent prosody features for each phoneme by conditioning on acoustic features, speaker information, and text features from past and future sentences.
Outcome:	The proposed model improves naturalness and prosody diversity with clear margins.

Copied to clipboard

Tianyi Tang, Hu Yiwen, Bingqian Li, Wenyang Luo, ZiJing Qin, Haoxiang Sun, Jiapeng Wang, Shiyi Xu, Xiaoxue Cheng, Geyang Guo, Han Peng, Bowen Zheng, Yiru Tang, Yingqian Min, Yushuo Chen, Jie Chen, Ranchi Zhao, Luran Ding, Yuhao Wang, Zican Dong, Xia Chunxuan, Junyi Li, Kun Zhou, Xin Zhao, Ji-Rong Wen

From Informal to Formal – Incorporating and Evaluating LLMs on Natural Language Requirements to Verifiable Formal Proofs (2025.acl-long)

Challenge:	a library to facilitate the development, use, and evaluation of large language models (LLMs) is presented.
Approach:	They propose a unified library to facilitate the development, use and evaluation of large language models (LLMs).
Outcome:	The proposed library is based on extensive experiments in a variety of evaluation settings.

Copied to clipboard

Jialun Cao, Yaojie Lu, Meiziniu Li, Haoyang Ma, Haokun Li, Mengda He, Cheng Wen, Le Sun, Hongyu Zhang, Shengchao Qin, Shing-Chi Cheung, Cong Tian

Reasoning

Challenge:	Recent studies in formal mathematical reasoning have shown an unstoppable growth trend.
Approach:	They constructed 18k high-quality instruction-response pairs across five mainstream formal specification languages and evaluated them against ten open-sourced LLMs.
Outcome:	The proposed model compared instruction-response pairs across five formal specification languages and found that the LLMs were good at writing proof segments when given either the code, or the detailed description of proof steps.

ChainLM: Empowering Large Language Models with Improved Chain-of-Thought Prompting (2024.lrec-main)

Copied to clipboard

Xiaoxue Cheng, Junyi Li, Wayne Xin Zhao, Ji-Rong Wen

Bridging Kernel Drivers and Virtual Device Models with LLM-Powered Automation (2026.acl-demo)

Challenge:	Existing CoT synthesis approaches focus on simpler reasoning tasks and result in inconsistent CoT prompts.
Approach:	They propose a framework for automatic generation of superior CoT prompts based on three major evolution strategies . they propose 'step-level debating' method where multiple debaters discuss each reasoning step to arrive at the correct answer.
Outcome:	The proposed framework can generate superior CoT prompts from a CoT dataset.

Copied to clipboard

Mingyu Wang, Bin Yu, Wenjian Lu, Zhi Wang, Gao Kefeng, Cheng Wen, Xu Lu, Cong Tian

Think More, Hallucinate Less: Mitigating Hallucinations via Dual Process of Fast and Slow Thinking (2025.findings-acl)

Challenge:	Linux kernel device drivers are tightly coupled with hardware, making them difficult to execute and test without physical devices.
Approach:	They present a tool that generates QEMU-based virtual devices directly from Linux driver source code.
Outcome:	The proposed tool generates QEMU-based virtual devices directly from Linux driver source code.

Copied to clipboard

Xiaoxue Cheng, Junyi Li, Xin Zhao, Ji-Rong Wen

Reasoning

Challenge:	Existing studies have focused on the issue of hallucination in large language models.
Approach:	They propose a framework that allows an explicit slow thinking generation process for mitigating hallucinations during inference.
Outcome:	The proposed framework outperforms baseline approaches on English and Chinese datasets.

LogToP: Logic Tree-of-Program with Table Instruction-tuned LLMs for Controlled Logical Table-to-Text Generation (2026.findings-eacl)

Copied to clipboard

Yupian Lin, Guangya Yu, Cheng Yuan, Huan Du, Hui Luo, Yuang Bian, Jingping Liu, Zhidong He, Wen Du, Tong Ruan

Small Agent Can Also Rock! Empowering Small Language Models as Hallucination Detector (2024.emnlp-main)

Challenge:	Existing LLMs are difficult to achieve satisfactory results in table-related tasks.
Approach:	They propose to develop a specialized logical table-to-text generation model that can be used for table-related tasks.
Outcome:	The proposed model achieves state-of-the-art on a Logic2Text dataset.

Copied to clipboard

Xiaoxue Cheng, Junyi Li, Xin Zhao, Hongzhi Zhang, Fuzheng Zhang, Di Zhang, Kun Gai, Ji-Rong Wen