Papers by Ke Zhou

Static Models, Dynamic World: A Unified Perspective on Temporal Perception in Large Language Models (2026.findings-acl)

Challenge:	Pre-trained language models (PLMs) have succeeded in natural language processing because they learn generic knowledge from a large corpus.
Approach:	They propose a method that allows pre-trained language models to explore simile knowledge from PLMs . they enhance PLM models with a multi-level simile recognition task that evaluates similes aplenty .
Outcome:	The proposed method can explore more accurate simile knowledge for PLMs.

Copied to clipboard

Chenhao Li, Dandan Song, Changzhi Zhou, Jun Yang, Yuhang Tian, Huipeng Ma, Guangyuan Feng, Luan Zhang, Xudong Li, Ke Duan

MathGenie: Generating Synthetic Data with Question Back-translation for Enhancing Mathematical Reasoning of LLMs (2024.acl-long)

Challenge:	Large language models are trained on static corpora but deployed in a dynamic world . a foundational tension remains between time and the ability to understand it .
Approach:	They formalize temporal queries in an information-theoretic framework based on parametric reachability of temporal premises and answers.
Outcome:	The proposed framework formalizes temporal queries in an information-theoretic framework based on parametric reachability of temporal premises and answers . the framework induces four temporal information regimes corresponding to internal reasoning, answer recency, premise anchoring, and genuine world indeterminacy .

Copied to clipboard

Zimu Lu, Aojun Zhou, Houxing Ren, Ke Wang, Weikang Shi, Junting Pan, Mingjie Zhan, Hongsheng Li

Reasoning

Challenge:	Existing models have demonstrated outstanding capabilities in mathematical reasoning, but there is a performance gap between open-source models and closed-source ones.
Approach:	They propose a method for generating diverse and reliable math problems by leveraging the ground-truth solutions of the seed data.
Outcome:	The proposed model outperforms open-source models across five representative mathematical reasoning datasets.

Contextual Knowledge Learning for Dialogue Generation (2023.acl-long)

Copied to clipboard

Wen Zheng, Natasa Milic-Frayling, Ke Zhou

Don’t Half-listen: Capturing Key-part Information in Continual Instruction Tuning (2025.acl-long)

Challenge:	Incorporating conversational context and knowledge into dialogue generation models has been essential for improving the quality of the generated responses.
Approach:	They propose a method to incorporate conversational context and knowledge into dialogue generation models . they use Latent Vectors to capture the relationship between context and knowing .
Outcome:	The proposed approach improves performance with two standard datasets and human evaluations.

Copied to clipboard

Yongquan He, Wenyuan Zhang, Xuancheng Huang, Peng Zhang, Lingxun Meng, Xiang Zhou, Ke Zeng, Xunliang Cai

PopAlign: Diversifying Contrasting Patterns for a More Comprehensive Alignment (2025.acl-long)

Challenge:	Existing methods to improve instruction tuning for large language models may cause catastrophic forgetting (CF) CF is a problem where previously learned abilities are degraded .
Approach:	They propose a continual instruction tuning method that uses key-part information gain to replay data and refine training objective.
Outcome:	The proposed method achieves superior performance on both seen and held-out tasks.

Copied to clipboard

Zekun Moore Wang, Shenzhi Wang, King Zhu, Jiaheng Liu, Ke Xu, Jie Fu, Wangchunshu Zhou, Wenhao Huang

Challenge:	Typical approaches to training large language models rely on limited contrasting patterns . contrasting data is limited and models are susceptible to harmful response tendencies .
Approach:	They propose a framework that integrates contrasting patterns across the prompt, model, and pipeline levels.
Outcome:	The proposed framework outperforms existing methods in the comparison of RQ1 and RQ2 . the proposed framework significantly outperformed existing methods, leading to more comprehensive alignment.

CoSQA: 20,000+ Web Queries for Code Search and Question Answering (2021.acl-long)

Copied to clipboard

Junjie Huang, Duyu Tang, Linjun Shou, Ming Gong, Ke Xu, Daxin Jiang, Ming Zhou, Nan Duan

Information Retrieval

Challenge:	Using deep neural networks to find codes is difficult . we present a dataset that includes 20,604 labels for natural language queries and codes .
Approach:	They introduce a contrastive learning method to enhance text-code matching . they find that CoSQA improves the accuracy of code question answering by 5.1% .
Outcome:	The proposed method improves the accuracy of code question answering by 5.1% and improves by 10.5% on a CodeBERT model.

PositionID: LLMs can Control Lengths, Copy and Paste with Explicit Positional Awareness (2024.findings-emnlp)

Copied to clipboard

Noah Wang, Feiyu Duan, Yibo Zhang, Wangchunshu Zhou, Ke Xu, Wenhao Huang, Jie Fu

MathCoder-VL: Bridging Vision and Code for Enhanced Multimodal Mathematical Reasoning (2025.findings-acl)

Challenge:	Large Language Models (LLMs) have impressive capabilities across various domains, including role-playing, creative writing, mathematical reasoning, and coding.
Approach:	They propose two methods to improve the model’s adherence to length constraints and copy-paste accuracy without compromising response quality.
Outcome:	The proposed methods improve the model’s adherence to length constraints and copy-paste accuracy without compromising response quality.

Copied to clipboard

Ke Wang, Junting Pan, Linda Wei, Aojun Zhou, Weikang Shi, Zimu Lu, Han Xiao, Yunqiao Yang, Houxing Ren, Mingjie Zhan, Hongsheng Li

Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compression (2021.emnlp-main)

Challenge:	Large Language Models (LMMs) struggle with simple tasks such as geometry, e.g., arithmetic, and reasoning.
Approach:	They propose to leverage code as supervision for cross-modal alignment . they propose to use FigCodifier and ImgCode-8.6M to synthesize novel mathematical figures .
Outcome:	The proposed model surpasses GPT-4o and Claude 3.5 Sonnet in the geometry problem-solving subset of MathVista, achieving improvements of 8.9% and 9.2%.

Copied to clipboard

Canwen Xu, Wangchunshu Zhou, Tao Ge, Ke Xu, Julian McAuley, Furu Wei

FPE2M2: Approaching Lossless and Efficient Quantization with Native Floating Point (2025.findings-acl)

Challenge:	Recent studies on compression of pretrained language models usually use preserved accuracy as the metric for evaluation.
Approach:	They propose two new metrics that measure how closely a compressed model mimics the original model.
Outcome:	The proposed metrics measure how closely a compressed model (i.e., student) mimics the original model (e.g., teacher).

Copied to clipboard

Ke Yi, Jianwei Zhang, Zhiying Xu, Xinlong Yang, Yang Zhou, Minmin Sun, Zengke Liu, Tong Zhang, Junyang Lin, Jingren Zhou

Green & Sustainable NLP

Challenge:	Auto-regressive decoding is a memory-bound job, meaning decoding performance is limited by the bandwidth rather than the computational capabilities of the GPU.
Approach:	They propose a framework that supports lossless weight-only quantization inference and validate it on Qwen and LLaMA Models.
Outcome:	The proposed framework achieves the highest efficiency with lossless accuracy on Qwen and LLaMA Models across various modalities.

Blow the Dog Whistle: A Chinese Dataset for Cant Understanding with Common Sense and World Knowledge (2021.naacl-main)

Copied to clipboard

Canwen Xu, Wangchunshu Zhou, Tao Ge, Ke Xu, Julian McAuley, Furu Wei

Mixture-of-Minds: Multi-Agent Reinforcement Learning for Table Understanding (2026.acl-long)

Challenge:	Cant is important for understanding advertising, comedies and dogwhistle politics . currently, there are very few resources available for the research of cant .
Approach:	They propose a large and diverse dataset for creating and understanding cant from a computational linguistics perspective.
Outcome:	The proposed dataset can be used to test word embedding similarity and pretrained language models.

Copied to clipboard

Yuhang Zhou, Mingrui Zhang, Ke Li, Mingyi Wang, Qiao Liu, Qifei Wang, Jiayi Liu, Fei Liu, Serena Li, Weiwei LI, Mingze Gao, Abhishek Kumar, Xiangjun Fan, Zhuokai Zhao, Lizhu Zhang

Walking in Others’ Shoes: How Perspective-Taking Guides Large Language Models in Reducing Toxicity and Bias (2024.emnlp-main)

Challenge:	Large language models (LLMs) have shown promise on understanding and reasoning over tables, but current approaches remain limited.
Approach:	They propose a multi-agent framework that decomposes table reasoning into three specialized roles: planning, coding, and answering.
Outcome:	The proposed framework decomposes table reasoning into three specialized roles: planning, coding, and answering.

Copied to clipboard

Rongwu Xu, Zian Zhou, Tianwei Zhang, Zehan Qi, Su Yao, Ke Xu, Wei Xu, Han Qiu

CTRLEval: An Unsupervised Reference-Free Metric for Evaluating Controlled Text Generation (2022.acl-long)

Challenge:	Existing prompting methods that require white-box access to the model or substantial training fail to simultaneously lessen toxicity and bias.
Approach:	They propose a strategy that encourages LLMs to integrate diverse human perspectives and self-regulate their responses by incorporating diverse human viewpoints.
Outcome:	The proposed approach can significantly diminish toxicity (up to 89%) and bias (up 73%) in LLMs’ responses.

Copied to clipboard

Pei Ke, Hao Zhou, Yankai Lin, Peng Li, Jie Zhou, Xiaoyan Zhu, Minlie Huang

Low-Resource NLP

Challenge:	Existing reference-free metrics have obvious limitations for evaluating controlled text generation models.
Approach:	They propose an unsupervised reference-free metric which evaluates controlled text generation from different aspects by formulating each aspect into multiple text infilling tasks.
Outcome:	The proposed metric has higher correlations with human judgments while obtaining better generalization of evaluating generated texts from different models and with different qualities.

From Implicit Graph Encoding to Explicit Evidence: A Training-Free LLM Framework for Temporal Knowledge Graph Reasoning (2026.findings-acl)

Copied to clipboard

Guo Tang, Ke Cheng, Huiming Fan, Heng Chang, Wenxiang Zheng, Xianhao Ou, Junjia Xiang, Ming Liu, Yujun Zhou, Li Lanyu, Bing Qin

DIFFA-2: A Practical Diffusion Large Language Model for General Audio Understanding (2026.findings-acl)

Challenge:	Existing Large Language Models (LLMs) struggle with implicit modality alignment and suboptimal graph linearization.
Approach:	They propose a training-free, test-time adaptive framework that reframes TKG prediction as explicit evidence-driven reasoning.
Outcome:	ExE-LLM outperforms fully trained graph neural networks on four benchmarks . it achieves SOTA performance in inductive settings, significantly outperforming fully trained neural networks .

Copied to clipboard

Jiaming Zhou, Xuxin Cheng, Shiwan Zhao, Yuhang Jia, Cao Liu, Ke Zeng, Xunliang Cai, Yong Qin

PACE: Prefix-Protected and Difficulty-Aware Compression for Efficient Reasoning (2026.findings-acl)

Challenge:	Autoregressive (AR) large audio language models are expensive in data and computation . prior work shows diffusion-based LALMs can improve audio understanding under matched settings .
Approach:	They propose a diffusion-based LALM that upgrades the speech encoder and employs dual semantic and acoustic adapters.
Outcome:	a new model improves over existing autoregressive large language models and is competitive to strong AR models . the proposed model can make use of limited training data and improve inference efficiency . a recent study shows that diffusion-based models can improve audio understanding .

Copied to clipboard

Ruixiang Feng, Yuntao Wen, Silin Zhou, Ke Shi, Yifan Wang, Ran Le, Zhenwei An, Zongchao Chen, Chen Yang, Guangyue Peng, Yiming Jia, Dongsheng Wang, Tao Zhang, Lisi Chen, Yang Song, Shen Gao, Shuo Shang

Responsible & Trustworthy NLP

Challenge:	Existing LRMs often suffer from "overthinking" and excessively long reasoning traces . a dual-level framework for length compression of LRM is proposed .
Approach:	They propose a framework for prefix-protected and difficulty-aware compression under hierarchical supervision.
Outcome:	The proposed framework reduces token usage while improving accuracy on math benchmarks.

RAG-Instruct: Boosting LLMs with Diverse Retrieval-Augmented Instructions (2025.emnlp-main)

Copied to clipboard

Wanlong Liu, Junying Chen, Ke Ji, Li Zhou, Wenyu Chen, Benyou Wang

Information Retrieval

Challenge:	Retrieval-Augmented Generation (RAG) has emerged as a key paradigm for enhancing large language models by incorporating external knowledge.
Approach:	They propose a method for synthesizing diverse and high-quality RAG instruction data based on any source corpus.
Outcome:	The proposed method outperforms existing methods in multiple tasks and achieves strong zero-shot performance.

Probability-Consistent Preference Optimization for Enhanced LLM Reasoning (2025.findings-acl)

Copied to clipboard

Yunqiao Yang, Houxing Ren, Zimu Lu, Ke Wang, Weikang Shi, Aojun Zhou, Junting Pan, Mingjie Zhan, Hongsheng Li

Reasoning

Challenge:	Recent advances in preference optimization have demonstrated significant potential for improving mathematical reasoning capabilities in large language models.
Approach:	They propose a framework that establishes two quantitative metrics for preference selection: surface-level answer correctness and intrinsic token-level probability consistency.
Outcome:	The proposed framework outperforms existing outcome-only criterion approaches across a diverse range of LLMs and benchmarks.

Structure-Aware Zero-Shot Relational Learning for Knowledge Graphs without External Knowledge (2026.findings-acl)

Copied to clipboard

Kuan Xu, Baoxin Zhang, Shuyue Fan, Ming Chen, Zhipeng Ke, Jian Yu, Xuezhong Zhou

MathCanvas: Intrinsic Visual Chain-of-Thought for Multimodal Mathematical Reasoning (2026.acl-long)

Challenge:	Existing methods for Zero-shot Relational Learning depend on external knowledge, resulting in increased annotation costs and limited practical applicability.
Approach:	They propose a structure-aware paradigm that performs ZRL without external knowledge . it leverages intrinsic structural patterns in KGs to bridge semantic correlations for new relations with existing ones.
Outcome:	The proposed paradigm achieves 10.66% improvement in MRR while reducing annotation costs and enhancing practical applicability on three real-world benchmarks.

Copied to clipboard

Weikang Shi, Aldrich Yu, Rongyao Fang, Houxing Ren, Ke Wang, Aojun Zhou, Changyao Tian, Xinyu Fu, Yuxuan Hu, Zimu Lu, Linjiang Huang, Si Liu, Rui Liu, Hongsheng Li

Comprehensive Benchmarking of Long-Form Speech Generation in Diverse Scenarios (2026.findings-acl)

Challenge:	Existing approaches to visual chain-of-thought are limited by external tools or fail to generate high-fidelity diagrams.
Approach:	They propose a framework to enable large multimodal models with VCoT capabilities . they pre-train a model on a 15.2M-pair corpus and teach it how to leverage visual aids .
Outcome:	The proposed framework unlocks complex, human-like visual reasoning in large language models . it pre-trains the model on a 15.2M-pair corpus and fine-tunes it on MathCanvas-Instruct .

Copied to clipboard

Changhao Pan, Rui Yang, Han Wang, Zhuan Zhou, Xuming He, Wenxiang Guo, Ziyue Jiang, Ruiqi Li, Yu Zhang, Chenyuhao Wen, Ke Lei, Xiang Yin, Jingyu Lu, Zhiyuan Zhu, Zhou Zhao

T2A-Feedback: Improving Basic Capabilities of Text-to-Audio Generation via Fine-grained AI Feedback (2025.acl-long)

Challenge:	Existing evaluation benchmarks for long-form speech are limited to limited domains, creating a significant gap with the diverse downstream applications.
Approach:	They propose a benchmark that decomposes "long-form speech quality" into specific, disentangled dimensions.
Outcome:	The proposed benchmark decomposes “long-form speech quality” into specific, disentangled dimensions.

Copied to clipboard

Zehan Wang, Ke Lei, Chen Zhu, Jiawei Huang, Sashuai Zhou, Luping Liu, Xize Cheng, Shengpeng Ji, Zhenhui Ye, Tao Jin, Zhou Zhao

Sinkhorn Distance Minimization for Knowledge Distillation (2024.lrec-main)

Challenge:	Text-to-audio (T2A) models still struggle to satisfy human preferences for prompt-following and acoustic quality when generating complex multi-event audio.
Approach:	They propose to use AI feedback learning to enhance basic capabilities of text-to-audio models . they use a large audio preference dataset to evaluate the model's capabilities .
Outcome:	The proposed model improves in simple and complex scenarios with AI feedback learning.

Copied to clipboard

Xiao Cui, Yulei Qin, Yuting Gao, Enwei Zhang, Zihan Xu, Tong Wu, Ke Li, Xing Sun, Wengang Zhou, Houqiang Li

Model-Based Simulation for Optimising Smart Reply (2023.acl-long)

Challenge:	Existing knowledge distillation methods investigate divergence measures but fail to deliver effective supervision when few distribution overlap exists between teacher and student.
Approach:	They propose a knowledge distillation method that exploits the Sinkhorn distance to ensure a nuanced assessment of the disparity between teacher and student distributions.
Outcome:	The proposed method outperforms state-of-the-art methods on all kinds of LLMs with encoder-only, encoder decoder, and decoded architectures.

Copied to clipboard

Benjamin Towle, Ke Zhou

Challenge:	Existing methods to learn to predict responses to messages are based on post-hoc diversification rather than learning to predict sets of responses.
Approach:	They propose a method that employs model-based simulation to discover high-value response sets by simulating possible user responses with a learned world model.
Outcome:	Empirically, the proposed method improves ROUGE score and Self-ROUGE scores on two public datasets compared to baselines.

Improving Grammatical Error Correction with Machine Translation Pairs (2020.findings-emnlp)

Copied to clipboard

Wangchunshu Zhou, Tao Ge, Chang Mu, Ke Xu, Furu Wei, Ming Zhou

Machine Translation

Challenge:	Existing methods to generate error-corrected sentence pairs for improving grammatical error correction are not available.
Approach:	They propose a method to generate error-corrected sentence pairs for improving grammatical error correction based on machine translation models of different qualities .
Outcome:	The proposed method can generate multiple error-corrected sentence pairs from Chinese to English text.

Capability Decomposition for Unified Information Extraction via Hierarchical Mixture-of-Experts (2026.acl-long)

Copied to clipboard

Jing Zhou, Peng Wang, Wenjun Ke, Jiajun Liu, Yao He

Responsible & Trustworthy NLP

Challenge:	Existing methods for IE tasks suffer from inconsistent schema representation and implicitly intermediate reasoning . UC-UIE adopts a low-rank adapted hierarchical Mixture-of-Experts adapter for UIE tasks .
Approach:	They propose a framework that decomposes IE reasoning into three universal capabilities . UC-UIE adopts a low-rank Adaptation adapter to fine-tune LLMs for IE tasks .
Outcome:	The proposed framework outperforms full-parameter tuning methods with 1.24% trainable parameters and outperformed existing methods in generalization and interpretability.

LM-Searcher: Cross-domain Neural Architecture Search with LLMs via Unified Numerical Encoding (2025.emnlp-main)

Copied to clipboard

Yuxuan Hu, Jihao Liu, Ke Wang, Jinliang Zheng, Weikang Shi, Manyuan Zhang, Qi Dou, Rui Liu, Aojun Zhou, Hongsheng Li

Approximation of Response Knowledge Retrieval in Knowledge-grounded Dialogue Generation (2020.findings-emnlp)

Challenge:	Recent advances in Large Language Models have opened new avenues for solving complex optimization problems, including Neural Architecture Search (NAS).
Approach:	They propose a framework that leverages LLMs for cross-domain neural architecture optimization without extensive domain-specific tuning.
Outcome:	The proposed framework achieves competitive performance in both in-domain and out-of-domain tasks.

Copied to clipboard

Wen Zheng, Natasa Milic-Frayling, Ke Zhou

DeKeyNLU: Enhancing Natural Language to SQL Generation through Task Decomposition and Keyword Extraction (2025.findings-emnlp)

Challenge:	Recent studies have focused on improving dialogue generation models that include knowledge related to the posts.
Approach:	They propose to use a novel method to generate responses from posts and related knowledge by injecting knowledge into dialogue generation models.
Outcome:	The proposed method outperforms baseline models in terms of knowledge relevance and quality.

Copied to clipboard

Jian Chen, Zhenyan Chen, Xuming Hu, Peilin Zhou, Yining Hua, Han Fang, Cissy Hing Yee Choy, Xinmei Ke, Jingfeng Luo, Zixuan Yuan

Programming Languages in NLP

Challenge:	NL2SQL provides a model-centric paradigm that simplifies database access for non-technical users . challenges such as inaccurate task decomposition and keyword extraction remain major bottlenecks .
Approach:	They propose a RAG-based NL2SQL pipeline that employs three modules for query understanding, entity retrieval, and generation to improve SQL generation accuracy.
Outcome:	The proposed pipeline improves the accuracy of query generation on BIRD and Spider datasets.

QiMeng-Attention: SOTA Attention Operator is generated by SOTA Attention Algorithm (2025.findings-acl)

Copied to clipboard

Qirui Zhou, Shaohui Peng, Weiqiang Xiong, Haixin Chen, Yuanbo Wen, Haochen Li, Ling Li, Qi Guo, Yongwei Zhao, Ke Gao, Ruizhi Chen, Yanjun Wu, Zhao Chen, Yunji Chen

Programming Languages in NLP

Challenge:	Existing LLMs cannot comprehend the complex data flow and computation process of the attention operator and utilize low-level primitive to exploit GPU performance.
Approach:	They propose an LLM-friendly Thinking Language (LLM-TL) that can decouple the generation of high-level optimization logic and low-level implementation on GPU and enhance LLMs’ understanding of attention operator.
Outcome:	The proposed method outshines existing LLMs on A100, RTX8000, and T4 GPUs, achieving a speed-up of up to 35.16.

MIO: A Foundation Model on Multimodal Tokens (2025.emnlp-main)

Copied to clipboard

Zekun Moore Wang, King Zhu, Chunpu Xu, Wangchunshu Zhou, Jiaheng Liu, Yibo Zhang, Jessie Wang, Ning Shi, Siyu Li, Yizhi Li, Haoran Que, Zhaoxiang Zhang, Yuanxing Zhang, Ge Zhang, Ke Xu, Jie Fu, Wenhao Huang

FRAG: A Flexible Modular Framework for Retrieval-Augmented Generation based on Knowledge Graphs (2025.findings-acl)

Challenge:	Existing models lack multimodal understanding capabilities, resulting in closed-source model that does not support multimodal interleaved sequences.
Approach:	They propose a foundation model built on multimodal tokens capable of understanding and generating speech, text, images, and videos in an end-to-end, autoregressive manner.
Outcome:	The proposed model is able to understand speech, text, images, and videos in an end-to-end, autoregressive manner.

Copied to clipboard

Zengyi Gao, Yukun Cao, Hairu Wang, Ao Ke, Yuan Feng, S Kevin Zhou, Xike Xie

Alignment with Fill-In-the-Middle for Enhancing Code Generation (2025.emnlp-main)

Challenge:	Existing methods for retrieval-augmented generation struggle with a trade-off between flexibility and retrieval quality.
Approach:	They propose a flexible modular KG-RAG framework that uses query text instead of KGs . they propose to use query text to infer the structural information of reasoning paths .
Outcome:	The proposed method achieves state-of-the-art performance with high efficiency and low resource consumption.

Copied to clipboard

Houxing Ren, Zimu Lu, Weikang Shi, Haotian Hou, Yunqiao Yang, Ke Wang, Aojun Zhou, Junting Pan, Mingjie Zhan, Hongsheng Li

Programming Languages in NLP

Challenge:	Existing methods for generating test cases with limited training data are not reliable and may be counterproductive.
Approach:	They propose a method that splits code snippets into smaller, granular blocks, creating more diverse DPO pairs from the same test cases.
Outcome:	The proposed approach shows significant improvements in code generation tasks on benchmark datasets such as HumanEval (+), MBPP (+), and APPS.

SATBench: Benchmarking LLMs’ Logical Reasoning via Automated Puzzle Generation from SAT Formulas (2025.emnlp-main)

Copied to clipboard

Anjiang Wei, Yuheng Wu, Yingjia Wan, Tarun Suresh, Huanmi Tan, Zhanke Zhou, Sanmi Koyejo, Ke Wang, Alex Aiken

Reasoning

Challenge:	SATBench is a benchmark for evaluating the logical reasoning capabilities of large language models (LLMs) through logical puzzles derived from Boolean satisfiability (SAT) problems.
Approach:	They propose a benchmark to evaluate logical reasoning capabilities of large language models (LLMs) using logical puzzles derived from Boolean satisfiability problems.
Outcome:	The proposed model achieves 65.0% accuracy on hard UNSAT problems, close to the baseline of 50%.

Enhancing the General Agent Capabilities of Low-Paramter LLMs through Tuning and Multi-Branch Reasoning (2024.findings-naacl)

Copied to clipboard

Qinhao Zhou, Zihan Zhang, Xiang Xiang, Ke Wang, Yuchuan Wu, Yongbin Li

Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting (2021.emnlp-main)

Challenge:	Open-source pre-trained Large Language Models exhibit strong language understanding and generation capabilities, making them highly successful in a variety of tasks.
Approach:	They propose a method to construct agent-specific data using GPT-4 and supervised fine-tuning . they find that supervised tunning can significantly reduce hallucination outputs and formatting errors in agent tasks .
Outcome:	The proposed method improves on five agent tasks of AgentBench.

Copied to clipboard

Wangchunshu Zhou, Tao Ge, Canwen Xu, Ke Xu, Furu Wei

Text Generation

Challenge:	Existing text infilling objectives for pretrained language models require self-supervision by masking out tokens or spans in text.
Approach:	They propose to extend text infilling to a self-supervised sequence-to-sequence (Seq2Sequen) task.
Outcome:	The proposed task improves the model's performance on various natural language generation tasks.

Enhancing AI Assisted Writing with One-Shot Implicit Negative Feedback (2024.emnlp-main)

Copied to clipboard

Benjamin Towle, Ke Zhou

Challenge:	Various systems have been proposed to draft and automate replies for users . yet, the heterogeneity of the inputs and architectures often renders it difficult to combine insights from user behaviour in one system to improve performance in another.
Approach:	They propose an approach that uses classifier guidance to controllably integrate implicit user feedback into the text generation process.
Outcome:	The proposed approach improves Rouge-L, generating the correct intent and generating an 86% win-rate on the multiWOZ and Schema-Guided Dialog datasets.

SimRPD: Optimizing Recruitment Proactive Dialogue Agents through Simulator-Based Data Evaluation and Selection (2026.acl-industry)

Copied to clipboard

Zhiyong Cao, Dunqiang Liu, Qi Dai, Haojun Xu, Huai Yuen Khor, Hao Wang, Huan He, Yafei Liu, Ke Ma, Ruqian Shi, Sicheng Zhou, Sijia Yao

Fin-STAR: Structure-as-Semantics to Resolve Implicitness in Financial Retrieval (2026.findings-acl)

Challenge:	High-quality data in training proactive dialogue agents is scarce, despite fine-tuning and reinforcement learning . a recent study has shown that the effectiveness of supervised fine-touring is limited by the lack of high-quality, domain-specific training data.
Approach:	They propose a framework for training recruitment proactive dialogue agents using a high-fidelity user simulator and a multi-dimensional evaluation framework based on Chain-of-Intention.
Outcome:	The proposed framework outperforms existing simulator-based data selection strategies in a real-world recruitment scenario.

Copied to clipboard

Yu Zou, Yan Chen, Lida He, Qi Zhou, Xiaorui Zhou, Aixi Zhong, Yi Wang, Wei Li, Qingyu Wang, Jiatao Li, Wei Gong, Jialei Zeng, Jingmei Zhao, Ke Jiang, Qing Li

Information Retrieval

Challenge:	Existing Retrieval-Augmented Generation systems treat structure as a physical navigational skeleton rather than intrinsic semantic knowledge.
Approach:	They propose a framework that redefining hierarchy as intrinsic semantics and uses snippets to enrich hierarchical lineage.
Outcome:	The proposed framework outperforms state-of-the-art hierarchical and graph-based benchmarks on FinTierQA Gold.

Understanding Large Language Model Vulnerabilities to Social Bias Attacks (2025.acl-long)

Copied to clipboard

Jiaxu Zhao, Meng Fang, Fanghua Ye, Ke Xu, Qin Zhang, Joey Tianyi Zhou, Mykola Pechenizkiy

Responsible & Trustworthy NLP

Challenge:	Large Language Models (LLMs) have demonstrated remarkable linguistic capabilities across tasks . however, there is a growing concern about their potential to perpetuate social biases .
Approach:	They evaluate LLMs across gender, racial, and religious bias types . they also explore cross-bias and multiple-biases attacks .
Outcome:	The proposed models are more susceptible to gender bias attacks than racial or religious biases.

RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models (2024.findings-acl)

Copied to clipboard

Noah Wang, Z.y. Peng, Haoran Que, Jiaheng Liu, Wangchunshu Zhou, Yuhan Wu, Hongcheng Guo, Ruitong Gan, Zehao Ni, Jian Yang, Man Zhang, Zhaoxiang Zhang, Wanli Ouyang, Ke Xu, Wenhao Huang, Jie Fu, Junran Peng

Scheduled DropHead: A Regularization Method for Transformer Models (2020.findings-emnlp)

Challenge:	Large Language Models (LLMs) have paved the way for complex tasks such as role-playing.
Approach:	They propose a framework to benchmark, elicit, and enhance role-playing abilities in Large Language Models.
Outcome:	The proposed framework improves role-playing abilities with 168,093 samples.

Copied to clipboard

Wangchunshu Zhou, Tao Ge, Furu Wei, Ming Zhou, Ke Xu

RecBase: Generative Foundation Model Pretraining for Zero-Shot Recommendation (2025.emnlp-main)

Challenge:	DropHead is a structured dropout method for regularizing multi-head attention . DropHed drops entire attention heads during training to prevent overfitting .
Approach:	They propose a structured dropout method specifically designed for regularizing multi-head attention mechanism . DropHead drops entire attention heads during training to prevent overfitting .
Outcome:	The proposed method can improve transformer models by 0.9 BLEU score on translation task and around 1.0 accuracy for various text classification tasks.

Copied to clipboard

Sashuai Zhou, Weinan Gan, Qijiong Liu, Ke Lei, Jieming Zhu, Hai Huang, Yan Xia, Ruiming Tang, Zhenhua Dong, Zhou Zhao

Improving BERT with Syntax-aware Local Attention (2021.findings-acl)

Challenge:	Existing methods for addressing item-level user interests are lacking in cross-domain generalization . RecBase model is domain-agnostic and can be used to enhance recommender systems' effectiveness .
Approach:	They propose a domain-agnostic foundational model pretrained with a recommendation-oriented objective that leverages a large-scale, heterogeneous, cross-domain corpus with unified textual representations and feature mappings to enhance cross- domain generalization.
Outcome:	The proposed model matches or surpasses baselines in zero-shot and cross-domain recommendation tasks on eight real-world datasets.

Copied to clipboard

Zhongli Li, Qingyu Zhou, Chao Li, Ke Xu, Yunbo Cao

Skip-Thinking: Chunk-wise Chain-of-Thought Distillation Enable Smaller Language Models to Reason Better and Faster (2025.emnlp-main)

Challenge:	Recent studies show that attention-based models benefit from more focused attention over local regions.
Approach:	They propose a syntax-aware local attention which restrains attention over syntactically relevant words.
Outcome:	The proposed model performs better on all benchmark datasets, including sentence classification and sequence labeling tasks.

Copied to clipboard

Xiaoshu Chen, Sihang Zhou, Ke Liang, Xiaoyu Sun, Xinwang Liu

HPSS: Heuristic Prompting Strategy Search for LLM Evaluators (2025.findings-acl)

Challenge:	Existing methods train small language models to learn long rationales in one iteration.
Approach:	They propose a method that uses a heuristic search to divide rationale into internal chunks . they propose CWT, which uses CWt to focus SLM on learning from only one chunk per iteration.
Outcome:	The proposed method can guide a large language model (LLM) in reasoning tasks.

Copied to clipboard

Bosi Wen, Pei Ke, Yufei Sun, Cunxiang Wang, Xiaotao Gu, Jinfeng Zhou, Jie Tang, Hongning Wang, Minlie Huang

CharacterGLM: Customizing Social Characters with Large Language Models (2024.emnlp-industry)

Challenge:	Existing efforts to optimize text evaluation prompts neglect the combinatorial impact of multiple factors, leading to insufficient optimization of the evaluation pipeline.
Approach:	They propose to integrate 8 key factors for evaluation prompts and integrate them into an algorithm that searches for well-behaved prompting strategies for LLM evaluators.
Outcome:	The proposed method outperforms existing methods and human-designed evaluation prompts on four evaluation tasks.

Copied to clipboard

Jinfeng Zhou, Zhuang Chen, Dazhen Wan, Bosi Wen, Yi Song, Jifan Yu, Yongkang Huang, Pei Ke, Guanqun Bi, Libiao Peng, JiaMing Yang, Xiyao Xiao, Sahand Sabour, Xiaohan Zhang, Wenjing Hou, Yijia Zhang, Yuxiao Dong, Hongning Wang, Jie Tang, Minlie Huang

Wav-BERT: Cooperative Acoustic and Linguistic Representation Learning for Low-Resource Speech Recognition (2021.findings-emnlp)

Challenge:	Character-based dialogue systems (CharacterDial) allow users to customize social characters for social interactions.
Approach:	They will collect a large-scale Chinese corpus of characters with diverse categories and behaviors and develop CharacterGLM models to address these challenges.
Outcome:	Experiments show that CharacterGLM outperforms most popular open- and closed-source LLMs and performs comparable to GPT-4.

Copied to clipboard

Guolin Zheng, Yubei Xiao, Ke Gong, Pan Zhou, Xiaodan Liang, Liang Lin

An Empirical Study on Parameter-Efficient Fine-Tuning for MultiModal Large Language Models (2024.findings-acl)

Challenge:	Existing methods to learn the transfer from speech to text are unexplored . how to solve the representation discrepancy of speech and text is unexplorable .
Approach:	They propose a cooperative acoustic and linguistic representation learning method to fuse and utilize contextual information of speech and text.
Outcome:	The proposed method outperforms existing methods on low-resource speech recognition.

Copied to clipboard

Xiongtao Zhou, Jie He, Yuhua Ke, Guangyao Zhu, Victor Gutierrez Basulto, Jeff Pan

Learn What Is Possible, Then Choose What Is Best: Disentangling One-To-Many Relations in Language Through Text-based Games (2022.findings-emnlp)

Challenge:	Multimodal Large Language Models fine-tuned with multimodal instruction-following data have demonstrated formidable capabilities in multimodal tasks.
Approach:	They propose to employ four PEFT methods to fine-tune the LLM component of open-source MLLMs.
Outcome:	The proposed method is the best performing on seven datasets, while fine-tuning the connector layers leads to improved performance in most MLLMs.

Copied to clipboard

Benjamin Towle, Ke Zhou

BERT-based Lexical Substitution (P19-1)

Challenge:	Language models pre-trained on large self-supervised corpora, followed by task-specific fine-tuning has become the dominant paradigm in NLP.
Approach:	They propose to train language models pre-trained on large self-supervised corpora, followed by task-specific fine-tuning on the target domain.
Outcome:	The proposed model improves on the previous state-of-the-art model on the Jericho Walkthroughs dataset by 49%.

Copied to clipboard

Wangchunshu Zhou, Tao Ge, Ke Xu, Furu Wei, Ming Zhou

I run as fast as a rabbit, can you? A Multilingual Simile Dialogues Datasets (2023.findings-acl)

Challenge:	Existing approaches to lexical substitution tend to overlook good substitute candidates that are not the synonyms of the target words in the lexicals and fail to take into account the substitution’s influence on the global context of the sentence.
Approach:	They propose an end-to-end BERT-based lexical substitution approach which proposes and validates substitute candidates without using annotated data or manually curated resources.
Outcome:	The proposed approach performs well in proposing and ranking substitute candidates, achieving the state-of-the-art results in both LS07 and LS14 benchmarks.

Copied to clipboard

Longxuan Ma, Wei-Nan Zhang, Shuhan Zhou, Churui Sun, Changxin Ke, Ting Liu

Natural Language Interfaces

Challenge:	A simile is a figure of speech that compares two different things via shared properties.
Approach:	They propose a multilingual simile dialogue dataset that can be used to study similes in real-life scenarios.
Outcome:	The proposed dataset is the largest manually annotated simile dataset and contains both English and Chinese data.

Hyperlink-induced Pre-training for Passage Retrieval in Open-domain Question Answering (2022.acl-long)

Copied to clipboard

Jiawei Zhou, Xiaoguang Li, Lifeng Shang, Lan Luo, Ke Zhan, Enrui Hu, Xinyu Zhang, Hao Jiang, Zhao Cao, Fan Yu, Xin Jiang, Qun Liu, Lei Chen

Pseudo-Bidirectional Decoding for Local Sequence Transduction (2020.findings-emnlp)

Challenge:	Existing methods to train dense passage retrieval have a large data gap between upstream and downstream relevance.
Approach:	They propose a method to pre-train the dense retriever with the text relevance induced by hyperlinks within Web documents.
Outcome:	The proposed method outperforms existing methods under different scenarios and in the open-domain question answering domain.

Copied to clipboard

Wangchunshu Zhou, Tao Ge, Ke Xu

Knowledge-Grounded Dialogue Generation with Term-level De-noising (2021.findings-acl)

Challenge:	Local sequence transduction tasks involve massive overlapping between source and target sequences . experimental results show that Pseudo-Bidirectional Decoding improves performance of standard seq2seq models.
Approach:	They propose a simple but versatile approach for local sequence transduction tasks . they propose to copy source tokens to decoder as pseudo future context .
Outcome:	The proposed approach improves the performance of standard seq2seq models on LST tasks.

Copied to clipboard

Wen Zheng, Natasa Milic-Frayling, Ke Zhou