Papers by Kun Zhao

Evaluating Open-Domain Dialogues in Latent Space with Next Sentence Prediction and Mutual Information (2023.acl-long)

Challenge:	Autoregressive (AR) and diffusion language models (DLMs) suffer from insufficient reasoning capabilities.
Approach:	They propose a fully connected Diffusion Language Model that uses a concept-level causal graph to guide attention to learn causal relationships between concepts.
Outcome:	The proposed model achieves a 12% improvement and 3.2 training speedup on the COT-OrderPerturb task, along with an average gain of 1.31% across six downstream reasoning tasks.

Copied to clipboard

Kun Zhao, Bohao Yang, Chenghua Lin, Wenge Rong, Aline Villavicencio, Xiaohui Cui

TexSmart: A System for Enhanced Natural Language Understanding (2021.acl-demo)

Challenge:	Existing evaluation methods for open-domain dialogues are difficult due to the one-to-many issue of the open- domain dialogues.
Approach:	They propose a learning-based automatic evaluation metric which can robustly evaluate open-domain dialogues by augmenting CVAEs with a Next Sentence Prediction objective and employing Mutual Information to model the semantic similarity of text in the latent space.
Outcome:	The proposed method can evaluate open-domain dialogues on two open- domain dialogue datasets.

Copied to clipboard

Lemao Liu, Haisong Zhang, Haiyun Jiang, Yangming Li, Enbo Zhao, Kun Xu, Linfeng Song, Suncong Zheng, Botong Zhou, Dick Zhu, Xiao Feng, Tao Chen, Tao Yang, Dong Yu, Feng Zhang, ZhanHui Kang, Shuming Shi

Information Extraction & Text Mining

Challenge:	TexSmart supports fine-grained named entity recognition (NER) Large-scale fine-granular entity types are expected to provide richer semantic information for downstream NLP applications.
Approach:	They introduce TexSmart, a text understanding system that supports fine-grained named entity recognition (NER) and enhanced semantic analysis functionalities.
Outcome:	The proposed system supports fine-grained named entity recognition (NER) and enhanced semantic analysis functions.

Effective Distillation of Table-based Reasoning Ability from LLMs (2024.lrec-main)

Copied to clipboard

Bohao Yang, Chen Tang, Kun Zhao, Chenghao Xiao, Chenghua Lin

Probing Multimodal Large Language Models for Global and Local Semantic Representations (2024.lrec-main)

Challenge:	Existing work on table-based reasoning distillation has focused on smaller models with limited performance.
Approach:	They propose a table-based reasoning distillation approach to distill LLMs into smaller models . their results show that a 220 million parameter model fine-tuned using distilled data improves performance .
Outcome:	The proposed model improves on a scientific table-to-text generation dataset and surpasses specific LLMs.

Copied to clipboard

Mingxu Tao, Quzhe Huang, Kun Xu, Liwei Chen, Yansong Feng, Dongyan Zhao

Diffusion-NAT: Self-Prompting Discrete Diffusion for Non-Autoregressive Text Generation (2024.eacl-long)

Challenge:	Existing studies have focused on the ability of MLLMs to generate single tokens one by one, while lacking studies about how their representation vectors can encode global multimodal information.
Approach:	They propose to use image-caption corpus to train Multimodal Large Language Models (MLLMs) . they find that the topmost layers encode more global semantic information .
Outcome:	The proposed models can encode more global semantic information, rather than the topmost layers, and perform better on visual-language entailment tasks.

Copied to clipboard

Kun Zhou, Yifan Li, Xin Zhao, Ji-Rong Wen

A Unified Framework for Synaesthesia Analysis (2023.findings-emnlp)

Challenge:	Existing non-autoregressive (NAR) text-to-text generation methods are unable to generate coherent and fluent texts due to discrete nature of text.
Approach:	They propose to integrate discrete diffusion models (DDM) into NAR text-to-text generation and integrate BART to improve the performance.
Outcome:	The proposed method outperforms competing methods and surpasses autoregressive methods on 7 datasets.

Copied to clipboard

Kun Sheng, Zhongqing Wang, Qingqing Zhao, Xiaotong Jiang, Guodong Zhou

Cognitive Modeling

Challenge:	Synaesthesia is a cognitive phenomenon structuring human thought and action, which makes understanding it challenging.
Approach:	They propose a framework for annotating synaesthetic elements and exploring their relationship . they propose to include sensory modalities, cues and stimuli in the framework .
Outcome:	The proposed framework yields state-of-the-art results, demonstrating its effectiveness.

Debiased Contrastive Learning of Unsupervised Sentence Representations (2022.acl-long)

Copied to clipboard

Kun Zhou, Beichen Zhang, Xin Zhao, Ji-Rong Wen

Enhancing Chain-of-Thought Reasoning via Neuron Activation Differential Analysis (2025.emnlp-main)

Challenge:	Recent studies have shown that contrastive learning improves pre-trained language models to derive high-quality sentence representations.
Approach:	They propose a framework to punish false negatives and generate noise-based negatives to guarantee the uniformity of the representation space.
Outcome:	The proposed framework improves pre-trained language models while pushing apart irrelevant negatives to guarantee the uniformity of the representation space.

Copied to clipboard

Yiru Tang, Kun Zhou, Yingqian Min, Xin Zhao, Jing Sha, Zhichao Sheng, Shijin Wang

Reasoning

Challenge:	Existing studies focus on optimizing external components of CoT, but lack internal explanations for the quality of the model's outputs.
Approach:	They propose an efficient method to identify reasoning-critical neurons by analyzing their activation patterns under reasoning chains of varying quality.
Outcome:	The proposed method shows that neurons in the feed-forward layers are critical in the generation of high-quality reasoning chains.

Evaluating Object Hallucination in Large Vision-Language Models (2023.emnlp-main)

Copied to clipboard

Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Xin Zhao, Ji-Rong Wen

Emphasising Structured Information: Integrating Abstract Meaning Representation into LLMs for Enhanced Open-Domain Dialogue Evaluation (2025.findings-emnlp)

Challenge:	Large vision-language models (LVLMs) suffer from object hallucinations, i.e., they tend to generate objects inconsistent with the target images in the descriptions.
Approach:	They propose to integrate powerful large vision-language models (LVLMs) they propose a polling-based query method to evaluate object hallucination .
Outcome:	The proposed model can evaluate object hallucination in a more stable and flexible way.

Copied to clipboard

Bohao Yang, Kun Zhao, Dong Liu, Chen Tang, Liang Zhan, Chenghua Lin

Evolving Knowledge Distillation with Large Language Models and Active Learning (2024.lrec-main)

Challenge:	Existing evaluation metrics struggle to evaluate adversarial negative examples . existing metrics struggle in handling adversarials, resulting in low correlations with human judgments.
Approach:	They propose a framework that integrates AMR and domain-specific language models for automatic open-domain dialogue evaluation.
Outcome:	The proposed evaluation framework achieves strong correlations with human judgments across multiple datasets.

Copied to clipboard

Chengyuan Liu, Fubang Zhao, Kun Kuang, Yangyang Kang, Zhuoren Jiang, Changlong Sun, Fei Wu

StructGPT: A General Framework for Large Language Model to Reason over Structured Data (2023.emnlp-main)

Challenge:	Existing studies have focused on the direct use of large language models for text generation and labeling, without fully exploring their potential to comprehend the target task and acquire valuable knowledge.
Approach:	They propose to distill the knowledge of large language models into smaller models by generating annotated data.
Outcome:	The proposed method improves the performance of small domain models while enhancing the ability of large language models.

Copied to clipboard

Jinhao Jiang, Kun Zhou, Zican Dong, Keming Ye, Xin Zhao, Ji-Rong Wen

CR-GIS: Improving Conversational Recommendation via Goal-aware Interest Sequence Modeling (2022.coling-1)

Challenge:	Experiments conducted on three types of structured data show that StructGPT greatly improves the performance of LLMs.
Approach:	They propose an iterative Reading-then-Reasoning framework to solve question answering tasks based on structured data.
Outcome:	The proposed framework improves the reasoning ability of large language models over structured data under the few-shot and zero-shot settings.

Copied to clipboard

Jinfeng Zhou, Bo Wang, Zhitong Yang, Dongming Zhao, Kun Huang, Ruifang He, Yuexian Hou

Small Pre-trained Language Models Can be Fine-tuned as Large Models via Over-Parameterization (2023.acl-long)

Challenge:	Existing methods to determine a goal item by sequentially tracking users’ interests ignore the rich goal-aware implicit interest sequence patterns in a dialog.
Approach:	They propose to model goal-aware implicit user interest sequence patterns in a dialog and a hierarchical Star Transformer to guide multi-turn utterances generation.
Outcome:	The proposed framework achieves more accurate recommendations with more fluent and coherent dialog utterances.

Copied to clipboard

Ze-Feng Gao, Kun Zhou, Peiyu Liu, Wayne Xin Zhao, Ji-Rong Wen

KG-Agent: An Efficient Autonomous Agent Framework for Complex Reasoning over Knowledge Graph (2025.acl-long)

Challenge:	Large pre-trained language models (PLMs) have shown remarkable performance in various natural language processing tasks, outperforming small PLMs by a large margin.
Approach:	They propose to scale up parameters of pre-trained language models only during fine-tuning to benefit from over-parameterization.
Outcome:	The proposed approach can significantly boost the fine-tuning performance of small PLMs and even help small PDMs outperform 3 parameterized larger ones.

Copied to clipboard

Jinhao Jiang, Kun Zhou, Xin Zhao, Yang Song, Chen Zhu, Hengshu Zhu, Ji-Rong Wen

Knowledge Representation

Challenge:	Existing methods to design the interaction strategy between large language models and knowledge graphs (KGs) are not effective for large language model (LLM)s to solve complex tasks due to the large volume and structured format of KG data.
Approach:	They propose an LLM-based agent framework that enables small LLMs to actively make decisions over knowledge graphs.
Outcome:	The proposed framework outperforms existing methods on in-domain and out-domain datasets using 10K samples.

DATA-CUBE: Data Curriculum for Instruction-based Sentence Representation Learning (2024.findings-acl)

Copied to clipboard

Yingqian Min, Kun Zhou, Dawei Gao, Xin Zhao, He Hu, Yaliang Li

Beyond the Last Frame: Process-aware Evaluation for Generative Video Reasoning (2026.acl-long)

Challenge:	Existing methods to improve sentence representation learning (SRL) ignore the potential interference problems across tasks and instances.
Approach:	They propose a multi-task instruction tuning method that arranges the order of multi- task data for training to minimize interference risks.
Outcome:	The proposed method can boost the performance of state-of-the-art methods.

Copied to clipboard

Yifan Li, YuKai Gu, Yingqian Min, Zikang Liu, Yifan Du, Kun Zhou, Min Yang, Xin Zhao, Minghui Qiu

LoraRetriever: Input-Aware LoRA Retrieval and Composition for Mixed Tasks in the Wild (2024.findings-acl)

Challenge:	Existing evaluation frameworks often rely on single-frame assessments, which can lead to outcome-hacking.
Approach:	They propose a process-aware evaluation paradigm that uses a hierarchical rubric to evaluate the validity of the intermediate steps and the final result.
Outcome:	The proposed model achieves POC@1.0 only about 20% and exhibits significant outcome-hacking.

Copied to clipboard

Ziyu Zhao, Leilei Gan, Guoyin Wang, Wangchunshu Zhou, Hongxia Yang, Kun Kuang, Fei Wu

Great~Truths~are ~Always ~Simple: A Rather Simple Knowledge Encoder for Enhancing the Commonsense Reasoning Capacity of Pre-Trained Models (2022.findings-naacl)

Challenge:	Low-Rank Adaptation (LoRA) is an effective yet efficient solution for fine-tuning large language models.
Approach:	They propose a low-rank Adaptation framework that retrieves and composes multiple LoRAs according to input prompts.
Outcome:	Experimental results show that LoraRetriever outperforms baselines in terms of performance and versatility.

Copied to clipboard

Jinhao Jiang, Kun Zhou, Ji-Rong Wen, Xin Zhao

Virtual Data Augmentation: A Robust and General Framework for Fine-tuning Pre-trained Models (2021.emnlp-main)

Challenge:	Existing approaches to enhance pre-trained language models (PTMs) with a knowledge-aware graph neural network (GNN) encoder that models a commonsense knowledge graph (CSKG) can't explain how external knowledge resources improve the reasoning capacity of PTMs.
Approach:	They propose to use relation features from CSKGs to enhance the reasoning capacity of pre-trained language models (PTMs) by encoding a commonsense knowledge graph (CSKG)
Outcome:	The proposed approach reduces the parameters for encoding CSKGs and improves on five benchmarks.

Copied to clipboard

Kun Zhou, Wayne Xin Zhao, Sirui Wang, Fuzheng Zhang, Wei Wu, Ji-Rong Wen

DyLex: Incorporating Dynamic Lexicons into BERT for Sequence Labeling (2021.emnlp-main)

Challenge:	Recent studies have shown that powerful pre-trained language models can be fooled by small perturbations or intentional attacks.
Approach:	They propose a framework for fine-tuning PLMs using a masked language model and Gaussian noise to augment semantically relevant examples with sufficient diversity.
Outcome:	The proposed framework improves the robustness of pre-trained language models and alleviates performance degradation under adversarial attacks.

Copied to clipboard

Baojun Wang, Zhao Zhang, Kun Xu, Guang-Yuan Hao, Yuyang Zhang, Lifeng Shang, Linlin Li, Xiao Chen, Xin Jiang, Qun Liu

CRSLab: An Open-Source Toolkit for Building Conversational Recommender System (2021.acl-demo)

Challenge:	Existing approaches to integrate lexical knowledge into deep learning models are limited by large-scale dynamic lexicons.
Approach:	They propose a plug-in lexicon incorporation approach for BERT based sequence labeling tasks . they adopt word-agnostic tag embeddings to avoid re-training the representation .
Outcome:	The proposed framework achieves new SOTA even with large scale lexicons, the authors show . they adopt word-agnostic tag embeddings to avoid re-training the representation .

Copied to clipboard

Kun Zhou, Xiaolei Wang, Yuanhang Zhou, Chenzhan Shang, Yuan Cheng, Wayne Xin Zhao, Yaliang Li, Ji-Rong Wen

SLIDE: A Framework Integrating Small and Large Language Models for Open-Domain Dialogues Evaluation (2024.findings-acl)

Challenge:	Existing studies on conversational recommender systems lack a unified and standardized implementation or comparison.
Approach:	They propose to use a unified framework and highly-decoupled modules to develop CRSs.
Outcome:	The proposed framework collects 6 commonly used human-annotated CRS datasets and implements 19 models that include advanced techniques such as graph neural networks and pre-training models.

Copied to clipboard

Kun Zhao, Bohao Yang, Chen Tang, Chenghua Lin, Liang Zhan

Enhancing Personalized Dialogue Generation with Contrastive Latent Variables: Combining Sparse and Dense Persona (2023.acl-long)

Challenge:	Existing approaches to evaluate open domain dialogues have a one-to-many problem . existing approaches lack commonsense reasoning biases and perform poorly in domain-specific scenarios.
Approach:	They propose a framework that leverages both a small, specialised model and LLMs for the evaluation of open-domain dialogues.
Outcome:	The proposed framework achieves state-of-the-art performance in both classification and evaluation tasks and exhibits better correlation with human judgements.

Copied to clipboard

Yihong Tang, Bo Wang, Miao Fang, Dongming Zhao, Kun Huang, Ruifang He, Yuexian Hou

RexUIE: A Recursive Method with Explicit Schema Instructor for Universal Information Extraction (2023.findings-emnlp)

Challenge:	Existing personalized dialogue agents model persona profiles from sparse or dense persona descriptions and dialogue histories.
Approach:	They propose a model that clusters dense persona descriptions into sparse categories and generates personalized responses from dialogue histories.
Outcome:	The proposed model improves on Chinese and English datasets.

Copied to clipboard

Chengyuan Liu, Fubang Zhao, Yangyang Kang, Jingyuan Zhang, Xiang Zhou, Changlong Sun, Kun Kuang, Fei Wu

Information Extraction & Text Mining

Challenge:	Named Entity Recognition (NER) and Relation Extraction (RE) models have limited success when extracting general schemas such as quadruples and quintuples.
Approach:	They propose a formal formulation that covers almost all extraction schemas and a Recursive Method with Explicit Schema Instructor for UIE.
Outcome:	The proposed method shows strong performance under full-shot and few-shot settings and achieves state-of-the-art results on the tasks of extracting complex schemas.

ViFT: Towards Visual Instruction-Free Fine-tuning for Large Vision-Language Models (2025.findings-emnlp)

Copied to clipboard

Zikang Liu, Kun Zhou, Xin Zhao, Dawei Gao, Yaliang Li, Ji-Rong Wen

TopKG: Target-oriented Dialog via Global Planning on Knowledge Graph (2022.coling-1)

Challenge:	Visual instruction tuning is the predominant technology in eliciting multimodal task-solving capabilities of large vision-language models.
Approach:	They propose a visual instruction-free fine-tuning framework for large vision-language models . they require only text-only instructions and image caption data during training .
Outcome:	The proposed framework is based on visual instruction tuning, but requires images as input . it can achieve state-of-the-art performance on several downstream benchmarks with less training data.

Copied to clipboard

Zhitong Yang, Bo Wang, Jinfeng Zhou, Yue Tan, Dongming Zhao, Kun Huang, Ruifang He, Yuexian Hou

Selection Bias Explorations and Debias Methods for Natural Language Sentence Matching Datasets (P19-1)

Challenge:	Existing target-oriented dialogs take a local and greedy strategy for response generation, where global planning is absent.
Approach:	They propose a global planning method for target-oriented dialog on a commonsense knowledge graph to adjust local response generation towards the global target.
Outcome:	The proposed method can reach the target with a higher success rate, fewer turns, and more coherent responses.

Copied to clipboard

Guanhua Zhang, Bing Bai, Jian Liang, Kun Bai, Shiyu Chang, Mo Yu, Conghui Zhu, Tiejun Zhao

CLEAR: Can Language Models Really Understand Causal Graphs? (2024.findings-emnlp)

Challenge:	Natural Language Sentence Matching (NLSM) is a popular NLP task.
Approach:	They propose to use QuoraQP to train and evaluate NLSM models using a selection bias framework.
Outcome:	The proposed framework can improve generalization ability of trained models and give more trustworthy evaluation results for real-world adoptions.

Copied to clipboard

Sirui Chen, Mengying Xu, Kun Wang, Xingyu Zeng, Rui Zhao, Shengjie Zhao, Chaochao Lu

More Than Catastrophic Forgetting: Integrating General Capabilities For Domain-Specific LLMs (2024.emnlp-main)

Challenge:	Existing language models lack a conceptual framework for understanding causal graphs, but there is still potential for improvement.
Approach:	They develop a framework to define causal graph understanding by assessing language models’ behaviors through four practical criteria derived from diverse disciplines.
Outcome:	The proposed framework defines three complexity levels and encompasses 20 causal graph-based tasks across 20 different levels.

Copied to clipboard

Chengyuan Liu, Yangyang Kang, Shihang Wang, Lizhi Qing, Fubang Zhao, Chao Wu, Changlong Sun, Kun Kuang, Fei Wu

Landmark Embedding: A Chunking-Free Embedding Method For Retrieval Augmented Long-Context Large Language Models (2024.acl-long)

Challenge:	a recent study shows that performance on general tasks decreases after Large Language Models are fine-tuned on domain-specific tasks.
Approach:	They propose a general capability integration approach to integrate general capabilities and domain knowledge within a single instance.
Outcome:	The proposed method improves performance on domain-specific tasks by integrating general capabilities and domain knowledge.

Copied to clipboard

Kun Luo, Zheng Liu, Shitao Xiao, Tong Zhou, Yubo Chen, Jun Zhao, Kang Liu

Extracting and Combining Abilities For Building Multi-lingual Ability-enhanced Large Language Models (2025.emnlp-main)

Challenge:	Existing methods for retrieval augmentation work with chunked contexts, which leads to poor quality of semantic representation and incomplete retrieval of useful information.
Approach:	They propose a method for retrieval augmentation of long-context language modeling using landmark embedding.
Outcome:	The proposed method outperforms existing retrieval methods with a notable advantage.

Copied to clipboard

Zhipeng Chen, Kun Zhou, Liang Song, Xin Zhao, Bingning Wang, Weipeng Chen, Ji-Rong Wen

YuLan-Mini: Pushing the Limits of Open Data-efficient Language Model (2025.acl-long)

Challenge:	Existing work relies on training with multi-lingual ability-related data, which may not be available for low-resource languages.
Approach:	They propose a multi-lingual ability-enhanced LLM that extracts language-agnostic ability-related weights from LLMs and combine them across different languages by simple addition and subtraction operations without training.
Outcome:	The proposed approach extracts language-agnostic ability-related weights from LLMs and combine them across different languages without training.

Copied to clipboard

Hu Yiwen, Huatong Song, Jie Chen, Jia Deng, Jiapeng Wang, Kun Zhou, Yutao Zhu, Jinhao Jiang, Zican Dong, Yang Lu, Xu Miao, Xin Zhao, Ji-Rong Wen

Question Tells You Where the Answer Is: Intention-aware Long-Context KV Cache Compression (2026.acl-long)

Challenge:	prevailing pre-training approaches for large language models involve several complexities.
Approach:	They propose a low-cost training recipe and a robust optimization approach to mitigate training instability . they also propose synthesis, curriculum, and data selection pipelines to integrate data .
Outcome:	The proposed model achieves top-tier performance among models with similar parameter scale . it is comparable to industry-leading models that require significantly more data .

Copied to clipboard

Liang Zhao, Xiaocheng Feng, Weihong Zhong, Lei Huang, Kun Zhu, Baoxin Wang, Dayong Wu, Guoping Hu, Ting Liu, Bing Qin

Challenge:	Recent methods to reduce the KV cache size fail to identify crucial KVs for generation while excluding others accurately, resulting in severe information loss.
Approach:	They propose an intention-aware KV cache eviction method that identifies and retains crucial KVs according to the attention distribution of intention, which semantically reflects the user’s goal and determines which part of the context is relevant.
Outcome:	The proposed method can maintain the model performance while reducing the KV cache size from 128K to 2K, leading to a 6.3x increase in decoding speed and 7.8x enhancement in memory efficiency compared to the default setting.

SimANS: Simple Ambiguous Negatives Sampling for Dense Text Retrieval (2022.emnlp-industry)

Copied to clipboard

Kun Zhou, Yeyun Gong, Xiao Liu, Wayne Xin Zhao, Yelong Shen, Anlei Dong, Jingwen Lu, Rangan Majumder, Ji-rong Wen, Nan Duan

Information Retrieval

Challenge:	Existing methods for sapping negatives from large document pool suffer from the uninformative or false negative problem.
Approach:	They propose a method to sample negatives from a large document pool using a new sampling probability distribution.
Outcome:	The proposed method can be used to sample more ambiguous negatives on four public and one industry datasets.

DORA: Dynamic Optimization Prompt for Continuous Reflection of LLM-based Agent (2025.coling-main)

Copied to clipboard

Kun Li, Tingzhang Zhao, Wei Zhou, Songlin Hu

MTGP: Multi-turn Target-oriented Dialogue Guided by Generative Global Path with Flexible Turns (2023.findings-acl)

Challenge:	Existing studies have shown that reflection can enhance performance, but our investigation reveals an undesirable pattern in reflection framework: effective self-reflection occurs primarily at the beginning of iterations, with subsequent attempts failing to produce further improvements.
Approach:	They propose a framework that generates task-adaptive reflection advice using an external open-source small language model.
Outcome:	The proposed framework generates task-adaptive and diverse reflection advice in MiniWoB++ and Alfworld environments.

Copied to clipboard

Anqi Liu, Bo Wang, Yue Tan, Dongming Zhao, Kun Huang, Ruifang He, Yuexian Hou

Towards Effective and Efficient Continual Pre-training of Large Language Models (2025.acl-long)

Challenge:	Existing approaches focus on global planning, which plans toward the target before the conversation.
Approach:	They propose to generate a global path as a natural language sentence instead of a sequence of nodes.
Outcome:	The proposed method has fewer turns, more coherent semantics, and higher success rate than baselines.

Copied to clipboard

Jie Chen, Zhipeng Chen, Jiapeng Wang, Kun Zhou, Yutao Zhu, Jinhao Jiang, Yingqian Min, Xin Zhao, Zhicheng Dou, Jiaxin Mao, Yankai Lin, Ruihua Song, Jun Xu, Xu Chen, Rui Yan, Zhewei Wei, Di Hu, Wenbing Huang, Ji-Rong Wen

Streaming Hallucination Detection in Long Chain-of-Thought Reasoning (2026.findings-acl)

Challenge:	Continual pre-training (CPT) has been an important approach for adapting language models to specific domains or tasks.
Approach:	They propose a Continual pre-training method that can greatly improve Chinese language ability and scientific reasoning ability of LLMs.
Outcome:	The proposed method can greatly improve Chinese language ability and scientific reasoning ability of LLMs.

Copied to clipboard

Haolang Lu, Minghui Pan, Ripeng LI, Guoshun Nan, Jialin Zhuang, Zijie Zhao, Zhongxiang Sun, Kun Wang, Yang Liu

Reasoning

Challenge:	Long chain-of-thought reasoning improves performance of large language models, yet hallucinations in such settings often emerge subtly and propagate across reasoning steps.
Approach:	They propose to treat step-level hallucination judgments as local observations and introduce a cumulative prefix-level signal that tracks the global evolution of the reasoning state over the entire trajectory.
Outcome:	The proposed method enables streaming hallucination detection in long CoT reasoning, providing real-time, interpretable evidence.

Incorporating Dynamic Semantics into Pre-Trained Language Model for Aspect-based Sentiment Analysis (2022.findings-acl)

Copied to clipboard

Kai Zhang, Kun Zhang, Mengdi Zhang, Hongke Zhao, Qi Liu, Wei Wu, Enhong Chen

Sentiment Analysis

Challenge:	Aspect-based sentiment analysis (ABSA) predicts sentiment polarity towards a specific aspect in a sentence.
Approach:	They propose to use a dynamic aspect-oriented semantics-based method to learn ABSA.
Outcome:	The proposed method can learn dynamic aspect-oriented semantics for ABSA on three benchmark datasets.

Reinforcing Agentic Search Via Reward Density Optimization (2026.acl-long)

Copied to clipboard

Kun Luo, Hongjin Qian, Zheng Liu, Ziyi Xia, Shitao Xiao, Zhao Cao, Siqi Bao, Jun Zhao, Kang Liu

Information Retrieval

Challenge:	Reinforcement Learning with Verifiable Rewards (RLVR) is a promising approach for enhancing agentic search, but its performance is often hindered by reward sparsity .
Approach:	They propose a new research problem to improve the reward obtained per unit of exploration cost by using a system that decomposes long-horizon tasks into intermediate objectives and assigns process-level rewards to provide denser learning signals.
Outcome:	The proposed framework outperforms strong baselines on several agentic search benchmarks and achieves comparable performance to that of advanced proprietary models.

Enhancing Zero-Shot Chain-of-Thought Reasoning in Large Language Models through Logic (2024.lrec-main)

Copied to clipboard

Xufeng Zhao, Mengdi Li, Wenhao Lu, Cornelius Weber, Jae Hee Lee, Kun Chu, Stefan Wermter

KMatrix: A Flexible Heterogeneous Knowledge Enhancement Toolkit for Large Language Model (2024.emnlp-demo)

Challenge:	Experimental evaluations of large language models demonstrate the efficacy of enhanced reasoning by logic.
Approach:	They propose a framework that uses symbolic logic to verify and rectify reasoning steps by steps.
Outcome:	The proposed framework improves the zero-shot chain-of-thought reasoning ability of large language models by verifying and rectifying the reasoning steps step by step.

Copied to clipboard

Shun Wu, Di Wu, Kun Luo, XueYou Zhang, Jun Zhao, Kang Liu

Continual Pre-training of Language Models for Math Problem Understanding with Syntax-Aware Memory Network (2022.acl-long)

Challenge:	Existing Knowledge-Enhanced Large Language Models (K-LLMs) toolkits focus on free-textual knowledge and lack robust datasets, models, and user-friendly experience.
Approach:	They propose a flexible heterogeneous knowledge enhancement toolkit to enhance Large Language Models (LLMs) using external knowledge.
Outcome:	KMatrix: a flexible heterogeneous knowledge enhancement toolkit for LLMs includes verbalizing-retrieval and parsing-query methods.

Copied to clipboard

Zheng Gong, Kun Zhou, Xin Zhao, Jing Sha, Shijin Wang, Ji-Rong Wen

KMatrix-2: A Comprehensive Heterogeneous Knowledge Collaborative Enhancement Toolkit for Large Language Model (2025.emnlp-demos)

Challenge:	a fundamental challenge in modeling math problems is how to fuse semantics of textual description and formulas.
Approach:	They propose a method to continually pre-train language models for improving understanding of math problems with syntax-aware memory networks.
Outcome:	The proposed approach outperforms competitive baselines on four math tasks.

Copied to clipboard

Shun Wu, Di Wu, Wangtao Sun, Ziyang Huang, Xiaowei Yuan, Kun Luo, XueYou Zhang, Shizhu He, Jun Zhao, Kang Liu

On the In-context Generation of Language Models (2024.emnlp-main)

Challenge:	Existing studies on K-LLMs systems focus on declarative knowledge and procedural knowledge (rules) .
Approach:	They propose to build a toolkit that supports comprehensive heterogeneous knowledge collaborative enhancement for Large Language Models (LLMs).
Outcome:	The proposed toolkit provides unified knowledge integration and joint knowledge retrieval methods to achieve more comprehensive heterogeneous knowledge collaborative enhancement.

Copied to clipboard

Zhongtao Jiang, Yuanzhe Zhang, Kun Luo, Xiaowei Yuan, Jun Zhao, Kang Liu

Towards Topic-Guided Conversational Recommender System (2020.coling-main)

Challenge:	Large language models (LLMs) have the ability of in-context generation (ICG) when given an in-text prompt, they can implicitly recognize the pattern of the examples and complete the prompt in the desired way.
Approach:	They propose a plausible latent variable model to model the distribution of pretrained corpora and formalize ICG as a problem of next topic prediction.
Outcome:	The proposed model can model the distribution of pretrained corpora and then formalize ICG as a problem of next topic prediction.

Copied to clipboard

Kun Zhou, Yuanhang Zhou, Wayne Xin Zhao, Xiaoke Wang, Ji-Rong Wen

ChatCoT: Tool-Augmented Chain-of-Thought Reasoning on Chat-based Large Language Models (2023.findings-emnlp)

Challenge:	Existing CRS datasets focus on immediate requests from users, while lack proactive guidance to the recommendation scenario.
Approach:	They propose a topic-guided conversational recommendation dataset . it incorporates topic threads to enforce natural semantic transitions towards the recommendation scenario .
Outcome:	The proposed approach is more reasonable and controllable than previous approaches.

Copied to clipboard

Zhipeng Chen, Kun Zhou, Beichen Zhang, Zheng Gong, Xin Zhao, Ji-Rong Wen

Reasoning

Challenge:	Large language models (LLMs) have excellent performance in evaluation benchmarks, but struggle in complex reasoning tasks.
Approach:	They propose a tool-augmented chain-of-thought reasoning framework for chat-based LLMs . they model chain- of-thoughting reasoning as multi-turn conversations to utilize tools .
Outcome:	The proposed framework can outperform state-of-the-art models on complex reasoning tasks.

Crafting Customisable Characters with LLMs: A Persona-Driven Role-Playing Agent Framework (2025.findings-emnlp)

Copied to clipboard

Bohao Yang, Dong Liu, Chenghao Xiao, Kun Zhao, Chen Tang, Chao Li, Lin Yuan, Yang Guang, Chenghua Lin

Challenge:	Large Language Models (LLMs) are capable of generating human-like text, but the potential for freely customisable characters remains underexplored.
Approach:	They propose a framework which employs Large Language Models to create freely customisable characters through personalised characteristic feature injection.
Outcome:	The proposed framework provides valuable insights for developing more accurate and customisable human simulacra.

Automatic ICD Coding via Interactive Shared Representation Networks with Self-distillation Mechanism (2021.acl-long)

Copied to clipboard

Tong Zhou, Pengfei Cao, Yubo Chen, Kang Liu, Jun Zhao, Kun Niu, Weifeng Chong, Shengping Liu

AesBiasBench: Evaluating Bias and Alignment in Multimodal Language Models for Personalized Image Aesthetic Assessment (2025.emnlp-main)

Challenge:	Existing methods for ICD coding ignore the long-tail of code frequency or noisy clinical notes.
Approach:	They propose to use an interactive shared representation network to model code co-occurrences while focusing on the clinical note's noteworthy part and extract valuable information through a self-distillation learning mechanism to solve the long-tail problem.
Outcome:	The proposed model reduces the long-tail of code frequency and noise in clinical notes and extracts valuable information through a self-distillation learning mechanism.

Copied to clipboard

Kun Li, Lai Man Po, Hongzheng Yang, Xuyuan Xu, Kangcheng Liu, Yuzhi Zhao

Unmasking Style Sensitivity: A Causal Analysis of Bias Evaluation Instability in Large Language Models (2025.acl-long)

Challenge:	Multimodal Large Language Models are increasingly used in Personalized Image Aesthetic Assessment (PIAA) however, their predictions may reflect subtle biases influenced by demographic factors such as gender, age, and education.
Approach:	They propose to evaluate MLLMs along two complementary dimensions: (1) stereotype bias and (2) alignment between model outputs and genuine human aesthetic preferences.
Outcome:	The proposed benchmark covers three subtasks: aesthetic perception, assessment, empathy and alignment between outputs and genuine human aesthetic preferences.

Copied to clipboard

Jiaxu Zhao, Meng Fang, Kun Zhang, Mykola Pechenizkiy

X-ray Made Simple: Lay Radiology Report Generation and Robust Evaluation (2026.findings-acl)

Challenge:	Existing methods to assess social biases in natural language processing models show unexpected instability when input texts undergo minor stylistic changes.
Approach:	They conduct a comprehensive analysis of how style transformations impact bias evaluation results . they find formal style transformation significantly affects bias scores . larger models show greater sensitivity to stylistic variations, they find .
Outcome:	The proposed method fails to detect appearance bias, sexual orientation bias, religious bias and religious bias in large language models.

Copied to clipboard

Kun Zhao, Chenghao Xiao, Sixing Yan, Haoteng Tang, William K. Cheung, Noura Al Moubayed, Liang Zhan, Chenghua Lin

Robustness in NLP

Challenge:	Technical language and templated nature of professional reports hinder patient comprehension and allow models to artificially boost lexical metrics such as BLEU by reproducing common report patterns.
Approach:	They propose a layman's RRG framework that leverages layperson-friendly language to enhance patient accessibility and promote robust evaluation and report generation by encouraging models to focus on semantic accuracy over rigid templates.
Outcome:	The proposed framework improves model performance with more layman-style data, compared to templated professional language and inflated lexical scores.

Demographics Should Not Be the Reason of Toxicity: Mitigating Discrimination in Text Classifications with Instance Weighting (2020.acl-main)

Copied to clipboard

Guanhua Zhang, Bing Bai, Junqi Zhang, Kun Bai, Conghui Zhu, Tiejun Zhao

Text Classification

Challenge:	Recent research has found that text classification datasets contain certain unintended biases, such as text containing demographic identity-terms that are more likely to be abusive.
Approach:	They propose a model-agnostic debiasing framework that recovers the non-discrimination distribution using instance weighting, which does not require extra resources or annotations apart from a pre-defined set of demographic identity-terms.
Outcome:	The proposed framework alleviates the unintended biases without hurting models’ generalization ability.

LLMBox: A Comprehensive Library for Large Language Models (2024.acl-demos)

Copied to clipboard

Tianyi Tang, Hu Yiwen, Bingqian Li, Wenyang Luo, ZiJing Qin, Haoxiang Sun, Jiapeng Wang, Shiyi Xu, Xiaoxue Cheng, Geyang Guo, Han Peng, Bowen Zheng, Yiru Tang, Yingqian Min, Yushuo Chen, Jie Chen, Ranchi Zhao, Luran Ding, Yuhao Wang, Zican Dong, Xia Chunxuan, Junyi Li, Kun Zhou, Xin Zhao, Ji-Rong Wen

Stick to the Facts: Learning towards a Fidelity-oriented E-Commerce Product Description Generation (D19-1)

Challenge:	a library to facilitate the development, use, and evaluation of large language models (LLMs) is presented.
Approach:	They propose a unified library to facilitate the development, use and evaluation of large language models (LLMs).
Outcome:	The proposed library is based on extensive experiments in a variety of evaluation settings.

Copied to clipboard

Zhangming Chan, Xiuying Chen, Yongliang Wang, Juntao Li, Zhiqiang Zhang, Kun Gai, Dongyan Zhao, Rui Yan

Text Generation

Challenge:	Existing models for product description generation do not take the product attribute information into account.
Approach:	They propose a model that takes the embedding and the entity label of each word into account . they establish a keyword memory that stores the entity labels as keys and keywords as values .
Outcome:	The proposed model increases the fidelity of the generated descriptions by 25%.

OS Agents: A Survey on MLLM-based Agents for Computer, Phone and Browser Use (2025.acl-long)

Copied to clipboard

Xueyu Hu, Tao Xiong, Biao Yi, Zishu Wei, Ruixuan Xiao, Yurun Chen, Jiasheng Ye, Meiling Tao, Xiangxin Zhou, Ziyu Zhao, Yuhuai Li, Shengze Xu, Shenzhi Wang, Xinchen Xu, Shuofei Qiao, Zhaokai Wang, Kun Kuang, Tieyong Zeng, Liang Wang, Jiwei Li, Yuchen Eleanor Jiang, Wangchunshu Zhou, Guoyin Wang, Keting Yin, Zhou Zhao, Hongxia Yang, Fan Wu, Shengyu Zhang, Fei Wu

ReasoningLM: Enabling Structural Subgraph Reasoning in Pre-trained Language Models for Question Answering over Knowledge Graph (2023.emnlp-main)

Challenge:	a new generation of (M)LLMs is enabling the creation of superintelligent AI assistants . OS Agents can complete tasks autonomously and have the potential to significantly enhance the lives of billions of users worldwide.
Approach:	They propose to build OS Agents that operate within operating systems' GUIs and GUIs . they examine evaluation metrics and benchmarks to identify promising directions .
Outcome:	The proposed agents are based on operating systems (OS) and operating systems frameworks.

Copied to clipboard

Jinhao Jiang, Kun Zhou, Xin Zhao, Yaliang Li, Ji-Rong Wen

What Makes for Good Visual Instructions? Synthesizing Complex Visual Reasoning Instructions for Visual Instruction Tuning (2025.coling-main)

Challenge:	Question Answering over Knowledge Graph (KGQA) aims to find answer entities for natural language questions based on knowledge graphs.
Approach:	They propose a subgraph-aware self-attention mechanism to imitate the graph neural network (GNN) based module to perform multi-hop reasoning on KG.
Outcome:	The proposed method surpasses state-of-the-art models by a large margin even with fewer updated parameters and less training data.

Copied to clipboard

Yifan Du, Hangyu Guo, Kun Zhou, Wayne Xin Zhao, Jinpeng Wang, Chuyuan Wang, Mingchen Cai, Ruihua Song, Ji-Rong Wen

Guiding Dialogue Agents to Complex Semantic Targets by Dynamically Completing Knowledge Graph (2023.findings-acl)

Challenge:	Experimental results show that visual instruction tuning improves performance of Multi-modal Large Language Models (MLLMs) to extend the application scope of Large Language Modells, a surge of work augments LLMs with vision encoders to endow the ability of multi-modal cognition and reasoning.
Approach:	They propose a systematic approach to create high-quality visual reasoning instructions using a synthesize-complicate-reformulate paradigm.
Outcome:	The proposed method improves performance of MLLMs by 27.86% and 27.60% on MME-Perception and MME Cognition.

Copied to clipboard

Yue Tan, Bo Wang, Anqi Liu, Dongming Zhao, Kun Huang, Ruifang He, Yuexian Hou

Large Language Models as Foundations for Next-Gen Dense Retrieval: A Comprehensive Empirical Assessment (2024.emnlp-main)

Challenge:	Existing knowledge graphs are incomplete in tracking complex semantic relations of the target-oriented dialogue.
Approach:	They combine methods of knowledge retrieval and relationship prediction to construct a context-related dynamic KG and a metric to evaluate the tracked path automatically.
Outcome:	The proposed method can control the agent more logically and smoothly toward the complex target.

Copied to clipboard

Kun Luo, Minghao Qin, Zheng Liu, Shitao Xiao, Jun Zhao, Kang Liu

Visually-augmented pretrained language models for NLP tasks without images (2023.acl-long)

Challenge:	Pre-trained language models have limited generalization capabilities and performance challenges.
Approach:	They evaluate 15 different backbone LLMs and non-LLMs to evaluate their performance . larger models and extensive pre-training consistently enhance in-domain accuracy and data efficiency .
Outcome:	The results show that larger models and extensive pre-training enhance in-domain accuracy and data efficiency.

Copied to clipboard

Hangyu Guo, Kun Zhou, Wayne Xin Zhao, Qinyu Zhang, Ji-Rong Wen

Aligning Recommendation and Conversation via Dual Imitation (2022.emnlp-main)

Challenge:	Existing approaches to improve pre-trained language models lack visual commonsense and semantics.
Approach:	They propose a visual-augmented approach to fine-tune pre-trained language models by using retrieved or generated images instead of relying on explicit images.
Outcome:	The proposed approach outperforms baselines on ten tasks and consistently outperformed other approaches.

Copied to clipboard

Jinfeng Zhou, Bo Wang, Minlie Huang, Dongming Zhao, Kun Huang, Ruifang He, Yuexian Hou

Improving Large Language Models via Fine-grained Reinforcement Learning with Minimum Editing Constraint (2024.findings-acl)

Challenge:	Existing conversational recommendation systems ignore the advantage of user interest shift in connecting recommendation and conversation, leading to an ineffective loose coupling structure.
Approach:	They propose a dual imitation to explicitly align recommendation and conversation paths . they propose to generate high-quality responses with accurate recommendations and coherent explanations .
Outcome:	The proposed model outperforms the state-of-the-art models on recommendation and conversation performance with automatic, human, and novel explainability metrics.

Copied to clipboard

Zhipeng Chen, Kun Zhou, Xin Zhao, Junchen Wan, Fuzheng Zhang, Di Zhang, Ji-Rong Wen

Not Everything is All You Need: Toward Low-Redundant Optimization for Large Language Model Alignment (2024.emnlp-main)

Challenge:	Existing reinforcement learning methods do not provide fine-grained supervision for complex reasoning tasks.
Approach:	They propose a reinforcement learning method that incorporates a generative model as the reward model and a token-level supervision model for RL training.
Outcome:	Experiments on 8 tasks show the proposed method is effective .

Copied to clipboard

Zhipeng Chen, Kun Zhou, Xin Zhao, Jingyuan Wang, Ji-Rong Wen

CAT: Causal Attention Tuning For Injecting Fine-grained Causal Knowledge into Large Language Models (2025.emnlp-main)

Challenge:	Experimental results show that large language models are struggling to align with human preference in complex tasks and scenarios.
Approach:	They propose a low-redundant alignment method that selects the top-10% most updated parameters in LLMs for alignment training.
Outcome:	The proposed method improves on 10 datasets and shows that it is redundant . it can be used to train LLMs on QA and ECQA datasets, but it is not feasible to test it on a large dataset.

Copied to clipboard

Kairong Han, Wenshuo Zhao, Ziyu Zhao, Ye Jun Jian, Lujia Pan, Kun Kuang

Small Agent Can Also Rock! Empowering Small Language Models as Hallucination Detector (2024.emnlp-main)

Challenge:	Existing fine-tuning paradigms focus on aligning LLMs with task-specific objectives.
Approach:	They propose a pipeline that leverages human priors to automatically generate token-level causal signals and introduce the Re-Attention mechanism to guide training.
Outcome:	The proposed pipeline achieves an average improvement of 5.76% on the STG dataset and 1.56% on downstream tasks.

Copied to clipboard

Xiaoxue Cheng, Junyi Li, Xin Zhao, Hongzhi Zhang, Fuzheng Zhang, Di Zhang, Kun Gai, Ji-Rong Wen

Search-in-Context: Efficient Multi-Hop QA over Long Contexts via Monte Carlo Tree Search with Dynamic KV Retrieval (2025.findings-acl)

Challenge:	Existing studies on hallucination detection rely heavily on closed-source LLMs such as GPT-4.
Approach:	They propose an LLM-based agent framework called HaluAgent that integrates LLMs, multi-functional toolbox and a memory mechanism for hallucination detection.
Outcome:	The proposed framework integrates the LLM, multi-functional toolbox, and can detect hallucinations on Chinese and English datasets.

Copied to clipboard

Jiabei Chen, Guang Liu, Shizhu He, Kun Luo, Yao Xu, Jun Zhao, Kang Liu

Parrot: Enhancing Multi-Turn Instruction Following for Large Language Models (2024.acl-long)

Challenge:	Existing approaches to multihop question answering (MHQA) over long contexts are often neglecting explicit reasoning or incurring expensive computational costs due to full-attention mechanisms over long contextuals.
Approach:	They propose a framework that integrates Monte Carlo Tree Search (MCTS) with dynamic key-value retrieval to enable iterative, context-aware reasoning.
Outcome:	The proposed framework integrates Monte Carlo Tree Search (MCTS) with dynamic key-value (KV) retrieval to enable iterative, context-aware reasoning.

Copied to clipboard

Yuchong Sun, Che Liu, Kun Zhou, Jinwen Huang, Ruihua Song, Xin Zhao, Fuzheng Zhang, Di Zhang, Kun Gai