Papers by Yan Zhu

Do Large Language Models excel in Complex Logical Reasoning with Formal Language? (2025.emnlp-main)

Challenge:	Cross-modal retrieval tasks are used to retrieve data from one modality or another based on a query from another modality.
Approach:	They propose a generative cross-modal retrieval framework based on coarse-to-fine semantic modeling . they propose combining K-Means and RQ-VAE to discretize multimodal data into token sequences that support autoregressive generation.
Outcome:	The proposed framework achieves excellent performance and efficiency in multimodal retrieval tasks.

Copied to clipboard

Jin Jiang, Jianing Wang, Yuchen Yan, Yang Liu, Jianhua Zhu, Mengdi Zhang, Liangcai Gao

CodeScope: An Execution-based Multilingual Multitask Multidimensional Benchmark for Evaluating LLMs on Code Understanding and Generation (2024.acl-long)

Challenge:	Existing studies on LLMs have focused on formal language, but evaluations of their performance are limited.
Approach:	They propose to use a formal language to evaluate LLMs across logical reasoning problems using formal languages.
Outcome:	The proposed model outperforms Instruct models in three dimensions, taxonomy of tasks, and format of trajectories, and achieves the best generalization performance across other languages.

Copied to clipboard

Weixiang Yan, Haitian Liu, Yunkun Wang, Yunzhe Li, Qian Chen, Wen Wang, Tingyu Lin, Weishan Zhao, Li Zhu, Hari Sundaram, Shuiguang Deng

Reasoning-to-Defend: Safety-Aware Reasoning Can Defend Large Language Models from Jailbreaking (2025.emnlp-main)

Challenge:	Existing benchmarks for evaluating the code understanding and generation capacities of Large Language Models are insufficient . existing benchmarks focus on a narrow range of popular programming languages and specific tasks .
Approach:	They propose an execution-based, multilingual, multitask evaluation benchmark for LLMs . they evaluate coding performance from three dimensions: length, difficulty, efficiency .
Outcome:	The proposed benchmark covers 43 programming languages and eight coding tasks.

Copied to clipboard

Junda Zhu, Lingyong Yan, Shuaiqiang Wang, Dawei Yin, Lei Sha

RAGEval: Scenario Specific RAG Evaluation Dataset Generation Framework (2025.acl-long)

Challenge:	Large Reasoning Models (LLMs) have demonstrated impressive performances across diverse domains, but how their safety benefits from enhanced reasoning capabilities against jailbreak queries remains unexplored.
Approach:	They propose a safety-aware reasoning paradigm that integrates a pivot token-based safety-based reasoning mechanism into LLMs’ generation process.
Outcome:	The proposed model improves the safety of large language models against jailbreak queries while minimizing attacks and maintaining the original performance.

Copied to clipboard

Kunlun Zhu, Yifan Luo, Dingling Xu, Yukun Yan, Zhenghao Liu, Shi Yu, Ruobing Wang, Shuo Wang, Yishan Li, Nan Zhang, Xu Han, Zhiyuan Liu, Maosong Sun

Text Generation

Challenge:	Existing evaluation metrics for RAG systems are lacking due to high costs of data construction and lack of factual accuracy.
Approach:	They propose a framework to evaluate RAG systems in specialized scenarios . they propose three new metrics to evaluate LLM-generated responses .
Outcome:	The proposed framework outperforms zero-shot and one-shot methods in terms of clarity, safety, conformity, and richness of generated samples.

LoRAMoE: Alleviating World Knowledge Forgetting in Large Language Models via MoE-Style Plugin (2024.acl-long)

Copied to clipboard

Shihan Dou, Enyu Zhou, Yan Liu, Songyang Gao, Wei Shen, Limao Xiong, Yuhao Zhou, Xiao Wang, Zhiheng Xi, Xiaoran Fan, Shiliang Pu, Jiang Zhu, Rui Zheng, Tao Gui, Qi Zhang, Xuanjing Huang

The Illusion of Specialization: Unveiling the Domain-Invariant "Standing Committee" in Mixture-of-Experts Models (2026.acl-long)

Challenge:	Experimental results show that, as the instruction data increases, LoRAMoE can significantly improve the ability to process downstream tasks, while maintaining the world knowledge stored in the LLM.
Approach:	They propose a framework that introduces several low-rank adapters and integrates them by using a router network to freeze the backbone model and force a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks.
Outcome:	The proposed framework freezes the backbone model and forces a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks, to alleviate world knowledge forgetting.

Copied to clipboard

Yan Wang, Yitao Xu, Nanhan Shen, Jinyan Su, Jimin Huang, Zining Zhu

Challenge:	Mixture of Experts models are widely assumed to achieve domain specialization through sparse routing.
Approach:	They propose a framework that analyzes routing behavior at the level of expert groups rather than individual experts.
Outcome:	The proposed framework analyzes routing behavior at the level of expert groups rather than individual experts.

SILC-EFSA: Self-aware In-context Learning Correction for Entity-level Financial Sentiment Analysis (2025.coling-main)

Copied to clipboard

Senbin Zhu, ChenYuan He, Hongde Liu, Pengcheng Dong, Hanjie Zhao, Yuchen Yan, Yuxiang Jia, Hongying Zan, Min Peng

Low-Resource NLP

Challenge:	Currently, most sentiment analysis corpora use sequence-level annotation.
Approach:	They propose a two-stage approach to financial entity-level sentiment analysis called Self-aware In-context Learning Correction.
Outcome:	The proposed approach achieves state-of-the-art on the largest English and Chinese financial entity-level sentiment analysis datasets to date.

Unified Demonstration Retriever for In-Context Learning (2023.acl-long)

Copied to clipboard

Xiaonan Li, Kai Lv, Hang Yan, Tianyang Lin, Wei Zhu, Yuan Ni, Guotong Xie, Xiaoling Wang, Xipeng Qiu

FastMem: Fast Memorization of Prompt Improves Context Awareness of Large Language Models (2024.findings-emnlp)

Challenge:	In-context learning is a new learning paradigm where a language model conditions on a few input-output pairs (demonstrations) and a test input, and directly outputs the prediction.
Approach:	They propose a single model to retrieve demonstrations for a wide range of tasks by combining training signals from various tasks into a unified list-wise ranking formulation by language model’s feedback.
Outcome:	The proposed model outperforms baselines on 30+ tasks across 13 task families and multiple data domains.

Copied to clipboard

Junyi Zhu, Shuochen Liu, Yu Yu, Bo Tang, Yibo Yan, Zhiyu Li, Feiyu Xiong, Tong Xu, Matthew Blaschko

QAP: A Quantum-Inspired Adaptive-Priority-Learning Model for Multimodal Emotion Recognition (2023.findings-acl)

Challenge:	Large language models struggle with context awareness, leading to inaccuracies in tasks requiring faithful adherence to provided information.
Approach:	They propose a method to enhance LLMs' context awareness by updating only the last Feed-Forward Network module to maximize the likelihood of the prompt before inference .
Outcome:	The proposed method improves the accuracy of Llama 3-8B-Inst on the NQ-SWAP dataset from 59.1% to 71.6% and reduces the output structure failure rate of Qwen 1.5-4B-Chat from 34.9% to 25.5%.

Copied to clipboard

Ziming Li, Yan Zhou, Yaxin Liu, Fuqing Zhu, Chuanpeng Yang, Songlin Hu

Masks Can be Learned as an Alternative to Experts (2025.acl-long)

Challenge:	Experimental results show that multimodal emotion recognition is a state-of-the-art technique . textual, visual and acoustic modalities are involved in multimodal video emotion recognition .
Approach:	They propose a quantum-inspired adaptive-priority-learning model to address the challenges . they use quantum state to model modal features and Q-attention to integrate three modalities .
Outcome:	Experimental results show that QAP improves on previous models.

Copied to clipboard

Peiyu Liu, Tianwen Wei, Bo Zhu, Xin Zhao, Shuicheng Yan

Browse and Concentrate: Comprehending Multimodal Content via Prior-LLM Context Fusion (2024.acl-long)

Challenge:	a recent study shows that sparse activation techniques can reduce inference performance without sacrificing performance.
Approach:	They propose to sparsify a pre-trained dense large language model into a mixture-of-experts architecture for faster inference.
Outcome:	The proposed approach is more efficient than one-shot sparsification techniques . it achieves 97% performance retention on downstream tasks with only 50% of parameters activated .

Copied to clipboard

Ziyue Wang, Chi Chen, Yiqi Zhu, Fuwen Luo, Peng Li, Ming Yan, Ji Zhang, Fei Huang, Maosong Sun, Yang Liu

BioT5: Enriching Cross-modal Integration in Biology with Chemical Knowledge and Natural Language Associations (2023.emnlp-main)

Challenge:	Multimodal Large Language Models (MLLMs) lack understanding of multi-image and interleaved inputs due to the visual features encoded by frozen encoders before being fed into the LLM backbone.
Approach:	They propose a two phase paradigm to enable in-depth multimodal context fusion prior to feeding the features into LLMs.
Outcome:	The proposed paradigm boosts the performance on 7 multi-image scenarios, contributing to increments on average accuracy by 2.13% and 7.60% against strong MLLMs baselines with 3B and 11B LLMs, respectively.

Copied to clipboard

Qizhi Pei, Wei Zhang, Jinhua Zhu, Kehan Wu, Kaiyuan Gao, Lijun Wu, Yingce Xia, Rui Yan

DemonAgent: Dynamically Encrypted Multi-Backdoor Implantation Attack on LLM-based Agent (2025.findings-emnlp)

Challenge:	et al., 2022) argue that the current models for drug discovery lack the ability to integrate molecules, proteins, and natural language.
Approach:	They propose a framework that integrates biological knowledge with chemical knowledge and natural language associations.
Outcome:	The proposed framework shows superior performance across a wide range of tasks.

Copied to clipboard

Pengyu Zhu, Zhenhong Zhou, Yuanhe Zhang, Shilinlu Yan, Kun Wang, Sen Su

Pingan Smart Health and SJTU at COIN - Shared Task: utilizing Pre-trained Language Models and Common-sense Knowledge in Machine Reading Tasks (D19-60)

Challenge:	a new method for detecting advanced backdoors is proposed to bypass safety audits.
Approach:	They propose a backdoor implantation strategy that introduces dynamic encryption to bypass safety audits.
Outcome:	The proposed method achieves an attack success rate approaching 100% while maintaining a detection rate of 0%.

Copied to clipboard

Xiepeng Li, Zhexi Zhang, Wei Zhu, Zheng Li, Yuan Ni, Peng Gao, Junchi Yan, Guotong Xie

Parallelism and Generation Order in Masked Diffusion Language Models: Limits Today, Potential Tomorrow (2026.findings-acl)

Challenge:	Existing approaches to represent knowledge in the low-dimensional space are to leverage large-scale unsupervised text corpus to train fixed or contextual representations.
Approach:	They propose to leverage large-scale unsupervised text corpus to train fixed or contextual language representations and to express knowledge into a knowledge graph (KG) they incorporate distributional representations of a KG onto the representations from pre-trained language models, via simply concatenation or multi-head attention.
Outcome:	The proposed models outperform the other models on the COIN: COmmonsense INference in Natural Language Processing (COIN) Workshop datasets.

Copied to clipboard

Yangyang Zhong, Yanmei Gu, Zhengqing Zang, Xiaomeng Li, Yuqi Ding, Xibei Jia, Yuting Shen, Zhenzhong Lan, Liwang Zhu, Weiping Liu, Junlin Zhou, Haisheng Liu, Zhong Xin Yu, Pengxin Luo, Donglian Qi, Yunfeng Yan, Junbo Zhao

TencentPretrain: A Scalable and Flexible Toolkit for Pre-training Models of Different Modalities (2023.acl-demo)

Challenge:	Autoregressive (AR) language models dominate modern natural language processing due to strong likelihood-based training objectives and reliable left-to-right decoding.
Approach:	They characterize MDLM behavior along two dimensions: parallelism strength and generation order . authors propose a Generate-then-Edit paradigm that mitigates dependency loss .
Outcome:	The proposed model improves on tasks that require "backward information" the Generate-then-Edit paradigm improves parallel decoding efficiency while reducing dependency loss.

Copied to clipboard

Zhe Zhao, Yudong Li, Cheng Hou, Jing Zhao, Rong Tian, Weijie Liu, Yiren Chen, Ningyuan Sun, Haoyan Liu, Weiquan Mao, Han Guo, Weigang Gou, Taiqiang Wu, Tao Zhu, Wenhang Shi, Chen Chen, Shan Huang, Sihong Chen, Liqun Liu, Feifei Li, Xiaoshuai Chen, Xingwu Sun, Zhanhui Kang, Xiaoyong Du, Linlin Shen, Kimmo Yan

TS-SQL: Test-driven Self-refinement for Text-to-SQL (2025.findings-emnlp)

Challenge:	Several pre-training models of different modalities are showing a rising trend of homogeneity in their model structures.
Approach:	They propose a toolkit that supports pre-training models of different modalities.
Outcome:	The proposed toolkit can match the performance of the original implementations on text, vision, and audio benchmarks.

Copied to clipboard

Wenbo Xu, Haifeng Zhu, Liang Yan, Chuanyi Liu, Peiyi Han, Shaoming Duan, Jeff Z. Pan

Programming Languages in NLP

Challenge:	null
Approach:	null
Outcome:	null

SLAM-Omni: Timbre-Controllable Voice Interaction System with Single-Stage Training (2025.findings-acl)

Copied to clipboard

Wenxi Chen, Ziyang Ma, Ruiqi Yan, Yuzhe Liang, Xiquan Li, Ruiyang Xu, Zhikang Niu, Yanqiao Zhu, Yifan Yang, Zhanxun Liu, Kai Yu, Yuxuan Hu, Jinyu Li, Yan Lu, Shujie Liu, Xie Chen

Speech & Audio in NLP

Challenge:	a new spoken dialogue system with single-stage training is demonstrating its low latency and high quality . SLAM-Omni achieves zero-shot timbre control by modeling spoken language with semantic tokens .
Approach:	They propose a timbre-controllable, end-to-end voice interaction system with single-stage training.
Outcome:	The proposed system outperforms previous models on 4 GPUs with limited data.

Intelligent Document Parsing: Towards End-to-end Document Parsing via Decoupled Content Parsing and Layout Grounding (2025.findings-emnlp)

Copied to clipboard

Hangdi Xing, Feiyu Gao, Qi Zheng, Zhaoqing Zhu, Zirui Shao, Ming Yan

LLM Agents for Education: Advances and Applications (2025.findings-emnlp)

Challenge:	Existing methods fragment document parsing into pipeline of separated subtasks, resulting in incomplete semantics and error propagation.
Approach:	They propose an end-to-end document parsing framework that leverages vision-language priors of MLLMs.
Outcome:	The proposed method surpasses existing methods significantly in document parsing . it leverages the vision-language priors of MLLMs to decouple parse and layout grounding based on visual information.

Copied to clipboard

Zhendong Chu, Shen Wang, Jian Xie, Tinghui Zhu, Yibo Yan, Jingheng Ye, Aoxiao Zhong, Xuming Hu, Jing Liang, Philip S. Yu, Qingsong Wen

Same Claim, Different Judgment: Benchmarking Scenario-Induced Bias in Multilingual Financial Misinformation Detection (2026.findings-acl)

Challenge:	Large Language Model (LLM) agents are transforming education by automating complex tasks and enhancing both teaching and learning processes.
Approach:	This survey analyzes recent advances in applying Large Language Model agents to educational settings . it highlights ethical issues, hallucination and overreliance, and integration with existing ecosystems .
Outcome:	The authors analyze the technologies enabling LLM agents and highlight key challenges in deploying them in educational settings.

Copied to clipboard

Zhiwei Liu, Yupeng Cao, Yuechen Jiang, Mohsinul Kabir, Polydoros Giannouris, Chen Xu, Ziyang Xu, Tianlei Zhu, Md. Tariquzzaman, Triantafillos Papadopoulos, Yan Wang, Lingfei Qian, Xueqing Peng, Zhuohan Xie, Ye Yuan, Saeed Almheiri, Abdulrazzaq Alnajjar, Ming-Bin Chen, Harry Stuart, Paul Thompson, Prayag Tiwari, Alejandro Lopez-Lira, Xue Liu, Jimin Huang, Sophia Ananiadou

Think-Search-Patch: A Retrieval-Augmented Reasoning Framework for Repository-Level Code Repair (2025.emnlp-industry)

Challenge:	Existing research on LLM biases has focused on direct questioning or general-purpose settings . pronounced behavioral biase despite their growing deployment in financial analysis, forecasting, and decision support.
Approach:	They propose a benchmark to evaluate behavioral biases of large language models in MFMD . they use a multilingual financial misinformation dataset to integrate these with misinformation claims .
Outcome:	The proposed benchmark evaluates behavioral biases of large language models across economic scenarios.

Copied to clipboard

Bojian Xiong, Yikun Lei, Xikai Liu, Shaowei Zhang, Pengyun Zhu, Yan Liu, Yongqi Leng, Ling Shi, Meizhi Zhong, Yurong Zhang, Yan Gao, null Yiwu, Yao Hu, Deyi Xiong

Information Retrieval

Challenge:	Large language models suffer from multiple-file coding scenarios with strong inter-file dependencies . experimental results show that large language models exhibit inadequate performance in multi-file scenarios .
Approach:	They propose a retrieval-augmented reasoning framework for repository-level code repair . they use a dataset to generate standardized patches based on the key snippets .
Outcome:	The proposed framework improves retrieval accuracy and repair success on SWE-bench Lite . it surpasses models with larger size in managing extensive code contexts and fixing bugs spanning across multiple files.

M³GQA: A Multi-Entity Multi-Hop Multi-Setting Graph Question Answering Benchmark (2025.acl-long)

Copied to clipboard

Boci Peng, Yongchao Liu, Xiaohe Bo, Jiaxin Guo, Yun Zhu, Xuanbo Fan, Chuntao Hong, Yan Zhang

Reinforcement Learning on Pre-Training Data (2026.acl-long)

Challenge:	GraphRAG systems have achieved remarkable progress in enhancing performance and reliability of large language models.
Approach:	They propose a GraphRAG benchmark focusing on multi-entity queries with six settings for comprehensive evaluation.
Outcome:	The proposed method can construct diverse data with semantically correct ground-truth reasoning paths.

Copied to clipboard

Siheng Li, Kejiao Li, Zenan Xu, Guanhua Huang, Kun Li, Haoyuan Wu, null Wujiajia, Zihao Zheng, Chenchen Zhang, Kun Shi, Xue Gong, Qi Yi, Ruibin Xiong, Tingqiang Xu, Yuhao Jiang, Jianfeng Yan, Yuyuan Zeng, Guanghui Xu, Jinbao Xue, Zhijiang xu, Zheng Fang, Shuai LI, Qibin Liu, Xiaoxue Li, Zhuoyu Li, Yangyu Tao, Fei Gao, Cheng Jiang, Bochao Wang, Kai Liu, Jianchen Zhu, Wai Lam, Bo Zhou, Di Wang

StraGo: Harnessing Strategic Guidance for Prompt Optimization (2024.findings-emnlp)

Challenge:	Recent progress in large language models is driven by scaling of training compute through pre-training with nexttoken prediction (NTP) or post-training (RL) Pre-training using NTP enables models to acquire extensive knowledge and skills from general data, but it suffers from data inefficiency and catastrophic forgetting in continual learning settings.
Approach:	They propose to scale training compute through pre-training with next-token prediction (NTP) or post-training by scaling reinforcement learning (RL) to improve learning from general data.
Outcome:	Experiments on multiple benchmarks and models show that the proposed approach improves continual pre-training and provides a strong foundation for post-training on Qwen3-8B-Base.

Copied to clipboard

Yurong Wu, Yan Gao, Bin Zhu, Zineng Zhou, Xiaodi Sun, Sheng Yang, Jian-Guang Lou, Zhiming Ding, Linjun Yang

DetectBench: Can Large Language Model Detect and Piece Together Implicit Evidence? (2024.findings-emnlp)

Challenge:	Existing methods for prompt optimization often lead to prompt drifting, wherein newly generated prompts canadversely impact previously successful cases while addressing failures.
Approach:	They propose a method to mitigate prompt drifting by integrating in-context learning to formulate specific, actionable strategies for prompt optimization.
Outcome:	The proposed approach mitigates prompt drifting by leveraging insights from both successful and failed cases to identify critical factors for achieving optimization objectives.

Copied to clipboard

Zhouhong Gu, Lin Zhang, Xiaoxuan Zhu, Jiangjie Chen, Wenhao Huang, Yikai Zhang, Shusen Wang, Zheyu Ye, Yan Gao, Hongwei Feng, Yanghua Xiao

Beyond Prompting: An Efficient Embedding Framework for Open-Domain Question Answering (2025.acl-long)

Challenge:	Existing LLMs' abilities to detect evidence in long contexts are far inferior to humans.
Approach:	They propose a benchmark to assess LLMs' abilities in evidence and multi-step commonsense reasoning within a long context.
Outcome:	The proposed method improves the performance of LLMs in evidence detection and commonsense reasoning.

Copied to clipboard

Zhanghao Hu, Hanqi Yan, Qinglin Zhu, Zhenyi Shen, Yulan He, Lin Gui

CNNSum: Exploring Long-Context Summarization with Large Language Models in Chinese Novels (2025.findings-acl)

Challenge:	Large language models (LLMs) have recently pushed open-domain question answering (ODQA) to new heights.
Approach:	They propose an embedding-level framework that enhances both the retriever and the reader by reordering query representations via lightweight linear layers under an unsupervised contrastive learning objective.
Outcome:	The proposed framework outperforms baselines in accuracy and efficiency across three open-source LLMs, three retrieval methods, and four ODQA benchmarks.

Copied to clipboard

Lingxiao Wei, He Yan, Lu Xiangju, Junmin Zhu, Jun Wang, Wei Zhang

ImaginE: An Imagination-Based Automatic Evaluation Metric for Natural Language Generation (2023.findings-eacl)

Challenge:	Currently, long-context summarization mainly relies on memory ability.
Approach:	They propose a multi-scale long-context summarization benchmark based on Chinese novels . they use human-driven annotations to analyze long-constituency models .
Outcome:	The proposed benchmark features human-driven annotations across four subsets with lengths ranging from 16k to 128k.

Copied to clipboard

Wanrong Zhu, Xin Wang, An Yan, Miguel Eckstein, William Yang Wang

Living in the Moment: Can Large Language Models Grasp Co-Temporal Reasoning? (2024.acl-long)

Challenge:	Existing evaluation methods for natural language generation rely on token-level or embedding-level comparisons with text references.
Approach:	They propose to use text-to-image generator to generate an image as the embodied imagination for the text snippet and compute the imagination similarity using contextual embeddings.
Outcome:	The proposed metric improves existing evaluation metrics’ correlations with human similarity judgments in both reference-based and reference-free scenarios.

Copied to clipboard

Zhaochen Su, Juntao Li, Jun Zhang, Tong Zhu, Xiaoye Qu, Pan Zhou, Yan Bowen, Yu Cheng, Min Zhang

Reasoning

Challenge:	Current temporal reasoning datasets are limited to questions about single or isolated events, falling short in mirroring the realistic temporal characteristics involving concurrent nature and intricate temporal interconnections.
Approach:	They propose a co-temporal Question Answering benchmark that contains four co-time scenarios with 4,748 samples for evaluating the co-timing abilities of large language models.
Outcome:	The proposed benchmarks show that current LLMs struggle on CoTempQA tasks even when enhanced with Chain of Thought methodologies.

Learning Goal-oriented Dialogue Policy with opposite Agent Awareness (2020.aacl-main)

Copied to clipboard

Zheng Zhang, Lizi Liao, Xiaoyan Zhu, Tat-Seng Chua, Zitao Liu, Yan Huang, Minlie Huang

Dialogue Systems & Conversational Agents

Challenge:	Existing approaches for goal-oriented dialogue policy learning focus on the target agent policy and treat the opposite agent policy as part of the environment.
Approach:	They propose a framework for policy learning in goal-oriented dialogues that uses the opposite agent's policy estimation to improve the target agent by regarding it as part of the target policy.
Outcome:	The proposed framework shows superior performance over state-of-the-art models on cooperative and competitive dialogue tasks.

Translating Headers of Tabular Data: A Pilot Study of Schema Translation (2021.emnlp-main)

Copied to clipboard

Kunrui Zhu, Yan Gao, Jiaqi Guo, Jian-Guang Lou

Multilinguality

Challenge:	Schema translation is not well studied in the community because of morphological difference and context difference between plain text and tabular data.
Approach:	They propose a schema translation model augmented with schema context . they model a target header and its context as a directed graph to represent their entities .
Outcome:	The proposed model outperforms state-of-the-art models on schema translation . it uses a graph to represent entity types and relations, and a relational-aware transformer .

Towards Effective and Efficient Continual Pre-training of Large Language Models (2025.acl-long)

Copied to clipboard

Jie Chen, Zhipeng Chen, Jiapeng Wang, Kun Zhou, Yutao Zhu, Jinhao Jiang, Yingqian Min, Xin Zhao, Zhicheng Dou, Jiaxin Mao, Yankai Lin, Ruihua Song, Jun Xu, Xu Chen, Rui Yan, Zhewei Wei, Di Hu, Wenbing Huang, Ji-Rong Wen

Enhancing Multimodal Unified Representations for Cross Modal Generalization (2025.findings-acl)

Challenge:	Continual pre-training (CPT) has been an important approach for adapting language models to specific domains or tasks.
Approach:	They propose a Continual pre-training method that can greatly improve Chinese language ability and scientific reasoning ability of LLMs.
Outcome:	The proposed method can greatly improve Chinese language ability and scientific reasoning ability of LLMs.

Copied to clipboard

Hai Huang, Yan Xia, Shengpeng Ji, Shulei Wang, Hanting Wang, Minghui Fang, Jieming Zhu, Zhenhua Dong, Sashuai Zhou, Zhou Zhao

CFBench: A Comprehensive Constraints-Following Benchmark for LLMs (2025.acl-long)

Challenge:	Existing studies on discrete unified representations overlook important distinctions between different dimensions of features.
Approach:	They propose to use a codebook to optimize unified representations from pretraining and fine- and coarse-grained disentangling to optimize the representations.
Outcome:	The proposed methods improve the interpretability of multimodal unified representations . they use training-free optimization of codebook and fine and coarse cross-modal disentangling .

Copied to clipboard

Tao Zhang, ChengLIn Zhu, Yanjun Shen, Wenjing Luo, Yan Zhang, Hao Liang, Tao Zhang, Fan Yang, Mingan Lin, Yujing Qiao, Weipeng Chen, Bin Cui, Wentao Zhang, Zenan Zhou

WebCPM: Interactive Web Search for Chinese Long-form Question Answering (2023.acl-long)

Challenge:	Existing evaluations of Large Language Models (LLMs) focus on fragmented constraints or narrow scenarios, but they overlook the comprehensiveness and authenticity of constraints from the user’s perspective.
Approach:	They propose a Chinese Comprehensive Constraints Following Benchmark for LLMs that compiles constraints from real-world instructions and constructs a systematic framework for constraint types.
Outcome:	The proposed framework integrates multi-dimensional assessment criteria with requirement prioritization, covering various perspectives of constraints, instructions, and requirement fulfillment.

Copied to clipboard

Yujia Qin, Zihan Cai, Dian Jin, Lan Yan, Shihao Liang, Kunlun Zhu, Yankai Lin, Xu Han, Ning Ding, Huadong Wang, Ruobing Xie, Fanchao Qi, Zhiyuan Liu, Maosong Sun, Jie Zhou

Information Retrieval

Challenge:	Long-form question answering requires two procedures: information retrieval and information synthesis.
Approach:	They propose a Chinese long-form question answering dataset called WebCPM . the dataset is based on a web search interface that engages with a search engine in real time .
Outcome:	The proposed dataset generates answers that are no worse than human-written ones . the dataset is the first Chinese LFQA dataset .

Neural Topic Modeling via Contextual and Graph Information Fusion (2025.emnlp-main)

Copied to clipboard

Jiyuan Liu, Jiaxing Yan, Chunjiang Zhu, Xingyu Liu, Li Qing, Yanghui Rao

AMPO: Automatic Multi-Branched Prompt Optimization (2024.emnlp-main)

Challenge:	Existing topic models generate uninformative and incoherent topics that hinder interpretable insights from managing textual data.
Approach:	They propose to incorporate contextual and graph information to improve the variational autoencoder framework by combining contextual and bag-of-words information.
Outcome:	The proposed framework generates more coherent and diverse topics on three benchmark datasets and achieves strong performance on automatic and manual evaluations.

Copied to clipboard

Sheng Yang, Yurong Wu, Yan Gao, Zineng Zhou, Bin Zhu, Xiaodi Sun, Jian-Guang Lou, Zhiming Ding, Anbang Hu, Yuan Fang, Yunsong Li, Junyan Chen, Linjun Yang

GenWebNovel: A Genre-oriented Corpus of Entities in Chinese Web Novels (2025.coling-main)

Challenge:	Existing prompt engineering techniques are limited to producing single flow instructions, struggling with handling diverse patterns.
Approach:	They propose an automatic prompt optimization method that iteratively develops a multi-branched prompt using failure cases as feedback.
Outcome:	The proposed method achieves the best results across five tasks and demonstrates significant optimization efficiency due to adoption of a minimal search strategy.

Copied to clipboard

Hanjie Zhao, Yuchen Yan, Senbin Zhu, Hongde Liu, Yuxiang Jia, Hongying Zan, Min Peng

Unleashing the Potential of Large Language Models through Spectral Modulation (2024.findings-emnlp)

Challenge:	Existing literature on nested entity recognition is insufficient partly due to insufficient annotated data.
Approach:	They propose a method that utilizes a pre-trained language model as an In-context learning example retriever to boost the performance of large language models.
Outcome:	The proposed method significantly enhances entity recognition, matching state-of-the-art (SOTA) models without additional training data.

Copied to clipboard

Peng Sun, Yao Zhu, Yunjian Zhang, Xiu Yan, Zizhe Wang, Xiangyang Ji

Fast Retrieval and Slow Reasoning for Explainable Multimodal Sentiment Analysis (2026.findings-acl)

Challenge:	Large Language Models (LLMs) have demonstrated impressive capabilities across various domains, garnering significant attention from both academia and industry.
Approach:	They propose to conduct spectral modulation in the parameter space of LLMs to integrate with various models in a plug-and-play manner.
Outcome:	The proposed approach improves performance by 10.12% with spectral modulation.

Copied to clipboard

Aoqiang Zhu, Min Hu, Yan Xing

Chain-of-Thought Compression Should Not Be Blind: V-Skip for Efficient Multimodal Reasoning via Dual-Path Anchoring (2026.acl-long)

Challenge:	Existing multimodal sentiment analysis methods rely on holistic fusion . such strategies introduce redundant information and obscure the decision process .
Approach:	They propose an interpretable framework that decomposes multimodal sentiment modeling into two cooperative pathways.
Outcome:	The proposed framework achieves competitive performance, higher efficiency, stronger robustness to noise, and clearer decision transparency than existing holistic fusion methods.

Copied to clipboard

Dongxu Zhang, Yiding Sun, Cheng Tan, Wenbiao Yan, Ning Yang, Jihua Zhu, Haijun Zhang

Detecting AI-Generated Content on Social Media with Multi-modal Language Models (2026.acl-industry)

Challenge:	Existing efforts to mitigate this via token compression fail due to its autoregressive nature . linguistically redundant tokens are erroneously pruned, leading to hallucinations .
Approach:	They propose a method that reformulates token pruning as a Visual-Anchored Information Bottleneck (VA-IB) optimization problem.
Outcome:	Experiments on Qwen2-VL and Llama-3.2 families show that the proposed model achieves a speedup with negligible accuracy loss.

Copied to clipboard

Chenyang Yang, Shen Yan, Yibo Yang, Litao Hu, Yuchen Liu, Yuan Zeng, Hanchao Yu, Yinan Zhu, Sumedha Singla, Brian Vanover, Huijun Qian, Zihao Wang, Fujun Liu, Aashu Singh, Jianyu Wang, Xuewen Zhang

Mirror: Multiple-perspective Self-Reflection Method for Knowledge-rich Reasoning (2024.acl-long)

Challenge:	Existing methods for AI-generated content detection face poor generalization to newer models, reliance on single modalities, and lack of interpretable explanations.
Approach:	They propose a model that curates diverse social media data and trains a vision-language model for detection and explanation.
Outcome:	The proposed model achieves state-of-the-art detection performance on public benchmarks and observes positive downstream impacts on user engagement.

Copied to clipboard

Hanqi Yan, Qinglin Zhu, Xinyu Wang, Lin Gui, Yulan He

Reasoning

Challenge:	Large language models (LLMs) struggle with knowledge-rich problems without external resources.
Approach:	They propose a Multiple-perspective self-reflection method that allows LLMs to reflect from multiple-perceptive clues, achieved through a heuristic interaction between a Navigator and a Reasoner.
Outcome:	The proposed method is superior to other self-reflection methods on five reasoning datasets.

Non-Autoregressive Neural Machine Translation with Consistency Regularization Optimized Variational Framework (2022.naacl-main)

Copied to clipboard

Minghao Zhu, Junli Wang, Chungang Yan

Machine Translation

Challenge:	Variational Autoencoder (VAE) is an effective framework to model the interdependency for non-autoregressive neural machine translation (NAT).
Approach:	They propose to use Variational Autoencoder to model interdependency for non-autoregressive neural machine translation (NAT) a posterior consistency regularization approach is proposed to improve translation quality .
Outcome:	The proposed model is 1.5/0.7 and 0.8/0.3 BLEU points faster than the baseline model.

COSMOS: Connectivity-Oriented Submodular Maximization for Optimal Subgraph Retrieval (2026.acl-long)

Copied to clipboard

Boci Peng, Xiao Liu, Boren Hu, Yun Zhu, Xuanbo Fan, Yanwei Yue, Chunyu Yang, Yan Zhang

Visualize Before You Write: Imagination-Guided Open-Ended Text Generation (2023.findings-eacl)

Challenge:	Existing paradigms treat facts independently or employ myopic search, failing to optimize collective subgraph utility.
Approach:	They propose a framework that formalizes evidence retrieval as a constrained submodular maximization problem.
Outcome:	The proposed framework captures the trade-off between information relevance and structural complexity.

Copied to clipboard

Wanrong Zhu, An Yan, Yujie Lu, Wenda Xu, Xin Wang, Miguel Eckstein, William Yang Wang

RecBase: Generative Foundation Model Pretraining for Zero-Shot Recommendation (2025.emnlp-main)

Challenge:	Existing tools for text-to-image synthesis can visualize machine imaginations for a given context.
Approach:	They propose a framework that uses machine-generated images to guide language models in open-ended text generation.
Outcome:	The proposed framework is effective on open-ended text generation tasks while showing minor degeneration.

Copied to clipboard

Sashuai Zhou, Weinan Gan, Qijiong Liu, Ke Lei, Jieming Zhu, Hai Huang, Yan Xia, Ruiming Tang, Zhenhua Dong, Zhou Zhao

Short Text Topic Modeling with Topic Distribution Quantization and Negative Sampling Decoder (2020.emnlp-main)

Challenge:	Existing methods for addressing item-level user interests are lacking in cross-domain generalization . RecBase model is domain-agnostic and can be used to enhance recommender systems' effectiveness .
Approach:	They propose a domain-agnostic foundational model pretrained with a recommendation-oriented objective that leverages a large-scale, heterogeneous, cross-domain corpus with unified textual representations and feature mappings to enhance cross- domain generalization.
Outcome:	The proposed model matches or surpasses baselines in zero-shot and cross-domain recommendation tasks on eight real-world datasets.

Copied to clipboard

Xiaobao Wu, Chunping Li, Yan Zhu, Yishu Miao

Generalized Category Discovery with Large Language Models in the Loop (2024.findings-acl)

Challenge:	Topic models for short texts suffer from data sparsity because of limited word co-occurrences.
Approach:	They propose a neural topic model with a new topic distribution quantization approach that generates peakier distributions that are more appropriate for modeling short texts.
Outcome:	The proposed model outperforms both strong traditional and neural baselines under extreme data sparsity scenes, producing high-quality topics.

Copied to clipboard

Wenbin An, Wenkai Shi, Feng Tian, Haonan Lin, QianYing Wang, Yaqiang Wu, Mingxiang Cai, Luyan Wang, Yan Chen, Haiping Zhu, Ping Chen

Is Cognition Consistent with Perception? Assessing and Mitigating Multimodal Knowledge Conflicts in Document Understanding (2025.emnlp-main)

Challenge:	Generalized Category Discovery (GCD) is a crucial task that aims to recognize both known and novel categories from a set of unlabeled data.
Approach:	They propose a framework that introduces Large Language Models into the training loop to generate category names without human effort.
Outcome:	The proposed framework outperforms SOTA models on three benchmark datasets and generates accurate category names for the discovered clusters.

Copied to clipboard

Zirui Shao, Feiyu Gao, Zhaoqing Zhu, Chuwei Luo, Hangdi Xing, Zhi Yu, Qi Zheng, Ming Yan, Jiajun Bu

DuQM: A Chinese Dataset of Linguistically Perturbed Natural Questions for Evaluating the Robustness of Question Matching Models (2022.emnlp-main)

Challenge:	Multimodal large language models (MLLMs) have shown impressive capabilities in document understanding due to different types of annotation noise in training.
Approach:	They propose a method to reduce C&P knowledge conflicts across all tested MLLMs . they propose to use annotation noise to train models to understand document content .
Outcome:	The proposed method reduces C&P knowledge conflicts across all tested MLLMs and enhances their performance in both cognitive and perceptual tasks.

Copied to clipboard

Hongyu Zhu, Yan Chen, Jing Yan, Jing Liu, Yu Hong, Ying Chen, Hua Wu, Haifeng Wang

SportQA: A Benchmark for Sports Understanding in Large Language Models (2024.naacl-long)

Challenge:	a comprehensive evaluation of QM models should be conducted on natural texts, not on artificial adversarial examples . ral models are often not robust to adversarials, which means they predict unexpected outputs .
Approach:	They use a Chinese dataset to evaluate the robustness of QM models . they show that the effect of artificial adversarial examples does not work on natural texts .
Outcome:	The proposed model is more robust than other models on natural questions with 32 linguistic perturbations.

Copied to clipboard

Haotian Xia, Zhengbang Yang, Yuqing Wang, Rhys Tracy, Yun Zhao, Dongdong Huang, Zezhi Chen, Yan Zhu, Yuan-fang Wang, Weining Shen

Revisiting Cross-Lingual Summarization: A Corpus-based Study and A New Benchmark with Improved Annotation (2023.acl-long)

Challenge:	SportQA is a benchmark specifically designed for evaluating Large Language Models (LLMs) sports knowledge is characterized by its fast pace, variety of types, abundance of strategies, and rich player narratives .
Approach:	They propose a benchmark specifically designed for evaluating Large Language Models in the context of sports understanding.
Outcome:	The proposed benchmark aims to bridge the gap between existing and specialized benchmarks in sports understanding.

Copied to clipboard

Yulong Chen, Huajian Zhang, Yijie Zhou, Xuefeng Bai, Yueguan Wang, Ming Zhong, Jianhao Yan, Yafu Li, Judy Li, Xianchao Zhu, Yue Zhang

Text Generation

Challenge:	Existing work on cross-lingual summarization (CLS) does not consider crosslingual sources for summarizing.
Approach:	They propose a cross-lingual conversation summarization benchmark that explicitly considers source context.
Outcome:	The proposed method surpasses baselines on ConvSumX and 3 widely-used manual annotations.

ConvLab-2: An Open-Source Toolkit for Building, Evaluating, and Diagnosing Dialogue Systems (2020.acl-demos)

Copied to clipboard

Qi Zhu, Zheng Zhang, Yan Fang, Xiang Li, Ryuichi Takanobu, Jinchao Li, Baolin Peng, Jianfeng Gao, Xiaoyan Zhu, Minlie Huang

Natural Language Interfaces

Challenge:	ConvLab-2 inherits Convlab's framework but integrates more powerful dialogue models and supports more datasets.
Approach:	They present ConvLab-2, an open-source toolkit that enables researchers to build task-oriented dialogue systems with state-of-the-art models and perform an end-to-end evaluation.
Outcome:	The new tool inherits ConvLab's framework and extends it by integrating many recently proposed state-of-the-art dialogue models.

Language Modeling with Sparse Product of Sememe Experts (D18-1)

Copied to clipboard

Yihong Gu, Jun Yan, Hao Zhu, Zhiyuan Liu, Ruobing Xie, Maosong Sun, Fen Lin, Leyu Lin

ATM: Adversarial Tuning Multi-agent System Makes a Robust Retrieval-Augmented Generator (2024.emnlp-main)

Challenge:	Existing language modeling methods rely on large-scale text data to learn the sequential patterns of words.
Approach:	They propose to use sememes to represent the implicit semantics behind words for language modeling . they propose to employ sememe-driven language models to fine-grained semem-level semantics .
Outcome:	Experiments on language modeling and the downstream application of headline generation show the effectiveness of SDLM.

Copied to clipboard

Junda Zhu, Lingyong Yan, Haibo Shi, Dawei Yin, Lei Sha

Unified Active Retrieval for Retrieval Augmented Generation (2024.findings-emnlp)

Challenge:	Large language models (LLMs) are proven to benefit a lot from retrieval-augmented generation (RAG) due to noisy and fabricating content, it is inevitable that RAG systems are vulnerable to these noises and prone to respond incorrectly.
Approach:	They propose to optimize retrieval-augmented generation (RGG) with an Adversarial Tuning Multi-agent system (ATM) ATM steers the Generator to have a robust perspective of useful documents for question answering with the help of an auxiliary Attacker agent.
Outcome:	The proposed system improves the retrieval-augmented generator with an auxiliary Attacker agent and can discriminate useful documents amongst fabrications.

Copied to clipboard

Qinyuan Cheng, Xiaonan Li, Shimin Li, Qin Zhu, Zhangyue Yin, Yunfan Shao, Linyang Li, Tianxiang Sun, Hang Yan, Xipeng Qiu

Information Retrieval

Challenge:	Existing active retrieval methods struggle with handling various types of instructions.
Approach:	They propose a unified active retrieval framework for retrieval-augmented generation . they propose to combine four orthogonal criteria into plug-and-play classification tasks .
Outcome:	The proposed framework outperforms existing methods on four representative types of user instructions on four types of instructions.

LIFBench: Evaluating the Instruction Following Performance and Stability of Large Language Models in Long-Context Scenarios (2025.acl-long)

Copied to clipboard

Xiaodong Wu, Minhao Wang, Yichen Liu, Xiaoming Shi, He Yan, Lu Xiangju, Junmin Zhu, Wei Zhang

Multimodal Text Style Transfer for Outdoor Vision-and-Language Navigation (2021.eacl-main)

Challenge:	Existing benchmarks rarely focus on instruction-following in long-context scenarios or stability on different inputs.
Approach:	They propose a scalable dataset to evaluate LLMs’ instruction-following capabilities and stability across long contexts.
Outcome:	The proposed method evaluates LLMs’ instruction-following capabilities and stability across long contexts.

Copied to clipboard

Wanrong Zhu, Xin Wang, Tsu-Jui Fu, An Yan, Pradyumna Narayana, Kazoo Sone, Sugato Basu, William Yang Wang

TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing (2021.acl-demo)

Challenge:	Outdoor vision-and-language navigation (VLN) tasks require visual grounding to generate correct actions.
Approach:	They propose a multimodal text style transfer learning approach to mitigate data scarcity in outdoor vision-and-language navigation tasks.
Outcome:	The proposed approach outperforms baseline models on the outdoor vision-and-language navigation task, improving task completion rate by 8.7% relative to the baseline models.

Copied to clipboard

Xiao Wang, Qin Liu, Tao Gui, Qi Zhang, Yicheng Zou, Xin Zhou, Jiacheng Ye, Yongxin Zhang, Rui Zheng, Zexiong Pang, Qinzhuo Wu, Zhengyan Li, Chong Zhang, Ruotian Ma, Zichu Fei, Ruijian Cai, Jun Zhao, Xingwu Hu, Zhiheng Yan, Yiding Tan, Yuan Hu, Qiyuan Bian, Zhihua Liu, Shan Qin, Bolin Zhu, Xiaoyu Xing, Jinlan Fu, Yue Zhang, Minlong Peng, Xiaoqing Zheng, Yaqian Zhou, Zhongyu Wei, Xipeng Qiu, Xuanjing Huang

BioT5+: Towards Generalized Biological Understanding with IUPAC Integration and Multi-task Tuning (2024.findings-acl)

Challenge:	Existing approaches to textual robustness evaluation focus on slightly modifying the input data, which maintains the original meaning and results in a different prediction.
Approach:	They propose a multilingual robustness evaluation toolkit for NLP that integrates universal text transformations, task-specific transformations and adversarial attack.
Outcome:	The toolkit includes universal text transformation, task-specific transformation, adversarial attack, subpopulation, and their combinations to provide comprehensive robustness analyses.

Copied to clipboard

Qizhi Pei, Lijun Wu, Kaiyuan Gao, Xiaozhuan Liang, Yin Fang, Jinhua Zhu, Shufang Xie, Tao Qin, Rui Yan

Divide-Then-Aggregate: An Efficient Tool Learning Method via Parallel Tool Invocation (2025.acl-long)

Challenge:	BioT5+ is an extension of the BioT5, but lacked a nuanced understanding of molecular structures.
Approach:	They propose a new bio-entity modeling framework, BioT5+, which integrates IUPAC names and molecule data.
Outcome:	The proposed model bridges the gap between molecular representations and textual descriptions and improves the grounded reasoning of bio-text and bio-sequences.

Copied to clipboard

Dongsheng Zhu, Weixian Shi, Zhengliang Shi, Zhaochun Ren, Shuaiqiang Wang, Lingyong Yan, Dawei Yin

Green & Sustainable NLP

Challenge:	Large Language Models (LLMs) demonstrate remarkable capabilities but their ability to autonomously execute complex real-world tasks remains limited.
Approach:	They propose a parallel tool invocation framework that decomposes tasks into parallel tool-using subtasks while aggregating results for subsequent decisions.
Outcome:	The proposed method significantly improves task performance while reducing token consumption and inference time.

ASTRA: An Automated Framework for Strategy Discovery, Retrieval, and Evolution for Jailbreaking LLMs (2026.acl-long)

Copied to clipboard

Xu Liu, Yan Chen, Kan Ling, Yichi Zhu, Hengrun Zhang, Guisheng Fan, Huiqun Yu