Papers by Yu Duan

Enhanced Data Synthesis for LLM through Reasoning Structures Generated by Hierarchical GFlowNet (2025.findings-acl)

Challenge:	Object categories are typically organized into a multi-granularity taxonomic hierarchy . traditional uni-modal approaches focus primarily on image features, revealing limitations in complex scenarios.
Approach:	They propose a framework that combines vision-language models with a deeper exploitation of the hierarchy.
Outcome:	The proposed framework shows significant improvements on 11 diverse visual recognition benchmarks.

Copied to clipboard

Tianpeng Bu, Minying Zhang, Hongtao Duan, Shurui Li, Lulu Hu, Yu Li

Reasoning

Challenge:	Existing methods to optimize instruction-response pairs lack a systematic design for the underlying reasoning structure.
Approach:	They propose a Reasoning Structure driven data Synthesis method that leverages a coarse-to-fine directed acyclic graph to construct reasoning structures efficiently.
Outcome:	The proposed method outperforms existing methods in 48.50%, 84.00%, 79.90% of the synthetic datasets trained on the proposed model.

PDC & DM-SFT: A Road for LLM SQL Bug-Fix Enhancing (2025.coling-industry)

Copied to clipboard

Yiwen Duan, Yonghong Yu, Xiaoming Zhao, Yichang Wu, Wenbo Liu

Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models (2026.findings-acl)

Challenge:	Existing code large language models focus on generating correct code, but struggle with bug repair.
Approach:	They propose a set of methods to enhance LLM’s SQL bug-fixing abilities by combining a data set construction and a supervised bug-fixed learning approach.
Outcome:	The proposed methods exceed current best performing model which size is much larger.

Copied to clipboard

Hengyuan Zhang, Zhihao Zhang, Ercong Nie, Mingyang Wang, Zunhai Su, Yiwei Wang, Qianli Wang, Shuzhou Yuan, Xufeng Duan, Qibo Xue, Zeping Yu, Chenming Shang, Xiao Liang, Jing Xiong, Hui Shen, Chaofan Tao, Zhengwu Liu, Senjie Jin, Zhiheng Xi, Dongdong Zhang, Sophia Ananiadou, Tao Gui, Ruobing Xie, Hayden Kwok-Hay So, Hinrich Schuetze, Xuanjing Huang, Qi Zhang, Ngai Wong

Explainability & Interpretability in NLP

Challenge:	Existing literature on mechanistic interpretation (MI) treats it as an observational science, leaving practical applications underexplored.
Approach:	They propose a survey structured around the pipeline to identify and improve MI models.
Outcome:	The proposed framework enables tangible improvements in Alignment, Capability, and Efficiency.

A Deep Relevance Model for Zero-Shot Document Filtering (P18-1)

Copied to clipboard

Chenliang Li, Wei Zhou, Feng Ji, Yu Duan, Haiqing Chen

Semantic Text Processing

Challenge:	Existing methods for document classification do not consider document filtering . existing methods do not include document filter.
Approach:	They propose a deep relevance model for zero-shot document filtering called DAZER . they use word embeddings to extract the relevance signals from word embeds .
Outcome:	The proposed model outperforms existing models on two document collections . it estimates the relevance between a document and a category by using seed words .

ProphetNet: Predicting Future N-gram for Sequence-to-SequencePre-training (2020.findings-emnlp)

Copied to clipboard

Weizhen Qi, Yu Yan, Yeyun Gong, Dayiheng Liu, Nan Duan, Jiusheng Chen, Ruofei Zhang, Ming Zhou

PlotGen-Bench: Evaluating VLMs on Generating Visualization Code from Diverse Plots across Multiple Libraries (2026.findings-acl)

Challenge:	Existing sequence-to-sequence models are optimized for future n-gram prediction and n stream self-attention mechanism.
Approach:	They propose a self-supervised objective called future n-gram prediction and the proposed n stream self-attention mechanism to optimize the model for sequence-to-sequence learning.
Outcome:	The proposed model achieves state-of-the-art on CNN/DailyMail, Gigaword, and SQuAD 1.1 benchmarks compared to the models using the same scale pre-training corpus.

Copied to clipboard

Yi Zhao, Zhen Yang, Shuaiqi Duan, Wenmeng Yu, Zhe Su, Jibing Gong, Jie Tang

ObfusLM: Privacy-preserving Language Model Service against Embedding Inversion Attacks (2025.acl-long)

Challenge:	PlotGen-Bench evaluates vision-language models' ability to generate executable visualization code from plots under realistic and complex visualization requirements.
Approach:	They propose a benchmark to evaluate plot-to-code generation in vision-language models . they use Matplot, Matplos, Mat3D, Mat4D, and Mat4E to evaluate their performance .
Outcome:	The proposed benchmark covers 9 major categories, 30 subcategories, and 3 core tasks . it covers 2D, 3D and animated plots across 5 widely used visualization libraries.

Copied to clipboard

Yu Lin, Ruining Yang, Yunlong Mao, Qizhi Zhang, Jue Hong, Quanwei Cai, Ye Wu, Huiqi Liu, Zhiyu Chen, Bing Duan, Sheng Zhong

Camouflaged Chinese Spam Content Detection with Semi-supervised Generative Active Learning (2020.acl-main)

Challenge:	Recent studies show that obfuscation techniques for MLaaS are susceptible to embedding inversion attacks (EIAs).
Approach:	They propose a model obfuscation framework that protects client inputs from embedding inversion attacks by obliviously obbing models.
Outcome:	The proposed framework outperforms existing works in utility by 10% with a nearly 80% resistance rate against embedding inversion attacks.

Copied to clipboard

Zhuoren Jiang, Zhe Gao, Yu Duan, Yangyang Kang, Changlong Sun, Qiong Zhang, Xiaozhong Liu

MdEval: Massively Multilingual Code Debugging (2026.findings-acl)

Challenge:	Existing active learning models for text spam detection tasks are based on pool-based active learning, but the annotating process is laborious and time consuming for humans.
Approach:	They propose a semi-supervised active learning model to address spam imbalances . they propose masked attention learning approach and character variation graph-enhanced augmentation procedure .
Outcome:	The proposed model can improve the performance of existing models for Chinese spam detection task.

Copied to clipboard

Shukai Liu, Linzheng Chai, Jian Yang, Jiajun Shi, He Zhu, Liran Wang, Jin Ke, Wei Zhang, Hualei Zhu, Shuyue Guo, Tao Sun, Jiaheng Liu, Yunlong Duan, Yu Hao, Liqun Yang, Guanglin Niu, Ge Zhang, Zhoujun Li

MoE-I2: Compressing Mixture of Experts Models through Inter-Expert Pruning and Intra-Expert Low-Rank Decomposition (2024.findings-emnlp)

Challenge:	Existing benchmarks primarily focus on Python and are limited in terms of language diversity.
Approach:	They propose a multilingual debugging benchmark that includes 3.9K test samples of 20 programming languages and introduces the debug instruction corpora MdEval-Instruct by injecting bugs into the correct multilingual queries and solutions.
Outcome:	The proposed benchmark includes 3.9K test samples of 20 programming languages and covers the automated program repair task, bug localization task, and bug identification task.

Copied to clipboard

Cheng Yang, Yang Sui, Jinqi Xiao, Lingyi Huang, Yu Gong, Yuanlin Duan, Wenqi Jia, Miao Yin, Yu Cheng, Bo Yuan

Don’t Get Lost in the Trees: Streamlining LLM Reasoning by Overcoming Tree Search Exploration Pitfalls (2025.acl-long)

Challenge:	emergence of Mixture of Experts (MoE) LLMs has significantly advanced the development of language models.
Approach:	They propose a two-stage compression method tailored for Mixture of Experts to reduce the model size and decrease the computational cost.
Outcome:	The proposed method reduces model size and improves inference efficiency while maintaining performance in various zero-shot tasks.

Copied to clipboard

Ante Wang, Linfeng Song, Ye Tian, Dian Yu, Haitao Mi, Xiangyu Duan, Zhaopeng Tu, Jinsong Su, Dong Yu

Green & Sustainable NLP

Challenge:	Recent advances in tree search algorithms guided by verifiers have significantly enhanced the reasoning capabilities of large language models (LLMs), but at the cost of increased computational resources.
Approach:	They propose an e ffici ent tree sear ch framework that is a plug-and-play system compatible with various tree search algorithms.
Outcome:	The proposed framework reduces computational costs and prioritizes resource allocation to harder tasks (Levels 3-4) over simpler ones (Level 1-2), addressing both over-exploration in basic problems and under-exploation in complex cases.

ProphetNet-X: Large-Scale Pre-training Models for English, Chinese, Multi-lingual, Dialog, and Code Generation (2021.acl-demo)

Copied to clipboard

Weizhen Qi, Yeyun Gong, Yu Yan, Can Xu, Bolun Yao, Bartuer Zhou, Biao Cheng, Daxin Jiang, Jiusheng Chen, Ruofei Zhang, Houqiang Li, Nan Duan

DSQG-Syn: Synthesizing High-quality Data for Text-to-SQL Parsing by Domain Specific Question Generation (2025.findings-naacl)

Challenge:	Existing models for pre-training are not convenient for users to find and set them up.
Approach:	They propose to extend ProphetNet into other domains and languages by pre-training models . they pre-train a cross-lingual generation model ProphetNet-Multi and a Chinese generation model .
Outcome:	The proposed models achieve new state-of-the-art on 10 benchmarks.

Copied to clipboard

Shaoming Duan, Youxuan Wu, Chuanyi Liu, Yuhao Zhang, Zirui Wang, Peiyi Han, Shengyuan Yu, Liang Yan, Yingwei Liang

Programming Languages in NLP

Challenge:	Existing methods for generating SQL queries using natural language questions produce inconsistent NLQ-SQL pairs.
Approach:	They propose a text-to-SQL data synthesis framework that generates domain-relevant questions . they synthesize NLQ-SqL pairs that are domain-specific and intent-consistent .
Outcome:	The proposed method outperforms closed-source LLMs on the Text-to-SQL task.

FastSeq: Make Sequence Generation Faster (2021.acl-demo)

Copied to clipboard

Yu Yan, Fei Hu, Jiusheng Chen, Nikhil Bhendawade, Ting Ye, Yeyun Gong, Nan Duan, Desheng Cui, Bingyu Chi, Ruofei Zhang

GLGE: A New General Language Generation Evaluation Benchmark (2021.findings-acl)

Challenge:	Transformer-based models have made tremendous impact in natural language generation, but inference speed is still a bottleneck due to large model size and intensive computing involved in auto-regressive decoding process.
Approach:	They propose an attention cache optimization, an efficient algorithm for detecting repeated n-grams, and an asynchronous generation pipeline with parallel I/O to accelerate sequence generation without loss of accuracy.
Outcome:	The proposed framework can accelerate the sequence generation by 4x to 9x with a simple one-line code change for a set of widely used and diverse models.

Copied to clipboard

Dayiheng Liu, Yu Yan, Yeyun Gong, Weizhen Qi, Hang Zhang, Jian Jiao, Weizhu Chen, Jie Fu, Linjun Shou, Ming Gong, Pengcheng Wang, Jiusheng Chen, Daxin Jiang, Jiancheng Lv, Ruofei Zhang, Winnie Wu, Ming Zhou, Nan Duan

Chunks as Arms: Multi-Armed Bandit-Guided Sampling for Long-Context LLM Preference Optimization (2026.acl-long)

Challenge:	Multi-task benchmarks focus on a range of Natural Language Understanding (NLU) tasks without considering the Natural Language Generation (NLG) models.
Approach:	They propose a multi-task benchmark for evaluating the generalization capabilities of NLG models across eight language generation tasks.
Outcome:	The proposed benchmarks are based on GLUE and Su-perGLUE for English and several other languages.

Copied to clipboard

Shaohua Duan, Pengcheng Huang, Xinze Li, Zhenghao Liu, Xiaoyuan Yi, Yukun Yan, Shuo Wang, Yu Gu, Ge Yu, Maosong Sun

Reasoning

Challenge:	Recent studies have explored fine-tuning Large Language Models with synthetic data to enhance their long-context capabilities.
Approach:	They propose a framework that leverages a Multi-Armed Bandit rollout strategy to identify the most informative chunks from the given long context for sampling high-quality and diverse responses.
Outcome:	The proposed framework achieves 4% improvement on long-context reasoning benchmarks on Llama and Qwen.

RikiNet: Reading Wikipedia Pages for Natural Question Answering (2020.acl-main)

Copied to clipboard

Dayiheng Liu, Yeyun Gong, Jie Fu, Yu Yan, Jiusheng Chen, Daxin Jiang, Jiancheng Lv, Nan Duan

Natural Language Interfaces

Challenge:	Using Wikipedia pages to answer open-domain questions remains challenging in natural language understanding.
Approach:	They propose a model which reads Wikipedia pages for natural question answering . it uses a dynamic paragraph dual-attention reader and a cascaded answer predictor .
Outcome:	The proposed model outperforms the human model on the Natural Questions dataset . it achieves 74.3 F1 and 57.9 F1 on long-answer and short-answer tasks .

Projecting Out the Malice: A Global Subspace Approach to LLM Detoxification (2026.acl-long)

Copied to clipboard

Zenghao Duan, Zhiyi Yin, Zhichao Shi, Liang Pang, Shaoling Jing, Zihe Huang, Jiayi Wu, Yu Yan, Jingcheng Deng, Huawei Shen, Xueqi Cheng

Challenge:	Large language models (LLMs) exhibit exceptional performance but pose inherent risks of generating toxic content.
Approach:	They propose a method that removes toxic subspaces from FFN parameters . they propose to use a lightweight method to eliminate toxic subespaces .
Outcome:	The proposed method achieves SOTA detoxification while preserving general capabilities without large-scale retraining.

Contrastive Attention Mechanism for Abstractive Sentence Summarization (D19-1)

Copied to clipboard

Xiangyu Duan, Hongfei Yu, Mingming Yin, Min Zhang, Weihua Luo, Yue Zhang

Summarization

Challenge:	Existing attention mechanisms for abstractive sentence summarization are based on rule-based methods and large-scale training corpora.
Approach:	They propose a contrastive attention mechanism that extends the sequence-to-sequence framework for abstractive sentence summarization task.
Outcome:	The proposed mechanism improves the state-of-the-art on the abstractive sentence summarization task.

An Inversion Attack Against Obfuscated Embedding Matrix in Language Model Inference (2024.emnlp-main)

Copied to clipboard

Yu Lin, Qizhi Zhang, Quanwei Cai, Jue Hong, Wu Ye, Huiqi Liu, Bing Duan

Semantic Text Processing

Challenge:	Recent studies have explored transforming user inputs to obfuscated embedded vectors, so that the data will not be eavesdropped by service providers.
Approach:	They propose to transform user inputs to obfuscated embedded vectors so that the data will not be eavesdropped by service providers.
Outcome:	The proposed inversion attack can recover user input 100% from the obfuscated vectors without a solid and deliberate security design and analysis .

from Benign import Toxic: Jailbreaking the Language Model via Adversarial Metaphors (2025.acl-long)

Copied to clipboard

Yu Yan, Sheng Sun, Zenghao Duan, Teli Liu, Min Liu, Zhiyi Yin, LeiJingyu LeiJingyu, Qi Li

Diverse, Controllable, and Keyphrase-Aware: A Corpus and Method for News Multi-Headline Generation (2020.emnlp-main)

Challenge:	Recent studies have exposed the risk of Large Language Models (LLMs) generating harmful content by jailbreak attacks.
Approach:	They propose a framework that exploits AdVersArial meTAphoR to induce LLMs to calibrate harmful metaphors for jailbreaking.
Outcome:	The proposed framework can successfully jailbreak Large Language Models (LLMs) by leveraging the AdVersArial meTAphoR (AVATAR) framework achieves state-of-the-art attack success rate across multiple advanced LLMs.

Copied to clipboard

Dayiheng Liu, Yeyun Gong, Yu Yan, Jie Fu, Bo Shao, Daxin Jiang, Jiancheng Lv, Nan Duan

Text Generation

Challenge:	Existing methods for news headline generation focus on producing a single short sentence . et al., 2017; Gehrmann e.t., 2018; Zhong ee., 2019) focus on single-headline generation.
Approach:	They propose a method to generate multiple headlines with keyphrases of user interests . they propose generating multiple keyphrase-relevant headlines using a transformer decoder .
Outcome:	The proposed method achieves state-of-the-art in terms of quality and diversity.

MemPO: Self-Memory Policy Optimization for Long-Horizon Agents (2026.findings-acl)

Copied to clipboard

Ruoran Li, Xinghua Zhang, Haiyang Yu, Shitong Duan, Xiang Li, Wenxin Xiang, Chonghua Liao, Xudong Guo, Yongbin Li, Jinli Suo

Challenge:	Existing methods for long-horizon agents introduce the external memory module and look up the relevant information from the stored memory, which prevents the model from proactively managing its memory content and aligning with the agent’s overarching task objectives.
Approach:	They propose an algorithm which enables agents to autonomously manage their memory during interaction with environment and selectively retain crucial information.
Outcome:	Extensive experiments show that the proposed algorithm achieves absolute F1 score gains of 25.98 over the base model and 7.1 over the previous SOTA baseline while preserving task performance.

Pre-train and Plug-in: Flexible Conditional Text Generation with Variational Auto-Encoders (2020.acl-main)

Copied to clipboard

Yu Duan, Canwen Xu, Jiaxin Pei, Jialong Han, Chenliang Li

A.S.E: A Repository-Level Benchmark for Evaluating Security in AI-Generated Code (2026.findings-acl)

Challenge:	Existing conditional generation models cannot handle emerging conditions due to their joint end-to-end learning fashion.
Approach:	They propose a framework for conditional text generation that decouples the text generation module from the condition representation module to allow "one-to-many" conditional generation.
Outcome:	The proposed framework decouples the text generation module from the condition representation module to allow “one-to-many” conditional generation.

Copied to clipboard

Keke Lian, Wang Bin, Lei Zhang, Libo Chen, Junjie Wang, Ziming Zhao, Yujiu Yang, Miaoqian Lin, Haotong Duan, Haoran Zhao, Shuang Liao, Mingda Guo, Quan Jiazheng, Yilu Zhong, Chenhao He, Chen Zichuan, Jie Wu, Haoling Li, Zhaoxuan Li, Jiongchi Yu, Hui LI, Dong Zhang

Programming Languages in NLP

Challenge:	Existing security evaluation benchmarks lack relevance to real-world AI programming tasks . current LLMs struggle with secure coding, research shows .
Approach:	They propose a repository-level evaluation benchmark to assess security of AI-generated code.
Outcome:	The proposed framework mirrors real-world AI programming tasks and offers valuable insights into the state of AI code generation.

Can Large Language Models Act as Ensembler for Multi-GNNs? (2025.emnlp-main)

Copied to clipboard

Hanqi Duan, Yao Cheng, Jianxiang Yu, Yao Liu, Xiang Li

TiMem: Temporal-Hierarchical Memory Consolidation for Long-Horizon Conversational Agents (2026.findings-acl)

Challenge:	Existing graph neural networks lack the inherent semantic understanding capability of rich textual attributes, limiting their effectiveness in applications.
Approach:	They propose a model that integrates multiple GNNs and LLMs to provide an ensemble for multi-GNNs.
Outcome:	The proposed model outperforms existing models in terms of semantic understanding of graph structures and graph structures.

Copied to clipboard

Kai Li, Xuanqing Yu, Ziyi Ni, Yi Zeng, Yao Xu, Zheqing Zhang, Xin Li, Jitao Sang, Xiaogang Duan, Xuelei Wang, Chengbao Liu, Jie Tan

Dialogue Systems & Conversational Agents

Challenge:	Existing memory frameworks provide limited support for temporally structured information across hierarchical levels, leading to fragmented memories and unstable long-horizon personalization.
Approach:	They propose a temporal–hierarchical memory framework that organizes conversations through a Temporal Memory Tree.
Outcome:	The proposed framework outperforms baselines while reducing the recalled memory length by 52.20%.

Factorized Transformer for Multi-Domain Neural Machine Translation (2020.findings-emnlp)

Copied to clipboard

Yongchao Deng, Hongfei Yu, Heng Yu, Xiangyu Duan, Weihua Luo

Multilinguality

Challenge:	Multi-domain Neural Machine Translation (MMT) is a challenging task due to the extreme diversity of cross-domain wording and phrasing style, and the imperfections of training data distribution.
Approach:	They propose a factorized NMT model that divides domain-shared knowledge into domain-specific ones that are private for each constituent domain.
Outcome:	The proposed model achieves state-of-the-art performance and opens up new perspectives for multi-domain and open-domain applications.

Confusion is the Final Barrier: Rethinking Jailbreak Evaluation and Investigating the Real Misuse Threat of LLMs (2025.findings-emnlp)

Copied to clipboard

Yu Yan, Sheng Sun, Zhe Wang, Yijun Lin, Zenghao Duan, Zhifei Zheng, Min Liu, Zhiyi Yin, Jianping Zhang