Papers by Zhiyuan Chen

Graph-GRPO: Stabilizing Multi-Agent Topology Learning via Group Relative Policy Optimization (2026.findings-acl)

Challenge:	Existing classification and regression models that only extract finer-grained information from magnetic resonance imaging (MRI) may not be effective for Alzheimer's disease (AD).
Approach:	They propose to use a 3D Adapter in a Vision Transformer to extract the patient's EHR information and questions related to the disease as text prompts.
Outcome:	The proposed model can discriminate and predict the corresponding MMSE score based on the extracted brain structural information and textual content .

Copied to clipboard

Yueyang Cang, Xiaoteng Zhang, Erlu Zhao, Zehua Ji, Yuhang Liu, Yuchen He, Zhiyuan Ning, Chen Yijun, Wenge Que, Li Shi

Joint Representation Learning of Cross-lingual Words and Entities via Attentive Distant Supervision (D18-1)

Challenge:	Recent approaches to optimize communication topology rely on single-sample policy gradients with absolute rewards.
Approach:	They propose a topology optimization framework that integrates Group Relative Policy Optimization.
Outcome:	The proposed topology optimization framework outperforms state-of-the-art methods on reasoning and code generation benchmarks.

Copied to clipboard

Yixin Cao, Lei Hou, Juanzi Li, Zhiyuan Liu, Chengjiang Li, Xu Chen, Tiansi Dong

Sparse Low-rank Adaptation of Pre-trained Language Models (2023.emnlp-main)

Challenge:	Existing methods for learning word and entity representations in monolingual settings are limited.
Approach:	They propose a method for joint representation learning of cross-lingual words and entities that captures mutually complementary knowledge and enables cross-linguistic inferences.
Outcome:	The proposed method captures mutually complementary knowledge and enables cross-lingual inferences among knowledge bases and texts.

Copied to clipboard

Ning Ding, Xingtai Lv, Qiaosen Wang, Yulin Chen, Bowen Zhou, Zhiyuan Liu, Maosong Sun

Emergent Relational Order in LLM Agent Societies: From Collective Affect to Authority Stratification (2026.findings-acl)

Challenge:	Existing methods for fine-tuning pre-trained large language models in a parameter-efficient manner are gaining traction within the research community.
Approach:	They propose a method of low-rank adaptation that enables dynamic adjustments to the intrinsic rank during the adaptation process.
Outcome:	The proposed approach outperforms the current method with a fixed and unalterable intrinsic rank and a low-rank adaptation process.

Copied to clipboard

Zhiyuan Ji, Xinyu Chen, Ziqi Dai, Shiyun Tang, Chunyu Wei, Yueguo Chen

Linguistic Theories

Challenge:	Fei Xiaotong’s Differential Order Pattern characterizes rural society as egocentric and relationally graded, with cooperation attenuating over social distance.
Approach:	They propose a multi-agent framework grounded in Affect Control Theory, Social Identity Theory, and Durkheimian collective affect.
Outcome:	Extensive simulations support interpreting Differential Order as a structure-sensitive emergent outcome of general social mechanisms.

From Adversarial Arms Race to Model-centric Evaluation: Motivating a Unified Automatic Robustness Evaluation Framework (2023.findings-acl)

Copied to clipboard

Yangyi Chen, Hongcheng Gao, Ganqu Cui, Lifan Yuan, Dehan Kong, Hanlu Wu, Ning Shi, Bo Yuan, Longtao Huang, Hui Xue, Zhiyuan Liu, Maosong Sun, Heng Ji

STARS: A Unified Framework for Singing Transcription, Alignment, and Refined Style Annotation (2025.findings-acl)

Challenge:	Existing models of robustness evaluation are incomprehensive, impractical, and invalid .
Approach:	They propose a framework for automatic robustness evaluation that shifts towards model-centric evaluation to further exploit the advantages of adversarial attacks.
Outcome:	The proposed framework is based on a model-centric evaluation protocol and a robustness evaluation protocol.

Copied to clipboard

Wenxiang Guo, Yu Zhang, Changhao Pan, Zhiyuan Zhu, Ruiqi Li, ZheTao Chen, Wenhao Xu, Fei Wu, Zhou Zhao

Speech & Audio in NLP

Challenge:	Existing automated singing annotation (ASA) methods tackle isolated aspects of the annotation pipeline.
Approach:	They propose a framework that addresses transcription, alignment, and refined style annotations.
Outcome:	The proposed framework delivers comprehensive multi-level annotations encompassing: (1) precise phoneme-audio alignment, (2) robust note transcription and temporal localization, (3) expressive vocal technique identification, and (4) global stylistic characterization including emotion and pace.

OpenPrompt: An Open-source Framework for Prompt-learning (2022.acl-demo)

Copied to clipboard

Ning Ding, Shengding Hu, Weilin Zhao, Yulin Chen, Zhiyuan Liu, Haitao Zheng, Maosong Sun

Diffusion Theory as a Scalpel: Detecting and Purifying Poisonous Dimensions in Pre-trained Language Models Caused by Backdoor or Bias (2023.findings-acl)

Challenge:	Prompt-learning is a new paradigm in natural language processing, adapting pre-trained language models to cloze-style prediction, autoregressive modeling, or sequence to sequence generation.
Approach:	They propose a framework for prompt-learning that integrates pre-trained language models with a unified framework.
Outcome:	The proposed framework is easy to use and flexible enough to integrate with other frameworks.

Copied to clipboard

Zhiyuan Zhang, Deli Chen, Hao Zhou, Fandong Meng, Jie Zhou, Xu Sun

Exploring Mode Connectivity for Pre-trained Language Models (2022.emnlp-main)

Challenge:	Existing methods to fine-tune pre-trained language models (PLMs) are not safe, since the fine-uning process is invisible to the user.
Approach:	They propose a technique to study the dynamic process of fine-tuning for finding poisonous dimensions using diffusion theory.
Outcome:	The proposed approach can detect poisonous dimensions with abnormal dynamics, purify them and fine-tune them on a clean dataset.

Copied to clipboard

Yujia Qin, Cheng Qian, Jing Yi, Weize Chen, Yankai Lin, Xu Han, Zhiyuan Liu, Maosong Sun, Jie Zhou

MAVEN-ARG: Completing the Puzzle of All-in-One Event Understanding Dataset with Event Argument Annotation (2024.acl-long)

Challenge:	Recent years have witnessed the prevalent application of pre-trained language models (PLMs) in NLP. From the perspective of parameter space, PLMs provide generic initialization, starting from which high-performance minima could be found.
Approach:	They investigate the geometric connections of different minima through the lens of mode connectivity, which measures whether two minima can be connected with a low-loss path.
Outcome:	The proposed model can be used to find low-loss paths between two minima, and to understand how their mode connectivity affects their task knowledge.

Copied to clipboard

Xiaozhi Wang, Hao Peng, Yong Guan, Kaisheng Zeng, Jianhui Chen, Lei Hou, Xu Han, Yankai Lin, Zhiyuan Liu, Ruobing Xie, Jie Zhou, Juanzi Li

LoRA-Flow: Dynamic LoRA Fusion for Large Language Models in Generative Tasks (2024.acl-long)

Challenge:	Existing datasets for event understanding have limited coverage due to complexity of tasks.
Approach:	They propose a dataset that augments MAVEN datasets with event argument annotations . they propose 98,591 events and 290,613 arguments obtained with laborious human annotation .
Outcome:	The proposed dataset is the first all-in-one dataset supporting event detection, event argument extraction, and event relation extraction.

Copied to clipboard

Hanqing Wang, Bowen Ping, Shuo Wang, Xu Han, Yun Chen, Zhiyuan Liu, Maosong Sun

Textual Backdoor Attacks Can Be More Harmful via Two Simple Tricks (2022.emnlp-main)

Challenge:	LoRA-Flow uses lightweight modules to customize large language models for downstream tasks . previous work on LoRA combination relied on task-level weights for each involved LoRA .
Approach:	They propose a LoRA-Flow approach that uses dynamic weights to adjust the impact of different LoRAs.
Outcome:	The proposed method outperforms baselines with task-level weights on six generative tasks.

Copied to clipboard

Yangyi Chen, Fanchao Qi, Hongcheng Gao, Zhiyuan Liu, Maosong Sun

MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing (2026.acl-industry)

Challenge:	Existing textual backdoor attacks are vulnerable to backdoors . researchers add extra training task to distinguish poisoned and clean data .
Approach:	They propose two tricks that make existing backdoor attacks much more harmful . first trick is to add an extra task to distinguish poisoned and clean data . second trick is using all the clean training data rather than the original clean data.
Outcome:	The proposed tricks can significantly improve attack performance in three tough situations including clean data fine-tuning, low-poisoning-rate, and label-consistent attacks.

Copied to clipboard

Junbo Niu, Zheng Liu, Zhuangcheng Gu, Bin Wang, Linke Ouyang, Zhiyuan Zhao, Tao Chu, Tianyao He, Fan Wu, Qintong Zhang, Zhenjiang Jin, Guang Liang, Rui Zhang, Wenzheng Zhang, Yuan Qu, Zhifei Ren, Yuefeng Sun, Zirui Tang, Boyu Niu, Yuanhong Zheng, Dongsheng Ma, Ziyang Miao, Hejun Dong, Siyi Qian, Junyuan Zhang, Fangdong Wang, Jingzhou Chen, Xiaomeng Zhao, Liqun Wei, Wei Li, Shasha Wang, RuiLiang Xu, Yuanyuan Cao, Lu Chen, Qianqian Wu, Huaiyu Gu, Lindong Lu, Dechen Lin, null Shenguanlin, Xuanhe Zhou, Linfeng Zhang, Yuhang Zang, Xiaoyi Dong, Jiaqi Wang, Bo Zhang, Lei Bai, Pei Chu, Weijia Li, Jiang Wu, Lijun Wu, Zhenxiang Li, Guangyu Wang, Zhongying Tu, Chao Xu, Kai Chen, Bowen Zhou, Dahua Lin, Wentao Zhang, Conghui He

Why Should Adversarial Perturbations be Imperceptible? Rethink the Research Paradigm in Adversarial NLP (2022.emnlp-main)

Challenge:	Document images are characterized by higher resolutions, denser content, and more complex structural layouts.
Approach:	They propose a 1.2B-parameter document parsing vision-language model that decouples layout analysis from local content recognition.
Outcome:	The proposed model surpasses general-purpose and domain-specific models on multiple benchmarks while maintaining significantly lower computational overhead.

Copied to clipboard

Yangyi Chen, Hongcheng Gao, Ganqu Cui, Fanchao Qi, Longtao Huang, Zhiyuan Liu, Maosong Sun

Exploring the Universal Vulnerability of Prompt-based Learning Paradigm (2022.findings-naacl)

Challenge:	Textual adversarial samples are often misrepresented in research on security, evaluation, explainability, and data augmentation.
Approach:	They propose to use adversarial samples to evaluate their methods on security tasks to demonstrate the real-world concerns rather than developing impractical methods.
Outcome:	The proposed method has higher practical value than the current benchmark.

Copied to clipboard

Lei Xu, Yangyi Chen, Ganqu Cui, Hongcheng Gao, Zhiyuan Liu

Few-shot Classification with Hypersphere Modeling of Prototypes (2023.findings-acl)

Challenge:	Prompt-based learning inherits the vulnerability from pre-training, where model predictions can be misled by inserting triggers into the text.
Approach:	They propose a potential solution to mitigate this vulnerability by injecting triggers into pre-trained language models using only plain text.
Outcome:	The proposed learning paradigm inherits the vulnerability from the pre-training stage . it can totally control or severely decrease the performance of prompt-based models .

Copied to clipboard

Ning Ding, Yulin Chen, Ganqu Cui, Xiaobin Wang, Haitao Zheng, Zhiyuan Liu, Pengjun Xie

Exploring Lottery Prompts for Pre-trained Language Models (2023.acl-long)

Challenge:	Existing methods for fewshot learning use embeddings in space, but they lack expressivity and are difficult to perform statistically.
Approach:	They propose a method where class information is represented by hyperspheres with dynamic sizes with two sets of learnable parameters: the hypersphere’s center and the radius.
Outcome:	The proposed method is much more expressive than embeddings and performs better than statistical modeling.

Copied to clipboard

Yulin Chen, Ning Ding, Xiaobin Wang, Shengding Hu, Haitao Zheng, Zhiyuan Liu, Pengjun Xie

PersLLM: A Personified Training Approach for Large Language Models (2025.findings-emnlp)

Challenge:	Existing approaches to optimize pre-trained language models are expensive and slow to scale.
Approach:	They propose to search for instance-level lottery prompts and generalize them to unseen data . they validate the assumption that for every instance, there is almost always a lottery prompt that induces the correct prediction from the PLM .
Outcome:	The proposed method can achieve comparable results with other gradient-free and optimization-free baselines.

Copied to clipboard

Zheni Zeng, Jiayi Chen, Huimin Chen, Yukun Yan, Yuxuan Chen, Zhenghao Liu, Zhiyuan Liu, Maosong Sun

Optima: Optimizing Effectiveness and Efficiency for LLM-Based Multi-Agent System (2025.findings-acl)

Challenge:	Large language models exhibit human-like intelligence, enabling them to simulate human behavior and support various applications that require both humanized communication and extensive knowledge reserves.
Approach:	They propose a framework for better data construction and model tuning to unlock the potential of LLM personification by using Chain-of-Thought prompting and anti-induction.
Outcome:	The proposed framework improves data construction and model tuning for insufficient data usage and rigid behavior patterns.

Copied to clipboard

Weize Chen, Jiarui Yuan, Chen Qian, Cheng Yang, Zhiyuan Liu, Maosong Sun

ChartCoder: Advancing Multimodal Large Language Model for Chart-to-Code Generation (2025.acl-long)

Challenge:	Large Language Models (LLMs) have emerged as powerful tools for a wide range of tasks, from * Equal Contribution.
Approach:	They propose a framework that enhances communication efficiency and task effectiveness in LLM-based multi-agent systems through training.
Outcome:	The proposed framework improves communication efficiency and task effectiveness on multi-agent tasks with 2.8x performance gain with less than 10% tokens on tasks requiring heavy information exchange.

Copied to clipboard

Xuanle Zhao, Xianzhen Luo, Qi Shi, Chi Chen, Shuo Wang, Zhiyuan Liu, Maosong Sun

AgentCPM-GUI: Building Mobile-Use Agents with Reinforcement Fine-Tuning (2025.emnlp-demos)

Challenge:	Existing open-source MLLMs fail to fully capture dense information embedded in charts . current models still face significant challenges in understanding and analyzing visual tasks such as captioning and question answering.
Approach:	They propose a chart-to-code MLLM which leverages Code LLMs as the language backbone to enhance the executability of the generated code.
Outcome:	The proposed model surpasses existing open-source models on chart-to-code benchmarks with only 7B parameters and provides lossless representations that contain all critical details.

Copied to clipboard

Zhong Zhang, Yaxi Lu, Yikun Fu, Yupeng Huo, Shenzhi Yang, Yesai Wu, Han Si, Xin Cong, Haotian Chen, Yankai Lin, Xie Xie, Wei Zhou, Wang Xu, Zhou Su, Zhongwu Zhai, Xiaoming Liu, null Meiyudong, Jianming Xu, Hongyan Tian, Chongyi Wang, Chi Chen, Yuan Yao, Zhiyuan Liu, Maosong Sun

Few-NERD: A Few-shot Named Entity Recognition Dataset (2021.acl-long)

Challenge:	Large language model agents have enabled GUI-based automation, but their deployment is limited by noisy data, poor generalization, and lack of support for non-English GUIs.
Approach:	They propose an 8B-parameter GUI agent built for robust and efficient on-device GUI interaction.
Outcome:	The proposed GUI agent achieves promising performance on five public benchmarks and proposed Chinese benchmark CAGUI.

Copied to clipboard

Ning Ding, Guangwei Xu, Yulin Chen, Xiaobin Wang, Xu Han, Pengjun Xie, Haitao Zheng, Zhiyuan Liu

Low-Resource NLP

Challenge:	Existing approaches to few-shot named entity recognition (NER) focus on coarse-grained entities with few examples, while most unseen entities are fine-grounded.
Approach:	They present a human-annotated few-shot named entity recognition dataset . they construct benchmark tasks to assess the generalization capability of models .
Outcome:	The proposed model is the first few-shot NER dataset and the largest human-crafted NER data set.

KBAlign: Efficient Self Adaptation on Specific Textual Knowledge Bases (2025.findings-emnlp)

Copied to clipboard

Zheni Zeng, Yuxuan Chen, Shi Yu, Ruobing Wang, Yukun Yan, Zhenghao Liu, Shuo Wang, Xu Han, Zhiyuan Liu, Maosong Sun

Controllable Preference Optimization: Toward Controllable Multi-Objective Alignment (2024.emnlp-main)

Challenge:	Existing methods for retrieval-augmented generation (RAG) are limited and fine-tuning incurs prohibitive costs of external signals.
Approach:	They propose a self-supervised framework that enhances RAG systems through efficient model adaptation.
Outcome:	The proposed framework achieves 90% of the performance gain obtained through GPT-4-supervised adaptation while relying entirely on self-annotation of much smaller models.

Copied to clipboard

Yiju Guo, Ganqu Cui, Lifan Yuan, Ning Ding, Zexu Sun, Bowen Sun, Huimin Chen, Ruobing Xie, Jie Zhou, Yankai Lin, Zhiyuan Liu, Maosong Sun

Challenge:	Existing algorithms for achieving optimal alignment are mostly unidirectional . a recent study suggests that large language models can be ground with evident preferences .
Approach:	They propose to ground large language models with evident preferences . they propose to use controllable preference optimization to specify different objectives .
Outcome:	The proposed models can provide responses that match various preferences among the ”3H” desiderata.

RepoGenesis: Benchmarking End-to-End Microservice Generation from Readme to Repository (2026.acl-long)

Copied to clipboard

Zhiyuan Peng, Xin Yin, Pu Zhao, Fangkai Yang, Lu Wang, Ran Jia, Xu Chen, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang

Challenge:	Existing benchmarks focus on isolated function/class-level generation, neglecting complete microservice repository generation.
Approach:	They propose a multilingual benchmark for repository-level end-to-end web microservice generation that reflects real-world development workflows.
Outcome:	The benchmark compared 106 repositories across 18 domains and 11 frameworks and 1,258 API endpoints and 2,335 test cases.

Large Language Model-based Human-Agent Collaboration for Complex Task Solving (2024.findings-emnlp)

Copied to clipboard

Xueyang Feng, Zhi-Yuan Chen, Yujia Qin, Yankai Lin, Xu Chen, Zhiyuan Liu, Ji-Rong Wen

Octopus: On-device language model for function calling of software APIs (2025.naacl-industry)

Challenge:	Recent advances in large language models have led to the development of LLM-based autonomous agents.
Approach:	They propose a Reinforcement Learning-based Human-Agent Collaboration method which trains a policy model to determine the most opportune stages for human intervention within the task-solving process.
Outcome:	The proposed method improves human-agent collaboration significantly through well-planned, limited human intervention.

Copied to clipboard

Wei Chen, Zhiyuan Li, Mingyuan Ma

Sub-Character Tokenization for Chinese Pretrained Language Models (2023.tacl-1)

Challenge:	Large Language Models (LLMs) are pivotal for advanced text processing and generation.
Approach:	They propose a framework to train on-device Large Language Models optimized for invoking software APIs.
Outcome:	The proposed model outperforms GPT-4 in API calling tasks while maintaining inference speed.

Copied to clipboard

Chenglei Si, Zhengyan Zhang, Yingfa Chen, Fanchao Qi, Xiaozhi Wang, Zhiyuan Liu, Yasheng Wang, Qun Liu, Maosong Sun

PEVL: Position-enhanced Pre-training and Prompt Tuning for Vision-language Models (2022.emnlp-main)

Challenge:	Existing tokenization methods for Chinese PLMs treat each character as an indivisible token, but ignore the unique feature of the writing system where additional linguistic information exists below the character level.
Approach:	They propose to encode Chinese characters into short sequences and construct Chinese vocabulary based on the encoded text.
Outcome:	The proposed tokenizers can tokenize inputs into much shorter sequences, improving computational efficiency.

Copied to clipboard

Yuan Yao, Qianyu Chen, Ao Zhang, Wei Ji, Zhiyuan Liu, Tat-Seng Chua, Maosong Sun

RouteMoA: Dynamic Routing without Pre-Inference Boosts Efficient Mixture-of-Agents (2026.acl-long)

Challenge:	Recent advances on self-supervised learning have led to powerful vision-language pre-training models that achieve state-of-the-art performance on a wide range of cross-modal tasks.
Approach:	They propose a vision-language pre-training framework that reformulates discretized object positions and language in a unified language modeling framework.
Outcome:	The proposed model improves performance on position-sensitive vision-language (VL) tasks and also improves on position insensitive tasks.

Copied to clipboard

Jize Wang, Han Wu, Zhiyuan You, Yiming Song, Yijun Wang, Zifei Shan, Yining Li, Songyang Zhang, Xinyi Le, Cailian Chen, Xinping Guan, Dacheng Tao

Beyond Natural Language: LLMs Leveraging Alternative Formats for Enhanced Reasoning and Communication (2024.findings-emnlp)

Challenge:	Existing methods for mixing-of-agents (MoA) lack model selection criteria and struggle with large model pools.
Approach:	They propose a mixture-of-agents framework with dynamic routing that uses a lightweight scorer to perform initial screening and refines the model scores through self- and cross-assessment.
Outcome:	The proposed framework outperforms existing methods for large model pools and tasks . it reduces cost by 89.8% and latency by 63.6% in the large-scale model pool.

Copied to clipboard

Weize Chen, Chenfei Yuan, Jiarui Yuan, Yusheng Su, Chen Qian, Cheng Yang, Ruobing Xie, Zhiyuan Liu, Maosong Sun

Stochastic Bridges as Effective Regularizers for Parameter-Efficient Tuning (2023.findings-acl)

Challenge:	Natural language (NL) has long been the predominant format for human cognition and communication, but its utility in LLMs has not been thoroughly examined.
Approach:	They propose to allow LLMs to choose the most suitable format before reasoning or communicating, and to automate the selection process.
Outcome:	The proposed format improves reasoning efficiency and reduces token usage while maintaining communicative effectiveness.

Copied to clipboard

Weize Chen, Xu Han, Yankai Lin, Zhiyuan Liu, Maosong Sun, Jie Zhou

READIN: A Chinese Multi-Task Benchmark with Realistic and Diverse Input Noises (2023.acl-long)

Challenge:	Existing methods for tuning pre-trained language models ignore the running cost and only optimize the terminal cost.
Approach:	They propose to use stochastic bridges to regularize intermediate states and use regularization as running cost of PETs.
Outcome:	The proposed methods can be used to tune large pre-trained language models . they can be compared to full-parameter fine-tuning by tuning a small number of parameters .

Copied to clipboard

Chenglei Si, Zhengyan Zhang, Yingfa Chen, Xiaozhi Wang, Zhiyuan Liu, Maosong Sun

ONION: A Simple and Effective Defense Against Textual Backdoor Attacks (2021.emnlp-main)

Challenge:	Existing benchmarks for Chinese inputs often lack a realistic representation of real-world noises.
Approach:	They construct a Chinese multi-task benchmark with REalistic and Diverse input noises . they use pinyin input and speech input to recruit speakers from diverse dialects based on their inputs - a feature that is important for Chinese NLP benchmarks if it is implemented in real-world applications.
Outcome:	The proposed benchmarks are based on four different tasks and are designed to maximize diversity.

Copied to clipboard

Fanchao Qi, Yangyi Chen, Mukai Li, Yuan Yao, Zhiyuan Liu, Maosong Sun

Fusing Highly Specialized Language Models for Comprehensive Expertise (2025.acl-long)

Challenge:	Backdoor attacks can manipulate the output of deep neural networks and possess high insidiousness.
Approach:	They propose a textual backdoor defense based on outlier word detection that can handle all the textual attacks.
Outcome:	The proposed method can handle all the textual backdoor attack situations.

Copied to clipboard

Ning Ding, Yulin Chen, Ganqu Cui, Xingtai Lv, Weilin Zhao, Kaiyan Zhang, Ruobing Xie, Bowen Zhou, Zhiyuan Liu, Maosong Sun

Automatic Construction of Sememe Knowledge Bases via Dictionaries (2021.findings-acl)

Challenge:	Existing models that focus on language, programming code, and mathematical symbols are not able to achieve mastery of all three domains simultaneously.
Approach:	They propose to fuse highly-specialized models that are already sufficiently trained on different domains to achieve a highly-specific model.
Outcome:	The proposed model could achieve mastery of the three crucial domains simultaneously.

Copied to clipboard

Fanchao Qi, Yangyi Chen, Fengyu Wang, Zhiyuan Liu, Xiao Chen, Maosong Sun

DeepNote: Note-Centric Deep Retrieval-Augmented Generation (2025.findings-emnlp)

Challenge:	Sememe knowledge bases (SKBs) are used to analyze natural language processing.
Approach:	They propose a method to build sememe knowledge bases from an existing dictionary . they propose to use existing dictionaries to build an English and a French SKB .
Outcome:	The proposed method is superior to HowNet, the most widely used SKB that takes decades to build manually.

Copied to clipboard

Ruobing Wang, Qingfei Zhao, Yukun Yan, Daren Zha, Yuxuan Chen, Shi Yu, Zhenghao Liu, Yixuan Wang, Shuo Wang, Xu Han, Zhiyuan Liu, Maosong Sun

Information Retrieval

Challenge:	. - (EN)
Approach:	. - (EN)
Outcome:	. - (EN)

Cross-Lingual Contrastive Learning for Fine-Grained Entity Typing for Low-Resource Languages (2022.acl-long)

Copied to clipboard

Xu Han, Yuqi Luo, Weize Chen, Zhiyuan Liu, Maosong Sun, Zhou Botong, Hao Fei, Suncong Zheng

ClueAnchor: Clue-Anchored Knowledge Reasoning Exploration and Optimization for Retrieval-Augmented Generation (2025.findings-emnlp)

Challenge:	Experimental results show that by applying our framework, we can easily learn effective FGET models for low-resource languages.
Approach:	They propose a cross-lingual contrastive learning framework to learn FGET models for low-resource languages.
Outcome:	The proposed framework can learn effective FGET models for low-resource languages even without human-labeled data.

Copied to clipboard

Hao Chen, Yukun Yan, Sen Mei, Wanxiang Che, Zhenghao Liu, Qi Shi, Xinze Li, Yuchun Fan, Pengcheng Huang, Qiushi Xiong, Zhiyuan Liu, Maosong Sun

Reasoning

Challenge:	Existing RAG systems often underutilize the retrieved documents, authors say . they fail to extract and integrate key clues needed to support faithful and interpretable reasoning .
Approach:	a new framework extracts key clues from retrieved content and generates multiple reasoning paths . the framework optimizes the model by selecting the most appropriate reasoning path .
Outcome:	Experiments show that ClueAnchor outperforms baseline RAG frameworks in completeness and robustness.

MAVEN-ERE: A Unified Large-scale Dataset for Event Coreference, Temporal, Causal, and Subevent Relation Extraction (2022.emnlp-main)

Copied to clipboard

Xiaozhi Wang, Yulin Chen, Ning Ding, Hao Peng, Zimu Wang, Yankai Lin, Xu Han, Lei Hou, Juanzi Li, Zhiyuan Liu, Peng Li, Jie Zhou

Robust and Scalable Model Editing for Large Language Models (2024.lrec-main)

Challenge:	Existing datasets only cover limited relation types at once, which prevents models from taking full advantage of relation interactions.
Approach:	They construct a large-scale human-annotated ERE dataset with improved annotation schemes to address these drawbacks.
Outcome:	The proposed dataset is larger than existing datasets of all the ERE tasks by at least an order of magnitude.

Copied to clipboard

Yingfa Chen, Zhengyan Zhang, Xu Han, Chaojun Xiao, Zhiyuan Liu, Chen Chen, Kuai Li, Tao Yang, Maosong Sun

A Close Look into the Calibration of Pre-trained Language Models (2023.acl-long)

Challenge:	Existing methods that ignore contextual knowledge fail to reliably fall back to parametric knowledge when presented with irrelevant context.
Approach:	They propose to use contextual knowledge to update and correct LLMs' knowledge by in-context editing instead of retraining.
Outcome:	The proposed method outperforms current state-of-the-art methods by a large margin on a dataset that contains irrelevant questions.

Copied to clipboard

Yangyi Chen, Lifan Yuan, Ganqu Cui, Zhiyuan Liu, Heng Ji

MiLiC-Eval: Benchmarking Multilingual LLMs for China’s Minority Languages (2025.findings-acl)

Challenge:	Pre-trained language models (PLMs) may fail in giving reliable estimates of their predictive uncertainty.
Approach:	They conduct fine-grained control experiments to study the dynamic change in PLMs’ calibration performance in training.
Outcome:	The proposed methods significantly reduce PLMs’ confidence in wrong predictions.

Copied to clipboard

Chen Zhang, Mingxu Tao, Zhiyuan Liao, Yansong Feng

Multilinguality

Challenge:	Large language models excel in high-resource languages but struggle with low-resourced languages . minority languages such as Tibetan, Uyghur, Kazakh, and Mongolian are marginalized in NLP research due to limited digital representation and the scarcity of training data.
Approach:	They propose a benchmark for minority languages in China that tracks the progress of large language models on low-resource languages.
Outcome:	The proposed benchmark focuses on underrepresented writing systems and syntax-intensive tasks.

A Query-Response Framework for Whole-Page Complex-Layout Document Image Translation with Relevant Regional Concentration (2025.findings-acl)

Copied to clipboard

Zhiyang Zhang, Yaping Zhang, Yupu Liang, Zhiyuan Chen, Lu Xiang, Yang Zhao, Yu Zhou, Chengqing Zong

Machine Translation

Challenge:	Existing methods for document image translation rely on the vanilla encoder-decoder paradigm . a novel dynamic aggregation mechanism is designed to enhance the text semantics in query features toward translation.
Approach:	They propose a Query-Response DIT framework that reformulates the DIT task into a parallel response/translation process of multiple queries.
Outcome:	The proposed framework improves translation quality on four translation directions on three benchmarks.

AgentRM: Enhancing Agent Generalization with Reward Modeling (2025.acl-long)

Copied to clipboard

Yu Xia, Jingru Fan, Weize Chen, Siyu Yan, Xin Cong, Zhong Zhang, Yaxi Lu, Yankai Lin, Zhiyuan Liu, Maosong Sun

The Right Time Matters: Data Arrangement Affects Zero-Shot Generalization in Instruction Tuning (2025.findings-acl)

Challenge:	Existing LLM-based agents have strong performance on held-in tasks, but their generalizability to unseen tasks remains poor.
Approach:	They propose a reward-based generalizable reward model to guide the policy model for effective test-time search.
Outcome:	The proposed agentRM outperforms existing agents on held-in tasks by 8.8 points on average.

Copied to clipboard

Bingxiang He, Ning Ding, Cheng Qian, Jia Deng, Ganqu Cui, Lifan Yuan, Haiwen Hong, Huan-ang Gao, Longtao Huang, Hui Xue, Huimin Chen, Zhiyuan Liu, Maosong Sun

Cross-Modal Coreference Alignment: Enabling Reliable Information Transfer in Omni-LLMs (2026.acl-long)

Challenge:	Existing work on instruction tuning has focused on task level, without considering that tasks are artificially defined and, to LLMs, merely consist of tokens and representations.
Approach:	They propose a training data arrangement framework that allows for continual learning and loss reduction.
Outcome:	The proposed framework promotes continual learning and loss reduction on unseen tasks.

Copied to clipboard

Hongcheng Liu, Yuhao Wang, Zhe Chen, Pingjie Wang, Zhiyuan Zhu, Yixuan Hou, Yanfeng Wang, Yu Wang

LoRE: Enhancing Search Relevance with Progressive Chain-of-Thought and Preference Alignment (2026.findings-acl)

Challenge:	Experiments on 13 Omni-LLMs reveal systematic weaknesses in cross-modal coreference . cross-module coreference is a crucial missing piece for advancing robust omni-modal reasoning.
Approach:	They propose a cross-modal coreference problem to evaluate and enhance Omni-LLMs' reasoning capabilities.
Outcome:	Experiments on 13 Omni-LLMs show they lack coreference-aware thinking patterns . the CROSSOMNI dataset yields significant performance gains and generalizes well to collaborative reasoning tasks.

Copied to clipboard

Chenji Lu, Zhuo Chen, Hui Zhao, Zhiyuan Zeng, Gang Zhao, Junjie Ren, null Lihaoran, Songyan Liu, Pengjie Wang, Chuan Yu, Jian Xu, Bo Zheng

Reasoning

Challenge:	E-commerce search relevance is a critical component of retrieval systems.
Approach:	They propose a large-generative model for search relevance that trains reasoning knowledge, multi-modal understanding and rule awareness into three core competencies.
Outcome:	The proposed model outperforms GPT-5 in Macro-F1 and achieves 27% online gain.

DIAG-NRE: A Neural Pattern Diagnosis Framework for Distantly Supervised Neural Relation Extraction (P19-1)

Copied to clipboard

Shun Zheng, Xu Han, Yankai Lin, Peilin Yu, Lu Chen, Ling Huang, Zhiyuan Liu, Wei Xu

BMCook: A Task-agnostic Compression Toolkit for Big Models (2022.emnlp-demos)

Challenge:	Existing methods for labeling relational facts require significant expert labor to write relation-specific patterns, which makes them too sophisticated to generalize quickly.
Approach:	They propose a neural pattern diagnosis framework that can summarize and refine relation-specific patterns with human experts in the loop.
Outcome:	The proposed framework can summarize and refine high-quality relational patterns from noise data with human experts in the loop.

Copied to clipboard

Zhengyan Zhang, Baitao Gong, Yingfa Chen, Xu Han, Guoyang Zeng, Weilin Zhao, Yanxu Chen, Zhiyuan Liu, Maosong Sun

EvolveBench: A Comprehensive Benchmark for Assessing Temporal Awareness in LLMs on Evolving Knowledge (2025.acl-long)

Challenge:	Existing efforts to compress medium-sized models for specific tasks have limited results.
Approach:	They propose a task-agnostic compression toolkit for big models that implements quantization, pruning, distillation and MoEfication methods.
Outcome:	The proposed tool improves performance on a model with 3 billion parameters by 12x . it also outperforms the original model on three typical NLP benchmarks.

Copied to clipboard

Zhiyuan Zhu, Yusheng Liao, Zhe Chen, Yuhao Wang, Yunfeng Guan, Yanfeng Wang, Yu Wang

Challenge:	Existing studies have explored how LLMs perceive time, but they often overlook the critical aspect of knowledge utilization.
Approach:	They propose a benchmark that evaluates temporal competence along five key dimensions: Cognition, Awareness, Trustworthiness and reasoning.
Outcome:	EvolveBench measures temporal competence along five key dimensions: Cognition, Awareness, Trustworthiness, Understanding and reasoning.

Boosting Inference Efficiency: Unleashing the Power of Parameter-Shared Pre-trained Language Models (2023.findings-emnlp)

Copied to clipboard

Weize Chen, Xiaoyue Xu, Xu Han, Yankai Lin, Ruobing Xie, Zhiyuan Liu, Maosong Sun, Jie Zhou

Cost-Optimal Grouped-Query Attention for Long-Context Modeling (2025.emnlp-main)

Challenge:	Parameter-shared pre-trained language models (PLMs) have emerged as a successful approach in resource-constrained environments.
Approach:	They propose a method to enhance the inference efficiency of parameter-shared PLMs by pre-training models that can achieve even greater acceleration.
Outcome:	The proposed method improves inference efficiency on autoregressive and autoencoding models.

Copied to clipboard

Yingfa Chen, Yutong Wu, Chenyang Song, Zhen Leng Thai, Xingyu Shen, Xu Han, Zhiyuan Liu, Maosong Sun

Does Memory Need Graphs? A Unified Framework and Empirical Analysis for Long-Term Dialog Memory (2026.acl-long)

Challenge:	Current GQA configurations overlook how context length influences inference cost .
Approach:	They propose a recipe for deriving cost-optimal GQA configurations that decouple the total head size from the hidden size and allow more flexible control over attention FLOPs.
Outcome:	The proposed configurations reduce memory usage and FLOPs by more than 50% compared to Llama-3's GQA, with no degradation in model capabilities.

Copied to clipboard

Sen Hu, Yuxiang Wei, Jiaxin Ran, Xueran Han, Zhiyuan Yao, Huacan Wang, Ronghao Chen, Lei Zou

Natural Language Interfaces

Challenge:	Existing literature on dialog memory systems is inconsistent on their effectiveness . empirical findings on graph structures are difficult to attribute to specific design choices .
Approach:	They propose a framework that decomposes dialog memory systems into core components . they conduct stage-wise experiments on LongMemEval and HaluMeM, and compare implementation details .
Outcome:	The proposed framework compares graph-based and non-graph memory architectures on long-term dialog memory systems.

Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger (2021.acl-long)

Copied to clipboard

Fanchao Qi, Mukai Li, Yangyi Chen, Zhengyan Zhang, Zhiyuan Liu, Yasheng Wang, Maosong Sun

Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer (2021.emnlp-main)

Challenge:	Existing methods for textual backdoor attacks insert additional contents into normal samples as triggers, causing detection and blocking of backdoors.
Approach:	They propose to use syntactic structure as trigger in textual backdoor attacks . they propose to achieve similar attack performance but have higher invisibility .
Outcome:	The proposed method achieves almost 100% success rate but has higher invisibility and stronger resistance to defenses than the insertion-based methods.

Copied to clipboard

Fanchao Qi, Yangyi Chen, Xurui Zhang, Mukai Li, Zhiyuan Liu, Maosong Sun

GUICourse: From General Vision Language Model to Versatile GUI Agent (2025.acl-long)

Challenge:	Experimental results show that popular NLP models are vulnerable to both adversarial and backdoor attacks based on text style transfer.
Approach:	They propose to conduct adversarial and backdoor attacks based on text style transfer . the authors propose to use text style to alter the style of a sentence .
Outcome:	The proposed methods show that popular models are vulnerable to both attacks based on text style transfer . the results show that the proposed methods perform better than baselines in many aspects .

Copied to clipboard

Wentong Chen, Junbo Cui, Jinyi Hu, Yujia Qin, Junjie Fang, Yue Zhao, Chongyi Wang, Jun Liu, Guirong Chen, Yupeng Huo, Yuan Yao, Yankai Lin, Zhiyuan Liu, Maosong Sun

Visual Data in NLP

Challenge:	Graphical User Interfaces (GUIs) are a pivotal medium for human-computer interaction.
Approach:	They propose a series of datasets for training visual-based GUI agents using general VLMs.
Outcome:	The proposed GUICourse datasets show that even a small-sized GUI agent performs better on GUI tasks.

Expose Backdoors on the Way: A Feature-Based Efficient Defense against Textual Backdoor Attacks (2022.findings-emnlp)

Copied to clipboard

Sishuo Chen, Wenkai Yang, Zhiyuan Zhang, Xiaohan Bi, Xu Sun

Experiential Co-Learning of Software-Developing Agents (2024.acl-long)

Challenge:	Existing online backdoor defense methods for NLP models focus on anomalies at input or output level, causing fragility to adaptive attacks and high computational cost.
Approach:	They propose a feature-based online defense method to detect poisoned samples . they use a distance-based anomaly score to distinguish poisones from clean samples based on feature-level regularization .
Outcome:	The proposed method outperforms existing methods in sentiment analysis and offense detection tasks.

Copied to clipboard

Chen Qian, Yufan Dang, Jiahao Li, Wei Liu, Zihao Xie, YiFei Wang, Weize Chen, Cheng Yang, Xin Cong, Xiaoyin Che, Zhiyuan Liu, Maosong Sun

Beat LLMs at Their Own Game: Zero-Shot LLM-Generated Text Detection via Querying ChatGPT (2023.emnlp-main)

Challenge:	Recent advances in large language models (LLMs) have brought significant changes to various domains, especially through autonomous agents.
Approach:	They propose a framework that lets agents learn shortcuts from their past tasks and use them for future task execution.
Outcome:	The proposed framework enables agents to tackle unseen software-developing tasks more effectively.

Copied to clipboard

Biru Zhu, Lifan Yuan, Ganqu Cui, Yangyi Chen, Chong Fu, Bingxiang He, Yangdong Deng, Zhiyuan Liu, Maosong Sun, Ming Gu

Beyond the Turn-Based Game: Enabling Real-Time Conversations with Duplex Models (2024.emnlp-main)

Challenge:	Large language models (LLMs) are capable of performing tasks but are likely to be misused.
Approach:	They propose a zero-shot black-box method to detect LLM-generated texts . they revise the text to be detected using the ChatGPT model .
Outcome:	The proposed method can detect LLM-generated texts with a zero-shot black-box model . it is based on intuition that the model will make fewer revisions to LLMs than to human-written texts .

Copied to clipboard

Xinrong Zhang, Yingfa Chen, Shengding Hu, Xu Han, Zihang Xu, Yuanwei Xu, Weilin Zhao, Maosong Sun, Zhiyuan Liu

Dialogue Systems & Conversational Agents

Challenge:	Large language models (LLMs) are increasingly permeating daily lives and require real-time interactions that mirror human conversations.
Approach:	They propose to use time-division-multiplexing to process queries and responses pseudo-simultaneously.
Outcome:	The proposed model can listen to users while generating output and adjust to provide instant feedback.

Efficient Low-Resource Language Adaptation via Multi-Source Dynamic Logit Fusion (2026.acl-long)

Copied to clipboard

Chen Zhang, Jiuheng Lin, Zhiyuan Liao, Yansong Feng

Weize Chen, Hao Zhu, Xu Han, Zhiyuan Liu, Maosong Sun

Challenge:	Proxy Tuning offers a logit-level strategy for introducing scaling effects, but it often fails in LRL settings because the large model’s weak LRL competence might overwhelm the knowledge of specialized smaller models.
Approach:	They propose a logit-based framework that balances LRL competence from a continually pretrained small model, task competence from high-resource language instruction tuning, and the scaling benefits of large models.
Outcome:	Experiments across four model families and eight LRLs show that TriMix outperforms single-model baselines and Proxy Tuning.

Copied to clipboard

SConU: Selective Conformal Uncertainty in Large Language Models (2025.acl-long)

Challenge:	a conceptually simple and effective method to quantify the similarity between relations is presented . identifying relations is a crucial problem for several information extraction tasks.
Approach:	They propose a method to quantify the similarity between relations in knowledge bases . they use a neural network to parameterize conditional probability distributions over entity pairs .
Outcome:	The proposed method significantly correlates with human judgments, the authors show . it could be incorporated into negative sampling and softmax classification to alleviate these mistakes.

Copied to clipboard

Zhiyuan Wang, Qingni Wang, Yue Zhang, Tianlong Chen, Xiaofeng Zhu, Xiaoshuang Shi, Kaidi Xu

Fully Hyperbolic Neural Networks (2022.acl-long)

Challenge:	Existing frameworks fail to identify outliers that violate the exchangeability assumption, leading to unbounded miscoverage rates and unactionable prediction sets.
Approach:	They propose a method that implements significance tests to determine whether a given sample deviates from the uncertainty distribution of the calibration set.
Outcome:	The proposed approach facilitates rigorous management of miscoverage rates across single-domain and interdisciplinary contexts, and enhances the efficiency of predictions.

Copied to clipboard

Weize Chen, Xu Han, Yankai Lin, Hexu Zhao, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

Challenge:	Existing hyperbolic neural networks encode features in the hyperbolical space yet formalize most of their operations in the tangent space.
Approach:	They propose a fully hyperbolic framework to build hyperbolical networks based on the Lorentz model by adapting Lorentzer transformations to formalize essential operations of neural networks.
Outcome:	The proposed framework has better performance on four NLP tasks compared with existing hyperbolic models .

Enhancing Legal Case Retrieval via Scaling High-quality Synthetic Query-Candidate Pairs (2024.emnlp-main)

Copied to clipboard

Cheng Gao, Chaojun Xiao, Zhenghao Liu, Huimin Chen, Zhiyuan Liu, Maosong Sun

Information Retrieval

Challenge:	Existing studies focus on case-to-case retrieval using lengthy queries, which does not match real-world scenarios.
Approach:	They propose a method to construct query-candidate pairs and build the largest LCR dataset to date, LEAD.
Outcome:	Experimental results show that the method can provide ample training signals for LCR models.

Prompt-learning for Fine-grained Entity Typing (2022.findings-emnlp)

Copied to clipboard

Ning Ding, Yulin Chen, Xu Han, Guangwei Xu, Xiaobin Wang, Pengjun Xie, Haitao Zheng, Zhiyuan Liu, Juanzi Li, Hong-Gee Kim

bert2BERT: Towards Reusable Pretrained Language Models (2022.acl-long)

Challenge:	Extensive experiments on fine-grained entity typing under fully supervised, few-shot, and zero-shot settings show the effectiveness of prompt-learning.
Approach:	They propose a prompt-learning pipeline that stimulates versatile knowledge of pre-trained language models (PLMs) by constructing entity-oriented verbalizers and templates and conducting masked language modeling.
Outcome:	The proposed approach can be applied to fine-grained entity typing in fully supervised, few-shot, and zero-shot scenarios.

Copied to clipboard

Cheng Chen, Yichun Yin, Lifeng Shang, Xin Jiang, Yujia Qin, Fengyu Wang, Zhi Wang, Xiao Chen, Zhiyuan Liu, Qun Liu

HeteroRAG: A Heterogeneous Retrieval-Augmented Generation Framework for Medical Vision Language Tasks (2026.findings-acl)

Challenge:	Pre-training large language models can be expensive and wasteful.
Approach:	They propose a method which can transfer the knowledge of an existing smaller pre-trained model to a large model through parameter initialization and a two-stage learning method to further accelerate the pre-training.
Outcome:	The proposed method can transfer the knowledge of an existing smaller pre-trained model to a large model through parameter initialization and significantly improve the pre-training efficiency of the large model.

Copied to clipboard

Zhe Chen, Yusheng Liao, Zhiyuan Zhu, Haolin Li, Hongcheng Liu, Yanfeng Wang, Yu Wang

Improving MLLM’s Document Image Machine Translation via Synchronously Self-reviewing Its OCR Proficiency (2025.findings-acl)

Challenge:	Medical large vision-language models suffer from factual inaccuracies and unreliable outputs.
Approach:	They propose a framework that enhances Med-LVLMs through heterogeneous knowledge sources.
Outcome:	The proposed framework improves Med-LVLMs through heterogeneous knowledge sources.

Copied to clipboard

Yupu Liang, Yaping Zhang, Zhiyang Zhang, Zhiyuan Chen, Yang Zhao, Lu Xiang, Chengqing Zong, Yu Zhou

Multilinguality

Challenge:	Multimodal Large Language Models (MLLMs) have shown strong performance in document image tasks, especially Optical Character Recognition (OCR). However, they struggle with Document Image Machine Translation (DIMT), which requires handling both cross-modal and cross-lingual challenges.
Approach:	They propose a novel fine-tuning paradigm that allows the model to generate OCR text before producing translation text, which allows it to leverage its strong monolingual OCR ability while learning to translate text across languages.
Outcome:	The proposed model can leverage its strong monolingual OCR ability while learning to translate text across languages.

Enhancing Chat Language Models by Scaling High-quality Instructional Conversations (2023.emnlp-main)

Copied to clipboard

Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Shengding Hu, Zhiyuan Liu, Maosong Sun, Bowen Zhou

Migician: Revealing the Magic of Free-Form Multi-Image Grounding in Multimodal Large Language Models (2025.findings-acl)

Challenge:	a recent study validates the effectiveness of chat language models by fine-tuning instruction data.
Approach:	They propose to use a large-scale dataset of instructional conversations to fine-tune a conversational model on instruction data.
Outcome:	The proposed model outperforms open-source models in key metrics including scale, average length, diversity, coherence, etc.

Copied to clipboard

You Li, Heyu Huang, Chi Chen, Kaiyu Huang, Chao Huang, Zonghao Guo, Zhiyuan Liu, Jinan Xu, Yuhua Li, Ruixuan Li, Maosong Sun

RealMem: Benchmarking LLMs in Real-World Memory-Driven Interaction (2026.findings-acl)

Challenge:	Existing MLLMs still struggle to achieve precise grounding in multi-image scenarios.
Approach:	They propose a Chain-of-Thought framework that integrates single-image grounding with multi-image comprehension to address this challenge.
Outcome:	The proposed model outperforms existing models in multi-image grounding tasks by 24.94% and surpasses larger 70B models.

Copied to clipboard

Haonan Bian, Zhiyuan Yao, Sen Hu, Zishan Xu, Shaolei Zhang, Yifu Guo, Ziliang Yang, Xueran Han, Huacan Wang, Ronghao Chen

Dialogue Systems & Conversational Agents

Challenge:	Existing benchmarks focus on casual conversation or task-oriented dialogue, failing to capture “long-term project-oriented” interactions where agents must track evolving goals.
Approach:	They propose a benchmark that simulates the dynamic evolution of memory in real-world projects.
Outcome:	The proposed benchmarks simulate the dynamic evolution of memory in real-world projects.

Multi-Modal Multi-Granularity Tokenizer for Chu Bamboo Slips (2025.coling-main)

Copied to clipboard

Yingfa Chen, Chenlong Hu, Cong Feng, Chenyang Song, Shi Yu, Xu Han, Zhiyuan Liu, Maosong Sun

Syntactic Text Processing

Challenge:	Using a multi-modal multi-granularity tokenizer, we analyze ancient Chinese scripts . a large proportion of the characters in ancient Chinese are rare or undeciphered .
Approach:	They propose a multi-modal multi-granularity tokenizer specifically designed for ancient Chinese scripts.
Outcome:	The proposed tokenizer improves on the part-of-speech tagging task on the Chu bamboo slip script.

Gradient-Based Adversarial Factual Consistency Evaluation for Abstractive Summarization (2021.emnlp-main)

Copied to clipboard

Zhiyuan Zeng, Jiaze Chen, Weiran Xu, Lei Li

ORMind: A Cognitive-Inspired End-to-End Reasoning Framework for Operations Research (2025.acl-industry)

Challenge:	Abstractive summarization models often produce inconsistent statements or false facts.
Approach:	They propose an efficient weak-supervised adversarial data augmentation approach to generate factual consistency datasets by backpropagating gradients on token embeddings.
Outcome:	The proposed model can make interpretable factual errors tracing on public datasets and is cost-effective.

Copied to clipboard

Zhiyuan Wang, Bokui Chen, Yinya Huang, Qingxing Cao, Ming He, Jianping Fan, Xiaodan Liang

Reasoning

Challenge:	Large Language Models (LLMs) have shown promising results in various domains, but their practical application in industry-relevant operations research presents significant challenges and opportunities.
Approach:	They propose a cognitive-inspired framework that enhances optimization through counterfactual reasoning . they use a workflow that transforms requirements into mathematical models and executable solver code .
Outcome:	Experiments show that ORMind outperforms existing methods in the NL4Opt dataset and ComplexOR dataset.

Exploring the Potential of Dense Information in Multimodal Alignment (2024.findings-acl)

Copied to clipboard

Zhiyuan Fan, Zhihong Chen, Benyou Wang

StateX: Enhancing RNN Recall via Post-training State Expansion (2026.findings-acl)

Challenge:	Existing methods to enhance captions have limitations such as insufficient detail and excessive hallucinations, resulting in compromised alignment and masking the true potential of dense information.
Approach:	They propose a pipeline that generates highly detailed captions for images that facilitates in-depth analysis of the potential for dense information in multimodal alignment.
Outcome:	The proposed pipeline significantly improves multimodal alignment and compositional reasoning abilities, surpassing hard negative samples.

Copied to clipboard

Xingyu Shen, Yingfa Chen, Zhen Leng Thai, Xu Han, Zhiyuan Liu, Maosong Sun

Different Tunes Played with Equal Skill: Exploring a Unified Optimization Subspace for Parameter-Efficient Tuning (2022.findings-emnlp)

Challenge:	Existing studies show that RNNs with large recurrent states are expensive to train . however, the ability to recall contextual information from long contexts is underperforms them in certain aspects.
Approach:	They propose a framework that expands the states of pre-trained RNNs by scaling them up to 1.3B . they use a recurrent architecture that compresses contextual information into a fixedsize state .
Outcome:	Experiments on models with up to 1.3B parameters show that StateX expands state sizes without incurring high post-training costs or compromising other capabilities.

Copied to clipboard

Jing Yi, Weize Chen, Yujia Qin, Yankai Lin, Ning Ding, Xu Han, Zhiyuan Liu, Maosong Sun, Jie Zhou

LLM×MapReduce: Simplified Long-Sequence Processing using Large Language Models (2025.acl-long)

Challenge:	Existing delta tuning algorithms freeze most of the parameters and only optimize minimal adaptive parameters.
Approach:	They propose to decompose DETs into a unified optimization subspace and conduct optimization within the subspace.
Outcome:	The proposed DETs achieve comparable performance to the original DET and can be transferred to another DET with non-trivial performance.

Copied to clipboard

Zihan Zhou, Chong Li, Xinyi Chen, Shuo Wang, Yu Chao, Zhili Li, Haoyu Wang, Qi Shi, Zhixing Tan, Xu Han, Xiaodong Shi, Zhiyuan Liu, Maosong Sun

Cross-Lingual Transfer of Cultural Knowledge: An Asymmetric Phenomenon (2025.acl-short)

Challenge:	Existing studies have focused on extending the context length of large language models (LLMs) due to their quadratic computational complexity and a lack of high-quality long training examples, most LLMs are trained with a limited window size.
Approach:	They propose a training-free framework that enables large language models to effectively process long texts using a divide-and-conquer strategy for comprehensive document understanding.
Outcome:	The proposed framework outperforms open-source and commercial long-context LLMs and is compatible with several models.

Copied to clipboard

Chen Zhang, Zhiyuan Liao, Yansong Feng

Cross-Lingual Transfer

Challenge:	Existing studies evaluate whether large language models handle global cultural diversity . however, mechanisms behind cultural knowledge acquisition remain unexplored .
Approach:	They propose an interpretable framework to study cultural knowledge transfer in large language models . they observe bidirectional cultural transfer between English and other high-resource languages .
Outcome:	The proposed framework ensures training data transparency and controls transfer effects.

Modeling Semantic Compositionality with Sememe Knowledge (P19-1)

Copied to clipboard

Fanchao Qi, Junjie Huang, Chenghao Yang, Zhiyuan Liu, Xiao Chen, Qun Liu, Maosong Sun

Enhancing Open-Domain Task-Solving Capability of LLMs via Autonomous Tool Integration from GitHub (2025.acl-long)

Challenge:	Semantic compositionality (SC) is defined as the phenomenon that the meaning of a complex linguistic unit can be composed of the meanings of its constituents.
Approach:	They propose to incorporate sememes into SC models and employ them in learning multiword expressions.
Outcome:	The proposed models achieve significant performance boost compared to baseline methods without sememe knowledge.

Copied to clipboard

Bohan Lyu, Xin Cong, Heyang Yu, Pan Yang, Cheng Qian, Zihe Wang, Yujia Qin, Yining Ye, Yaxi Lu, Chen Qian, Zhong Zhang, Yukun Yan, Yankai Lin, Zhiyuan Liu, Maosong Sun

Learning Diverse Responses with Prefix-Conditioned Supervised Fine-Tuning (2026.acl-long)

Challenge:	Existing approaches lack flexibility to address diverse and ever-evolving user queries in open domains.
Approach:	They propose to evaluate LLMs on open-domain knowledge that requires tools to solve diverse and ever-evolving user queries.
Outcome:	The proposed system outperforms baselines in the open domain task-solving benchmark.

Copied to clipboard

Zhiyuan Fan, Guanqiao Chen, Yanyi Huang, Mingkuan Zhao, Dadi Guo, Yi R. Fung

ProSparse: Introducing and Enhancing Intrinsic Activation Sparsity within Large Language Models (2025.coling-main)

Challenge:	Large language models exhibit highly homogeneous, repetitive responses, resulting in inefficient exploration.
Approach:	They propose a method that constructs semantically consistent yet distributionally distinct prior contents to different responses and decouple the one-to-many mapping.
Outcome:	The proposed method improves absolute performance by 5.3% and increases generation diversity by 198.3% on average while significantly enhancing output diversity and test-time scaling.

Copied to clipboard

Chenyang Song, Xu Han, Zhengyan Zhang, Shengding Hu, Xiyu Shi, Kuai Li, Chen Chen, Zhiyuan Liu, Guangli Li, Tao Yang, Maosong Sun