Papers by Maosong Sun

RSMeM: Knowledge-Enhanced Memory Evolution for Remote Sensing Agents with Systematic Evaluation (2026.acl-long)

Challenge:	Existing methods for creating versatile MLLMs rely on joint training with paired instruction data, which is resource-intensive and challenging to extend to new modalities.
Approach:	They propose a new paradigm for multimodal large language models by reusing modality encoders and merging LLM parameters.
Outcome:	The proposed model retains the modal understanding capabilities of each original model.

Copied to clipboard

Bingxian Wu, Yu Zhang, Zonghao Guo, Tang Liu, Chen Qian, Yuxiang Lu, Xingbo Du, Yanghao Li, Yidan Zhang, Chi Chen, Ling Yao, Chenghu Zhou, Maosong Sun

Automatic Label Sequence Generation for Prompting Sequence-to-sequence Models (2022.coling-1)

Challenge:	Existing RS agents built on general-purpose LLMs are domain-agnostic, resulting in brittle and error-prone workflows.
Approach:	They propose a knowledge-enhanced memory evolution mechanism that bootstraps RS agents with pre-distilled domain knowledge and iteratively integrates online experience for robust multi-step tool execution.
Outcome:	Experiments show that the new model improves tool-use performance and accuracy . iteratively, iteration of the model integrates online experience for robust multi-step tool execution .

Copied to clipboard

Zichun Yu, Tianyu Gao, Zhengyan Zhang, Yankai Lin, Zhiyuan Liu, Maosong Sun, Jie Zhou

Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification (2022.acl-long)

Challenge:	Prompting has shown to be sample efficient compared to fine-tuning with pre-trained models.
Approach:	They propose a fully automatic prompting method that uses natural language prompts on sequence-to-sequence models and a beam search method to generate a large amount of label sequence candidates.
Outcome:	The proposed method significantly outperforms other no-manual-design methods on single label words and generates large amount of label sequence candidates.

Copied to clipboard

Shengding Hu, Ning Ding, Huadong Wang, Zhiyuan Liu, Jingang Wang, Juanzi Li, Wei Wu, Maosong Sun

Sparse Low-rank Adaptation of Pre-trained Language Models (2023.emnlp-main)

Challenge:	Recent studies suggest that pre-trained language models have gained rich knowledge during pre-training.
Approach:	They propose to tune pre-trained language models with task-specific prompts to improve and stabilize prompttuning.
Outcome:	Extensive experiments on zero and few-shot text classification tasks show that prompt-tuning improves and stabilizes prompttun-ing.

Copied to clipboard

Ning Ding, Xingtai Lv, Qiaosen Wang, Yulin Chen, Bowen Zhou, Zhiyuan Liu, Maosong Sun

Denoising Distantly Supervised Open-Domain Question Answering (P18-1)

Challenge:	Existing methods for fine-tuning pre-trained large language models in a parameter-efficient manner are gaining traction within the research community.
Approach:	They propose a method of low-rank adaptation that enables dynamic adjustments to the intrinsic rank during the adaptation process.
Outcome:	The proposed approach outperforms the current method with a fixed and unalterable intrinsic rank and a low-rank adaptation process.

Copied to clipboard

Yankai Lin, Haozhe Ji, Zhiyuan Liu, Maosong Sun

Question Answering

Challenge:	Existing DS-QA models ignore rich information contained in other paragraphs and are noisy . Existing systems rely on pre-identified relevant texts, which do not always exist in real-world QA scenarios.
Approach:	They propose a model which uses a paragraph selector to filter out noisy paragraphs and a reader to extract the correct answer from denoised paragraphs.
Outcome:	The proposed model can capture useful information from noisy data and achieve significant improvements on open domain question answering.

From Adversarial Arms Race to Model-centric Evaluation: Motivating a Unified Automatic Robustness Evaluation Framework (2023.findings-acl)

Copied to clipboard

Yangyi Chen, Hongcheng Gao, Ganqu Cui, Lifan Yuan, Dehan Kong, Hanlu Wu, Ning Shi, Bo Yuan, Longtao Huang, Hui Xue, Zhiyuan Liu, Maosong Sun, Heng Ji

CLaMP 3: Universal Music Information Retrieval Across Unaligned Modalities and Unseen Languages (2025.findings-acl)

Challenge:	Existing models of robustness evaluation are incomprehensive, impractical, and invalid .
Approach:	They propose a framework for automatic robustness evaluation that shifts towards model-centric evaluation to further exploit the advantages of adversarial attacks.
Outcome:	The proposed framework is based on a model-centric evaluation protocol and a robustness evaluation protocol.

Copied to clipboard

Shangda Wu, Guo Zhancheng, Ruibin Yuan, Junyan Jiang, SeungHeon Doh, Gus Xia, Juhan Nam, Xiaobing Li, Feng Yu, Maosong Sun

RAGEval: Scenario Specific RAG Evaluation Dataset Generation Framework (2025.acl-long)

Challenge:	Music information retrieval (MIR) is a field that aims at developing computational tools for processing, organizing, and accessing music data.
Approach:	They propose a framework that aligns music modalities with multilingual text in a shared representation space.
Outcome:	Experiments show CLaMP 3 performs state-of-the-art on multiple MIR tasks . it surpasses baselines and shows excellent generalization in multimodal and multilingual contexts .

Copied to clipboard

Kunlun Zhu, Yifan Luo, Dingling Xu, Yukun Yan, Zhenghao Liu, Shi Yu, Ruobing Wang, Shuo Wang, Yishan Li, Nan Zhang, Xu Han, Zhiyuan Liu, Maosong Sun

A Goal Without a Plan Is Just a Wish: Efficient and Effective Global Planner Training for Long-Horizon Agent Tasks (2026.acl-long)

Challenge:	Existing evaluation metrics for RAG systems are lacking due to high costs of data construction and lack of factual accuracy.
Approach:	They propose a framework to evaluate RAG systems in specialized scenarios . they propose three new metrics to evaluate LLM-generated responses .
Outcome:	The proposed framework outperforms zero-shot and one-shot methods in terms of clarity, safety, conformity, and richness of generated samples.

Copied to clipboard

Shuzheng Si, Haozhe Zhao, Kangyang Luo, Gang Chen, Fanchao Qi, Minjia Zhang, Baobao Chang, Maosong Sun

TritonBench: Benchmarking Large Language Model Capabilities for Generating Triton Operators (2025.findings-acl)

Challenge:	Recent advances in large language models (LLMs) have leapt from static chatbots to versatile agents that tackle complex tasks such as science experiments.
Approach:	They propose a plan-and-execute framework and propose 'EAGLET' to enhance the executor agent's planning abilities without human effort.
Outcome:	The proposed method outperforms existing methods on three long-horizon tasks and reduces training costs by 8 compared to baselines.

Copied to clipboard

Jianling Li, ShangZhan Li, Zhenye Gao, Qi Shi, Yuxuan Li, Zefan Wang, Jiacheng Huang, WangHaojie WangHaojie, Jianrong Wang, Xu Han, Zhiyuan Liu, Maosong Sun

RepoAgent: An LLM-Powered Open-Source Framework for Repository-level Code Documentation Generation (2024.emnlp-demo)

Challenge:	Triton is a high-level Python-like programming language for building efficient GPU kernels.
Approach:	They propose a TritonBench benchmark that provides a comprehensive evaluation of Tritonic operators on widely deployed GPUs.
Outcome:	The proposed benchmarks show that current LLMs struggle to generate efficient Triton operators on widely deployed GPUs aligned with industry applications.

Copied to clipboard

Qinyu Luo, Yining Ye, Shihao Liang, Zhong Zhang, Yujia Qin, Yaxi Lu, Yesai Wu, Xin Cong, Yankai Lin, Yingli Zhang, Xiaoyin Che, Zhiyuan Liu, Maosong Sun

Fine-grained Fact Verification with Kernel Graph Attention Network (2020.acl-main)

Challenge:	Xia et al., 2018) demonstrate that a large language model can generate and maintain high-quality code documentation.
Approach:	They propose a large language model powered open-source framework for generating, maintaining, and updating code documentation.
Outcome:	The proposed framework generates high-quality documentation for the entire project.

Copied to clipboard

Zhenghao Liu, Chenyan Xiong, Maosong Sun, Zhiyuan Liu

Coreferential Reasoning Learning for Language Representation (2020.emnlp-main)

Challenge:	Existing methods for fact verification are based on dot-product attentions, but kernel-based attentions focus more on relevant evidence sentences and meaningful clues in the evidence graph.
Approach:	They propose a kernel-based attention network which conducts more fine-grained fact verification with kernel-basic attentions.
Outcome:	The proposed task achieves a 70.38% FEVER score and significantly outperforms existing fact verification models on FEVER, a large-scale benchmark for fact verification.

Copied to clipboard

Deming Ye, Yankai Lin, Jiaju Du, Zhenghao Liu, Peng Li, Maosong Sun, Zhiyuan Liu

On LLM-Based Scientific Inductive Reasoning Beyond Equations (2025.emnlp-main)

Challenge:	Existing language representation models cannot explicitly handle coreference, which is essential to the coherent understanding of the whole discourse.
Approach:	They propose a language representation model that captures coreferential relations in context.
Outcome:	The proposed model can achieve significant improvements on downstream NLP tasks while maintaining comparable performance to baseline models on other common NLP task.

Copied to clipboard

Brian S. Lin, Jiaxin Yuan, Zihan Zhou, Shouli Wang, Shuo Wang, Cunliang Kong, Qi Shi, Yuxuan Li, Liner Yang, Zhiyuan Liu, Maosong Sun

OpenPrompt: An Open-source Framework for Prompt-learning (2022.acl-demo)

Challenge:	Existing research on inductive reasoning models emphasizes rule design without grounding them in specific scenarios.
Approach:	They propose to use LLMs to learn underlying patterns from limited examples in entirely new environments.
Outcome:	The proposed benchmark evaluates the inductive reasoning abilities of large language models in scientific settings.

Copied to clipboard

Ning Ding, Shengding Hu, Weilin Zhao, Yulin Chen, Zhiyuan Liu, Haitao Zheng, Maosong Sun

Exploring Mode Connectivity for Pre-trained Language Models (2022.emnlp-main)

Challenge:	Prompt-learning is a new paradigm in natural language processing, adapting pre-trained language models to cloze-style prediction, autoregressive modeling, or sequence to sequence generation.
Approach:	They propose a framework for prompt-learning that integrates pre-trained language models with a unified framework.
Outcome:	The proposed framework is easy to use and flexible enough to integrate with other frameworks.

Copied to clipboard

Yujia Qin, Cheng Qian, Jing Yi, Weize Chen, Yankai Lin, Xu Han, Zhiyuan Liu, Maosong Sun, Jie Zhou

Packed Levitated Marker for Entity and Relation Extraction (2022.acl-long)

Challenge:	Recent years have witnessed the prevalent application of pre-trained language models (PLMs) in NLP. From the perspective of parameter space, PLMs provide generic initialization, starting from which high-performance minima could be found.
Approach:	They investigate the geometric connections of different minima through the lens of mode connectivity, which measures whether two minima can be connected with a low-loss path.
Outcome:	The proposed model can be used to find low-loss paths between two minima, and to understand how their mode connectivity affects their task knowledge.

Copied to clipboard

Deming Ye, Yankai Lin, Peng Li, Maosong Sun

Knowledge Inheritance for Pre-trained Language Models (2022.naacl-main)

Challenge:	Existing work on entity and relation extraction ignores the interrelation between spans . a novel approach to extract better span representations from pre-trained languages is needed .
Approach:	They propose a span representation approach that packs Levitated Markers to consider interrelation between spans.
Outcome:	The proposed model improves on baselines on six NER benchmarks and achieves a 4.1%-4.3% strict relation F1 improvement with higher speed over previous state-of-the-art models.

Copied to clipboard

Yujia Qin, Yankai Lin, Jing Yi, Jiajie Zhang, Xu Han, Zhengyan Zhang, Yusheng Su, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

Pass off Fish Eyes for Pearls: Attacking Model Selection of Pre-trained Models (2022.acl-long)

Challenge:	Existing large-scale pre-trained language models are mainly trained from scratch individually, ignoring that many well-taught PLMs are available.
Approach:	They propose a pre-training framework called knowledge inheritance and propose auxiliary supervision to efficiently learn larger PLMs.
Outcome:	The proposed framework can be used to train large-scale language models with huge parameters and a large dataset can be adapted to domain adaptation and knowledge transfer.

Copied to clipboard

Biru Zhu, Yujia Qin, Fanchao Qi, Yangdong Deng, Zhiyuan Liu, Maosong Sun, Ming Gu

FaithLens: Detecting and Explaining Faithfulness Hallucination (2026.findings-acl)

Challenge:	Existing feature-based model selection methods are vulnerable to fine-tuning and are not reliable indicators for the PTM’s transferability.
Approach:	They propose feature-based model selection methods which assess PTMs’ transferability to a specific task in a fast way without fine-tuning.
Outcome:	The proposed methods can make FMS mistakenly judge transferability of models and can be combined with the backdoor attack to misguide the FMS to select poisoned models.

Copied to clipboard

Shuzheng Si, Qingyi Wang, Haozhe Zhao, Yuzhuo Bai, Guanqiao Chen, Kangyang Luo, Gang Chen, Fanchao Qi, Minjia Zhang, Baobao Chang, Maosong Sun

Seq1F1B: Efficient Sequence-Level Pipeline Parallelism for Large Language Model Training (2025.naacl-long)

Challenge:	Recent progress in large language models (LLMs) has revolutionized text generation.
Approach:	They propose a faithfulness hallucination detection model that can provide binary predictions and corresponding explanations to improve trustworthiness.
Outcome:	The proposed model outperforms advanced models on 12 diverse tasks.

Copied to clipboard

Sun Ao, Weilin Zhao, Xu Han, Cheng Yang, Xinrong Zhang, Zhiyuan Liu, Chuan Shi, Maosong Sun

LoRA-Flow: Dynamic LoRA Fusion for Large Language Models in Generative Tasks (2024.acl-long)

Challenge:	Current PP methods face severe bottlenecks, including pipeline bubbles and memory footprint.
Approach:	They propose a sequence-level one-forward-one-backward (1F1B) PP method for training LLMs on long sequences with high throughput and memory efficiency.
Outcome:	The proposed method achieves 1.14X training throughput with half memory footprint compared to baseline methods . it trains an LLM with 30B parameters on sequences up to 64k tokens using 64X NVIDIA A100 GPUs .

Copied to clipboard

Hanqing Wang, Bowen Ping, Shuo Wang, Xu Han, Yun Chen, Zhiyuan Liu, Maosong Sun

More Data, More Relations, More Context and More Openness: A Review and Outlook for Relation Extraction (2020.aacl-main)

Challenge:	LoRA-Flow uses lightweight modules to customize large language models for downstream tasks . previous work on LoRA combination relied on task-level weights for each involved LoRA .
Approach:	They propose a LoRA-Flow approach that uses dynamic weights to adjust the impact of different LoRAs.
Outcome:	The proposed method outperforms baselines with task-level weights on six generative tasks.

Copied to clipboard

Xu Han, Tianyu Gao, Yankai Lin, Hao Peng, Yaoliang Yang, Chaojun Xiao, Zhiyuan Liu, Peng Li, Jie Zhou, Maosong Sun

Tell Me More! Towards Implicit User Intention Understanding of Language Model Driven Agents (2024.acl-long)

Challenge:	Existing methods for extracting relational facts from text have been successful . but with explosion of Web text, human knowledge is increasing drastically .
Approach:	They propose to improve relation extraction methods to extract relational facts from text . they analyze existing methods and show promising directions towards more powerful RE .
Outcome:	The proposed methods can extract relational facts from text, but they are still lacking in the current field.

Copied to clipboard

Cheng Qian, Bingxiang He, Zhong Zhuang, Jia Deng, Yujia Qin, Xin Cong, Zhong Zhang, Jie Zhou, Yankai Lin, Zhiyuan Liu, Maosong Sun

Textual Backdoor Attacks Can Be More Harmful via Two Simple Tricks (2022.emnlp-main)

Challenge:	Current language model-driven agents lack mechanisms for effective user participation, which is crucial given the vagueness commonly found in user instructions.
Approach:	They propose a benchmark to inspect users’ implicit intentions through explicit queries and a model expert as the upstream in agent design to enhance user-agent interaction.
Outcome:	The proposed approach excels at identifying vague user tasks, recovering and summarizing critical missing information, setting precise and necessary agent execution goals, and minimizing redundant tool usage, thus boosting overall efficiency.

Copied to clipboard

Yangyi Chen, Fanchao Qi, Hongcheng Gao, Zhiyuan Liu, Maosong Sun

LLM×MapReduce-V3: Enabling Interactive In-Depth Survey Generation through a MCP-Driven Hierarchically Modular Agent System (2025.emnlp-demos)

Challenge:	Existing textual backdoor attacks are vulnerable to backdoors . researchers add extra training task to distinguish poisoned and clean data .
Approach:	They propose two tricks that make existing backdoor attacks much more harmful . first trick is to add an extra task to distinguish poisoned and clean data . second trick is using all the clean training data rather than the original clean data.
Outcome:	The proposed tricks can significantly improve attack performance in three tough situations including clean data fine-tuning, low-poisoning-rate, and label-consistent attacks.

Copied to clipboard

Yu Chao, Siyu Lin, Xiaorong Wang, Zhu Zhang, Zihan Zhou, Haoyu Wang, Shuo Wang, Jie Zhou, Zhiyuan Liu, Maosong Sun

Challenge:	Generating high-quality long-form survey articles poses significant challenges to AI Agent systems.
Approach:	They propose a hierarchically modular agent system for long-form survey generation . they use atomic models to implement skeleton initialization, digest construction, and skelet refinement . human evaluations demonstrate system surpasses representative baselines .
Outcome:	The proposed system surpasses representative baselines in both content depth and length, highlighting the strength of MCP-based modular planning.

Improving Back-Translation with Uncertainty-based Confidence Estimation (D19-1)

Copied to clipboard

Shuo Wang, Yang Liu, Chao Wang, Huanbo Luan, Maosong Sun

Why Should Adversarial Perturbations be Imperceptible? Rethink the Research Paradigm in Adversarial NLP (2022.emnlp-main)

Challenge:	Despite the success of low-resource neural machine translation, there is a data scarcity problem in many languages . large-scale, high-quality, and widecoverage bilingual corpora do not exist for most language pairs .
Approach:	They propose to quantify confidence of NMT models based on model uncertainty . they propose to use uncertainty-based confidence measures to improve back-translation .
Outcome:	The proposed model outperforms conventional statistical machine translation (SMT) on Chinese-English and English-German translation tasks.

Copied to clipboard

Yangyi Chen, Hongcheng Gao, Ganqu Cui, Fanchao Qi, Longtao Huang, Zhiyuan Liu, Maosong Sun

Adversarial Training for Weakly Supervised Event Detection (N19-1)

Challenge:	Textual adversarial samples are often misrepresented in research on security, evaluation, explainability, and data augmentation.
Approach:	They propose to use adversarial samples to evaluate their methods on security tasks to demonstrate the real-world concerns rather than developing impractical methods.
Outcome:	The proposed method has higher practical value than the current benchmark.

Copied to clipboard

Xiaozhi Wang, Xu Han, Zhiyuan Liu, Maosong Sun, Peng Li

KACC: A Multi-task Benchmark for Knowledge Abstraction, Concretization and Completion (2021.findings-acl)

Challenge:	Detecting and identifying events is an important subtask of event extraction.
Approach:	They build a large event-related candidate set with good coverage and apply an adversarial training mechanism to iteratively identify informative instances from the candidate set and filter out those noisy ones.
Outcome:	The proposed method significantly outperforms the state-of-the-art methods on two real-world datasets.

Copied to clipboard

Jie Zhou, Shengding Hu, Xin Lv, Cheng Yang, Zhiyuan Liu, Wei Xu, Jie Jiang, Juanzi Li, Maosong Sun

OpenDelta: A Plug-and-play Library for Parameter-efficient Adaptation of Pre-trained Models (2023.acl-demo)

Challenge:	Existing studies focus on partial aspects of knowledge abstraction, concretization, and completion (KACC).
Approach:	They propose a unified knowledge graph benchmark to improve existing benchmarks . they collect new datasets that contain larger concept graphs and cross-view links .
Outcome:	The proposed benchmark improves existing benchmarks in terms of dataset scale, task coverage, and difficulty.

Copied to clipboard

Shengding Hu, Ning Ding, Weilin Zhao, Xingtai Lv, Zhen Zhang, Zhiyuan Liu, Maosong Sun

Automatic Poetry Generation with Mutual Reinforcement Learning (D18-1)

Challenge:	Existing implementations that modify the code of the backbone PTMs and hard-code specific delta tuning methods for each PTM have limited the practicality and flexibility of delta tuning.
Approach:	They propose an open-source library that provides a plug-and-play implementation of delta tuning methods for pre-trained models.
Outcome:	The proposed methods eliminate the need to modify the backbone PTMs’ code, making OpenDelta compatible with different, even novel PTM.

Copied to clipboard

Xiaoyuan Yi, Maosong Sun, Ruoyu Li, Wenhao Li

Browse and Concentrate: Comprehending Multimodal Content via Prior-LLM Context Fusion (2024.acl-long)

Challenge:	Existing models for automatic poetry generation are based on maximum likelihood estimation (MLE) MLE-based models tend to remember common patterns of the poetry corpus, which results in loss-evaluation mismatch.
Approach:	They propose to model the criteria and use them as explicit rewards to guide gradient update by reinforcement learning to motivate the model to pursue higher scores.
Outcome:	The proposed model outperforms the current state-of-the-art model and improves on Chinese poetry.

Copied to clipboard

Ziyue Wang, Chi Chen, Yiqi Zhu, Fuwen Luo, Peng Li, Ming Yan, Ji Zhang, Fei Huang, Maosong Sun, Yang Liu

Meta-Information Guided Meta-Learning for Few-Shot Relation Classification (2020.coling-main)

Challenge:	Multimodal Large Language Models (MLLMs) lack understanding of multi-image and interleaved inputs due to the visual features encoded by frozen encoders before being fed into the LLM backbone.
Approach:	They propose a two phase paradigm to enable in-depth multimodal context fusion prior to feeding the features into LLMs.
Outcome:	The proposed paradigm boosts the performance on 7 multi-image scenarios, contributing to increments on average accuracy by 2.13% and 7.60% against strong MLLMs baselines with 3B and 11B LLMs, respectively.

Copied to clipboard

Bowen Dong, Yuan Yao, Ruobing Xie, Tianyu Gao, Xu Han, Zhiyuan Liu, Fen Lin, Leyu Lin, Maosong Sun

Revealing the Attention Floating Mechanism in Masked Diffusion Models (2026.findings-acl)

Challenge:	Existing meta-learning models rely on implicit instance statistics and are unreliability and weak interpretability.
Approach:	They propose a meta-information guided meta-learning framework that uses semantics to guide meta- learning . experimental results demonstrate the effectiveness of the proposed framework .
Outcome:	The proposed framework can establish connections between instance-based information and semantic-based data, enabling faster initialization and adaptation.

Copied to clipboard

Xin Dai, Pengcheng Huang, Zhenghao Liu, Shuo Wang, Yukun Yan, Chaojun Xiao, Yu Gu, Ge Yu, Maosong Sun

PersLLM: A Personified Training Approach for Large Language Models (2025.findings-emnlp)

Challenge:	Masked diffusion models (MDMs) leverage bidirectional attention and a denoising process.
Approach:	They investigate the attention behaviors of Masked diffusion models by revealing the phenomenon of Attention Floating.
Outcome:	The proposed model doubles the performance of autoregressive models in knowledge-intensive tasks.

Copied to clipboard

Zheni Zeng, Jiayi Chen, Huimin Chen, Yukun Yan, Yuxuan Chen, Zhenghao Liu, Zhiyuan Liu, Maosong Sun

Optima: Optimizing Effectiveness and Efficiency for LLM-Based Multi-Agent System (2025.findings-acl)

Challenge:	Large language models exhibit human-like intelligence, enabling them to simulate human behavior and support various applications that require both humanized communication and extensive knowledge reserves.
Approach:	They propose a framework for better data construction and model tuning to unlock the potential of LLM personification by using Chain-of-Thought prompting and anti-induction.
Outcome:	The proposed framework improves data construction and model tuning for insufficient data usage and rigid behavior patterns.

Copied to clipboard

Weize Chen, Jiarui Yuan, Chen Qian, Cheng Yang, Zhiyuan Liu, Maosong Sun

Adapting Open Domain Fact Extraction and Verification to COVID-FACT through In-Domain Language Modeling (2020.findings-emnlp)

Challenge:	Large Language Models (LLMs) have emerged as powerful tools for a wide range of tasks, from * Equal Contribution.
Approach:	They propose a framework that enhances communication efficiency and task effectiveness in LLM-based multi-agent systems through training.
Outcome:	The proposed framework improves communication efficiency and task effectiveness on multi-agent tasks with 2.8x performance gain with less than 10% tokens on tasks requiring heavy information exchange.

Copied to clipboard

Zhenghao Liu, Chenyan Xiong, Zhuyun Dai, Si Sun, Maosong Sun, Zhiyuan Liu

Segment, Mask, and Predict: Augmenting Chinese Word Segmentation with Self-Supervision (2021.emnlp-main)

Challenge:	Existing methods to verify scientifically false online information are limited by the lack of training data in the scientific domain.
Approach:	They propose an in-domain language modeling method for fact extraction and verification systems . they use SCIFACT to extract scientifically false online information .
Outcome:	The proposed method improves accuracy 30% on SCIFACT dataset . state-of-the-art model achieves only 46.6% precision, which is hard to be trusted for users.

Copied to clipboard

Mieradilijiang Maimaiti, Yang Liu, Yuanhang Zheng, Gang Chen, Kaiyu Huang, Ji Zhang, Huanbo Luan, Maosong Sun

ChartCoder: Advancing Multimodal Large Language Model for Chart-to-Code Generation (2025.acl-long)

Challenge:	Recent state-of-the-art (SOTA) effective neural network methods have been used in Chinese word segmentation (CWS) However, the robustness of the previous neural methods is limited by the large-scale annotated corpus.
Approach:	They propose a self-supervised Chinese word segmentation approach with a straightforward and effective architecture.
Outcome:	The proposed approach outperforms previous methods on 9 different CWS datasets with single criterion training and multiple criteria training and achieves better robustness.

Copied to clipboard

Xuanle Zhao, Xianzhen Luo, Qi Shi, Chi Chen, Shuo Wang, Zhiyuan Liu, Maosong Sun

Looking Beyond Text: Reducing Language Bias in Large Vision-Language Models via Multimodal Dual-Attention and Soft-Image Guidance (2025.emnlp-main)

Challenge:	Existing open-source MLLMs fail to fully capture dense information embedded in charts . current models still face significant challenges in understanding and analyzing visual tasks such as captioning and question answering.
Approach:	They propose a chart-to-code MLLM which leverages Code LLMs as the language backbone to enhance the executability of the generated code.
Outcome:	The proposed model surpasses existing open-source models on chart-to-code benchmarks with only 7B parameters and provides lossless representations that contain all critical details.

Copied to clipboard

Haozhe Zhao, Shuzheng Si, Liang Chen, Yichi Zhang, Maosong Sun, Baobao Chang, Minjia Zhang

Visual Data in NLP

Challenge:	Large vision-language models (LVLMs) have been criticized for their language bias.
Approach:	They propose to use a dual-attention mechanism to construct separate attention for visual and text inputs to enhance integration of visual inputs across models.
Outcome:	Experiments show that the proposed model debiases LVLMs from their language bias, enhancing visual comprehension and reducing hallucinations without additional resources.

AgentCPM-GUI: Building Mobile-Use Agents with Reinforcement Fine-Tuning (2025.emnlp-demos)

Copied to clipboard

Zhong Zhang, Yaxi Lu, Yikun Fu, Yupeng Huo, Shenzhi Yang, Yesai Wu, Han Si, Xin Cong, Haotian Chen, Yankai Lin, Xie Xie, Wei Zhou, Wang Xu, Zhou Su, Zhongwu Zhai, Xiaoming Liu, null Meiyudong, Jianming Xu, Hongyan Tian, Chongyi Wang, Chi Chen, Yuan Yao, Zhiyuan Liu, Maosong Sun

KBAlign: Efficient Self Adaptation on Specific Textual Knowledge Bases (2025.findings-emnlp)

Challenge:	Large language model agents have enabled GUI-based automation, but their deployment is limited by noisy data, poor generalization, and lack of support for non-English GUIs.
Approach:	They propose an 8B-parameter GUI agent built for robust and efficient on-device GUI interaction.
Outcome:	The proposed GUI agent achieves promising performance on five public benchmarks and proposed Chinese benchmark CAGUI.

Copied to clipboard

Zheni Zeng, Yuxuan Chen, Shi Yu, Ruobing Wang, Yukun Yan, Zhenghao Liu, Shuo Wang, Xu Han, Zhiyuan Liu, Maosong Sun

Document Segmentation Matters for Retrieval-Augmented Generation (2025.findings-acl)

Challenge:	Existing methods for retrieval-augmented generation (RAG) are limited and fine-tuning incurs prohibitive costs of external signals.
Approach:	They propose a self-supervised framework that enhances RAG systems through efficient model adaptation.
Outcome:	The proposed framework achieves 90% of the performance gain obtained through GPT-4-supervised adaptation while relying entirely on self-annotation of much smaller models.

Copied to clipboard

Zhitong Wang, Cheng Gao, Chaojun Xiao, Yufei Huang, Shuzheng Si, Kangyang Luo, Yuzhuo Bai, Wenhao Li, Tangjian Duan, Chuancheng Lv, Guoshan Lu, Gang Chen, Fanchao Qi, Maosong Sun

Judge as A Judge: Improving the Evaluation of Retrieval-Augmented Generation through the Judge-Consistency of Large Language Models (2025.findings-acl)

Challenge:	Existing rule-based chunking methods lead to suboptimal splits, where overly large chunks introduce irrelevant information and small chunks lack semantic coherence.
Approach:	They propose a method that leverages document summaries as pseudo-instructions to guide chunking by computing semantic similarity between sentences and the summary.
Outcome:	Experiments on multiple open-domain question-answering benchmarks show that PIC significantly improves retrieval accuracy (Hits@k) and end-to-end QA performance (Exact Match) without any additional training.

Copied to clipboard

Shuliang Liu, Xinze Li, Zhenghao Liu, Yukun Yan, Cheng Yang, Zheni Zeng, Zhiyuan Liu, Maosong Sun, Ge Yu

Controllable Preference Optimization: Toward Controllable Multi-Objective Alignment (2024.emnlp-main)

Challenge:	Existing evaluation metrics cannot fairly evaluate the outputs of RAG models during training and evaluation.
Approach:	They propose a method which prompts LLMs to generate different judgments based on various combinations of judgment dimensions and utilizes the judge-consistency to evaluate these judgments.
Outcome:	The proposed method generates more accurate evaluations for RAG models across different RAG model and datasets.

Copied to clipboard

Yiju Guo, Ganqu Cui, Lifan Yuan, Ning Ding, Zexu Sun, Bowen Sun, Huimin Chen, Ruobing Xie, Jie Zhou, Yankai Lin, Zhiyuan Liu, Maosong Sun

Challenge:	Existing algorithms for achieving optimal alignment are mostly unidirectional . a recent study suggests that large language models can be ground with evident preferences .
Approach:	They propose to ground large language models with evident preferences . they propose to use controllable preference optimization to specify different objectives .
Outcome:	The proposed models can provide responses that match various preferences among the ”3H” desiderata.

APB: Accelerating Distributed Long-Context Inference by Passing Compressed Context Blocks across GPUs (2025.acl-long)

Copied to clipboard

Yuxiang Huang, Mingye Li, Xu Han, Chaojun Xiao, Weilin Zhao, Sun Ao, Hao Zhou, Jie Zhou, Zhiyuan Liu, Maosong Sun

Turn the Combination Lock: Learnable Textual Backdoor Attacks via Word Substitution (2021.acl-long)

Challenge:	Long-context inference is crucial for advancing large language models, but its prefill speed remains a bottleneck.
Approach:	They propose an efficient long-context inference framework that leverages multi-host approximate attention to enhance prefill speed.
Outcome:	The proposed framework achieves speedups of 9.2, 4.2, and 1.6 without any degradation in performance.

Copied to clipboard

Fanchao Qi, Yuan Yao, Sophia Xu, Zhiyuan Liu, Maosong Sun

Reducing Word Omission Errors in Neural Machine Translation: A Contrastive Learning Approach (P19-1)

Challenge:	Recent studies show that neural natural language processing models are vulnerable to backdoor attacks.
Approach:	They propose to inject neural models with backdoors activated by word substitution . their results raise a serious alarm to the security of NLP models, they argue .
Outcome:	The proposed backdoors are activated by a learnable combination of word substitution and exhibit higher invisibility than previous methods.

Copied to clipboard

Zonghan Yang, Yong Cheng, Yang Liu, Maosong Sun

GEAR: Graph-based Evidence Aggregating and Reasoning for Fact Verification (P19-1)

Challenge:	Existing methods for reducing word omission errors in neural machine translation are prone to omit essential words on the source side.
Approach:	They propose a contrastive learning approach to reduce word omission errors in NMT by omitting words.
Outcome:	The proposed approach achieves better translation performance than baseline methods on Chinese-to-English, German-to English, and Russian-toEnglish translation tasks.

Copied to clipboard

Jie Zhou, Xu Han, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, Maosong Sun

Prompt Tuning for Discriminative Pre-trained Language Models (2022.findings-acl)

Challenge:	Existing methods to extract information from evidence are unable to grasp relational and logical information among the evidence.
Approach:	They propose a graph-based evidence aggregating and reasoning framework to integrate evidence from multiple pieces of evidence.
Outcome:	The proposed framework achieves significant performance improvements on a large-scale benchmark dataset.

Copied to clipboard

Yuan Yao, Bowen Dong, Ao Zhang, Zhengyan Zhang, Ruobing Xie, Zhiyuan Liu, Leyu Lin, Maosong Sun, Jianyong Wang

Word-level Textual Adversarial Attacking as Combinatorial Optimization (2020.acl-main)

Challenge:	Recent studies have shown promising results of prompt tuning in stimulating pre-trained language models (PLMs) for natural language processing tasks.
Approach:	They propose a prompt tuning framework that reformulates NLP tasks into a discriminative language modeling problem.
Outcome:	The proposed framework improves on text classification and question answering tasks and prevents unstable tuning problems in low-resource settings.

Copied to clipboard

Yuan Zang, Fanchao Qi, Chenghao Yang, Zhiyuan Liu, Meng Zhang, Qun Liu, Maosong Sun

Sub-Character Tokenization for Chinese Pretrained Language Models (2023.tacl-1)

Challenge:	Existing word-level attack models are far from perfect because of unsuitable search space reduction methods and inefficient optimization algorithms.
Approach:	They propose a novel adversarial adversarialist model that incorporates word substitution and particle swarm optimization to solve two problems separately.
Outcome:	The proposed model achieves much higher success rates and crafts more high-quality adversarial examples as compared to baseline methods.

Copied to clipboard

Chenglei Si, Zhengyan Zhang, Yingfa Chen, Fanchao Qi, Xiaozhi Wang, Zhiyuan Liu, Yasheng Wang, Qun Liu, Maosong Sun

PEVL: Position-enhanced Pre-training and Prompt Tuning for Vision-language Models (2022.emnlp-main)

Challenge:	Existing tokenization methods for Chinese PLMs treat each character as an indivisible token, but ignore the unique feature of the writing system where additional linguistic information exists below the character level.
Approach:	They propose to encode Chinese characters into short sequences and construct Chinese vocabulary based on the encoded text.
Outcome:	The proposed tokenizers can tokenize inputs into much shorter sequences, improving computational efficiency.

Copied to clipboard

Yuan Yao, Qianyu Chen, Ao Zhang, Wei Ji, Zhiyuan Liu, Tat-Seng Chua, Maosong Sun

Musical Score Understanding Benchmark: Evaluating Large Language Models’ Comprehension of Complete Musical Scores (2026.acl-long)

Challenge:	Recent advances on self-supervised learning have led to powerful vision-language pre-training models that achieve state-of-the-art performance on a wide range of cross-modal tasks.
Approach:	They propose a vision-language pre-training framework that reformulates discretized object positions and language in a unified language modeling framework.
Outcome:	The proposed model improves performance on position-sensitive vision-language (VL) tasks and also improves on position insensitive tasks.

Copied to clipboard

Congren Dai, Yue Yang, Krinos Li, Huichi Zhou, Shijie Liang, Zhang Bo, Enyang Liu, Ge Jin, Hongran An, Haosen Zhang, Peiyuan Jing, KinHei Lee, Zhenxuan Zhang, Xiaobing Li, Maosong Sun

Evade the Trap of Mediocrity: Promoting Diversity and Novelty in Text Generation via Concentrating Attention (2022.emnlp-main)

Challenge:	Existing benchmarks for musical score understanding are narrow in scope, focusing on isolated fragments, short excerpts, or multiple-choice formulations, rather than supporting holistic reasoning over entire scores.
Approach:	They propose a benchmark for score-level musical understanding across textual and visual modalities.
Outcome:	The musical score understanding benchmark contains 1,800 question-answer pairs from works by Bach, Beethoven, Chopin, Debussy, and others.

Copied to clipboard

Wenhao Li, Xiaoyuan Yi, Jinyi Hu, Maosong Sun, Xing Xie

TR-BERT: Dynamic Token Reduction for Accelerating BERT Inference (2021.naacl-main)

Challenge:	Recent studies have shown that powerful Transformer architectures produce dull high-frequency phrases, severely hurting the diversity and novelty of generated text.
Approach:	They propose a method to control the sharpness of the attention distribution by python code and use it to learn a Bayesian approximation of posterior attention.
Outcome:	The proposed method improves diversity and novelty while maintaining comparable quality on conditional and unconditional generation tasks.

Copied to clipboard

Deming Ye, Yankai Lin, Yufei Huang, Maosong Sun

Enhancing Long-Chain Reasoning Distillation through Error-Aware Self-Reflection (2026.findings-acl)

Challenge:	Existing pre-trained language models (PLMs) are expensive in inference, making them impractical in resource-limited real-world applications.
Approach:	They propose a dynamic token reduction approach to accelerate PLMs' inference by adapting the layer number of each token to avoid redundant calculation.
Outcome:	The proposed approach speeds up BERT by 2-5 times and improves performance in long-text tasks with less computation.

Copied to clipboard

Zhuoyang Wu, Xinze Li, Zhenghao Liu, Yukun Yan, Zhiyuan Liu, Minghe Yu, Cheng Yang, Yu Gu, Ge Yu, Maosong Sun

Beyond Natural Language: LLMs Leveraging Alternative Formats for Enhanced Reasoning and Communication (2024.findings-emnlp)

Challenge:	Existing studies treat SLMs as student models and use long-form Chains-of-Thought (CoTs) as supervision signals for Supervised Fine-Tuning (SFT). Existing research focuses on distilling reasoning ability from LLMs to enhance the mathematical reasoning performance of small-scale models.
Approach:	They propose a framework that refines teacher CoTs through an error-aware reflection process to enable the student model to construct more tailored teacher Cots.
Outcome:	Experiments on multiple mathematical reasoning benchmarks show that ORION improves performance by more than 2% over all baselines.

Copied to clipboard

Weize Chen, Chenfei Yuan, Jiarui Yuan, Yusheng Su, Chen Qian, Cheng Yang, Ruobing Xie, Zhiyuan Liu, Maosong Sun

OpenNRE: An Open and Extensible Toolkit for Neural Relation Extraction (D19-3)

Challenge:	Natural language (NL) has long been the predominant format for human cognition and communication, but its utility in LLMs has not been thoroughly examined.
Approach:	They propose to allow LLMs to choose the most suitable format before reasoning or communicating, and to automate the selection process.
Outcome:	The proposed format improves reasoning efficiency and reduces token usage while maintaining communicative effectiveness.

Copied to clipboard

Xu Han, Tianyu Gao, Yuan Yao, Deming Ye, Zhiyuan Liu, Maosong Sun

Stochastic Bridges as Effective Regularizers for Parameter-Efficient Tuning (2023.findings-acl)

Challenge:	OpenNRE provides a framework to implement neural relation extraction (RE) . the toolkit provides various functional modules based on TensorFlow and PyTorch .
Approach:	OpenNRE is an open-source framework to implement neural relation extraction models. they also release an online system to meet real-time extraction without any training and deployment.
Outcome:	OpenNRE provides a framework to implement neural models for relation extraction (RE) the toolkit also includes an online system to meet real-time extraction without training and deployment .

Copied to clipboard

Weize Chen, Xu Han, Yankai Lin, Zhiyuan Liu, Maosong Sun, Jie Zhou

READIN: A Chinese Multi-Task Benchmark with Realistic and Diverse Input Noises (2023.acl-long)

Challenge:	Existing methods for tuning pre-trained language models ignore the running cost and only optimize the terminal cost.
Approach:	They propose to use stochastic bridges to regularize intermediate states and use regularization as running cost of PETs.
Outcome:	The proposed methods can be used to tune large pre-trained language models . they can be compared to full-parameter fine-tuning by tuning a small number of parameters .

Copied to clipboard

Chenglei Si, Zhengyan Zhang, Yingfa Chen, Xiaozhi Wang, Zhiyuan Liu, Maosong Sun

ONION: A Simple and Effective Defense Against Textual Backdoor Attacks (2021.emnlp-main)

Challenge:	Existing benchmarks for Chinese inputs often lack a realistic representation of real-world noises.
Approach:	They construct a Chinese multi-task benchmark with REalistic and Diverse input noises . they use pinyin input and speech input to recruit speakers from diverse dialects based on their inputs - a feature that is important for Chinese NLP benchmarks if it is implemented in real-world applications.
Outcome:	The proposed benchmarks are based on four different tasks and are designed to maximize diversity.

Copied to clipboard

Fanchao Qi, Yangyi Chen, Mukai Li, Yuan Yao, Zhiyuan Liu, Maosong Sun

FastFiD: Improve Inference Efficiency of Open Domain Question Answering via Sentence Selection (2024.acl-long)

Challenge:	Backdoor attacks can manipulate the output of deep neural networks and possess high insidiousness.
Approach:	They propose a textual backdoor defense based on outlier word detection that can handle all the textual attacks.
Outcome:	The proposed method can handle all the textual backdoor attack situations.

Copied to clipboard

Yufei Huang, Xu Han, Maosong Sun

Natural Language Interfaces

Challenge:	Open Domain Question Answering (ODQA) is a longstanding task in Natural Language Processing that involves generating an answer solely based on a given question.
Approach:	They propose a novel approach that executes sentence selection on the encoded passages to enhance the inference speed while reducing the context length required for generating answers.
Outcome:	The proposed approach can increase inference speed by 2.3X-5.7X while maintaining the model’s performance.

Fusing Highly Specialized Language Models for Comprehensive Expertise (2025.acl-long)

Copied to clipboard

Ning Ding, Yulin Chen, Ganqu Cui, Xingtai Lv, Weilin Zhao, Kaiyan Zhang, Ruobing Xie, Bowen Zhou, Zhiyuan Liu, Maosong Sun

Ouroboros: Generating Longer Drafts Phrase by Phrase for Faster Speculative Decoding (2024.emnlp-main)

Challenge:	Existing models that focus on language, programming code, and mathematical symbols are not able to achieve mastery of all three domains simultaneously.
Approach:	They propose to fuse highly-specialized models that are already sufficiently trained on different domains to achieve a highly-specific model.
Outcome:	The proposed model could achieve mastery of the three crucial domains simultaneously.

Copied to clipboard

Weilin Zhao, Yuxiang Huang, Xu Han, Wang Xu, Chaojun Xiao, Xinrong Zhang, Yewei Fang, Kaihuo Zhang, Zhiyuan Liu, Maosong Sun

Challenge:	Speculative decoding is a widely used method that accelerates the generation process of large language models (LLMs) drafting efficiency has become a bottleneck in the final speedup of speculative drafting, therefore generating longer drafts at less cost can lead to better speedup.
Approach:	They propose a method that uses existing model to drafting and target LLM to verify draft in a low-cost parallel manner.
Outcome:	The proposed method can achieve speedups of up to 2.4 over speculative decoding and 3.9 over vanilla decoding without fine-tuning draft and target models.

Emergent Modularity in Pre-trained Transformers (2023.findings-acl)

Copied to clipboard

Zhengyan Zhang, Zhiyuan Zeng, Yankai Lin, Chaojun Xiao, Xiaozhi Wang, Xu Han, Zhiyuan Liu, Ruobing Xie, Maosong Sun, Jie Zhou

Going “Deeper”: Structured Sememe Prediction via Transformer with Tree Attention (2022.findings-acl)

Challenge:	Existing studies on pre-trained Transformers show that they learn fine-grained neuron functions.
Approach:	They examine the presence of modularity in pre-trained Transformers . they focus on Mixture-of-Experts, a promising candidate for modularity .
Outcome:	The proposed structure stabilizes at the early stage, which is faster than neuron stabilization.

Copied to clipboard

Yining Ye, Fanchao Qi, Zhiyuan Liu, Maosong Sun

Decoder Tuning: Efficient Language Understanding as Decoding (2023.acl-long)

Challenge:	Existing studies ignore hierarchical structures of sememes in sememe-based semantic description systems.
Approach:	They propose a structured sememe prediction problem to predict a sememes tree with hierarchical structures rather than a set of sememas.
Outcome:	The proposed model outperforms baseline models and shows its effectiveness . it predicts a sememe tree with hierarchical structures rather than a set of sememes .

Copied to clipboard

Ganqu Cui, Wentao Li, Ning Ding, Longtao Huang, Zhiyuan Liu, Maosong Sun

FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation (D18-1)

Challenge:	Existing approaches to adapt pre-trained models with parameters frozen are based on input-side adaptation, which requires thousands of API queries.
Approach:	They propose to train a model-as-a-service (MaaS) setting to provide only the inference APIs for users . they argue that input-side adaptation could be arduous due to the lack of gradient signals .
Outcome:	The proposed model outperforms state-of-the-art algorithms with a 200x speed-up.

Copied to clipboard

Xu Han, Hao Zhu, Pengfei Yu, Ziyun Wang, Yuan Yao, Zhiyuan Liu, Maosong Sun

Low-Resource NLP

Challenge:	Empirical results show that even the most competitive few-shot learning models struggle on this task, especially as compared with humans.
Approach:	They propose a Few-Shot Relation Classification Dataset consisting of 70, 000 sentences on 100 relations derived from Wikipedia and annotated by crowdworkers.
Outcome:	The proposed methods perform well on the most competitive few-shot learning models, especially as compared with humans.

CLaMP 2: Multimodal Music Information Retrieval Across 101 Languages Using Large Language Models (2025.findings-naacl)

Copied to clipboard

Shangda Wu, Yashan Wang, Ruibin Yuan, Guo Zhancheng, Xu Tan, Ge Zhang, Monan Zhou, Jing Chen, Xuefeng Mu, Yuejie Gao, Yuanliang Dong, Jiafeng Liu, Xiaobing Li, Feng Yu, Maosong Sun

From Unaligned to Aligned: Scaling Multilingual LLMs with Multi-Way Parallel Corpora (2025.emnlp-main)

Challenge:	Current music information retrieval systems struggle to meet linguistic diversity challenges . current systems struggle with text queries in non-English languages .
Approach:	They propose a music information retrieval system that supports both ABC notation and MIDI . CLaMP 2 includes a multilingual text encoder and a multiple-modal music encoder .
Outcome:	The proposed system achieves state-of-the-art results in multilingual semantic search and music classification across modalities.

Copied to clipboard

Yingli Shen, Wen Lai, Shuo Wang, Ge Gao, Kangyang Luo, Alexander Fraser, Maosong Sun

MEIC-DT: Memory-Efficient Incremental Clustering for Long-Text Coreference Resolution with Dual-Threshold Constraints (2026.findings-acl)

Challenge:	Experiments show that models trained on multi-way parallel data outperform those trained on unaligned data.
Approach:	They propose a large-scale, high-quality multi-way parallel corpus based on TED Talks that spans 113 languages with up to 50 languages aligned in parallel.
Outcome:	The proposed model outperforms models trained on unaligned multilingual data on six multilingual benchmarks.

Copied to clipboard

Kangyang Luo, Shuzheng Si, Yuzhuo Bai, Cheng Gao, Zhitong Wang, Cheng Huang, Yingli Shen, Yufeng Han, Wenhao Li, Cunliang Kong, Maosong Sun

Fantastic Semantics and Where to Find Them: Investigating Which Layers of Generative LLMs Reflect Lexical Semantics (2024.findings-acl)

Challenge:	Existing supervised neural methods are underexplored for coreference resolution, especially in incremental clustering.
Approach:	They propose a dual-threshold incremental clustering approach based on a lightweight Transformer.
Outcome:	Experiments on common benchmarks show that MEIC-DT achieves highly competitive coreference performance under stringent memory constraints.

Copied to clipboard

Zhu Liu, Cunliang Kong, Ying Liu, Maosong Sun

Automatic Construction of Sememe Knowledge Bases via Dictionaries (2021.findings-acl)

Challenge:	Existing research suggests that contextual representations of large language models exhibit subpar performance in downstream tasks, struggling to fully capture the semantic nuances of words.
Approach:	They investigate the bottom-up evolution of lexical semantics for a popular LLM . they probing its hidden states at the end of each layer using a contextualized word identification task .
Outcome:	The proposed model is able to encode lexical semantics in lower layers while achieving weaker induction in higher layers.

Copied to clipboard

Fanchao Qi, Yangyi Chen, Fengyu Wang, Zhiyuan Liu, Xiao Chen, Maosong Sun

Adversarial Multi-lingual Neural Relation Extraction (C18-1)

Challenge:	Sememe knowledge bases (SKBs) are used to analyze natural language processing.
Approach:	They propose a method to build sememe knowledge bases from an existing dictionary . they propose to use existing dictionaries to build an English and a French SKB .
Outcome:	The proposed method is superior to HowNet, the most widely used SKB that takes decades to build manually.

Copied to clipboard

Xiaozhi Wang, Xu Han, Yankai Lin, Zhiyuan Liu, Maosong Sun

DeepNote: Note-Centric Deep Retrieval-Augmented Generation (2025.findings-emnlp)

Challenge:	Existing models cannot capture consistency and diversity of relation patterns in different languages.
Approach:	They propose an adversarial multi-lingual neural relation extraction model which considers consistency and diversity among languages.
Outcome:	The proposed model outperforms the state-of-the-art models on real-world datasets.

Copied to clipboard

Ruobing Wang, Qingfei Zhao, Yukun Yan, Daren Zha, Yuxuan Chen, Shi Yu, Zhenghao Liu, Yixuan Wang, Shuo Wang, Xu Han, Zhiyuan Liu, Maosong Sun

Information Retrieval

Challenge:	. - (EN)
Approach:	. - (EN)
Outcome:	. - (EN)

MatPlotAgent: Method and Evaluation for LLM-Based Agentic Scientific Data Visualization (2024.findings-acl)

Copied to clipboard

Zhiyu Yang, Zihan Zhou, Shuo Wang, Xin Cong, Xu Han, Yukun Yan, Zhenghao Liu, Zhixing Tan, Pengyuan Liu, Dong Yu, Zhiyuan Liu, Xiaodong Shi, Maosong Sun

Value Compass Benchmarks: A Comprehensive, Generative and Self-Evolving Platform for LLMs’ Value Evaluation (2025.acl-demo)

Challenge:	Scientific data visualization is an essential process in research, but its use of large language models remains unexplored.
Approach:	They propose a model-agnostic LLM agent framework to automate scientific data visualization tasks.
Outcome:	The proposed framework improves performance of commercial and open-source models.

Copied to clipboard

Jing Yao, Xiaoyuan Yi, Shitong Duan, Jindong Wang, Yuzhuo Bai, Muhua Huang, Yang Ou, Scarlett Li, Peng Zhang, Tun Lu, Zhicheng Dou, Maosong Sun, James Evans, Xing Xie

Challenge:	Current evaluation methods for large language models face two key challenges: 1. evaluation validity and 2. Result interpretation reduce the pluralistic and incommensurable values to one-dimensional scores.
Approach:	They propose a platform for comprehensive value diagnosis of large language models (LLMs) that provides a generative evaluation paradigm that automatically creates real-world test items co-evolving with ever-advancing LLMs.
Outcome:	The proposed platform provides a framework for comprehensive value diagnosis of large language models (LLMs) with fine-grained scores and case studies across 27 value dimensions for 33 leading LLMs, customized comparisons, and visualized analysis of LLM’s alignment with cultural values.

End-to-End Unsupervised Vision-and-Language Pre-training with Referring Expression Matching (2022.emnlp-main)

Copied to clipboard

Chi Chen, Peng Li, Maosong Sun, Yang Liu

Incorporating Chinese Characters of Words for Lexical Sememe Prediction (P18-1)

Challenge:	Existing unsupervised vision-and-language pre-training methods take pre-extracted region-based visual features from external object detectors, which limits flexibility and reduces computational efficiency.
Approach:	They propose an unsupervised vision-and-language pre-training task that predicts which patches contain an object referred to in natural language from the encoded visual features.
Outcome:	The proposed approach outperforms existing methods and obtains state-of-the-art results on four vision-and-language tasks.

Copied to clipboard

Huiming Jin, Hao Zhu, Zhiyuan Liu, Ruobing Xie, Maosong Sun, Fen Lin, Leyu Lin

Exploring the Impact of Model Scaling on Parameter-Efficient Tuning (2023.emnlp-main)

Challenge:	Existing methods of lexical sememe prediction rely on external context information of words to represent meaning.
Approach:	They propose a character-enhanced sememe prediction framework for Chinese language that takes advantage of internal character information and external context information.
Outcome:	The proposed framework outperforms state-of-the-art methods on a Chinese sememe knowledge base and maintains robust performance even for low-frequency words.

Copied to clipboard

Yusheng Su, Chi-Min Chan, Jiali Cheng, Yujia Qin, Yankai Lin, Shengding Hu, Zonghan Yang, Ning Ding, Xingzhi Sun, Guotong Xie, Zhiyuan Liu, Maosong Sun

Mask-Align: Self-Supervised Neural Word Alignment (2021.acl-long)

Challenge:	Parameter-efficient tuning (PET) methods can drive large pre-trained language models by training only minimal parameters.
Approach:	They propose a parameter-efficient tuning method that is compatible with a tunable module and uses a random number generator to optimize fewer table parameters.
Outcome:	The proposed method is compatible with a tunable module and tested on 11 NLP tasks.

Copied to clipboard

Chi Chen, Maosong Sun, Yang Liu

Machine Translation

Challenge:	Word alignment is an important task in many natural language processing tasks.
Approach:	They propose a self-supervised word alignment model that takes advantage of the full context on the target side.
Outcome:	The proposed model outperforms previous unsupervised models and obtains state-of-the-art results on four language pairs.

BMInf: An Efficient Toolkit for Big Model Inference and Tuning (2022.acl-demo)

Copied to clipboard

Xu Han, Guoyang Zeng, Weilin Zhao, Zhiyuan Liu, Zhengyan Zhang, Jie Zhou, Jun Zhang, Jia Chao, Maosong Sun

MetaMem: Evolving Meta-Memory for Knowledge Utilization through Self-Reflective Symbolic Optimization (2026.findings-acl)

Challenge:	Recent years, pre-trained language models (PLMs) have achieved promising results on various NLP tasks.
Approach:	They propose an open-source toolkit for big model inference and tuning which can support big model tuning at extremely low computation cost.
Outcome:	The proposed toolkit can support big model inference and tuning at extremely low computation cost.

Copied to clipboard

Haidong Xin, Xinze Li, Zhenghao Liu, Yukun Yan, Shuo Wang, Cheng Yang, Yu Gu, Ge Yu, Maosong Sun

Learning to Copy for Automatic Post-Editing (D19-1)

Challenge:	Existing memory systems can support long-horizon human-LLM interactions by persisting historical interactions beyond limited context windows.
Approach:	They propose a framework that augments memory systems with a self-evolving meta-memory . meta-meso is iteratively distilling transferable knowledge utilization experiences . results show MetaMem outperforms strong baselines by over 3.6% .
Outcome:	The proposed framework outperforms baselines by over 3.6% in the long-horizon human-LLM interaction.

Copied to clipboard

Xuancheng Huang, Yang Liu, Huanbo Luan, Jingfang Xu, Maosong Sun

Machine Translation

Challenge:	Automatic post-editing (APE) is an important task in natural language processing.
Approach:	They propose a method that explicitly models how to copy words from a machine translation to a correct translation.
Outcome:	The proposed method outperforms all published methods on the WMT 2016-2017 datasets.

An Extensible Plug-and-Play Method for Multi-Aspect Controllable Text Generation (2023.acl-long)

Copied to clipboard

Xuancheng Huang, Zijun Liu, Peng Li, Tao Li, Maosong Sun, Yang Liu

FewRel 2.0: Towards More Challenging Few-Shot Relation Classification (D19-1)

Challenge:	Multi-aspect controllable text generation has attracted increasing attention . but the mutual interference of multiple prefixes limits its extensibility to training-time unseen combinations.
Approach:	They propose to use trainable gates to normalize the intervention of prefixes to restrain the interference.
Outcome:	The proposed approach outperforms baselines on constraint accuracy, text quality, and extensibility.

Copied to clipboard

Tianyu Gao, Xu Han, Hao Zhu, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

Cross-lingual Lexical Sememe Prediction (D18-1)

Challenge:	Few-shot domain adaptation and NOTA detection are two real-world challenges for few-shot relation classification models.
Approach:	They propose a task to investigate two aspects of few-shot relation classification models . they build upon the FewRel dataset by adding a new test set in a different domain .
Outcome:	The proposed task can evaluate few-shot domain adaptation and few- shot none-of-the-above detection on a new domain and NOTA relation choice.

Copied to clipboard

Fanchao Qi, Yankai Lin, Maosong Sun, Hao Zhu, Ruobing Xie, Zhiyuan Liu

Cross-Lingual Contrastive Learning for Fine-Grained Entity Typing for Low-Resource Languages (2022.acl-long)

Challenge:	Sememes are defined as the minimum semantic units of human languages . but most languages do not have sememe-based linguistic knowledge bases . a new framework is proposed to predict sememes for words in other languages based on semems .
Approach:	They propose a framework to model correlations between sememes and multi-lingual words in low-dimensional semantic space for sememe prediction.
Outcome:	The proposed model improves on baseline methods on real-world datasets.

Copied to clipboard

Xu Han, Yuqi Luo, Weize Chen, Zhiyuan Liu, Maosong Sun, Zhou Botong, Hao Fei, Suncong Zheng

Open Hierarchical Relation Extraction (2021.naacl-main)

Challenge:	Experimental results show that by applying our framework, we can easily learn effective FGET models for low-resource languages.
Approach:	They propose a cross-lingual contrastive learning framework to learn FGET models for low-resource languages.
Outcome:	The proposed framework can learn effective FGET models for low-resource languages even without human-labeled data.

Copied to clipboard

Kai Zhang, Yuan Yao, Ruobing Xie, Xu Han, Zhiyuan Liu, Fen Lin, Leyu Lin, Maosong Sun

ClueAnchor: Clue-Anchored Knowledge Reasoning Exploration and Optimization for Retrieval-Augmented Generation (2025.findings-emnlp)

Challenge:	Existing OpenRE methods cast different relation types in isolation without considering their hierarchical dependency.
Approach:	They propose a framework to establish bidirectional connections between OpenRE and relation hierarchies by integrating hierarchy information into relation representations.
Outcome:	The proposed framework outperforms state-of-the-art models on relation clustering and hierarchy expansion.

Copied to clipboard

Hao Chen, Yukun Yan, Sen Mei, Wanxiang Che, Zhenghao Liu, Qi Shi, Xinze Li, Yuchun Fan, Pengcheng Huang, Qiushi Xiong, Zhiyuan Liu, Maosong Sun

FR-Spec: Accelerating Large-Vocabulary Language Models via Frequency-Ranked Speculative Sampling (2025.acl-long)

Challenge:	Existing RAG systems often underutilize the retrieved documents, authors say . they fail to extract and integrate key clues needed to support faithful and interpretable reasoning .
Approach:	a new framework extracts key clues from retrieved content and generates multiple reasoning paths . the framework optimizes the model by selecting the most appropriate reasoning path .
Outcome:	Experiments show that ClueAnchor outperforms baseline RAG frameworks in completeness and robustness.

Copied to clipboard

Weilin Zhao, Tengyu Pan, Xu Han, Yudi Zhang, Sun Ao, Yuxiang Huang, Kaihuo Zhang, Weilun Zhao, Yuxuan Li, Jie Zhou, Hao Zhou, Jianyong Wang, Maosong Sun, Zhiyuan Liu

Train No Evil: Selective Masking for Task-Guided Pre-Training (2020.emnlp-main)

Challenge:	Speculative sampling is an efficient way to accelerate the auto-regressive generation process of large language models.
Approach:	They propose a frequency-ranked speculative sampling framework that optimizes draft candidate selection through vocabulary space compression.
Outcome:	Experiments show that FR-Spec reduces LM Head computation overhead by 75% while ensuring the equivalence of the final output distribution.

Copied to clipboard

Yuxian Gu, Zhengyan Zhang, Xiaozhi Wang, Zhiyuan Liu, Maosong Sun

AutoReproduce: Automatic AI Experiment Reproduction with Paper Lineage (2026.acl-long)

Challenge:	Pre-trained language models can't capture domain-specific and task-specific patterns because of the task-agnostic pre-training stage.
Approach:	They propose a task-guided pre-training stage with selective masking between general pre-train and fine-tuning to learn domain-specific patterns.
Outcome:	The proposed method can achieve comparable or even better performance with less than 50% of computation cost.

Copied to clipboard

Xuanle Zhao, Zilin Sang, Yuxuan Li, Qi Shi, Weilun Zhao, Shuo Wang, Duzhen Zhang, Xu Han, Zhiyuan Liu, Maosong Sun

Challenge:	Efficient reproduction of research papers requires deep domain expertise.
Approach:	They propose a framework that systematically mines implicit knowledge from the cited literature to reproduce experimental code in a complete, end-to-end manner.
Outcome:	The proposed framework surpasses baselines across all metrics and reproduces experimental code in a complete, end-to-end manner.

MoEfication: Transformer Feed-forward Layers are Mixtures of Experts (2022.findings-acl)

Copied to clipboard

Zhengyan Zhang, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

Transfer Learning for Sequence Generation: from Single-source to Multi-source (2021.acl-long)

Challenge:	Recent work has shown that feed-forward networks (FFNs) in pre-trained Transformers are a key component, storing various linguistic and factual knowledge.
Approach:	They propose to convert a model into its MoE version with the same parameters and build expert routers to decide which experts will be used for each input.
Outcome:	The proposed model can use 10% to 30% of FFN parameters while maintaining over 95% original performance.

Copied to clipboard

Xuancheng Huang, Jingfang Xu, Maosong Sun, Yang Liu

OpenAttack: An Open-source Textual Adversarial Attack Toolkit (2021.acl-demo)

Challenge:	Recent studies have shown that pretrained models are effective for low-resource downstream tasks.
Approach:	They propose a two-stage finetuning method to transfer pretrained models to MSG tasks by concatenating multiple sources into a single long sequence.
Outcome:	The proposed model outperforms baselines on the WMT17 APE task and multi-source translation task using the WTM14 test set.

Copied to clipboard

Guoyang Zeng, Fanchao Qi, Qianrui Zhou, Tingji Zhang, Zixian Ma, Bairu Hou, Yuan Zang, Zhiyuan Liu, Maosong Sun

A Top-down Graph-based Tool for Modeling Classical Semantic Maps: A Case Study of Supplementary Adverbs (2025.naacl-long)

Challenge:	Various attack models are distinct and implemented with different programming frameworks and settings, which hinders quick utilization and fair comparison of attack models.
Approach:	They propose an open-source textual adversarial attack toolkit to solve these issues by combining 15 typical attack models into one toolkit.
Outcome:	The proposed toolkit supports all attack types, multilinguality, and parallel processing.

Copied to clipboard

Zhu Liu, Cunliang Kong, Ying Liu, Maosong Sun

Structured Data in NLP

Challenge:	Semantic map models (SMMs) construct a network-like conceptual space from cross-linguistic instances or forms based on the connectivity hypothesis.
Approach:	They propose a graph-based algorithm that automatically generates conceptual spaces and SMMs in a top-down manner.
Outcome:	The proposed model is compared with human annotations and other automated methods on cross-linguistic supplementary adverbs.

Robust and Scalable Model Editing for Large Language Models (2024.lrec-main)

Copied to clipboard

Yingfa Chen, Zhengyan Zhang, Xu Han, Chaojun Xiao, Zhiyuan Liu, Chen Chen, Kuai Li, Tao Yang, Maosong Sun

Open Relation Extraction: Relational Knowledge Transfer from Supervised Data to Unsupervised Data (D19-1)

Challenge:	Existing methods that ignore contextual knowledge fail to reliably fall back to parametric knowledge when presented with irrelevant context.
Approach:	They propose to use contextual knowledge to update and correct LLMs' knowledge by in-context editing instead of retraining.
Outcome:	The proposed method outperforms current state-of-the-art methods by a large margin on a dataset that contains irrelevant questions.

Copied to clipboard

Ruidong Wu, Yuan Yao, Xu Han, Ruobing Xie, Zhiyuan Liu, Fen Lin, Leyu Lin, Maosong Sun

GLTW: Joint Improved Graph Transformer and LLM via Three-Word Language for Knowledge Graph Completion (2025.findings-acl)

Challenge:	Existing methods to extract relational facts from open domain corpora are time-consuming and human-intensive.
Approach:	They propose a framework to learn similarity metrics of relations from labeled data . they propose to transfer relational knowledge to identify novel relations in unlabeled data.
Outcome:	Experiments on two real-world datasets show that the proposed framework improves compared with state-of-the-art methods.

Copied to clipboard

Kangyang Luo, Yuzhuo Bai, Cheng Gao, Shuzheng Si, Zhu Liu, Yingli Shen, Zhitong Wang, Cunliang Kong, Wenhao Li, Yufei Huang, Ye Tian, Xuantang Xiong, Lei Han, Maosong Sun

Parameter-efficient Weight Ensembling Facilitates Task-level Knowledge Transfer (2023.acl-short)

Challenge:	Existing knowledge graphs lack the ability to integrate structural information into LLMs and output predictions deterministically.
Approach:	They propose a method which encodes structural information of KGs and merges it with LLMs to enhance KGC performance.
Outcome:	The proposed method improves the performance of KG Completion datasets on KGs by integrating structural information with LLMs.

Copied to clipboard

Xingtai Lv, Ning Ding, Yujia Qin, Zhiyuan Liu, Maosong Sun

AgentRM: Enhancing Agent Generalization with Reward Modeling (2025.acl-long)

Challenge:	Recent studies show that large pre-trained language models can be adapted to particular tasks in a parameter-efficient manner.
Approach:	They propose to use lightweight parameters to transfer them between tasks to obtain similarity between tasks.
Outcome:	The proposed methods show an improvement of 5%8% over baselines and could largely facilitate task-level knowledge transfer.

Copied to clipboard

Yu Xia, Jingru Fan, Weize Chen, Siyu Yan, Xin Cong, Zhong Zhang, Yaxi Lu, Yankai Lin, Zhiyuan Liu, Maosong Sun

The Right Time Matters: Data Arrangement Affects Zero-Shot Generalization in Instruction Tuning (2025.findings-acl)

Challenge:	Existing LLM-based agents have strong performance on held-in tasks, but their generalizability to unseen tasks remains poor.
Approach:	They propose a reward-based generalizable reward model to guide the policy model for effective test-time search.
Outcome:	The proposed agentRM outperforms existing agents on held-in tasks by 8.8 points on average.

Copied to clipboard

Bingxiang He, Ning Ding, Cheng Qian, Jia Deng, Ganqu Cui, Lifan Yuan, Haiwen Hong, Huan-ang Gao, Longtao Huang, Hui Xue, Huimin Chen, Zhiyuan Liu, Maosong Sun

On Transferability of Prompt Tuning for Natural Language Processing (2022.naacl-main)

Challenge:	Existing work on instruction tuning has focused on task level, without considering that tasks are artificially defined and, to LLMs, merely consist of tokens and representations.
Approach:	They propose a training data arrangement framework that allows for continual learning and loss reduction.
Outcome:	The proposed framework promotes continual learning and loss reduction on unseen tasks.

Copied to clipboard

Yusheng Su, Xiaozhi Wang, Yujia Qin, Chi-Min Chan, Yankai Lin, Huadong Wang, Kaiyue Wen, Zhiyuan Liu, Peng Li, Juanzi Li, Lei Hou, Maosong Sun, Jie Zhou

CODIS: Benchmarking Context-dependent Visual Comprehension for Multimodal Large Language Models (2024.acl-long)

Challenge:	Pre-trained language models (PLMs) can achieve comparable performance to full-parameter fine-tuning by tuning a few soft prompts, but require much more training time than fine-timing.
Approach:	They empirically investigate the transferability of soft prompts across different downstream tasks and PLMs to determine what decides prompt transferability.
Outcome:	The proposed method can achieve comparable performance to full-parameter fine-tuning by tuning a few soft prompts, but requires much more training time than fine-timing.

Copied to clipboard

Fuwen Luo, Chi Chen, Zihao Wan, Zhaolu Kang, Qidong Yan, Yingjie Li, Xiaolong Wang, Siyu Wang, Ziyue Wang, Xiaoyue Mi, Peng Li, Ning Ma, Maosong Sun, Yang Liu

QuoteR: A Benchmark of Quote Recommendation for Writing (2022.acl-long)

Challenge:	Multimodal large language models have demonstrated promising results in a variety of tasks that combine vision and language.
Approach:	They propose a benchmark to assess the ability of models to use contextual information in free-form text to enhance visual comprehension.
Outcome:	The proposed model fails to extract and utilize contextual information to improve understanding of images.

Copied to clipboard

Fanchao Qi, Yanhui Yang, Jing Yi, Zhili Cheng, Zhiyuan Liu, Maosong Sun

Challenge:	Existing methods to recommend quotes are evaluated on unpublished datasets .
Approach:	They propose to build a dataset that is open and contains three parts including English, standard Chinese and classical Chinese.
Outcome:	The proposed model outperforms existing methods on all three parts of QuoteR.

Plug-and-Play Knowledge Injection for Pre-trained Language Models (2023.acl-long)

Copied to clipboard

Zhengyan Zhang, Zhiyuan Zeng, Yankai Lin, Huadong Wang, Deming Ye, Chaojun Xiao, Xu Han, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

Sememe Prediction for BabelNet Synsets using Multilingual and Multimodal Information (2022.findings-acl)

Challenge:	Existing knowledge injection methods are not suitable for enhancing pre-trained language models with external knowledge bases.
Approach:	They propose a plug-and-play knowledge injection method where knowledge bases are injected into frozen existing downstream models by a knowledge plugin.
Outcome:	The proposed method improves the performance of knowledge injection on knowledge-driven tasks while keeping model parameters frozen.

Copied to clipboard

Fanchao Qi, Chuancheng Lv, Zhiyuan Liu, Xiaojun Meng, Maosong Sun, Hai-Tao Zheng

FPT: Improving Prompt Tuning Efficiency via Progressive Training (2022.findings-emnlp)

Challenge:	Existing sememe KBs only cover a few languages, which hinders the wide utilization of sememes.
Approach:	They propose to build a multilingual sememe KB based on a dictionary called BabelNet . they use multilingual synonyms, multilingual glosses and images to encode sememes .
Outcome:	The proposed model outperforms previous methods in terms of MAP and F1 scores.

Copied to clipboard

Yufei Huang, Yujia Qin, Huadong Wang, Yichun Yin, Maosong Sun, Zhiyuan Liu, Qun Liu

OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems (2024.acl-long)

Challenge:	Recent prompt tuning (PT) has gained increasing attention as a parameter-efficient way of tuning pre-trained language models (PLMs).
Approach:	They propose a prompt tuning algorithm that uses a small-scale partial PLM and progressively expands its depth and width until the full-model size.
Outcome:	The proposed method could save over 30% of training computations while achieving comparable performance.

Copied to clipboard

Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, Maosong Sun

UNIKIE-BENCH: Benchmarking Large Multimodal Models for Key Information Extraction in Visual Documents (2026.acl-long)

Challenge:	Large Language Models (LLMs) and Large Multimodal Models have exceeded general human capabilities in various tasks.
Approach:	They present an Olympiad-level bilingual multimodal scientific benchmark featuring 8,476 problems from Olympiad level mathematics and physics competitions.
Outcome:	The best performing model, GPT-4V, attains an average score of 17.97% on OlympiadBench, with a mere 10.74% in physics, highlighting the benchmark rigor and the intricacy of physical reasoning.

Copied to clipboard

Yifan Ji, Zhipeng Xu, Zhenghao Liu, Zulong Chen, Qian Zhang, Zhibo Yang, Junyang Lin, Yu Gu, Ge Yu, Maosong Sun

ImCoref-CeS: An Improved Lightweight Pipeline for Coreference Resolution with LLM-based Checker-Splitter Refinement (2026.acl-long)

Challenge:	Recent Large Multimodal Models (LMMs) have shown promising potential for performing end-to-end KIE directly from document images.
Approach:	They propose a benchmark to evaluate the performance of Large Multimodal Models (LMMs) using a constrained-category KIE track and an open-categorical KIE Track.
Outcome:	Experiments on 15 state-of-the-art LMMs show performance degradation under diverse schema definitions, long-tail key fields, and complex layouts, along with pronounced performance disparities across different document types and scenarios.

Copied to clipboard

Kangyang Luo, Yuzhuo Bai, Shuzheng Si, Cheng Gao, Zhitong Wang, Yingli Shen, Wenhao Li, Zhu Liu, Yufeng Han, Jiayi Wu, Cunliang Kong, Maosong Sun

Entity-Duet Neural Ranking: Understanding the Role of Knowledge Graph Semantics in Neural Information Retrieval (P18-1)

Challenge:	Existing supervised neural methods for coreference resolution are underexplored . current methods rely on small language models, but their potential is underexploited .
Approach:	They propose a framework that integrates an enhanced supervised model with LLM-based reasoning.
Outcome:	The proposed method surpasses existing state-of-the-art methods in coreference resolution.

Copied to clipboard

Zhenghao Liu, Chenyan Xiong, Maosong Sun, Zhiyuan Liu

Legal Judgment Prediction via Topological Learning (D18-1)

Challenge:	Entity-oriented search and neural-IR push the boundary of search engines from two different aspects.
Approach:	They propose an Entity-Duet Neural Ranking Model which integrates knowledge graphs into neural search systems.
Outcome:	The proposed model improves generalization ability of neural ranking models on a commercial search log.

Copied to clipboard

Haoxi Zhong, Zhipeng Guo, Cunchao Tu, Chaojun Xiao, Zhiyuan Liu, Maosong Sun

Continual Knowledge Distillation for Neural Machine Translation (2023.acl-long)

Challenge:	Existing studies focus on a specific subtask of judgment prediction and ignore the dependencies among subtasks.
Approach:	They propose a topological multi-task learning framework that incorporates multiple subtasks and DAG dependencies into judgment prediction.
Outcome:	The proposed model improves on baselines on all judgment prediction tasks.

Copied to clipboard

Yuanchi Zhang, Peng Li, Maosong Sun, Yang Liu

From Scaffolding to Assimilation: Progressive Structural Internalization for Format-Constrained Creative Text Generation (2026.findings-acl)

Challenge:	Current parallel corpora are not publicly accessible but trained models are more readily available.
Approach:	They propose a method to take advantage of existing translation models to improve one model of interest.
Outcome:	The proposed method improves on Chinese-English and German-English datasets and is robust to malicious models.

Copied to clipboard

Wenhao Li, Yuwei Yang, Xiaoqing Wu, Yufeng Han, Cunliang Kong, Yuzhuo Bai, Xin Cong, Maosong Sun

A Simple but Effective Pluggable Entity Lookup Table for Pre-trained Language Models (2022.acl-short)

Challenge:	Existing paradigms rely on unreliable prompting or rigid constrained decoding strategies to achieve aesthetic unity.
Approach:	They propose a framework to embed external constraints into the model’s intrinsic intuition and use it to generate open-ended creative texts.
Outcome:	The proposed framework surpasses baselines in both strict constraint adherence and literary aesthetics.

Copied to clipboard

Deming Ye, Yankai Lin, Peng Li, Maosong Sun, Zhiyuan Liu

BMCook: A Task-agnostic Compression Toolkit for Big Models (2022.emnlp-demos)

Challenge:	Existing pre-trained language models cannot recall factual knowledge of entities exhibited in large-scale corpora, especially those rare entities.
Approach:	They propose to build a pluggable Entity Lookup Table (PELT) on demand by aggregating the entity’s output representations of multiple occurrences in the corpora.
Outcome:	The proposed model can transfer entity knowledge from out-of-domain corpora into PLMs with different architectures.

Copied to clipboard

Zhengyan Zhang, Baitao Gong, Yingfa Chen, Xu Han, Guoyang Zeng, Weilin Zhao, Yanxu Chen, Zhiyuan Liu, Maosong Sun

XQA: A Cross-lingual Open-domain Question Answering Dataset (P19-1)

Challenge:	Existing efforts to compress medium-sized models for specific tasks have limited results.
Approach:	They propose a task-agnostic compression toolkit for big models that implements quantization, pruning, distillation and MoEfication methods.
Outcome:	The proposed tool improves performance on a model with 3 billion parameters by 12x . it also outperforms the original model on three typical NLP benchmarks.

Copied to clipboard

Jiahua Liu, Yankai Lin, Zhiyuan Liu, Maosong Sun

Natural Language Interfaces

Challenge:	Open-domain question answering aims to answer questions through text retrieval and reading comprehension . but, the success of these models relies on a massive volume of training data, which is not available in other languages . a new dataset aims at investigating cross-lingual OpenQA .
Approach:	They propose to use a dataset for cross-lingual OpenQA research to test models . they use XQA dataset to train models with large volumes of labeled data .
Outcome:	The proposed model achieves best results in almost all target languages while the performance is lower than that of English.

Boosting Inference Efficiency: Unleashing the Power of Parameter-Shared Pre-trained Language Models (2023.findings-emnlp)

Copied to clipboard

Weize Chen, Xiaoyue Xu, Xu Han, Yankai Lin, Ruobing Xie, Zhiyuan Liu, Maosong Sun, Jie Zhou

Chunks as Arms: Multi-Armed Bandit-Guided Sampling for Long-Context LLM Preference Optimization (2026.acl-long)

Challenge:	Parameter-shared pre-trained language models (PLMs) have emerged as a successful approach in resource-constrained environments.
Approach:	They propose a method to enhance the inference efficiency of parameter-shared PLMs by pre-training models that can achieve even greater acceleration.
Outcome:	The proposed method improves inference efficiency on autoregressive and autoencoding models.

Copied to clipboard

Shaohua Duan, Pengcheng Huang, Xinze Li, Zhenghao Liu, Xiaoyuan Yi, Yukun Yan, Shuo Wang, Yu Gu, Ge Yu, Maosong Sun

Cost-Optimal Grouped-Query Attention for Long-Context Modeling (2025.emnlp-main)

Challenge:	Recent studies have explored fine-tuning Large Language Models with synthetic data to enhance their long-context capabilities.
Approach:	They propose a framework that leverages a Multi-Armed Bandit rollout strategy to identify the most informative chunks from the given long context for sampling high-quality and diverse responses.
Outcome:	The proposed framework achieves 4% improvement on long-context reasoning benchmarks on Llama and Qwen.

Copied to clipboard

Yingfa Chen, Yutong Wu, Chenyang Song, Zhen Leng Thai, Xingyu Shen, Xu Han, Zhiyuan Liu, Maosong Sun

Graph Neural Networks with Generated Parameters for Relation Extraction (P19-1)

Challenge:	Current GQA configurations overlook how context length influences inference cost .
Approach:	They propose a recipe for deriving cost-optimal GQA configurations that decouple the total head size from the hidden size and allow more flexible control over attention FLOPs.
Outcome:	The proposed configurations reduce memory usage and FLOPs by more than 50% compared to Llama-3's GQA, with no degradation in model capabilities.

Copied to clipboard

Hao Zhu, Yankai Lin, Zhiyuan Liu, Jie Fu, Tat-Seng Chua, Maosong Sun

Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger (2021.acl-long)

Challenge:	Existing graph neural networks can only process multi-hop relational reasoning on pre-defined graphs and cannot be directly applied in natural language relational reasoning.
Approach:	They propose a graph neural network with generated parameters using natural language sentences as inputs.
Outcome:	The proposed model can process relational reasoning on graphs and in natural language processing tasks.

Copied to clipboard

Fanchao Qi, Mukai Li, Yangyi Chen, Zhengyan Zhang, Zhiyuan Liu, Yasheng Wang, Maosong Sun

Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer (2021.emnlp-main)

Challenge:	Existing methods for textual backdoor attacks insert additional contents into normal samples as triggers, causing detection and blocking of backdoors.
Approach:	They propose to use syntactic structure as trigger in textual backdoor attacks . they propose to achieve similar attack performance but have higher invisibility .
Outcome:	The proposed method achieves almost 100% success rate but has higher invisibility and stronger resistance to defenses than the insertion-based methods.

Copied to clipboard

Fanchao Qi, Yangyi Chen, Xurui Zhang, Mukai Li, Zhiyuan Liu, Maosong Sun

GUICourse: From General Vision Language Model to Versatile GUI Agent (2025.acl-long)

Challenge:	Experimental results show that popular NLP models are vulnerable to both adversarial and backdoor attacks based on text style transfer.
Approach:	They propose to conduct adversarial and backdoor attacks based on text style transfer . the authors propose to use text style to alter the style of a sentence .
Outcome:	The proposed methods show that popular models are vulnerable to both attacks based on text style transfer . the results show that the proposed methods perform better than baselines in many aspects .

Copied to clipboard

Wentong Chen, Junbo Cui, Jinyi Hu, Yujia Qin, Junjie Fang, Yue Zhao, Chongyi Wang, Jun Liu, Guirong Chen, Yupeng Huo, Yuan Yao, Yankai Lin, Zhiyuan Liu, Maosong Sun

Visual Data in NLP

Challenge:	Graphical User Interfaces (GUIs) are a pivotal medium for human-computer interaction.
Approach:	They propose a series of datasets for training visual-based GUI agents using general VLMs.
Outcome:	The proposed GUICourse datasets show that even a small-sized GUI agent performs better on GUI tasks.

StableToolBench: Towards Stable Large-Scale Benchmarking on Tool Learning of Large Language Models (2024.findings-acl)

Copied to clipboard

Zhicheng Guo, Sijie Cheng, Hao Wang, Shihao Liang, Yujia Qin, Peng Li, Zhiyuan Liu, Maosong Sun, Yang Liu

DecorateLM: Data Engineering through Corpus Rating, Tagging, and Editing with Language Models (2024.emnlp-main)

Challenge:	Large Language Models (LLMs) have witnessed remarkable advancements in recent years, prompting the exploration of tool learning.
Approach:	They propose a virtual API server and stable evaluation system to assess the stability of large-scale real-time APIs.
Outcome:	The proposed benchmarks demonstrate the stability of the proposed system and its caching system.

Copied to clipboard

Ranchi Zhao, Zhen Thai, Yifan Zhang, Shengding Hu, Jie Zhou, Yunqi Ba, Jie Cai, Zhiyuan Liu, Maosong Sun

Recurrence Boosts Diversity! Revisiting Recurrent Latent Variable in Transformer-Based Variational AutoEncoder for Diverse Text Generation (2022.findings-emnlp)

Challenge:	Large Language Models (LLMs) are pre-trained on vast datasets composed of billions of tokens harvested from diverse text sources.
Approach:	They propose a data engineering method to refine the pretraining corpus through data rating, tagging and editing.
Outcome:	The proposed method improves the quality of the pretraining corpus by enhancing 100 billion tokens of the training corpus.

Copied to clipboard

Jinyi Hu, Xiaoyuan Yi, Wenhao Li, Maosong Sun, Xing Xie

WebCPM: Interactive Web Search for Chinese Long-form Question Answering (2023.acl-long)

Challenge:	Variational Auto-Encoder (VAE) has been widely adopted in text generation due to its ability to learn flexible representations.
Approach:	They propose a Transformer-based recurrent VAE structure that imposes recurrence on segment-wise latent variables with arbitrarily separated text segments and constructs the posterior distribution with residual parameterization.
Outcome:	The proposed structure can deduce a non-zero lower bound of the KL term and enhance the entanglement of each segment and preceding latent variables, providing a theoretical guarantee of generation diversity.

Copied to clipboard

Yujia Qin, Zihan Cai, Dian Jin, Lan Yan, Shihao Liang, Kunlun Zhu, Yankai Lin, Xu Han, Ning Ding, Huadong Wang, Ruobing Xie, Fanchao Qi, Zhiyuan Liu, Maosong Sun, Jie Zhou

Information Retrieval

Challenge:	Long-form question answering requires two procedures: information retrieval and information synthesis.
Approach:	They propose a Chinese long-form question answering dataset called WebCPM . the dataset is based on a web search interface that engages with a search engine in real time .
Outcome:	The proposed dataset generates answers that are no worse than human-written ones . the dataset is the first Chinese LFQA dataset .

Aligning Large Language Models to Follow Instructions and Hallucinate Less via Effective Data Filtering (2025.acl-long)

Copied to clipboard

Shuzheng Si, Haozhe Zhao, Gang Chen, Cheng Gao, Yuzhuo Bai, Zhitong Wang, Kaikai An, Kangyang Luo, Chen Qian, Fanchao Qi, Baobao Chang, Maosong Sun

LEVEN: A Large-Scale Chinese Legal Event Detection Dataset (2022.findings-acl)

Challenge:	Existing studies show that training LLMs on data containing unfamiliar knowledge during instruction tuning can encourage hallucinations.
Approach:	They propose a framework that measures how familiar the LLM is with instruction data and introduce an expert-aligned reward model to ensure the quality of selected samples.
Outcome:	The proposed framework reduces hallucinations while maintaining a competitive ability to follow instructions.

Copied to clipboard

Feng Yao, Chaojun Xiao, Xiaozhi Wang, Zhiyuan Liu, Lei Hou, Cunchao Tu, Juanzi Li, Yun Liu, Weixing Shen, Maosong Sun

Manual Evaluation Matters: Reviewing Test Protocols of Distantly Supervised Relation Extraction (2021.findings-acl)

Challenge:	Existing legal event detection datasets only cover incomprehensive event types and have limited annotated data.
Approach:	They present a large-scale Chinese legal event detection dataset . they use legal events as side information to promote downstream applications .
Outcome:	The proposed method improves 2.2 points precision in low-resource judgment prediction and 1.5 points precision for unsupervised case retrieval.

Copied to clipboard

Tianyu Gao, Xu Han, Yuzhuo Bai, Keyue Qiu, Zhiyu Xie, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

TopWORDS-Seg: Simultaneous Text Segmentation and Word Discovery for Open-Domain Chinese Texts via Bayesian Inference (2022.acl-long)

Challenge:	Distantly supervised relation extraction (RE) has attracted much attention in the past few years . previous methods to evaluate models manually or directly on autolabeled data have produced inaccurate evaluations .
Approach:	They propose to use distant supervision to generate large-scale autolabeled data . they build manually-annotated test sets for two DS-RE datasets and evaluate models .
Outcome:	The proposed method produces 53% wrong labels at the entity pair level in the popular NYT10 dataset.

Copied to clipboard

Changzai Pan, Maosong Sun, Ke Deng

Experiential Co-Learning of Software-Developing Agents (2024.acl-long)

Challenge:	No existing methods can achieve effective text segmentation and word discovery in open domain Chinese texts.
Approach:	They propose a Bayesian-based method that can achieve effective text segmentation and word discovery in open domain.
Outcome:	The proposed method enjoys robust performance and transparent interpretation when no training corpus and domain vocabulary are available.

Copied to clipboard

Chen Qian, Yufan Dang, Jiahao Li, Wei Liu, Zihao Xie, YiFei Wang, Weize Chen, Cheng Yang, Xin Cong, Xiaoyin Che, Zhiyuan Liu, Maosong Sun

Beat LLMs at Their Own Game: Zero-Shot LLM-Generated Text Detection via Querying ChatGPT (2023.emnlp-main)

Challenge:	Recent advances in large language models (LLMs) have brought significant changes to various domains, especially through autonomous agents.
Approach:	They propose a framework that lets agents learn shortcuts from their past tasks and use them for future task execution.
Outcome:	The proposed framework enables agents to tackle unseen software-developing tasks more effectively.

Copied to clipboard

Biru Zhu, Lifan Yuan, Ganqu Cui, Yangyi Chen, Chong Fu, Bingxiang He, Yangdong Deng, Zhiyuan Liu, Maosong Sun, Ming Gu

Self-Supervised Quality Estimation for Machine Translation (2021.emnlp-main)

Challenge:	Large language models (LLMs) are capable of performing tasks but are likely to be misused.
Approach:	They propose a zero-shot black-box method to detect LLM-generated texts . they revise the text to be detected using the ChatGPT model .
Outcome:	The proposed method can detect LLM-generated texts with a zero-shot black-box model . it is based on intuition that the model will make fewer revisions to LLMs than to human-written texts .

Copied to clipboard

Yuanhang Zheng, Zhixing Tan, Meng Zhang, Mieradilijiang Maimaiti, Huanbo Luan, Maosong Sun, Qun Liu, Yang Liu

Beyond the Turn-Based Game: Enabling Real-Time Conversations with Duplex Models (2024.emnlp-main)

Challenge:	Training QE models require massive parallel data with hand-crafted quality annotations, which are time-consuming and labor-intensive to obtain.
Approach:	They propose a self-supervised method to evaluate machine-translated sentences without references by recovering masked target words.
Outcome:	The proposed method outperforms previous unsupervised methods on several QE tasks in different language pairs and domains.

Copied to clipboard

Xinrong Zhang, Yingfa Chen, Shengding Hu, Xu Han, Zihang Xu, Yuanwei Xu, Weilin Zhao, Maosong Sun, Zhiyuan Liu

Dialogue Systems & Conversational Agents

Challenge:	Large language models (LLMs) are increasingly permeating daily lives and require real-time interactions that mirror human conversations.
Approach:	They propose to use time-division-multiplexing to process queries and responses pseudo-simultaneously.
Outcome:	The proposed model can listen to users while generating output and adjust to provide instant feedback.

Stylistic Chinese Poetry Generation via Unsupervised Style Disentanglement (D18-1)

Copied to clipboard

Cheng Yang, Maosong Sun, Xiaoyuan Yi, Wenhao Li

Low-Resource NLP

Challenge:	Automatic Chinese poetry generation is one of the first attempts towards computer writing.
Approach:	They propose a model which requires no supervised style labeling to generate stylistic poems . they incorporate mutual information, a concept in information theory, into modeling .
Outcome:	The proposed model generates stylistic poems without losing fluency and coherency . it is based on mutual information, a concept in information theory .

Knowledge Representation Learning with Contrastive Completion Coding (2021.findings-emnlp)

Copied to clipboard

Bo Ouyang, Wenbing Huang, Runfa Chen, Zhixing Tan, Yang Liu, Maosong Sun, Jihong Zhu

UltraEval: A Lightweight Platform for Flexible and Comprehensive Evaluation for LLMs (2024.acl-demos)

Challenge:	Existing knowledge representation learning methods suffer from immaturity on tackling potentially-imperfect knowledge graphs and highly-imbalanced positive-negative instances during training.
Approach:	They propose a framework for knowledge representation learning that incorporates two functional components to achieve robust embedding for each entity/relation.
Outcome:	The proposed framework achieves better convergence against state-of-the-art methods on several benchmarks.

Copied to clipboard

Chaoqun He, Renjie Luo, Shengding Hu, Ranchi Zhao, Jie Zhou, Hanghao Wu, Jiajie Zhang, Xu Han, Zhiyuan Liu, Maosong Sun

Weize Chen, Hao Zhu, Xu Han, Zhiyuan Liu, Maosong Sun

Challenge:	Existing evaluation platforms are complex and poorly modularized, hindering seamless incorporation into researcher’s workflows.
Approach:	They propose a lightweight evaluation framework characterized by lightweight, comprehensiveness, modularity, and efficiency that integrates models, data, and metrics into a unified evaluation workflow.
Outcome:	The proposed evaluation framework is lightweight, comprehensive, modular, and efficient.

Copied to clipboard

CheckRLM: Effective Knowledge–Thought Coherence Checking in Retrieval-Augmented Reasoning (2026.acl-long)

Challenge:	a conceptually simple and effective method to quantify the similarity between relations is presented . identifying relations is a crucial problem for several information extraction tasks.
Approach:	They propose a method to quantify the similarity between relations in knowledge bases . they use a neural network to parameterize conditional probability distributions over entity pairs .
Outcome:	The proposed method significantly correlates with human judgments, the authors show . it could be incorporated into negative sampling and softmax classification to alleviate these mistakes.

Copied to clipboard

Dingling Xu, Ruobing Wang, Qingfei Zhao, Yukun Yan, Zhichun Wang, Daren Zha, Shi Yu, Zhenghao Liu, Shuo Wang, Xu Han, Maosong Sun

Neural Quality Estimation with Multiple Hypotheses for Grammatical Error Correction (2021.naacl-main)

Challenge:	Reasoning Language Models (RLMs) have improved performance on complex tasks by extending the reasoning chain, but they are prone to factual errors, especially in knowledge-intensive tasks.
Approach:	They propose a framework that improves the reliability of the reasoning process by timely checking and correcting factual errors.
Outcome:	The proposed framework outperforms baselines and shows that it mitigates error accumulation with lower costs.

Copied to clipboard

Zhenghao Liu, Xiaoyuan Yi, Maosong Sun, Liner Yang, Tat-Seng Chua

Fully Hyperbolic Neural Networks (2022.acl-long)

Challenge:	Existing GEC models produce spurious corrections or fail to detect lots of errors.
Approach:	They propose a neural network for GEC quality estimation with multiple hypotheses . VERNet establishes interactions among hypothese based on reasoning graph .
Outcome:	The proposed model achieves state-of-the-art grammatical error detection performance and best quality estimation results on four GEC datasets.

Copied to clipboard

Weize Chen, Xu Han, Yankai Lin, Hexu Zhao, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

Challenge:	Existing hyperbolic neural networks encode features in the hyperbolical space yet formalize most of their operations in the tangent space.
Approach:	They propose a fully hyperbolic framework to build hyperbolical networks based on the Lorentz model by adapting Lorentzer transformations to formalize essential operations of neural networks.
Outcome:	The proposed framework has better performance on four NLP tasks compared with existing hyperbolic models .

Enhancing Legal Case Retrieval via Scaling High-quality Synthetic Query-Candidate Pairs (2024.emnlp-main)

Copied to clipboard

Cheng Gao, Chaojun Xiao, Zhenghao Liu, Huimin Chen, Zhiyuan Liu, Maosong Sun

Information Retrieval

Challenge:	Existing studies focus on case-to-case retrieval using lengthy queries, which does not match real-world scenarios.
Approach:	They propose a method to construct query-candidate pairs and build the largest LCR dataset to date, LEAD.
Outcome:	Experimental results show that the method can provide ample training signals for LCR models.

Variator: Accelerating Pre-trained Models with Plug-and-Play Compression Modules (2023.findings-emnlp)

Copied to clipboard

Chaojun Xiao, Yuqi Luo, Wenbin Zhang, Pengle Zhang, Xu Han, Yankai Lin, Zhengyan Zhang, Ruobing Xie, Zhiyuan Liu, Maosong Sun, Jie Zhou

Enhancing Multilingual Capabilities of Large Language Models through Self-Distillation from Resource-Rich Languages (2024.acl-long)

Challenge:	Large language models (LLMs) have been successful on NLP tasks but require huge parameter sizes and computational resources.
Approach:	They propose a parameter-efficient acceleration method that enhances computational efficiency through plug-and-play compression plugins.
Outcome:	The proposed method saves 53% computational costs using only 0.9% additional parameters with a performance drop of less than 2%.

Copied to clipboard

Yuanchi Zhang, Yile Wang, Zijun Liu, Shuo Wang, Xiaolong Wang, Peng Li, Maosong Sun, Yang Liu

Denoising Relation Extraction from Document-level Distant Supervision (2020.emnlp-main)

Challenge:	Contemporary large language models (LLMs) are pre-trained on multilingual corpora, but their performance lags behind in most languages compared to a few resource-rich languages.
Approach:	They propose a method that leverages the internal capabilities of large language models on resource-rich languages to enhance multilingual performance.
Outcome:	The proposed method improves multilingual performance while minimizing impact on original performance in resource-rich languages.

Copied to clipboard

Chaojun Xiao, Yuan Yao, Ruobing Xie, Xu Han, Zhiyuan Liu, Maosong Sun, Fen Lin, Leyu Lin

HMEAE: Hierarchical Modular Event Argument Extraction (D19-1)

Challenge:	Existing methods to generate auto-labeled sentences for relation extraction (RE) are difficult to extend to document-level relation extraction as noise from DS may be even multiplied in documents.
Approach:	They propose a pre-trained model which de-emphasizes noisy DS data via multiple pre-training tasks.
Outcome:	The proposed model can capture useful information from noisy data and achieve promising results on the large-scale DocRE benchmark.

Copied to clipboard

Xiaozhi Wang, Ziqi Wang, Xu Han, Zhiyuan Liu, Juanzi Li, Peng Li, Maosong Sun, Jie Zhou, Xiang Ren

Empirical Analysis of Decoding Biases in Masked Diffusion Models (2026.acl-long)

Challenge:	Existing event extraction methods classify each argument role independently, ignoring conceptual correlations between different argument roles.
Approach:	They propose a Hierarchical Modular Event Argument Extraction model to provide inductive bias from the concept hierarchy of event argument roles.
Outcome:	The proposed model outperforms existing methods on real-world datasets and shows that it leverages useful knowledge from the concept hierarchy.

Copied to clipboard

Pengcheng Huang, Tianming Liu, Zhenghao Liu, Yukun Yan, Shuo Wang, Tong Xiao, Zulong Chen, Maosong Sun

Won’t Get Fooled Again: Answering Questions with False Premises (2023.acl-long)

Challenge:	Existing MDMs employ uncertainty-based decoding strategies that limit their reasoning ability and ultimately degrade generation quality.
Approach:	They propose a framework that regularizes uncertainty-based decoding by incorporating two complementary priors to shape global decoding trajectories and promote content informativeness.
Outcome:	The proposed framework outperforms existing decoding strategies by more than 7% while achieving comparable performance to autoregressive models of similar parameter scales.

Copied to clipboard

Shengding Hu, Yifan Luo, Huadong Wang, Xingyi Cheng, Zhiyuan Liu, Maosong Sun

Question Answering

Challenge:	Pre-trained language models (PLMs) are often easily deceived by tricky questions such as “How many eyes does the sun have?” .
Approach:	They annotate a FalseQA dataset containing 2365 human-written FPQs and find that PLMs are capable of discriminating FPqs by fine-tuning on moderate numbers.
Outcome:	The proposed model can discriminate on FPQs by fine-tuning on moderate numbers of examples and generate reasonable explanations for false premise questions.

Learning to Generate Structured Output with Schema Reinforcement Learning (2025.acl-long)

Copied to clipboard

Yaxi Lu, Haolun Li, Xin Cong, Zhong Zhang, Yesai Wu, Yankai Lin, Zhiyuan Liu, Fangming Liu, Maosong Sun

Learn and Consolidate: Continual Adaptation for Zero-Shot and Multilingual Neural Machine Translation (2023.emnlp-main)

Challenge:	Recent advances in large language models have facilitated the development of intelligent applications like automatic web search (Qin et al., 2023) Several methods exist for generating JSON strings from LLMs, including Prompting but often miss certain schemas.
Approach:	They propose to use 40K different JSON schemas to assess models' ability to generate valid JSON outputs.
Outcome:	The proposed model improves both in generating JSON outputs and downstream tasks.

Copied to clipboard

Kaiyu Huang, Peng Li, Junpeng Liu, Maosong Sun, Yang Liu

Fine-Grained Legal Argument-Pair Extraction via Coarse-Grained Pre-training (2024.lrec-main)

Challenge:	Existing multilingual neural machine translation models perform poorly on language pairs with no parallel corpus.
Approach:	They propose a two-stage approach that encourages original models to acquire language-agnostic multilingual representations from new data and preserves the model architecture without introducing parameters.
Outcome:	The proposed approach improves performance in translation directions where existing models are weak and mitigates degeneration in the well-performing translation directions, offering flexibility in the real-world scenario.

Copied to clipboard

Chaojun Xiao, Yutao Sun, Yuan Yao, Xu Han, Wenbin Zhang, Zhiyuan Liu, Maosong Sun

Long-Chain Reasoning Distillation via Adaptive Prefix Alignment (2026.acl-long)

Challenge:	Current methods conceptualize LAE as a supervised sentence-pair classification problem and necessitate extensive manual annotations.
Approach:	They propose a model that focuses on fine-grained alignment of argument pairs building upon coarse-grain complaint-defense pairs.
Outcome:	The proposed model outperforms baseline models by 3.7 and 2.4 points on average.

Copied to clipboard

Zhenghao Liu, Zhuoyang Wu, Xinze Li, Yukun Yan, Shuo Wang, Zulong Chen, Yu Gu, Ge Yu, Maosong Sun

On the Language Coverage Bias for Neural Machine Translation (2021.findings-acl)

Challenge:	Large Language Models (LLMs) have demonstrated remarkable reasoning capabilities, especially in solving complex mathematical problems.
Approach:	They propose a framework that exploits teacher CoTs for distillation through adaptive prefix alignment.
Outcome:	The proposed framework outperforms baseline models on multiple mathematical reasoning benchmarks by over 3%.

Copied to clipboard

Shuo Wang, Zhaopeng Tu, Zhixing Tan, Shuming Shi, Maosong Sun, Yang Liu

EchoMLLM: Incentivizing Echocardiographic Video Understanding with Keyframe Grounding and Report Generation (2026.findings-acl)

Challenge:	Language coverage bias is important for neural machine translation because of the target-original training data.
Approach:	They propose two approaches to alleviate the language coverage bias problem by explicitly distinguishing between the source-and target-original training data.
Outcome:	The proposed methods improve translation tasks on both back-and forward-translation and their tagged variants.

Copied to clipboard

Heyu Huang, Wanran Sun, Chi Chen, Bo Chen, Zonghao Guo, Yuhua Li, Ruixuan Li, Kunlun He, Maosong Sun

Visual Data in NLP

Challenge:	Echocardiography analysis requires a dual capability: rigorous quantitative keyframe localization and comprehensive qualitative synthesis.
Approach:	They propose a unified framework designed for real-world echocardiography video understanding.
Outcome:	a new framework is designed to support real-world echocardiography video understanding . it reduces temporal grounding errors by up to 76% and improves report generation quality by 65% .

ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning (2021.acl-long)

Copied to clipboard

Yujia Qin, Yankai Lin, Ryuichi Takanobu, Zhiyuan Liu, Peng Li, Heng Ji, Minlie Huang, Maosong Sun, Jie Zhou

CodRED: A Cross-Document Relation Extraction Dataset for Acquiring Knowledge in the Wild (2021.emnlp-main)

Challenge:	Existing pre-training objectives do not explicitly model relational facts in text . Experimental results show that ERICA can improve typical PLMs on several language understanding tasks, including relation extraction, entity typing and question answering.
Approach:	They propose a contrastive learning framework ERICA to obtain a deep understanding of entities and relations in text.
Outcome:	The proposed framework can improve PLMs on several language understanding tasks, especially under low-resource settings.

Copied to clipboard

Yuan Yao, Jiaju Du, Yankai Lin, Peng Li, Zhiyuan Liu, Jie Zhou, Maosong Sun

WantWords: An Open-source Online Reverse Dictionary System (2020.emnlp-demos)

Challenge:	Existing relation extraction methods focus on extracting relational facts between entity pairs within single sentences or documents.
Approach:	They present a problem of cross-document relation extraction (CRE) using human annotations.
Outcome:	The proposed dataset is the first human-annotated cross-document RE dataset . it shows that it is challenging to existing RE methods including strong BERT-based models.

Copied to clipboard

Fanchao Qi, Lei Zhang, Yanhui Yang, Zhiyuan Liu, Maosong Sun

Generating Major Types of Chinese Classical Poetry in a Uniformed Framework (2020.lrec-1)

Challenge:	Existing reverse dictionary systems only support English reverse dictionary queries . a reverse dictionary can help people who can't remember a word from memory .
Approach:	They propose an online reverse dictionary system that outperforms other reverse dictionary systems . it supports Chinese and English-Chinese as well as Chinese-English cross-lingual reverse dictionary queries .
Outcome:	The proposed reverse dictionary outperforms other reverse dictionary systems on performance . it supports Chinese and English-Chinese as well as Chinese-English queries .

Copied to clipboard

Jinyi Hu, Maosong Sun

UltraLink: An Open-Source Knowledge-Enhanced Multilingual Supervised Fine-tuning Dataset (2024.acl-long)

Challenge:	Chinese classical poetry is one of the most valuable literary and cultural heritages of China . it has many particular characteristics in its language structure, ranging from form, sound to meaning . a proposed uniformed framework for generating major types of Chinese classical poems is proposed .
Approach:	They propose a uniformed framework for generating major types of Chinese classical poems . they use a form- stressed weighting method to strengthen the control to the form of the generated poems a proposed framework is incorporated into Jiuge, the most influential Chinese classical poetry generation system developed by Tsinghua University.
Outcome:	The proposed framework can generate Chinese classical poems of major types with high quality in form and content.

Copied to clipboard

Haoyu Wang, Shuo Wang, Yukun Yan, Xujia Wang, Zhiyu Yang, Yuzhuang Xu, Zhenghao Liu, Liner Yang, Ning Ding, Xu Han, Zhiyuan Liu, Maosong Sun

DebugBench: Evaluating Debugging Capability of Large Language Models (2024.findings-acl)

Challenge:	Open-source large language models (LLMs) have gained strength across diverse fields, but the majority of studies focus on English.
Approach:	They propose a knowledge-grounded data augmentation approach to elicit more language-specific knowledge of LLMs by enhancing their ability to serve users from different countries.
Outcome:	The proposed method can prune the language-agnostic supervised fine-tuning dataset without any performance degradation.

Copied to clipboard

Runchu Tian, Yining Ye, Yujia Qin, Xin Cong, Yankai Lin, Yinxu Pan, Yesai Wu, Hui Haotian, Liu Weichuan, Zhiyuan Liu, Maosong Sun

AutoVecCoder: Teaching LLMs to Generate Explicitly Vectorized Code (2026.findings-acl)

Challenge:	Large language models (LLMs) have demonstrated exceptional coding capabilities, but their debugging capabilities remain relatively unexplored.
Approach:	They propose a debugging benchmark consisting of 4,253 LLMs with four major bug categories and 18 minor types in C++, Java, and Python.
Outcome:	The proposed benchmark covers four major bug categories and 18 minor types in C++, Java, and Python.

Copied to clipboard

ShangZhan Li, Xinyu Yin, Xuanyu Jin, Ye He, Yuxin Zhou, Yuxuan Li, Xu Han, Wanxiang Che, Qi Shi, Ting Liu, Maosong Sun

Programming Languages in NLP

Challenge:	Current development practices face a dichotomy between automation and performance.
Approach:	They propose a framework to empower LLMs with the capability of automated explicit vectorization.
Outcome:	The proposed framework achieves state-of-the-art performance on the SSE and AVX subsets of SimdBench.

Self-Knowledge Guided Retrieval Augmentation for Large Language Models (2023.findings-emnlp)

Copied to clipboard

Yile Wang, Peng Li, Maosong Sun, Yang Liu

Hierarchical Relation Extraction with Coarse-to-Fine Grained Attention (D18-1)

Challenge:	Large language models (LLMs) have shown superior performance without task-specific fine-tuning due to the computational costs.
Approach:	They propose a method which lets LLMs refer to the questions they have previously encountered and adaptively call for external resources when dealing with new questions.
Outcome:	The proposed method outperforms chain-of-thought based and fully retrieval-based methods on multiple datasets and outperformed chain- of-though, chatGPT and InstructGPT.

Copied to clipboard

Xu Han, Pengfei Yu, Zhiyuan Liu, Maosong Sun, Peng Li

Improving the Transformer Translation Model with Document-Level Context (D18-1)

Challenge:	Existing methods for relation extraction use knowledge graphs to automatically label training data . but, it suffers from the wrong labeling problem because not all sentences containing two entities can express their relations in KGs .
Approach:	They propose a distant supervision approach to automatically label training instances . they integrate hierarchical information of relations into distantly supervised relation extraction .
Outcome:	The proposed model outperforms baseline models on a large-scale dataset.

Copied to clipboard

Jiacheng Zhang, Huanbo Luan, Maosong Sun, Feifei Zhai, Jingfang Xu, Min Zhang, Yang Liu

Enhancing Chat Language Models by Scaling High-quality Instructional Conversations (2023.emnlp-main)

Challenge:	Existing models for document-level context translation ignore documentlevel context.
Approach:	They propose a document-level context encoder to represent document- level context and integrate it into the Transformer model.
Outcome:	Experiments on NIST Chinese-English and IWSLT French-English datasets show that the proposed translation model outperforms the Transformer model significantly.

Copied to clipboard

Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Shengding Hu, Zhiyuan Liu, Maosong Sun, Bowen Zhou

Migician: Revealing the Magic of Free-Form Multi-Image Grounding in Multimodal Large Language Models (2025.findings-acl)

Challenge:	a recent study validates the effectiveness of chat language models by fine-tuning instruction data.
Approach:	They propose to use a large-scale dataset of instructional conversations to fine-tune a conversational model on instruction data.
Outcome:	The proposed model outperforms open-source models in key metrics including scale, average length, diversity, coherence, etc.

Copied to clipboard

You Li, Heyu Huang, Chi Chen, Kaiyu Huang, Chao Huang, Zonghao Guo, Zhiyuan Liu, Jinan Xu, Yuhua Li, Ruixuan Li, Maosong Sun

Jiuge: A Human-Machine Collaborative Chinese Classical Poetry Generation System (P19-3)

Challenge:	Existing MLLMs still struggle to achieve precise grounding in multi-image scenarios.
Approach:	They propose a Chain-of-Thought framework that integrates single-image grounding with multi-image comprehension to address this challenge.
Outcome:	The proposed model outperforms existing models in multi-image grounding tasks by 24.94% and surpasses larger 70B models.

Copied to clipboard

Guo Zhipeng, Xiaoyuan Yi, Maosong Sun, Wenhao Li, Cheng Yang, Jiannan Liang, Huimin Chen, Yuhui Zhang, Ruoyu Li

Language Modeling with Sparse Product of Sememe Experts (D18-1)

Challenge:	Existing systems for automatic poetry generation are model-oriented, resulting in poor user participation.
Approach:	They propose a human-machine collaborative Chinese classical poetry generation system called Jiuge . Jiuge allows users to revise unsatisfied parts of a generated poem draft repeatedly .
Outcome:	The proposed system allows users to revise unsatisfied parts of a generated poem draft repeatedly.

Copied to clipboard

Yihong Gu, Jun Yan, Hao Zhu, Zhiyuan Liu, Ruobing Xie, Maosong Sun, Fen Lin, Leyu Lin

Try to Substitute: An Unsupervised Chinese Word Sense Disambiguation Method Based on HowNet (2020.coling-main)

Challenge:	Existing language modeling methods rely on large-scale text data to learn the sequential patterns of words.
Approach:	They propose to use sememes to represent the implicit semantics behind words for language modeling . they propose to employ sememe-driven language models to fine-grained semem-level semantics .
Outcome:	Experiments on language modeling and the downstream application of headline generation show the effectiveness of SDLM.

Copied to clipboard

Bairu Hou, Fanchao Qi, Yuan Zang, Xurui Zhang, Zhiyuan Liu, Maosong Sun

Fuse It More Deeply! A Variational Transformer with Layer-Wise Latent Variable Inference for Text Generation (2022.naacl-main)

Challenge:	Existing unsupervised methods for word sense disambiguation cannot work for HowNet-based WSD because of its uniqueness.
Approach:	They propose a method which exploits the masked language model task of pre-trained language models to conduct word sense disambiguation using a lexical knowledge base as the sense inventory.
Outcome:	The proposed method achieves significantly better performance than baseline methods.

Copied to clipboard

Jinyi Hu, Xiaoyuan Yi, Wenhao Li, Maosong Sun, Xing Xie

Continual Relation Learning via Episodic Memory Activation and Reconsolidation (2020.acl-main)

Challenge:	Variational Auto-Encoders are often used for text generation tasks due to the sequential nature of the text.
Approach:	They propose a variational Transformer framework that learns a series of layer-wise latent variables with each inferred from those of lower layers and tightly coupled with the hidden states by low-rank tensor product.
Outcome:	The proposed framework can learn latent variables from lower layers and incorporate more information.

Copied to clipboard

Xu Han, Yi Dai, Tianyu Gao, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

Challenge:	Existing methods to learn incessantly emerging novel relations are overfitting the few memorized examples of old relations, causing confusion among existing relations.
Approach:	They introduce episodic memory activation and reconsolidation (EMAR) to continual relation learning.
Outcome:	The proposed method outperforms state-of-the-art models in catastrophic forgetting old relations.

Alternated Training with Synthetic and Authentic Data for Neural Machine Translation (2021.findings-acl)

Copied to clipboard

Rui Jiao, Zonghan Yang, Maosong Sun, Yang Liu

Multi-Modal Multi-Granularity Tokenizer for Chu Bamboo Slips (2025.coling-main)

Challenge:	Existing approaches to synthesizing data in NMT focus on leveraging monolingual data in training.
Approach:	They propose alternated training with synthetic and authentic data to improve NMT models' performance.
Outcome:	The proposed approach improves Chinese-English and German-English translation tasks over strong baselines.

Copied to clipboard

Yingfa Chen, Chenlong Hu, Cong Feng, Chenyang Song, Shi Yu, Xu Han, Zhiyuan Liu, Maosong Sun

OpenKE: An Open Toolkit for Knowledge Embedding (D18-2)

Challenge:	Using a multi-modal multi-granularity tokenizer, we analyze ancient Chinese scripts . a large proportion of the characters in ancient Chinese are rare or undeciphered .
Approach:	They propose a multi-modal multi-granularity tokenizer specifically designed for ancient Chinese scripts.
Outcome:	The proposed tokenizer improves on the part-of-speech tagging task on the Chu bamboo slip script.

Copied to clipboard

Xu Han, Shulin Cao, Xin Lv, Yankai Lin, Zhiyuan Liu, Maosong Sun, Juanzi Li

Weakly Supervised Vision-and-Language Pre-training with Relative Representations (2023.acl-long)

Challenge:	Existing knowledge embedding tools are available for embeddable knowledge graphs.
Approach:	They propose a unified framework and various fundamental models to embed knowledge graphs into a continuous low-dimensional space.
Outcome:	The toolkit and pre-trained embeddings are available on http://openke.thunlp.org/.

Copied to clipboard

Chi Chen, Peng Li, Maosong Sun, Yang Liu

Put It Back: Entity Typing with Language Model Enhancement (D18-1)

Challenge:	Weakly supervised vision-and-language pre-training (WVLP) uses only local descriptions of images as cross-modal anchors to construct weakly-aligned image-text pairs for pre- training.
Approach:	They propose to take a small number of aligned image-text pairs as anchors and represent each unaligned image and text by its similarities to these anchors.
Outcome:	The proposed model reduces the cost of pre-training while maintaining decent performance on downstream tasks.

Copied to clipboard

Ji Xin, Hao Zhu, Xu Han, Zhiyuan Liu, Maosong Sun

GATEAU: Selecting Influential Samples for Long Context Alignment (2025.emnlp-main)

Challenge:	Existing Entity typing models suffer from noisy labels due to distant supervision .
Approach:	They propose to enhance existing entity typing models with language model enhancement to measure compatibility between context sentences and labels.
Outcome:	The proposed model significantly outperforms the state-of-the-art model on benchmark datasets and is available on github.

Copied to clipboard

Shuzheng Si, Haozhe Zhao, Gang Chen, Yunshui Li, Kangyang Luo, Chuancheng Lv, Kaikai An, Fanchao Qi, Baobao Chang, Maosong Sun

Challenge:	Existing studies have attempted to scale up the available data volume by synthesizing long instruction-following samples, but a lack of a well-defined strategy for ensuring data quality may introduce low-quality samples and restrict the model’s performance.
Approach:	They propose a framework to identify influential samples enriched with long-range dependency relations that can be used to align large language models to handle instructions with extremely long contexts.
Outcome:	The proposed framework identifies samples with long-range dependency relations and shows that the model trained on these samples exhibits better instruction-following and long-context understanding capabilities.

A Multi-answer Multi-task Framework for Real-world Machine Reading Comprehension (D18-1)

Copied to clipboard

Jiahua Liu, Wan Wei, Maosong Sun, Hao Chen, Yantao Du, Dekang Lin

StateX: Enhancing RNN Recall via Post-training State Expansion (2026.findings-acl)

Challenge:	Existing models of machine reading comprehension (MRC) are based on cloze style questions or crowdworkers given a short passage from well-edited sources.
Approach:	They propose a multi-answer multi-task framework that uses multiple reference answers for multiple questions.
Outcome:	The proposed model increases the ROUGE-L score on the DuReader dataset from 44.18, the previous state-of-the-art, to 51.09 .

Copied to clipboard

Xingyu Shen, Yingfa Chen, Zhen Leng Thai, Xu Han, Zhiyuan Liu, Maosong Sun

Different Tunes Played with Equal Skill: Exploring a Unified Optimization Subspace for Parameter-Efficient Tuning (2022.findings-emnlp)

Challenge:	Existing studies show that RNNs with large recurrent states are expensive to train . however, the ability to recall contextual information from long contexts is underperforms them in certain aspects.
Approach:	They propose a framework that expands the states of pre-trained RNNs by scaling them up to 1.3B . they use a recurrent architecture that compresses contextual information into a fixedsize state .
Outcome:	Experiments on models with up to 1.3B parameters show that StateX expands state sizes without incurring high post-training costs or compromising other capabilities.

Copied to clipboard

Jing Yi, Weize Chen, Yujia Qin, Yankai Lin, Ning Ding, Xu Han, Zhiyuan Liu, Maosong Sun, Jie Zhou

Lingxi: A Diversity-aware Chinese Modern Poetry Generation System (2023.acl-demo)

Challenge:	Existing delta tuning algorithms freeze most of the parameters and only optimize minimal adaptive parameters.
Approach:	They propose to decompose DETs into a unified optimization subspace and conduct optimization within the subspace.
Outcome:	The proposed DETs achieve comparable performance to the original DET and can be transferred to another DET with non-trivial performance.

Copied to clipboard

Xinran Zhang, Maosong Sun, Jiafeng Liu, Xiaobing Li

ELLE: Efficient Lifelong Pre-training for Emerging Data (2022.findings-acl)

Challenge:	Chinese modern poetry generation is a challenging task because of the word segmentation problem and decoding methods . the decoding method may induce repetition and boredom and lower the diversity of generated poetry.
Approach:	They propose a Chinese word segmentation-based decoding system that incorporates Chinese word segments into tokenization.
Outcome:	The proposed system can achieve high vocabulary coverage rate with a reasonable vocabulary size.

Copied to clipboard

Yujia Qin, Jiajie Zhang, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

Better Robustness by More Coverage: Adversarial and Mixup Data Augmentation for Robust Finetuning (2021.findings-acl)

Challenge:	Existing pre-trained language models are typically trained with static data, ignoring that streaming data of various sources may continuously grow.
Approach:	They propose to use function preserved model expansion to expand existing PLM's width and depth to improve efficiency of knowledge acquisition.
Outcome:	The proposed model improves pre-training efficiency and performance over existing models on BERT and GPT.

Copied to clipboard

Chenglei Si, Zhengyan Zhang, Fanchao Qi, Zhiyuan Liu, Yasheng Wang, Qun Liu, Maosong Sun

LLM×MapReduce: Simplified Long-Sequence Processing using Large Language Models (2025.acl-long)

Challenge:	Pretrained language models perform poorly under adversarial attacks due to the large search space.
Approach:	They propose a method to cover a much larger proportion of the attack search space by adding textual adversarial examples during training.
Outcome:	The proposed method covers a much larger proportion of the attack search space.

Copied to clipboard

Zihan Zhou, Chong Li, Xinyi Chen, Shuo Wang, Yu Chao, Zhili Li, Haoyu Wang, Qi Shi, Zhixing Tan, Xu Han, Xiaodong Shi, Zhiyuan Liu, Maosong Sun

A Template-based Method for Constrained Neural Machine Translation (2022.emnlp-main)

Challenge:	Existing studies have focused on extending the context length of large language models (LLMs) due to their quadratic computational complexity and a lack of high-quality long training examples, most LLMs are trained with a limited window size.
Approach:	They propose a training-free framework that enables large language models to effectively process long texts using a divide-and-conquer strategy for comprehensive document understanding.
Outcome:	The proposed framework outperforms open-source and commercial long-context LLMs and is compatible with several models.

Copied to clipboard

Shuo Wang, Peng Li, Zhixing Tan, Zhaopeng Tu, Maosong Sun, Yang Liu

Learning from Context or Names? An Empirical Study on Neural Relation Extraction (2020.emnlp-main)

Challenge:	Existing methods to solve this problem can not satisfy the following three desiderata: (1) high translation quality, (2) high match accuracy, and (3) low latency.
Approach:	They propose a template-based method that can provide high translation quality and match accuracy and a low latency inference.
Outcome:	The proposed method outperforms baselines in lexically and structurally constrained translation tasks and can be used in a variety of applications.

Copied to clipboard

Hao Peng, Tianyu Gao, Xu Han, Yankai Lin, Peng Li, Zhiyuan Liu, Maosong Sun, Jie Zhou

Few-Shot Charge Prediction with Discriminative Legal Attributes (C18-1)

Challenge:	Existing datasets may leak shallow heuristics via entity mentions, thus contributing to the high performance on RE benchmarks.
Approach:	They propose an entity-masked contrastive framework for relation extraction to gain a deeper understanding on textual context and type information while avoiding rote memorization of entities.
Outcome:	The proposed framework improves the effectiveness and robustness of neural models in different RE scenarios.

Copied to clipboard

Zikun Hu, Xiang Li, Cunchao Tu, Zhiyuan Liu, Maosong Sun

Plug-and-Play Document Modules for Pre-trained Models (2023.acl-long)

Challenge:	Existing works on charge prediction perform well on high-frequency charges but are not capable of predicting few-shot charges with limited cases.
Approach:	They propose an attribute-attentive charge prediction model to infer attributes and charges simultaneously . they propose to use discriminative attributes as the internal mapping between fact descriptions and charges .
Outcome:	The proposed model outperforms baseline models on real-world datasets by more than 50% . the proposed model can predict the attributes and charges simultaneously .

Copied to clipboard

Chaojun Xiao, Zhengyan Zhang, Xu Han, Chi-Min Chan, Yankai Lin, Zhiyuan Liu, Xiangyang Li, Zhonghua Li, Zhao Cao, Maosong Sun

ERNIE: Enhanced Language Representation with Informative Entities (P19-1)

Challenge:	Large-scale pre-trained models have been widely adopted for document-oriented NLP tasks, such as question answering.
Approach:	They propose to decouple document encoding from downstream tasks by introducing a document plugin into the backbone of a PTM.
Outcome:	The proposed model can encode documents once and for all across different scenarios.

Copied to clipboard

Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang, Maosong Sun, Qun Liu

How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligence (2020.acl-main)

Challenge:	Existing pre-trained language models rarely consider incorporating knowledge graphs (KGs) Existing models capture rich semantic patterns from plain text and can be fine-tuned to improve performance of NLP tasks.
Approach:	They propose to incorporate knowledge graphs into pre-trained language models to enhance language representation with external knowledge.
Outcome:	The proposed model can take full advantage of lexical, syntactic, and knowledge information simultaneously.

Copied to clipboard

Haoxi Zhong, Chaojun Xiao, Cunchao Tu, Tianyang Zhang, Zhiyuan Liu, Maosong Sun

Challenge:	Legal Artificial Intelligence (LegalAI) focuses on applying artificial intelligence to help legal tasks.
Approach:	They introduce the history, current state, and future directions of research in LegalAI . they illustrate the tasks from the perspectives of legal professionals and NLP researchers .
Outcome:	The proposed system can reduce heavy and redundant work for legal professionals . it can also provide a reliable reference to those who are not familiar with the legal domain .

Modeling Semantic Compositionality with Sememe Knowledge (P19-1)

Copied to clipboard

Fanchao Qi, Junjie Huang, Chenghao Yang, Zhiyuan Liu, Xiao Chen, Qun Liu, Maosong Sun

IsOBS: An Information System for Oracle Bone Script (2020.emnlp-demos)

Challenge:	Semantic compositionality (SC) is defined as the phenomenon that the meaning of a complex linguistic unit can be composed of the meanings of its constituents.
Approach:	They propose to incorporate sememes into SC models and employ them in learning multiword expressions.
Outcome:	The proposed models achieve significant performance boost compared to baseline methods without sememe knowledge.

Copied to clipboard

Xu Han, Yuzhuo Bai, Keyue Qiu, Zhiyuan Liu, Maosong Sun

Challenge:	Oracle bone script (OBS) documents are the oldest continuously-used writing system in the world and are important for linguistic and historical research.
Approach:	They construct an information system for OBS to symbolize, serialize, and store OBS data at the character-level using efficient databases and retrieval modules.
Outcome:	The proposed system symbolizes, serializes, and stores OBS data at the character-level, based on efficient databases and retrieval modules.

Enhancing Open-Domain Task-Solving Capability of LLMs via Autonomous Tool Integration from GitHub (2025.acl-long)

Copied to clipboard

Bohan Lyu, Xin Cong, Heyang Yu, Pan Yang, Cheng Qian, Zihe Wang, Yujia Qin, Yining Ye, Yaxi Lu, Chen Qian, Zhong Zhang, Yukun Yan, Yankai Lin, Zhiyuan Liu, Maosong Sun

ProSparse: Introducing and Enhancing Intrinsic Activation Sparsity within Large Language Models (2025.coling-main)

Challenge:	Existing approaches lack flexibility to address diverse and ever-evolving user queries in open domains.
Approach:	They propose to evaluate LLMs on open-domain knowledge that requires tools to solve diverse and ever-evolving user queries.
Outcome:	The proposed system outperforms baselines in the open domain task-solving benchmark.

Copied to clipboard

Chenyang Song, Xu Han, Zhengyan Zhang, Shengding Hu, Xiyu Shi, Kuai Li, Chen Chen, Zhiyuan Liu, Guangli Li, Tao Yang, Maosong Sun

DocRED: A Large-Scale Document-Level Relation Extraction Dataset (P19-1)

Challenge:	Activation sparsity is a promising paradigm for accelerating model inference . few large language models achieve high activation spar and comparable performance .
Approach:	They propose a method to achieve activation sparsity and acceleration in large language models . they introduce ReLU activation and adopt progressive sparse regularization .
Outcome:	The proposed method achieves high activation sparsity and comparable model performance.

Copied to clipboard

Yuan Yao, Deming Ye, Peng Li, Xu Han, Yankai Lin, Zhenghao Liu, Zhiyuan Liu, Lixin Huang, Jie Zhou, Maosong Sun

LEGENT: Open Platform for Embodied Agents (2024.acl-demos)

Challenge:	Existing relation extraction methods focus on extracting intra-sentence relations for single entities.
Approach:	They propose a relation extraction dataset from Wikipedia and Wikidata with three features . document-level relation extraction is a task to identify relational facts between entities .
Outcome:	The proposed dataset is the largest human-annotated dataset for document-level RE from plain text.

Copied to clipboard

Zhili Cheng, Zhitong Wang, Jinyi Hu, Shengding Hu, An Liu, Yuge Tu, Pengkai Li, Lei Shi, Zhiyuan Liu, Maosong Sun