Papers by Tao Xu

Lil: Less is Less When Applying Post-Training Sparse-Attention Algorithms in Long-Decode Stage (2026.findings-acl)

Challenge:	Representative models like LLaVA and MiniGPT-4 have great capabilities in various tasks.
Approach:	They propose a unified model to represent various multi-modal tasks using a single representation.
Outcome:	The proposed model outperforms existing models in a variety of tasks while maintaining generality and scalability.

Copied to clipboard

Junhao Hu, Fangze Li, Mingtao Xu, Feifan Meng, Shiju Zhao, Tiancheng Hu, Ting Peng, Anmin Liu, Wenrui Huang, Chenxu Liu, Ziyue Hua, Tao Xie

Challenge:	Prior work typically decomposes inference into prefill and decode stages, with the decode stage dominating total latency.
Approach:	They propose an algorithm that detects threshold where information loss exceeds information gain during sparse decoding to reduce token consumption by up to 90% and a marginal accuracy degradation of less than 2%.
Outcome:	The proposed algorithm reduces token consumption by 90% with a marginal accuracy degradation of less than 2% across reasoning-intensive benchmarks.

SimulSpeech: End-to-End Simultaneous Speech to Text Translation (2020.acl-main)

Copied to clipboard

Yi Ren, Jinglin Liu, Xu Tan, Chen Zhang, Tao Qin, Zhou Zhao, Tie-Yan Liu

Biology-Instructions: A Dataset and Benchmark for Multi-Omics Sequence Understanding Capability of Large Language Models (2025.findings-emnlp)

Challenge:	SimulSpeech is an end-to-end simultaneous speech to text translation system . conventional approaches to simultaneous speech translation divide the translation process into two stages .
Approach:	They develop an end-to-end simultaneous speech to text translation system which translates speech in source language to text in target language concurrently.
Outcome:	The proposed system achieves reasonable BLEU scores and lower delay compared to full-sentence translation model.

Copied to clipboard

Haonan He, Yuchen Ren, Yining Tang, Ziyang Xu, Junxian Li, Minghao Yang, Di Zhang, Yuan Dong, Tao Chen, Shufei Zhang, Yuqiang Li, Nanqing Dong, Wanli Ouyang, Dongzhan Zhou, Peng Ye

TexSmart: A System for Enhanced Natural Language Understanding (2021.acl-demo)

Challenge:	Biology-Instructions is the first large-scale instruction-tuning dataset for multi-omics biological sequences.
Approach:	They propose a large-scale instruction-tuning dataset for multi-omics biological sequences . they propose 'chatMultiOmics' to overcome limitations of current LLMs on multi-ome tasks .
Outcome:	The proposed dataset bridges LLMs and complex biological sequence-related tasks while maintaining conversational fluency.

Copied to clipboard

Lemao Liu, Haisong Zhang, Haiyun Jiang, Yangming Li, Enbo Zhao, Kun Xu, Linfeng Song, Suncong Zheng, Botong Zhou, Dick Zhu, Xiao Feng, Tao Chen, Tao Yang, Dong Yu, Feng Zhang, ZhanHui Kang, Shuming Shi

Probing Multimodal Large Language Models for Global and Local Semantic Representations (2024.lrec-main)

Challenge:	TexSmart supports fine-grained named entity recognition (NER) Large-scale fine-granular entity types are expected to provide richer semantic information for downstream NLP applications.
Approach:	They introduce TexSmart, a text understanding system that supports fine-grained named entity recognition (NER) and enhanced semantic analysis functionalities.
Outcome:	The proposed system supports fine-grained named entity recognition (NER) and enhanced semantic analysis functions.

Copied to clipboard

Mingxu Tao, Quzhe Huang, Kun Xu, Liwei Chen, Yansong Feng, Dongyan Zhao

Revisiting Over-Smoothness in Text to Speech (2022.acl-long)

Challenge:	Existing studies have focused on the ability of MLLMs to generate single tokens one by one, while lacking studies about how their representation vectors can encode global multimodal information.
Approach:	They propose to use image-caption corpus to train Multimodal Large Language Models (MLLMs) . they find that the topmost layers encode more global semantic information .
Outcome:	The proposed models can encode more global semantic information, rather than the topmost layers, and perform better on visual-language entailment tasks.

Copied to clipboard

Yi Ren, Xu Tan, Tao Qin, Zhou Zhao, Tie-Yan Liu

Incentivizing Strong Reasoning from Weak Supervision (2026.eacl-long)

Challenge:	Non-autoregressive text to speech models ignore correlation in time and frequency domains, causing blurry results.
Approach:	They revisit the problem of over-smoothness in non-autoregressive text to speech models . they use methods that reduce complexity of data distributions and improve modeling methods .
Outcome:	The proposed models achieve better voice quality and faster inference speed than autoregressive models.

Copied to clipboard

Yige Yuan, Teng Xiao, Shuchang Tao, Xue Wang, Jinyang Gao, Bolin Ding, Bingbing Xu

ContextBLIP: Doubly Contextual Alignment for Contrastive Image Retrieval from Linguistically Complex Descriptions (2024.findings-acl)

Challenge:	Large language models (LLMs) have demonstrated impressive performance on reasoning-intensive tasks, but enhancing their reasoning abilities typically relies on expensive high-quality demonstrations and reinforcement learning.
Approach:	They propose to incentivize reasoning abilities of large language models without expensive demonstrations and reinforcement learning.
Outcome:	The proposed model can recover 94% of the gains of expensive RL at a fraction of the cost.

Copied to clipboard

Honglin Lin, Siyu Li, Guoshun Nan, Chaoyue Tang, Xueting Wang, Jingxin Xu, Rong Yankai, Zhouzhili Zhouzhili, Yutong Gao, Qimei Cui, Xiaofeng Tao

Visual Data in NLP

Challenge:	Existing approaches to image retrieval from contextual descriptions (IRCD) lag behind human performance in IRCD.
Approach:	They propose a method that relies on a doubly contextual alignment scheme for challenging IRCD.
Outcome:	The proposed method can yield comparable results with GPT-4V, despite fewer parameters.

Rehearsal-free Continual Language Learning via Efficient Parameter Isolation (2023.acl-long)

Copied to clipboard

Zhicheng Wang, Yufang Liu, Tao Ji, Xiaoling Wang, Yuanbin Wu, Congcong Jiang, Ye Chao, Zhencong Han, Ling Wang, Xu Shao, Wenqiu Zeng

Green & Sustainable NLP

Challenge:	Existing methods for learning continual tasks do not cache history data, which makes the problem more challenging.
Approach:	They propose a method that allocates a small portion of private parameters and learns them with a shared pre-trained model.
Outcome:	The proposed method is comparable to existing methods and comparable to those using historical data.

Data Interpreter: An LLM Agent for Data Science (2025.findings-acl)

Copied to clipboard

Sirui Hong, Yizhang Lin, Bang Liu, Bangbang Liu, Binhao Wu, Ceyao Zhang, Danyang Li, Jiaqi Chen, Jiayi Zhang, Jinlin Wang, Li Zhang, Lingyao Zhang, Min Yang, Mingchen Zhuge, Taicheng Guo, Tuo Zhou, Wei Tao, Robert Tang, Xiangtao Lu, Xiawu Zheng, Xinbing Liang, Yaying Fei, Yuheng Cheng, Yongxin Ni, Zhibin Gou, Zongze Xu, Yuyu Luo, Chenglin Wu

Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compression (2021.emnlp-main)

Challenge:	Large Language Models (LLMs) excel in various domains but face challenges when applied to data science workflows due to their complex, multi-stage nature.
Approach:	They propose a hierarchical graph-based agent that represents complexity and a progressive strategy for step-by-step verification, refinement, and consistent context management.
Outcome:	The proposed agent surpasses state-of-the-art baselines on the MATH dataset and performs better on InfiAgent-DABench.

Copied to clipboard

Canwen Xu, Wangchunshu Zhou, Tao Ge, Ke Xu, Julian McAuley, Furu Wei

One Time of Interaction May Not Be Enough: Go Deep with an Interaction-over-Interaction Network for Response Selection in Dialogues (P19-1)

Challenge:	Recent studies on compression of pretrained language models usually use preserved accuracy as the metric for evaluation.
Approach:	They propose two new metrics that measure how closely a compressed model mimics the original model.
Outcome:	The proposed metrics measure how closely a compressed model (i.e., student) mimics the original model (e.g., teacher).

Copied to clipboard

Chongyang Tao, Wei Wu, Can Xu, Wenpeng Hu, Dongyan Zhao, Rui Yan

Natural Language Interfaces

Challenge:	Currently, retrieval-based dialogues are performed in shallow ways . a recent study investigated the problem of context-response matching in open-domain .
Approach:	They propose a model that lets utterance-response interaction go deep by stacking interaction blocks.
Outcome:	The proposed model outperforms state-of-the-art methods on three benchmark data sets.

MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing (2026.acl-industry)

Copied to clipboard

Junbo Niu, Zheng Liu, Zhuangcheng Gu, Bin Wang, Linke Ouyang, Zhiyuan Zhao, Tao Chu, Tianyao He, Fan Wu, Qintong Zhang, Zhenjiang Jin, Guang Liang, Rui Zhang, Wenzheng Zhang, Yuan Qu, Zhifei Ren, Yuefeng Sun, Zirui Tang, Boyu Niu, Yuanhong Zheng, Dongsheng Ma, Ziyang Miao, Hejun Dong, Siyi Qian, Junyuan Zhang, Fangdong Wang, Jingzhou Chen, Xiaomeng Zhao, Liqun Wei, Wei Li, Shasha Wang, RuiLiang Xu, Yuanyuan Cao, Lu Chen, Qianqian Wu, Huaiyu Gu, Lindong Lu, Dechen Lin, null Shenguanlin, Xuanhe Zhou, Linfeng Zhang, Yuhang Zang, Xiaoyi Dong, Jiaqi Wang, Bo Zhang, Lei Bai, Pei Chu, Weijia Li, Jiang Wu, Lijun Wu, Zhenxiang Li, Guangyu Wang, Zhongying Tu, Chao Xu, Kai Chen, Bowen Zhou, Dahua Lin, Wentao Zhang, Conghui He

Green & Sustainable NLP

Challenge:	Document images are characterized by higher resolutions, denser content, and more complex structural layouts.
Approach:	They propose a 1.2B-parameter document parsing vision-language model that decouples layout analysis from local content recognition.
Outcome:	The proposed model surpasses general-purpose and domain-specific models on multiple benchmarks while maintaining significantly lower computational overhead.

Blow the Dog Whistle: A Chinese Dataset for Cant Understanding with Common Sense and World Knowledge (2021.naacl-main)

Copied to clipboard

Canwen Xu, Wangchunshu Zhou, Tao Ge, Ke Xu, Julian McAuley, Furu Wei

ADAM: Dense Retrieval Distillation with Adaptive Dark Examples (2024.findings-acl)

Challenge:	Cant is important for understanding advertising, comedies and dogwhistle politics . currently, there are very few resources available for the research of cant .
Approach:	They propose a large and diverse dataset for creating and understanding cant from a computational linguistics perspective.
Outcome:	The proposed dataset can be used to test word embedding similarity and pretrained language models.

Copied to clipboard

Chongyang Tao, Chang Liu, Tao Shen, Can Xu, Xiubo Geng, Binxing Jiao, Daxin Jiang

“I’ve Decided to Leak”: Probing Internals Behind Prompt Leakage Intents (2025.emnlp-main)

Challenge:	Existing methods to retrieve data from multiple encoders are too trivial for the teacher to distinguish, preventing the teacher from transferring abundant dark knowledge to the student.
Approach:	They propose a knowledge distillation framework that can better transfer the dark knowledge held in the teacher with adaptive dark examples.
Outcome:	The proposed framework can better transfer the dark knowledge held in the teacher with adaptive dark examples.

Copied to clipboard

Jianshuo Dong, Yutong Zhang, Liu Yan, Zhenyu Zhong, Tao Wei, Ke Xu, Minlie Huang, Chao Zhang, Han Qiu

Unsupervised Pivot Translation for Distant Languages (P19-1)

Challenge:	Large language models (LLMs) exhibit prompt leakage vulnerabilities, raising intellectual property and confidentiality concerns.
Approach:	They use probing techniques to capture LLMs’ intent-related internal representations and show that they internalize prompt leakage intents in their hidden states before generating tokens.
Outcome:	The proposed probes achieve 90%+ AUROC across all tested models, even when applied to new system prompts and attacks.

Copied to clipboard

Yichong Leng, Xu Tan, Tao Qin, Xiang-Yang Li, Tie-Yan Liu

OctoBench: Benchmarking Scaffold-Aware Instruction Following in Repository-Grounded Agentic Coding (2026.acl-long)

Challenge:	Unsupervised neural machine translation (NMT) is a popular method for transferring information between languages.
Approach:	They propose an unsupervised pivot translation method which translates a language to a distant language through multiple hops.
Outcome:	The proposed method improves translation on 20 languages and 294 distant languages on 20 different languages and language pairs.

Copied to clipboard

Deming Ding, Shichun Liu, Enhui Yang, Jiahang Lin, Ziying Chen, Shihan Dou, Honglin Guo, Weiyu Cheng, Pengyu Zhao, Chengjun Xiao, Qunhong Zeng, Qi Zhang, Xuanjing Huang, Qidi Xu, Tao Gui

Programming Languages in NLP

Challenge:	coding scaffolds that follow heterogeneous instructions remain under-examined in software engineering . coding models are capable software agents, but their ability to follow constraints remains under-explored .
Approach:	They introduce OctoBench, which benchmarks scaffold-aware instruction following in agentic coding.
Outcome:	The proposed benchmark aims to accelerate the development of more scaffold-aware agents.

Enhancing Online Recruitment with Category-Aware MoE and LLM-based Data Augmentation (2026.acl-industry)

Copied to clipboard

Minping Chen, Bing Xu, Zulong Chen, Chuanfei Xu, Ying Zhou, Zui Tao, Zeyi Wen

MMAC: A Multilingual, Multimodal Alignment Framework for Cultural Grounding Evaluation (2026.acl-long)

Challenge:	Existing methods to measure the matching degree of a job and a candidate face several challenges, such as low-quality job descriptions and similar candidate-job pairs.
Approach:	They propose a large language model-based method that polishes and rewrites low-quality job descriptions by leveraging chain-of-thought prompts and category-aware Mixture of Experts (MoE) module incorporates category embeddings to dynamically assign weights to the experts and learns more distinguishable patterns for similar candidate-job pairs.
Outcome:	The proposed method surpasses existing methods by 2.40% in AUC and 7.46% in GAUC and boosts click-through conversion rate (CTCVR) by 19.4% in online tests, saving millions of CNY in external headhunting expenses.

Copied to clipboard

Weihua Zheng, Zhengyuan Liu, Tanmoy Chakraborty, Weiwen Xu, Xiaoxue Gao, Bryan Chen Zhengyu Tan, Bowei Zou, Chang Liu, Yujia Hu, Xing Xie, Xiaoyuan Yi, Jing Yao, Chaojun Wang, Long Li, Rui Liu, Huiyao Liu, Koji Inoue, Ryuichi Sumida, Tatsuya Kawahara, Fan Xu, Lingyu Ye, Wei Tian, Dongjun Kim, Jimin Jung, Jaehyung Seo, Nadya Yuki Wangsajaya, Pham Minh Duc, Ojasva Saxena, Palash Nandi, Xiyan Tao, Wiwik Karlina, Tuan Luong, Keertana Arun Vasan, Roy Ka-Wei Lee, Nancy F. Chen

MMDialog: A Large-scale Multi-turn Dialogue Dataset Towards Multi-modal Open-domain Conversation (2023.acl-long)

Challenge:	Existing models lack cultural alignment across modalities and languages . a new framework to assess cultural awareness across linguistics and languages is needed .
Approach:	They propose a framework that integrates tri-modally aligned cultural benchmarks and a five-dimensional evaluation protocol to assess cross-country awareness disparities.
Outcome:	The proposed framework assesses cultural awareness disparities across modalities and languages . it is the first dataset aligned at the input level across text, image, and speech .

Copied to clipboard

Jiazhan Feng, Qingfeng Sun, Can Xu, Pu Zhao, Yaming Yang, Chongyang Tao, Dongyan Zhao, Qingwei Lin

Natural Language Interfaces

Challenge:	MMDialog is a dataset of 1.08 million real-world dialogues with 1.53 million unique images across 4,184 topics.
Approach:	They propose to use a curated set of 1.08 million dialogues with 1.53 million unique images to generalize the open domain.
Outcome:	The proposed system can predict responses to multi-modal content with state-of-the-art techniques and measure their performance.

PCL: Peer-Contrastive Learning with Diverse Augmentations for Unsupervised Sentence Embeddings (2022.emnlp-main)

Copied to clipboard

Qiyu Wu, Chongyang Tao, Tao Shen, Can Xu, Xiubo Geng, Daxin Jiang

Mobile-Bench: An Evaluation Benchmark for LLM-based Mobile Agents (2024.acl-long)

Challenge:	Existing approaches to learning sentence embeddings in unsupervised manner depend on mono-augmenting . existing approaches depend on augmenting biases and thus corrupt the quality of sentence embeds.
Approach:	They propose a method to augment a sentence with a semantically-close positive instance to construct contrastive pairs in unsupervised manner.
Outcome:	The proposed method improves performance on STS benchmarks and compares with existing methods.

Copied to clipboard

Shihan Deng, Weikai Xu, Hongda Sun, Wei Liu, Tao Tan, Liujianfeng Liujianfeng, Ang Li, Jian Luan, Bin Wang, Rui Yan, Shuo Shang

Dense Information Flow for Neural Machine Translation (N18-1)

Challenge:	Existing benchmarks for LLM-based mobile agents are insufficient to evaluate their capabilities.
Approach:	They propose a benchmark to evaluate LLM-based mobile agents' planning capabilities . they expand UI operations by incorporating 103 APIs to accelerate task completion .
Outcome:	The proposed benchmarks are based on 103 collected APIs and real user queries . the data is categorized into three distinct groups: SAST, SAMT, and MAMT .

Copied to clipboard

Yanyao Shen, Xu Tan, Di He, Tao Qin, Tie-Yan Liu

Neural Response Generation with Meta-words (P19-1)

Challenge:	Recent advances in deep neural networks have improved learning performance for NMT . Residual connections allow features from previous layers to be accumulated to the next layer easily.
Approach:	They propose a densely connected NMT architecture that can train more efficiently for NMT.
Outcome:	The proposed architecture improves learning performance and attention quality on multiple datasets.

Copied to clipboard

Can Xu, Wei Wu, Chongyang Tao, Huang Hu, Matt Schuerman, Ying Wang

A Fast and High-quality Text-to-Speech Method with Compressed Auxiliary Corpus and Limited Target Speaker Corpus (2024.lrec-main)

Challenge:	Experimental results show that meta-words can be used to generate open domain dialogues . human-machine conversation is a fundamental problem in NLP .
Approach:	They propose a goal-tracking memory network that formalizes meta-word expression as a target in response generation and manages the generation process with a state memory panel and a controller.
Outcome:	The proposed model outperforms state-of-the-art generation models in response relevance, response diversity, and accuracy.

Copied to clipboard

Ye Tao, Chaofeng Lu, Meng Liu, Kai Xu, Tianyu Liu, Yunlong Tian, Yongjie Du

Synergistic Interplay between Search and Large Language Models for Information Retrieval (2024.acl-long)

Challenge:	Existing methods to generate high-quality speech with limited target speaker corpus require extensive training data.
Approach:	They propose an auxiliary corpus compression algorithm that reduces the training cost while the naturalness of synthesized speech is not significantly degraded.
Outcome:	The proposed method significantly reduces training costs while maintaining the naturalness of synthesized speech.

Copied to clipboard

Jiazhan Feng, Chongyang Tao, Xiubo Geng, Tao Shen, Can Xu, Guodong Long, Dongyan Zhao, Daxin Jiang

Towards Understanding Omission in Dialogue Summarization (2023.acl-long)

Challenge:	Information retrieval (IR) is an indispensable technique for locating relevant resources from vast amounts of data.
Approach:	They propose a framework that facilitates information refinement through synergy between RMs and LLMs.
Outcome:	The proposed framework improves the performance of large-scale retrieval benchmarks on web searches and low-resource retrieval tasks.

Copied to clipboard

Yicheng Zou, Kaitao Song, Xu Tan, Zhongkai Fu, Qi Zhang, Dongsheng Li, Tao Gui

LONGAGENT: Achieving Question Answering for 128k-Token-Long Documents through Multi-Agent Collaboration (2024.emnlp-main)

Challenge:	Existing methods for dialogue summarization are far from satisfactory . omission is a major factor in affecting the quality of summarizing, but few studies have explored the problem .
Approach:	They propose a dataset that provides high-quality omission labels for dialogue summarization . they propose to use this dataset to detect omitted dialogue utterances .
Outcome:	The proposed dataset improves summarization quality by providing ground-truth omission labels . the proposed dataset and codes are publicly available .

Copied to clipboard

Jun Zhao, Can Zu, Xu Hao, Yi Lu, Wei He, Yiwen Ding, Tao Gui, Qi Zhang, Xuanjing Huang

Question Answering

Challenge:	Large language models (LLMs) have been successful in understanding language and processing text, but their cost prohibits their practical applications.
Approach:	They propose a multi-agent collaboration method that breaks down lengthy documents into smaller, more manageable chunks and organizes the member agents to read their assigned chunks.
Outcome:	The proposed method achieves 16.42% and 1.63% accuracy gains over existing models on single-hop and multi-hop QA settings.

FastCorrect 2: Fast Error Correction on Multiple Candidates for Automatic Speech Recognition (2021.findings-emnlp)

Copied to clipboard

Yichong Leng, Xu Tan, Rui Wang, Linchen Zhu, Jin Xu, Wenjie Liu, Linquan Liu, Xiang-Yang Li, Tao Qin, Edward Lin, Tie-Yan Liu

Beyond Boundaries: Learning a Universal Entity Taxonomy across Datasets and Languages for Open Named Entity Recognition (2025.coling-main)

Challenge:	Error correction is widely used in automatic speech recognition (ASR) to post-process the generated sentence.
Approach:	They propose a fast correction model that takes multiple ASR candidates as input for better correction accuracy.
Outcome:	The proposed model can reduce the word error rate (WER) with multiple candidates by 3.2% and 2.6%.

Copied to clipboard

Yuming Yang, Wantong Zhao, Caishuang Huang, Junjie Ye, Xiao Wang, Huiyuan Zheng, Yang Nan, Yuran Wang, Xueying Xu, Kaixin Huang, Yunke Zhang, Tao Gui, Qi Zhang, Xuanjing Huang

Beyond Perplexity: Multi-dimensional Safety Evaluation of LLM Compression (2024.findings-emnlp)

Challenge:	Current Large Language Models struggle with complex entity taxonomies in open domains and lack NER capabilities.
Approach:	They propose a dataset to guide LLMs' generalization in Open NER under a universal entity taxonomy.
Outcome:	The proposed model outperforms GPT-4 in 3 out-of-domain benchmarks across 15 datasets and 6 languages.

Copied to clipboard

Zhichao Xu, Ashim Gupta, Tao Li, Oliver Bentham, Vivek Srikumar

Reinforcement Learning on Pre-Training Data (2026.acl-long)

Challenge:	Prior work on compression prioritizes preserving perplexity, which is analogous to training loss.
Approach:	They examine the impact of model compression along four dimensions: degeneration harm, representational harm, dialect bias, and language modeling and downstream task performance.
Outcome:	The proposed compression methods can lead to unexpected consequences, the authors show . quantization preserves bias while pruning degrades quickly.

Copied to clipboard

Siheng Li, Kejiao Li, Zenan Xu, Guanhua Huang, Kun Li, Haoyuan Wu, null Wujiajia, Zihao Zheng, Chenchen Zhang, Kun Shi, Xue Gong, Qi Yi, Ruibin Xiong, Tingqiang Xu, Yuhao Jiang, Jianfeng Yan, Yuyuan Zeng, Guanghui Xu, Jinbao Xue, Zhijiang xu, Zheng Fang, Shuai LI, Qibin Liu, Xiaoxue Li, Zhuoyu Li, Yangyu Tao, Fei Gao, Cheng Jiang, Bochao Wang, Kai Liu, Jianchen Zhu, Wai Lam, Bo Zhou, Di Wang

Low-Bit Quantization Favors Undertrained LLMs (2025.acl-long)

Challenge:	Recent progress in large language models is driven by scaling of training compute through pre-training with nexttoken prediction (NTP) or post-training (RL) Pre-training using NTP enables models to acquire extensive knowledge and skills from general data, but it suffers from data inefficiency and catastrophic forgetting in continual learning settings.
Approach:	They propose to scale training compute through pre-training with next-token prediction (NTP) or post-training by scaling reinforcement learning (RL) to improve learning from general data.
Outcome:	Experiments on multiple benchmarks and models show that the proposed approach improves continual pre-training and provides a strong foundation for post-training on Qwen3-8B-Base.

Copied to clipboard

Xu Ouyang, Tao Ge, Thomas Hartvigsen, Zhisong Zhang, Haitao Mi, Dong Yu

Challenge:	Larger models or those trained on fewer tokens exhibit less quantization-induced degradation (QiD), while smaller, well-trained models face significant performance losses.
Approach:	They propose to use QiD to measure an LLM’s training levels and determine the number of training tokens required for fully training LLMs of various sizes.
Outcome:	The proposed scaling laws can predict the quantization performance of different-sized LLMs trained with tokens.

BCL: Bayesian In-Context Learning Framework for Information Extraction (2026.findings-acl)

Copied to clipboard

Haoliang Liu, Chengkun Cai, Xu Zhao, Han Zhu, Shizhou Huang, Xinglin Zhang, Tao Chen, Jenq-Neng Hwang, Zhang Huaping, Lei Li

Low-Resource NLP

Challenge:	Existing information extraction (IE) tasks rely on in-context learning with large language models.
Approach:	They propose a Bayesian-based in-context learning framework that refines label representations across IE tasks using particle filtering and Bayes updates.
Outcome:	The proposed framework improves performance over existing methods (up to 30%) it underperforms one-shot prompting by a substantial margin on NER tasks and CodeIE fails on RE tasks with near-zero micro-F1.

Knowledge-Grounded Dialogue Generation with Pre-trained Language Models (2020.emnlp-main)

Copied to clipboard

Xueliang Zhao, Wei Wu, Can Xu, Chongyang Tao, Dongyan Zhao, Rui Yan

DeepKE: A Deep Learning Based Knowledge Extraction Toolkit for Knowledge Base Population (2022.emnlp-demos)

Challenge:	Empirical results indicate that pre-trained language models can significantly outperform state-of-the-art methods in both automatic evaluation and human judgment.
Approach:	They propose to equip a pre-trained language model with a knowledge selection module to generate knowledge-grounded dialogues.
Outcome:	The proposed model outperforms state-of-the-art methods in evaluation and human judgment.

Copied to clipboard

Ningyu Zhang, Xin Xu, Liankuan Tao, Haiyang Yu, Hongbin Ye, Shuofei Qiao, Xin Xie, Xiang Chen, Zhoubo Li, Lei Li

OEE-CFC: A Dataset for Open Event Extraction from Chinese Financial Commentary (2024.findings-emnlp)

Challenge:	Existing knowledge extraction tools are not complete due to emerging entities and relations in real-world applications.
Approach:	They propose an open-source knowledge extraction toolkit DeepKE that supports low-resource, document-level and multimodal scenarios in the knowledge base population.
Outcome:	The proposed toolkit supports low-resource, document-level and multimodal scenarios in the knowledge base population.

Copied to clipboard

Qizhi Wan, Changxuan Wan, Rong Hu, Dexi Liu, Xu Wenwu, Kang Xu, Zou Meihua, Liu Tao, Jie Yang, Zhenwei Xiong

GroundingGPT: Language Enhanced Multi-modal Grounding Model (2024.acl-long)

Challenge:	Existing corpora with unconventional entities serving as event arguments lack rich multi-events and shared arguments.
Approach:	They develop an open event template that includes 21 event argument roles and an open corpus supporting open event extraction.
Outcome:	The proposed corpus includes 17,469 events, 44,221 arguments, 3,644 complex arguments, and 5,898 shared arguments.

Copied to clipboard

Zhaowei Li, Qi Xu, Dong Zhang, Hang Song, YiQing Cai, Qi Qi, Ran Zhou, Junting Pan, Zefeng Li, Vu Tu, Zhida Huang, Tao Wang

InferenceDynamics: Adaptive LLM Routing through Structured Capability and Knowledge Profiling (2026.acl-long)

Challenge:	Existing multi-modal large language models focus on capturing global information while neglecting the fine-grained local information in multimodal inputs.
Approach:	They propose an end-to-end language enhanced multi-modal grounding model that performs fine-grained grounding tasks for image, video and audio.
Outcome:	The proposed model achieves impressive fine-grained understanding of multi-modal inputs while maintaining or improving its global comprehension capabilities.

Copied to clipboard

Haochen Shi, Tianshi Zheng, Weiqi Wang, Baixuan Xu, Chunyang Li, Chunkit Chan, Tao Fan, Yangqiu Song

Harder Task Needs More Experts: Dynamic Routing in MoE Models (2024.acl-long)

Challenge:	Large Language Model (LLM) routing is a pivotal technique for navigating a diverse landscape of LLMs.
Approach:	They propose a flexible and scalable multi-dimensional routing framework that models the capability and knowledge of models.
Outcome:	The proposed framework can be used to generalize and identify top-performing models for group-level routing using modern benchmarks including MMLU-Pro, GPQA, BigGenBench, and LiveBench.

Copied to clipboard

Quzhe Huang, Zhenwei An, Nan Zhuang, Mingxu Tao, Chen Zhang, Yang Jin, Kun Xu, Kun Xu, Liwei Chen, Songfang Huang, Yansong Feng

TreeRAG: Unleashing the Power of Hierarchical Storage for Enhanced Knowledge Retrieval in Long Documents (2025.findings-acl)

Challenge:	Unlike existing MoE approaches that rely on fixed TopK Routing, our dynamic expert selection framework dynamically allocates experts based on the confidence level in expert selection for each input.
Approach:	They propose a dynamic expert selection framework that dynamically allocates experts based on the confidence level in expert selection for each input.
Outcome:	The proposed method achieves an average improvement of 0.7% with less than 90% activated parameters and outperforms dense models in QA and machine translation tasks.

Copied to clipboard

Wenyu Tao, Xiaofen Xing, Yirong Chen, Linyi Huang, Xiangmin Xu

Information Retrieval

Challenge:	Traditional RAG frameworks struggle to retrieve all relevant knowledge points . a new approach to retrieve long documents is proposed to improve performance in NLP .
Approach:	They propose a tree-based approach to document knowledge retrieval that preserves hierarchical structure . treeRAG is a key technique for enhancing the text generation capabilities of Large Language Models .
Outcome:	The proposed approach improves recall quality and precision compared to existing methods and better performance to question-answering tasks.

DeepRapper: Neural Rap Generation with Rhyme and Rhythm Modeling (2021.acl-long)

Copied to clipboard

Lanqing Xue, Kaitao Song, Duocai Wu, Xu Tan, Nevin L. Zhang, Tao Qin, Wei-Qiang Zhang, Tie-Yan Liu

Concept rather than Document: Context Compression via AMR-based Conceptual Entropy (2026.findings-acl)

Challenge:	Existing systems for rap generation focus on rhyming lyrics but ignore rhythmic beats . rap lyrics need to be semantically meaningful and fashionable to convey interesting stories .
Approach:	They develop a Transformer-based rap generation system that can model both rhymes and rhythms.
Outcome:	The proposed system generates high-quality raps with rhymes and rhythms . it is based on a Transformer-based language model .

Copied to clipboard

Kaize Shi, Xueyao Sun, Xiaohui Tao, Lin Li, Qika Lin, Guandong Xu

Robust and Scalable Model Editing for Large Language Models (2024.lrec-main)

Challenge:	Existing methods rely on lexical or token-level features that fragment semantic units and fail to capture conceptually essential content.
Approach:	They propose an unsupervised framework leveraging Abstract Meaning Representation to preserve essential information while filtering irrelevant text.
Outcome:	The proposed framework outperforms RAG and existing baselines while preserving essential information while filtering irrelevant text.

Copied to clipboard

Yingfa Chen, Zhengyan Zhang, Xu Han, Chaojun Xiao, Zhiyuan Liu, Chen Chen, Kuai Li, Tao Yang, Maosong Sun

Improving Grammatical Error Correction with Machine Translation Pairs (2020.findings-emnlp)

Challenge:	Existing methods that ignore contextual knowledge fail to reliably fall back to parametric knowledge when presented with irrelevant context.
Approach:	They propose to use contextual knowledge to update and correct LLMs' knowledge by in-context editing instead of retraining.
Outcome:	The proposed method outperforms current state-of-the-art methods by a large margin on a dataset that contains irrelevant questions.

Copied to clipboard

Wangchunshu Zhou, Tao Ge, Chang Mu, Ke Xu, Furu Wei, Ming Zhou

Machine Translation

Challenge:	Existing methods to generate error-corrected sentence pairs for improving grammatical error correction are not available.
Approach:	They propose a method to generate error-corrected sentence pairs for improving grammatical error correction based on machine translation models of different qualities .
Outcome:	The proposed method can generate multiple error-corrected sentence pairs from Chinese to English text.

Leveraging Large Language Models for NLG Evaluation: Advances and Challenges (2024.emnlp-main)

Copied to clipboard

Zhen Li, Xiaohan Xu, Tao Shen, Can Xu, Jia-Chen Gu, Yuxuan Lai, Chongyang Tao, Shuai Ma

TemplateRL: Structured Template-Guided Reinforcement Learning for LLM Reasoning (2026.findings-acl)

Challenge:	introducing Large Language Models (LLMs) has opened new avenues for assessing generated content quality, e.g., coherence, creativity, and context relevance.
Approach:	They propose a taxonomy for organizing existing LLM-based evaluation metrics and a structured framework to understand and compare them.
Outcome:	The proposed taxonomy offers a framework to understand and compare LLM-based evaluation methods.

Copied to clipboard

Jinyang Wu, Chonghua Liao, Mingkuan Feng, Shuai Zhang, Zhengqi Wen, Haoran Luo, Ling Yang, Huazhe Xu, Jianhua Tao

MobileVLM: A Vision-Language Model for Better Intra- and Inter-UI Understanding (2024.findings-emnlp)

Challenge:	Existing RL methods rely on unstructured self-sampling to fit scalar rewards, resulting in inefficient rollouts.
Approach:	They propose a structured template-guided RL framework that augments policy optimization with explicit template guidance.
Outcome:	Experiments show that TemplateRL outperforms GRPO and GRPI by 99% on AIME and 41% on AMC with superior stability on weak models and remarkable cross-domain generalization.

Copied to clipboard

Qinzhuo Wu, Weikai Xu, Wei Liu, Tao Tan, Liujian Liujianfeng, Ang Li, Jian Luan, Bin Wang, Shuo Shang

AEQ-Bench: Measuring Empathy of Omni-Modal Large Models (2026.findings-acl)

Challenge:	Recent mobile AI agents based on VLMs lack basic mobile capabilities due to their pre-trained nature.
Approach:	They propose a mobile AI agent based on VLMs that includes additional pre-training stages to enhance both intra- and inter-UI understanding.
Outcome:	The proposed model outperforms existing VLMs on the Chinese mobile dataset Mobile3M .

Copied to clipboard

Xuan Luo, Lewei Yao, Lanqing Hong, Kai Chen, Dehua Tao, Daxin Tan, Yukun Deng, Ruifeng Xu, Jing Li

Monotonic Paraphrasing Improves Generalization of Language Model Prompting (2024.findings-emnlp)

Challenge:	Existing benchmarks focus on cognitive abilities, such as knowledge retrieval, complex reasoning, and instruction following, largely overlooking empathy evaluation.
Approach:	They propose to benchmark two core empathetic capabilities of omnimodal large models (OLMs) generating empatries by comprehending affective cues from multi-modal inputs and judging empathy of audio responses without relying on text transcription.
Outcome:	The proposed benchmark outperforms existing models with audio output capabilities but is unreliable for evaluating fine-grained paralinguistic expressiveness.

Copied to clipboard

Qin Liu, Fei Wang, Nan Xu, Tianyi Yan, Tao Meng, Muhao Chen

Re-Reading Improves Reasoning in Large Language Models (2024.emnlp-main)

Challenge:	Large language models (LLMs) have demonstrated remarkable proficiency in zero-shot decision making and instruction following.
Approach:	They propose an end-to-end decoding strategy that paraphrases given prompts or instructions into their lower perplexity counterparts based on an ensemble of a paraphrase LM for prompt rewriting, and a target LM that constrains the generation for lower perxity.
Outcome:	The proposed method can efficiently paraphrase the original prompt without altering its semantic meaning while decreasing the perplexity of each generation as calculated by the target LM.

Copied to clipboard

Xiaohan Xu, Chongyang Tao, Tao Shen, Can Xu, Hongbo Xu, Guodong Long, Jian-Guang Lou, Shuai Ma

PromDA: Prompt-based Data Augmentation for Low-Resource NLU Tasks (2022.acl-long)

Challenge:	Unlike thought-eliciting prompting methods, RE2 shifts the focus to the input by processing questions twice, thereby enhancing the understanding process.
Approach:	They introduce a simple, yet general and effective prompting method, RE2, which rereads the question as input.
Outcome:	The proposed method demonstrates strong generality and compatibility with most thought-eliciting prompting methods, including CoT.

Copied to clipboard

Yufei Wang, Can Xu, Qingfeng Sun, Huang Hu, Chongyang Tao, Xiubo Geng, Daxin Jiang

Low-Resource NLP

Challenge:	Existing approaches to build labeled training data from domain-specific data are expensive to obtain.
Approach:	They propose a Prompt-based Data Augmentation model which only trains small-scale Soft Promptes in frozen Pre-trained Language Models.
Outcome:	The proposed model outperforms several baseline models on four benchmarks and is complementary with unlabeled in-domain data.

Momoka-RAG: MCTS-Organized Mapping of Knowledge Associations for Long-Document Retrieval Augmented Generation (2026.findings-acl)

Copied to clipboard

Wenyu Tao, Xiaofen Xing, Zeliang Li, Xiangmin Xu

Information Retrieval

Challenge:	Existing frameworks that rely on fixed-length chunking are unsuitable for long-document tasks due to their passive and mechanical approach to knowledge structure.
Approach:	They propose a framework that utilizes Monte Carlo Tree Search to proactively uncover connections among chunks and construct optimal semantic information paths with the objective of completing semantic relationships.
Outcome:	The proposed framework achieves higher precision while maintaining competitive recall compared to other RAG frameworks.

Take Care of Your Prompt Bias! Investigating and Mitigating Prompt Bias in Factual Knowledge Extraction (2024.lrec-main)

Copied to clipboard

Ziyang Xu, Keqin Peng, Liang Ding, Dacheng Tao, Xiliang Lu

V-GameGym: Visual Game Generation for Code Large Language Models (2026.findings-acl)

Challenge:	Recent research shows that pre-trained language models suffer from “prompt bias” in factual knowledge extraction.
Approach:	They propose a representation-based approach to mitigate prompt bias during inference time by querying the model and removing it from its internal representations to generate debiased representations.
Outcome:	The proposed approach corrects the overfitted performance caused by prompt bias and significantly improves prompt retrieval capability.

Copied to clipboard

Wei Zhang, Jian Yang, Renshuai Tao, Linzheng Chai, Shuyue Guo, Jiajun Wu, Xiaoming Chen, Ganqu Cui, Ning Ding, Xander Xu, HU Wei, Bowen Zhou

Toward Optimal LLM Alignments Using Two-Player Games (2025.findings-emnlp)

Challenge:	Existing code-related benchmarks focus on single modality rather than visual game development.
Approach:	They propose a multimodal benchmark for evaluating code large language models in visual game generation that integrates a clustering-based curation methodology and a pipeline for visual code synthesis.
Outcome:	The proposed framework assesses code generation and visual game generation using a sandbox environment.

Copied to clipboard

Rui Zheng, Hongyi Guo, Zhihan Liu, Xiaoying Zhang, Yuanshun Yao, Xiaojun Xu, Zhaoran Wang, Zhiheng Xi, Tao Gui, Qi Zhang, Xuanjing Huang, Yang Liu, Hang Li

BERT-of-Theseus: Compressing BERT by Progressive Module Replacing (2020.emnlp-main)

Challenge:	Alignment of large language models (LLM) is a process that ensures the model’s responses to user prompts align with human intentions and social values.
Approach:	They propose an alignment method based on a two-agent game consisting of an adversarial agent and a defensive agent.
Outcome:	The proposed method improves on a two-agent game with an adversarial agent and a defensive agent.

Copied to clipboard

Canwen Xu, Wangchunshu Zhou, Tao Ge, Furu Wei, Ming Zhou

Beyond Error Propagation in Neural Machine Translation: Characteristics of Language Also Matter (D18-1)

Challenge:	a novel approach to compress neural networks by progressive module replacement is proposed . a number of techniques have been proposed to compress pretraining and fine-tuning models .
Approach:	They propose a model compression approach that divides BERT into modules and builds their compact substitutes.
Outcome:	The proposed approach outperforms existing knowledge distillation approaches on GLUE benchmark . it is based on a model that divides the original BERT into several modules and builds their substitutes .

Copied to clipboard

Lijun Wu, Xu Tan, Di He, Fei Tian, Tao Qin, Jianhuang Lai, Tie-Yan Liu

World Models with Hints of Large Language Models for Goal Achieving (2025.naacl-long)

Challenge:	Neural machine translation suffers from exposure bias and error propagation problem.
Approach:	They conduct a series of analyses to deeply understand the accuracy drop problem . they find that the left part of the translated sentence is often better than its right part .
Outcome:	The results show that the left part of the translated sentence is often better than its right part in left-to-right decoding models.

Copied to clipboard

Zeyuan Liu, Ziyu Huan, Xiyao Wang, Jiafei Lyu, Jian Tao, Xiu Li, Furong Huang, Huazhe Xu

Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting (2021.emnlp-main)

Challenge:	Existing methods address this by adding intrinsic rewards, but they fail to provide meaningful guidance in long-horizon decision-making tasks with large state and action spaces lacking purposeful exploration.
Approach:	They propose a multi-modal model-based RL approach that integrates the proposed hinting subgoals into the model rollouts to encourage goal discovery and reaching in challenging tasks.
Outcome:	The proposed model outperforms existing methods in challenging, sparse-reward environments such as HomeGrid, Crafter, and Minecraft by 41.8%, 21.1%, and 9.9%.

Copied to clipboard

Wangchunshu Zhou, Tao Ge, Canwen Xu, Ke Xu, Furu Wei

TegTok: Augmenting Text Generation via Task-specific and Open-world Knowledge (2022.findings-acl)

Challenge:	Existing text infilling objectives for pretrained language models require self-supervision by masking out tokens or spans in text.
Approach:	They propose to extend text infilling to a self-supervised sequence-to-sequence (Seq2Sequen) task.
Outcome:	The proposed task improves the model's performance on various natural language generation tasks.

Copied to clipboard

Chao-Hong Tan, Jia-Chen Gu, Chongyang Tao, Zhen-Hua Ling, Can Xu, Huang Hu, Xiubo Geng, Daxin Jiang

Neighbors Are Not Strangers: Improving Non-Autoregressive Translation under Low-Frequency Lexical Constraints (2022.naacl-main)

Challenge:	Generating natural and informative texts has been a long-standing problem in NLP.
Approach:	They propose to augment TExt Generation via Task-specific and Open-world Knowledge in a unified framework.
Outcome:	The proposed model can learn what and how to generate on two text generation tasks.

Copied to clipboard

Chun Zeng, Jiangjie Chen, Tianyi Zhuang, Rui Xu, Hao Yang, Qin Ying, Shimin Tao, Yanghua Xiao

Machine Translation

Challenge:	Existing approaches to lexically constrained neural machine translation suffer from high latency.
Approach:	They propose a plug-in algorithm for non-autoregressive translation for this problem . they propose ACT to familiarize the model with the source-side context of constraints .
Outcome:	The proposed model improves over the backbone constrained NAT model in constraint preservation and translation quality, especially for rare constraints.

Playing 20 Question Game with Policy-Based Reinforcement Learning (D18-1)

Copied to clipboard

Huang Hu, Xianchao Wu, Bingfeng Luo, Chongyang Tao, Can Xu, Wei Wu, Zhan Chen

Natural Language Interfaces

Challenge:	The 20 Questions (Q20) game encourages deductive reasoning and creativity.
Approach:	They propose a policy-based Reinforcement Learning method which learns optimal question selection . the method is robust to noisy answers and uses a reward network to estimate the more informative reward .
Outcome:	The proposed method outperforms an entropy-based engineering system and has competitive performance in noisy-free simulation environment.

ZeroGen: Efficient Zero-shot Learning via Dataset Generation (2022.emnlp-main)

Copied to clipboard

Jiacheng Ye, Jiahui Gao, Qintong Li, Hang Xu, Jiangtao Feng, Zhiyong Wu, Tao Yu, Lingpeng Kong

SAKI-RAG: Mitigating Context Fragmentation in Long-Document RAG via Sentence-level Attention Knowledge Integration (2025.emnlp-main)

Challenge:	Existing approaches to generate training data with pre-trained language models have been found effective in various scenarios.
Approach:	They propose an unsupervised zero-shot learning method that generates a dataset from scratch and trains a tiny task model under supervision of the synthesized dataset.
Outcome:	The proposed method is annotated-free and efficient, but can provide useful insights from the perspective of data-free model-agnostic knowledge distillation and unreferenced text generation evaluation.

Copied to clipboard

Wenyu Tao, Xiaofen Xing, Zeliang Li, Xiangmin Xu

Chunking

Challenge:	Traditional Retrieval-Augmented Generation (RAG) frameworks segment documents into larger chunks to preserve contextual coherence . however, such chunking methods lead to fragmented contexts, isolated chunk semantics, and broken inter-chunk relationships .
Approach:	They propose a framework that maintains granular chunks while recovering their intrinsic semantic connections.
Outcome:	The proposed framework achieves better recall and precision compared to other RAG frameworks in long-document retrieval scenarios.

Double Path Networks for Sequence to Sequence Learning (C18-1)

Copied to clipboard

Kaitao Song, Xu Tan, Di He, Jianfeng Lu, Tao Qin, Tie-Yan Liu

Scheduled DropHead: A Regularization Method for Transformer Models (2020.findings-emnlp)

Challenge:	Existing approaches for Sequence to Sequence learning have been developed . convolutional neural networks and self-attention networks are the most popular .
Approach:	They propose to integrate convolutional and self-attention layers into a double path network for sequence to sequence learning.
Outcome:	The proposed method significantly improves performance over state-of-the-art systems.

Copied to clipboard

Wangchunshu Zhou, Tao Ge, Furu Wei, Ming Zhou, Ke Xu

FOCUS: Forging Originality through Contrastive Use in Self-Plagiarism for Language Models (2024.findings-acl)

Challenge:	DropHead is a structured dropout method for regularizing multi-head attention . DropHed drops entire attention heads during training to prevent overfitting .
Approach:	They propose a structured dropout method specifically designed for regularizing multi-head attention mechanism . DropHead drops entire attention heads during training to prevent overfitting .
Outcome:	The proposed method can improve transformer models by 0.9 BLEU score on translation task and around 1.0 accuracy for various text classification tasks.

Copied to clipboard

Kaixin Lan, Tao Fang, Derek Wong, Yabo Xu, Lidia Chao, Cecilia Zhao

A Study of Syntactic Multi-Modality in Non-Autoregressive Machine Translation (2022.naacl-main)

Challenge:	Existing methods to generate original text using pre-trained language models are problematic as they are trained on corpora constructed by human authors.
Approach:	They propose a unique “self-plagiarism” contrastive decoding strategy that modifies prompts in LLMs to develop an amateur model and a professional model.
Outcome:	The proposed method enables the development of an amateur model and a professional model while maintaining its standard language model status.

Copied to clipboard

Kexun Zhang, Rui Wang, Xu Tan, Junliang Guo, Yi Ren, Tao Qin, Tie-Yan Liu

Multilinguality

Challenge:	Non-autoregressive translation models suffer from the multi-modality problem when a source sentence corresponds to multiple correct translations.
Approach:	They propose to decompose the syntactic multi-modality problem into short- and long-range models and evaluate them on synthesized and real datasets.
Outcome:	The proposed loss functions can handle short- and long-range syntactic multi-modalities better than existing models.

CCPrefix: Counterfactual Contrastive Prefix-Tuning for Many-Class Classification (2024.eacl-long)

Copied to clipboard

Yang Li, Canran Xu, Guodong Long, Tao Shen, Chongyang Tao, Jing Jiang

Multilingual Neural Machine Translation with Language Clustering (D19-1)

Challenge:	Existing methods to improve pre-trained language models for many-class classification suffer from verbalizer ambiguity . a significant disparity exists between the pre-training and fine-tuning stages of the model .
Approach:	They propose a method to tune pre-trained language models to a broad spectrum of tasks . they use an instance-dependent soft prefix to complement language verbalizers in many-class classification .
Outcome:	The proposed method outperforms baselines on many-class datasets.

Copied to clipboard

Xu Tan, Jiale Chen, Di He, Yingce Xia, Tao Qin, Tie-Yan Liu

Multilinguality

Challenge:	Existing work on multilingual neural machine translation has been neglected due to its burdensome training process.
Approach:	They develop a framework that clusters languages into different groups and trains one multilingual model for each cluster.
Outcome:	The proposed model reduces the cost of training and improves translation accuracy.

Maria: A Visual Experience Powered Conversational Agent (2021.acl-long)

Copied to clipboard

Zujie Liang, Huang Hu, Can Xu, Chongyang Tao, Xiubo Geng, Yining Chen, Fan Liang, Daxin Jiang

MPC-BERT: A Pre-Trained Language Model for Multi-Party Conversation Understanding (2021.acl-long)

Challenge:	Existing studies focus on grounding conversational agents on text-only corpora, but they lack the perception ability to our physical world.
Approach:	They propose to ground conversational agents on images retrieved from large-scale image indexes . they propose to use visual knowledge to generate informative responses based on the extracted knowledge .
Outcome:	The proposed agent outperforms state-of-the-art methods on automatic metrics and human evaluation.

Copied to clipboard

Jia-Chen Gu, Chongyang Tao, Zhenhua Ling, Can Xu, Xiubo Geng, Daxin Jiang

Structured Self-Supervised Pretraining for Commonsense Knowledge Graph Completion (2021.tacl-1)

Challenge:	Existing models for multi-party conversation represent interlocutors and utterances individually . existing methods ignore complicated structure of MPC which may provide crucial interlocutor and tertiary semantics.
Approach:	They propose a pre-trained model for multi-party conversation that considers learning who says what to whom in a unified model with elaborated self-supervised tasks.
Outcome:	The proposed model outperforms existing models on three downstream tasks at two benchmarks.

Copied to clipboard

Jiayuan Huang, Yangkai Du, Shuting Tao, Kun Xu, Pengtao Xie

Towards Robust Ranker for Text Retrieval (2023.findings-acl)

Challenge:	Existing approaches focus on generating concepts that have direct and obvious relationships with existing concepts and lack an ability to generate unobvious concepts.
Approach:	They propose a general graph-to-paths pretraining framework that leverages high-order structures in CKGs to capture high-level relationships between concepts.
Outcome:	The proposed framework can capture high-order relationships between concepts in four special cases: long path, path-to-path, router, and graph-node-path.

Copied to clipboard

Yucheng Zhou, Tao Shen, Xiubo Geng, Chongyang Tao, Can Xu, Guodong Long, Binxing Jiao, Daxin Jiang

Multi-Grained Knowledge Distillation for Named Entity Recognition (2021.naacl-main)

Challenge:	Existing methods for text retrieval are based on a 'retrieval & rerank' pipeline, which uses a fast retriever to fetch a set of top document candidates, while a robust ranker is based upon a weak negative mining during contrastive learning.
Approach:	They propose a multi-adversarial training strategy that leverages multiple retrievers as generators to challenge a ranker.
Outcome:	The proposed model outperforms the existing de facto ranker training paradigms on the passage retrieval benchmarks using BM25-reranking, full-ranking and retriever distillation.

Copied to clipboard

Xuan Zhou, Xiao Zhang, Chenyang Tao, Junya Chen, Bing Xu, Wei Wang, Jing Xiao

Green & Sustainable NLP

Challenge:	Pre-trained big models have delivered top performance in Seq2seq modeling, but their deployments in real-world applications are often hindered by excessive computations and memory demands.
Approach:	They propose a distillation scheme to efficiently transfer knowledge from big models to their cheaper counterparts.
Outcome:	The proposed scheme maximizes the assimilation of knowledge from the teacher model to the student model.

OS Agents: A Survey on MLLM-based Agents for Computer, Phone and Browser Use (2025.acl-long)

Copied to clipboard

Xueyu Hu, Tao Xiong, Biao Yi, Zishu Wei, Ruixuan Xiao, Yurun Chen, Jiasheng Ye, Meiling Tao, Xiangxin Zhou, Ziyu Zhao, Yuhuai Li, Shengze Xu, Shenzhi Wang, Xinchen Xu, Shuofei Qiao, Zhaokai Wang, Kun Kuang, Tieyong Zeng, Liang Wang, Jiwei Li, Yuchen Eleanor Jiang, Wangchunshu Zhou, Guoyin Wang, Keting Yin, Zhou Zhao, Hongxia Yang, Fan Wu, Shengyu Zhang, Fei Wu

Feedback-Driven Tool-Use Improvements in Large Language Models via Automated Build Environments (2026.findings-acl)

Challenge:	a new generation of (M)LLMs is enabling the creation of superintelligent AI assistants . OS Agents can complete tasks autonomously and have the potential to significantly enhance the lives of billions of users worldwide.
Approach:	They propose to build OS Agents that operate within operating systems' GUIs and GUIs . they examine evaluation metrics and benchmarks to identify promising directions .
Outcome:	The proposed agents are based on operating systems (OS) and operating systems frameworks.

Copied to clipboard

Junjie Ye, Changhao Jiang, Zhengyin Du, Yufei Xu, Xuesong Yao, Zhiheng Xi, Xiaoran Fan, Qi Zhang, Tao Gui, Xuanjing Huang, Jiecao Chen

MusicBERT: Symbolic Music Understanding with Large-Scale Pre-Training (2021.findings-acl)

Challenge:	Currently, there are no efficient reinforcement learning (RL) frameworks specifically designed for tool use.
Approach:	They propose an automated environment construction pipeline that incorporates scenario decomposition, document generation, function integration, complexity scaling, and localized deployment to enable high-quality training environments without external tools.
Outcome:	The proposed framework significantly improves the models’ tool-use performance without degrading their general capabilities.

Copied to clipboard

Mingliang Zeng, Xu Tan, Rui Wang, Zeqian Ju, Tao Qin, Tie-Yan Liu

UnihanLM: Coarse-to-Fine Chinese-Japanese Language Model Pretraining with the Unihan Database (2020.aacl-main)

Challenge:	Symbolic music understanding is useful for many music applications, but lack of training data hinders representation learning.
Approach:	They propose a pre-trained model for music understanding that uses symbolic music data to train music representations.
Outcome:	The proposed model improves on four music understanding tasks.

Copied to clipboard

Canwen Xu, Tao Ge, Chenliang Li, Furu Wei

ToolHop: A Query-Driven Benchmark for Evaluating Large Language Models in Multi-Hop Tool Use (2025.acl-long)

Challenge:	Chinese and Japanese share many characters with similar surface morphology.
Approach:	They propose a Chinese-Japanese pretrained masked language model with a coarse-to-fine training approach to exploit the shared knowledge across the languages.
Outcome:	The proposed model is effective on mono- and cross-lingual Chinese and Japanese tasks.

Copied to clipboard

Junjie Ye, Zhengyin Du, Xuesong Yao, Weijian Lin, Yufei Xu, Zehui Chen, Zaiyuan Wang, Sining Zhu, Zhiheng Xi, Siyu Yuan, Tao Gui, Qi Zhang, Xuanjing Huang, Jiecao Chen

ProphetChat: Enhancing Dialogue Generation with Simulation of Future Conversation (2022.acl-long)

Challenge:	Effective evaluation of multi-hop tool use is critical for analyzing the understanding, reasoning, and function-calling capabilities of large language models.
Approach:	They propose a dataset that provides rigorous evaluation of multi-hop tool use.
Outcome:	The proposed model achieves 49.04% accuracy across five model families.

Copied to clipboard

Chang Liu, Xu Tan, Chongyang Tao, Zhenxin Fu, Dongyan Zhao, Tie-Yan Liu, Rui Yan

Batch-ICL: Effective, Efficient, and Order-Agnostic In-Context Learning (2024.findings-acl)

Challenge:	generative dialogue models use dialogue histories to generate the response . however, generating a response based on the historical information is not easy .
Approach:	They propose a framework that utilizes simulated dialogue futures to enhance response generation.
Outcome:	The proposed framework can generate better responses over strong baselines on two open-domain dialogue datasets.

Copied to clipboard

Kaiyi Zhang, Ang Lv, Yuhan Chen, Hansen Ha, Tao Xu, Rui Yan

Cross-Linguistic Syntactic Difference in Multilingual BERT: How Good is It and How Does It Affect Transfer? (2022.emnlp-main)

Challenge:	In-context learning (ICL) is a meta-optimization process that affects performance . we develop a batch-based inference algorithm that is order-agnostic to ICL examples .
Approach:	They develop an order-agnostic inference algorithm that aggregates ICL examples in batches . they find it outperforms most permutations of ICL, and it even exceeds the best order .
Outcome:	The proposed method outperforms standard ICL examples while reducing computational resources.

Copied to clipboard

Ningyu Xu, Tao Gui, Ruotian Ma, Qi Zhang, Jingting Ye, Menghan Zhang, Xuanjing Huang

Extract and Attend: Improving Entity Translation in Neural Machine Translation (2023.findings-acl)

Challenge:	Multilingual BERT (mBERT) has demonstrated considerable cross-lingual syntactic ability, but it is not well understood what leads to this variation and whether it fairly reflects difference between languages.
Approach:	They propose to use multilingual BERT to enable zero-shot cross-lingual transfer of syntactic knowledge between different languages by generating grammatical relations in 24 different languages.
Outcome:	The results show that the distance between the distributions of different languages is highly consistent with the syntactic difference in terms of linguistic formalisms.

Copied to clipboard

Zixin Zeng, Rui Wang, Yichong Leng, Junliang Guo, Shufang Xie, Xu Tan, Tao Qin, Tie-Yan Liu

Machine Translation

Challenge:	Existing methods to improve entity translation in Neural machine translation still suffer from inaccurate translation of entities due to the lack of entity training instances.
Approach:	They propose an extract-and-tend approach to enhance entity translation in NMT by extracting entities from a dictionary and attending to them with a prefix.
Outcome:	Experiments on En-Zh and En-Ru show that the proposed approach improves translation accuracy and translation quality.

BERT-based Lexical Substitution (P19-1)

Copied to clipboard

Wangchunshu Zhou, Tao Ge, Ke Xu, Furu Wei, Ming Zhou

Breaking the Impasse: Dual-Scale Evolutionary Policy Training for Social Language Agents (2026.acl-long)

Challenge:	Existing approaches to lexical substitution tend to overlook good substitute candidates that are not the synonyms of the target words in the lexicals and fail to take into account the substitution’s influence on the global context of the sentence.
Approach:	They propose an end-to-end BERT-based lexical substitution approach which proposes and validates substitute candidates without using annotated data or manually curated resources.
Outcome:	The proposed approach performs well in proposing and ranking substitute candidates, achieving the state-of-the-art results in both LS07 and LS14 benchmarks.

Copied to clipboard

Minzheng Wang, Run Luo, Yanbo Wang, Zichen Liu, Yuqiao Tan, Tao Tan, Nan Xu, Lu Wang, Wenji Mao

QCRD: Quality-guided Contrastive Rationale Distillation for Large Language Models (2025.emnlp-main)

Challenge:	Reinforcement Learning with Verifiable Rewards (RLVR) is effective for closed-ended tasks, but it is not applicable to open-ended social language games.
Approach:	They propose a method that uses a time-scaled evolutionary perception mechanism to detect impasse by quantifying dual-scale value baseline divergence alongside match entropy.
Outcome:	Experiments on multiple social language games show that the proposed method outperforms baselines and avoids policy degeneration.

Copied to clipboard

Wei Wang, Zhaowei Li, Qi Xu, YiQing Cai, Hang Song, Qi Qi, Ran Zhou, Zhida Huang, Tao Wang, Li Xiao

One2Set: Generating Diverse Keyphrases as a Set (2021.acl-long)

Challenge:	Recent research has focused on smaller, task-specific models enhanced by distilling knowledge from LLMs, but the diversity and quality of negative knowledge remains understudied.
Approach:	They propose a quality-guided contrastive rationale distillation framework that aims to enhance reasoning capabilities through contrastive knowledge learning.
Outcome:	The proposed method consistently outperforms existing distillation techniques yielding higher-quality rationales.

Copied to clipboard

Jiacheng Ye, Tao Gui, Yichao Luo, Yige Xu, Qi Zhang

Challenge:	Recent keyphrase generation models are wrongly imposing a predefined order on keyphrases . a new training paradigm is proposed to concatenate keyphrase sequences in parallel .
Approach:	They propose a training paradigm that concatenates keyphrases in a predefined order . they propose combining a fixed set of learned control codes with a bipartite matching mechanism .
Outcome:	The proposed model outperforms the state-of-the-art methods on multiple benchmarks.

Pseudo-Bidirectional Decoding for Local Sequence Transduction (2020.findings-emnlp)

Copied to clipboard

Wangchunshu Zhou, Tao Ge, Ke Xu

MathFlow: Enhancing the Perceptual Flow of MLLMs for Visual Mathematical Problems (2026.acl-long)

Challenge:	Local sequence transduction tasks involve massive overlapping between source and target sequences . experimental results show that Pseudo-Bidirectional Decoding improves performance of standard seq2seq models.
Approach:	They propose a simple but versatile approach for local sequence transduction tasks . they propose to copy source tokens to decoder as pseudo future context .
Outcome:	The proposed approach improves the performance of standard seq2seq models on LST tasks.

Copied to clipboard

Shuhang Chen, Hangjie Yuan, Yunqiu Xu, Pengwei Liu, Tao Feng, Jun Cen, Zeying Huang, Yi Yang

Reasoning

Challenge:	Existing multimodal large language models (MLLMs) exhibit significant limitations when extracting essential information and reasoned properties from diagrams and performing complex reasoning based on these visual inputs.
Approach:	They propose a benchmark that provides a fine-grained evaluation of MLLMs’ perception and reasoning capabilities.
Outcome:	The proposed benchmark shows that existing MLLMs exhibit limitations when extracting essential information and reasoned properties from diagrams and performing complex reasoning based on these visual inputs.

ProSparse: Introducing and Enhancing Intrinsic Activation Sparsity within Large Language Models (2025.coling-main)

Copied to clipboard

Chenyang Song, Xu Han, Zhengyan Zhang, Shengding Hu, Xiyu Shi, Kuai Li, Chen Chen, Zhiyuan Liu, Guangli Li, Tao Yang, Maosong Sun