Papers by Tao Ma

TextMixer: Mixing Multiple Inputs for Privacy-Preserving Inference (2023.findings-emnlp)

Challenge:	Existing speech codecs struggle to balance high-quality reconstruction with semantically rich representations, limiting their effectiveness in both generative and understanding tasks.
Approach:	They propose a neural speech codec with semantic-acoustic dual-stream quantization that disentangles semantic and acousian modeling into two dedicated streams.
Outcome:	The proposed codec outperforms state-of-the-art speech tokenizers in auto-propagating text-to-speech models.

Copied to clipboard

Xin Zhou, Yi Lu, Ruotian Ma, Tao Gui, Qi Zhang, Xuanjing Huang

PPC-GPT: Federated Task-Specific Compression of Large Language Models via Pruning and Chain-of-Thought Distillation (2025.emnlp-main)

Challenge:	Pre-trained language models (PLMs) are often deployed as cloud services, enabling users to upload textual data and perform inference remotely.
Approach:	They propose a privacy-preserving inference framework called MixPi which aims to obfuscate a user's private input by mixing it with multiple other inputs.
Outcome:	The proposed framework surpasses existing privacy-preserving methods on token and sentence classification tasks.

Copied to clipboard

Tao Fan, Guoqiang Ma, Yuanfeng Song, Lixin Fan, Qiang Yang

LoopCoder: Scaling Code Intelligence via Looped Language Models (2026.findings-acl)

Challenge:	Large Language Models (LLMs) are becoming more popular and are gaining widespread use in artificial intelligence.
Approach:	They propose a unified framework that addresses both privacy preservation and model compression in federated settings.
Outcome:	The proposed framework maintains competitive performance comparable to full-sized LLMs while ensuring robust privacy protection through its federated architecture.

Copied to clipboard

Jian Yang, Wei Zhang, Shuyue Guo, Yizhi LI, Linzheng Chai, Zhengmao Ye, Shukai Liu, Yuyang Song, Jiajun Wu, Che Liu, Tianyu Zheng, Siwei Wu, Leo L, Xudong Ma, Chuan Hao, Ran Tao, Yan Xing, Jianzhou Wang, Mingjie Tang, Aishan Liu, Zhoujun Li, Xianglong Liu, Weifeng Lv, Bryan Dai

Unlocking the Potential of Model Merging for Low-Resource Languages (2024.findings-emnlp)

Challenge:	Large language models have mastered syntax-level code generation, but complex algorithmic reasoning remains a challenge.
Approach:	They propose a recurrent inductive bias that aligns with the recursive nature of programming logic.
Outcome:	The proposed model achieves comparable performance to standard dense models with more parameters.

Copied to clipboard

Mingxu Tao, Chen Zhang, Quzhe Huang, Tianyao Ma, Songfang Huang, Dongyan Zhao, Yansong Feng

MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing (2026.acl-industry)

Challenge:	Adapting large language models (LLMs) to new languages requires continual pre-training followed by supervised fine-tuning.
Approach:	They propose a model merging solution that integrates LLMs with distinct capabilities into a single model without additional training.
Outcome:	The proposed model merging outperforms CT-then-SFT in low-resource languages with scarce data.

Copied to clipboard

Junbo Niu, Zheng Liu, Zhuangcheng Gu, Bin Wang, Linke Ouyang, Zhiyuan Zhao, Tao Chu, Tianyao He, Fan Wu, Qintong Zhang, Zhenjiang Jin, Guang Liang, Rui Zhang, Wenzheng Zhang, Yuan Qu, Zhifei Ren, Yuefeng Sun, Zirui Tang, Boyu Niu, Yuanhong Zheng, Dongsheng Ma, Ziyang Miao, Hejun Dong, Siyi Qian, Junyuan Zhang, Fangdong Wang, Jingzhou Chen, Xiaomeng Zhao, Liqun Wei, Wei Li, Shasha Wang, RuiLiang Xu, Yuanyuan Cao, Lu Chen, Qianqian Wu, Huaiyu Gu, Lindong Lu, Dechen Lin, null Shenguanlin, Xuanhe Zhou, Linfeng Zhang, Yuhang Zang, Xiaoyi Dong, Jiaqi Wang, Bo Zhang, Lei Bai, Pei Chu, Weijia Li, Jiang Wu, Lijun Wu, Zhenxiang Li, Guangyu Wang, Zhongying Tu, Chao Xu, Kai Chen, Bowen Zhou, Dahua Lin, Wentao Zhang, Conghui He

Green & Sustainable NLP

Challenge:	Document images are characterized by higher resolutions, denser content, and more complex structural layouts.
Approach:	They propose a 1.2B-parameter document parsing vision-language model that decouples layout analysis from local content recognition.
Outcome:	The proposed model surpasses general-purpose and domain-specific models on multiple benchmarks while maintaining significantly lower computational overhead.

A Confidence-based Partial Label Learning Model for Crowd-Annotated Named Entity Recognition (2023.findings-acl)

Copied to clipboard

Limao Xiong, Jie Zhou, Qunxi Zhu, Xiao Wang, Yuanbin Wu, Qi Zhang, Tao Gui, Xuanjing Huang, Jin Ma, Ying Shan

Graph Reasoning Paradigm: Structured and Symbolic Reasoning with Topology-Aware Reinforcement Learning for Large Language Models (2026.acl-long)

Challenge:	Existing models for named entity recognition (NER) are based on large-scale labeled datasets, which always obtain using crowdsourcing.
Approach:	They propose a CONfidence-based partial Label Learning method to integrate prior and posterior confidences for crowd-annotated named entity recognition models.
Outcome:	The proposed model improves on real-world and synthetic datasets compared with baselines.

Copied to clipboard

Runxuan Liu, Xianhao Ou, Xinyan Ma, Jiyuan Wang, Jiafeng Liang, Jiaqi Li, Tao He, Zheng Chu, Rongchuan Mu, Zekun Wang, Baoxin Wang, Dayong Wu, Ming Liu, Shijin Wang, Guoping Hu, Bing Qin

Unveiling and Consulting Core Experts in Retrieval-Augmented MoE-based LLMs (2024.emnlp-main)

Challenge:	Existing methods for long chain-of-thought (LCoT) are coarse-grained, reward hacking, and poor generalization.
Approach:	They propose a Long Chain-of-Thought (LCoT) model that integrates reinforcement learning with verifiable rewards with a process-aware verification approach.
Outcome:	The proposed model improves reasoning and code generation tasks while reducing the cost of training and performance bottlenecks.

Copied to clipboard

Xin Zhou, Ping Nie, Yiwen Guo, Haojie Wei, Zhanqiu Zhang, Pasquale Minervini, Ruotian Ma, Tao Gui, Qi Zhang, Xuanjing Huang

Information Retrieval

Challenge:	Existing research seeks to enhance RAG performance by retrieving higher-quality documents or designing RAG-specific LLMs, but internal mechanisms that contribute to RAG’s effectiveness remain underexplored.
Approach:	They propose to examine the internal mechanisms within the popular Mixture-of-Expert (MoE)-based LLMs and examine their ability to improve RAG by examining expert activations.
Outcome:	The proposed method significantly improved the ability of Large Language Models (LLMs) to solve knowledge-intensive tasks.

PFDial: A Structured Dialogue Instruction Fine-tuning Method Based on UML Flowcharts (2025.findings-acl)

Copied to clipboard

Ming Zhang, Yuhui Wang, Yujiong Shen, Tingyi Yang, Changhao Jiang, Yilong Wu, Shihan Dou, Qinhao Chen, Zhiheng Xi, Zhihao Zhang, Yi Dong, Zhen Wang, Zhihui Fei, Mingyang Wan, Tao Liang, Guojun Ma, Qi Zhang, Tao Gui, Xuanjing Huang

Dialogue Systems & Conversational Agents

Challenge:	Large Language Models (LLMs) have shown remarkable progress in dialogue and reasoning, but they struggle to solve strictly constrained dialogue tasks.
Approach:	They construct a dataset that contains 12,705 high-quality Chinese dialogue instructions from 440 flowcharts containing 5,055 process nodes.
Outcome:	The proposed model outperforms GPT-4o models on backward transitions and outperformed GPT-42 models on the same dataset.

FedCoT: Federated Chain-of-Thought Distillation for Large Language Models (2025.findings-emnlp)

Copied to clipboard

Tao Fan, Weijing Chen, Yan Kang, Guoqiang Ma, Hanlin Gu, Yuanfeng Song, Lixin Fan, Qiang Yang

FACTrial: Factorized Clinical Contrastive Training for Scalable Patient-Trial Retrieval (2026.acl-long)

Challenge:	Large Language Models (LLMs) have emerged as a transformative force in artificial intelligence, demonstrating exceptional proficiency across various tasks.
Approach:	They propose a federated framework for the Chain-of-Thought distillation of knowledge from LLMs to SLMs, while adhering to privacy requirements.
Outcome:	The proposed framework ensures secure knowledge transfer from an LLM on a high-powered server to an SLM on resource-constrained client while adhering to privacy requirements.

Copied to clipboard

Xuanren Chen, Chongyang Tao, Tao Shen, Shuai Ma

Information Retrieval

Challenge:	Existing approaches to patient–trial retrieval rely on generic semantic matching and zero-shot transfer.
Approach:	They propose a factorized contrastive training framework that synthesizes diagnosis-aware supervision for scalable patient–trial retrieval.
Outcome:	Experiments show that the proposed framework improves quality and recall coverage.

Coarse-to-fine Few-shot Learning for Named Entity Recognition (2023.findings-acl)

Copied to clipboard

Ruotian Ma, Zhang Lin, Xuanting Chen, Xin Zhou, Junzhe Wang, Tao Gui, Qi Zhang, Xiang Gao, Yun Wen Chen

Group-SAE: Efficient Training of Sparse Autoencoders for Large Language Models via Layer Groups (2025.emnlp-main)

Challenge:	Existing few-shot NER solutions do not consider sub-class discrimination and various granularity of new classes during coarse training.
Approach:	They propose a method that uses a cluster-based prototype loss to learn group-wise discriminative representations of coarse-grained classes and a mixture prototype loss for learning the representations.
Outcome:	The proposed method shows superior performance over baseline methods on in-domain and cross-domain settings with various target granularity.

Copied to clipboard

Davide Ghilardi, Federico Belotti, Marco Molinari, Tao Ma, Matteo Palmonari

Text-Guided Multi-Scale Frequency Representation Adaptation (2026.acl-long)

Challenge:	Sparse Autoencoders (SAEs) are a promising unsupervised approach for understanding the representations of layers of Large Language Models (LLMs).
Approach:	They propose a method that groups similar models and trains a single SAE per group based on representational similarity across layers.
Outcome:	Experiments on Pythia family models show that the proposed method significantly accelerates training with minimal impact on reconstruction quality and comparable downstream task performance and interpretability over baseline SAEs trained layer by layer.

Copied to clipboard

Weicai Yan, Xinhua Ma, Wang Lin, Tao Jin

Making Parameter-efficient Tuning More Efficient: A Unified Framework for Classification Tasks (2022.coling-1)

Challenge:	Existing methods for fine-tuning visual signals are limited by their size and complexity.
Approach:	They propose a multi-scale frequency-based fine-tuning method that integrates textual information and performs multi-level fine- tuning of visual signals in the frequency domain.
Outcome:	Extensive experiments on multimodal models, including CLIP and LLaVA, demonstrate that the proposed method significantly improves performance and efficiency with minimal cost and fast convergence within one epoch.

Copied to clipboard

Xin Zhou, Ruotian Ma, Yicheng Zou, Xuanting Chen, Tao Gui, Qi Zhang, Xuanjing Huang, Rui Xie, Wei Wu

Lost in Pronunciation: Detecting Chinese Offensive Language Disguised by Phonetic Cloaking Replacement (2025.emnlp-industry)

Challenge:	Large pre-trained language models (PLMs) have demonstrated superior performance in industrial applications.
Approach:	They propose a framework that re-uses existing parameter-efficient methods with a unified classifier.
Outcome:	The proposed framework improves the efficiency of existing parameter-efficient methods with a unified classifier.

Copied to clipboard

Haotan Guo, Jianfei He, Jiayuan Ma, Hongbin Na, Zimu Wang, Haiyang Zhang, Qi Chen, Wei Wang, Zijing Shi, Tao Shen, Ling Chen

Unlocking Data-free Low-bit Quantization with Matrix Decomposition for KV Cache Compression (2024.acl-long)

Challenge:	Phonetic Cloaking Replacement (PCR) is a problem in content moderation in China.
Approach:	They organize PCR into a four-way surface-form taxonomy and compile PCR-ToxiCN, a dataset of 500 phonetically cloaked offensive posts gathered from the RedNote platform.
Outcome:	The proposed model achieves only an F1-score and zero-shot chain-of-thought prompting pushes performance even lower.

Copied to clipboard

Peiyu Liu, Ze-Feng Gao, Xin Zhao, Yipeng Ma, Tao Wang, Ji-Rong Wen

Green & Sustainable NLP

Challenge:	Existing methods to compress KV cache compromise precision or require extra data for calibration, limiting their practicality in LLM deployment.
Approach:	They propose a low-bit quantization technique based on tensor decomposition to effectively compress KV cache.
Outcome:	The proposed method reduces memory footprint and performance by 75% . it is compared with existing methods that compromise precision or require extra data for calibration .

C3: A Bilingual Benchmark for Spoken Dialogue Models Exploring Challenges in Complex Conversations (2025.emnlp-main)

Copied to clipboard

Chengqian Ma, Wei Tao, Steven Y. Guo

Speech & Audio in NLP

Challenge:	Recent developments in spoken dialogue models have created a gap in understanding their effectiveness in comprehending and emulating human conversations.
Approach:	They present a benchmark dataset which comprises 1,079 instances in English and Chinese to examine their effectiveness in emulating human conversations.
Outcome:	The proposed model outperforms existing models in English and Chinese by using an LLM-based evaluation method that closely aligns with human judgment.

Unsupervised Text Style Transfer for Controllable Intensity (2026.findings-eacl)

Copied to clipboard

Shuhuan Gu, Wenbiao Tao, Xinchen Ma, Kangkang He, Ye Guo, Xiang Li, Yunshi Lan

Low-Resource NLP

Challenge:	Unsupervised Text Style Transfer (UTST) aims to transfer the stylistic properties of a given text without parallel text pairs.
Approach:	They propose a SFT-then-PPO paradigm to fine-tune an LLM with parallel data and reward functions for distinguishing stylistic intensity in hierarchical levels.
Outcome:	The proposed system can transfer stylistic properties without parallel text pairs even for adjacent levels of intensity.

ProcWorld: Benchmarking Large Model Planning in Reachability-Constrained Environments (2025.emnlp-main)

Copied to clipboard

Dong Wang, Xinghang Li, Zhengshen Zhang, Jirong Liu, Xiao Ma, Hanbo Zhang, Tao Kong, Huaping Liu

ChatMusician: Understanding and Generating Music Intrinsically with LLM (2024.findings-acl)

Challenge:	Existing benchmarks for embodied spatial reasoning and long-term planning are non-trivial due to the combinatorial complexity of long-horizon abstract reasoning.
Approach:	They propose a large-scale benchmark for partially observable embodied spatial reasoning and long-term planning with large language models and vision language models.
Outcome:	The proposed model performs better in 16 task types, 5,000 rooms, and over 10 million evaluation trajectories with diverse data distribution.

Copied to clipboard

Ruibin Yuan, Hanfeng Lin, Yi Wang, Zeyue Tian, Shangda Wu, Tianhao Shen, Ge Zhang, Yuhang Wu, Cong Liu, Ziya Zhou, Liumeng Xue, Ziyang Ma, Qin Liu, Tianyu Zheng, Yizhi Li, Yinghao Ma, Yiming Liang, Xiaowei Chi, Ruibo Liu, Zili Wang, Chenghua Lin, Qifeng Liu, Tao Jiang, Wenhao Huang, Wenhu Chen, Jie Fu, Emmanouil Benetos, Gus Xia, Roger Dannenberg, Wei Xue, Shiyin Kang, Yike Guo

Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task (D18-1)

Challenge:	Despite LLMs' impressive capabilities in musical knowledge, music reasoning remains an unsolved task.
Approach:	They propose an open-source large language model (LLM) that integrates intrinsic musical abilities into LLaMA2 and GPT-3.5.
Outcome:	The proposed model can understand and generate music with a pure text tokenizer without external multi-modal neural structures or tokenizers.

Copied to clipboard

Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang, Dragomir Radev

Programming Languages in NLP

Challenge:	Existing datasets for semantic parsing are too small in terms of number of programs for training modern data-intensive models.
Approach:	They propose a large-scale complex and cross-domain semantic parsing task for a database . they use a dataset with 10,181 questions and 5,693 unique complex SQL queries .
Outcome:	The proposed task is different from previous tasks because it uses the same database and program . the best model achieves only 9.7% exact matching accuracy on a database split setting.

Characterizing the Impacts of Instances on Robustness (2023.findings-acl)

Copied to clipboard

Rui Zheng, Zhiheng Xi, Qin Liu, Wenbin Lai, Tao Gui, Qi Zhang, Xuanjing Huang, Jin Ma, Ying Shan, Weifeng Ge

Robustness in NLP

Challenge:	Existing defense approaches focus on developing new model structures or training algorithms, but they do little to tap the potential of training instances.
Approach:	They propose a method that can distinguish between robust and non-robust instances according to the model’s sensitivity to perturbations on individual instances during training.
Outcome:	The proposed method can distinguish between robust and non-robust instances according to the model’s sensitivity to perturbations on individual instances during training.

Leveraging Large Language Models for NLG Evaluation: Advances and Challenges (2024.emnlp-main)

Copied to clipboard

Zhen Li, Xiaohan Xu, Tao Shen, Can Xu, Jia-Chen Gu, Yuxuan Lai, Chongyang Tao, Shuai Ma

TriPlay-RL: Tri-Role Self-Play Reinforcement Learning for LLM Safety Alignment (2026.acl-long)

Challenge:	introducing Large Language Models (LLMs) has opened new avenues for assessing generated content quality, e.g., coherence, creativity, and context relevance.
Approach:	They propose a taxonomy for organizing existing LLM-based evaluation metrics and a structured framework to understand and compare them.
Outcome:	The proposed taxonomy offers a framework to understand and compare LLM-based evaluation methods.

Copied to clipboard

Zhewen Tan, Wenhan Yu, Jianfeng Si, Tongxin Liu, Kaiqi Guan, Huiyan Jin, Jiawen Tao, Xiaokun Yuan, Xiangzheng Zhang, Duohe Ma, Tong Yang, Lin Sun

Robustness in NLP

Challenge:	Existing approaches to safety alignment of large language models rely on costly manual annotations or human review.
Approach:	They propose a closed-loop reinforcement learning framework called TriPlay-RL that enables iterative collaboration among three roles with near-zero manual annotation.
Outcome:	The proposed framework achieves 20%–50% improvement in adversarial effectiveness while preserving high output diversity while achieving 10%–30% gains in safety performance without degrading general reasoning capability.

Re-Reading Improves Reasoning in Large Language Models (2024.emnlp-main)

Copied to clipboard

Xiaohan Xu, Chongyang Tao, Tao Shen, Can Xu, Hongbo Xu, Guodong Long, Jian-Guang Lou, Shuai Ma

Scattered Hypothesis Generation for Open-Ended Event Forecasting (2026.findings-acl)

Challenge:	Unlike thought-eliciting prompting methods, RE2 shifts the focus to the input by processing questions twice, thereby enhancing the understanding process.
Approach:	They introduce a simple, yet general and effective prompting method, RE2, which rereads the question as input.
Outcome:	The proposed method demonstrates strong generality and compatibility with most thought-eliciting prompting methods, including CoT.

Copied to clipboard

He Chang, Zhulin Tao, Lifang Yang, Xianglin Huang, Yunshan Ma

Challenge:	Existing methods for event forecasting focus on the most probable outcomes, neglecting the intrinsic uncertainty of real-world events.
Approach:	They propose a reinforcement learning framework that optimizes inclusiveness and diversity of the hypothesis by integrating validity-gated score into the overall objective.
Outcome:	The proposed framework outperforms baselines on two real-world benchmark datasets.

TextFusion: Privacy-Preserving Pre-trained Model Inference via Token Fusion (2022.emnlp-main)

Copied to clipboard

Xin Zhou, Jinzhu Lu, Tao Gui, Ruotian Ma, Zichu Fei, Yuran Wang, Yong Ding, Yibo Cheung, Qi Zhang, Xuanjing Huang

Learning “O” Helps for Learning More: Handling the Unlabeled Entity Problem for Class-incremental NER (2023.acl-long)

Challenge:	Existing methods to preserve inference privacy are available as cloud services . however, the risk of privacy leakage remains, according to recent studies .
Approach:	They propose a method to preserve inference privacy by fusing token representations in the cloud.
Outcome:	The proposed method preserves inference privacy without sacrificing performance on different scenarios.

Copied to clipboard

Ruotian Ma, Xuanting Chen, Zhang Lin, Xin Zhou, Junzhe Wang, Tao Gui, Qi Zhang, Xiang Gao, Yun Wen Chen

XTREME-UP: A User-Centric Scarce-Data Benchmark for Under-Represented Languages (2023.findings-emnlp)

Challenge:	Existing Named Entity Recognition systems are typically trained on a large-scale dataset with predefined entity classes, then deployed for entity recognition on the test data without further adaptation or refinement.
Approach:	They propose a representation learning method that adaptively detects entity clusters in "O" and two effective distance-based relabeling strategies for better learning the old classes.
Outcome:	The proposed method achieves 10.62% improvement over the baseline methods.

Copied to clipboard

Sebastian Ruder, Jonathan Clark, Alexander Gutkin, Mihir Kale, Min Ma, Massimo Nicosia, Shruti Rijhwani, Parker Riley, Jean-Michel Sarr, Xinyi Wang, John Wieting, Nitish Gupta, Anna Katanova, Christo Kirov, Dana Dickinson, Brian Roark, Bidisha Samanta, Connie Tao, David Adelani, Vera Axelrod, Isaac Caswell, Colin Cherry, Dan Garrette, Reeve Ingle, Melvin Johnson, Dmitry Panteleev, Partha Talukdar

FedMKT: Federated Mutual Knowledge Transfer for Large and Small Language Models (2025.coling-main)

Challenge:	Existing datasets are often informed by established research directions in the NLP community.
Approach:	They propose a benchmark to evaluate the capabilities of language models across 88 under-represented languages over 9 key user-centric technologies including ASR, OCR, MT, and information access tasks.
Outcome:	The proposed benchmark evaluates the capabilities of language models across 88 under-represented languages over 9 key user-centric technologies including ASR, OCR, MT, and information access tasks.

Copied to clipboard

Tao Fan, Guoqiang Ma, Yan Kang, Hanlin Gu, Yuanfeng Song, Lixin Fan, Kai Chen, Qiang Yang

Bridging the Editing Gap in LLMs: FineEdit for Precise and Targeted Text Modifications (2025.findings-emnlp)

Challenge:	Recent research in large language models (LLMs) has focused on enabling clients to fine-tune their locally deployed homogeneous LLMs collaboratively or on transferring knowledge from server-based LLM to small language models at downstream clients.
Approach:	They propose a parameter-efficient federated mutual knowledge transfer framework for large and small language models that allows for token alignment and selective knowledge transfer between client-side LLMs and a server-side SLM.
Outcome:	The proposed framework enhances the performance of both LLMs and SLMs with clients' unique domain insights while preserving the server's LLM and client's unique domain insight.

Copied to clipboard

Yiming Zeng, Wanhao Yu, Zexin Li, Tao Ren, Yu Ma, Jinghan Cao, Xiyan Chen, Tingting Yu

Challenge:	a recent study shows that large language models can perform precise text editing tasks.
Approach:	InstrEditBench is a benchmark dataset that compares 30,000 structured editing tasks . experimental evaluations show FineEdit outperforms state-of-the-art models .
Outcome:	The proposed model outperforms state-of-the-art models on single-turn edits and mistral-7B-OpenOrca on direct edits.

CoTrust: Privacy-Preserving Collaboration Between Large and Small Language Models in Trusted Execution Environments (2026.findings-acl)

Copied to clipboard

Zhenya Ma, Tingyi Wang, Yongheng Deng, Ziqing Qiao, Yinggui Wang, Tao Wei, Lei Wang, Ju Ren

FedProxy: Federated Fine-Tuning of LLMs via Proxy SLMs and Heterogeneity-Aware Fusion (2026.acl-long)

Challenge:	Large language models (LLMs) provide powerful text generation capabilities, but accessing sensitive user inputs raises privacy concerns.
Approach:	They propose a privacy-preserving collaborative inference framework that combines large language models with small language models inside TEE to preserve privacy.
Outcome:	Experiments show that CoTrust outperforms unconstrained LLMs on multiple question answering and summarization benchmarks while maintaining strong privacy protection.

Copied to clipboard

Tao Fan, Guoqiang Ma, Yuanfeng Song, Lixin Fan, Kai Chen, Qiang Yang

WebQuality: A Large-scale Multi-modal Web Page Quality Assessment Dataset with Multiple Scoring Dimensions (2025.naacl-long)

Challenge:	Existing methods for fine-tuning Large Language Models (LLMs) suffer from a performance bottleneck . Existing approaches like Offsite-Tuning (OT) secure the LLMs IP .
Approach:	They propose a framework that replaces weak adapters with a unified, powerful Proxy Small Language Model (SLM) they propose 'resource-friendly' compression and 'robust optimization' to handle data heterogeneity.
Outcome:	Experiments show that FedProxy outperforms OT and centralized fine-tuning methods.

Copied to clipboard

Tao Zhang, Yige Wang, ZhuHangyu ZhuHangyu, Li Xin, Chen Xiang, Tian Hua Zhou, Jin Ma

3AM: An Ambiguity-Aware Multi-Modal Machine Translation Dataset (2024.lrec-main)

Challenge:	Existing studies on web page quality assessment neglect the aspect of web page content.
Approach:	They propose a Chinese dataset for web page quality assessment . the dataset includes over 65,000 detailed an-notations spanning four sub-dimensions .
Outcome:	The proposed dataset includes over 65,000 detailed an-notations spanning four sub-dimensions and incorporates elements such as HTML+CSS, text, and visual screenshot.

Copied to clipboard

Xinyu Ma, Xuebo Liu, Derek F. Wong, Jun Rao, Bei Li, Liang Ding, Lidia S. Chao, Dacheng Tao, Min Zhang

Machine Translation

Challenge:	Existing studies have shown that visual information in existing MMT datasets is insufficient, causing models to disregard it and overestimate their capabilities.
Approach:	They propose to use 3AM to create an ambiguity-aware multimodal machine translation dataset.
Outcome:	The proposed dataset includes more ambiguity and a greater variety of captions and images than other MMT datasets.

MuVER: Improving First-Stage Entity Retrieval with Multi-View Entity Representations (2021.emnlp-main)

Copied to clipboard

Xinyin Ma, Yong Jiang, Nguyen Bach, Tao Wang, Zhongqiang Huang, Fei Huang, Weiming Lu

Representation Learning

Challenge:	Recent advances in entity retrieval ignore the property that meanings of entity mentions diverge in different contexts and are related to various portions of descriptions.
Approach:	They propose a novel approach that constructs multi-view representations for entity descriptions and approximates the optimal view for mentions via a heuristic searching method.
Outcome:	The proposed approach achieves state-of-the-art performance on ZESHEL and improves quality of candidates on three standard Entity Linking datasets.

Do Vision-Language Models Have Internal World Models? Towards an Atomic Evaluation (2025.findings-acl)

Copied to clipboard

Qiyue Gao, Xinyu Pi, Kevin Liu, Junrong Chen, Ruolan Yang, Xinqi Huang, Xinyu Fang, Lu Sun, Gautham Kishore, Bo Ai, Stone Tao, Mengyang Liu, Jiaxi Yang, Chao-Jung Lai, Chuanyang Jin, Jiannan Xiang, Benhao Huang, Zeming Chen, David Danks, Hao Su, Tianmin Shu, Ziqiao Ma, Lianhui Qin, Zhiting Hu

On Vision Features in Multimodal Machine Translation (2022.acl-long)

Challenge:	Recent studies have evaluated and shown limitations in specific capabilities such as visual understanding, but a systematic evaluation of VLMs’ fundamental WM abilities remains absent.
Approach:	They propose a framework that assesses perception and prediction to provide an atomic evaluation of VLMs as WMs.
Outcome:	The proposed framework assesses perception and prediction abilities on 15 latest VLMs and compares them to human-level models.

Copied to clipboard

Bei Li, Chuanhao Lv, Zefan Zhou, Tao Zhou, Tong Xiao, Anxiang Ma, JingBo Zhu

Making Harmful Behaviors Unlearnable for Large Language Models (2024.findings-acl)

Challenge:	Recent work on multimodal machine translation (MMT) has focused on the way of incorporating vision features into translation but little attention is given to the quality of vision models.
Approach:	They develop a selective attention model to study the patch-level contribution of an image in multimodal machine translation.
Outcome:	The proposed model is able to learn translation from the visual modality on probing tasks and is compared with existing models.

Copied to clipboard

Xin Zhou, Yi Lu, Ruotian Ma, Yujian Wei, Tao Gui, Qi Zhang, Xuanjing Huang

SENT: Sentence-level Distant Relation Extraction via Negative Training (2021.acl-long)

Challenge:	Large language models (LLMs) are often customized by fine-tuning for the requirements of different domains.
Approach:	They propose a controllable training framework to make undesired behaviors unlearnable during the fine-tuning process.
Outcome:	The proposed framework makes undesired behaviors unlearnable during the fine-tuning process while preserving the ability to learn other information.

Copied to clipboard

Ruotian Ma, Tao Gui, Linyang Li, Qi Zhang, Xuanjing Huang, Yaqian Zhou

Template-free Prompt Tuning for Few-shot NER (2022.naacl-main)

Challenge:	Existing methods for relation extraction use bag labels, which introduce noise, to train the model.
Approach:	They propose to use negative training to train a model using complementary labels to separate the noisy data from the training data.
Outcome:	The proposed method improves on previous methods on sentence-level evaluation and de-noise effect.

Copied to clipboard

Ruotian Ma, Xin Zhou, Tao Gui, Yiding Tan, Linyang Li, Qi Zhang, Xuanjing Huang

Prompt Refinement with Image Pivot for Text-to-Image Generation (2024.acl-long)

Challenge:	Prompt-based methods have been successfully applied in few-shot learning tasks . however, when applied to token-level labeling tasks, it would be time-consuming to enumerate the template queries over all potential entity spans.
Approach:	They propose a method to reformulate NER tasks as LM problems without templates.
Outcome:	The proposed method is 30.12 times faster than the template-based method under few-shot settings.

Copied to clipboard

Jingtao Zhan, Qingyao Ai, Yiqun Liu, Yingwei Pan, Ting Yao, Jiaxin Mao, Shaoping Ma, Tao Mei

Multilinguality

Challenge:	Recent advances in text-to-image generation have markedly expanded the boundaries of digital artistry, enabling the creation of visually compelling images with unprecedented ease.
Approach:	They propose to decompose the prompt refinement process into two tasks: inferring user-preferred images from user languages and translating them into system languages.
Outcome:	Experiments show that PRIP outperforms baselines and transfers to unseen systems in a zero-shot manner.

Cross-Linguistic Syntactic Difference in Multilingual BERT: How Good is It and How Does It Affect Transfer? (2022.emnlp-main)

Copied to clipboard

Ningyu Xu, Tao Gui, Ruotian Ma, Qi Zhang, Jingting Ye, Menghan Zhang, Xuanjing Huang

Route Sparse Autoencoder to Interpret Large Language Models (2025.emnlp-main)

Challenge:	Multilingual BERT (mBERT) has demonstrated considerable cross-lingual syntactic ability, but it is not well understood what leads to this variation and whether it fairly reflects difference between languages.
Approach:	They propose to use multilingual BERT to enable zero-shot cross-lingual transfer of syntactic knowledge between different languages by generating grammatical relations in 24 different languages.
Outcome:	The results show that the distance between the distributions of different languages is highly consistent with the syntactic difference in terms of linguistic formalisms.

Copied to clipboard

Wei Shi, Sihang Li, Tao Liang, Mingyang Wan, Guojun Ma, Xiang Wang, Xiangnan He

Explainability & Interpretability in NLP

Challenge:	Sparse autoencoders (SAEs) extract interpretable and monosemantic features in large language models . prior work focused on feature extraction from a single layer, failing to capture activations that span multiple layers.
Approach:	They propose a framework that integrates a routing mechanism with a shared SAE to efficiently extract features from multiple layers.
Outcome:	The proposed framework extracts features from multiple layers while incurring minimal parameter overhead while achieving high interpretability and flexibility.

TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing (2021.acl-demo)

Copied to clipboard

Xiao Wang, Qin Liu, Tao Gui, Qi Zhang, Yicheng Zou, Xin Zhou, Jiacheng Ye, Yongxin Zhang, Rui Zheng, Zexiong Pang, Qinzhuo Wu, Zhengyan Li, Chong Zhang, Ruotian Ma, Zichu Fei, Ruijian Cai, Jun Zhao, Xingwu Hu, Zhiheng Yan, Yiding Tan, Yuan Hu, Qiyuan Bian, Zhihua Liu, Shan Qin, Bolin Zhu, Xiaoyu Xing, Jinlan Fu, Yue Zhang, Minlong Peng, Xiaoqing Zheng, Yaqian Zhou, Zhongyu Wei, Xipeng Qiu, Xuanjing Huang

Robustness in NLP

Challenge:	Existing approaches to textual robustness evaluation focus on slightly modifying the input data, which maintains the original meaning and results in a different prediction.
Approach:	They propose a multilingual robustness evaluation toolkit for NLP that integrates universal text transformations, task-specific transformations and adversarial attack.
Outcome:	The toolkit includes universal text transformation, task-specific transformation, adversarial attack, subpopulation, and their combinations to provide comprehensive robustness analyses.

ED2LM: Encoder-Decoder to Language Model for Faster Document Re-ranking Inference (2022.findings-acl)

Copied to clipboard

Kai Hui, Honglei Zhuang, Tao Chen, Zhen Qin, Jing Lu, Dara Bahri, Ji Ma, Jai Gupta, Cicero Nogueira dos Santos, Yi Tay, Donald Metzler

Multi-grained Named Entity Recognition (P19-1)

Challenge:	State-of-the-art neural models typically encode document-query pairs using cross-attention for re-ranking.
Approach:	They propose to fine tune a pretrained encoder-decoder model using document to query generation.
Outcome:	The proposed model achieves comparable results to more expensive approaches while being 6.8X faster.

Copied to clipboard

Congying Xia, Chenwei Zhang, Tao Yang, Yaliang Li, Nan Du, Xian Wu, Wei Fan, Fenglong Ma, Philip Yu

TextObfuscator: Making Pre-trained Language Model a Privacy Protector via Obfuscating Word Representations (2023.findings-acl)

Challenge:	Existing approaches treat Named Entity Recognition (NER) as a sequence labeling task.
Approach:	They propose a framework for Multi-Grained Named Entity Recognition where multiple entities or entity mentions in a sentence could be non-overlapping or totally nested.
Outcome:	The proposed framework outperforms current state-of-the-art frameworks by 4.4% in terms of the F1 score among nested/non-overlapping NER tasks.

Copied to clipboard

Xin Zhou, Yi Lu, Ruotian Ma, Tao Gui, Yuran Wang, Yong Ding, Yibo Zhang, Qi Zhang, Xuanjing Huang