Papers by Tao Qi

RE-Matching: A Fine-Grained Semantic Matching Method for Zero-Shot Relation Extraction (2023.acl-long)

Challenge:	Existing processes that reward for each step are one-directional and lack a mechanism to model the distance to the final target.
Approach:	They propose a process supervision model that evaluates the correctness of previous steps and the probability of future success.
Outcome:	The proposed model outperforms existing supervision models like ORM and PRM on reasoning tasks and improves solution re-design.

Copied to clipboard

Jun Zhao, WenYu Zhan, Xin Zhao, Qi Zhang, Tao Gui, Zhongyu Wei, Junzhe Wang, Minlong Peng, Mingming Sun

UnifiedMLLM: Enabling Unified Representation for Multi-modal Multi-tasks With Large Language Model (2025.findings-naacl)

Challenge:	Existing methods for zero-shot relation extraction lack explicit modeling of matching pattern . et al. (2018) show that our method achieves higher matching accuracy and faster inference speed .
Approach:	They propose a fine-grained semantic matching method tailored for zero-shot relation extraction . they decompose sentence-level similarity score into entity matching score and context matching score .
Outcome:	The proposed method achieves higher matching accuracy and faster inference speed than state-of-the-art methods.

Copied to clipboard

Zhaowei Li, Wei Wang, YiQing Cai, Qi Xu, Pengyu Wang, Dong Zhang, Hang Song, Botian Jiang, Zhida Huang, Tao Wang

FinToolSyn: A forward synthesis Framework for Financial Tool-Use Dialogue Data with Dynamic Tool Retrieval (2026.findings-acl)

Challenge:	Representative models like LLaVA and MiniGPT-4 have great capabilities in various tasks.
Approach:	They propose a unified model to represent various multi-modal tasks using a single representation.
Outcome:	The proposed model outperforms existing models in a variety of tasks while maintaining generality and scalability.

Copied to clipboard

Caishuang Huang, Yang Qiao, Rongyu Zhang, Junjie Ye, Pu Lu, null Wuwenxi, Meng Zhou, Xiku Du, Qi Zhang, Tao Gui, Xuanjing Huang

Privacy-Preserving News Recommendation Model Learning (2020.findings-emnlp)

Challenge:	Existing data synthesis methods rely on static tools to generate queries . this approach fails to capture the implicit, event-driven nature of real-world needs .
Approach:	They propose a forward synthesis framework to generate high-quality financial dialogues . they construct a repository of 43,066 tools and synthesize over 148k dialogue instances .
Outcome:	Experiments show that models trained on FinToolSyn achieve a 21.06% improvement . the framework is designed to generate high-quality financial dialogues .

Copied to clipboard

Tao Qi, Fangzhao Wu, Chuhan Wu, Yongfeng Huang, Xing Xie

FOLIO: Natural Language Reasoning with First-Order Logic (2024.emnlp-main)

Challenge:	Existing news recommendation methods rely on centralized storage of user behavior data for model training, which may lead to privacy concerns and risks due to the privacy-sensitive nature of user behaviors.
Approach:	They propose a privacy-preserving method where user behavior data is locally stored on user devices to train accurate news recommendation models.
Outcome:	The proposed method can train accurate news recommendation models without centralized storage of user behavior data.

Copied to clipboard

Simeng Han, Hailey Schoelkopf, Yilun Zhao, Zhenting Qi, Martin Riddell, Wenfei Zhou, James Coady, David Peng, Yujie Qiao, Luke Benson, Lucy Sun, Alexander Wardle-Solano, Hannah Szabó, Ekaterina Zubova, Matthew Burtell, Jonathan Fan, Yixin Liu, Brian Wong, Malcolm Sailor, Ansong Ni, Linyong Nan, Jungo Kasai, Tao Yu, Rui Zhang, Alexander Fabbri, Wojciech Kryscinski, Semih Yavuz, Ye Liu, Xi Lin, Shafiq Joty, Yingbo Zhou, Caiming Xiong, Rex Ying, Arman Cohan, Dragomir Radev

Llama SLayer 8B: Shallow Layers Hold the Key to Knowledge Injection (2024.findings-emnlp)

Challenge:	Existing benchmarks for logical reasoning in large language models lack language naturalness or limited complexity.
Approach:	They propose to use first-order logic annotations to evaluate logical reasoning capabilities of large language models.
Outcome:	The proposed dataset evaluates the FOL reasoning ability of supervised fine-tuning on medium-sized language models.

Copied to clipboard

Tianxiang Chen, Zhentao Tan, Tao Gong, Yue Wu, Qi Chu, Bin Liu, Jieping Ye, Nenghai Yu

Challenge:	Existing methods to augment pre-trained large language models require extensive computational efforts and massive data volumes, challenging the widespread accessibility of LLM research.
Approach:	They propose a post-pretraining strategy of selectively enhancing shallow layers while pruning less effective deep ones to augment pretrained large language models.
Outcome:	The proposed approach improves performance on the corpus of code & math and a legal corpus and is widely applicable.

AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments (2026.acl-long)

Copied to clipboard

Zhiheng Xi, Dingwen Yang, Jiaqi Liu, Jixuan Huang, Honglin Guo, Baodai Huang, Tinggang Chen, Qi Zhang, Zhonghang Lu, Chenyu Liu, Jiajun Sun, Jiazheng Zhang, Dingwei Zhu, Xin Guo, Junzhe Wang, Zhihao Zhang, Yuming Yang, Junjie Ye, Minghe Gao, Dongrui Liu, Jiaming Ji, Guohao Li, Tao Gui, Qi Zhang, Xuanjing Huang

LoRAMoE: Alleviating World Knowledge Forgetting in Large Language Models via MoE-Style Plugin (2024.acl-long)

Challenge:	Existing benchmarks evaluate agents in simplified, idealized settings, relying on pre-packaged tool interfaces, overlooking critical steps, and assume inputs are clean and fully specified.
Approach:	They propose a framework that evaluates language agents in simplified, idealized settings . they show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 .
Outcome:	Experiments on 15 proprietary and open-source models show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 .

Copied to clipboard

Shihan Dou, Enyu Zhou, Yan Liu, Songyang Gao, Wei Shen, Limao Xiong, Yuhao Zhou, Xiao Wang, Zhiheng Xi, Xiaoran Fan, Shiliang Pu, Jiang Zhu, Rui Zheng, Tao Gui, Qi Zhang, Xuanjing Huang

Causal Intervention Improves Implicit Sentiment Analysis (2022.coling-1)

Challenge:	Experimental results show that, as the instruction data increases, LoRAMoE can significantly improve the ability to process downstream tasks, while maintaining the world knowledge stored in the LLM.
Approach:	They propose a framework that introduces several low-rank adapters and integrates them by using a router network to freeze the backbone model and force a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks.
Outcome:	The proposed framework freezes the backbone model and forces a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks, to alleviate world knowledge forgetting.

Copied to clipboard

Siyin Wang, Jie Zhou, Changzhi Sun, Junjie Ye, Tao Gui, Qi Zhang, Xuanjing Huang

Sentiment Analysis

Challenge:	Existing neural models struggle with implicit sentiment analysis because they latch onto spurious correlations, resulting in poor generalization and robustness.
Approach:	They propose a CausaL intervention model for implicit sEntiment ANalysis using instrumental variable to eliminate confounding causal effects and extract the pure causal effect between sentence and sentiment.
Outcome:	The proposed model extracts the pure causal effect between sentence and sentiment using instrumental variable.

TextMixer: Mixing Multiple Inputs for Privacy-Preserving Inference (2023.findings-emnlp)

Copied to clipboard

Xin Zhou, Yi Lu, Ruotian Ma, Tao Gui, Qi Zhang, Xuanjing Huang

MINER: Improving Out-of-Vocabulary Named Entity Recognition from an Information Theoretic Perspective (2022.acl-long)

Challenge:	Pre-trained language models (PLMs) are often deployed as cloud services, enabling users to upload textual data and perform inference remotely.
Approach:	They propose a privacy-preserving inference framework called MixPi which aims to obfuscate a user's private input by mixing it with multiple other inputs.
Outcome:	The proposed framework surpasses existing privacy-preserving methods on token and sentence classification tasks.

Copied to clipboard

Xiao Wang, Shihan Dou, Limao Xiong, Yicheng Zou, Qi Zhang, Tao Gui, Liang Qiao, Zhanzhan Cheng, Xuanjing Huang

Length Generalization of Causal Transformers without Position Encoding (2024.findings-acl)

Challenge:	Named Entity Recognition models are feature-engineering and machine learning based.
Approach:	They propose a new NER learning framework that uses entity mentions to improve model performance.
Outcome:	The proposed model achieves better performance on OOV entities on various settings and datasets.

Copied to clipboard

Jie Wang, Tao Ji, Yuanbin Wu, Hang Yan, Tao Gui, Qi Zhang, Xuanjing Huang, Xiaoling Wang

A Lexicon-Based Graph Neural Network for Chinese NER (D19-1)

Challenge:	Besides Transformers without position encodings, the success of NoPE provides a new way to overcome the challenge of generalizing to longer sentences.
Approach:	They propose a parameter-efficient tuning for searching attention heads’ best temperature hyper-parameters, which substantially expands NoPE’s context size.
Outcome:	The proposed tuning significantly expands NoPE's context size, allowing it to generalize to longer sentences with state-of-the-art generalization algorithms.

Copied to clipboard

Tao Gui, Yicheng Zou, Qi Zhang, Minlong Peng, Jinlan Fu, Zhongyu Wei, Xuanjing Huang

Improving Discriminative Capability of Reward Models in RLHF Using Contrastive Learning (2024.emnlp-main)

Challenge:	Chinese named entity recognition models are vulnerable to word ambiguities due to the lack of global semantics and chain structure.
Approach:	They propose a lexicon-based graph neural network with global semantics to solve word ambiguities in Chinese named entity recognition (NER) Lexicons are used to construct the graph and provide word-level features.
Outcome:	The proposed model improves on four NER datasets on Chinese characters, potential words, and the whole-sentence semantics.

Copied to clipboard

Lu Chen, Rui Zheng, Binghai Wang, Senjie Jin, Caishuang Huang, Junjie Ye, Zhihao Zhang, Yuhao Zhou, Zhiheng Xi, Tao Gui, Qi Zhang, Xuanjing Huang

Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models (2026.findings-acl)

Challenge:	Current methods rely on ranking losses to teach reward model to assess preferences, but they are susceptible to noise and ambiguous data, often failing to deeply understand human intentions.
Approach:	They propose a method that incorporates contrastive learning into the reward modeling process to enhance generalization and stabilize the reinforcement learning training process.
Outcome:	The proposed method enhances generalization of the reward model, stabilizes the reinforcement learning training process, and improves the final alignment with human preferences.

Copied to clipboard

Hengyuan Zhang, Zhihao Zhang, Ercong Nie, Mingyang Wang, Zunhai Su, Yiwei Wang, Qianli Wang, Shuzhou Yuan, Xufeng Duan, Qibo Xue, Zeping Yu, Chenming Shang, Xiao Liang, Jing Xiong, Hui Shen, Chaofan Tao, Zhengwu Liu, Senjie Jin, Zhiheng Xi, Dongdong Zhang, Sophia Ananiadou, Tao Gui, Ruobing Xie, Hayden Kwok-Hay So, Hinrich Schuetze, Xuanjing Huang, Qi Zhang, Ngai Wong

Explainability & Interpretability in NLP

Challenge:	Existing literature on mechanistic interpretation (MI) treats it as an observational science, leaving practical applications underexplored.
Approach:	They propose a survey structured around the pipeline to identify and improve MI models.
Outcome:	The proposed framework enables tangible improvements in Alignment, Capability, and Efficiency.

Governance in Motion: Co-evolution of Constitutions and AI models for Scalable Safety (2025.emnlp-main)

Copied to clipboard

Chenhao Huang, Ziyu Shen, Yicong Ren, Huiyuan Zheng, Jiazheng Zhang, Mingxu Chai, Ming Zhang, Shihan Dou, Fan Mo, Jie Shi, Tao Gui, Qi Zhang, Xuanjing Huang

ProofInfer: Generating Proof via Iterative Hierarchical Inference (2022.emnlp-main)

Challenge:	Existing approaches to align large language models with human preferences lack flexibility . static alignment preferences lack the ability to correct misaligned behaviors as they emerge .
Approach:	They propose a framework that enables dynamic and continuous alignment of large language models with human preferences.
Outcome:	The proposed framework improves safety and accuracy of a 7B model with human annotations.

Copied to clipboard

Zichu Fei, Qi Zhang, Xin Zhou, Tao Gui, Xuanjing Huang

Uni-FedRec: A Unified Privacy-Preserving News Recommendation Framework for Model Training and Online Serving (2021.findings-emnlp)

Challenge:	Existing proof generation models focus on generating several proof paths instead of a whole tree.
Approach:	They propose a method that generates the proof tree via iterative hierarchical inference . they propose coding the proof as plain text without losing structure information .
Outcome:	The proposed proof generation model significantly improves performance on widely-used datasets.

Copied to clipboard

Tao Qi, Fangzhao Wu, Chuhan Wu, Yongfeng Huang, Xing Xie

Inductive Relation Inference of Knowledge Graph Enhanced by Ontology Information (2023.findings-emnlp)

Challenge:	Existing news recommendation methods rely on user behavior data to model user interests and user interests.
Approach:	They propose a unified news recommendation framework that uses user data locally stored in user clients to train models and serve users in a privacy-preserving way.
Outcome:	The proposed framework outperforms baseline methods and effectively protects user privacy.

Copied to clipboard

Wentao Zhou, Jun Zhao, Tao Gui, Qi Zhang, Xuanjing Huang

Knowledge Representation

Challenge:	Existing methods to inference knowledge graphs lack ontology information, which is often too sparse.
Approach:	They propose a knowledge graph inductive inference method that fuses ontology information to learn the semantic information of entities.
Outcome:	The proposed method outperforms large language models like ChatGPT on two benchmark datasets and improves the MRR metrics by 15.4% and 44.1%, respectively.

DARM: Distribution-Aware Reward Modeling by Alleviating Biases from Low Preference-Context Dependency Data (2026.acl-long)

Copied to clipboard

Shaofan Liu, Guoqiang Zhang, Shihan Dou, Huiyuan Zheng, Yiming Zhou, Junjie Ye, Shaowen Wang, Shichun Liu, Jiazheng Zhang, Tao Gui, Qi Zhang, Xuanjing Huang

Challenge:	Existing methods for training reward models are vulnerable to context neglect and degraded accuracy.
Approach:	They propose distribution-aware reward modeling that augments the RM objective with a conditional mutual information regularizer that maximizes context and the predicted reward conditioned on the response.
Outcome:	The proposed model improves performance in RLHF and improves accuracy in other settings.

Reviews Meet Graphs: Enhancing User and Item Representations for Recommendation with Hierarchical Attentive Graph Neural Network (D19-1)

Copied to clipboard

Chuhan Wu, Fangzhao Wu, Tao Qi, Suyu Ge, Yongfeng Huang, Xing Xie

TransferTOD: A Generalizable Chinese Multi-Domain Task-Oriented Dialogue System with Transfer Capabilities (2024.emnlp-main)

Challenge:	Existing methods to learn user and item representations from review texts do not take into account the user-user and item-item relatedness of the user.
Approach:	They propose to use review content and user-item graphs to integrate them as different views.
Outcome:	The proposed approach can learn user and item representations from review content and user-item graphs.

Copied to clipboard

Ming Zhang, Caishuang Huang, Yilong Wu, Shichun Liu, Huiyuan Zheng, Yurui Dong, Yujiong Shen, Shihan Dou, Jun Zhao, Junjie Ye, Qi Zhang, Tao Gui, Xuanjing Huang

A Confidence-based Partial Label Learning Model for Crowd-Annotated Named Entity Recognition (2023.findings-acl)

Challenge:	Current datasets cater to user-led systems and are limited to predefined specific scenarios and slots.
Approach:	They propose to use a Chinese dialogue dataset to train a model that authentically simulates human-computer dialogues in 30 popular life service scenarios.
Outcome:	The proposed model achieves a joint accuracy of 75.09% in out-of-domain evaluations . it also achieves notable abilities in slot filling and questioning .

Copied to clipboard

Limao Xiong, Jie Zhou, Qunxi Zhu, Xiao Wang, Yuanbin Wu, Qi Zhang, Tao Gui, Xuanjing Huang, Jin Ma, Ying Shan

AgentV-RL: Scaling Reward Modeling with Agentic Verifier (2026.findings-acl)

Challenge:	Existing models for named entity recognition (NER) are based on large-scale labeled datasets, which always obtain using crowdsourcing.
Approach:	They propose a CONfidence-based partial Label Learning method to integrate prior and posterior confidences for crowd-annotated named entity recognition models.
Outcome:	The proposed model improves on real-world and synthetic datasets compared with baselines.

Copied to clipboard

Jiazheng Zhang, Ziche Fu, Zhiheng Xi, Wenqing Jing, Mingxu Chai, Wei He, Guoqiang Zhang, Chenghao Fan, Chenxin An, Wenxiang Chen, Zhicheng Liu, Haojie Pan, Dingwei Zhu, Tao Gui, Qi Zhang, Xuanjing Huang

LFKQG: A Controlled Generation Framework with Local Fine-tuning for Question Generation over Knowledge Bases (2022.coling-1)

Challenge:	Existing approaches to improve LLM reasoning are limited in complex domains and lack external grounding makes verifiers unreliable on computation-intensive tasks.
Approach:	They propose a framework that transforms reward modeling into a multi-turn, tool-augmented deliberative process.
Outcome:	The proposed framework surpasses state-of-the-art ORMs by 25.2% under parallel and sequential TTS.

Copied to clipboard

Zichu Fei, Xin Zhou, Tao Gui, Qi Zhang, Xuanjing Huang

Unveiling and Consulting Core Experts in Retrieval-Augmented MoE-based LLMs (2024.emnlp-main)

Challenge:	Existing KBQG models focus on the most relevant part of the answer entity, while neglecting the rest of the subgraph.
Approach:	They propose a controlled generation framework for Question Generation over Knowledge Bases that generates questions with out-of-vocabulary (OOV) predicates.
Outcome:	The proposed framework outperforms existing methods significantly on three widely-used benchmark datasets SimpleQuestion, PathQuestions, and WebQuestIONS.

Copied to clipboard

Xin Zhou, Ping Nie, Yiwen Guo, Haojie Wei, Zhanqiu Zhang, Pasquale Minervini, Ruotian Ma, Tao Gui, Qi Zhang, Xuanjing Huang

Information Retrieval

Challenge:	Existing research seeks to enhance RAG performance by retrieving higher-quality documents or designing RAG-specific LLMs, but internal mechanisms that contribute to RAG’s effectiveness remain underexplored.
Approach:	They propose to examine the internal mechanisms within the popular Mixture-of-Expert (MoE)-based LLMs and examine their ability to improve RAG by examining expert activations.
Outcome:	The proposed method significantly improved the ability of Large Language Models (LLMs) to solve knowledge-intensive tasks.

Orthogonal Subspace Learning for Language Model Continual Learning (2023.findings-emnlp)

Copied to clipboard

Xiao Wang, Tianze Chen, Qiming Ge, Han Xia, Rong Bao, Rui Zheng, Qi Zhang, Tao Gui, Xuanjing Huang

PFDial: A Structured Dialogue Instruction Fine-tuning Method Based on UML Flowcharts (2025.findings-acl)

Challenge:	Existing methods for continual learning in language models suffer catastrophic forgetting when learning sequential tasks.
Approach:	They propose an orthogonal low-rank adaptation approach for continual learning in language models that uses orthogons to learn sequentially.
Outcome:	The proposed approach outperforms state-of-the-art methods on continual learning benchmarks and preserves generalization ability of LLMs on unseen tasks.

Copied to clipboard

Ming Zhang, Yuhui Wang, Yujiong Shen, Tingyi Yang, Changhao Jiang, Yilong Wu, Shihan Dou, Qinhao Chen, Zhiheng Xi, Zhihao Zhang, Yi Dong, Zhen Wang, Zhihui Fei, Mingyang Wan, Tao Liang, Guojun Ma, Qi Zhang, Tao Gui, Xuanjing Huang

Flooding-X: Improving BERT’s Resistance to Adversarial Attacks via Loss-Restricted Fine-Tuning (2022.acl-long)

Challenge:	Large Language Models (LLMs) have shown remarkable progress in dialogue and reasoning, but they struggle to solve strictly constrained dialogue tasks.
Approach:	They construct a dataset that contains 12,705 high-quality Chinese dialogue instructions from 440 flowcharts containing 5,055 process nodes.
Outcome:	The proposed model outperforms GPT-4o models on backward transitions and outperformed GPT-42 models on the same dataset.

Copied to clipboard

Qin Liu, Rui Zheng, Bao Rong, Jingyi Liu, ZhiHua Liu, Zhanzhan Cheng, Liang Qiao, Tao Gui, Qi Zhang, Xuanjing Huang

Farewell to Aimless Large-scale Pretraining: Influential Subset Selection for Language Model (2023.findings-acl)

Challenge:	Existing approaches to generating adversarial perturbations scale up the cost of training computational complexity by the number of gradient steps it takes to obtain the adversarials.
Approach:	They propose a flood method which aims at better generalization and a criterion to bring hyper-parameter-dependent flooding into effect with a narrowed-down search space by measuring how the gradient steps taken within one epoch affect the loss of each batch.
Outcome:	The proposed method improves BERT’s resistance to textual adversarial attacks by a large margin and achieves state-of-the-art robust accuracy on various text classification and GLUE tasks.

Copied to clipboard

Xiao Wang, Weikang Zhou, Qi Zhang, Jie Zhou, SongYang Gao, Junzhe Wang, Menghan Zhang, Xiang Gao, Yun Wen Chen, Tao Gui

Domain Generalization via Causal Adjustment for Cross-Domain Sentiment Analysis (2024.lrec-main)

Challenge:	Pretrained language models have achieved remarkable success in various natural language processing tasks.
Approach:	They propose to use end-task knowledge to select a tiny subset of pretraining corpus to influence performance.
Outcome:	The proposed model outperforms pretrained models on eight datasets covering four domains with 0.45% of the data and a three-orders-of-magnitude lower computational cost.

Copied to clipboard

Siyin Wang, Jie Zhou, Qin Chen, Qi Zhang, Tao Gui, Xuanjing Huang

Sentiment Analysis

Challenge:	Existing approaches to domain adaptation fail to generalize well on unknown test data.
Approach:	They propose a backdoor adjustment-based causal model to disentangle domain-specific and domain-invariant representations that play essential roles in tackling domain shift.
Outcome:	The proposed model disentangles domain-specific and domain-invariant representations that play essential roles in tackling domain shift.

Counteracting the Matthew Effect in Self-Improvement of LVLMs through Head-Tail Re-balancing (2026.acl-long)

Copied to clipboard

Xin Guo, Zhiheng Xi, Yiwen Ding, Yitao Zhai, Xiaowei Shi, Xunliang Cai, Tao Gui, Qi Zhang, Xuanjing Huang

Neural News Recommendation with Heterogeneous User Behavior (D19-1)

Challenge:	Large vision language models have impressive reasoning capabilities across complex multimodal tasks.
Approach:	They propose to use distribution-reshaping and trajectory-rebalancing to improve visual reasoning capabilities.
Outcome:	Experiments on Qwen2-VL-7B-Instruct and InternVL2.5-4B models show that their methods outperform baselines by 3.86 points.

Copied to clipboard

Chuhan Wu, Fangzhao Wu, Mingxiao An, Tao Qi, Jianqiang Huang, Yongfeng Huang, Xing Xie

Mitigating Tail Narrowing in LLM Self-Improvement via Socratic-Guided Sampling (2025.naacl-long)

Challenge:	Existing news recommendation methods rely on news click history to model user interest, but data sparsity is a problem . other kinds of user behaviors such as webpage browsing and search queries can provide useful clues of users’ news reading interest.
Approach:	They propose to exploit heterogeneous user behaviors to learn news representations from their titles via CNN networks and apply attention networks to select important words.
Outcome:	The proposed approach exploits heterogeneous user behaviors on a real-world dataset.

Copied to clipboard

Yiwen Ding, Zhiheng Xi, Wei He, Lizhuoyuan Lizhuoyuan, Yitao Zhai, Shi Xiaowei, Xunliang Cai, Tao Gui, Qi Zhang, Xuanjing Huang

OctoBench: Benchmarking Scaffold-Aware Instruction Following in Repository-Grounded Agentic Coding (2026.acl-long)

Challenge:	Large language models (LLMs) generate solutions themselves and iteratively train on filtered, high-quality rationales, but performance reaches a ceiling after a few iterations.
Approach:	They propose a strategy to improve the efficiency of sampling heavy-tailed data by using Socratic-style guidance signals to help LLMs reasoning with complex queries.
Outcome:	The proposed approach is effective on difficult queries and on held-out tasks, while requiring human supervision.

Copied to clipboard

Deming Ding, Shichun Liu, Enhui Yang, Jiahang Lin, Ziying Chen, Shihan Dou, Honglin Guo, Weiyu Cheng, Pengyu Zhao, Chengjun Xiao, Qunhong Zeng, Qi Zhang, Xuanjing Huang, Qidi Xu, Tao Gui

Programming Languages in NLP

Challenge:	coding scaffolds that follow heterogeneous instructions remain under-examined in software engineering . coding models are capable software agents, but their ability to follow constraints remains under-explored .
Approach:	They introduce OctoBench, which benchmarks scaffold-aware instruction following in agentic coding.
Outcome:	The proposed benchmark aims to accelerate the development of more scaffold-aware agents.

Self-Polish: Enhance Reasoning in Large Language Models via Problem Refinement (2023.findings-emnlp)

Copied to clipboard

Zhiheng Xi, Senjie Jin, Yuhao Zhou, Rui Zheng, Songyang Gao, Jia Liu, Tao Gui, Qi Zhang, Xuanjing Huang

AgentGym: Evaluating and Training Large Language Model-based Agents across Diverse Environments (2025.acl-long)

Challenge:	Existing prompting methods have been used to enhance multistep reasoning capabilities of large language models, but they have overlooked the potential of formulating higher-quality problems.
Approach:	They propose a method that starts from the problem side and refines problems to be more comprehensible and solvable for models.
Outcome:	The proposed method achieves notable and consistent effectiveness on five reasoning benchmarks across different models.

Copied to clipboard

Zhiheng Xi, Yiwen Ding, Wenxiang Chen, Boyang Hong, Honglin Guo, Junzhe Wang, Xin Guo, Dingwen Yang, Chenyang Liao, Wei He, Songyang Gao, Lu Chen, Rui Zheng, Yicheng Zou, Tao Gui, Qi Zhang, Xipeng Qiu, Xuanjing Huang, Zuxuan Wu, Yu-Gang Jiang

Multi-Programming Language Sandbox for LLMs (2025.acl-demo)

Challenge:	Large language models (LLMs) are promising foundations to build generally-capable agents . however, the community lacks a unified interactive framework that covers diverse environments for comprehensive evaluation of agents.
Approach:	They propose a framework that features 7 real-world scenarios, 14 environments, and 89 tasks for unified, real-time, and concurrent agent interaction.
Outcome:	The proposed framework features 7 real-world scenarios, 14 environments, and 89 tasks for unified, real-time, and concurrent agent interaction.

Copied to clipboard

Shihan Dou, Jiazheng Zhang, Jianxiang Zang, Yunbo Tao, Weikang Zhou, Haoxiang Jia, Shichun Liu, Yuming Yang, Shenxi Wu, Zhiheng Xi, Muling Wu, Rui Zheng, Changze Lv, Limao Xiong, Shaoqing Zhang, Lin Zhang, Wenyu Zhan, Rongxiang Weng, Jingang Wang, Xunliang Cai, Yueming Wu, Ming Wen, Yixin Cao, Tao Gui, Xipeng Qiu, Qi Zhang, Xuanjing Huang

Coarse-to-fine Few-shot Learning for Named Entity Recognition (2023.findings-acl)

Challenge:	MPLSandbox is an out-of-the-box multi-programming language sandbox designed to provide unified and comprehensive feedback from compiler and analysis tools for Large Language Models (LLMs).
Approach:	They propose a multi-programming language sandbox that provides unified feedback from compilers and analysis tools for Large Language Models.
Outcome:	The proposed multi-language sandbox can provide comprehensive feedback from compilers and analysis tools for large language models (LLMs).

Copied to clipboard

Ruotian Ma, Zhang Lin, Xuanting Chen, Xin Zhou, Junzhe Wang, Tao Gui, Qi Zhang, Xiang Gao, Yun Wen Chen

NoisyTune: A Little Noise Can Help You Finetune Pretrained Language Models Better (2022.acl-short)

Challenge:	Existing few-shot NER solutions do not consider sub-class discrimination and various granularity of new classes during coarse training.
Approach:	They propose a method that uses a cluster-based prototype loss to learn group-wise discriminative representations of coarse-grained classes and a mixture prototype loss for learning the representations.
Outcome:	The proposed method shows superior performance over baseline methods on in-domain and cross-domain settings with various target granularity.

Copied to clipboard

Uncertainty Aware Learning for Language Model Alignment (2024.acl-long)

Challenge:	Existing methods for finetuning pretrained language models (PLMs) have risks in overfitting the pretraining tasks and data, which may lead to suboptimal performance.
Approach:	They propose a method which adds noise to parameters of PLMs before fine-tuning.
Outcome:	The proposed method can be used on GLUE English and XTREME multilingual benchmarks.

Copied to clipboard

Yikun Wang, Rui Zheng, Liang Ding, Qi Zhang, Dahua Lin, Dacheng Tao

Enhancing Contrastive Learning with Noise-Guided Attack: Towards Continual Relation Extraction in the Wild (2024.acl-long)

Challenge:	Existing alignment strategies that focus on diverse and high-quality data often overlook the intrinsic uncertainty of tasks, learning all data samples equally.
Approach:	They propose to introduce the sample uncertainty into the alignment of different task scenarios by a simple fashion by setting the label smoothing value of training according to the uncertainty of individual samples.
Outcome:	The proposed model outperforms standard supervised fine-tuning on high-entropy tasks and complex low-entropic tasks.

Copied to clipboard

Ting Wu, Jingyi Liu, Rui Zheng, Tao Gui, Qi Zhang, Xuanjing Huang

Rescue: Ranking LLM Responses with Partial Ordering to Improve Response Generation (2024.acl-srw)

Challenge:	Existing methods for continual relation extraction (CRE) excel in preserving old knowledge but falter when confronted with contaminated data streams.
Approach:	They propose a noise-resistant contrastive framework for continual relation extraction (CRE) that preserves old knowledge while learning incremental corrupted relations.
Outcome:	The proposed framework outperforms state-of-the-art methods on various benchmarks with increasing noise rates.

Copied to clipboard

Yikun Wang, Rui Zheng, Haoming Li, Qi Zhang, Tao Gui, Fei Liu

Text Generation

Challenge:	Customizing LLMs for a specific task involves separating high-quality responses from lower-quality ones. Obtaining a large volume of expert-annotated data is costly for most tasks.
Approach:	They propose a method that trains the model to prioritize the best responses from a pool of candidates created for a task using ranking metrics.
Outcome:	The proposed method is more robust, less sensitive to noise, and can be achieved with limited human annotations or through heuristic methods.

Outcome Accuracy is Not Enough: Aligning the Reasoning Process of Reward Models (2026.acl-long)

Copied to clipboard

Binghai Wang, Yantao Liu, Yuxuan Liu, Tianyi Tang, Shenzhi Wang, Chang Gao, Chujie Zheng, Yichang Zhang, Le Yu, Shixuan Liu, Tao Gui, Qi Zhang, Xuanjing Huang, Bowen Yu, Fei Huang, Junyang Lin

Reading Order Matters: Information Extraction from Visually-rich Documents by Token Path Prediction (2023.emnlp-main)

Challenge:	Recent studies observe a phenomenon where reward models achieve high accuracy on static datasets but fail to generalize effectively during RLHF.
Approach:	They propose a method that combines rationale consistency with outcome accuracy to improve performance on RM-Bench and JudgeBench.
Outcome:	The proposed method surpasses baselines on RM-Bench and JudgeBench by an average of 5% and improves creative writing tasks by 7%.

Copied to clipboard

Chong Zhang, Ya Guo, Yi Tu, Huan Chen, Jinyang Tang, Huijia Zhu, Qi Zhang, Tao Gui

MulDimIF: A Multi-Dimensional Constraint Framework for Evaluating and Improving Instruction Following in Large Language Models (2026.findings-acl)

Challenge:	Recent advances in multimodal pre-trained models have significantly improved information extraction from visually-rich documents (VrDs).
Approach:	They propose a method to predict token sequences within visually-rich documents by a simple prediction head.
Outcome:	The proposed method can be used to predict token mentions as token sequences within documents.

Copied to clipboard

Junjie Ye, Caishuang Huang, Zhuohan Chen, Wenjie Fu, Chenyuan Yang, Leyi Yang, Yilong Wu, Peng Wang, Meng Zhou, Xiaolong Yang, Tao Gui, Qi Zhang, Zhongchao Shi, Jianping Fan, Xuanjing Huang

LongHeads: Multi-Head Attention is Secretly a Long Context Processor (2024.findings-emnlp)

Challenge:	Existing research has focused on constraint categories, offering little guidance for improving instruction following abilities.
Approach:	They propose a multi-dimensional constraint framework that allows for instruction following . they construct 9,106 code-verifiable samples and evaluate 18 LLMs .
Outcome:	The proposed framework improves instruction following performance without compromising general performance.

Copied to clipboard

Yi Lu, Xin Zhou, Wei He, Jun Zhao, Tao Ji, Tao Gui, Qi Zhang, Xuanjing Huang

Green & Sustainable NLP

Challenge:	Large language models struggle to process lengthy inputs due to limited length generalization and attention’s quadratic computational demands.
Approach:	They propose a training-free framework that allows each head to attend to important context chunks instead of allowing each head a full sentence .
Outcome:	The proposed framework unlocks multi-head attention's untapped potential by allowing each head to attend to important context chunks instead of the full sentence.

MARS-Bench: A Multi-turn Athletic Real-world Scenario Benchmark for Dialogue Evaluation (2025.findings-emnlp)

Copied to clipboard

Chenghao Yang, Yinbo Luo, Zhoufutu Wen, Qi Chu, Tao Gong, Longxiang Liu, Kaiyuan Zhang, Jianpeng Jiao, Ge Zhang, Wenhao Huang, Nenghai Yu

Towards Understanding Omission in Dialogue Summarization (2023.acl-long)

Challenge:	Large Language Models (LLMs) have been widely adopted in real-world dialogue applications, but their robustness is criticized all along.
Approach:	They propose to use play-by-play text commentary to build a multi-turn athletic real-world scenario dialogue benchmark to evaluate three critical aspects of multi-turned conversations: ultra multi- turn, interactive multi-twist, and cross-turn tasks.
Outcome:	The proposed benchmarks outperform open-source LLMs on three critical aspects of multi-turn conversations: ultra multi-turned, interactive multi- turn, and cross-turn tasks.

Copied to clipboard

Yicheng Zou, Kaitao Song, Xu Tan, Zhongkai Fu, Qi Zhang, Dongsheng Li, Tao Gui

Making Parameter-efficient Tuning More Efficient: A Unified Framework for Classification Tasks (2022.coling-1)

Challenge:	Existing methods for dialogue summarization are far from satisfactory . omission is a major factor in affecting the quality of summarizing, but few studies have explored the problem .
Approach:	They propose a dataset that provides high-quality omission labels for dialogue summarization . they propose to use this dataset to detect omitted dialogue utterances .
Outcome:	The proposed dataset improves summarization quality by providing ground-truth omission labels . the proposed dataset and codes are publicly available .

Copied to clipboard

Xin Zhou, Ruotian Ma, Yicheng Zou, Xuanting Chen, Tao Gui, Qi Zhang, Xuanjing Huang, Rui Xie, Wei Wu

LONGAGENT: Achieving Question Answering for 128k-Token-Long Documents through Multi-Agent Collaboration (2024.emnlp-main)

Challenge:	Large pre-trained language models (PLMs) have demonstrated superior performance in industrial applications.
Approach:	They propose a framework that re-uses existing parameter-efficient methods with a unified classifier.
Outcome:	The proposed framework improves the efficiency of existing parameter-efficient methods with a unified classifier.

Copied to clipboard

Jun Zhao, Can Zu, Xu Hao, Yi Lu, Wei He, Yiwen Ding, Tao Gui, Qi Zhang, Xuanjing Huang

Question Answering

Challenge:	Large language models (LLMs) have been successful in understanding language and processing text, but their cost prohibits their practical applications.
Approach:	They propose a multi-agent collaboration method that breaks down lengthy documents into smaller, more manageable chunks and organizes the member agents to read their assigned chunks.
Outcome:	The proposed method achieves 16.42% and 1.63% accuracy gains over existing models on single-hop and multi-hop QA settings.

LLMEval-Fair: A Large-Scale Longitudinal Study on Robust and Fair Evaluation of Large Language Models (2026.acl-long)

Copied to clipboard

Ming Zhang, Yujiong Shen, Jingyi Deng, Yuhui Wang, Huayu Sha, Kexin Tan, Qiyuan Peng, Yue Zhang, Junzhe Wang, Shichun Liu, Yueyuan Huang, Jingqi Tong, Changhao Jiang, Yilong Wu, Zhihao Zhang, Mingqi Wu, Mingxu Chai, Zhiheng Xi, Shihan Dou, Tao Gui, Qi Zhang, Xuanjing Huang

Beyond Boundaries: Learning a Universal Entity Taxonomy across Datasets and Languages for Open Named Entity Recognition (2025.coling-main)

Challenge:	Existing evaluation of Large Language Models on static benchmarks is vulnerable to data contamination and leaderboard overfitting.
Approach:	LLMEval-Fair framework provides a framework for dynamic evaluation of Large Language Models . evaluators use a proprietary bank of 220k graduate-level questions to analyze model data .
Outcome:	LLMEval-Fair provides robust and credible evaluation framework for Large Language Models . it provides a strong empirical validation for the dynamic evaluation paradigm .

Copied to clipboard

Yuming Yang, Wantong Zhao, Caishuang Huang, Junjie Ye, Xiao Wang, Huiyuan Zheng, Yang Nan, Yuran Wang, Xueying Xu, Kaixin Huang, Yunke Zhang, Tao Gui, Qi Zhang, Xuanjing Huang

Reinforcement Learning on Pre-Training Data (2026.acl-long)

Challenge:	Current Large Language Models struggle with complex entity taxonomies in open domains and lack NER capabilities.
Approach:	They propose a dataset to guide LLMs' generalization in Open NER under a universal entity taxonomy.
Outcome:	The proposed model outperforms GPT-4 in 3 out-of-domain benchmarks across 15 datasets and 6 languages.

Copied to clipboard

Siheng Li, Kejiao Li, Zenan Xu, Guanhua Huang, Kun Li, Haoyuan Wu, null Wujiajia, Zihao Zheng, Chenchen Zhang, Kun Shi, Xue Gong, Qi Yi, Ruibin Xiong, Tingqiang Xu, Yuhao Jiang, Jianfeng Yan, Yuyuan Zeng, Guanghui Xu, Jinbao Xue, Zhijiang xu, Zheng Fang, Shuai LI, Qibin Liu, Xiaoxue Li, Zhuoyu Li, Yangyu Tao, Fei Gao, Cheng Jiang, Bochao Wang, Kai Liu, Jianchen Zhu, Wai Lam, Bo Zhou, Di Wang

Measuring Data Diversity for Instruction Tuning: A Systematic Analysis and A Reliable Metric (2025.acl-long)

Challenge:	Recent progress in large language models is driven by scaling of training compute through pre-training with nexttoken prediction (NTP) or post-training (RL) Pre-training using NTP enables models to acquire extensive knowledge and skills from general data, but it suffers from data inefficiency and catastrophic forgetting in continual learning settings.
Approach:	They propose to scale training compute through pre-training with next-token prediction (NTP) or post-training by scaling reinforcement learning (RL) to improve learning from general data.
Outcome:	Experiments on multiple benchmarks and models show that the proposed approach improves continual pre-training and provides a strong foundation for post-training on Qwen3-8B-Base.

Copied to clipboard

Yuming Yang, Yang Nan, Junjie Ye, Shihan Dou, Xiao Wang, Shuo Li, Huijie Lv, Tao Gui, Qi Zhang, Xuanjing Huang

CQG: A Simple and Effective Controlled Generation Framework for Multi-hop Question Generation (2022.acl-long)

Challenge:	Existing studies have explored various diversity-aware data selection methods to construct high-quality datasets and enhance model performance.
Approach:	They propose to use data diversity to measure instruction tuning of large language models.
Outcome:	The proposed diversity metric outperforms existing methods on simulated and real-world data and shows that it captures diversity variations and achieves a 0.97 correlation with instruction tuning.

Copied to clipboard

Zichu Fei, Qi Zhang, Tao Gui, Di Liang, Sirui Wang, Wei Wu, Xuanjing Huang

Text Generation

Challenge:	Current models can not ensure the complexity of generated questions, so they may generate shallow questions that can be answered without multi-hop reasoning.
Approach:	They propose a controlled framework to generate multi-hop questions that contain key entities in multi- hop reasoning chains and a novel Transformer-based decoder to guarantee that key entities appear in the questions.
Outcome:	The proposed model outperforms the state-of-the-art model 25% on HotpotQA.

Lost in Pronunciation: Detecting Chinese Offensive Language Disguised by Phonetic Cloaking Replacement (2025.emnlp-industry)

Copied to clipboard

Haotan Guo, Jianfei He, Jiayuan Ma, Hongbin Na, Zimu Wang, Haiyang Zhang, Qi Chen, Wei Wang, Zijing Shi, Tao Shen, Ling Chen

PDF-to-Tree: Parsing PDF Text Blocks into a Tree (2024.findings-emnlp)

Challenge:	Phonetic Cloaking Replacement (PCR) is a problem in content moderation in China.
Approach:	They organize PCR into a four-way surface-form taxonomy and compile PCR-ToxiCN, a dataset of 500 phonetically cloaked offensive posts gathered from the RedNote platform.
Outcome:	The proposed model achieves only an F1-score and zero-shot chain-of-thought prompting pushes performance even lower.

Copied to clipboard

Yue Zhang, Zhihao Zhang, Wenbin Lai, Chong Zhang, Tao Gui, Qi Zhang, Xuanjing Huang

Challenge:	Existing studies try to extract one universal reading order for PDF files, however, some applications, like Retrieval Augmented Generation, require breaking long articles into sections and subsections for better indexing.
Approach:	They propose a new task and dataset, PDF-to-Tree, which organizes the text blocks of a PDF into a tree structure.
Outcome:	The proposed parser achieves 93.93% accuracy, surpassing baseline methods by 6.72%.

When to Trust LLMs: Aligning Confidence with Response Quality (2024.findings-acl)

Copied to clipboard

Shuchang Tao, Liuyi Yao, Hanxing Ding, Yuexiang Xie, Qi Cao, Fei Sun, Jinyang Gao, Huawei Shen, Bolin Ding

Challenge:	Existing methods express reliability by confidence level, but lack objective guidance . Existing approaches express reliability but lack guidance on when to trust LLMs .
Approach:	They propose a reward-based approach to align confidence with quality to ensure reliability . they propose 'conqORD' to help model to verbalize greater confidence for higher quality responses .
Outcome:	Experiments show that CONQORD significantly improves confidence and response accuracy . the proposed approach can be used to determine reliability of large language models .

Transferring from Formal Newswire Domain with Hypernet for Twitter POS Tagging (D18-1)

Copied to clipboard

Tao Gui, Qi Zhang, Jingjing Gong, Minlong Peng, Di Liang, Keyu Ding, Xuanjing Huang

Syntactic Text Processing

Challenge:	Existing POS tagging methods for Twitter use labeled newswire text . however, Twitter users tend to mimic formal media expressions and develop linguistically informal styles.
Approach:	They propose to use newswire text to learn POS tagging for Twitter while twitter users are developing linguistically informal styles.
Outcome:	The proposed method achieves better performance than state-of-the-art methods on three different datasets.

GroundingGPT: Language Enhanced Multi-modal Grounding Model (2024.acl-long)

Copied to clipboard

Zhaowei Li, Qi Xu, Dong Zhang, Hang Song, YiQing Cai, Qi Qi, Ran Zhou, Junting Pan, Zefeng Li, Vu Tu, Zhida Huang, Tao Wang

Knowledge Poisoning Attacks on Medical Multi-Modal Retrieval-Augmented Generation (2026.acl-long)

Challenge:	Existing multi-modal large language models focus on capturing global information while neglecting the fine-grained local information in multimodal inputs.
Approach:	They propose an end-to-end language enhanced multi-modal grounding model that performs fine-grained grounding tasks for image, video and audio.
Outcome:	The proposed model achieves impressive fine-grained understanding of multi-modal inputs while maintaining or improving its global comprehension capabilities.

Copied to clipboard

Peiru Yang, Haoran Zheng, Tong Ju, Shiting Wang, Wanchun Ni, Jiajun Liu, Shangguang Wang, Yongfeng Huang, Tao Qi

Modeling the Q-Diversity in a Min-max Play Game for Robust Optimization (2023.findings-acl)

Challenge:	Existing studies have investigated knowledge poisoning attacks in medical RAG systems . knowledge poison attacks can disrupt model outputs and undermine system reliability .
Approach:	They propose a knowledge poisoning framework that injects misinformation into textual data . they propose to use paired visual data as a query-agnostic trigger to promote retrieval .
Outcome:	The proposed framework produces clinically plausible but incorrect generations on five LLMs and datasets.

Copied to clipboard

Ting Wu, Rui Zheng, Tao Gui, Qi Zhang, Xuanjing Huang

Attentive Pooling with Learnable Norms for Text Representation (2020.acl-main)

Challenge:	Existing methods for minimizing the worst-case loss of annotated groups are lacking in practice due to expensive annotations and privacy issues.
Approach:	They propose a distributionally robust optimization framework that relaxes group identification into direct parameterization by using an interactive training mode.
Outcome:	The proposed method outperforms state-of-the-art methods on synthetic and real-world text classification tasks.

Copied to clipboard

Chuhan Wu, Fangzhao Wu, Tao Qi, Xiaohui Cui, Yongfeng Huang

Unveiling Linguistic Regions in Large Language Models (2024.acl-long)

Challenge:	Existing pooling methods that use fixed pooling norms may not be optimal for learning text representations in different tasks.
Approach:	They propose to learn pooling norms in an end-to-end manner to automatically find the optimal ones for text representation in different tasks.
Outcome:	The proposed approach improves on four benchmark datasets on a neural NLP model.

Copied to clipboard

Zhihao Zhang, Jun Zhao, Qi Zhang, Tao Gui, Xuanjing Huang

HieRec: Hierarchical User Interest Modeling for Personalized News Recommendation (2021.acl-long)

Challenge:	Existing studies on how LLMs achieve cross-lingual alignment and generalization have not explored the intrinsic mechanisms of how they achieve crosslingual alignment.
Approach:	They propose to remove a core region that corresponds to linguistic competence and set parameters to zero to reduce performance across 30 different languages.
Outcome:	The proposed model can be used to perform tasks requiring abstract knowledge and reasoning in complex languages.

Copied to clipboard

Tao Qi, Fangzhao Wu, Chuhan Wu, Peiru Yang, Yang Yu, Xing Xie, Yongfeng Huang

RethinkingTMSC: An Empirical Study for Target-Oriented Multimodal Sentiment Classification (2023.findings-emnlp)

Challenge:	Existing news recommendation methods learn a single user embedding for each user from their previous behaviors to represent their overall interest. Existing methods only learn 'one' embeddable representation vectors to model user interest.
Approach:	They propose a news recommendation method with hierarchical user interest modeling that captures user interest in news rather than a single user embedding.
Outcome:	The proposed method can better capture multi-grained user interest in news.

Copied to clipboard

Junjie Ye, Jie Zhou, Junfeng Tian, Rui Wang, Qi Zhang, Tao Gui, Xuanjing Huang

Information Retrieval

Challenge:	Recent studies have shown that current TMSC systems rely on textual information, and the progress in tackling this task has slowed down.
Approach:	They propose to integrate both visual and textual information to improve the performance of TMSC by considering multimodal information.
Outcome:	The proposed model integrates both visual and textual information to improve performance.

PlugAT: A Plug and Play Module to Defend against Textual Adversarial Attack (2022.coling-1)

Copied to clipboard

Rui Zheng, Rong Bao, Qin Liu, Tao Gui, Qi Zhang, Xuanjing Huang, Rui Xie, Wei Wu

Subspace Defense: Discarding Adversarial Perturbations by Learning a Subspace for Clean Signals (2024.lrec-main)

Challenge:	Recent studies have shown that adversarial examples can be easily fooled by adversarially perturbed examples.
Approach:	They propose a pluggable defense module PlugAT to provide robust predictions by adding a few trainable parameters to the model inputs while keeping the original model frozen.
Outcome:	The proposed model improves robustness over several strong baselines whilst training only 9.1% parameters.

Copied to clipboard

Rui Zheng, Yuhao Zhou, Zhiheng Xi, Tao Gui, Qi Zhang, Xuanjing Huang

Detecting Adversarial Samples through Sharpness of Loss Landscape (2023.findings-acl)

Challenge:	Existing models that extract discrete inputs into fixed-length representations are vulnerable to adversarial attacks that place perturbations on clean inputs to fool DNNs.
Approach:	They propose to inspect the subspaces of sample features through spectral analysis to better understand adversarial attacks.
Outcome:	The proposed strategy enables the model to inherently suppress adversaries, which boosts model robustness and motivates new directions of effective adversarial defense.

Copied to clipboard

Rui Zheng, Shihan Dou, Yuhao Zhou, Qin Liu, Tao Gui, Qi Zhang, Zhongyu Wei, Xuanjing Huang, Menghan Zhang

Read Extensively, Focus Smartly: A Cross-document Semantic Enhancement Method for Visual Documents NER (2022.coling-1)

Challenge:	Existing studies have shown that adversarial samples are more vulnerable than normal ones to textual adversarials.
Approach:	They propose a simple and effective sharpness-based detector that can distinguish adversarial samples by maximizing the loss increment within the region where the inference sample is located.
Outcome:	The proposed method outperforms previous detection methods by large margins on three text classification tasks.

Copied to clipboard

Jun Zhao, Xin Zhao, WenYu Zhan, Tao Gui, Qi Zhang, Liang Qiao, Zhanzhan Cheng, Shiliang Pu

Visual Data in NLP

Challenge:	Existing methods to extract entities from visually-rich documents ignore the inherent multimodality of VRDs and thus the suboptimal results are achieved.
Approach:	They propose a multimodal semantic enhancement method that filters redundant information in the current document and a cross-document information awareness technique to enrich the entity-related context.
Outcome:	The proposed method outperforms existing methods on two documents understanding benchmarks covering eight languages.

Low-Resource Dialogue Summarization with Domain-Agnostic Multi-Source Pretraining (2021.emnlp-main)

Copied to clipboard

Yicheng Zou, Bolin Zhu, Xingwu Hu, Tao Gui, Qi Zhang

P4: Plug-and-Play Discrete Prompting for Large Language Models Personalization (2024.findings-acl)

Challenge:	Existing methods for low-resource dialogue summarization neglect the difference between dialogues and conventional articles.
Approach:	They propose a multi-source pretraining paradigm to leverage external summary data . they exploit large-scale in-domain non-summary data to separate dialogue encoder and summary decoder .
Outcome:	The proposed model can be used to better leverage external summary data.

Copied to clipboard

Yuansen Zhang, Xiao Wang, Tianze Chen, Jiayi Fu, Tao Gui, Qi Zhang

Connectivity Patterns are Task Embeddings (2023.findings-acl)

Challenge:	Large Language Models (LLMs) exhibit impressive capabilities in following instructions, but manually prompting them to exhibit certain personalities may result in sub-optimal performance.
Approach:	They propose a plug-and-play prompting method to manipulate Large Language Models with distinct human-like personality traits by appending discrete personalized suffixes to query or dialog histories and focusing exclusively on influential tokens.
Outcome:	The proposed method outperforms other prompting methods and model editing methods on four models ranging from 1.1B to 13B and achieves 79.9% accuracy in customizing LLMs’ personalities.

Copied to clipboard

Zhiheng Xi, Rui Zheng, Yuansen Zhang, Xuanjing Huang, Zhongyu Wei, Minlong Peng, Mingming Sun, Qi Zhang, Tao Gui

Reward Modeling Requires Automatic Adjustment Based on Data Quality (2024.findings-emnlp)

Challenge:	Existing methods for predicting inter-task transferability are sparse and task-specific.
Approach:	They propose a method that uses connectivity patterns of neurons as a unique identifier associated with a task.
Outcome:	The proposed method outperforms baselines in predicting inter-task transferability across data regimes and transfer settings while keeping high efficiency in computation and storage.

Copied to clipboard

Binghai Wang, Rui Zheng, Lu Chen, Zhiheng Xi, Wei Shen, Yuhao Zhou, Dong Yan, Tao Gui, Qi Zhang, Xuanjing Huang

Challenge:	Reinforcement Learning from Human Feedback (RLHF) is a method for aligning language models with human values.
Approach:	They propose a method that automatically adjusts reward modeling based on data quality . they use preference data to train a reward model that is more aligned with human values .
Outcome:	The proposed method stabilizes reward model training and significantly improves alignment performance on human preference datasets.

Characterizing the Impacts of Instances on Robustness (2023.findings-acl)

Copied to clipboard

Rui Zheng, Zhiheng Xi, Qin Liu, Wenbin Lai, Tao Gui, Qi Zhang, Xuanjing Huang, Jin Ma, Ying Shan, Weifeng Ge

Open Set Relation Extraction via Unknown-Aware Training (2023.acl-long)

Challenge:	Existing defense approaches focus on developing new model structures or training algorithms, but they do little to tap the potential of training instances.
Approach:	They propose a method that can distinguish between robust and non-robust instances according to the model’s sensitivity to perturbations on individual instances during training.
Outcome:	The proposed method can distinguish between robust and non-robust instances according to the model’s sensitivity to perturbations on individual instances during training.

Copied to clipboard

Jun Zhao, Xin Zhao, WenYu Zhan, Qi Zhang, Tao Gui, Zhongyu Wei, Yun Wen Chen, Xiang Gao, Xuanjing Huang

Controllable Contamination Detection for Reliable LLM Evaluation with Statistical Guarantees (2026.acl-long)

Challenge:	Existing supervised relation extraction methods can still misclassify unknown relations into known relations due to the lack of supervision signals.
Approach:	They propose a method that regularizes the model by dynamically synthesizing negative instances that can provide the missing supervision signals.
Outcome:	The proposed method achieves SOTA unknown relation detection without compromising the classification of known relations.

Copied to clipboard

Zheng Zhang, Qi Liu, Siyuan Liang, Ning Li, Zirui Hu, Weibo Gao, Rui Li, Zhenya Huang, Leszek Rutkowski, Baosheng Yu, Dacheng Tao

Multi-stage Distillation Framework for Cross-Lingual Semantic Similarity Matching (2022.findings-naacl)

Challenge:	Existing training data detectors fail to detect clean samples from contaminated test sets . existing methods fail to identify clean samples due to black-box nature of LLMs .
Approach:	They propose a framework that detects and filters contaminated evaluation data . they propose 'failure detection' to reduce the proportion of contaminated samples mistakenly retained .
Outcome:	The proposed framework reduces false discovery rate (FDR) under valid FDR control while maintaining evaluation consistency.

Copied to clipboard

Kunbo Ding, Weijie Liu, Yuejian Fang, Zhe Zhao, Qi Ju, Xuefeng Yang, Rong Tian, Zhu Tao, Haoyan Liu, Han Guo, Xingyu Bai, Weiquan Mao, Yudong Li, Weigang Guo, Taiqiang Wu, Ningyuan Sun

ToolSword: Unveiling Safety Issues of Large Language Models in Tool Learning Across Three Stages (2024.acl-long)

Challenge:	Existing studies have shown that cross-lingual knowledge distillation can improve the performance of pre-trained models for cross-linguistic similarity matching tasks.
Approach:	They propose a multi-stage distillation framework for constructing a small-size but high-performance cross-lingual model using contrastive learning, bottleneck, and parameter recurrent strategies.
Outcome:	The proposed model can compress the size of XLM-R and MiniLM by more than 50% while the performance is only reduced by about 1%.

Copied to clipboard

Junjie Ye, Sixian Li, Guanyu Li, Caishuang Huang, Songyang Gao, Yilong Wu, Qi Zhang, Tao Gui, Xuanjing Huang

Black-Box Membership Inference Attacks for Video Training Data in Multimodal Large Language Models (2026.acl-long)

Challenge:	Existing research focuses on enhancing LLMs capabilities through tool utilization.
Approach:	They propose a framework to investigate safety issues in large language models in tool learning . they propose malicious queries and jailbreak attacks in the input stage .
Outcome:	The proposed framework investigates six safety scenarios for LLMs in tool learning . the data will be released upon acceptance of the proposed framework .

Copied to clipboard

Jinrui Wang, Zhenfeng Gao, Wendan Wang, Huili Wang, Zichen Qin, Linjie Zhu, Hongke Fu, Shangguang Wang, Tao Qi

Robust Lottery Tickets for Pre-trained Language Models (2022.acl-long)

Challenge:	Existing methods assess model memorization of key semantic concepts within a video but do not provide reliable evidence that a specific video was used during training.
Approach:	They propose a black-box MIA framework that can provide reliable evidence of specific video data usage for training multimodal large language models.
Outcome:	The proposed framework can provide reliable evidence of specific video data usage for training multimodal large language models.

Copied to clipboard

Rui Zheng, Bao Rong, Yuhao Zhou, Di Liang, Sirui Wang, Wei Wu, Tao Gui, Qi Zhang, Xuanjing Huang

SDAR-VL: Stable and Efficient Block-wise Diffusion for Vision-Language Understanding (2026.acl-long)

Challenge:	Recent studies have shown that pre-trained language models contain smaller matching subnetworks that are not robust to adversarial examples.
Approach:	They propose a method to find robust tickets hidden in pre-trained language models by learning binary weight masks and an adversarial loss objective to guide the search.
Outcome:	The proposed method improves on previous work on adversarial robustness evaluation.

Copied to clipboard

Shuang Cheng, Yuhua Jiang, Zineng Zhou, Dawei Liu, Tao Wang, Linfeng Zhang, Biqing Qi, Bowen Zhou

ORTicket: Let One Robust BERT Ticket Transfer across Different Tasks (2024.lrec-main)

Challenge:	Existing block-wise discrete diffusion models lack robust autoregressive (AR) decoders.
Approach:	They propose a block-wise discrete diffusion framework for large-scale vision-language understanding with a progressive beta noise curriculum.
Outcome:	The proposed framework improves training efficiency, convergence stability, and task performance over conventional block diffusion.

Copied to clipboard

Yuhao Zhou, Wenxiang Chen, Rui Zheng, Zhiheng Xi, Tao Gui, Qi Zhang, Xuanjing Huang

TextFusion: Privacy-Preserving Pre-trained Model Inference via Token Fusion (2022.emnlp-main)

Challenge:	Pretrained language models are susceptible to subtle perturbations and require multiple adversarial training during fine-tuning to improve their robustness.
Approach:	They propose a novel adversarial defense method ORTicket that fine-tunes a model for downstream tasks.
Outcome:	The proposed method achieves comparable robustness to other defense methods while maintaining the efficiency of fine-tuning.

Copied to clipboard

Xin Zhou, Jinzhu Lu, Tao Gui, Ruotian Ma, Zichu Fei, Yuran Wang, Yong Ding, Yibo Cheung, Qi Zhang, Xuanjing Huang

Toward Optimal LLM Alignments Using Two-Player Games (2025.findings-emnlp)

Challenge:	Existing methods to preserve inference privacy are available as cloud services . however, the risk of privacy leakage remains, according to recent studies .
Approach:	They propose a method to preserve inference privacy by fusing token representations in the cloud.
Outcome:	The proposed method preserves inference privacy without sacrificing performance on different scenarios.

Copied to clipboard

Rui Zheng, Hongyi Guo, Zhihan Liu, Xiaoying Zhang, Yuanshun Yao, Xiaojun Xu, Zhaoran Wang, Zhiheng Xi, Tao Gui, Qi Zhang, Xuanjing Huang, Yang Liu, Hang Li

Learning “O” Helps for Learning More: Handling the Unlabeled Entity Problem for Class-incremental NER (2023.acl-long)

Challenge:	Alignment of large language models (LLM) is a process that ensures the model’s responses to user prompts align with human intentions and social values.
Approach:	They propose an alignment method based on a two-agent game consisting of an adversarial agent and a defensive agent.
Outcome:	The proposed method improves on a two-agent game with an adversarial agent and a defensive agent.

Copied to clipboard

Ruotian Ma, Xuanting Chen, Zhang Lin, Xin Zhou, Junzhe Wang, Tao Gui, Qi Zhang, Xiang Gao, Yun Wen Chen

Hi-Transformer: Hierarchical Interactive Transformer for Efficient and Effective Long Document Modeling (2021.acl-short)

Challenge:	Existing Named Entity Recognition systems are typically trained on a large-scale dataset with predefined entity classes, then deployed for entity recognition on the test data without further adaptation or refinement.
Approach:	They propose a representation learning method that adaptively detects entity clusters in "O" and two effective distance-based relabeling strategies for better learning the old classes.
Outcome:	The proposed method achieves 10.62% improvement over the baseline methods.

Copied to clipboard

CASN:Class-Aware Score Network for Textual Adversarial Detection (2023.acl-long)

Challenge:	Existing approaches to model long documents are difficult due to the quadratic complexity of text length.
Approach:	They propose a hierarchical interactive Transformer for efficient long document modeling.
Outcome:	Extensive experiments on three benchmark datasets validate the efficiency and effectiveness of Hi-Transformer in long document modeling.

Copied to clipboard

Rong Bao, Rui Zheng, Liang Ding, Qi Zhang, Dacheng Tao

Enhancing LLM-based Search Agents via Contribution Weighted Group Relative Policy Optimization (2026.acl-long)

Challenge:	Existing approaches to counteract adversarial attacks can be divided into two directions, adversarials defense and adversarially detection.
Approach:	They propose a score-based generative method to implicitly model the data distribution using a log-density distribution and supervised contrastive learning to guide the estimation using label information.
Outcome:	The proposed method improves on three text classification tasks on four advanced attack algorithms.

Copied to clipboard

Junzhe Wang, Zhiheng Xi, Yajie Yang, Hao Luo, Shihan Dou, Tao Gui, Qi Zhang

Information Retrieval

Challenge:	Existing approaches to training large language models suffer from unstable value estimation, whereas outcome supervision struggles with credit assignment due to sparse, trajectory-level rewards.
Approach:	They propose a framework that integrates process supervision into group relative policy optimization.
Outcome:	The proposed framework outperforms standard GRPO on knowledge-intensive benchmarks by 5.0% and 6.3% on Qwen3-1.7B.

Which Reasoning Trajectories Teach Students to Reason Better? A Simple Metric of Informative Alignment (2026.acl-long)

Copied to clipboard

Yuming Yang, Mingyoung Lai, Wanxu Zhao, Xiaoran Fan, Zhiheng Xi, Mingqi Wu, Chiyue Huang, Jun Zhao, Haijun Lv, Jian Tong, Yunhua Zhou, Yicheng Zou, Qipeng Guo, Tao Gui, Qi Zhang, Xuanjing Huang

Two Birds with One Stone: Unified Model Learning for Both Recall and Ranking in News Recommendation (2022.findings-acl)

Challenge:	Existing methods assess suitability primarily through student likelihood, favoring trajectories that align closely with the student model’s current behavior but overlooking more informative ones.
Approach:	They propose a Rank–Surprisal Ratio metric that captures both alignment and informativeness to assess the suitability of a reasoning trajectory.
Outcome:	The proposed metric captures both alignment and informativeness to assess the suitability of a reasoning trajectory.

Copied to clipboard

Heterogeneous Graph Neural Networks for Keyphrase Generation (2021.emnlp-main)

Challenge:	Existing news recommender systems conduct news recall and ranking separately with different models, but maintaining multiple models leads to high computational cost and high latency.
Approach:	They propose a unified method for recall and ranking in news recommendation that uses historical news click behaviors to extract user embeddings for ranking from the user's attention query.
Outcome:	The proposed method improves recall and ranking efficiency and effectiveness on a benchmark dataset.

Copied to clipboard

Jiacheng Ye, Ruijian Cai, Tao Gui, Qi Zhang

Efficient Adversarial Training with Robust Early-Bird Tickets (2022.emnlp-main)

Challenge:	Existing approaches for keyphrase generation generate uncontrollable and inaccurate absent keyphrases.
Approach:	They propose a graph-based method that captures explicit knowledge from related references.
Outcome:	The proposed model improves on baseline keyphrase generation models on multiple benchmarks.

Copied to clipboard

Zhiheng Xi, Rui Zheng, Tao Gui, Qi Zhang, Xuanjing Huang

UER: An Open-Source Toolkit for Pre-training Models (D19-3)

Challenge:	Existing methods to improve the robustness of pre-trained language models are expensive because of the need to generate adversarial examples via gradient descent.
Approach:	They propose an adversarial optimization method that searches for robust tickets with structured sparsity in the early stage and fine-tunes tickets in the remaining time.
Outcome:	The proposed method achieves up to 7 13 training speedups while maintaining comparable or even better robustness compared to the most competitive state-of-the-art methods.

Copied to clipboard

Zhe Zhao, Hui Chen, Jinbin Zhang, Xin Zhao, Tao Liu, Wei Lu, Xi Chen, Haotang Deng, Qi Ju, Xiaoyong Du

Modeling Layout Reading Order as Ordering Relations for Visually-rich Document Understanding (2024.emnlp-main)

Challenge:	Existing work on pre-training models have shown that it is important to use a framework to deploy various pre- training models efficiently.
Approach:	They propose an assemble-on-demand pre-training toolkit that assembles pre-trained models on demand and encapsulates them with rich modules.
Outcome:	The proposed framework can reproduce state-of-the-art models or develop models that remain unexplored.

Copied to clipboard

Chong Zhang, Yi Tu, Yixi Zhao, Chenshu Yuan, Huan Chen, Yue Zhang, Mingxu Chai, Ya Guo, Huijia Zhu, Qi Zhang, Tao Gui

MM-Doc-R1: Training Agents for Long Document Visual Question Answering through Multi-turn Reinforcement Learning (2026.findings-acl)

Challenge:	Existing models of layout reading order do not convey the complete reading order information in the layout.
Approach:	They propose to model layout reading order as ordering relations over layout elements . they propose a reading-order-relation-enhancing pipeline to improve model performance .
Outcome:	The proposed model outperforms existing models on a visual-rich document dataset and on eight cross-domain VrD-IE/QA tasks without targeted optimization.

Copied to clipboard

Jiahang Lin, Kai Hu, Binghai Wang, Yuhao Zhou, Zhiheng Xi, Honglin Guo, Shichun Liu, Junzhe Wang, Shihan Dou, Enyu Zhou, Hang Yan, Zhenhua Han, Tao Gui, Qi Zhang, Xuanjing Huang

Visual Data in NLP

Challenge:	Existing work on long document visual question answering is based on Retrieval-Augmented Generation (RAG) where textual or visual content is encoded into embeddings and relevance is determined by similarity scores with respect to the original query.
Approach:	They propose a framework that employs an agentic, vision-aware workflow to address long document visual question answering through iterative information discovery and synthesis.
Outcome:	The proposed framework outperforms existing RL systems by 10.4% on the MMLongbench-Doc benchmark and demonstrates superior training performance over GRPO.

LLMEval-Med: A Real-world Clinical Benchmark for Medical LLMs with Physician Validation (2025.findings-emnlp)

Copied to clipboard

Ming Zhang, Yujiong Shen, Zelin Li, Huayu Sha, Binze Hu, Yuhui Wang, Chenhao Huang, Shichun Liu, Jingqi Tong, Changhao Jiang, Mingxu Chai, Zhiheng Xi, Shihan Dou, Tao Gui, Qi Zhang, Xuanjing Huang

TL-Training: A Task-Feature-Based Framework for Training Large Language Models in Tool Use (2025.findings-emnlp)

Challenge:	Current medical benchmarks have limitations in question design, data sources and evaluation methods.
Approach:	They propose a new benchmark covering five core medical areas . it includes 2,996 questions created from real-world electronic health records .
Outcome:	The proposed model covers five core medical areas and includes 2,996 questions created from real-world electronic health records and expert-designed clinical scenarios.

Copied to clipboard

Junjie Ye, Yilong Wu, Sixian Li, Yuming Yang, Zhiheng Xi, Tao Gui, Qi Zhang, Xuanjing Huang, Peng Wang, Zhongchao Shi, Jianping Fan, Zhengyin Du

PTUM: Pre-training User Model from Unlabeled User Behaviors via Self-supervision (2020.findings-emnlp)

Challenge:	a new approach to training large language models (LLMs) overlooks task-specific characteristics in tool use, leading to performance bottlenecks.
Approach:	They propose a task-feature-based framework that mitigates the effects of suboptimal training data . they use a dataset to train large-scale LLMs and a reward mechanism tailored to error categories .
Outcome:	The proposed framework matches or surpasses open- and closed-source LLMs in tool-use performance using only 1,217 training data points.

Copied to clipboard

Chuhan Wu, Fangzhao Wu, Tao Qi, Jianxun Lian, Yongfeng Huang, Xing Xie

ToolEyes: Fine-Grained Evaluation for Tool Learning Capabilities of Large Language Models in Real-world Scenarios (2025.coling-main)

Challenge:	Existing methods for user modeling cannot exploit useful information in unlabeled data . Existing models only model task-specific user information and do not exploit universal user information encoded in user behaviors.
Approach:	They propose to pre-train user models from large-scale unlabeled user behavior data.
Outcome:	The proposed method can model relatedness between historical and future behaviors on two real-world datasets.

Copied to clipboard

Junjie Ye, Guanyu Li, SongYang Gao, Caishuang Huang, Yilong Wu, Sixian Li, Xiaoran Fan, Shihan Dou, Tao Ji, Qi Zhang, Tao Gui, Xuanjing Huang

Robust Membership Inference for Large Language Models under Adversarial Generative Corruption (2026.acl-long)

Challenge:	Existing evaluations of tool learning focus on validation of tools for large language models with expected outcomes, but this focus ignores the complex capabilities required for LLMs to effectively use tools.
Approach:	They propose a fine-grained system for evaluation of large language models’ tool learning capabilities in authentic scenarios.
Outcome:	The proposed system examines seven real-world scenarios, analyzing five dimensions crucial to LLMs in tool learning: format alignment, intent comprehension, behavior planning, tool selection, and answer organization.

Copied to clipboard

Yuanhong Huang, Huili Wang, Xueying Bai, Jinrui Wang, Jiajun Liu, Ziqin Wang, Wanchun Ni, Shangguang Wang, Tao Qi

PP-Rec: News Recommendation with Personalized User Interest and Time-aware News Popularity (2021.acl-long)

Challenge:	Membership inference attacks are a promising tool for auditing training data of LLMs . existing methods rely on the assumption that LLM's assign higher confidence scores to training samples than to non-training ones.
Approach:	They propose a membership inference framework that can be robust against adversarial MIAs.
Outcome:	The proposed framework can be robust against adversarial MIA methods and AIGT detectors while maintaining the performance of baselines.

Copied to clipboard

Tao Qi, Fangzhao Wu, Chuhan Wu, Yongfeng Huang

Challenge:	Existing personalized news recommendation methods have difficulties in making accurate recommendations to cold-start users.
Approach:	They propose to incorporate news popularity information to improve cold-start recommendations . they propose to use a popularity-aware user encoder to eliminate popularity bias .
Outcome:	The proposed method improves accuracy and diversity of personalized news recommendation on two real-world datasets.

SentiRec: Sentiment Diversity-aware Neural News Recommendation (2020.aacl-main)

Copied to clipboard

Neural News Recommendation with Multi-Head Self-Attention (D19-1)

Sentiment Analysis

Challenge:	Existing news recommendation methods rank candidate news based on relevance to users’ historical browsed news, but if browsed data is dominated by certain kinds of sentiment, the model may recommend news with the same sentiment.
Approach:	They propose a sentiment diversity-aware neural news recommendation approach which can recommend news with more diverse sentiment without performance sacrifices.
Outcome:	The proposed approach can improve the sentiment diversity in news recommendation without performance sacrifice.

Copied to clipboard

Chuhan Wu, Fangzhao Wu, Suyu Ge, Tao Qi, Yongfeng Huang, Xing Xie

Parrot: A Training Pipeline Enhances Both Program CoT and Natural Language CoT for Reasoning (2025.emnlp-main)

Challenge:	Precisely modeling news and users is critical for news recommendation, and capturing the contexts of words and news is important to learn news and user representations.
Approach:	They propose a neural news recommendation approach with multi-head self-attention to model the interactions between words and news and use multi-headed self- attention to capture relatedness between the news.
Outcome:	The proposed approach can learn representations from news titles by modeling the interactions between words and users and capture relatedness between the news.

Copied to clipboard

Senjie Jin, Lu Chen, Zhiheng Xi, Yuhui Wang, Sirui Song, Yuhao Zhou, Xinbo Zhang, Peng Sun, Hong Lu, Tao Gui, Qi Zhang, Xuanjing Huang

RoTBench: A Multi-Level Benchmark for Evaluating the Robustness of Large Language Models in Tool Learning (2024.emnlp-main)

Challenge:	Existing work focuses on enabling models to generate natural language chain-of-thought rationales or leverage executable and verifiable code, such as Python.
Approach:	They propose a novel training pipeline that integrates sequential P-CoT and N-Co T generation and a subtask hybrid training strategy to facilitate natural language transferability.
Outcome:	The proposed training pipeline improves both N-CoT and P-Co T performance over the RL baseline.

Copied to clipboard

Junjie Ye, Yilong Wu, Songyang Gao, Caishuang Huang, Sixian Li, Guanyu Li, Xiaoran Fan, Qi Zhang, Tao Gui, Xuanjing Huang

Actively Supervised Clustering for Open Relation Extraction (2023.acl-long)

Challenge:	Current research emphasizes LLMs’ capacity to utilize tools in well-structured environments while overlooking their stability when confronted with the inevitable noise of the real world.
Approach:	They propose a multi-level benchmark to evaluate the robustness of large language models in tool learning by establishing five external environments with varying levels of noise.
Outcome:	The proposed model outperforms the GPT-4 model in tool learning in three critical phases: tool selection, parameter identification, and content filling.

Copied to clipboard

Jun Zhao, Yongxin Zhang, Qi Zhang, Tao Gui, Zhongyu Wei, Minlong Peng, Mingming Sun

Lost in the Context: Insufficient and Distracted Attention to Contexts in Preference Modeling (2025.acl-long)

Challenge:	Existing methods for Open Relation Extraction (OpenRE) use a two-stage pipeline, which learns relation representations and assignments in the first stage, then manually labels relation for each cluster.
Approach:	They propose a method that performs relation learning and relation labeling simultaneously without a significant increase in human effort.
Outcome:	The proposed method improves existing SOTA methods by 13.8% and 10.6% on two datasets.

Copied to clipboard

Shihan Dou, Jiayi Chen, Chenhao Huang, Feng Chen, Wei Chengzhi, Huiyuan Zheng, Shichun Liu, Yan Liu, Chenxiao Liu, Chao Xin, Lin Yan, Zongzhang Zhang, Tao Gui, Qi Zhang, Xuanjing Huang

Challenge:	Existing reward models concatenate contexts and responses, but they often ignore crucial segments of the context that are important for evaluating the response quality.
Approach:	They propose a reward model that evaluates the response quality based on a given context and assigns a rewards reward.
Outcome:	The proposed framework significantly improves preference modeling by increasing attention to relevant information within the context and achieves better generalizability.

RoCoIns: Enhancing Robustness of Large Language Models through Code-Style Instructions (2024.lrec-main)

Copied to clipboard

Yuansen Zhang, Xiao Wang, Zhiheng Xi, Han Xia, Tao Gui, Qi Zhang, Xuanjing Huang

Measuring Human Contribution in AI-Assisted Content Generation (2026.acl-long)

Challenge:	Large Language Models (LLMs) have shown remarkable capabilities in following human instructions and solving NLU tasks.
Approach:	They propose to use code style instructions to replace typically natural language instructions to provide more precise instructions and strengthen the robustness of LLMs.
Outcome:	The proposed method outperforms natural language models on eight robustness datasets and achieves an improvement of 5.68% in test set accuracy and a reduction of 5.66 points in Attack Success Rate (ASR).

Copied to clipboard

Yueqi Xie, Tao Qi, Jingwei Yi, Xiyuan Yang, Ryan Whalen, Junming Huang, Qian Ding, Yu Xie, Xing Xie, Fangzhao Wu

Text Generation

Challenge:	generative AI has created a new way to generate content with humans . varying degrees of human contribution in content generation poses significant challenges for the delineation of originality .
Approach:	They propose a framework to measure human contribution in AI-assisted content generation by calculating mutual information between human input and AI-aided output relative to self-information of AI-assist output.
Outcome:	The proposed measure discriminates between varying degrees of human contribution across multiple creative domains and is validated in real-world applications.

Self-Demos: Eliciting Out-of-Demonstration Generalizability in Large Language Models (2024.findings-naacl)

Copied to clipboard

Wei He, Shichun Liu, Jun Zhao, Yiwen Ding, Yi Lu, Zhiheng Xi, Tao Gui, Qi Zhang, Xuanjing Huang

Making Harmful Behaviors Unlearnable for Large Language Models (2024.findings-acl)

Challenge:	Existing methods that rely on limited demos and out-of-demonstration (OOD) queries fail when faced with out- of-demotion queries.
Approach:	They propose a query-aware prompting method that elicits the inherent generalizability of large language models by query-based demo generation.
Outcome:	The proposed method outperforms state-of-the-art methods in the OOD setting and two public math benchmarks.

Copied to clipboard

Xin Zhou, Yi Lu, Ruotian Ma, Yujian Wei, Tao Gui, Qi Zhang, Xuanjing Huang

SENT: Sentence-level Distant Relation Extraction via Negative Training (2021.acl-long)

Challenge:	Large language models (LLMs) are often customized by fine-tuning for the requirements of different domains.
Approach:	They propose a controllable training framework to make undesired behaviors unlearnable during the fine-tuning process.
Outcome:	The proposed framework makes undesired behaviors unlearnable during the fine-tuning process while preserving the ability to learn other information.

Copied to clipboard

Ruotian Ma, Tao Gui, Linyang Li, Qi Zhang, Xuanjing Huang, Yaqian Zhou

UniDrop: A Simple yet Effective Technique to Improve Transformer without Extra Cost (2021.naacl-main)

Challenge:	Existing methods for relation extraction use bag labels, which introduce noise, to train the model.
Approach:	They propose to use negative training to train a model using complementary labels to separate the noisy data from the training data.
Outcome:	The proposed method improves on previous methods on sentence-level evaluation and de-noise effect.

Copied to clipboard

Zhen Wu, Lijun Wu, Qi Meng, Yingce Xia, Shufang Xie, Tao Qin, Xinyu Dai, Tie-Yan Liu

StepCoder: Improving Code Generation with Reinforcement Learning from Compiler Feedback (2024.acl-long)

Challenge:	Existing approaches to improve the performance of natural language processing models are over-parameterized and overfitted.
Approach:	They propose an approach to integrate dropout techniques into the training of Transformer models.
Outcome:	The proposed approach can achieve 1.5 BLEU improvement on IWSLT14 translation tasks and better accuracy for the classification even using strong pre-trained RoBERTa as backbone.

Copied to clipboard

Shihan Dou, Yan Liu, Haoxiang Jia, Enyu Zhou, Limao Xiong, Junjie Shan, Caishuang Huang, Xiao Wang, Xiaoran Fan, Zhiheng Xi, Yuhao Zhou, Tao Ji, Rui Zheng, Qi Zhang, Tao Gui, Xuanjing Huang

Programming Languages in NLP

Challenge:	Existing work integrates reinforcement learning with compiler feedback to enhance code generation quality but the long code generated by LLMs makes RL exploration ineffective.
Approach:	They propose a framework that integrates reinforcement learning and compiler feedback to enhance code generation quality.
Outcome:	The proposed framework outperforms state-of-the-art approaches in corresponding benchmarks and integrates reinforcement learning with compiler feedback to improve code generation quality.

Towards Economical Inference: Enabling DeepSeek’s Multi-Head Latent Attention in Any Transformer-based LLMs (2025.acl-long)

Copied to clipboard

Tao Ji, Bin Guo, Yuanbin Wu, Qipeng Guo, Shenlixing Shenlixing, Chenzhan Chenzhan, Xipeng Qiu, Qi Zhang, Tao Gui

A Relation-Oriented Clustering Method for Open Relation Extraction (2021.emnlp-main)

Challenge:	Multi-head Latent Attention (MLA) is an innovative architecture designed to ensure efficient and economical inference by significantly compressing the Key-Value (KV) cache into a latent vector.
Approach:	They propose a data-efficient fine-tuning method for transitioning from MHA to MLA using a latent vector cache.
Outcome:	The proposed architecture reduces the KV cache size of Llama2-7B by 92.19%, with only 1% drop in LongBench performance.

Copied to clipboard

Jun Zhao, Tao Gui, Qi Zhang, Yaqian Zhou

MIND: A Large-scale Dataset for News Recommendation (2020.acl-main)

Challenge:	Existing methods for open relation extraction (OpenRE) are designed for predefined relations, which cannot deal with new emerging relations in the real world.
Approach:	They propose a relation-oriented clustering model that leverages readily available labeled data to learn a relationship-oriented representation.
Outcome:	The proposed model reduces error rate by 29.2% and 15.7% on two datasets compared with current SOTA methods.

Copied to clipboard

Fangzhao Wu, Ying Qiao, Jiun-Hung Chen, Chuhan Wu, Tao Qi, Jianxun Lian, Danyang Liu, Xing Xie, Jianfeng Gao, Winnie Wu, Ming Zhou

RealBehavior: A Framework for Faithfully Characterizing Foundation Models’ Human-like Behavior Mechanisms (2023.findings-emnlp)

Challenge:	Personalized news recommendation is an important technique for personalized news service.
Approach:	They propose to build a large-scale news recommendation dataset from Microsoft News . they demonstrate that news recommendation relies on the quality of news content understanding .
Outcome:	The proposed dataset contains 1 million users and more than 160k English news articles, each of which has rich textual content such as title, abstract and body.

Copied to clipboard

Enyu Zhou, Rui Zheng, Zhiheng Xi, Songyang Gao, Xiaoran Fan, Zichu Fei, Jingting Ye, Tao Gui, Qi Zhang, Xuanjing Huang

Challenge:	Existing studies on human-like behaviors in foundation models do not verify their faithfulness . a simple application of psychological tools cannot faithfully characterize all human-type behaviors .
Approach:	They propose a framework to characterize humanoid behaviors in foundation models . they argue that a simple application of psychological tools cannot faithfully characterize all human-like behaviors .
Outcome:	The proposed framework assesses the faithfulness of results based on reproducibility, internal consistency, and generalizability.

Divide and Conquer: Text Semantic Matching with Disentangled Keywords and Intents (2022.findings-acl)

Copied to clipboard

Yicheng Zou, Hongwei Liu, Tao Gui, Junzhe Wang, Qi Zhang, Meng Tang, Haixiang Li, Daniell Wang

Analyzing the Effects of Supervised Fine-Tuning on Model Knowledge from Token and Parameter Levels (2025.emnlp-main)

Challenge:	Existing text semantic matching models do not provide granularity for text comparison.
Approach:	They propose a simple yet effective training strategy for text semantic matching by disentangling keywords from intents.
Outcome:	The proposed approach achieves stable performance improvements against a wide range of models on three benchmarks.

Copied to clipboard

Junjie Ye, Yuming Yang, Yang Nan, Shuo Li, Qi Zhang, Tao Gui, Xuanjing Huang, Peng Wang, Zhongchao Shi, Jianping Fan

VRPO: Rethinking Value Modeling for Robust RL under Noisy Supervision in LLM Post-Training (2026.acl-long)

Challenge:	Large language models (LLMs) acquire substantial world knowledge during pretraining, which is further shaped by post-training techniques such as supervised fine-tuning (SFT).
Approach:	They evaluate closed-book question answering (CBQA) performance across five LLMs from the LLaMA-2 and LLama-3 families and examine the impact of supervised fine-tuning on model knowledge.
Outcome:	The proposed model performance is 14% worse than models fine-tuned on 1,920 samples and 12% worse on 240 samples.

Copied to clipboard

Dingwei Zhu, Shihan Dou, Zhiheng Xi, Senjie Jin, Guoqiang Zhang, Jiazheng Zhang, Junjie Ye, Mingxu Chai, Enyu Zhou, Ming Zhang, Yuhui Wang, Caishuang Huang, Chenhao Huang, Yunke Zhang, Yuran Wang, Tao Gui, Qi Zhang, Xipeng Qiu, Xuanjing Huang

Feedback-Driven Tool-Use Improvements in Large Language Models via Automated Build Environments (2026.findings-acl)

Challenge:	Reinforcement Learning (RL) in real-world environments often suffers from ambiguous or incomplete supervision.
Approach:	They propose a framework that enhances value modeling for robust RL in LLM post-training by integrating auxiliary losses guided by entropy and perplexity from a frozen language model and variational information bottleneck.
Outcome:	The proposed framework outperforms baselines on multi-turn dialogue, math reasoning, and science QA with rule-based and model-based rewards.

Copied to clipboard

Junjie Ye, Changhao Jiang, Zhengyin Du, Yufei Xu, Xuesong Yao, Zhiheng Xi, Xiaoran Fan, Qi Zhang, Tao Gui, Xuanjing Huang, Jiecao Chen

Rethinking Expert Trajectory Utilization in LLM Post-training for Mathematical Reasoning (2026.acl-long)

Challenge:	Currently, there are no efficient reinforcement learning (RL) frameworks specifically designed for tool use.
Approach:	They propose an automated environment construction pipeline that incorporates scenario decomposition, document generation, function integration, complexity scaling, and localized deployment to enable high-quality training environments without external tools.
Outcome:	The proposed framework significantly improves the models’ tool-use performance without degrading their general capabilities.

Copied to clipboard

Bowen Ding, Yuhan Chen, Jiayang Lyu, Jiyao Yuan, Qi Zhu, Shuangshuang Tian, Dantong Zhu, Futing Wang, Heyuan Deng, Fei Mi, Lifeng Shang, Tao Lin

Loose lips sink ships: Mitigating Length Bias in Reinforcement Learning from Human Feedback (2023.findings-emnlp)

Challenge:	evolving generic Large Language Models into specialized Large Reasoning Models requires effective post-training.
Approach:	They propose a plasticity-ceiling framework to harness expert trajectories . they establish the Sequential SFT-then-RL pipeline as the superior standard .
Outcome:	The proposed framework overcomes stability and premature convergence deficits in synchronized approaches.

Copied to clipboard

Wei Shen, Rui Zheng, Wenyu Zhan, Jun Zhao, Shihan Dou, Tao Gui, Qi Zhang, Xuanjing Huang

ImgTrojan: Jailbreaking Vision-Language Models with ONE Image (2025.naacl-long)

Challenge:	Experimental results prove that language models can learn from human feedback better, irrespective of sequence length . emergence of length bias often induces the model to favor longer outputs .
Approach:	They propose to separate reward modeling from the influence of sequence length by using the Product-of-Experts technique.
Outcome:	The proposed approach shows that language models perform better regardless of sequence length . the main expert is focused on understanding human intents, while the biased expert targets the identification and capture of length bias.

Copied to clipboard

Xijia Tao, Shuai Zhong, Lei Li, Qi Liu, Lingpeng Kong

From Scores to Preferences: Redefining Evaluation Paradigm for Speech Quality Reward Modeling (2026.findings-acl)

Challenge:	Existing studies on the safety of large language models (LLMs) with human values have focused on the integration of multi-modal user input into these models.
Approach:	They propose a method to bypass safety constraints of large language models by using poisoned images instead of original textual captions.
Outcome:	The proposed attack bypasses safety constraints of large language models (VLMs) by replacing the original textual captions with malicious jailbreak prompts.

Copied to clipboard

Yifei Cao, Changhao Jiang, Jiabao Zhuang, Jiajun Sun, Ming Zhang, Zhiheng Xi, Hui Li, Shihan Dou, Yuran Wang, Yunke Zhang, Tao Ji, Tao Gui, Qi Zhang, Xuanjing Huang

Template-free Prompt Tuning for Few-shot NER (2022.naacl-main)

Challenge:	Experimental results show that the MOS-aware GRM significantly improves fine-grained speech quality discrimination.
Approach:	They propose a MOS-aware reward model that incorporates MOS gap into reward function during reinforcement learning.
Outcome:	The proposed model significantly improves fine-grained speech quality discrimination.

Copied to clipboard

Ruotian Ma, Xin Zhou, Tao Gui, Yiding Tan, Linyang Li, Qi Zhang, Xuanjing Huang

Uncertainty-Aware Label Refinement for Sequence Labeling (2020.emnlp-main)

Challenge:	Prompt-based methods have been successfully applied in few-shot learning tasks . however, when applied to token-level labeling tasks, it would be time-consuming to enumerate the template queries over all potential entity spans.
Approach:	They propose a method to reformulate NER tasks as LM problems without templates.
Outcome:	The proposed method is 30.12 times faster than the template-based method under few-shot settings.

Copied to clipboard

Tao Gui, Jiacheng Ye, Qi Zhang, Zhengyan Li, Zichu Fei, Yeyun Gong, Xuanjing Huang

Green & Sustainable NLP

Challenge:	Conditional random fields (CRF) for label decoding have been a problem for many tasks.
Approach:	They propose a two-stage label decoding framework that model long-term label dependencies while being much more computationally efficient.
Outcome:	The proposed method outperforms the CRF-based methods and greatly accelerates the inference process.

ToolHop: A Query-Driven Benchmark for Evaluating Large Language Models in Multi-Hop Tool Use (2025.acl-long)

Copied to clipboard

Junjie Ye, Zhengyin Du, Xuesong Yao, Weijian Lin, Yufei Xu, Zehui Chen, Zaiyuan Wang, Sining Zhu, Zhiheng Xi, Siyu Yuan, Tao Gui, Qi Zhang, Xuanjing Huang, Jiecao Chen

A Lexicon-Based Supervised Attention Model for Neural Sentiment Analysis (C18-1)

Challenge:	Effective evaluation of multi-hop tool use is critical for analyzing the understanding, reasoning, and function-calling capabilities of large language models.
Approach:	They propose a dataset that provides rigorous evaluation of multi-hop tool use.
Outcome:	The proposed model achieves 49.04% accuracy across five model families.

Copied to clipboard

Yicheng Zou, Tao Gui, Qi Zhang, Xuanjing Huang

Mitigating Object Hallucinations in MLLMs via Multi-Frequency Perturbations (2025.findings-emnlp)

Challenge:	Existing attention models do not take full advantage of sentiment lexicons, which provide rich sentiment information and play a critical role in sentiment analysis.
Approach:	They propose a lexicon-based supervised attention model which allows a neural network to focus on the sentiment content, thus generating sentiment-informative representations.
Outcome:	The proposed model outperforms existing models on three large-scale sentiment classification datasets.

Copied to clipboard

Shuo Li, Jiajun Sun, Guodong Zheng, Xiaoran Fan, Yujiong Shen, Yi Lu, Zhiheng Xi, Yuming Yang, Wenming Tan, Tao Ji, Tao Gui, Qi Zhang, Xuanjing Huang

Distill Visual Chart Reasoning Ability from LLMs to MLLMs (2025.findings-emnlp)

Challenge:	Multimodal large language models have demonstrated remarkable performance in visual-language tasks, but their authenticity is often compromised by object hallucinations.
Approach:	They propose a multi-frequency perturbation method that leverages both low-frequency and high-frequency features of images to perturb visual feature representations and explicitly suppress redundant frequency-domain features during inference.
Outcome:	The proposed method significantly mitigates object hallucinations across various model architectures.

Copied to clipboard

Wei He, Zhiheng Xi, Wanxu Zhao, Xiaoran Fan, Yiwen Ding, Zifei Shan, Tao Gui, Qi Zhang, Xuanjing Huang

When Agents Look the Same: Quantifying Distillation-Induced Similarity in Tool-Use Behaviors (2026.acl-long)

Challenge:	a new method for generating chart annotations is proposed to improve visual reasoning in multimodal large language models.
Approach:	They propose a code-as-intermediary translation method for distilling visual reasoning abilities from LLMs to MLLMs.
Outcome:	The proposed method is cost-effective, efficient and scalable.

Copied to clipboard

Chenghao Yang, Yuning Zhang, Zhoufutu Wen, Tao Gong, Jiaheng Liu, Qi Chu, Nenghai Yu

Green & Sustainable NLP

Challenge:	Existing metrics fail to distinguish mandatory behaviors required for task success from non-mandatory patterns that reflect a model’s autonomous preferences.
Approach:	They propose to use response pattern similarity and action graph similarity to isolate non-mandatory behaviors from mandatory behaviors.
Outcome:	Evaluating 18 models from 8 providers on -Bench and 2-Bench against Claude Sonnet 4.5, the authors find that within-family model pairs score 5.9 pp higher in response pattern similarity and action graph similarity .

Cross-Linguistic Syntactic Difference in Multilingual BERT: How Good is It and How Does It Affect Transfer? (2022.emnlp-main)

Copied to clipboard

Ningyu Xu, Tao Gui, Ruotian Ma, Qi Zhang, Jingting Ye, Menghan Zhang, Xuanjing Huang

TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing (2021.acl-demo)

Challenge:	Multilingual BERT (mBERT) has demonstrated considerable cross-lingual syntactic ability, but it is not well understood what leads to this variation and whether it fairly reflects difference between languages.
Approach:	They propose to use multilingual BERT to enable zero-shot cross-lingual transfer of syntactic knowledge between different languages by generating grammatical relations in 24 different languages.
Outcome:	The results show that the distance between the distributions of different languages is highly consistent with the syntactic difference in terms of linguistic formalisms.

Copied to clipboard

Xiao Wang, Qin Liu, Tao Gui, Qi Zhang, Yicheng Zou, Xin Zhou, Jiacheng Ye, Yongxin Zhang, Rui Zheng, Zexiong Pang, Qinzhuo Wu, Zhengyan Li, Chong Zhang, Ruotian Ma, Zichu Fei, Ruijian Cai, Jun Zhao, Xingwu Hu, Zhiheng Yan, Yiding Tan, Yuan Hu, Qiyuan Bian, Zhihua Liu, Shan Qin, Bolin Zhu, Xiaoyu Xing, Jinlan Fu, Yue Zhang, Minlong Peng, Xiaoqing Zheng, Yaqian Zhou, Zhongyu Wei, Xipeng Qiu, Xuanjing Huang

Named Entity Recognition in Multi-level Contexts (2020.aacl-main)

Challenge:	Existing approaches to textual robustness evaluation focus on slightly modifying the input data, which maintains the original meaning and results in a different prediction.
Approach:	They propose a multilingual robustness evaluation toolkit for NLP that integrates universal text transformations, task-specific transformations and adversarial attack.
Outcome:	The toolkit includes universal text transformation, task-specific transformation, adversarial attack, subpopulation, and their combinations to provide comprehensive robustness analyses.

Copied to clipboard

Yubo Chen, Chuhan Wu, Tao Qi, Zhigang Yuan, Yongfeng Huang

NewsBERT: Distilling Pre-trained Language Model for Intelligent News Application (2021.findings-emnlp)

Challenge:	Existing methods for named entity recognition are unsatisfactory for recognizing entities in limited or ambiguous sentence-level contexts.
Approach:	They propose a framework to incorporate multi-level contexts for named entity recognition using TagLM as a baseline model and an auxiliary task to mine word-level contextual information.
Outcome:	The proposed framework is based on a set of sentence-level contexts and a document-level task to mine word-level contextual information.

Copied to clipboard

Chuhan Wu, Fangzhao Wu, Yang Yu, Tao Qi, Yongfeng Huang, Qi Liu

LoRACoE: Improving Large Language Model via Composition-based LoRA Expert (2025.emnlp-main)

Challenge:	Existing language models are pre-trained and distilled on general corpus like Wikipedia, which has gaps with the news domain and may be suboptimal for news intelligence.
Approach:	They propose a method to distill existing language models on Wikipedia to enable efficient news intelligence.
Outcome:	The proposed model can be used to build and test a news intelligence application on Wikipedia and Wikipedia.

Copied to clipboard

Guanyu Li, Zhiheng Xi, Zhihao Zhang, Boyang Hong, Tao Gui, Qi Zhang, Xuanjing Huang

QCRD: Quality-guided Contrastive Rationale Distillation for Large Language Models (2025.emnlp-main)

Challenge:	Recent studies show that the Mixture of Experts architecture improves performance of large language models.
Approach:	They propose a method to build static experts using LoRA parameters . they propose to use rank-level parameters to build experts based on rank-based parameters based in LoRA module.
Outcome:	The proposed method improves task performance across a broader range of tasks.

Copied to clipboard

Wei Wang, Zhaowei Li, Qi Xu, YiQing Cai, Hang Song, Qi Qi, Ran Zhou, Zhida Huang, Tao Wang, Li Xiao

One2Set: Generating Diverse Keyphrases as a Set (2021.acl-long)

Challenge:	Recent research has focused on smaller, task-specific models enhanced by distilling knowledge from LLMs, but the diversity and quality of negative knowledge remains understudied.
Approach:	They propose a quality-guided contrastive rationale distillation framework that aims to enhance reasoning capabilities through contrastive knowledge learning.
Outcome:	The proposed method consistently outperforms existing distillation techniques yielding higher-quality rationales.

Copied to clipboard

Jiacheng Ye, Tao Gui, Yichao Luo, Yige Xu, Qi Zhang

Challenge:	Recent keyphrase generation models are wrongly imposing a predefined order on keyphrases . a new training paradigm is proposed to concatenate keyphrase sequences in parallel .
Approach:	They propose a training paradigm that concatenates keyphrases in a predefined order . they propose combining a fixed set of learned control codes with a bipartite matching mechanism .
Outcome:	The proposed model outperforms the state-of-the-art methods on multiple benchmarks.

Beyond Scaling: Measuring and Predicting the Upper Bound of Knowledge Retention in Language Model Pre-Training (2026.acl-long)

Copied to clipboard

Changhao Jiang, Ming Zhang, Yifei Cao, Junjie Ye, Xiaoran Fan, Shihan Dou, Zhiheng Xi, Jiajun Sun, Yi Dong, Yujiong Shen, Jingqi Tong, Baoyu Fan, Tao Gui, Qi Zhang, Xuanjing Huang

TextObfuscator: Making Pre-trained Language Model a Privacy Protector via Obfuscating Word Representations (2023.findings-acl)

Challenge:	Existing methods to predict performance of large language models are lacking . authors propose a size-dependent mutual information predictor for closed-book question answering accuracy .
Approach:	They propose a size-dependent mutual information predictor that integrates knowledge frequency, knowledge specificity, and model size to forecast closed-book question answering accuracy.
Outcome:	The proposed method outperforms baseline models and achieves R2 > 0.7 in predicting QA accuracy without additional training.

Copied to clipboard

Xin Zhou, Yi Lu, Ruotian Ma, Tao Gui, Yuran Wang, Yong Ding, Yibo Zhang, Qi Zhang, Xuanjing Huang

Navigating the OverKill in Large Language Models (2024.acl-long)

Challenge:	Existing inference services are plagued by privacy concerns, such as sharing sensitive data with service providers.
Approach:	They propose a framework for protecting inference privacy by applying random perturbations to clustered representations.
Outcome:	The proposed framework protects inference privacy by applying random perturbations to clustered representations.

Copied to clipboard

Chenyu Shi, Xiao Wang, Qiming Ge, Songyang Gao, Xianjun Yang, Tao Gui, Qi Zhang, Xuanjing Huang, Xun Zhao, Dahua Lin