Papers by Zheng Fu

LIFTED: Multimodal Clinical Trial Outcome Prediction via Large Language Models and Mixture-of-Experts (2025.findings-emnlp)

Challenge:	Existing techniques for extending context capabilities in LLMs require additional training procedures and access to datasets with long context (e.g., sequences of 32K tokens).
Approach:	They propose a solution to extend context capabilities in Large Language Models by training a single process over a sequence of 4K tokens.
Outcome:	The proposed solution significantly reduces the cost of continual-pretraining or fine-tuning over short sequences and improves robustness to diverse relative positions.

Copied to clipboard

Wenhao Zheng, Liaoyaqi Wang, Dongshen Peng, Hongxia Xu, Yun Li, Hongtu Zhu, Tianfan Fu, Huaxiu Yao

Enhancing Hyperbole and Metaphor Detection with Their Bidirectional Dynamic Interaction and Emotion Knowledge (2025.acl-long)

Challenge:	Clinical trials are costly and pivotal processes that require substantial expenses . a new approach to integrate multimodal data for clinical outcome prediction is needed .
Approach:	a proposed framework transforms modality-specific data into natural language descriptions . a sparse Mixture-of-Experts mechanism then identifies shared patterns across modalities .
Outcome:	a proposed framework outperforms baseline methods in predicting clinical trial outcomes . it transforms modality-specific data into natural language descriptions, encoded via unified encoders .

Copied to clipboard

Li Zheng, Sihang Wang, Hao Fei, Zuquan Peng, Fei Li, Jianming Fu, Chong Teng, Donghong Ji

Sentiment Analysis

Challenge:	Existing methods for hyperbole and metaphor detection focus on superficial text features, ignoring the associations of hyperbola and metaphor . Existing frameworks focus on identifying superficial text, focusing on superficial features .
Approach:	They propose an emotion-guided hyperbole and metaphor detection framework based on bidirectional dynamic interaction.
Outcome:	The proposed framework outperforms baseline methods on four datasets.

Model Merging for Knowledge Editing (2025.acl-industry)

Copied to clipboard

Zichuan Fu, Xian Wu, Guojing Li, Yingying Zhang, Yefeng Zheng, Tianshi Ming, Yejing Wang, Wanyu Wang, Xiangyu Zhao

Reinforcement Learning with Semantic Rewards Enables Low-Resource Language Expansion without Alignment Tax (2026.findings-acl)

Challenge:	Existing knowledge editing approaches struggle with sequential editing scenarios and harm the general capabilities of the model.
Approach:	They propose a framework that combines robust supervised fine-tuning and model merging for knowledge editing to combine supervised and supervised learning.
Outcome:	The proposed approach outperforms existing methods in sequential editing while preserving the original performance of the model.

Copied to clipboard

Zeli Su, Ziyin Zhang, Zhou Liu, Xuexian Song, Zhankai Xu, Longfei Zheng, Xiaolu Zhang, Rong Fu, Guixian Xu, Wentao Zhang

Light-PEFT: Lightening Parameter-Efficient Fine-Tuning via Early Pruning (2024.findings-acl)

Challenge:	Extending large language models to low-resource languages often incurs an "alignment tax" token-level fine-tuning enforces token-level surface imitation on narrow and biased data distributions.
Approach:	They propose a semantic-space alignment paradigm powered by group-level semantic rewards instead of likelihood maximization.
Outcome:	The proposed model acquires low-resource capa- bilities while mitigating alignment tax on Tibetan–Chinese machine translation and Ti- betan headline generation.

Copied to clipboard

Naibin Gu, Peng Fu, Xiyu Liu, Bowen Shen, Zheng Lin, Weiping Wang

Enhancing Uncertainty-Based Hallucination Detection with Stronger Focus (2023.emnlp-main)

Challenge:	Existing methods for fine-tuning large language models are inefficient and redundant . a light-PEFT framework can be used to prune redundant parameters during training .
Approach:	They propose a parameter-efficient fine-tuning framework that freezes most parameters of the foundation model and finetuns only a small number of parameters.
Outcome:	The proposed framework achieves training and inference speedup, reduces memory usage, and maintains comparable performance and plug-and-play feature of PEFT.

Copied to clipboard

Tianhang Zhang, Lin Qiu, Qipeng Guo, Cheng Deng, Yue Zhang, Zheng Zhang, Chenghu Zhou, Xinbing Wang, Luoyi Fu

Hierarchical Enhancement Framework for Aspect-based Argument Mining (2023.findings-emnlp)

Challenge:	Existing methods for detecting hallucinations in LLMs rely on external knowledge for reference retrieval or require sampling multiple responses for consistency verification.
Approach:	They propose a reference-free, uncertainty-based method for detecting hallucinations in Large Language Models that imitates human focus in factuality checking from three aspects: focus on the most informative keywords; focus on unreliable tokens in historical context; focus of token properties such as token type and token frequency.
Outcome:	The proposed method achieves state-of-the-art performance across all evaluation metrics and eliminates the need for additional information.

Copied to clipboard

Yujie Fu, Yang Li, Suge Wang, Xiaoli Li, Deyu Li, Jian Liao, JianXing Zheng

Are We Using the Right Benchmark: An Evaluation Framework for Visual Token Compression Methods (2026.acl-long)

Challenge:	Existing methods have primarily treated ABAM as a nested named entity recognition problem, overlooking the need for tailored strategies to effectively address the specific challenges of ABA M tasks.
Approach:	They propose a layer-based Hierarchical Enhancement Framework (HEF) for Aspect-Based Argument Mining and introduce three new components to improve the performance and accuracy.
Outcome:	Experiments on multiple datasets and tasks verify the effectiveness of the proposed framework and components.

Copied to clipboard

Chenfei Liao, Wensong Wang, Zichen Wen, Xu Zheng, Yiyu Wang, Haocong He, Yuanhuiyi Lyu, Lutao Jiang, Xin Zou, Yuqian Fu, Bin Ren, Linfeng Zhang, Xuming Hu

AgentThink: A Unified Framework for Tool-Augmented Chain-of-Thought Reasoning in Vision-Language Models for Autonomous Driving (2025.findings-emnlp)

Challenge:	Recent efforts to accelerate inference in Multimodal Large Language Models have focused on visual token compression.
Approach:	They propose a framework that leverages downsampling as a discriminator to denoise existing benchmarks.
Outcome:	The proposed evaluation framework leverages downsampling as a discriminator to denoise existing benchmarks.

Copied to clipboard

Kangan Qian, Sicong Jiang, Yang Zhong, Ziang Luo, Zilin Huang, Tianze Zhu, Kun Jiang, Mengmeng Yang, Zheng Fu, Jinyu Miao, Yining Shi, He Zhe Lim, Li Liu, Tianbao Zhou, Hongyi Wang, Huang Yu, Yifei Hu, Guang Li, Guang Chen, Hao Ye, Lijun Sun, Diange Yang

AdapTime: Enabling Adaptive Temporal Reasoning in Large Language Models (2026.findings-acl)

Challenge:	Vision-Language Models struggle with hallucinations, inefficient reasoning, and limited real-world validation hinders accurate perception and robust step-by-step reasoning.
Approach:	AgentThink integrates Chain-of-Thought reasoning with dynamic, agent-style tool invocation for autonomous driving tasks.
Outcome:	Experiments on the DriveLMM-o1 benchmark show AgentThink significantly boosts overall reasoning scores by 53.91% and enhances answer accuracy by 33.54% .

Copied to clipboard

Yimin Deng, Yejing Wang, Zhenxi Lin, Zichuan Fu, Guoshuai Zhao, Derong Xu, Yefeng Zheng, Xiangyu Zhao, Xian Wu, Li Zhu, Xueming Qian

Past, Present, and Future: Conversational Emotion Recognition through Structural Modeling of Psychological Knowledge (2021.findings-emnlp)

Challenge:	Existing methods for temporal reasoning are limited and apply a fixed pipeline to all questions.
Approach:	They propose an adaptive temporal reasoning method that dynamically executes reasoning steps based on context and task requirements.
Outcome:	Experiments on two temporal QA benchmarks show the proposed method works.

Copied to clipboard

Jiangnan Li, Zheng Lin, Peng Fu, Weiping Wang

Think Hard Only When Needed: A Hybrid Best-of-N and Beam Search for Efficient Test-Time Compute (2026.findings-eacl)

Challenge:	Conversational Emotion Recognition (CER) is a task to predict the emotion of an utterance in the context of a conversation.
Approach:	They propose a pSychological-Knowledge-Aware Interaction Graph to model the emotional state of an utterance in the context of a conversation.
Outcome:	The proposed method achieves state-of-the-art and competitive performance on four popular CER datasets.

Copied to clipboard

Hyewon Suh, Chaojian Li, Cheng-Jhih Shih, Zheng Wang, Kejing Xia, Yonggan Fu, Yingyan Celine Lin

Tandem: Riding Together with Large and Small Language Models for Efficient Reasoning (2026.findings-acl)

Challenge:	Large language models (LLMs) exhibit remarkable reasoning and planning capabilities, yet their substantial inference-time cost significantly impedes deployment in resourceconstrained applications.
Approach:	They propose a hybrid inference pipeline that combines beam search and Best-of-N . THROW generates shorter initial trajectories and evaluates them using PRMs .
Outcome:	THROW achieves 1.54 and 14.38 latency speedups and 35.7% and 80.4% token reductions on average compared to Best-of-N and beam search .

Copied to clipboard

Zichuan Fu, Xian Wu, Guojing Li, Yejing Wang, Yijun Chen, Zhao Zihao, Luo Yixuan, Hanyu Yan, Yefeng Zheng, Xiangyu Zhao

DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environments (2025.emnlp-main)

Challenge:	Recent advances in large language models (LLMs) have catalyzed the rise of reasoningintensive inference paradigms, where models perform explicit step-by-step reasoning before generating final answers.
Approach:	They propose a large-small LLM collaboration framework that synergizes large and small language models to achieve high-quality reasoning with significantly reduced computational cost.
Outcome:	The proposed framework outperforms the mentor LLM while preserving the benefits of the thinking paradigm of LLMs.

Copied to clipboard

Yuxiang Zheng, Dayuan Fu, Xiangkun Hu, Xiaojie Cai, Lyumanshan Ye, Pengrui Lu, Pengfei Liu

MASPO: Unifying Gradient Utilization, Probability Mass, and Signal Reliability for Robust and Sample-Efficient LLM Reasoning (2026.acl-long)

Challenge:	Large Language Models (LLMs) with web search capabilities show significant potential for deep research.
Approach:	They introduce a framework for end-to-end training of LLM-based deep research agents . they implement a specialized multi-agent architecture where browsing agents extract relevant information from various webpage structures.
Outcome:	The proposed framework improves on open-domain research tasks by 28.9 points over prompt engineering and 7.2 points over RAG-based RL agents.

Copied to clipboard

Xiaoliang Fu, Jiaye Lin, Yangyi Fang, Binbin Zheng, Chaowen Hu, Zekai Shao, Cong Qin, Lu Pan, Ke Zeng, Xunliang Cai

From log 𝜋 to 𝜋: Taming Divergence in Soft Clipping via Bilateral Decoupled Decay of Probability Gradient Weight (2026.acl-long)

Challenge:	Existing RLVR algorithms rely on rigid, uniform, and symmetric trust region mechanisms . current algorithms lack robustness, asymmetric signal reliability and inefficient gradient utilization .
Approach:	They propose a framework to harmonize three dimensions of RLVR algorithms, a paper argues . a binary cutoff is used to discard valuable reinforcement signals, they argue .
Outcome:	The proposed framework outperforms baselines in evaluating a robust RLVR solution.

Copied to clipboard

Xiaoliang Fu, Jiaye Lin, Yangyi Fang, Chaowen Hu, Cong Qin, Zekai Shao, Binbin Zheng, Lu Pan, Ke Zeng

Training-free LLM Merging for Multi-task Learning (2025.acl-long)

Challenge:	Standard algorithms for Large Language Models (LLMs) enforce stability via "hard clipping" but relying on log-probability gradient yields divergent weights as probabilities vanish, destabilizing LLM training.
Approach:	They propose a decoupled gradient policy optimization that uses a decay mechanism to decouple the probability of a boundary token.
Outcome:	The proposed algorithm outperforms baselines on various mathematical benchmarks.

Copied to clipboard

Zichuan Fu, Xian Wu, Yejing Wang, Wanyu Wang, Shanshan Ye, Hongzhi Yin, Yi Chang, Yefeng Zheng, Xiangyu Zhao

ChatMusician: Understanding and Generating Music Intrinsically with LLM (2024.findings-acl)

Challenge:	Large Language Models (LLMs) have demonstrated exceptional capabilities across diverse natural language processing tasks.
Approach:	They propose a training-free method for unifying different specialized LLMs into a single model using model-wise and layer-wise pruning and scaling.
Outcome:	The proposed method outperforms existing merging techniques and surpasses models fine-tuned on combined datasets in most scenarios.

Copied to clipboard

Ruibin Yuan, Hanfeng Lin, Yi Wang, Zeyue Tian, Shangda Wu, Tianhao Shen, Ge Zhang, Yuhang Wu, Cong Liu, Ziya Zhou, Liumeng Xue, Ziyang Ma, Qin Liu, Tianyu Zheng, Yizhi Li, Yinghao Ma, Yiming Liang, Xiaowei Chi, Ruibo Liu, Zili Wang, Chenghua Lin, Qifeng Liu, Tao Jiang, Wenhao Huang, Wenhu Chen, Jie Fu, Emmanouil Benetos, Gus Xia, Roger Dannenberg, Wei Xue, Shiyin Kang, Yike Guo

Towards Robust Visual Question Answering: Making the Most of Biased Samples via Contrastive Learning (2022.findings-emnlp)

Challenge:	Despite LLMs' impressive capabilities in musical knowledge, music reasoning remains an unsolved task.
Approach:	They propose an open-source large language model (LLM) that integrates intrinsic musical abilities into LLaMA2 and GPT-3.5.
Outcome:	The proposed model can understand and generate music with a pure text tokenizer without external multi-modal neural structures or tokenizers.

Copied to clipboard

Qingyi Si, Yuanxin Liu, Fandong Meng, Zheng Lin, Peng Fu, Yanan Cao, Weiping Wang, Jie Zhou

A Multi-Expert Structural-Semantic Hybrid Framework for Unveiling Historical Patterns in Temporal Knowledge Graphs (2025.findings-acl)

Challenge:	Recent studies have shown that biased samples can be brittle for VQA models . however, the improvements on OOD data severely sacrifice the performance on the in-distribution (ID) data.
Approach:	They propose a contrastive learning approach that exploits biased samples for unbiased information that contributes to reasoning.
Outcome:	The proposed method achieves competitive performance on the OOD dataset while maintaining robustness on the ID dataset.

Copied to clipboard

Yimin Deng, Yuxia Wu, Yejing Wang, Guoshuai Zhao, Li Zhu, Qidong Liu, Derong Xu, Zichuan Fu, Xian Wu, Yefeng Zheng, Xiangyu Zhao, Xueming Qian

Knowledge Representation

Challenge:	Existing methods focus on graph structure learning or semantic reasoning, lacking the capability to capture the inherent differences between historical and non-historical events.
Approach:	They propose a temporal knowledge graph reasoning framework that integrates both structural and semantic information to guide the reasoning process for different events.
Outcome:	The proposed framework integrates structural and semantic information to predict future events . it can provide evidence for many downstream tasks, including situation analysis and political decision making .

TokenSelect: Efficient Long-Context Inference and Length Extrapolation for LLMs via Dynamic Token-Level KV Cache Selection (2025.emnlp-main)

Copied to clipboard

Wei Wu, Zhuoshi Pan, Kun Fu, Chao Wang, Liyi Chen, Yunchu Bai, Tianfu Wang, Zheng Wang, Hui Xiong

Skill-Based Few-Shot Selection for In-Context Learning (2023.emnlp-main)

Challenge:	Rapid advances in Large Language Models have spurred demand for processing extended context sequences . however, performance degradation due to sequence lengths out-of-distribution and excessively long inference times are limiting LLMs in long-context scenarios.
Approach:	They propose a training-free method for efficient and accurate long-context inference . they selectively involves a few critical KV cache tokens in attention calculation .
Outcome:	The proposed method speeds up attention computation and accelerates inference time while reducing selection overhead.

Copied to clipboard

Shengnan An, Bo Zhou, Zeqi Lin, Qiang Fu, Bei Chen, Nanning Zheng, Weizhu Chen, Jian-Guang Lou

Learning to Win Lottery Tickets in BERT Transfer via Task-agnostic Mask Training (2022.naacl-main)

Challenge:	Existing methods based on pre-trained embeddings can be easily biased by surface features that are not important for the target task.
Approach:	They propose a skill-based few-shot selection method for in-context learning . it generates skill-specific descriptions for each test case and candidate example .
Outcome:	The proposed method significantly outperforms existing methods in five cross-domain semantic parsing datasets and six backbone models.

Copied to clipboard

Yuanxin Liu, Fandong Meng, Zheng Lin, Peng Fu, Yanan Cao, Weiping Wang, Jie Zhou

DIVE into MoE: Diversity-Enhanced Reconstruction of Large Language Models from Dense into Mixture-of-Experts (2025.acl-long)

Challenge:	Recent studies show pre-trained language models contain matching subnetworks that have similar transfer learning performance as the original PLM.
Approach:	They propose to prune matching subnetworks using magnitude-based pruning . they propose to optimize the subnetwork structure towards the pre-training objectives .
Outcome:	The proposed method is more efficient in searching subnetworks and advantageous when fine-tuning within a range of data scarcity.

Copied to clipboard

Yuchen Feng, Bowen Shen, Naibin Gu, Jiaxuan Zhao, Peng Fu, Zheng Lin, Weiping Wang

Question-Interlocutor Scope Realized Graph Modeling over Key Utterances for Dialogue Reading Comprehension (2023.findings-acl)

Challenge:	Existing methods for reconstruction of large language models overlook diversity among experts, leading to potential redundancy.
Approach:	They propose a pruning-based expert reconstruction method that prunes a specific LLM and retrains it on routers, experts and normalization modules.
Outcome:	The proposed method outperforms pruning and MoE reconstruction methods on Llama-style models with open-source training corpora.

Copied to clipboard

Jiangnan Li, Mo Yu, Fandong Meng, Zheng Lin, Peng Fu, Weiping Wang, Jie Zhou

Natural Language Interfaces

Challenge:	Compared to standard RC tasks, dialogue reading comprehension (DRC) has raised challenges because of the complex speaker information and noisy dialogue context.
Approach:	They propose a new method for dialogue reading comprehension that extracts answers from dialogues by using key-utterances-extracting methods and a Question-Interlocutor Scope Realized Graph.
Outcome:	The proposed method achieves state-of-the-art performance against previous works.

A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challenges (2025.findings-acl)

Copied to clipboard

Yibo Yan, Jiamin Su, Jianxiang He, Fangteng Fu, Xu Zheng, Yuanhuiyi Lyu, Kun Wang, Shen Wang, Qingsong Wen, Xuming Hu

MultiDx: A Multi-Source Knowledge Integration Framework towards Diagnostic Reasoning (2026.findings-acl)

Challenge:	This survey provides the first comprehensive analysis of mathematical reasoning in the era of multimodal large language models . integrating large language model with mathematical reasoning tasks is becoming significant as AI advances .
Approach:	They review over 200 studies published since 2021 and examine the state-of-the-art developments in Math-LLMs . they identify five major challenges hindering the realization of AGI in this domain .
Outcome:	The authors examine the state-of-the-art developments in Math-LLMs with a focus on multimodal settings.

Copied to clipboard

Yimin Deng, Zhenxi Lin, Yejing Wang, Guoshuai Zhao, Pengyue Jia, Zichuan Fu, Derong Xu, Yefeng Zheng, Xiangyu Zhao, Li Zhu, Xian Wu, Xueming Qian

Check It Again:Progressive Visual Question Answering via Visual Entailment (2021.acl-long)

Challenge:	Existing approaches focus on diagnostic reasoning based on internal model knowledge or static knowledge bases.
Approach:	They propose a two-stage diagnostic reasoning framework that integrates multi-perspective evidence to generate a diagnostic prediction.
Outcome:	The proposed method generates suspected diagnoses and reasoning traces from web search, SOAP-formatted case, and clinical case database.

Copied to clipboard

Qingyi Si, Zheng Lin, Ming yu Zheng, Peng Fu, Weiping Wang

Visual Data in NLP

Challenge:	Existing approaches to Visual Question Answering (VQA) only address superficial correlations between image and answer.
Approach:	They propose a select-and-rerank progressive framework based on Visual Entailment to address this problem.
Outcome:	The proposed framework improves on the Visual Question Answering (VQA) task with 7.55% accuracy.

Doc-V*: Coarse-to-Fine Interactive Visual Reasoning for Multi-Page Document VQA (2026.acl-long)

Copied to clipboard

Yuanlei Zheng, Pei Fu, Hang Li, Ziyang Wang, Yuyi Zhang, Wenyu Ruan, Xiaojin Zhang, Zhongyu Wei, Zhenbo Luo, Jian Luan, Wei Chen, Xiang Bai

Can Graph Neural Networks Learn Language with Extremely Weak Text Supervision? (2025.acl-long)

Challenge:	Existing OCR-free approaches to document visual question answering are brittle and passive.
Approach:	They propose an OCR-free agentic framework that casts multi-page DocVQA as sequential evidence aggregation.
Outcome:	The proposed framework outperforms open-source and proprietary models in five benchmarks and improves out-of-domain performance by 47.9% over baseline.

Copied to clipboard

Zihao Li, Lecheng Zheng, Bowen Jin, Dongqi Fu, Baoyu Jing, Yikun Ban, Jingrui He, Jiawei Han

Target Really Matters: Target-aware Contrastive Learning and Consistency Regularization for Few-shot Stance Detection (2022.coling-1)

Challenge:	Graph Neural Networks (GNNs) with CLIP pipeline are difficult because of the scarcity of labeled data and text supervision, different levels of downstream tasks, and conceptual gaps between domains.
Approach:	They propose a multi-modal prompt learning paradigm to adapt pre-trained GNNs to downstream tasks with weak text supervision.
Outcome:	The proposed model can generalize graphs to unseen classes with weak text supervision.

Copied to clipboard

Rui Liu, Zheng Lin, Huishan Ji, Jiangnan Li, Peng Fu, Weiping Wang

Modeling Intra and Inter-modality Incongruity for Multi-Modal Sarcasm Detection (2020.findings-emnlp)

Challenge:	stance detection is a task to identify attitudes from opinions towards certain targets, but it is expensive and time-consuming . stance detector is based on labeled data, but unlabeled data can be collected easier .
Approach:	They propose a semi-supervised framework for few-shot stance detection that uses unlabeled data to learn more distinguishable representations for different targets.
Outcome:	The proposed framework achieves state-of-the-art performance on multiple benchmark datasets.

Copied to clipboard

Hongliang Pan, Zheng Lin, Peng Fu, Yatao Qi, Weiping Wang

Sentiment Analysis

Challenge:	Existing methods for sarcasm detection ignore the incongruity character in sarcasm, which is often manifested between modalities or within modalités.
Approach:	They propose to capture inter-modality incongruity in a text-based model by using a self-attention mechanism and a co-attention model to model the contradiction within the text.
Outcome:	The proposed model achieves state-of-the-art on a public multi-modal sarcasm detection dataset.

A Gradient Control Method for Backdoor Attacks on Parameter-Efficient Tuning (2023.acl-long)

Copied to clipboard

Naibin Gu, Peng Fu, Xiyu Liu, Zhengxiao Liu, Zheng Lin, Weiping Wang

Adapt Once, Thrive with Updates: Transferable Parameter-Efficient Fine-Tuning on Evolving Base Models (2025.acl-long)

Challenge:	Parameter-Efficient Tuning (PET) fine-tunes pre-trained language models for downstream tasks, but a large reduction in the number of attackable parameters will greatly affect the effectiveness of backdoor attacks, resulting in backdoor forgetting.
Approach:	They propose a gradient control method to consolidate the attack effect by freezing most parameters of the pre-trained model and fine-tuning only a small number of parameters.
Outcome:	The proposed method improves sentiment classification and spam detection, and can be applied to different tasks.

Copied to clipboard

Naibin Gu, Peng Fu, Xiyu Liu, Ke Ma, Zheng Lin, Weiping Wang

Slot Dependency Modeling for Zero-Shot Cross-Domain Dialogue State Tracking (2022.coling-1)

Challenge:	Parameter-efficient fine-tuning (PEFT) is a common method for fine- tuning large language models . however, once updated, PEFT modules suffer performance degradation on newer versions .
Approach:	They propose a method that enhances the PEFT module by focusing on the task-specific pattern while reducing its dependence on certain knowledge in the base model.
Outcome:	Experiments show that PEFT modules can maintain performance on updated models without re-tuning . the proposed approach can be used in real-world applications with large model sizes .

Copied to clipboard

Qingyue Wang, Yanan Cao, Piji Li, Yanhe Fu, Zheng Lin, Li Guo

Low-Resource NLP

Challenge:	Existing zero-shot learning methods ignore slot dependencies in a multidomain dialogue . experimental results show the effectiveness of our proposed method over existing state-of-art generation methods .
Approach:	They propose to use slot prompts combination, slot values demonstration and slot constraint object to model slot-slot dependency, slot-value dependency and slot-context dependency respectively.
Outcome:	The proposed method outperforms state-of-the-art methods under zero-shot/few-shot settings.

BeamLoRA: Beam-Constraint Low-Rank Adaptation (2025.acl-long)

Copied to clipboard

Naibin Gu, Zhenyu Zhang, Xiyu Liu, Peng Fu, Zheng Lin, Shuohuan Wang, Yu Sun, Hua Wu, Weiping Wang, Haifeng Wang

COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuning (2025.findings-naacl)

Challenge:	Low-Rank Adaptation (LoRA) is one of the most efficient parameter-efficient fine-tuning methods.
Approach:	They propose to conceptualize each LoRA module as a beam where each rank corresponds to a potential sub-solution.
Outcome:	The proposed method improves performance on three base models and 12 datasets.

Copied to clipboard

Yuelin Bai, Xeron Du, Yiming Liang, Leo Jin, Junting Zhou, Ziqiang Liu, Feiteng Fang, Mingshan Chang, Tianyu Zheng, Xincheng Zhang, Nuo Ma, Zekun Moore Wang, Ruibin Yuan, Haihong Wu, Hongquan Lin, Wenhao Huang, Jiajun Zhang, Chenghua Lin, Jie Fu, Min Yang, Shiwen Ni, Ge Zhang

How Do In-Context Examples Affect Compositional Generalization? (2023.acl-long)

Challenge:	Existing datasets for Chinese instruction tuning are not well-aligned with Chinese users’ interaction patterns.
Approach:	They propose to use Chinese instruction tuning datasets to improve instruction fine-tuning for Chinese users.
Outcome:	The proposed dataset shows that Chinese models achieve competitive performance in diverse benchmarks.

Copied to clipboard

Shengnan An, Zeqi Lin, Qiang Fu, Bei Chen, Nanning Zheng, Jian-Guang Lou, Dongmei Zhang

Low-Resource NLP

Challenge:	In-context learning paradigms that focus on large corpus are limiting compositional generalization performance.
Approach:	They propose a test suite to investigate in-context compositional generalization . they propose to use examples that are structurally similar to the test case .
Outcome:	The proposed test suite investigates in-context compositional generalization performance . it finds that the performance can be affected by the selection of in-const examples .

Decompose, Prioritize, and Eliminate: Dynamically Integrating Diverse Representations for Multimodal Named Entity Recognition (2024.lrec-main)

Copied to clipboard

Zihao Zheng, Zihan Zhang, Zexin Wang, Ruiji Fu, Ming Liu, Zhongyuan Wang, Bing Qin

Investigating and Enhancing the Robustness of Large Multimodal Models Against Temporal Inconsistency (2025.acl-long)

Challenge:	Existing research on multi-modal Named Entity Recognition (MNER) does not integrate all multi-modal representations to provide rich contextual information to improve NER.
Approach:	They propose an iterative reasoning framework that integrates all the diverse multi-modal representations following the strategy of "decompose, prioritize, and eliminate" . they propose to use hierarchically connected fusion layers to prioritize transitions from "easy-to-hard" and "coarse-to fine"
Outcome:	The proposed framework integrates all the diverse multi-modal representations following the strategy of "decompose, prioritize, and eliminate".

Copied to clipboard

Jiafeng Liang, Shixin Jiang, Xuan Dong, Ning Wang, Zheng Chu, Hui Su, Jinlan Fu, Ming Liu, See-Kiong Ng, Bing Qin

Robustness in NLP

Challenge:	Large Multimodal Models (LMMs) have demonstrated impressive performance on general video comprehension benchmarks, but their robustness needs to be thoroughly investigated for broader applications.
Approach:	They propose a temporal robustness benchmark which introduces temporal inconsistency perturbations separately at the visual and textual modalities to assess the robustness of models.
Outcome:	The proposed method improves the model’s robustness and reliability in temporal analysis.

NavA3: Understanding Any Instruction, Navigating Anywhere, Finding Anything (2026.acl-long)

Copied to clipboard

Lingfeng Zhang, Xiaoshuai Hao, Yingbo Tang, Haoxiang Fu, Xinyu Zheng, Pengwei Wang, Zhongyuan Wang, Wenbo Ding, Shanghang Zhang

Language Prior Is Not the Only Shortcut: A Benchmark for Shortcut Learning in VQA (2022.findings-emnlp)

Challenge:	Existing embodied navigation methods struggle with such tasks due to their limitations in comprehending high-level human instructions and localizing objects with an open vocabulary.
Approach:	They propose a hierarchical framework for long-horizon navigation that integrates human instructions with 3D scene views.
Outcome:	The proposed model achieves SOTA results and can complete long-horizon navigation tasks across different robot embodiments in real-world environments.

Copied to clipboard

Qingyi Si, Fandong Meng, Mingyu Zheng, Zheng Lin, Yuanxin Liu, Peng Fu, Yanan Cao, Weiping Wang, Jie Zhou

Question Answering

Challenge:	Visual Question Answering (VQA) models are prone to learn the shortcut solution formed by dataset biases rather than the intended solution.
Approach:	They propose a dataset that considers varying types of shortcuts by constructing different distribution shifts in multiple OOD test sets.
Outcome:	The proposed dataset considers varying types of shortcuts by constructing different distribution shifts in multiple OOD test sets.

TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing (2021.acl-demo)

Copied to clipboard

Xiao Wang, Qin Liu, Tao Gui, Qi Zhang, Yicheng Zou, Xin Zhou, Jiacheng Ye, Yongxin Zhang, Rui Zheng, Zexiong Pang, Qinzhuo Wu, Zhengyan Li, Chong Zhang, Ruotian Ma, Zichu Fei, Ruijian Cai, Jun Zhao, Xingwu Hu, Zhiheng Yan, Yiding Tan, Yuan Hu, Qiyuan Bian, Zhihua Liu, Shan Qin, Bolin Zhu, Xiaoyu Xing, Jinlan Fu, Yue Zhang, Minlong Peng, Xiaoqing Zheng, Yaqian Zhou, Zhongyu Wei, Xipeng Qiu, Xuanjing Huang

Robustness in NLP

Challenge:	Existing approaches to textual robustness evaluation focus on slightly modifying the input data, which maintains the original meaning and results in a different prediction.
Approach:	They propose a multilingual robustness evaluation toolkit for NLP that integrates universal text transformations, task-specific transformations and adversarial attack.
Outcome:	The toolkit includes universal text transformation, task-specific transformation, adversarial attack, subpopulation, and their combinations to provide comprehensive robustness analyses.

OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement (2024.findings-acl)

Copied to clipboard

Tianyu Zheng, Ge Zhang, Tianhao Shen, Xueling Liu, Bill Yuchen Lin, Jie Fu, Wenhu Chen, Xiang Yue

Programming Languages in NLP

Challenge:	OpenCodeInterpreter-33B provides a high level of performance for code generation, executing, and iterative refinement.
Approach:	They propose a family of open-source code systems for generating, executing, and iteratively refining code.
Outcome:	The OpenCodeInterpreter-33B performs well on humanEval, MBPP, and EvalPlus benchmarks.

Compressing and Debiasing Vision-Language Pre-Trained Models for Visual Question Answering (2023.emnlp-main)

Copied to clipboard

Qingyi Si, Yuanxin Liu, Zheng Lin, Peng Fu, Yanan Cao, Weiping Wang

Robustness in NLP

Challenge:	Existing studies on VQA models have found that they suffer from dataset biases and inefficient memory footprints.
Approach:	They investigate whether a VLP can be compressed and debiased simultaneously by searching sparse and robust subnetworks.
Outcome:	The proposed compression and debiasing pipelines outperform the debiased full VLPs on VQA tasks.

LAMB: A Training-Free Method to Enhance the Long-Context Understanding of SSMs via Attention-Guided Token Filtering (2025.acl-short)

Copied to clipboard

Zhifan Ye, Zheng Wang, Kejing Xia, Jihoon Hong, Leshu Li, Lexington Whalen, Cheng Wan, Yonggan Fu, Yingyan Celine Lin, Souvik Kundu

Challenge:	Recent work attributes performance degradation to an exponential decay in hidden-state memory.
Approach:	They propose a token filtering strategy that is training-free and attention-guided . they propose 'LAMB' to preserve critical tokens during inference .
Outcome:	The proposed token filtering improves long-context performance by 30.35% over state-of-the-art methods on benchmarks.

CBP-Tuning: Efficient Local Customization for Black-box Large Language Models (2025.emnlp-main)

Copied to clipboard

Jiaxuan Zhao, Naibin Gu, Yuchen Feng, Xiyu Liu, Peng Fu, Zheng Lin, Weiping Wang

Learning to Compose Representations of Different Encoder Layers towards Improving Compositional Generalization (2023.findings-emnlp)

Challenge:	Customized black-box prompt tuning is a new approach to customize large language models . however, as models grow, the resources required for training and deployment become increasingly expensive .
Approach:	They propose a framework that facilitates efficient local customization while preserving bidirectional privacy.
Outcome:	The proposed framework facilitates efficient local customization while preserving bidirectional privacy.

Copied to clipboard

Lei Lin, Shuangtao Li, Yafang Zheng, Biao Fu, Shan Liu, Yidong Chen, Xiaodong Shi