Papers by Jie Fu

LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing (2024.emnlp-main)

Challenge:	Existing techniques for extending context capabilities in LLMs require additional training procedures and access to datasets with long context (e.g., sequences of 32K tokens).
Approach:	They propose a solution to extend context capabilities in Large Language Models by training a single process over a sequence of 4K tokens.
Outcome:	The proposed solution significantly reduces the cost of continual-pretraining or fine-tuning over short sequences and improves robustness to diverse relative positions.

Copied to clipboard

Jiangshu Du, Yibo Wang, Wenting Zhao, Zhongfen Deng, Shuaiqi Liu, Renze Lou, Henry Zou, Pranav Narayanan Venkit, Nan Zhang, Mukund Srinath, Haoran Zhang, Vipul Gupta, Yinghui Li, Tao Li, Fei Wang, Qin Liu, Tianlin Liu, Pengzhi Gao, Congying Xia, Chen Xing, Cheng Jiayang, Zhaowei Wang, Ying Su, Raj Shah, Ruohao Guo, Jing Gu, Haoran Li, Kangda Wei, Zihao Wang, Lu Cheng, Surangika Ranathunga, Meng Fang, Jie Fu, Fei Liu, Ruihong Huang, Eduardo Blanco, Yixin Cao, Rui Zhang, Philip Yu, Wenpeng Yin

Challenge:	a comparative analysis of paper (meta-)reviews by large language models (LLMs) aims to identify and distinguish LLMs from human activities .
Approach:	They present a comparative analysis to identify and distinguish LLM activities from human activities.
Outcome:	The proposed analysis aims to improve recognition of instances when someone implicitly uses LLMs for reviewing activities.

TIGS: An Inference Algorithm for Text Infilling with Gradient Search (P19-1)

Copied to clipboard

Dayiheng Liu, Jie Fu, Pengfei Liu, Jiancheng Lv

CamoQuery: Language-Guided Reasoning Camouflaged Object Segmentation (2026.acl-long)

Challenge:	Text infilling is an under-explored challenge in the field of text generation.
Approach:	They propose an iterative inference algorithm based on gradient search that can be broadly applied to any sequence generative model for text infilling tasks.
Outcome:	The proposed method performs well on three different text infilling tasks with different mask ratios and mask strategies compared with five state-of-the-art methods.

Copied to clipboard

Tianxin Han, Qing Dong, Xingwei Wang, Jie Jia, Gang Wu, Bowen Yang, Fu Zhang

HERB: Measuring Hierarchical Regional Bias in Pre-trained Language Models (2022.findings-aacl)

Challenge:	Existing methods for camouflaged object segmentation are limited to vision-only mask prediction under fixed task assumptions.
Approach:	They propose a language-guided reasoning camouflaged object segmentation task that generates an intent-consistent segmentation mask from an image and an implicit query text instruction.
Outcome:	The proposed task can generate an intent-consistent segmentation mask from a camouflaged image and an implicit query text instruction.

Copied to clipboard

Yizhi Li, Ge Zhang, Bohao Yang, Chenghua Lin, Anton Ragni, Shi Wang, Jie Fu

PopAlign: Diversifying Contrasting Patterns for a More Comprehensive Alignment (2025.acl-long)

Challenge:	Existing methods do not examine social groups categorised by geographical information, leaving the region-related biases in pre-trained LMs unexplored.
Approach:	They propose a hierarchical regional bias evaluation method to quantify regional bias in pre-trained language models.
Outcome:	The proposed method evaluates regional bias with regard to comprehensive topics and measures potential regional bias that can be propagated to downstream tasks.

Copied to clipboard

Zekun Moore Wang, Shenzhi Wang, King Zhu, Jiaheng Liu, Ke Xu, Jie Fu, Wangchunshu Zhou, Wenhao Huang

Challenge:	Typical approaches to training large language models rely on limited contrasting patterns . contrasting data is limited and models are susceptible to harmful response tendencies .
Approach:	They propose a framework that integrates contrasting patterns across the prompt, model, and pipeline levels.
Outcome:	The proposed framework outperforms existing methods in the comparison of RQ1 and RQ2 . the proposed framework significantly outperformed existing methods, leading to more comprehensive alignment.

PositionID: LLMs can Control Lengths, Copy and Paste with Explicit Positional Awareness (2024.findings-emnlp)

Copied to clipboard

Noah Wang, Feiyu Duan, Yibo Zhang, Wangchunshu Zhou, Ke Xu, Wenhao Huang, Jie Fu

Simple and Effective Curriculum Pointer-Generator Networks for Reading Comprehension over Long Narratives (P19-1)

Challenge:	Large Language Models (LLMs) have impressive capabilities across various domains, including role-playing, creative writing, mathematical reasoning, and coding.
Approach:	They propose two methods to improve the model’s adherence to length constraints and copy-paste accuracy without compromising response quality.
Outcome:	The proposed methods improve the model’s adherence to length constraints and copy-paste accuracy without compromising response quality.

Copied to clipboard

Yi Tay, Shuohang Wang, Anh Tuan Luu, Jie Fu, Minh C. Phan, Xingdi Yuan, Jinfeng Rao, Siu Cheung Hui, Aston Zhang

Machine Reading Comprehension

Challenge:	Using a pointer-generator framework for reading/sampling over large documents, we propose a framework for learning over long narratives where documents easily span over thousands of tokens.
Approach:	They propose a curriculum learning (CL) based pointer-generator framework for reading/sampling over large documents, enabling diverse training of the neural model based on the notion of alternating contextual difficulty.
Outcome:	The proposed framework improves on the NarrativeQA reading comprehension benchmark and reaches state-of-the-art performance.

Rhetorically Controlled Encoder-Decoder for Modern Chinese Poetry Generation (P19-1)

Copied to clipboard

Zhiqiang Liu, Zuohui Fu, Jie Cao, Gerard de Melo, Yik-Cheung Tam, Cheng Niu, Jie Zhou

AnyGPT: Unified Multimodal LLM with Discrete Sequence Modeling (2024.acl-long)

Challenge:	Rhetoric is a vital element in modern Chinese poetry, and plays an essential role in improving its aesthetics. however, to date, it has not been considered in research on automatic poetry generation.
Approach:	They propose a rhetorically controlled encoder-decoder for modern Chinese poetry generation . their model captures various rhetorical patterns in an encoder and incorporates mixtures .
Outcome:	The proposed model outperforms state-of-the-art methods in terms of fluency, coherence, meaningfulness, and rhetorical aesthetics.

Copied to clipboard

Jun Zhan, Junqi Dai, Jiasheng Ye, Yunhua Zhou, Dong Zhang, Zhigeng Liu, Xin Zhang, Ruibin Yuan, Ge Zhang, Linyang Li, Hang Yan, Jie Fu, Tao Gui, Tianxiang Sun, Yu-Gang Jiang, Xipeng Qiu

HyperMoE: Towards Better Mixture of Experts via Transferring Among Experts (2024.acl-long)

Challenge:	Existing language models that use discrete representations for unified processing of various modalities are limited to text generation and do not include multimodal output.
Approach:	They propose a multimodal language model that utilizes discrete representations for unified processing of various modalities.
Outcome:	The proposed model can be trained stably without any alterations to existing models or training paradigms.

Copied to clipboard

Hao Zhao, Zihan Qiu, Huijia Wu, Zili Wang, Zhaofeng He, Jie Fu

Finite State Automata Inside Transformers with Chain-of-Thought: A Mechanistic Study on State Tracking (2025.acl-long)

Challenge:	Existing methods for enhancing performance through increased use of expert knowledge often result in diminishing sparsity during expert selection.
Approach:	They propose a framework that integrates the computational processes of MoE with the concept of knowledge transferring in multi-task learning.
Outcome:	The proposed framework outperforms existing methods under identical conditions concerning the number of experts.

Copied to clipboard

Yifan Zhang, Wenyu Du, Dongming Jin, Jie Fu, Zhi Jin

Structure Learning for Neural Module Networks (D19-64)

Challenge:	Existing studies show that Chain-of-thought (CoT) can enhance the performance of large language models (LLMs) however, there is limited understanding of the algorithms that Transformer+CoT can learn.
Approach:	They propose two metrics to evaluate Transformer+CoT's state tracking capabilities and identify the circuit responsible for tracking the world state.
Outcome:	The proposed model achieves 100% accuracy for each state, highlighting an implicit finite state automaton (FSA) embedded within the model.

Copied to clipboard

Vardaan Pahuja, Jie Fu, Sarath Chandar, Christopher Pal

Challenge:	Neural Module Networks are a class of neural networks that involve human-specified neural modules . current models only learn the parameters of the modules and/or the order of their execution .
Approach:	They propose to learn internal structure and sequence without extra supervisory signals . they use dynamically composable modules which are then assembled into a layout .
Outcome:	The proposed model performs comparable to models using hand-designed modules.

Enhancing Language Model Hypernetworks with Restart: A Study on Optimization (2025.naacl-long)

Copied to clipboard

Yihan Zhang, Jie Fu, Rongrong Ji, Jie Chen

Cognitive Scaffold: From Fluid Context to Crystallized Memory for Long-Horizon DeepResearch Agents (2026.acl-long)

Challenge:	a comprehensive investigation into optimization strategies for hypernetworks remains lacking.
Approach:	They propose restart optimization strategies to improve hypernetworks' performance for language models.
Outcome:	The proposed restart strategy improves hypernetworks' performance for language models, compared to conventional deep neural networks.

Copied to clipboard

Qiuyuan Ai, Zenghuang Fu, Zhaoyang Li, Ping Jiang, Haoyu Wu, Jie Song, Guannan He

Reasoning

Challenge:	Scaling LLM-based agents to long-horizon deep research is constrained by context-noise trade-off . solving a single query may require hundreds of interactions with noisy environments .
Approach:	They propose a factorized memory architecture that decouples the cognitive state into a Fluid Working Context for immediate reasoning and a persistent Knowledge Graph for long-term retention.
Outcome:	The Cognitive Scaffold outperforms baselines on Xbench-DeepSearch, BrowseComp-ZH, and GAIA . it achieves 74.7% Avg@3 and 87.0% Pass@3 on xbench, browseComp, and 88.3% Pass@3.

Multi-Level Cross-Modal Alignment for Speech Relation Extraction (2024.emnlp-main)

Copied to clipboard

Liang Zhang, Zhen Yang, Biao Fu, Ziyao Lu, Liangying Shao, Shiyu Liu, Fandong Meng, Jie Zhou, Xiaoli Wang, Jinsong Su

Unifying Discrete and Continuous Representations for Unsupervised Paraphrase Generation (2023.emnlp-main)

Challenge:	Existing studies use synthetic speech to train and evaluate SpeechRE models, hindering their development . modality gap issue limits performance of existing models, limiting future researches .
Approach:	They propose to use speech data to train and evaluate SpeechRE models by using real speech . they propose to train a cross-modal alignment model to bridge the modality gap .
Outcome:	The proposed model can train to bridge the modality gap between speech encoder and text decoder . the proposed model is based on two real SpeechRE datasets .

Copied to clipboard

Mingfeng Xue, Dayiheng Liu, Wenqiang Lei, Jie Fu, Jian Lan, Mei Li, Baosong Yang, Jun Xie, Yidan Zhang, Dezhong Peng, Jiancheng Lv

Text Generation

Challenge:	Existing unsupervised paraphrase generation methods require large-scale, manually annotated paraphrase datasets, which are labor-intensive to build.
Approach:	They propose a self-supervised pseudo-data construction method that generates diverse pseudo-paraphrases in distinct surface structures for a given sentence.
Outcome:	The proposed method generates diverse pseudo-paraphrases in distinct surface structures for a given sentence.

Interactive Language Learning by Question Answering (D19-1)

Copied to clipboard

Xingdi Yuan, Marc-Alexandre Côté, Jie Fu, Zhouhan Lin, Chris Pal, Yoshua Bengio, Adam Trischler

Reconciliation of Pre-trained Models and Prototypical Neural Networks in Few-shot Named Entity Recognition (2022.findings-emnlp)

Challenge:	Existing machine reading comprehension tasks lack interactive information-seeking component of comprehension.
Approach:	They propose a question-asking task that asks questions in a text-based environment . they propose QAit, which uses a game generator to build models that include deep reinforcement learning agents.
Outcome:	The proposed task poses questions about existence, location, and attributes of objects found in environment.

Copied to clipboard

Youcheng Huang, Wenqiang Lei, Jie Fu, Jiancheng Lv

On Large Language Models’ Hallucination with Regard to Known Facts (2024.naacl-long)

Challenge:	Existing methods do not consider that pre-trained models contain a prominently large amount of information regarding word frequencies, thus biasing prototypical neural networks against learning word entities.
Approach:	They propose a one-line-code normalization method to reconcile such a mismatch with empirical and theoretical grounds and propose 'references' for the model enhancement.
Outcome:	The proposed method outperforms the state-of-the-art models on nine benchmark datasets and is comparable to the state of the art.

Copied to clipboard

Che Jiang, Biqing Qi, Xiangyu Hong, Dayuan Fu, Yang Cheng, Fandong Meng, Mo Yu, Bowen Zhou, Jie Zhou

Interactive Machine Comprehension with Information Seeking Agents (2020.acl-main)

Challenge:	Large language models are successful in answering factoid questions but are also prone to hallucination.
Approach:	They propose self-reporting to the model when faced with such limitations.
Outcome:	The proposed classifier can detect hallucinations with an 88% success rate and can be used to answer factoid questions with correct answer knowledge.

Copied to clipboard

Xingdi Yuan, Jie Fu, Marc-Alexandre Côté, Yi Tay, Chris Pal, Adam Trischler

CIF-Bench: A Chinese Instruction-Following Benchmark for Evaluating the Generalizability of Large Language Models (2024.findings-acl)

Challenge:	Existing machine reading comprehension (MRC) models do not scale effectively to real-world applications like web-level information retrieval and question answering (QA).
Approach:	They propose a method that reframes existing machine reading comprehension (MRC) datasets as interactive, partially observable environments.
Outcome:	The proposed method "occludes" the majority of a document’s text and adds context-sensitive commands that reveal "glimpses" of the hidden text to a model.

Copied to clipboard

Yizhi Li, Ge Zhang, Xingwei Qu, Jiali Li, Zhaoqun Li, Noah Wang, Hao Li, Ruibin Yuan, Yinghao Ma, Kai Zhang, Wangchunshu Zhou, Yiming Liang, Lei Zhang, Lei Ma, Jiajun Zhang, Zuowen Li, Wenhao Huang, Chenghua Lin, Jie Fu

MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcare (2022.lrec-1)

Challenge:	a recent study shows that large language models have limited generalization in low-resource languages like Chinese.
Approach:	They propose to evaluate the zero-shot generalizability of large language models to the Chinese language . they release only half of the dataset publicly, with the remainder kept private .
Outcome:	The Chinese Instruction-Following Benchmark evaluates the generalizability of LLMs to the Chinese language.

Copied to clipboard

Shaoxiong Ji, Tianlin Zhang, Luna Ansari, Jie Fu, Prayag Tiwari, Erik Cambria

Would you Rather? A New Benchmark for Learning Machine Alignment with Cultural Values and Social Preferences (2020.acl-main)

Challenge:	Existing pretrained language models for mental health detection are inadequate . one in four people worldwide suffers from mental disorders .
Approach:	They train and release two pretrained masked language models to benefit machine learning for mental healthcare research . they demonstrate that language representations pretrained in the target domain improve the performance of mental health detection tasks.
Outcome:	The proposed models improve mental health detection tasks on several benchmarks and are available for free.

Copied to clipboard

Yi Tay, Donovan Ong, Jie Fu, Alvin Chan, Nancy Chen, Anh Tuan Luu, Chris Pal

Challenge:	Existing studies on optimal decision-making are limited and only consider individuals in isolation.
Approach:	They propose a task and corpus for learning alignments between machine and human preferences based on a gamified voting game .
Outcome:	The proposed task and corpus show that current state-of-the-art NLP models still leave much room for improvement.

ChatMusician: Understanding and Generating Music Intrinsically with LLM (2024.findings-acl)

Copied to clipboard

Ruibin Yuan, Hanfeng Lin, Yi Wang, Zeyue Tian, Shangda Wu, Tianhao Shen, Ge Zhang, Yuhang Wu, Cong Liu, Ziya Zhou, Liumeng Xue, Ziyang Ma, Qin Liu, Tianyu Zheng, Yizhi Li, Yinghao Ma, Yiming Liang, Xiaowei Chi, Ruibo Liu, Zili Wang, Chenghua Lin, Qifeng Liu, Tao Jiang, Wenhao Huang, Wenhu Chen, Jie Fu, Emmanouil Benetos, Gus Xia, Roger Dannenberg, Wei Xue, Shiyin Kang, Yike Guo

Huatuo-26M, a Large-scale Chinese Medical QA Dataset (2025.findings-naacl)

Challenge:	Despite LLMs' impressive capabilities in musical knowledge, music reasoning remains an unsolved task.
Approach:	They propose an open-source large language model (LLM) that integrates intrinsic musical abilities into LLaMA2 and GPT-3.5.
Outcome:	The proposed model can understand and generate music with a pure text tokenizer without external multi-modal neural structures or tokenizers.

Copied to clipboard

Xidong Wang, Jianquan Li, Shunian Chen, Yuxuan Zhu, Xiangbo Wu, Zhiyi Zhang, Xiaolong Xu, Junying Chen, Jie Fu, Xiang Wan, Anningzhe Gao, Benyou Wang

Defending Against Weight-Poisoning Backdoor Attacks for Parameter-Efficient Fine-Tuning (2024.findings-naacl)

Challenge:	Large Language Models are a powerful tool for medical research, but the data is a bottleneck.
Approach:	They propose to use the largest ever medical Question Answering dataset with 26 Million QA pairs as a fine-tuning data for training large language models.
Outcome:	The proposed dataset demonstrates that it can be used to train large language models and improves zero-shot performance on other datasets.

Copied to clipboard

Shuai Zhao, Leilei Gan, Anh Tuan Luu, Jie Fu, Lingjuan Lyu, Meihuizi Jia, Jinming Wen

Responsible & Trustworthy NLP

Challenge:	Existing methods for parameter-efficient fine-tuning (PEFT) are not effective for weight-poisoning backdoor attacks.
Approach:	They propose a parameter-efficient fine-tuning (PEFT) method that updates only a limited set of model parameters and provides a robust defense against weight-poisoning backdoor attacks.
Outcome:	The proposed method identifies poisoned samples through confidence and is robust against weight-poisoning backdoor attacks.

Unlocking Emergent Modularity in Large Language Models (2024.naacl-long)

Copied to clipboard

Zihan Qiu, Zeyu Huang, Jie Fu

Towards Robust Visual Question Answering: Making the Most of Biased Samples via Contrastive Learning (2022.findings-emnlp)

Challenge:	Existing MNNs are explicit, with predefined modular architectures and individual modules expected to implement distinct functions.
Approach:	They propose to unlock emergent modularity in language models by fine-tuning them as Mixture-of-Experts (MoEs) EMoE is robust to various configurations and can scale up to Large Language Models .
Outcome:	The proposed models can be fine-tuned as Mixture-of-Expert (MoE) counterparts without introducing any extra parameters.

Copied to clipboard

Qingyi Si, Yuanxin Liu, Fandong Meng, Zheng Lin, Peng Fu, Yanan Cao, Weiping Wang, Jie Zhou

On Orthogonality Constraints for Transformers (2021.acl-short)

Challenge:	Recent studies have shown that biased samples can be brittle for VQA models . however, the improvements on OOD data severely sacrifice the performance on the in-distribution (ID) data.
Approach:	They propose a contrastive learning approach that exploits biased samples for unbiased information that contributes to reasoning.
Outcome:	The proposed method achieves competitive performance on the OOD dataset while maintaining robustness on the ID dataset.

Copied to clipboard

Aston Zhang, Alvin Chan, Yi Tay, Jie Fu, Shuohang Wang, Shuai Zhang, Huajie Shao, Shuochao Yao, Roy Ka-Wei Lee

CMDAG: A Chinese Metaphor Dataset with Annotated Grounds as CoT for Boosting Metaphor Generation (2024.lrec-main)

Challenge:	a dedicated study on orthogonality constraints for transformers has been lacking . plug-and-play constraints increase the BLEU of transformers .
Approach:	They propose to use plug-and-play constraints to encourage matrices to be orthogonal for numerical stability.
Outcome:	The proposed constraint increases the BLEU on the large-scale WMT’16 EnDe benchmark by a factor of 28.4 to 29.6.

Copied to clipboard

Yujie Shao, Xinrong Yao, Xingwei Qu, Chenghua Lin, Shi Wang, Wenhao Huang, Ge Zhang, Jie Fu

Challenge:	Metaphors are a prominent linguistic device in human language and literature, as they add color, imagery, and emphasis to enhance effective communication.
Approach:	They propose a large-scale high quality annotated Chinese Metaphor Corpus . they use a set of guidelines to ensure the accuracy and consistency of their annotations .
Outcome:	The proposed corpus generates metaphors that resonate more with real-world intuition.

GLGE: A New General Language Generation Evaluation Benchmark (2021.findings-acl)

Copied to clipboard

Dayiheng Liu, Yu Yan, Yeyun Gong, Weizhen Qi, Hang Zhang, Jian Jiao, Weizhu Chen, Jie Fu, Linjun Shou, Ming Gong, Pengcheng Wang, Jiusheng Chen, Daxin Jiang, Jiancheng Lv, Ruofei Zhang, Winnie Wu, Ming Zhou, Nan Duan

A Closer Look into Mixture-of-Experts in Large Language Models (2025.findings-naacl)

Challenge:	Multi-task benchmarks focus on a range of Natural Language Understanding (NLU) tasks without considering the Natural Language Generation (NLG) models.
Approach:	They propose a multi-task benchmark for evaluating the generalization capabilities of NLG models across eight language generation tasks.
Outcome:	The proposed benchmarks are based on GLUE and Su-perGLUE for English and several other languages.

Copied to clipboard

Ka Man Lo, Zeyu Huang, Zihan Qiu, Zili Wang, Jie Fu

MIO: A Foundation Model on Multimodal Tokens (2025.emnlp-main)

Challenge:	Mixture-of-experts (MoE) architectures are gaining increasing attention for their unique properties and remarkable performance.
Approach:	They propose a mixture-of-experts architecture that allows for model scaling without sacrificing computational efficiency.
Outcome:	The proposed model increases model size without sacrificing computational efficiency . the proposed model is modular and can be used by a broad spectrum of practitioners .

Copied to clipboard

Zekun Moore Wang, King Zhu, Chunpu Xu, Wangchunshu Zhou, Jiaheng Liu, Yibo Zhang, Jessie Wang, Ning Shi, Siyu Li, Yizhi Li, Haoran Que, Zhaoxiang Zhang, Yuanxing Zhang, Ge Zhang, Ke Xu, Jie Fu, Wenhao Huang

Graph Neural Networks with Generated Parameters for Relation Extraction (P19-1)

Challenge:	Existing models lack multimodal understanding capabilities, resulting in closed-source model that does not support multimodal interleaved sequences.
Approach:	They propose a foundation model built on multimodal tokens capable of understanding and generating speech, text, images, and videos in an end-to-end, autoregressive manner.
Outcome:	The proposed model is able to understand speech, text, images, and videos in an end-to-end, autoregressive manner.

Copied to clipboard

Hao Zhu, Yankai Lin, Zhiyuan Liu, Jie Fu, Tat-Seng Chua, Maosong Sun

Information Extraction & Text Mining

Challenge:	Existing graph neural networks can only process multi-hop relational reasoning on pre-defined graphs and cannot be directly applied in natural language relational reasoning.
Approach:	They propose a graph neural network with generated parameters using natural language sentences as inputs.
Outcome:	The proposed model can process relational reasoning on graphs and in natural language processing tasks.

RikiNet: Reading Wikipedia Pages for Natural Question Answering (2020.acl-main)

Copied to clipboard

Dayiheng Liu, Yeyun Gong, Jie Fu, Yu Yan, Jiusheng Chen, Daxin Jiang, Jiancheng Lv, Nan Duan

Natural Language Interfaces

Challenge:	Using Wikipedia pages to answer open-domain questions remains challenging in natural language understanding.
Approach:	They propose a model which reads Wikipedia pages for natural question answering . it uses a dynamic paragraph dual-attention reader and a cascaded answer predictor .
Outcome:	The proposed model outperforms the human model on the Natural Questions dataset . it achieves 74.3 F1 and 57.9 F1 on long-answer and short-answer tasks .

Prototype-based HyperAdapter for Sample-Efficient Multi-task Tuning (2023.emnlp-main)

Copied to clipboard

Hao Zhao, Jie Fu, Zhaofeng He

Responsible & Trustworthy NLP

Challenge:	Existing methods for fine-tuning pre-trained language models are limited to low-data regimes and require learning different modules to adapt to diverse tasks.
Approach:	They propose a framework for parameter-efficient fine-tuning that trains modules per task . they use an instance-dense retriever and a prototypical hypernetwork to generate conditional modules .
Outcome:	The proposed framework outperforms existing methods on multi-task learning and few-shot transfer learning.

Unlocking Continual Learning Abilities in Language Models (2024.findings-emnlp)

Copied to clipboard

Wenyu Du, Shuang Cheng, Tongxu Luo, Zihan Qiu, Zeyu Huang, Ka Chun Cheung, Reynold Cheng, Jie Fu

Learning to Win Lottery Tickets in BERT Transfer via Task-agnostic Mask Training (2022.naacl-main)

Challenge:	Existing approaches to learning models (LMs) incorporate old task data or task-wise inductive bias into LMs, but old data and accurate task information are often unavailable or costly to collect.
Approach:	They propose a rehearsal-free method that updates model parameters with large magnitudes . they found that the L1-normalized magnitude distribution is different when different task data is used .
Outcome:	The proposed method improves accuracy and performance on four CL benchmarks.

Copied to clipboard

Yuanxin Liu, Fandong Meng, Zheng Lin, Peng Fu, Yanan Cao, Weiping Wang, Jie Zhou

Lightweight and Efficient Neural Natural Language Processing with Quaternion Networks (P19-1)

Challenge:	Recent studies show pre-trained language models contain matching subnetworks that have similar transfer learning performance as the original PLM.
Approach:	They propose to prune matching subnetworks using magnitude-based pruning . they propose to optimize the subnetwork structure towards the pre-training objectives .
Outcome:	The proposed method is more efficient in searching subnetworks and advantageous when fine-tuning within a range of data scarcity.

Copied to clipboard

Yi Tay, Aston Zhang, Anh Tuan Luu, Jinfeng Rao, Shuai Zhang, Shuohang Wang, Jie Fu, Siu Cheung Hui

Green & Sustainable NLP

Challenge:	Existing models for natural language processing are heavily parameterized and memory inefficient.
Approach:	They propose a series of lightweight and memory efficient neural architectures for NLP tasks . they propose quaternion algebra and hypercomplex spaces for computation .
Outcome:	The proposed models enable expressive inter-component interactions and significantly reduce parameter size without loss of performance.

Question-Interlocutor Scope Realized Graph Modeling over Key Utterances for Dialogue Reading Comprehension (2023.findings-acl)

Copied to clipboard

Jiangnan Li, Mo Yu, Fandong Meng, Zheng Lin, Peng Fu, Weiping Wang, Jie Zhou

Natural Language Interfaces

Challenge:	Compared to standard RC tasks, dialogue reading comprehension (DRC) has raised challenges because of the complex speaker information and noisy dialogue context.
Approach:	They propose a new method for dialogue reading comprehension that extracts answers from dialogues by using key-utterances-extracting methods and a Question-Interlocutor Scope Realized Graph.
Outcome:	The proposed method achieves state-of-the-art performance against previous works.

Text Editing as Imitation Game (2022.findings-emnlp)

Copied to clipboard

Ning Shi, Bin Tang, Bo Yuan, Longtao Huang, Yewen Pu, Jie Fu, Zhouhan Lin

Syntactic Text Processing

Challenge:	Text editing is an important domain of processing tasks to edit the text in a localized fashion, such as text simplification.
Approach:	They propose a nonautoregressive decoder for state-to-action demonstrations that parallels the decoding while retaining the dependencies between tokens.
Outcome:	The proposed model outperforms the autoregressive baselines on a suite of Arithmetic Equation benchmarks in terms of performance, efficiency, and robustness.

SciMMIR: Benchmarking Scientific Multi-modal Information Retrieval (2024.findings-acl)

Copied to clipboard

Siwei Wu, Yizhi Li, Kang Zhu, Ge Zhang, Yiming Liang, Kaijing Ma, Chenghao Xiao, Haoran Zhang, Bohao Yang, Wenhu Chen, Wenhao Huang, Noura Al Moubayed, Jie Fu, Chenghua Lin

RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models (2024.findings-acl)

Challenge:	Multi-modal information retrieval (MMIR) is a rapidly evolving field . current benchmarks for image-text pairings overlook the scientific domain .
Approach:	They develop a scientific domain-specific MMIR benchmark to evaluate image-text pairings using open-access research paper corpora.
Outcome:	The proposed benchmarks are based on 530K image-text pairs extracted from scientific documents with detailed captions.

Copied to clipboard

Noah Wang, Z.y. Peng, Haoran Que, Jiaheng Liu, Wangchunshu Zhou, Yuhan Wu, Hongcheng Guo, Ruitong Gan, Zehao Ni, Jian Yang, Man Zhang, Zhaoxiang Zhang, Wanli Ouyang, Ke Xu, Wenhao Huang, Jie Fu, Junran Peng

Efficient Domain Continual pretraining by Mitigating the Stability Gap (2025.acl-long)

Challenge:	Large Language Models (LLMs) have paved the way for complex tasks such as role-playing.
Approach:	They propose a framework to benchmark, elicit, and enhance role-playing abilities in Large Language Models.
Outcome:	The proposed framework improves role-playing abilities with 168,093 samples.

Copied to clipboard

Yiduo Guo, Jie Fu, Huishuai Zhang, Dongyan Zhao

Prompt as Triggers for Backdoor Attack: Examining the Vulnerability in Language Models (2023.emnlp-main)

Challenge:	Continual pretraining is an important approach for Large Language Models to improve their performance in target domains, learn new topics and languages, and even boost their general capabilities.
Approach:	They propose a training strategy that mitigates instability by increasing the number of epochs, along with two data sampling strategies targeting data domain relevance and corpus distribution.
Outcome:	The proposed training strategy improves the average medical task performance of the OpenLlama-3B model from 36.2% to 40.7% using only 40% of the original training budget, while also enhancing general task performance without causing forgetting.

Copied to clipboard

Shuai Zhao, Jinming Wen, Anh Luu, Junbo Zhao, Jie Fu

Responsible & Trustworthy NLP

Challenge:	ProAttack is a novel and efficient method for performing clean-label backdoor attacks based on the prompt, which uses the prompt itself as a trigger.
Approach:	They propose a method for performing clean-label backdoor attacks based on the prompt, which uses the prompt itself as a trigger.
Outcome:	The proposed method achieves state-of-the-art performance on several NLP tasks, particularly in few-shot settings.

OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models (2025.acl-long)

Copied to clipboard

Siming Huang, Tianhao Cheng, Jason Klein Liu, Weidi Xu, Jiaran Hao, Liuyihan Song, Yang Xu, Jian Yang, Jiaheng Liu, Chenchen Zhang, Linzheng Chai, Ruifeng Yuan, Xianzhen Luo, Qiufeng Wang, YuanTao Fan, Qingfu Zhu, Zhaoxiang Zhang, Yang Gao, Jie Fu, Qian Liu, Houyi Li, Ge Zhang, Yuan Qi, Xu Yinghui, Wei Chu, Zili Wang

Diverse, Controllable, and Keyphrase-Aware: A Corpus and Method for News Multi-Headline Generation (2020.emnlp-main)

Challenge:	Code LLMs lack reproducible data pipelines and training protocols for reproducible advancements in code intelligence.
Approach:	They propose a top-tier code LLM that releases model weights and inference code . reproducible data pipelines, rigorous experimental ablation results and training protocols are included .
Outcome:	The proposed model achieves comparable performance to leading models and serves as an "open cookbook" reproducible training data, rigorous experimental ablation results, and detailed training protocols are also included in the model.

Copied to clipboard

Dayiheng Liu, Yeyun Gong, Yu Yan, Jie Fu, Bo Shao, Daxin Jiang, Jiancheng Lv, Nan Duan

Text Generation

Challenge:	Existing methods for news headline generation focus on producing a single short sentence . et al., 2017; Gehrmann e.t., 2018; Zhong ee., 2019) focus on single-headline generation.
Approach:	They propose a method to generate multiple headlines with keyphrases of user interests . they propose generating multiple keyphrase-relevant headlines using a transformer decoder .
Outcome:	The proposed method achieves state-of-the-art in terms of quality and diversity.

P-Tuning: Prompt Tuning Can Be Comparable to Fine-tuning Across Scales and Tasks (2022.acl-short)

Copied to clipboard

Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Tam, Zhengxiao Du, Zhilin Yang, Jie Tang

COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuning (2025.findings-naacl)

Challenge:	Existing methods of prompt tuning cannot handle hard sequence labeling tasks.
Approach:	They propose to optimize prompt tuning to tune continuous prompts with a frozen language model.
Outcome:	The proposed method matches finetuning with prompt tuning while having only 0.1%-3% tuned parameters.

Copied to clipboard

Yuelin Bai, Xeron Du, Yiming Liang, Leo Jin, Junting Zhou, Ziqiang Liu, Feiteng Fang, Mingshan Chang, Tianyu Zheng, Xincheng Zhang, Nuo Ma, Zekun Moore Wang, Ruibin Yuan, Haihong Wu, Hongquan Lin, Wenhao Huang, Jiajun Zhang, Chenghua Lin, Jie Fu, Min Yang, Shiwen Ni, Ge Zhang

AgencyBench: Benchmarking the Frontiers of Autonomous Agents in 1M-Token Real-World Contexts (2026.acl-long)

Challenge:	Existing datasets for Chinese instruction tuning are not well-aligned with Chinese users’ interaction patterns.
Approach:	They propose to use Chinese instruction tuning datasets to improve instruction fine-tuning for Chinese users.
Outcome:	The proposed dataset shows that Chinese models achieve competitive performance in diverse benchmarks.

Copied to clipboard

Keyu Li, Junhao Shi, Yang Xiao, Mohan Jiang, Jie Sun, Yunze Wu, Dayuan Fu, Shijie Xia, Xiaojie Cai, Tianze Xu, Weiye Si, Wenjie Li, Dequan Wang, Pengfei Liu

Challenge:	Existing benchmarks focus on single agentic capability, failing to capture long-horizon real-world scenarios.
Approach:	They propose a benchmark that evaluates 6 agentic capabilities across 32 real-world scenarios.
Outcome:	Experiments show that closed-source models outperform open-source model (48.4% vs 32.1%) integrating models with advanced scaffolds to form autonomous agents is a paradigm shift.

TeachMaster: Generative Teaching via Code (2026.acl-industry)

Copied to clipboard

Yuheng Wang, Runde Yang, Lin Wu, Jie Zhang, Jingru Fan, Tianle Zhou, Ruoyu Fu, Huatao Li, Ruijie Shi, Siheng Chen, Weinan E, Chen Qian

Language Prior Is Not the Only Shortcut: A Benchmark for Shortcut Learning in VQA (2022.findings-emnlp)

Challenge:	Existing methods for creating video content are limited by high costs and slow update cycles.
Approach:	They propose a paradigm shifting educators from manual creators to high-level directors who focus on pedagogical intents while agents handle execution.
Outcome:	The proposed framework reduces production costs to 0.3% of traditional course videos and provides a robust solution for scalable education.

Copied to clipboard

Qingyi Si, Fandong Meng, Mingyu Zheng, Zheng Lin, Yuanxin Liu, Peng Fu, Yanan Cao, Weiping Wang, Jie Zhou