Papers by Xiao Fu
Say What You Mean! Large Language Models Speak Too Positively about Negative Commonsense Knowledge (2023.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have been studied for their ability to store and utilize positive knowledge. |
| Approach: | They propose to use a constrained keywords-to-sentence generation task and a Boolean question answering task to probe large language models on negative commonsense knowledge. |
| Outcome: | The proposed tasks show that LLMs fail to generate valid sentences grounded in negative commonsense knowledge, yet they can correctly answer yes-or-no questions. |
IFlyEA: A Chinese Essay Assessment System with Automated Rating, Review Generation, and Recommendation (2021.acl-demo)
Copied to clipboard
| Challenge: | Automated Essay Assessment (AEA) aims to judge students’ writing proficiency in an automatic way. |
| Approach: | They propose to use Chinese AEA system IFlyEssayAssess to evaluate essays written by native Chinese students from primary and junior schools. |
| Outcome: | The proposed system provides application services for essay scoring, review generation, recommendation, and explainable analytical visualization. |
ExplainaBoard: An Explainable Leaderboard for NLP (2021.acl-demo)
Copied to clipboard
Pengfei Liu, Jinlan Fu, Yang Xiao, Weizhe Yuan, Shuaichen Chang, Junqi Dai, Yixin Liu, Zihuiwen Ye, Graham Neubig
| Challenge: | Using leaderboards, researchers can track the performance of various systems on various NLP tasks. |
| Approach: | They propose a new conceptualization and implementation of NLP evaluation using a leaderboard. |
| Outcome: | The ExplainaBoard is an evaluation tool for natural language processing (NLP) it covers more than 400 systems, 50 datasets, 40 languages, and 12 tasks. |
E-KAR: A Benchmark for Rationalizing Natural Language Analogical Reasoning (2022.findings-acl)
Copied to clipboard
Jiangjie Chen, Rui Xu, Ziquan Fu, Wei Shi, Zhongqiao Li, Xinbo Zhang, Changzhi Sun, Lei Li, Yanghua Xiao, Hao Zhou
| Challenge: | Existing benchmarks to test word analogy do not reveal the underneath process of analogical reasoning of neural models. |
| Approach: | They propose an explanation benchmark for analogical reasoning using a Civil Service exam . they use a free-text explanation scheme to explain whether an analogy should be drawn . |
| Outcome: | The proposed benchmark is very challenging for state-of-the-art models, it is found. |
Safety Alignment in NLP Tasks: Weakly Aligned Summarization as an In-Context Attack (2024.acl-long)
Copied to clipboard
| Challenge: | Recent developments in balancing usefulness and safety of large language models raise a critical question . current attacks, especially adversarial ones that manipulate malicious prompts, often aim to manipulate the input . |
| Approach: | They show that LLMs can effectively summarize malicious long documents but often refuse to translate them. |
| Outcome: | The findings highlight a vulnerability in LLMs that can't translate or summarize documents . the study focuses on LLM models, Gemini and GPT-4, which can' be exploited . |
SceneGenAgent: Precise Industrial Scene Generation with Coding Agent (2025.acl-long)
Copied to clipboard
| Challenge: | Recent work on scene generation focuses on generating 3D scenes from textual descriptions . however, the task of generating industrial scenes with LLMs is complex and requires precise measurements and positioning . |
| Approach: | They propose an LLM-based agent for generating industrial scenes through C# code. |
| Outcome: | Experiments show that LLMs powered by SceneGenAgent exceed their original performance . the agent achieves 81.0% success rate in real-world industrial scene generation tasks . |
Data-Efficient Selection via Grammatical Complexity in Continual Pre-training of Domain-Specific LLMs (2025.emnlp-main)
Copied to clipboard
Yizhou Ying, Geng Zhang, Cui Danxin, Chengyu Du, Guanglei Yue, Sihang Jiang, Jiaqing Liang, Yifei Fu, Hailin Hu, Yanghua Xiao
| Challenge: | Existing data selection strategies for continual pre-training of large language models often rely on scarce labeled data or computationally expensive LLMs. |
| Approach: | They propose an annotation-independent data selection framework for CPT that evaluates grammatical complexity using lexical diversity and syntactic complexity. |
| Outcome: | The proposed framework outperforms baselines on a financial dataset and surpasses full-data training by 1.7% using only 20% of the data. |
Sentipolis: Emotion-Aware Agents for Social Simulations (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in reasoning and long-context memory are making large language models (LLMs) appear increasingly human-like, which has led researchers to adopt LLM agents as a substrate for social simulation. |
| Approach: | They propose a framework for emotionally stateful agents that integrates continuous Pleasure-Arousal-Dominance representation, dual-speed emotion dynamics, and emotion–memory coupling. |
| Outcome: | The proposed framework improves emotional grounded behavior, boosting communication, and emotional continuity across thousands of interactions over multiple base models and evaluators. |
Neural Multitask Learning for Simile Recognition (D18-1)
Copied to clipboard
| Challenge: | Simile is a special type of metaphor, where comparators such as like and as are used to compare two objects. |
| Approach: | They propose a neural network framework for simile sentence classification, simile component extraction and language modeling. |
| Outcome: | The proposed framework outperforms rule-based and feature-based approaches in simile sentence classification and simile component extraction tasks. |
Are All the Datasets in Benchmark Necessary? A Pilot Study of Dataset Evaluation for Text Classification (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing benchmark datasets contribute little to discriminating top-scoring systems, while those less used datasets exhibit impressive discriminative power. |
| Approach: | They examine the distinguishability of benchmark datasets when comparing different systems . they find that existing benchmark dataset contribute little to discriminating top-scoring systems - whereas those less used datasets exhibit impressive discriminative power. |
| Outcome: | The proposed datasets are released on DataLab. |
MARK: Multi-agent Collaboration with Ranking Guidance for Text-attributed Graph Clustering (2025.findings-acl)
Copied to clipboard
Yiwei Fu, Yuxing Zhang, Chunchun Chen, JianwenMa JianwenMa, Quan Yuan, Rong-Cheng Tu, Xinli Huang, Wei Ye, Xiao Luo, Minghua Deng
| Challenge: | Existing approaches to cluster graphs with GNNs are limited due to label scarcity. |
| Approach: | They propose to leverage large language models to enhance text-attributed graph clustering by using three LLMs as ranking-based supervision signals. |
| Outcome: | The proposed approach generates reliable guidance using collaboration of three LLM-based agents as ranking-based supervision signals. |
From Remembering to Metacognition: Do Existing Benchmarks Accurately Evaluate LLMs? (2025.findings-emnlp)
Copied to clipboard
Geng Zhang, Yizhou Ying, Sihang Jiang, Jiaqing Liang, Guanglei Yue, Yifei Fu, Hailin Hu, Yanghua Xiao
| Challenge: | Existing benchmark datasets focus on low-level cognitive tasks while providing limited coverage of higher-level reasoning skills. |
| Approach: | They analyze the cognitive depth of popular LLM benchmarks using Bloom’s Taxonomy to evaluate both the cognitive and knowledge dimensions. |
| Outcome: | The results show that incorporating higher-level cognitive instructions into the current instruction fine-tuning process improves model performance. |
ALPS: Attention Localization and Pruning Strategy for Efficient Adaptation of Large Language Models (2025.findings-acl)
Copied to clipboard
Hao Chen, Haoze Li, Zhiqing Xiao, Lirong Gao, Qi Zhang, Xiaomeng Hu, Ningtao Wang, Xing Fu, Junbo Zhao
| Challenge: | Prior research has focused on optimizing general-purpose large language models to downstream tasks . however, these approaches inherently introduce data dependency, which hinders generalization and reusability. |
| Approach: | They propose an algorithm that localizes the most task-sensitive attention heads and prunes by restricting attention training updates to these heads, thereby reducing alignment costs. |
| Outcome: | The proposed algorithm achieves 2% performance improvement over baselines on three tasks while localizing the most task-sensitive attention heads. |
P4: Plug-and-Play Discrete Prompting for Large Language Models Personalization (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) exhibit impressive capabilities in following instructions, but manually prompting them to exhibit certain personalities may result in sub-optimal performance. |
| Approach: | They propose a plug-and-play prompting method to manipulate Large Language Models with distinct human-like personality traits by appending discrete personalized suffixes to query or dialog histories and focusing exclusively on influential tokens. |
| Outcome: | The proposed method outperforms other prompting methods and model editing methods on four models ranging from 1.1B to 13B and achieves 79.9% accuracy in customizing LLMs’ personalities. |
TokenTiming: A Dynamic Alignment Method for Universal Speculative Decoding Model Pairs (2026.acl-long)
Copied to clipboard
| Challenge: | Speculative decoding (SD) is a useful tool for accelerating large language models . but its utility is limited by a fundamental constraint: draft and target models must share the same vocabulary . |
| Approach: | They propose an algorithm that uses a draft token sequence to get a new target token sequence and then uses DTW to build a mapping to transfer probability distributions. |
| Outcome: | The proposed method shows 1.57x speedup on various tasks. |
ViDove: A Translation Agent System with Multimodal Context and Memory-Augmented Reasoning (2025.emnlp-demos)
Copied to clipboard
Yichen Lu, Wei Dai, Jiaen Liu, Ching Wing Kwok, Zongheng Wu, Xudong Xiao, Ao Sun, Sheng Fu, Jianyuan Zhan, Yian Wang, Takatomo Saito, Sicheng Lai
| Challenge: | Recent advances in Large Language Models (LLMs) have demonstrated remarkable capabilities in Machine Translation (MT) tasks. |
| Approach: | They propose a translation agent system designed for multimodal input that leverages visual and contextual background information to enhance the translation process. |
| Outcome: | The proposed translation agent achieves significantly higher translation quality in subtitle generation and general translation tasks compared to previous state-of-the-art systems. |
SciText2Eq: Assessing LLMs for Explainable Equation Generation for Scientific Creativity (2026.findings-acl)
Copied to clipboard
| Challenge: | Prior work has addressed problems in unstructured grounding, multi-equation dependency, and human-aligned evaluation. |
| Approach: | They construct a dataset of scientific texts and evaluate it using an explainable equation generation workflow using automatic metrics and human judgments. |
| Outcome: | The proposed model achieves moderate performance on lexical and syntactic similarity, but struggles with semantic accuracy. |
SciMMIR: Benchmarking Scientific Multi-modal Information Retrieval (2024.findings-acl)
Copied to clipboard
Siwei Wu, Yizhi Li, Kang Zhu, Ge Zhang, Yiming Liang, Kaijing Ma, Chenghao Xiao, Haoran Zhang, Bohao Yang, Wenhu Chen, Wenhao Huang, Noura Al Moubayed, Jie Fu, Chenghua Lin
| Challenge: | Multi-modal information retrieval (MMIR) is a rapidly evolving field . current benchmarks for image-text pairings overlook the scientific domain . |
| Approach: | They develop a scientific domain-specific MMIR benchmark to evaluate image-text pairings using open-access research paper corpora. |
| Outcome: | The proposed benchmarks are based on 530K image-text pairs extracted from scientific documents with detailed captions. |
AIGT: AI Generative Table Based on Prompt (2025.coling-main)
Copied to clipboard
| Challenge: | Tabular data is an essential resource for many fields, but current methods do not fully utilize the rich information available in tables. |
| Approach: | They propose a method that utilizes metadata information to generate tabular data . they propose long-token partitioning algorithms that enable AIGT to model tables of any scale . |
| Outcome: | The proposed approach achieves state-of-the-art on 14 out of 20 public datasets and two real industry datasets within the Alipay risk control system. |
Lexicon Enhanced Chinese Sequence Labeling Using BERT Adapter (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for Chinese sequence labelling only fuse lexicon features via a shallow and random initialized sequence layer and do not integrate them into the bottom layers of BERT. |
| Approach: | They propose a Lexicon Enhanced BERT model which integrates external lexicon knowledge into BERT layers directly by a lexiccon Adapter layer. |
| Outcome: | The proposed model integrates external lexicon knowledge into BERT layers directly by a Lexicon Adapter layer. |
DataLab: A Platform for Data Analysis and Intervention (2022.acl-demo)
Copied to clipboard
Yang Xiao, Jinlan Fu, Weizhe Yuan, Vijay Viswanathan, Zhoumianze Liu, Yixin Liu, Graham Neubig, Pengfei Liu
| Challenge: | Existing tools and research focus on how to interpret and manipulate data, despite its crucial role in machine learning, . existing tools and researchers focus on systems on top of existing data, rather than how to use it. |
| Approach: | They propose a unified data-oriented platform that allows users to interactively analyze the characteristics of data and provides a standard interface for many data processing operations. |
| Outcome: | The proposed platform allows users to analyze the characteristics of data and provides a standardized interface so that many data processing operations can be provided within a single interface. |
GumbelSoft: Diversified Language Model Watermarking via the GumbelMax-trick (2024.acl-long)
Copied to clipboard
| Challenge: | Large language models generate human-like content, but they also pose a problem with generation diversity, negatively impacting generation diversity and user experience. |
| Approach: | They propose a Logits-Addition watermark and three variants that aim to enhance diversity to overcome generation diversity challenges. |
| Outcome: | The Logits-Addition watermark outperforms the Logits+Trick-based watermark in diversity tests and outperformed other decoding-based methods by 0.1 to 0.3. |
Anchoring the Cache: Mitigating Contextual Hallucination in KV-Compressed Long-Context Summarization (2026.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that KV cache compression can increase hallucination scores in LLMs . modern LLM models support extremely long sequences, but their impact on model hallucinosity remains underexplored. |
| Approach: | They propose a decoding-phase strategy that selectively removes generated KV pairs from retrieval heads responsible for retrieving critical information from source context. |
| Outcome: | The proposed method reduces hallucination across multiple models and datasets while preserving computational efficiency. |
P-Tuning: Prompt Tuning Can Be Comparable to Fine-tuning Across Scales and Tasks (2022.acl-short)
Copied to clipboard
| Challenge: | Existing methods of prompt tuning cannot handle hard sequence labeling tasks. |
| Approach: | They propose to optimize prompt tuning to tune continuous prompts with a frozen language model. |
| Outcome: | The proposed method matches finetuning with prompt tuning while having only 0.1%-3% tuned parameters. |
AgencyBench: Benchmarking the Frontiers of Autonomous Agents in 1M-Token Real-World Contexts (2026.acl-long)
Copied to clipboard
Keyu Li, Junhao Shi, Yang Xiao, Mohan Jiang, Jie Sun, Yunze Wu, Dayuan Fu, Shijie Xia, Xiaojie Cai, Tianze Xu, Weiye Si, Wenjie Li, Dequan Wang, Pengfei Liu
| Challenge: | Existing benchmarks focus on single agentic capability, failing to capture long-horizon real-world scenarios. |
| Approach: | They propose a benchmark that evaluates 6 agentic capabilities across 32 real-world scenarios. |
| Outcome: | Experiments show that closed-source models outperform open-source model (48.4% vs 32.1%) integrating models with advanced scaffolds to form autonomous agents is a paradigm shift. |
Transparent and Scrutable Recommendations Using Natural Language User Profiles (2024.acl-long)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) rely on implicit or explicit feedback from users to suggest new items, resulting in a lack of transparency and a user's ability to scrutinize and modify their preferences. |
| Approach: | They propose to use a natural language (NL) user profile to summarize a user's preferences and then use it to fine-tune a LLM using only NL profiles to make transparent and scrutable recommendations. |
| Outcome: | The proposed model performs on two benchmarking rating prediction datasets and is comparable to existing models. |
TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing (2021.acl-demo)
Copied to clipboard
Xiao Wang, Qin Liu, Tao Gui, Qi Zhang, Yicheng Zou, Xin Zhou, Jiacheng Ye, Yongxin Zhang, Rui Zheng, Zexiong Pang, Qinzhuo Wu, Zhengyan Li, Chong Zhang, Ruotian Ma, Zichu Fei, Ruijian Cai, Jun Zhao, Xingwu Hu, Zhiheng Yan, Yiding Tan, Yuan Hu, Qiyuan Bian, Zhihua Liu, Shan Qin, Bolin Zhu, Xiaoyu Xing, Jinlan Fu, Yue Zhang, Minlong Peng, Xiaoqing Zheng, Yaqian Zhou, Zhongyu Wei, Xipeng Qiu, Xuanjing Huang
| Challenge: | Existing approaches to textual robustness evaluation focus on slightly modifying the input data, which maintains the original meaning and results in a different prediction. |
| Approach: | They propose a multilingual robustness evaluation toolkit for NLP that integrates universal text transformations, task-specific transformations and adversarial attack. |
| Outcome: | The toolkit includes universal text transformation, task-specific transformation, adversarial attack, subpopulation, and their combinations to provide comprehensive robustness analyses. |
Distilling Script Knowledge from Large Language Models for Constrained Language Planning (2023.acl-long)
Copied to clipboard
Siyu Yuan, Jiangjie Chen, Ziquan Fu, Xuyang Ge, Soham Shah, Charles Jankowski, Yanghua Xiao, Deqing Yang
| Challenge: | Existing work exploits language models to plan for abstract goals of stereotypical activities, but leaves more specific goals with multi-facet constraints understudied. |
| Approach: | They propose an over-generate-then-filter approach to improve large language models on constrained language planning task by distilling a constrained script dataset. |
| Outcome: | The proposed approach improves the constrained language planning ability of large language models on constraint faithfulness and also in smaller LMs. |