Papers by Tao Ji

Length Generalization of Causal Transformers without Position Encoding (2024.findings-acl)

Challenge:	Existing benchmarks evaluate agents in simplified, idealized settings, relying on pre-packaged tool interfaces, overlooking critical steps, and assume inputs are clean and fully specified.
Approach:	They propose a framework that evaluates language agents in simplified, idealized settings . they show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 .
Outcome:	Experiments on 15 proprietary and open-source models show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 .

Copied to clipboard

Jie Wang, Tao Ji, Yuanbin Wu, Hang Yan, Tao Gui, Qi Zhang, Xuanjing Huang, Xiaoling Wang

Rehearsal-free Continual Language Learning via Efficient Parameter Isolation (2023.acl-long)

Challenge:	Besides Transformers without position encodings, the success of NoPE provides a new way to overcome the challenge of generalizing to longer sentences.
Approach:	They propose a parameter-efficient tuning for searching attention heads’ best temperature hyper-parameters, which substantially expands NoPE’s context size.
Outcome:	The proposed tuning significantly expands NoPE's context size, allowing it to generalize to longer sentences with state-of-the-art generalization algorithms.

Copied to clipboard

Zhicheng Wang, Yufang Liu, Tao Ji, Xiaoling Wang, Yuanbin Wu, Congcong Jiang, Ye Chao, Zhencong Han, Ling Wang, Xu Shao, Wenqiu Zeng

WebWISE: Unlocking Web Interface Control for LLMs via Sequential Exploration (2024.findings-naacl)

Challenge:	Existing methods for learning continual tasks do not cache history data, which makes the problem more challenging.
Approach:	They propose a method that allocates a small portion of private parameters and learns them with a shared pre-trained model.
Outcome:	The proposed method is comparable to existing methods and comparable to those using historical data.

Copied to clipboard

Heyi Tao, Sethuraman T V, Michal Shlapentokh-Rothman, Tanmay Gupta, Heng Ji, Derek Hoiem

Word Reordering for Zero-shot Cross-lingual Structured Prediction (2021.emnlp-main)

Challenge:	Prior work to control software has used reinforcement learning (RL), requiring many demonstrations and trials to learn simple interaction tasks.
Approach:	They propose a Large Language Model to automatically perform web software tasks using click, scroll, and text in- put operations using filtered Document Object Models as observations.
Outcome:	The proposed method performs better on the MiniWob++ benchmark with only one in-context example.

Copied to clipboard

Tao Ji, Yong Jiang, Tao Wang, Zhongqiang Huang, Fei Huang, Yuanbin Wu, Xiaoling Wang

Low-Resource NLP

Challenge:	Current sentence encoders are word order sensitive, resulting in poor performance . Adapting word order from one language to another is key in cross-lingual structured prediction.
Approach:	They propose a new module to organize words following the source language order . they build structured prediction models with bag-of-words inputs and introduce a module to do this .
Outcome:	The proposed model significantly improves target language performance for languages that are distant from the source language.

AudioVSR: Enhancing Video Speech Recognition with Audio Data (2024.emnlp-main)

Copied to clipboard

Xiaoda Yang, Xize Cheng, Jiaqi Duan, Hongshun Qiu, Minjie Hong, Minghui Fang, Shengpeng Ji, Jialong Zuo, Zhiqing Hong, Zhimeng Zhang, Tao Jin

SParC: Cross-Domain Semantic Parsing in Context (P19-1)

Challenge:	Recent work has shown poor performance with non-Indo-European languages . previous work primarily utilizes video information to build VSR models .
Approach:	They propose a generative model for data inflation that integrates synthetic data with authentic visual data to enhance the VSR model.
Outcome:	The proposed model improves on the audio-visual alignment problem in audio-video tasks.

Copied to clipboard

Tao Yu, Rui Zhang, Michihiro Yasunaga, Yi Chern Tan, Xi Victoria Lin, Suyi Li, Heyang Er, Irene Li, Bo Pang, Tao Chen, Emily Ji, Shreya Dixit, David Proctor, Sungrok Shim, Jonathan Kraft, Vincent Zhang, Caiming Xiong, Richard Socher, Dragomir Radev

Programming Languages in NLP

Challenge:	Xu et al., 2017): a dataset for cross-domain semantic parsing in context with 4,298 question sequences.
Approach:	They present a dataset for cross-domainSemanticParsing inContext that consists of 4,298 coherent question sequences.
Outcome:	The proposed dataset demonstrates that it has greater semantic diversity and can be generalized to unseen domains due to its cross-domain nature and the unseened databases at test time.

The Role of Visual Modality in Multimodal Mathematical Reasoning: Challenges and Insights (2025.acl-long)

Copied to clipboard

Yufang Liu, Yao Du, Tao Ji, Jianing Wang, Yang Liu, Yuanbin Wu, Aimin Zhou, Mengdi Zhang, Xunliang Cai

LongHeads: Multi-Head Attention is Secretly a Long Context Processor (2024.findings-emnlp)

Challenge:	Existing models that leverage visual information do not improve math reasoning performance . authors suggest that visual information is important for multimodal reasoning .
Approach:	They propose a dataset to require image reliance for problem-solving and challenge models with similar, yet distinct, images that change the correct answer.
Outcome:	The proposed model performance is unaffected by changes to or removal of images in the dataset.

Copied to clipboard

Yi Lu, Xin Zhou, Wei He, Jun Zhao, Tao Ji, Tao Gui, Qi Zhang, Xuanjing Huang

Typology Guided Multilingual Position Representations: Case on Dependency Parsing (2023.findings-acl)

Challenge:	Large language models struggle to process lengthy inputs due to limited length generalization and attention’s quadratic computational demands.
Approach:	They propose a training-free framework that allows each head to attend to important context chunks instead of allowing each head a full sentence .
Outcome:	The proposed framework unlocks multi-head attention's untapped potential by allowing each head to attend to important context chunks instead of the full sentence.

Copied to clipboard

Tao Ji, Yuanbin Wu, Xiaoling Wang

Syntactic Text Processing

Challenge:	Recent multilingual models benefit from strong unified semantic representation models, but conflicting linguistic regularities may break the effectiveness of word position features in multilingual learning.
Approach:	They propose to combine prior knowledge from typology features and existing position vectors to create a position generation network which combines prior knowledge of a language's position space and typological characterization.
Outcome:	The proposed model can achieve the best multilingual parsing results by combining prior knowledge from typology features and existing position vectors.

T2A-Feedback: Improving Basic Capabilities of Text-to-Audio Generation via Fine-grained AI Feedback (2025.acl-long)

Copied to clipboard

Zehan Wang, Ke Lei, Chen Zhu, Jiawei Huang, Sashuai Zhou, Luping Liu, Xize Cheng, Shengpeng Ji, Zhenhui Ye, Tao Jin, Zhou Zhao

Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration (2024.naacl-long)

Challenge:	Text-to-audio (T2A) models still struggle to satisfy human preferences for prompt-following and acoustic quality when generating complex multi-event audio.
Approach:	They propose to use AI feedback learning to enhance basic capabilities of text-to-audio models . they use a large audio preference dataset to evaluate the model's capabilities .
Outcome:	The proposed model improves in simple and complex scenarios with AI feedback learning.

Copied to clipboard

Zhenhailong Wang, Shaoguang Mao, Wenshan Wu, Tao Ge, Furu Wei, Heng Ji

You Never Know a Person, You Only Know Their Defenses: Detecting Levels of Psychological Defense Mechanisms in Supportive Conversations (2026.findings-acl)

Challenge:	Existing work on LLMs that only enhance reasoning abilities, but which lack factual hallucination and slow-thinking capabilities, argues that SPP is a cognitive synergist.
Approach:	They propose a Solo Performance Prompting (SPP) that transforms a single LLM into a cognitive synergist by engaging in multi-turn self-collaboration with multiple personas.
Outcome:	The proposed model reduces factual hallucination and maintains strong reasoning abilities on three challenging tasks .

Copied to clipboard

Hongbin Na, Zimu Wang, Zhaoming Chen, Peilin Zhou, Yining Hua, Grace Ziqi Zhou, Haiyang Zhang, Tao Shen, Wei Wang, John Torous, Shaoxiong Ji, Ling Chen

Dialogue Systems & Conversational Agents

Challenge:	Psychological defenses are strategies people use to manage distress.
Approach:	They propose a dialogue corpus with help seeker utterances labeled for defense level and a DMRS Co-Pilot pipeline that provides evidence-based pre-annotations.
Outcome:	The proposed framework reduces annotation time by 24.0% in a counterbalanced study.

Graph-based Dependency Parsing with Graph Neural Networks (P19-1)

Copied to clipboard

Tao Ji, Yuanbin Wu, Man Lan

Zero-Shot Event Detection Based on Ordered Contrastive Learning and Prompt-Based Prediction (2022.findings-naacl)

Challenge:	In graph-based dependency parsers, learning representations is gaining in importance, and we use graph neural networks to learn the representations.
Approach:	They propose to use graph neural networks to learn dependency tree nodes and propose to add a new aggregation function to the system.
Outcome:	The proposed model achieves the best UAS and LAS on PTB (96.0%, 94.3%) without using external resources.

Copied to clipboard

Senhui Zhang, Tao Ji, Wendi Ji, Xiaoling Wang

Semantic Text Processing

Challenge:	Existing zero-shot event detection methods do not work for unseen types . supervised methods require predefined event types or external tools .
Approach:	They propose a framework to detect events from unstructured text without annotating samples . they propose to use ordered contrastive learning and prompt-based prediction to identify trigger words .
Outcome:	The proposed model detects events more effectively and accurately than state-of-the-art methods.

A Unified Encoding of Structures in Transition Systems (2021.emnlp-main)

Copied to clipboard

Tao Ji, Yong Jiang, Tao Wang, Zhongqiang Huang, Fei Huang, Yuanbin Wu, Xiaoling Wang

IPIGuard: A Novel Tool Dependency Graph-Based Defense Against Indirect Prompt Injection in LLM Agents (2025.emnlp-main)

Challenge:	Existing approaches to encode dynamic structures only encode partial information of structures.
Approach:	They propose a new attention-based encoder unifying all structures in a transition system.
Outcome:	The proposed method significantly improves the test speed and achieves the best transition-based model.

Copied to clipboard

Hengyu An, Jinghuai Zhang, Tianyu Du, Chunyi Zhou, Qingming Li, Tao Lin, Shouling Ji

Responsible & Trustworthy NLP

Challenge:	Existing methods for detecting Indirect Prompt Injection (IPI) attacks rely on assumptions about the model's inherent security, which lacks structural constraints on agent behaviors.
Approach:	They propose a novel task execution paradigm that models the agents’ task execution process as a traversal over a planned Tool Dependency Graph (TDG).
Outcome:	The proposed model reduces unintended tool invocations triggered by injected instructions, enhancing robustness against IPI attacks.

Investigating and Mitigating Object Hallucinations in Pretrained Vision-Language (CLIP) Models (2024.emnlp-main)

Copied to clipboard

Yufang Liu, Tao Ji, Changzhi Sun, Yuanbin Wu, Aimin Zhou

ToolEyes: Fine-Grained Evaluation for Tool Learning Capabilities of Large Language Models in Real-world Scenarios (2025.coling-main)

Challenge:	Existing studies have revealed that Large Vision-Language Models suffer from hallucinations in practice, including object hallucines, spatial hallucinos, attribute hallucinications, etc.
Approach:	They propose to use CLIP model to mitigate object hallucinations by using a data augmentation method to create negative samples with a variety of hallucinian issues.
Outcome:	The proposed method mitigates object hallucinations and can be used as a visual encoder, effectively alleviating the object halluination issue in LVLMs.

Copied to clipboard

Junjie Ye, Guanyu Li, SongYang Gao, Caishuang Huang, Yilong Wu, Sixian Li, Xiaoran Fan, Shihan Dou, Tao Ji, Qi Zhang, Tao Gui, Xuanjing Huang

DVAGen: Dynamic Vocabulary Augmented Generation (2025.emnlp-demos)

Challenge:	Existing evaluations of tool learning focus on validation of tools for large language models with expected outcomes, but this focus ignores the complex capabilities required for LLMs to effectively use tools.
Approach:	They propose a fine-grained system for evaluation of large language models’ tool learning capabilities in authentic scenarios.
Outcome:	The proposed system examines seven real-world scenarios, analyzing five dimensions crucial to LLMs in tool learning: format alignment, intent comprehension, behavior planning, tool selection, and answer organization.

Copied to clipboard

Wei Du, Nuowei Liu, Jie Wang, Jiahao Kuang, Tao Ji, Xiaoling Wang, Yuanbin Wu

Taming Text-to-Image Synthesis for Novices: User-centric Prompt Generation via Multi-turn Guidance (2025.emnlp-main)

Challenge:	Existing dynamic vocabulary approaches struggle to generalize to novel or out-of-vocabulary words, limiting their flexibility in handling diverse token combinations.
Approach:	They propose an open-source framework for training, evaluation, and visualization of dynamic vocabulary-augmented language models.
Outcome:	The proposed framework validates the effectiveness of dynamic vocabulary-augmented language models on modern LLMs and shows support for batch inference significantly improving inference throughput.

Copied to clipboard

Yilun Liu, Minggui He, Feiyu Yao, Yuhe Ji, Shimin Tao, Jingzhou Du, Justin Li, Jian Gao, Zhang Li, Hao Yang, Boxing Chen, Osamu Yoshie

Low-Resource NLP

Challenge:	Existing solutions for text-to-image synthesis are sensitive on textual prompts, posing a challenge for novice users.
Approach:	They propose a dialogue-based TIS prompt generation model that emphasizes user experience for novice users.
Outcome:	The proposed model emphasizes user experience for novice users . it improves user-centricity score while maintaining a competitive quality of synthesized images.

Parameter-Efficient Prompt Tuning Makes Generalized and Calibrated Neural Text Retrievers (2023.findings-emnlp)

Copied to clipboard

Weng Tam, Xiao Liu, Kaixuan Ji, Lilong Xue, Jiahua Liu, Tao Li, Yuxiao Dong, Jie Tang

Responsible & Trustworthy NLP

Challenge:	Prompt tuning is a technique that updates few parameters in pre-trained models for language understanding and generation tasks.
Approach:	They propose to leverage prompt tuning for neural text retrieval to improve generalization and cross-domain generalization.
Outcome:	The proposed approach can mitigate the two issues faced by fine-tuning retrieval methods and improve the out-of-domain zero-shot generalization of the retrieval models.

ABC-Bench: Benchmarking Agentic Backend Coding in Real-World Development (2026.findings-acl)

Copied to clipboard

Jie Yang, Honglin Guo, Li Ji, Jiazheng Zhou, Rui Zheng, Zhikai Lei, Shuo Zhang, Zhiheng Xi, Shichun Liu, Yuxin Wang, Bo Wang, Yining Zheng, Tao Gui, Xipeng Qiu

Fine-grained Coordinated Cross-lingual Text Stream Alignment for Endless Language Knowledge Acquisition (D18-1)

Challenge:	Large Language Models (LLMs) have redefined the role of AI in software engineering . current benchmarks focus on localized code generation, but neglect dynamic, full-process requirements of real-world engineering.
Approach:	They propose a benchmark to evaluate agentic backend coding within a realistic, executable workflow.
Outcome:	The ABC-Bench benchmark evaluates agentic backend coding within a realistic, executable workflow.

Copied to clipboard

Tao Ge, Qing Dou, Heng Ji, Lei Cui, Baobao Chang, Zhifang Sui, Furu Wei, Ming Zhou

Multilinguality

Challenge:	a novel information network decipherment paradigm is proposed for fine-grained coordinated cross-lingual text stream alignment.
Approach:	They propose to use Burst Information Networks as media to represent text streams . they propose a simple yet effective information network decipherment algorithm with diverse clues .
Outcome:	The proposed approach outperforms existing approaches on bilingual lexicon extraction from coordinated text streams and can harvest high-quality alignments from large amounts of streaming data for endless language knowledge mining.

StepCoder: Improving Code Generation with Reinforcement Learning from Compiler Feedback (2024.acl-long)

Copied to clipboard

Shihan Dou, Yan Liu, Haoxiang Jia, Enyu Zhou, Limao Xiong, Junjie Shan, Caishuang Huang, Xiao Wang, Xiaoran Fan, Zhiheng Xi, Yuhao Zhou, Tao Ji, Rui Zheng, Qi Zhang, Tao Gui, Xuanjing Huang

Programming Languages in NLP

Challenge:	Existing work integrates reinforcement learning with compiler feedback to enhance code generation quality but the long code generated by LLMs makes RL exploration ineffective.
Approach:	They propose a framework that integrates reinforcement learning and compiler feedback to enhance code generation quality.
Outcome:	The proposed framework outperforms state-of-the-art approaches in corresponding benchmarks and integrates reinforcement learning with compiler feedback to improve code generation quality.

Towards Economical Inference: Enabling DeepSeek’s Multi-Head Latent Attention in Any Transformer-based LLMs (2025.acl-long)

Copied to clipboard

Tao Ji, Bin Guo, Yuanbin Wu, Qipeng Guo, Shenlixing Shenlixing, Chenzhan Chenzhan, Xipeng Qiu, Qi Zhang, Tao Gui

MMAPS: End-to-End Multi-Grained Multi-Modal Attribute-Aware Product Summarization (2024.lrec-main)

Challenge:	Multi-head Latent Attention (MLA) is an innovative architecture designed to ensure efficient and economical inference by significantly compressing the Key-Value (KV) cache into a latent vector.
Approach:	They propose a data-efficient fine-tuning method for transitioning from MHA to MLA using a latent vector cache.
Outcome:	The proposed architecture reduces the KV cache size of Llama2-7B by 92.19%, with only 1% drop in LongBench performance.

Copied to clipboard

Tao Chen, Ze Lin, Hui Li, Jiayi Ji, Yiyi Zhou, Guanbin Li, Rongrong Ji

From Scores to Preferences: Redefining Evaluation Paradigm for Speech Quality Reward Modeling (2026.findings-acl)

Challenge:	Existing product summarization methods lack end-to-end product summaries and multi-grained multi-modal modeling.
Approach:	They propose an end-to-end multi-grained multi-modal attribute-aware product summarization method that jointly models product attributes and generates product summaries.
Outcome:	The proposed method outperforms state-of-the-art product summarization methods on a large-scale Chinese e-commence dataset.

Copied to clipboard

Yifei Cao, Changhao Jiang, Jiabao Zhuang, Jiajun Sun, Ming Zhang, Zhiheng Xi, Hui Li, Shihan Dou, Yuran Wang, Yunke Zhang, Tao Ji, Tao Gui, Qi Zhang, Xuanjing Huang

Mitigating Object Hallucinations in MLLMs via Multi-Frequency Perturbations (2025.findings-emnlp)

Challenge:	Experimental results show that the MOS-aware GRM significantly improves fine-grained speech quality discrimination.
Approach:	They propose a MOS-aware reward model that incorporates MOS gap into reward function during reinforcement learning.
Outcome:	The proposed model significantly improves fine-grained speech quality discrimination.

Copied to clipboard

Shuo Li, Jiajun Sun, Guodong Zheng, Xiaoran Fan, Yujiong Shen, Yi Lu, Zhiheng Xi, Yuming Yang, Wenming Tan, Tao Ji, Tao Gui, Qi Zhang, Xuanjing Huang

ED2LM: Encoder-Decoder to Language Model for Faster Document Re-ranking Inference (2022.findings-acl)

Challenge:	Multimodal large language models have demonstrated remarkable performance in visual-language tasks, but their authenticity is often compromised by object hallucinations.
Approach:	They propose a multi-frequency perturbation method that leverages both low-frequency and high-frequency features of images to perturb visual feature representations and explicitly suppress redundant frequency-domain features during inference.
Outcome:	The proposed method significantly mitigates object hallucinations across various model architectures.

Copied to clipboard

Kai Hui, Honglei Zhuang, Tao Chen, Zhen Qin, Jing Lu, Dara Bahri, Ji Ma, Jai Gupta, Cicero Nogueira dos Santos, Yi Tay, Donald Metzler

Generation with Dynamic Vocabulary (2024.emnlp-main)

Challenge:	State-of-the-art neural models typically encode document-query pairs using cross-attention for re-ranking.
Approach:	They propose to fine tune a pretrained encoder-decoder model using document to query generation.
Outcome:	The proposed model achieves comparable results to more expensive approaches while being 6.8X faster.

Copied to clipboard

Yanting Liu, Tao Ji, Changzhi Sun, Yuanbin Wu, Xiaoling Wang