Papers by Lu Ji

HYRR: Hybrid Infused Reranking for Passage Retrieval (2024.lrec-main)

Challenge:	Existing benchmarks evaluate agents in simplified, idealized settings, relying on pre-packaged tool interfaces, overlooking critical steps, and assume inputs are clean and fully specified.
Approach:	They propose a framework that evaluates language agents in simplified, idealized settings . they show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 .
Outcome:	Experiments on 15 proprietary and open-source models show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 .

Copied to clipboard

Jing Lu, Keith Hall, Ji Ma, Jianmo Ni

Information Retrieval

Challenge:	Existing passage retrieval systems typically adopt a two-stage retrieve-then-rerank pipeline.
Approach:	They propose a framework for training robust reranking models using hybrid retrievers . they propose HYRR framework that allows users to select training data using hybrids .
Outcome:	The proposed framework is robust to different first-stage retrieval settings.

Few-Shot Multimodal Named Entity Recognition Based on Mutlimodal Causal Intervention Graph (2024.lrec-main)

Copied to clipboard

Feihong Lu, Xiaocui Yang, Qian Li, Qingyun Sun, Ke Jiang, Cheng Ji, Jianxin Li

Platforms for Non-speakers Annotating Names in Any Language (P18-4)

Challenge:	Existing methods for multimodal named entity recognition are limited due to limited resources.
Approach:	They propose a Few-shot Multimodal Named Entity Recognition task to address these relation types by constructing a multimodal graph and a new multimodal causal intervention strategy.
Outcome:	The proposed model improves on two multimodal named entity recognition datasets.

Copied to clipboard

Ying Lin, Cash Costello, Boliang Zhang, Di Lu, Heng Ji, James Mayfield, Paul McNamee

Challenge:	Traditionally, native speakers of a language have been asked to annotate a corpus in that language.
Approach:	They propose two annotation platforms that allow an English speaker to annotate names for any language without knowing the language.
Outcome:	The proposed annotations achieved state-of-the-art performance on two surprise languages and ten languages at TAC-KBP EDL2017.

Entity-aware Image Caption Generation (D18-1)

Copied to clipboard

Di Lu, Spencer Whitehead, Lifu Huang, Heng Ji, Shih-Fu Chang

Efficient Attentions for Long Document Summarization (2021.naacl-main)

Challenge:	Existing image captioning approaches generate generic descriptions of visual content and ignore background information.
Approach:	They propose a task which generates informative image captions using images and hashtags as input.
Outcome:	The proposed model outperforms unimodal baselines significantly with evaluation metrics on a dataset from Flickr.

Copied to clipboard

Luyang Huang, Shuyang Cao, Nikolaus Parulian, Heng Ji, Lu Wang

Breaking the Noise Barrier: LLM-Guided Semantic Filtering and Enhancement for Multi-Modal Entity Alignment (2025.emnlp-main)

Challenge:	Existing models that use full attentions have quadratic computational and memory complexities, and are too costly for long documents.
Approach:	They propose an efficient encoder-decoder attention with head-wise positional strides to effectively pinpoint salient information from the source.
Outcome:	The proposed model can process ten times more tokens than current models that use full attentions.

Copied to clipboard

Chenglong Lu, Chenxiao Li, Jingwei Cheng, Yongquan Ji, Guoqing Chen, Fu Zhang

Memorizing is Not Enough: Deep Knowledge Injection Through Reasoning (2025.acl-long)

Challenge:	Multi-modal entity alignment (MMEA) aims to identify equivalent entities between two multimodal knowledge graphs.
Approach:	They propose a novel LLMguided MMEA framework that prioritizes noise reduction before fusion.
Outcome:	The proposed framework prioritizes noise reduction before fusion and improves semantics on the noisy FB YG dataset.

Copied to clipboard

Ruoxi Xu, Yunjie Ji, Boxi Cao, Yaojie Lu, Hongyu Lin, Xianpei Han, Ben He, Yingfei Sun, Xiangang Li, Le Sun

ELISA-EDL: A Cross-lingual Entity Extraction, Linking and Localization System (N18-5)

Challenge:	Existing knowledge injection frameworks focus on knowledge memorization and retrieval, but static nature of large language models leads to outdated information as the real world evolves or when adapting to domain-specific knowledge.
Approach:	They propose a four-tier knowledge injection framework that defines the levels of knowledge injection: memorization, retrieval, reasoning, and association.
Outcome:	The proposed framework defines the levels of knowledge injection: memorization, retrieval, reasoning, and association.

Copied to clipboard

Boliang Zhang, Ying Lin, Xiaoman Pan, Di Lu, Jonathan May, Kevin Knight, Heng Ji

Information Extraction & Text Mining

Challenge:	ELISA-EDL is a cross-lingual entity extraction, linking and localization system for Wikipedia languages.
Approach:	They propose a cross-lingual entity extraction, linking and localization system for English speakers . it extracts entities from unstructured text in any of 282 Wikipedia languages and links them to English knowledge bases .
Outcome:	The proposed system extracts entity mentions from Wikipedia and links them to English knowledge bases and visualizes locations related to disaster topics on a world heatmap.

Critic-CoT: Boosting the Reasoning Abilities of Large Language Model via Chain-of-Thought Critic (2025.findings-acl)

Copied to clipboard

Xin Zheng, Jie Lou, Boxi Cao, Xueru Wen, Yuqiu Ji, Hongyu Lin, Yaojie Lu, Xianpei Han, Debing Zhang, Le Sun

Amphista: Bi-directional Multi-head Decoding for Accelerating LLM Inference (2025.naacl-long)

Challenge:	Existing approaches to improve the reasoning performance of large language models rely on intuitive instance-level feedback, which limits the reasoning capabilities.
Approach:	They propose a framework that pushes LLMs toward System-2-like critic capability by using a step-wise CoT reasoning paradigm and automatic construction of weak-supervision data without human annotation.
Outcome:	The proposed model significantly improves task-solving performance by filtering out invalid solutions or iterative refinement.

Copied to clipboard

Zeping Li, Xinlong Yang, Ziheng Gao, Ji Liu, Guanchen Li, Zhuang Liu, Dong Li, Jinzhang Peng, Lu Tian, Emad Barsoum

Machine Translation

Challenge:	Existing methods such as Medusa lack adequate information interaction between different drafting heads.
Approach:	They propose an enhanced speculative decoding framework that builds upon Medusa and integrates a drafting block capable of parallel inference.
Outcome:	The proposed framework outperforms Medusa in terms of head accuracy and latency.

Cross-lingual Structure Transfer for Zero-resource Event Extraction (2020.lrec-1)

Copied to clipboard

Di Lu, Ananya Subburathinam, Heng Ji, Jonathan May, Shih-Fu Chang, Avi Sil, Clare Voss

Information Extraction & Text Mining

Challenge:	Existing approaches for information extraction only use name tagging . Currently, most successful cross-lingual transfer learning methods are limited to sequence labeling .
Approach:	They propose a share-and-transfer framework to transfer graph structures across languages . they propose to convert sentences in any language to language-universal graph structures .
Outcome:	The proposed framework performs comparable to state-of-the-art models on three languages without annotations.

On-Policy Self-Alignment with Fine-grained Knowledge Feedback for Hallucination Mitigation (2025.findings-acl)

Copied to clipboard

Xueru Wen, Jie Lou, Xinyu Lu, Yuqiu Ji, Xinyan Guan, Yaojie Lu, Hongyu Lin, Ben He, Xianpei Han, Debing Zhang, Le Sun

LongHeads: Multi-Head Attention is Secretly a Long Context Processor (2024.findings-emnlp)

Challenge:	Large language models exhibit behavior that deviates from the boundaries of their knowledge during response generation.
Approach:	They propose a framework that allows large language models to explore their knowledge boundaries and self-correct generation behavior through fine-grained feedback signals.
Outcome:	The proposed framework enables LLMs to explore their knowledge boundaries and self-correct generation behavior through fine-grained feedback signals.

Copied to clipboard

Yi Lu, Xin Zhou, Wei He, Jun Zhao, Tao Ji, Tao Gui, Qi Zhang, Xuanjing Huang

Green & Sustainable NLP

Challenge:	Large language models struggle to process lengthy inputs due to limited length generalization and attention’s quadratic computational demands.
Approach:	They propose a training-free framework that allows each head to attend to important context chunks instead of allowing each head a full sentence .
Outcome:	The proposed framework unlocks multi-head attention's untapped potential by allowing each head to attend to important context chunks instead of the full sentence.

ModRWKV: Transformer Multimodality in Linear Time (2025.emnlp-main)

Copied to clipboard

Jiale Kang, Ziyin Yue, Qingyu Yin, Rui Jiang, Weile Li, Zening Lu, Zhouran Ji

Semantic Text Processing

Challenge:	Currently, multimodal studies are based on large language models with quadratic-complexity Transformer architectures.
Approach:	They propose a decoupled multimodal framework built upon the RWKV7 architecture as its LLM backbone and a lightweight architecture to achieve multi-source information fusion.
Outcome:	The proposed framework achieves multi-source information fusion through dynamically adaptable heterogeneous modality encoders.

RSVP: Reasoning Segmentation via Visual Prompting and Multi-modal Chain-of-Thought (2025.acl-long)

Copied to clipboard

Yi Lu, Jiawang Cao, Yongliang Wu, Bozheng Li, Licheng Tang, Yangguang Ji, Chong Wu, Jay Wu, Wenbo Zhu

Discrete Argument Representation Learning for Interactive Argument Pair Identification (2021.naacl-main)

Challenge:	Recent advances in multi-modal learning have enhanced MLLMs' ability to reason about visual content.
Approach:	They propose a framework that unifies multi-step multimodal reasoning with grounded visual understanding.
Outcome:	The proposed framework surpasses state-of-the-art methods by +6.5 gIoU and +9.2 cIou on ReasonSeg and achieves 49.7 mAP on SegInW under zero-shot settings.

Copied to clipboard

Lu Ji, Zhongyu Wei, Jing Li, Qi Zhang, Xuanjing Huang

Representation Learning

Challenge:	Existing research on monological argumentation covers claims generation, argument structure prediction, and essay scoring.
Approach:	They propose to identify argument pairs from two posts with opposite stances to a certain topic.
Outcome:	The proposed framework outperforms competing models on a large-scale dataset . it also proves that it is useful for analyzing argument pairs from two posts .

Error Analysis of Uyghur Name Tagging: Language-specific Techniques and Remaining Challenges (L18-1)

Copied to clipboard

Halidanmu Abudukelimu, Abudoukelimu Abulizi, Boliang Zhang, Xiaoman Pan, Di Lu, Heng Ji, Yang Liu

Syntactic Text Processing

Challenge:	despite efforts at name tagging, there is limited understanding on the performance ceiling . despite the high-resource language, there are very few natural language processing tools available .
Approach:	They propose to use a machine learning model to identify Uyghur name tagger errors . they conclude that such a model is unlikely to be effective for Uygur, or low-resource languages .
Outcome:	The proposed model is unlikely to be effective for Uyghur, or low-resource languages in general, the authors argue . they show that the proposed model can be used for high-res languages with superficial features .

SAM3-I: Segment Anything with Instructions (2026.acl-long)

Copied to clipboard

Jingjing Li, Yue Feng, Yuchen Guo, Jincai Huang, Wei Ji, Qi Bi, Yongri Piao, Miao Zhang, Xiaoqi Zhao, Qiang Chen, Shihao Zou, Huchuan Lu, Li Cheng

Low-Resource NLP

Challenge:	Existing methods for concept-level grounding and instruction-level reasoning use coarse representations and iterative mask filtering.
Approach:	They propose an instruction-following extension of the Segment Anything Model 3 family that unifies concept-level grounding and instruction-level reasoning within a single segmentation framework.
Outcome:	Experiments show that SAM3-I achieves appealing performance across referring and reasoning-based segmentation while maintaining its strong concept recall ability.

Large Dual Encoders Are Generalizable Retrievers (2022.emnlp-main)

Copied to clipboard

Jianmo Ni, Chen Qu, Jing Lu, Zhuyun Dai, Gustavo Hernandez Abrego, Ji Ma, Vincent Zhao, Yi Luan, Keith Hall, Ming-Wei Chang, Yinfei Yang

Semantic Text Processing

Challenge:	Experimental results show that dual encoders outperform sparse and dense retrievers on the BEIR dataset significantly.
Approach:	They challenge belief that bottleneck layer is too limited for out-of-domain generalization . they scale up the model while keeping bottleneck as a single dot-product with a fixed size .
Outcome:	The proposed model outperforms sparse and dense retrievers on the BEIR dataset significantly.

Don’t Go To Extremes: Revealing the Excessive Sensitivity and Calibration Limitations of LLMs in Implicit Hate Speech Detection (2024.acl-long)

Copied to clipboard

Min Zhang, Jianfeng He, Taoran Ji, Chang-Tien Lu

VoxMind: An End-to-End Agentic Spoken Dialogue System (2026.acl-long)

Challenge:	Several studies have examined whether large language models exhibit bias or discrimination against individuals or groups in terms of protected attributes like race, gender, or religion.
Approach:	They evaluate LLMs' ability to detect implicit hate speech and express confidence in their responses by considering prompt patterns and mainstream uncertainty estimation methods.
Outcome:	The proposed models exhibit two extremes: (1) excessive sensitivity towards groups or topics that may cause fairness issues, resulting in misclassifying benign statements as hate speech; (2) confidence scores for each method excessively concentrate on a fixed range, remaining unchanged regardless of the dataset’s complexity.

Copied to clipboard

Tianle Liang, Yifu Chen, Shengpeng Ji, Yijun Chen, Zhiyang Jia, Jingyu Lu, Fan Zhuo, Xueyi Pu, Yangzhuo Li, Zhou Zhao

Natural Language Interfaces

Challenge:	Existing research on end-to-end spoken dialogue models has focused on core perception and generation, with limited exploration of tool-augmented extensions.
Approach:	They propose a framework to equip end-to-end spoken dialogue models with comprehensive agentic abilities by leveraging a 470-hour AgentChat dataset.
Outcome:	The proposed framework outperforms Gemini-2.5-Pro on spoken agent tasks while maintaining general conversational quality.

Incorporating Argument-Level Interactions for Persuasion Comments Evaluation using Co-attention Model (C18-1)

Copied to clipboard

Lu Ji, Zhongyu Wei, Xiangkun Hu, Yang Liu, Qi Zhang, Xuanjing Huang

WebSRC: A Dataset for Web-Based Structural Reading Comprehension (2021.emnlp-main)

Challenge:	Existing research explores different text features of reply comments on word level and ignores interactions between participants.
Approach:	They propose a co-attention mechanism based neural network to capture interactions between participants on argument level to better model dialogical argumentation.
Outcome:	The proposed model outperforms state-of-the-art methods on a publicly available dataset showing that it extracts interactive argument pairs from the original post and the reply.

Copied to clipboard

Xingyu Chen, Zihan Zhao, Lu Chen, JiaBao Ji, Danyang Zhang, Ao Luo, Yuxuan Xiong, Kai Yu

Visual Attention Model for Name Tagging in Multimodal Social Media (P18-1)

Challenge:	Using a web page and a question, a machine can't understand the contents of web pages.
Approach:	They propose a novel dataset for web-based structural reading comprehension that consists of 400K question-answer pairs and a dataset of 6.4K web pages.
Outcome:	The proposed dataset consists of 400K question-answer pairs, collected from 6.4K web pages with corresponding HTML source code, screenshots, and metadata.

Copied to clipboard

Di Lu, Leonardo Neves, Vitor Carvalho, Ning Zhang, Heng Ji

Cheems: A Practical Guidance for Building and Evaluating Chinese Reward Models from Scratch (2025.acl-long)

Challenge:	Name tagging is a key task for language understanding, but is often limited by the short textual components.
Approach:	They propose a novel model architecture based on visual attention that outperforms other methods . they use multimodal datasets to analyze the name tagging task on social media .
Outcome:	The proposed model outperforms existing methods and significantly outperformed existing methods.

Copied to clipboard

Xueru Wen, Jie Lou, Zichao Li, Yaojie Lu, XingYu XingYu, Yuqiu Ji, Guohai Xu, Hongyu Lin, Ben He, Xianpei Han, Le Sun, Debing Zhang

Challenge:	Existing Chinese resources are small in scale and limited to specific domains, making them insufficient for LLM post-training.
Approach:	They propose a Chinese-annotated reward model and a preference dataset to address this gap . they evaluate Chinese RMs on CheemsBench and construct an RM that captures human preferences .
Outcome:	The proposed RM achieves state-of-the-art performance on CheemsBench and CheeMePreference.

ReFLAIR: Enhancing Multimodal Reasoning via Structured Reflection and Reward-Guided Learning (2025.findings-emnlp)

Copied to clipboard

Jiazhou Ji, Xinru Lu

WavAlign: Enhancing Intelligence and Expressiveness in Spoken Dialogue Models via Adaptive Hybrid Post-Training (2026.findings-acl)

Challenge:	Existing training methods for large models do not address the trade-off between reflection and accuracy.
Approach:	a unified framework teaches large models to perform structured reflection via an explicit $think re-think answer $ format and hybrid reward learning.
Outcome:	The proposed framework improves model performance on mathematical benchmarks and reduces inference cost by nearly 23%.

Copied to clipboard

Yifu Chen, Shengpeng Ji, Qian Chen, Tianle Liang, Yangzhuo Li, Ziqing Wang, Wen Wang, Jingyu Lu, Haoxiao Wang, Xueyi Pu, Fan Zhuo, Zhou Zhao

Dialogue Systems & Conversational Agents

Challenge:	End-to-end spoken dialogue models have higher potential ceiling in expressiveness and perceptual ability than cascaded systems.
Approach:	They propose a modality-aware adaptive post-training recipe that constrains preference updates to the semantic channel and improves acoustic behavior via explicit anchoring.
Outcome:	The proposed model improves speech quality and expressiveness across spoken dialogue benchmarks and architectures.

Mitigating Object Hallucinations in MLLMs via Multi-Frequency Perturbations (2025.findings-emnlp)

Copied to clipboard

Shuo Li, Jiajun Sun, Guodong Zheng, Xiaoran Fan, Yujiong Shen, Yi Lu, Zhiheng Xi, Yuming Yang, Wenming Tan, Tao Ji, Tao Gui, Qi Zhang, Xuanjing Huang

Capturing Latent Modal Association For Multimodal Entity Alignment (2025.findings-emnlp)

Challenge:	Multimodal large language models have demonstrated remarkable performance in visual-language tasks, but their authenticity is often compromised by object hallucinations.
Approach:	They propose a multi-frequency perturbation method that leverages both low-frequency and high-frequency features of images to perturb visual feature representations and explicitly suppress redundant frequency-domain features during inference.
Outcome:	The proposed method significantly mitigates object hallucinations across various model architectures.

Copied to clipboard

Yongquan Ji, Jingwei Cheng, Fu Zhang, Chenglong Lu

Representation Learning

Challenge:	Existing methods for multimodal entity alignment overlook the quality of input modality embeddings during modality interaction, amplifying noise propagation while suppressing discriminative feature representations.
Approach:	They propose a model for capturing latent modal association for multimodal entity alignment using a self-attention mechanism to enhance salient information while attenuating noise within individual modality embeddings.
Outcome:	The proposed model achieves an absolute 3.1% higher Hits@1 score than the sota method.

ARGUS: Policy-Adaptive Ad Governance via Evolving Reinforcement with Adversarial Umpiring (2026.acl-industry)

Copied to clipboard

Deyi Ji, Junyu Lu, Xuanyi Liu, Liqun Liu, Hailong Zhang, Peng Shu, Huan Yu, Jie Jiang, Tianrun Chen, Lanyun Zhu

ED2LM: Encoder-Decoder to Language Model for Faster Document Re-ranking Inference (2022.findings-acl)

Challenge:	Existing regulatory policies create label inconsistencies and reasoning ambiguities in historical datasets.
Approach:	They propose a policy-adaptive governance system that enables evolving reinforcement through multi-agent adversarial umpiring.
Outcome:	The proposed system outperforms fine-tuning baselines on industrial and public datasets . it enables evolving reinforcement through multi-agent adversarial umpiring .

Copied to clipboard

Kai Hui, Honglei Zhuang, Tao Chen, Zhen Qin, Jing Lu, Dara Bahri, Ji Ma, Jai Gupta, Cicero Nogueira dos Santos, Yi Tay, Donald Metzler

ReviewGrounder: Improving Review Substantiveness with Rubric-Guided, Tool-Integrated Agents (2026.acl-long)

Challenge:	State-of-the-art neural models typically encode document-query pairs using cross-attention for re-ranking.
Approach:	They propose to fine tune a pretrained encoder-decoder model using document to query generation.
Outcome:	The proposed model achieves comparable results to more expensive approaches while being 6.8X faster.

Copied to clipboard

Zhuofeng Li, Yi Lu, Dongfu Jiang, Haoxiang Zhang, Yuyang Bai, Chuan Li, Yu Wang, Shuiwang Ji, Jianwen Xie, Yu Zhang