Papers by Yi Ji

DeepMaven: Deep Question Answering on Long-Distance Movie/TV Show Videos with Multimedia Knowledge Extraction and Synthesis (2023.eacl-main)

Challenge:	Current claims detection methods focus on sentence analysis, ignoring other attributes . a key element of identifying misinformation is detecting the claims and the arguments that have been presented.
Approach:	They propose a benchmark for attribute-aware claim detection in the news domain . they extend the problem to include extraction of additional attributes related to each claim .
Outcome:	The proposed system performs well on the test, but human performance is still poor.

Copied to clipboard

Yi Fung, Han Wang, Tong Wang, Ali Kebarighotbi, Mohit Bansal, Heng Ji, Prem Natarajan

MIMIR: A Customizable Agent Tuning Platform for Enhanced Scientific Applications (2024.emnlp-demo)

Challenge:	Long video content understanding poses a challenging set of research questions as it involves long-distance, cross-media reasoning and knowledge awareness.
Approach:	They propose a framework which extracts events, entities, and relations from the rich multimedia content in long videos to pre-construct movie knowledge graphs.
Outcome:	The proposed framework performs competitively for both the new DeepMovieQA and the pre-existing MovieQA dataset.

Copied to clipboard

Xiangru Tang, Chunyuan Deng, Hanminwang Hanminwang, Haoran Wang, Yilun Zhao, Wenqi Shi, Yi Fung, Wangchunshu Zhou, Jiannan Cao, Heng Ji, Arman Cohan, Mark Gerstein

RESIN: A Dockerized Schema-Guided Cross-document Cross-lingual Cross-media Information Extraction and Event Tracking System (2021.naacl-demos)

Challenge:	Large language models (LLMs) have evolved into interactive agents capable of planning, tool use, and task execution across various tasks.
Approach:	They propose a platform that leverages large language models to generate agent-tuning data for fine-tuneing smaller, specialized models.
Outcome:	MIMIR enables large models to simulate various roles and create interaction data, which can then be used to fine-tune open-source models like LLaMA2.

Copied to clipboard

Haoyang Wen, Ying Lin, Tuan Lai, Xiaoman Pan, Sha Li, Xudong Lin, Ben Zhou, Manling Li, Haoyu Wang, Hongming Zhang, Xiaodong Yu, Alexander Dong, Zhenhailong Wang, Yi Fung, Piyush Mishra, Qing Lyu, Dídac Surís, Brian Chen, Susan Windisch Brown, Martha Palmer, Chris Callison-Burch, Carl Vondrick, Jiawei Han, Dan Roth, Shih-Fu Chang, Heng Ji

Visual Data in NLP

Challenge:	We present a new information extraction system that can construct temporal event graphs from news documents.
Approach:	They propose a temporal event graph extraction system that can extract news documents . they extend the system from sentence-level event extraction to cross-document cross-media event extraction .
Outcome:	The proposed system can extract temporal event graphs from news documents in multiple languages and multiple data modalities.

LongHeads: Multi-Head Attention is Secretly a Long Context Processor (2024.findings-emnlp)

Copied to clipboard

Yi Lu, Xin Zhou, Wei He, Jun Zhao, Tao Ji, Tao Gui, Qi Zhang, Xuanjing Huang

Green & Sustainable NLP

Challenge:	Large language models struggle to process lengthy inputs due to limited length generalization and attention’s quadratic computational demands.
Approach:	They propose a training-free framework that allows each head to attend to important context chunks instead of allowing each head a full sentence .
Outcome:	The proposed framework unlocks multi-head attention's untapped potential by allowing each head to attend to important context chunks instead of the full sentence.

Schema-Guided Culture-Aware Complex Event Simulation with Multi-Agent Role-Play (2024.emnlp-demo)

Copied to clipboard

Sha Li, Revanth Gangi Reddy, Khanh Nguyen, Qingyun Wang, Yi Fung, Chi Han, Jiawei Han, Kartik Natarajan, Clare Voss, Heng Ji

Challenge:	Complex news events require swift responses from government and society, authors say . relying on historical events to project the future is insufficient, they say - a simulator for complex news events is needed .
Approach:	They propose a controllable complex news event simulator guided by event schema and user-provided assumptions . they incorporate a geo-diverse commonsense and cultural norm-aware knowledge enhancement component .
Outcome:	The proposed simulator achieves higher coherence and appropriateness than existing models.

Demonstration Augmentation for Zero-shot In-context Learning (2024.findings-acl)

Copied to clipboard

Yi Su, Yunpeng Tai, Yixin Ji, Juntao Li, Yan Bowen, Min Zhang

Low-Resource NLP

Challenge:	Large Language Models (LLMs) have demonstrated an impressive capability known as In-context Learning (ICL), which enables them to acquire knowledge from textual demonstrations without the need for parameter updates.
Approach:	They propose to use model’s previously predicted historical samples as demonstrations for subsequent ones to improve model’ s performance.
Outcome:	The proposed method significantly outperforms the previous method and its predecessors in terms of inference cost and time.

Cross-document Misinformation Detection based on Event Graph Reasoning (2022.naacl-main)

Copied to clipboard

Xueqing Wu, Kung-Hsiang Huang, Yi Fung, Heng Ji

Reasoning

Challenge:	Existing methods for misinformation detection are limited to judging each document in isolation.
Approach:	They propose a task of cross-document misinformation detection that detects fake news from a cluster of topically related news documents.
Outcome:	The proposed method outperforms existing methods by up to 7 F1 points on this task.

RSVP: Reasoning Segmentation via Visual Prompting and Multi-modal Chain-of-Thought (2025.acl-long)

Copied to clipboard

Yi Lu, Jiawang Cao, Yongliang Wu, Bozheng Li, Licheng Tang, Yangguang Ji, Chong Wu, Jay Wu, Wenbo Zhu

MACAROON: Training Vision-Language Models To Be Your Engaged Partners (2024.findings-emnlp)

Challenge:	Recent advances in multi-modal learning have enhanced MLLMs' ability to reason about visual content.
Approach:	They propose a framework that unifies multi-step multimodal reasoning with grounded visual understanding.
Outcome:	The proposed framework surpasses state-of-the-art methods by +6.5 gIoU and +9.2 cIou on ReasonSeg and achieves 49.7 mAP on SegInW under zero-shot settings.

Copied to clipboard

Shujin Wu, Yi Fung, Sha Li, Yixin Wan, Kai-Wei Chang, Heng Ji

COVID-19 Literature Knowledge Graph Construction and Drug Repurposing Report Generation (2021.naacl-demos)

Challenge:	Large vision-language models (LVLMs) generate detailed responses even when questions are ambiguous or unanswerable, leading to hallucinations and bias issues.
Approach:	They propose a three-tiered hierarchy for questions of invalid, ambiguous, and personalizable nature to measure the proactive engagement capabilities of LVLMs.
Outcome:	The proposed model generates contrastive response pairs for unlabeled questions, achieving 0.84 AAR, while maintaining comparable performance on general tasks.

Copied to clipboard

Qingyun Wang, Manling Li, Xuan Wang, Nikolaus Parulian, Guangxing Han, Jiawei Ma, Jingxuan Tu, Ying Lin, Ranran Haoran Zhang, Weili Liu, Aabhas Chauhan, Yingjun Guan, Bangzheng Li, Ruisong Li, Xiangchen Song, Yi Fung, Heng Ji, Jiawei Han, Shih-Fu Chang, James Pustejovsky, Jasmine Rah, David Liem, Ahmed ELsayed, Martha Palmer, Clare Voss, Cynthia Schneider, Boyan Onyshkevych

Structured Data in NLP

Challenge:	a new framework to digest relevant biomedical knowledge is needed to combat COVID-19 . quantity of research results is a bottleneck, and false information promoted in publications .
Approach:	a team of researchers has developed a framework to extract multimedia knowledge elements from scientific literature to combat COVID-19.
Outcome:	a new framework extracts fine-grained multimedia knowledge elements from scientific literature . it provides detailed contextual sentences, subfigures, and knowledge subgraphs as evidence . the framework is based on a case study of drug repurposing .

Beware of Model Collapse! Fast and Stable Test-time Adaptation for Robust Question Answering (2023.emnlp-main)

Copied to clipboard

Yi Su, Yixin Ji, Juntao Li, Hai Ye, Min Zhang

Robustness in NLP

Challenge:	Pre-trained language models (PLMs) have achieved great success in question answering, but their robustness is insufficient to support their practical applications.
Approach:	They propose a method which regularizes the model's output and an efficient side block to reduce its inference time.
Outcome:	The proposed method achieves comparable or better results than previous TTA methods at a speed close to vanilla forward propagation, which is 1.8 to 4.4 speedup compared to previous methods.

Non-Autoregressive Sentence Ordering (2023.findings-emnlp)

Copied to clipboard

Yi Bin, Wenhao Shi, Bin Ji, Jipeng Zhang, Yujuan Ding, Yang Yang

Reasoning

Challenge:	Existing sentence ordering approaches only leverage unilateral dependencies during decoding and cannot fully explore the semantic dependency between sentences.
Approach:	They propose a non-autoregressive ordering network that explores bilateral dependencies between sentences and predicts sentences for each position in parallel.
Outcome:	The proposed model outperforms existing autoregressive sentence ordering approaches and yields competitive performance compared with the state-of-the-arts.

R-Tuning: Instructing Large Language Models to Say ‘I Don’t Know’ (2024.naacl-long)

Copied to clipboard

Hanning Zhang, Shizhe Diao, Yong Lin, Yi Fung, Qing Lian, Xingyao Wang, Yangyi Chen, Heng Ji, Tong Zhang

PUPPET: Neural-Symbolic Standardized Patients for Mental Health (2026.acl-long)

Challenge:	Existing methods for instruction tuning force the model to complete a sentence no matter whether it knows the knowledge or not.
Approach:	They propose a new approach to tuning large language models to refrain from answering questions beyond its parametric knowledge by identifying the disparity in parametric and parametric information.
Outcome:	The proposed approach improves a model’s ability to answer known questions and refrain from answering unknown questions.

Copied to clipboard

Chen Xu, Yu ji, Zhenyu Lv, Yang Yi, Yizhe Yang, Luyao Ji, Chaoyi Chen, Xianyang Wang, Tian Lan, Zhihua Wang, Juan Wang, Xunde Dong, Fuze Tian, Qunxi Dong, Bin Hu

InfoSurgeon: Cross-Media Fine-grained Information Consistency Checking for Fake News Detection (2021.acl-long)

Challenge:	Existing LLM-based training approaches lack faithful responses to clinical errors and explainable feedback.
Approach:	They propose a neural-symbolic virtual standardized patient governed by an OBSERVE-THINK-BEHAVE architecture that embeds LLM reasoning into a symbolic system where experts implant causal associations between intervention logic and patient mental states.
Outcome:	The proposed model outperforms baselines in faithfulness and pedagogical value.

Copied to clipboard

Yi Fung, Christopher Thomas, Revanth Gangi Reddy, Sandeep Polisetty, Heng Ji, Shih-Fu Chang, Kathleen McKeown, Mohit Bansal, Avi Sil

PropaInsight: Toward Deeper Understanding of Propaganda in Terms of Techniques, Appeals, and Intent (2025.coling-main)

Challenge:	a novel approach to detect fake news is needed due to training data scarcity . current methods focus on document-level fake news detection using lexical features and semantic embeddings .
Approach:	They propose a novel benchmark for fake news detection at the knowledge element level . they propose synthesis method which manipulates knowledge elements to generate noisy training data .
Outcome:	The proposed method outperforms the state-of-the-art in detecting misinformation . it yields fine-grained explanations and outperformed the current methods .

Copied to clipboard

Jiateng Liu, Lin Ai, Zizhou Liu, Payam Karisani, Zheng Hui, Yi Fung, Preslav Nakov, Julia Hirschberg, Heng Ji

Sentiment Analysis

Challenge:	Existing research on propaganda detection does not capture the motives behind the content or its broader impact.
Approach:	They propose a framework that dissects propaganda into techniques, arousal appeals, and underlying intent.
Outcome:	The proposed framework improves performance in a wide range of scenarios and can be used to identify and categorize propaganda techniques.

Defining a New NLP Playground (2023.findings-emnlp)

Copied to clipboard

Sha Li, Chi Han, Pengfei Yu, Carl Edwards, Manling Li, Xingyao Wang, Yi Fung, Charles Yu, Joel Tetreault, Eduard Hovy, Heng Ji

Semantic Text Processing

Challenge:	Recent explosion of performance of large language models (LLMs) has changed the field more abruptly and seismically than any other shift in the field’s 80 year history.
Approach:	They propose 20+ PhD-dissertation-worthy research directions to define a new NLP playground by combining theoretical analysis, new and challenging problems, learning paradigms and interdisciplinary applications.
Outcome:	The proposed research will cover theoretical analysis, new and challenging problems, learning paradigms and interdisciplinary applications.

Large Dual Encoders Are Generalizable Retrievers (2022.emnlp-main)

Copied to clipboard

Jianmo Ni, Chen Qu, Jing Lu, Zhuyun Dai, Gustavo Hernandez Abrego, Ji Ma, Vincent Zhao, Yi Luan, Keith Hall, Ming-Wei Chang, Yinfei Yang

Semantic Text Processing

Challenge:	Experimental results show that dual encoders outperform sparse and dense retrievers on the BEIR dataset significantly.
Approach:	They challenge belief that bottleneck layer is too limited for out-of-domain generalization . they scale up the model while keeping bottleneck as a single dot-product with a fixed size .
Outcome:	The proposed model outperforms sparse and dense retrievers on the BEIR dataset significantly.

TARN-VIST: Topic Aware Reinforcement Network for Visual Storytelling (2024.lrec-main)

Copied to clipboard

Weiran Chen, Xin Li, Jiaqi Su, Guiqian Zhu, Ying Li, Yi Ji, Chunping Liu

Decoding the Silent Majority: Inducing Belief Augmented Social Graph with Large Language Model for Response Forecasting (2023.emnlp-main)

Challenge:	Existing methods for visual storytelling ignore latent topic information.
Approach:	They propose a topic-aware reinforcement network for VIsual StoryTelling that takes topic information into account to generate a coherent story.
Outcome:	The proposed method outperforms most of the competing models across multiple evaluation metrics.

Copied to clipboard

Chenkai Sun, Jinning Li, Yi Fung, Hou Chan, Tarek Abdelzaher, ChengXiang Zhai, Heng Ji

Agenda-Driven Question Generation: A Case Study in the Courtroom Domain (2024.lrec-main)

Challenge:	Existing approaches to forecast news media responses have limited exploration of how to best process and utilize these important features.
Approach:	They propose a framework that leverages a large language model to induce a belief-centered graph on top of an existent social network, along with graph-based propagation to capture social dynamics.
Outcome:	The proposed framework surpasses state-of-the-art in experimental evaluations for both zero-shot and supervised settings, demonstrating its effectiveness in response forecasting.

Copied to clipboard

Yi Fung, Anoop Kumar, Aram Galstyan, Heng Ji, Prem Natarajan

Text Generation

Challenge:	Existing automated question generation methods focus on unstructured text and lack agenda and background documents as context.
Approach:	They propose to leverage large language models for CourtQG by fine-tuning them on two auxiliary tasks, agenda explanation and question type prediction.
Outcome:	The proposed method generates better questions according to standard metrics when compared to several baselines.

NORMSAGE: Multi-Lingual Multi-Cultural Norm Discovery from Conversations On-the-Fly (2023.emnlp-main)

Copied to clipboard

Yi Fung, Tuhin Chakrabarty, Hao Guo, Owen Rambow, Smaranda Muresan, Heng Ji

Dialogue Systems & Conversational Agents

Challenge:	Existing methods to understand acceptable behavior have focused on a single culture and manually built datasets from non-conversational settings.
Approach:	They propose a framework to automatically extract culture-specific norms from multi-lingual conversations.
Outcome:	The proposed framework extracts culture-specific norms from multi-lingual conversations.

Visual-Textual Alignment for Graph Inference in Visual Dialog (2020.coling-main)

Copied to clipboard

Tianling Jiang, Yi Ji, Chunping Liu, Hailin Shao

PaperRobot: Incremental Draft Generation of Scientific Ideas (P19-1)

Challenge:	Existing approaches to visual dialog do not understand semantic dependencies between visual and textual contents.
Approach:	They propose a Visual-Textual Alignment for Graph Inference network that makes up the lack of structural inference in visual dialog.
Outcome:	The proposed model outperforms existing models on a VisDial dataset.

Copied to clipboard

Qingyun Wang, Lifu Huang, Zhiying Jiang, Kevin Knight, Heng Ji, Mohit Bansal, Yi Luan

Structured Data in NLP

Challenge:	a paper robot can read existing papers and create new nodes or links in the knowledge graphs.
Approach:	They propose to automate the creation of new ideas by predicting links from the background KGs.
Outcome:	The proposed paper automates three tasks: read existing papers, create new ideas, predict links . the paper generated abstracts, conclusion and future work sections, and new titles are chosen over human-written ones up to 30%, 24% and 12% of the time.

Mitigating Object Hallucinations in MLLMs via Multi-Frequency Perturbations (2025.findings-emnlp)

Copied to clipboard

Shuo Li, Jiajun Sun, Guodong Zheng, Xiaoran Fan, Yujiong Shen, Yi Lu, Zhiheng Xi, Yuming Yang, Wenming Tan, Tao Ji, Tao Gui, Qi Zhang, Xuanjing Huang

Do LVLMs Understand Charts? Analyzing and Correcting Factual Errors in Chart Captioning (2024.findings-acl)

Challenge:	Multimodal large language models have demonstrated remarkable performance in visual-language tasks, but their authenticity is often compromised by object hallucinations.
Approach:	They propose a multi-frequency perturbation method that leverages both low-frequency and high-frequency features of images to perturb visual feature representations and explicitly suppress redundant frequency-domain features during inference.
Outcome:	The proposed method significantly mitigates object hallucinations across various model architectures.

Copied to clipboard

Kung-Hsiang Huang, Mingyang Zhou, Hou Pong Chan, Yi Fung, Zhenhailong Wang, Lingyu Zhang, Shih-Fu Chang, Heng Ji

ED2LM: Encoder-Decoder to Language Model for Faster Document Re-ranking Inference (2022.findings-acl)

Challenge:	LVLMs are known for producing text that is factually inconsistent with visual input . factuality of generated captions for structured visuals has not been studied as much .
Approach:	They propose a typology of factual errors in captions generated by large vision-language models . they propose CHOCOLATE, a visual entailment model that outperforms current models based on this analysis .
Outcome:	The proposed model outperforms current models in evaluating caption factuality.

Copied to clipboard

Kai Hui, Honglei Zhuang, Tao Chen, Zhen Qin, Jing Lu, Dara Bahri, Ji Ma, Jai Gupta, Cicero Nogueira dos Santos, Yi Tay, Donald Metzler

ReviewGrounder: Improving Review Substantiveness with Rubric-Guided, Tool-Integrated Agents (2026.acl-long)

Challenge:	State-of-the-art neural models typically encode document-query pairs using cross-attention for re-ranking.
Approach:	They propose to fine tune a pretrained encoder-decoder model using document to query generation.
Outcome:	The proposed model achieves comparable results to more expensive approaches while being 6.8X faster.

Copied to clipboard

Zhuofeng Li, Yi Lu, Dongfu Jiang, Haoxiang Zhang, Yuyang Bai, Chuan Li, Yu Wang, Shuiwang Ji, Jianwen Xie, Yu Zhang

Enhanced Chart Understanding via Visual Language Pre-training on Plot Table Pairs (2023.findings-acl)

Challenge:	Rapid rise in AI conference submissions has driven increasing exploration of large language models (LLMs) for peer review support.
Approach:	They propose a peer review benchmarking tool based on paper-specific rubrics and a rubric-guided framework that decomposes reviewing into drafting and grounding stages.
Outcome:	The proposed framework outperforms baselines with stronger/larger backbones in both alignment with human judgments and rubric-based review quality across 8 dimensions.

Copied to clipboard

Mingyang Zhou, Yi Fung, Long Chen, Christopher Thomas, Heng Ji, Shih-Fu Chang