Papers by Heng Li

Measuring the Effect of Influential Messages on Varying Personas (2023.acl-short)

Challenge:	We propose a new task to extract events and their arguments from multimedia documents . traditional methods target text, images or videos, but multimedia content is distributed via multimedia .
Approach:	They propose a method that encodes structured representations of semantic information from textual and visual data into a common embedding space.
Outcome:	The proposed method achieves 4.0% and 9.8% absolute gains on text event argument role labeling and visual event extraction.

Copied to clipboard

Chenkai Sun, Jinning Li, Hou Pong Chan, ChengXiang Zhai, Heng Ji

Sentiment Analysis

Challenge:	a new task estimates the response a persona might have upon seeing a news message . a first benchmark dataset is used to evaluate the performance of the proposed task .
Approach:	They propose a task to estimate the response a persona might have upon seeing a news message.
Outcome:	The proposed task estimates the response a persona might have upon seeing a news message.

GLEN: General-Purpose Event Detection for Thousands of Types (2023.emnlp-main)

Copied to clipboard

Sha Li, Qiusi Zhan, Kathryn Conger, Martha Palmer, Heng Ji, Jiawei Han

Learning from Mistakes: Negative Reasoning Samples Enhance Out-of-Domain Generalization (2026.acl-long)

Challenge:	ACE 2005 2 is the first large-scale event extraction dataset with 205K event mentions and 3,465 different types.
Approach:	They propose to use the DWD Overlay to map PropBank rolesets to a large distantlysupervised training dataset with partial labels to make event extraction more accessible.
Outcome:	The proposed model performs better than baselines including InstructGPT and ACE 2005 2 despite being 18 years old . key limitations of ACE include its small event ontology of 33 types, small dataset size of around 600 documents and restricted domain (with a significant portion concentrated on military conflicts).

Copied to clipboard

Tian Xueyun, MingHua Ma, Bingbing Xu, Nuoyan Lyu, Wei Li, Heng Dong, Zheng Chu, Yuanzhuo Wang, Huawei Shen

Joint Multimedia Event Extraction from Video and Article (2021.findings-emnlp)

Challenge:	Recent studies show that supervised fine-tuning (SFT) is a common approach for reasoning in large language models.
Approach:	They propose to use supervised fine-tuning (SFT) on chain-of-thought trajectories demonstrations . they find that incorporating negative traxories yields substantial OOD generalization gains .
Outcome:	The proposed scheme yields 5.51% OOD gain over positive-only training.

Copied to clipboard

Brian Chen, Xudong Lin, Christopher Thomas, Manling Li, Shoya Yoshida, Lovish Chum, Heng Ji, Shih-Fu Chang

PEARL: Self-Evolving Assistant for Time Management with Reinforcement Learning (2026.acl-long)

Challenge:	Existing methods to extract multimedia events from video and text are limited to video and images.
Approach:	They propose a task to jointly extract events from video and text documents . they propose 'self-supervised' cross-modal event coreference model and cross-mod transformer architecture .
Outcome:	The proposed method achieves 6.0% and 5.8% absolute F-score gain on video-article pairs . the proposed method can resolve coreference and extract multimodal event frames more accurately than existing methods.

Copied to clipboard

Bingxuan Li, Jeonghwan Kim, Cheng Qian, Xiusi Chen, Eitan Anzenberg, Niran Kundapur, Heng Ji

Challenge:	Scheduling logistics associated with meetings can amount to hours each week . a lack of automation can cause a lot of errors and can be costly .
Approach:	They propose a benchmark for long-horizon calendar conflict resolution that automates calendar conflict management . they propose 'pearl' that augments the language agent with an external preference memory .
Outcome:	a new benchmark for long-horizon calendar conflict resolution shows that existing agents perform poorly with high error rates.

Harmful Factuality: LLMs Correcting What They Shouldn’t (2026.findings-eacl)

Copied to clipboard

Mingchen Li, Hanzhi Zhang, Heng Fan, Junhua Ding, Yunhe Feng

Responsible & Trustworthy NLP

Challenge:	Large Language Models (LLMs) are trained for factual accuracy, but can conflict with the critical demand for source fidelity.
Approach:	They propose a reproducible framework to elicit and measure HFH using controlled entity-level perturbations and strategic entity selection.
Outcome:	The proposed framework reduces HFH rates by 50% across summarization, rephrasing, and QA tasks.

One QuantLLM for ALL: Fine-tuning Quantized LLMs Once for Efficient Deployments (2025.acl-long)

Copied to clipboard

Ke Yi, Yuhui Xu, Heng Chang, Yuan Meng, Tong Zhang, Jia Li

LiveCANNBench: Benchmark SWE AI Coding for Ascend CANN (2026.findings-acl)

Challenge:	Quantization has shown promise for Large Language Models, but current methods require lengthy training to alleviate quantization loss.
Approach:	They propose to decouple weights and incorporate Low-Rank adapters to reduce weight sharing . they validate the approach on LLaMA2 families and Mistral on downstream evaluation .
Outcome:	The proposed approach shows high performance while reducing deployment time faced with multiple scenarios.

Copied to clipboard

Sijie Wang, Kai Zhao, Wee Peng Tay, Shuo Zhang, Chengwen Liu, Quanjiang Guo, Ren Junhao, Xin Li, Heng Lian, Jingdi Lei, Rui She, Huacan Wang, Ronghao Chen

Challenge:	Recent advances in agents have enabled multi-file, multi-language, and dependency-aware AI coding.
Approach:	They propose an SWE-level benchmark for AI coding in the Huawei Ascend CANN software stack.
Outcome:	The proposed benchmark is constructed from real-world CANN repositories and consists of over 400 task instances spanning multiple file, multi-language, and execution-aware coding challenges.

The Future is not One-dimensional: Complex Event Schema Induction by Graph Modeling for Event Prediction (2021.emnlp-main)

Copied to clipboard

Manling Li, Sha Li, Zhenhailong Wang, Lifu Huang, Kyunghyun Cho, Heng Ji, Jiawei Han, Clare Voss

MP-RNA: Unleashing Multi-species RNA Foundation Model via Calibrated Secondary Structure Prediction (2024.findings-emnlp)

Challenge:	Event schemas encode knowledge of stereotypical structures of events and their connections . previous work on event schema induction focuses on atomic events or linear temporal sequences .
Approach:	They propose a Temporal Complex Event Schema: a graph-based schema representation that encompasses events, arguments, temporal connections and argument relations.
Outcome:	The proposed model outperforms existing models on HITS@1 by 17.8%.

Copied to clipboard

From Word to World: Can Large Language Models be Implicit Text-based World Models? (2026.acl-long)

Challenge:	Experimental evaluations demonstrate that our RNA FM consistently outperforms existing RNA .
Approach:	They propose to use filtered high-fidelity structure annotations to enhance the modeling ability of FMs in single nucleotide resolution tasks.
Outcome:	The proposed model outperforms existing RNA FMs on four genomic benchmarks and achieves top-tier results on DNA genomic benchmark.

Copied to clipboard

Yixia Li, Hongru Wang, Jiahao Qiu, Zhenfei Yin, Dongdong Zhang, Cheng Qian, Zeping Li, Xiaoteng Ma, Guanhua Chen, Heng Ji

Reaction Miner: An Integrated System for Chemical Reaction Extraction from Textual Data (2023.emnlp-demo)

Challenge:	Agentic learning increasingly hinges on interaction, yet real-world experience is expensive, limited, and often irreversible at inference time.
Approach:	They propose a framework that reframes language modeling as next-state prediction under interaction.
Outcome:	The proposed framework evaluates world models in text-based environments . it shows that sufficiently trained models capture coherent environment dynamics .

Copied to clipboard

Ming Zhong, Siru Ouyang, Yizhu Jiao, Priyanka Kargupta, Leo Luo, Yanzhen Shen, Bobby Zhou, Xianrui Zhong, Xuan Liu, Hongxiang Li, Jinfeng Xiao, Minhao Jiang, Vivian Hu, Xuan Wang, Heng Ji, Martin Burke, Huimin Zhao, Jiawei Han

PQR: Improving Dense Retrieval via Potential Query Modeling (2025.acl-long)

Challenge:	Reaction Miner is a system designed to extract chemical reactions from raw scientific PDFs.
Approach:	They propose a system that extracts chemical reactions directly from raw scientific PDFs.
Outcome:	The proposed system can extract chemical reactions from raw scientific PDFs.

Copied to clipboard

Junfeng Kang, Rui Li, Qi Liu, Yanjiang Chen, Zheng Zhang, Junzhe Jiang, Heng Yu, Yu Su

Multimodal Causal Reasoning Benchmark: Challenging Multimodal Large Language Models to Discern Causal Links Across Modalities (2025.findings-acl)

Challenge:	Existing training data is sparse, with each document associated with one or a few labeled queries.
Approach:	They propose a training-free potential query retrieval framework to address this problem . they use a Gaussian mixture distribution to model all potential queries for a document .
Outcome:	The proposed method is able to capture comprehensive semantic information from a document with multiple queries.

Copied to clipboard

Zhiyuan Li, Heng Wang, Dongnan Liu, Chaoyi Zhang, Ao Ma, Jieting Long, Weidong Cai

ProgGen: Generating Named Entity Recognition Datasets Step-by-step with Self-Reflexive Large Language Models (2024.findings-acl)

Challenge:	Existing MLLMs lack robustness in multimodal causal reasoning compared to their performance in textual settings.
Approach:	They propose a novel multimodal chain-of-thought (CoT) reasoning benchmark that leverages siamese images and text pairs to challenge MLLMs.
Outcome:	The proposed benchmark leverages siamese images and text pairs to challenge MLLMs.

Copied to clipboard

Yuzhao Heng, Chunyuan Deng, Yitong Li, Yue Yu, Yinghao Li, Rongzhi Zhang, Chao Zhang

Why Does New Knowledge Create Messy Ripple Effects in LLMs? (2024.emnlp-main)

Challenge:	Large Language Models (LLMs) exhibit remarkable adaptability across domains, but they are often not suitable for structured knowledge extraction tasks such as named entity recognition (NER).
Approach:	They propose a method that instructs LLMs to self-reflect on the specific domain and generates domain-relevant attributes for creating attribute-rich training data.
Outcome:	The proposed method produces NER datasets in domains with domain-relevant attributes and generates entity terms and NER context data around these entities.

Copied to clipboard

Jiaxin Qin, Zixuan Zhang, Chi Han, Pengfei Yu, Manling Li, Heng Ji

Human-in-the-loop Schema Induction (2023.acl-demo)

Challenge:	Existing research has focused on post-training knowledge editing (KE) for language models to ensure that knowledge remains accurate and up-to-date.
Approach:	They propose to use a GradSim indicator to detect when and why updated knowledge ripples in language models.
Outcome:	The proposed indicator GradSim shows that LMs that fail to handle ripple effects have low GradSIM.

Copied to clipboard

Tianyi Zhang, Isaac Tham, Zhaoyi Hou, Jiaxuan Ren, Leon Zhou, Hainiu Xu, Li Zhang, Lara Martin, Rotem Dror, Sha Li, Heng Ji, Martha Palmer, Susan Windisch Brown, Reece Suchocki, Chris Callison-Burch

From Implicit Graph Encoding to Explicit Evidence: A Training-Free LLM Framework for Temporal Knowledge Graph Reasoning (2026.findings-acl)

Challenge:	Existing approaches to event-centric natural language understanding (NLU) have been limited to linear and temporal ones.
Approach:	They propose a human-in-the-loop schema induction system powered by GPT-3 . they show that it transfers to new domains more easily than previous approaches .
Outcome:	The proposed system transfers to new domains more easily than previous approaches and reduces human curation.

Copied to clipboard

Guo Tang, Ke Cheng, Huiming Fan, Heng Chang, Wenxiang Zheng, Xianhao Ou, Junjia Xiang, Ming Liu, Yujun Zhou, Li Lanyu, Bing Qin

EVEDIT: Event-based Knowledge Editing for Deterministic Knowledge Propagation (2024.emnlp-main)

Challenge:	Existing Large Language Models (LLMs) struggle with implicit modality alignment and suboptimal graph linearization.
Approach:	They propose a training-free, test-time adaptive framework that reframes TKG prediction as explicit evidence-driven reasoning.
Outcome:	ExE-LLM outperforms fully trained graph neural networks on four benchmarks . it achieves SOTA performance in inductive settings, significantly outperforming fully trained neural networks .

Copied to clipboard

Jiateng Liu, Pengfei Yu, Yuji Zhang, Sha Li, Zixuan Zhang, Ruhi Sarikaya, Kevin Small, Heng Ji

Can Language Models Follow Multiple Turns of Entangled Instructions? (2025.findings-emnlp)

Challenge:	Existing knowledge editing approaches only operate on (subject, relation, object) triple . current methods are limited to (substance, relation) triple, causing low confidence in their answers.
Approach:	They propose a task of event-based knowledge editing that pairs facts with event descriptions to improve model confidence.
Outcome:	The proposed method improves model confidence by 55.6% while maintaining the naturalness of generation.

Copied to clipboard

Chi Han, Xin Liu, Haodong Wang, Shiyang Li, Jingfeng Yang, Haoming Jiang, Zhengyang Wang, Qingyu Yin, Liang Qiu, Changlong Yu, Yifan Gao, Zheng Li, Bing Yin, Jingbo Shang, Heng Ji

Document-Level Event Argument Extraction by Conditional Generation (2021.naacl-main)

Challenge:	Despite of significant achievements in improving instruction-following capabilities of large language models, the ability to process multiple potentially entangled or conflicting instructions remains a considerable challenge.
Approach:	They construct multi-turn instruction with 1.1K high-quality multi-turned conversations using the human-in-the-loop approach and examine their capabilities.
Outcome:	The proposed model shows that it is difficult to integrate multiple turns and balance competing objectives when instructions intersect or conflict.

Copied to clipboard

Sha Li, Heng Ji, Jiawei Han

Event-Centric Natural Language Processing (2021.acl-tutorials)

Challenge:	Existing event extraction models have been limited to the sentence level . this formulation signifies a misalignment between the information seeking behavior and the informative seeking behavior.
Approach:	They propose a document-level neural event argument extraction model by formulating the task as conditional generation following event templates.
Outcome:	The proposed model achieves 7.6% F1 and 5.7% F1 over the best baseline on the document-level event extraction dataset WikiEvents and 9.3% F1 on the informative argument extraction task.

Copied to clipboard

Muhao Chen, Hongming Zhang, Qiang Ning, Manling Li, Heng Ji, Kathleen McKeown, Dan Roth

Challenge:	This tutorial will provide an introduction to various methods for automating the extraction, conceptualization and prediction of events and their relations.
Approach:	This tutorial will provide an introduction to various methods for automating events and their relations, and a wide range of NLU and commonsense understanding tasks.
Outcome:	This tutorial will provide an introduction to various methods for automating extraction, conceptualization and prediction of events and their relations, and a wide range of NLU and commonsense understanding tasks.

EMCompress: Video-LLMs with Endomorphic Multimodal Compression (2026.findings-acl)

Copied to clipboard

Zheyu Fan, Jiateng Liu, Yuji Zhang, Zihan Wang, Yi R. Fung, Manling Li, Heng Ji

Code4Struct: Code Generation for Few-Shot Event Structure Prediction (2023.acl-long)

Challenge:	Static, sparse frame sampling either dilutes evidence across task-irrelevant segments at significant cost or misses fine-grained temporal semantics altogether.
Approach:	They propose a novel task that compresses multimodal input while preserving answer invariance across reasonable downstream models.
Outcome:	The proposed task surpasses prior methods by 0.40 F-1 with competitive query rewriting.

Copied to clipboard

Xingyao Wang, Sha Li, Heng Ji

Document-level Event Factuality Identification via Machine Reading Comprehension Frameworks with Transfer Learning (2022.coling-1)

Challenge:	Large Language Models (LLMs) trained on a mixture of text and code have demonstrated impressive capability in translating natural language (NL) into structured code.
Approach:	They propose to use programming language (PL) inheritance and type annotations to translate text into code to tackle structured prediction tasks.
Outcome:	The proposed model outperforms existing models on 20-shot data by 29.5% absolute F1.

Copied to clipboard

Zhong Qian, Heng Zhang, Peifeng Li, Qiaoming Zhu, Guodong Zhou

Instruct and Extract: Instruction Tuning for On-Demand Information Extraction (2023.emnlp-main)

Challenge:	Document-level Event Factuality Identification (DEFI) is a fundamental and crucial task in NLP.
Approach:	They propose a framework for document-level event factuality identification (DEFI) they propose to use Span-Extraction and Multiple-Choice to model DEFI as machine reading comprehension tasks .
Outcome:	The proposed model outperforms state-of-the-art models on a document-based event factuality task . it uses Span-Extraction (Ext) and Multiple-Choice (Mch) knowledge to extract knowledge from large-scale MRC corpus .

Copied to clipboard

Yizhu Jiao, Ming Zhong, Sha Li, Ruining Zhao, Siru Ouyang, Heng Ji, Jiawei Han

Open Relation and Event Type Discovery with Type Abstraction (2022.emnlp-main)

Challenge:	Large language models with instruction-following capabilities are not suitable for long-tail ad hoc extraction use cases for non-expert users.
Approach:	They propose a task that follows instructions to extract the desired content from the associated text and present it in a structured tabular format.
Outcome:	The proposed paradigm outperforms existing open-source models of similar size in terms of information extraction.

Copied to clipboard

Sha Li, Heng Ji, Jiawei Han

Enhancing Advanced Visual Reasoning Ability of Large Language Models (2024.emnlp-main)

Challenge:	Conventional "closed-world" information extraction methods rely on human ontologies to define scope for extraction.
Approach:	They propose a type abstraction approach where models are prompted to generalize and name the type . they use the similarity between inferred names to induce clusters .
Outcome:	The proposed method is complementary to token representations on relation extraction and event extraction datasets.

Copied to clipboard

Zhiyuan Li, Dongnan Liu, Chaoyi Zhang, Heng Wang, Tengfei Xue, Weidong Cai

RESIN: A Dockerized Schema-Guided Cross-document Cross-lingual Cross-media Information Extraction and Event Tracking System (2021.naacl-demos)

Challenge:	Recent advances in Vision-Language (VL) research have sparked new benchmarks for complex visual reasoning, challenging models’ advanced reasoning ability.
Approach:	They propose a novel multi-modal in-context learning methodology to enhance LLMs’ contextual understanding and reasoning.
Outcome:	The proposed model achieves SOTA performance among all visual reasoning tasks and achieves a 'higher level of accuracy' than previous models.

Copied to clipboard

Haoyang Wen, Ying Lin, Tuan Lai, Xiaoman Pan, Sha Li, Xudong Lin, Ben Zhou, Manling Li, Haoyu Wang, Hongming Zhang, Xiaodong Yu, Alexander Dong, Zhenhailong Wang, Yi Fung, Piyush Mishra, Qing Lyu, Dídac Surís, Brian Chen, Susan Windisch Brown, Martha Palmer, Chris Callison-Burch, Carl Vondrick, Jiawei Han, Dan Roth, Shih-Fu Chang, Heng Ji

Visual Data in NLP

Challenge:	We present a new information extraction system that can construct temporal event graphs from news documents.
Approach:	They propose a temporal event graph extraction system that can extract news documents . they extend the system from sentence-level event extraction to cross-document cross-media event extraction .
Outcome:	The proposed system can extract temporal event graphs from news documents in multiple languages and multiple data modalities.

EscapeBench: Towards Advancing Creative Intelligence of Language Model Agents (2025.acl-long)

Copied to clipboard

Cheng Qian, Peixuan Han, Qinyu Luo, Bingxiang He, Xiusi Chen, Yuji Zhang, Hongyi Du, Jiarui Yao, Xiaocheng Yang, Denghui Zhang, Yunzhu Li, Heng Ji

Schema-Guided Culture-Aware Complex Event Simulation with Multi-Agent Role-Play (2024.emnlp-demo)

Challenge:	Existing language model agents excel in planning and reasoning, but lack creativity in unfamiliar environments.
Approach:	They propose a benchmark suite of room escape game environments to challenge agents with creative reasoning, unconventional tool use and iterative problem-solving to uncover implicit goals.
Outcome:	The proposed framework can perform with 40% fewer steps and hints and performs robustly across difficulty levels.

Copied to clipboard

Sha Li, Revanth Gangi Reddy, Khanh Nguyen, Qingyun Wang, Yi Fung, Chi Han, Jiawei Han, Kartik Natarajan, Clare Voss, Heng Ji

Challenge:	Complex news events require swift responses from government and society, authors say . relying on historical events to project the future is insufficient, they say - a simulator for complex news events is needed .
Approach:	They propose a controllable complex news event simulator guided by event schema and user-provided assumptions . they incorporate a geo-diverse commonsense and cultural norm-aware knowledge enhancement component .
Outcome:	The proposed simulator achieves higher coherence and appropriateness than existing models.

RESIN-11: Schema-guided Event Prediction for 11 Newsworthy Scenarios (2022.naacl-demo)

Copied to clipboard

Xinya Du, Zixuan Zhang, Sha Li, Pengfei Yu, Hongwei Wang, Tuan Lai, Xudong Lin, Ziqi Wang, Iris Liu, Ben Zhou, Haoyang Wen, Manling Li, Darryl Hannan, Jie Lei, Hyounghun Kim, Rotem Dror, Haoyu Wang, Michael Regan, Qi Zeng, Qing Lyu, Charles Yu, Carl Edwards, Xiaomeng Jin, Yizhu Jiao, Ghazaleh Kazeminejad, Zhenhailong Wang, Chris Callison-Burch, Mohit Bansal, Carl Vondrick, Jiawei Han, Dan Roth, Shih-Fu Chang, Martha Palmer, Heng Ji

Improving Consistency for Text Summarization with Energy Functions (2023.findings-emnlp)

Challenge:	Existing methods for event prediction are incomplete and noisy.
Approach:	They propose to use news-related event schemas to extract newsworthy events . they build a demo website and include a video demonstrating the framework .
Outcome:	The proposed framework can be applied to a wide variety of newsworthy scenarios.

Copied to clipboard

Qi Zeng, Qingyu Yin, Zheng Li, Yifan Gao, Sreyashi Nag, Zhengyang Wang, Bing Yin, Heng Ji, Chao Zhang

Summarization

Challenge:	Current abstractive summarization models generate inconsistent content due to the inherently noisy dataset and the discrepancy between maximum likelihood estimation based training objectives and consistency measurements.
Approach:	They propose a new consistency taxonomy that categorizes inconsistent content into faithfulness, factuality, and self-supportiveness.
Outcome:	Experiments on XSUM and CNN/DM datasets show that EnergySum mitigates the trade-off between accuracy and consistency.

Syntax-aware Multi-task Graph Convolutional Networks for Biomedical Relation Extraction (D19-62)

Copied to clipboard

Diya Li, Heng Ji

InstOptima: Evolutionary Multi-objective Instruction Optimization via Large Language Model-based Instruction Operators (2023.findings-emnlp)

Challenge:	80% of the data sets for relation extraction tasks are negative instances, resulting in a lack of syntactic information between two entity mentions.
Approach:	They propose a graph convolutional networks model that incorporates dependency parsing and contextualized embedding to capture comprehensive contextual information.
Outcome:	The proposed model achieves state-of-the-art F-score on the 2013 drug-drug interaction extraction task.

Copied to clipboard

MACAROON: Training Vision-Language Models To Be Your Engaged Partners (2024.findings-emnlp)

Challenge:	Existing studies focus on automating instruction generation but do not consider other objectives that impact instruction quality.
Approach:	They propose an approach that treats instruction generation as an evolutionary multi-objective optimization problem.
Outcome:	The proposed approach improves fine-tuning performance and the generation of high-quality instructions.

Copied to clipboard

Shujin Wu, Yi Fung, Sha Li, Yixin Wan, Kai-Wei Chang, Heng Ji

Deep Learning on Graphs for Natural Language Processing (2021.naacl-tutorials)

Challenge:	Large vision-language models (LVLMs) generate detailed responses even when questions are ambiguous or unanswerable, leading to hallucinations and bias issues.
Approach:	They propose a three-tiered hierarchy for questions of invalid, ambiguous, and personalizable nature to measure the proactive engagement capabilities of LVLMs.
Outcome:	The proposed model generates contrastive response pairs for unlabeled questions, achieving 0.84 AAR, while maintaining comparable performance on general tasks.

Copied to clipboard

Lingfei Wu, Yu Chen, Heng Ji, Yunyao Li

CA-GAR: Context-Aware Alignment of LLM Generation for Document Retrieval (2025.findings-acl)

Challenge:	Graph Neural Networks (GNNs) are powerful tools for non-Euclidean data modeling and are used in many graph-related NLP tasks.
Approach:	This tutorial will cover applying deep learning on graph techniques to NLP using Graph Neural Networks (GNNs) Graph4NLP is the first library for researchers and practitioners for easy use of GNNs for various NLP tasks.
Outcome:	This tutorial will cover the latest developments in deep learning on graph techniques and their applications in various NLP tasks.

Copied to clipboard

Heng Yu, Junfeng Kang, Rui Li, Qi Liu, Liyang He, Zhenya Huang, Shuanghong Shen, Junyu Lu

COVID-19 Literature Knowledge Graph Construction and Drug Repurposing Report Generation (2021.naacl-demos)

Challenge:	Recent techniques such as Generation-Augmented Retrieval (GAR) and Generative Document Retrieleval (GDR) leverage LLMs to enhance retrieval performance but face key challenges: GAR’s generated content may not always align with the target document corpus, while GDR limits the generative capacity of LLM.
Approach:	They propose a Context-Aware Generation-Augmented Retrieval approach which integrates corpus information into their generation process.
Outcome:	Experimental results show that CA-GAR outperforms existing methods on seven tasks and four non-English languages.

Copied to clipboard

Qingyun Wang, Manling Li, Xuan Wang, Nikolaus Parulian, Guangxing Han, Jiawei Ma, Jingxuan Tu, Ying Lin, Ranran Haoran Zhang, Weili Liu, Aabhas Chauhan, Yingjun Guan, Bangzheng Li, Ruisong Li, Xiangchen Song, Yi Fung, Heng Ji, Jiawei Han, Shih-Fu Chang, James Pustejovsky, Jasmine Rah, David Liem, Ahmed ELsayed, Martha Palmer, Clare Voss, Cynthia Schneider, Boyan Onyshkevych

HyperAdaLoRA: Accelerating LoRA Rank Allocation During Training via Hypernetworks without Sacrificing Performance (2026.findings-acl)

Challenge:	a new framework to digest relevant biomedical knowledge is needed to combat COVID-19 . quantity of research results is a bottleneck, and false information promoted in publications .
Approach:	a team of researchers has developed a framework to extract multimedia knowledge elements from scientific literature to combat COVID-19.
Outcome:	a new framework extracts fine-grained multimedia knowledge elements from scientific literature . it provides detailed contextual sentences, subfigures, and knowledge subgraphs as evidence . the framework is based on a case study of drug repurposing .

Copied to clipboard

Hao Zhang, Zhenjia Li, Yifan Gao, Xi Xiao, Heng Zhang, Shuyang Zhang, null Xiaoxincc, Bo Huang, Yuhang Wu, Tianyang Wang, Hao Xu

Event Schema Induction with Double Graph Autoencoders (2022.naacl-main)

Challenge:	Low-Rank Adaptation (LoRA) assumes a uniform rank r for each incremental matrix, not accounting for the varying significance of weight matrices across modules and layers.
Approach:	They propose a framework that allows for faster convergence of low-rank adaptive models . they use a hypernetwork to prune the outputs of the hypernetworks to generate parameters .
Outcome:	The proposed framework accelerates convergence of AdaLoRA by leveraging a hypernetwork.

Copied to clipboard

Xiaomeng Jin, Manling Li, Heng Ji

Current Agents Fail to Leverage World Model as Tool for Foresight (2026.acl-long)

Challenge:	Experimental results show that a new method for learning event schemas from historical events is effective.
Approach:	They propose a new event schema induction framework which captures global dependencies among nodes in event graphs.
Outcome:	Experimental results show that the proposed model can learn event schemas with global consistency.

Copied to clipboard

Cheng Qian, Emre Can Acikgoz, Bingxuan Li, Xiusi Chen, Yuji Zhang, Bingxiang He, Qinyu Luo, Gokhan Tur, Dilek Hakkani-Tür, Yunzhu Li, Heng Ji

COVID-19 Claim Radar: A Structured Claim Extraction and Tracking System (2022.acl-demo)

Challenge:	Generative world models could be used to enhance agents' cognition . agents are expected to operate in settings where tasks unfold over long horizons and involve intricate chains of interdependent decisions.
Approach:	They propose to use vision-language models as external simulators to enhance cognition . they find that agents rarely invoke simulation and misuse predicted rollouts .
Outcome:	The proposed model could be used to predict future states rather than short-horizon reasoning . the model could also be used for real-world planning and robotics .

Copied to clipboard

Manling Li, Revanth Gangi Reddy, Ziqi Wang, Yi-shyuan Chiang, Tuan Lai, Pengfei Yu, Zixuan Zhang, Heng Ji

Keep Meeting Summaries on Topic: Abstractive Multi-Modal Meeting Summarization (P19-1)

Challenge:	a new system extracts supporting and refuting claims from COVID-19 related news . the system is publicly available at GitHub and DockerHub, with complete documentation.
Approach:	They propose a COVID-19 Claim Radar system that extracts supporting and refuting claims . the system leverages Wikidata as the hub to consolidate coreferential knowledge elements .
Outcome:	The system extracts supporting and refuting claims from COVID-19 pandemic information . it leverages Wikidata as the hub to merge coreferential knowledge elements .

Copied to clipboard

Manling Li, Lingyu Zhang, Heng Ji, Richard J. Radke

Eval-RAR: Evaluation-Driven Retrieval-Augmented Reasoning via Reinforcement Learning (2026.findings-acl)

Challenge:	Existing models for extractive summarization of meetings are unfocused and lack content coverage.
Approach:	They propose a multi-modal hierarchical attention model that prioritizes segmentation and summarization . they propose to use multi-level hierarchies to narrow down the focus into topically-relevant segments .
Outcome:	The proposed model outperforms the state-of-the-art with BLEU and ROUGE measures.

Copied to clipboard

Heng Yu, Rui Li, Qi Liu, Wenjun Feng, Junfeng Kang, Yi Zhan

RESIN-EDITOR: A Schema-guided Hierarchical Event Graph Visualizer and Editor (2023.emnlp-demo)

Challenge:	Existing methods for retrieval-augmented generation fail to provide explicit supervision for internal reasoning process.
Approach:	They propose an Evaluation-driven Retrieval-Augmented Reasoning framework that uses reinforcement learning and a fine-grained evaluation reward to optimize the process.
Outcome:	Eval-RAR outperforms existing methods on QA benchmarks on seven single-hop and multi-hop tasks.

Copied to clipboard

Khanh Duy Nguyen, Zixuan Zhang, Reece Suchocki, Sha Li, Martha Palmer, Susan Windisch Brown, Jiawei Han, Heng Ji

Connecting the Dots: Event Graph Schema Induction with Path Language Modeling (2020.emnlp-main)

Challenge:	Existing IE tools for atomic events are limited when applied to such complex events.
Approach:	They propose to use event schemas to guide the organization of complex events and to edit hierarchical graphs.
Outcome:	The proposed tool outperforms existing IE visualization tools in both IE result analysis and general model improvements.

Copied to clipboard

Manling Li, Qi Zeng, Ying Lin, Kyunghyun Cho, Heng Ji, Jonathan May, Nathanael Chambers, Clare Voss

The Best Defense is Attack: Repairing Semantics in Textual Adversarial Examples (2024.emnlp-main)

Challenge:	Existing methods to automate event extraction focus on uncertainty, re-occurring events and multiple hypotheses.
Approach:	They propose a new Event Graph Schema where two event types are connected through multiple paths involving entities that fill important roles in a coherent story.
Outcome:	The proposed model is highly effective at inducing salient and coherent schemas.

Copied to clipboard

Chem-FINESE: Validating Fine-Grained Few-shot Entity Extraction through Text Reconstruction (2024.findings-eacl)

Robustness in NLP

Challenge:	Recent studies have revealed the vulnerability of pre-trained language models to adversarial attacks.
Approach:	They propose a novel approach to repair adversarial examples using an adversarial detector.
Outcome:	The proposed approach is effective in various adversarial attack scenarios.

Copied to clipboard

Qingyun Wang, Zixuan Zhang, Hongxiang Li, Xuan Liu, Jiawei Han, Huimin Zhao, Heng Ji

Timeline Summarization based on Event Graph Compression via Time-Aware Optimal Transport (2021.emnlp-main)

Challenge:	Existing frameworks for fine-grained few-shot entity extraction are difficult to implement in the chemical domain due to the information overload of scientific papers.
Approach:	They propose a sequence-to-sequence based few-shot entity extraction approach . it uses a seq2seq entity extractor and a self-validation module to reconstruct original input sentence .
Outcome:	The proposed framework achieves 8.26% and 6.84% performance gains on two datasets.

Copied to clipboard

Manling Li, Tengfei Ma, Mo Yu, Lingfei Wu, Tian Gao, Heng Ji, Kathleen McKeown

How do Language Models Reshape Entity Alignment? A Survey of LM-Driven EA Methods: Advances, Benchmarks, and Future (2025.emnlp-main)

Challenge:	Existing methods for timeline summarization ignore the events’ intra-structures and inter-structure connections.
Approach:	They propose to represent news articles as an event-graph, thus compressing the whole graph to its salient sub-graph.
Outcome:	The proposed method significantly improves on the state-of-the-art on three real-world datasets, including two public benchmarks and a Timeline100 dataset.

Copied to clipboard

Zerui Chen, Huiming Fan, Qianyu Wang, Tao He, Ming Liu, Heng Chang, Weijiang Yu, Ze Li, Bing Qin

Unraveling the Mystery: Defending Against Jailbreak Attacks Via Unearthing Real Intention (2025.coling-main)

Challenge:	Entity alignment (EA) is critical for knowledge graph (KG) integration.
Approach:	They propose a taxonomy that categorizes methods in three stages: data preparation, feature embedding, and alignment.
Outcome:	The proposed taxonomy categorizes methods in three key stages: data preparation, feature embedding, and alignment.

Copied to clipboard

Yanhao Li, Hongshen Chen, Heng Zhang, Zhiwei Ge, Tianhao Li, Sulong Xu, Guibo Luo

A Language-First Approach for Procedure Planning (2023.findings-acl)

Challenge:	Large Language Models (LLMs) are increasingly vulnerable to elusive and implicit intentions, causing security risks and compromising user experience.
Approach:	They propose a method to detect and mitigate implicit jailbreak attacks using LLMs by unearthing real intentions and a greedy gradient-based algorithm to remove the least important parts of a sentence.
Outcome:	The proposed method reduces attacks success rate and Harmful Score while maintaining overall model performance.

Copied to clipboard

Jiateng Liu, Sha Li, Zhenhailong Wang, Manling Li, Heng Ji

Incorporating Graph Attention Mechanism into Knowledge Graph Reasoning Based on Deep Reinforcement Learning (D19-1)

Challenge:	Developing intelligent agents requires the ability to produce plans on the fly based on visual observations.
Approach:	They propose a language-first procedure planning framework with a modularized design . they first align current and goal observations with corresponding steps and then use a pre-trained LM to predict intermediate steps.
Outcome:	The proposed framework matches state-of-the-art procedures on COIN and CrossTask benchmarks.

Copied to clipboard

Heng Wang, Shuangyin Li, Rong Pan, Mingzhi Mao

Defining a New NLP Playground (2023.findings-emnlp)

Challenge:	Existing methods for learning knowledge Graphs are incomplete and therefore need well-pretraining.
Approach:	They propose a deep reinforcement learning based model which incorporates LSTM and Graph Attention Mechanism as the memory components.
Outcome:	The proposed model can get rid of the pretraining process and achieve state-of-the-art performance compared with the other models.

Copied to clipboard

Sha Li, Chi Han, Pengfei Yu, Carl Edwards, Manling Li, Xingyao Wang, Yi Fung, Charles Yu, Joel Tetreault, Eduard Hovy, Heng Ji

ReSo: A Reward-driven Self-organizing LLM-based Multi-Agent System for Reasoning Tasks (2025.emnlp-main)

Challenge:	Recent explosion of performance of large language models (LLMs) has changed the field more abruptly and seismically than any other shift in the field’s 80 year history.
Approach:	They propose 20+ PhD-dissertation-worthy research directions to define a new NLP playground by combining theoretical analysis, new and challenging problems, learning paradigms and interdisciplinary applications.
Outcome:	The proposed research will cover theoretical analysis, new and challenging problems, learning paradigms and interdisciplinary applications.

Copied to clipboard

Heng Zhou, Hejia Geng, Xiangyuan Xue, Li Kang, Yiran Qin, Zhiyong Wang, Zhenfei Yin, Lei Bai

Incorporating Factuality Inference to Identify Document-level Event Factuality (2023.findings-acl)

Challenge:	Multi-agent systems (MAS) are limited by poor flexibility and scalability, with underdeveloped optimization strategies.
Approach:	They propose a task graph generation and a reward-driven two-stage agent selection process to integrate multi-agent systems to improve their reasoning capabilities.
Outcome:	The proposed model outperforms existing methods on Math-MAS and SciBench-MAS SciBech, while other methods completely fail.

Copied to clipboard

Heng Zhang, Peifeng Li, Zhong Qian, Xiaoxu Zhu

Challenge:	Document-level Event Factuality Identification (DEFI) is a task for identifying the degree of certainty that a specific event occurs in a document.
Approach:	They propose a document-level event factuality inference task to bridge the sentence-level and document- level event factualality semantically.
Outcome:	The proposed model outperforms the baselines on the public English and Chinese DLEF datasets.

Analyzing and Internalizing Complex Policy Documents for LLM Agents (2026.acl-long)

Copied to clipboard

Jiateng Liu, Zhenhailong Wang, Xiaojiang Huang, Yingjie Li, Xiang Li, Chenlei Guo, Xing Fan, Ruhi Sarikaya, Heng Ji

Boosting Text Augmentation via Hybrid Instance Filtering Framework (2023.findings-acl)

Challenge:	Large language model agents rely on in-context policy documents to act as effective user assistants.
Approach:	They propose an agentic benchmark generator with Controllable Complexity in agent policy across four levels to evaluate agents under increasing complexity.
Outcome:	The proposed method outperforms the baseline in data-sparse and high-complexity settings.

Copied to clipboard

Text2DB: Integration-Aware Information Extraction with Large Language Model Agents (2024.findings-acl)

Challenge:	Existing text augmentation methods generate instances with shifted feature spaces, which leads to a drop in performance on large datasets.
Approach:	They propose a hybrid instance-filtering framework that generates instances with shifted feature spaces, which leads to a drop in performance on augmented data.
Outcome:	The proposed framework outperforms state-of-the-art methods on three classification tasks and nine public datasets.

Copied to clipboard

Yizhu Jiao, Sha Li, Sizhe Zhou, Heng Ji, Jiawei Han

New Frontiers of Information Extraction (2022.naacl-tutorials)

Challenge:	Current methods for information extraction (IE) focus on integrating IE output with the database . a long-overlooked question is what counts as "relevant knowledge"
Approach:	They propose a task that emphasizes integration of IE output and the database . they introduce a benchmark and an LLM agent framework for this task .
Outcome:	The proposed task integrates IE output and the target database (or knowledge base) it meets common demands such as data infilling, row population, and column addition .

Copied to clipboard

Muhao Chen, Lifu Huang, Manling Li, Ben Zhou, Heng Ji, Dan Roth

Dynamic Global Memory for Document-level Argument Extraction (2022.acl-long)

Challenge:	Information extraction (IE) is the process of automatically extracting structural information from unstructured or semi-structured data.
Approach:	This tutorial will provide an introduction to recent advances in IE by answering several important research questions.
Outcome:	The tutorial will address several important research questions and outline directions for further investigation.

Copied to clipboard

Xinya Du, Sha Li, Heng Ji

DP-GTR: Differentially Private Prompt Protection via Group Text Rewriting (2025.findings-emnlp)

Challenge:	Recent work on document-level event argument extraction is restricted by sequence length constraints and ignores global context between events.
Approach:	They propose to construct a document memory store to extract contextual event information and leverage it to implicitly and explicitly help with decoding of arguments for later events.
Outcome:	The proposed framework outperforms prior methods and is more robust to adversarially annotated examples with constrained decoding design.

Copied to clipboard

Mingchen Li, Heng Fan, Song Fu, Junhua Ding, Yunhe Feng

Responsible & Trustworthy NLP

Challenge:	Existing methods for prompt privacy focus on document-level rewriting, neglecting rich, multi-granular representations of text.
Approach:	a framework that leverages local differential privacy and composition theorem via group text rewriting is proposed . the framework is compatible with existing rewrite techniques and is publicly available at anonymous.4open.science for reproducibility.
Outcome:	DP-GTR is the first framework to integrate document-level and word-level information while exploiting in-context learning to improve privacy and utility.

UniRAG: Unified Query Understanding Method for Retrieval Augmented Generation (2025.acl-long)

Copied to clipboard

Rui Li, Liyang He, Qi Liu, Zheng Zhang, Heng Yu, Yuyang Ye, Linbo Zhu, Yu Su

Modeling Aspect Sentiment Coherency via Local Sentiment Aggregation (2024.findings-eacl)

Challenge:	Existing query augmentation methods face knowledge update lag and hallucinations in large language models (LLMs) Existing methods face two key challenges: (1) separation of query augmented and encoding tasks, which hinders information sharing and introduces cumulative errors; (2) difficulty of selecting optimal augmentation strategy for different scenarios.
Approach:	They propose a unified framework for query understanding in RAG that integrates internal and external knowledge to enhance query augmentation and encoding tasks.
Outcome:	The proposed framework outperforms traditional query augmentation methods in five knowledge-intensive benchmark tasks in both closed and open domain question answering.

Copied to clipboard

Decoding the Silent Majority: Inducing Belief Augmented Social Graph with Large Language Model for Response Forecasting (2023.emnlp-main)

Challenge:	Existing studies have not explored aspect sentiment coherency, including its implications in adversarial defense.
Approach:	They propose a local sentiment aggregation paradigm that models aspect sentiment coherency . they demonstrate the capability of LSA in adversarial defense .
Outcome:	The proposed model outperforms existing models and achieves state-of-the-art sentiment classification performance.

Copied to clipboard

Chenkai Sun, Jinning Li, Yi Fung, Hou Chan, Tarek Abdelzaher, ChengXiang Zhai, Heng Ji

ROMA: Real-time Omni-Multimodal Assistant with Interactive Streaming Understanding (2026.findings-acl)

Challenge:	Existing approaches to forecast news media responses have limited exploration of how to best process and utilize these important features.
Approach:	They propose a framework that leverages a large language model to induce a belief-centered graph on top of an existent social network, along with graph-based propagation to capture social dynamics.
Outcome:	The proposed framework surpasses state-of-the-art in experimental evaluations for both zero-shot and supervised settings, demonstrating its effectiveness in response forecasting.

Copied to clipboard

Tian Xueyun, Wei Li, Bingbing Xu, Heng Dong, Yuanzhuo Wang, Huawei Shen

Unveiling the Spectrum of Data Contamination in Language Model: A Survey from Detection to Remediation (2024.findings-acl)

Challenge:	Existing omni-multimodal large language models lack incomplete modality support or lack autonomous proactive monitoring.
Approach:	They propose a real-time omni-multimodal assistant for unified reactive and proactive interaction that decouples response initiation from generation to ensure precise triggering without task conflict.
Outcome:	The proposed model achieves state-of-the-art performance on proactive tasks while competing in reactive settings.

Copied to clipboard

Chunyuan Deng, Yilun Zhao, Yuzhao Heng, Yitong Li, Jiannan Cao, Xiangru Tang, Arman Cohan

GAIA: A Fine-grained Multimedia Knowledge Extraction System (2020.acl-demos)

Challenge:	Data contamination is a problem in Large language models due to the reliance on extensive internet-derived training corpora.
Approach:	They present a survey on the topic of data contamination in large language models.
Outcome:	The results of the first survey on data contamination in large language models provide a comprehensive guide for NLP researchers seeking a systematic understanding of the issue.

Copied to clipboard

Manling Li, Alireza Zareian, Ying Lin, Xiaoman Pan, Spencer Whitehead, Brian Chen, Bo Wu, Heng Ji, Shih-Fu Chang, Clare Voss, Daniel Napierski, Marjorie Freedman

Visual Data in NLP

Challenge:	Open source knowledge extraction tools are used for many real-world applications, but there is no comprehensive system for KE.
Approach:	They propose a multimedia knowledge extraction system that takes multimedia data from various sources and languages as input and creates a coherent, structured knowledge base.
Outcome:	The system achieves top performance at the recent NIST TAC SM-KBP2019 evaluation.

Copied to clipboard

Guangwei Zhang, Jianing Zhu, Cheng Qian, Neil Zhenqiang Gong, Rada Mihalcea, Zhaozhuo Xu, Jingrui He, Jiaqi W. Ma, Chaowei Xiao, Bo Li, Ahmed Abbasi, Dongwon Lee, Heng Ji, Denghui Zhang

Paraphrasing

Challenge:	Copyright Detective is the first interactive forensic system for detecting, analyzing, and visualizing potential copyright risks in LLM outputs.
Approach:	They propose a system that detects copyright infringements and visualizes them . they use content recall testing, paraphrase-level similarity analysis and persuasive jailbreak probing .
Outcome:	The proposed system detects, analyzes, and visualizes potential copyright risks in LLM outputs.

ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning (2021.acl-long)

Copied to clipboard

Yujia Qin, Yankai Lin, Ryuichi Takanobu, Zhiyuan Liu, Peng Li, Heng Ji, Minlie Huang, Maosong Sun, Jie Zhou

ViStruct: Visual Structural Knowledge Extraction via Curriculum Guided Code-Vision Representation (2023.emnlp-main)

Challenge:	Existing pre-training objectives do not explicitly model relational facts in text . Experimental results show that ERICA can improve typical PLMs on several language understanding tasks, including relation extraction, entity typing and question answering.
Approach:	They propose a contrastive learning framework ERICA to obtain a deep understanding of entities and relations in text.
Outcome:	The proposed framework can improve PLMs on several language understanding tasks, especially under low-resource settings.

Copied to clipboard

Yangyi Chen, Xingyao Wang, Manling Li, Derek Hoiem, Heng Ji

OpenPI-C: A Better Benchmark and Stronger Baseline for Open-Vocabulary State Tracking (2023.findings-acl)

Challenge:	State-of-the-art vision-language models have limited performance in structural knowledge extraction, such as relations between objects.
Approach:	They propose to leverage the inherent structure of programming language to depict visual structural information in a well-organized structured format.
Outcome:	The proposed framework improves visual structural knowledge extraction on visual structure prediction tasks.

Copied to clipboard

Xueqing Wu, Sha Li, Heng Ji

Text Clustering

Challenge:	Existing work on state tracking assumes both entities and state space are known, which limits their applicability.
Approach:	They propose a two-stage model that refines the state change prediction conditioned on entities predicted from the first stage.
Outcome:	The proposed model improves on the cleaned dataset and the evaluation metric on the proposed model.

Biomedical Event Extraction based on Knowledge-driven Tree-LSTM (N19-1)

Copied to clipboard

Diya Li, Lifu Huang, Heng Ji, Jiawei Han

Closing the Modality Reasoning Gap for Speech Large Language Models (2026.acl-long)

Challenge:	Biomedical event extraction requires domain-specific knowledge and deep understanding of complex contexts.
Approach:	They propose a knowledge base-driven tree-structured long short-term memory networks framework . tree-LSTM framework incorporates dependency structures and entity properties from ontologies .
Outcome:	The proposed framework is based on the BioNLP shared task with Genia dataset and achieves state-of-the-art results.

Copied to clipboard

Chaoren Wang, Heng Lu, Xueyao Zhang, Shujie Liu, Yan Lu, Jinyu Li, Zhizheng Wu

LoRAExit: Empowering Dynamic Modulation of LLMs in Resource-limited Settings using Low-rank Adapters (2024.findings-emnlp)

Challenge:	Recent advances in Speech Large Language Models have a modality reasoning gap that is not addressed by prior work.
Approach:	They propose a reinforcement-learning framework that aligns text-conditioned and speech-conditioned trajectories through an asymmetric reward design.
Outcome:	Experiments on MMSU and OBQA show that the proposed framework narrows the modality reasoning gap and achieves state-of-the-art performance among 7B-scale Speech LLMs.

Copied to clipboard

Jiacheng Liu, Peng Tang, Xiaofeng Hou, Chao Li, Pheng-Ann Heng

Responsible & Trustworthy NLP

Challenge:	Large Language Models (LLMs) have exhibited remarkable performance across various natural language processing tasks, but deployment on resource-limited settings remains a challenge.
Approach:	They propose a dynamic inference architecture that leverages low-rank adaptors for efficient deployment of LLMs.
Outcome:	The proposed architecture significantly improves performance when deployed on resource-limited settings.

Open-Domain Hierarchical Event Schema Induction by Incremental Prompting and Verification (2023.acl-long)

Copied to clipboard

Sha Li, Ruining Zhao, Manling Li, Heng Ji, Chris Callison-Burch, Jiawei Han

Low-Resource NLP

Challenge:	Recent methods for event schema induction use information extraction systems to construct event graph instances from documents . compared to the previous state-of-the-art closed-domain schema inducing model, human assessors were able to cover 10% more events when translating the schemas into coherent stories .
Approach:	They propose to treat event schemas as commonsense knowledge that can be derived from large language models.
Outcome:	The proposed method simplifies the schema induction process and improves readability.

Learning Shared Semantic Space for Speech-to-Text Translation (2021.findings-acl)

Copied to clipboard

Chi Han, Mingxuan Wang, Heng Ji, Lei Li

Multimedia Generative Script Learning for Task Planning (2023.findings-acl)

Challenge:	End-to-end speech translation (ST) has been treated as an independent task . however, the modality gap has rendered MT data and its end-to end models incompatible with their ST counterparts.
Approach:	They propose to bridge the representation gap between text and audio inputs by projecting audio and text features to a common semantic representation.
Outcome:	The proposed model improves performance on two ST benchmarks and achieves 27.1 BLEU on MuST-C EN-DE.

Copied to clipboard

Qingyun Wang, Manling Li, Hou Pong Chan, Lifu Huang, Julia Hockenmaier, Girish Chowdhary, Heng Ji

The Law of Knowledge Overshadowing: Towards Understanding, Predicting and Preventing LLM Hallucination (2025.findings-acl)

Challenge:	Goal-oriented generative script learning aims to generate subsequent steps to reach a specific goal . ability to capture historical states in visual modalities provides detailed information not covered by text .
Approach:	They propose a goal-oriented generative script learning task to generate subsequent steps by tracking historical states in both text and vision modalities.
Outcome:	The proposed task outperforms baselines in three aspects of the current task.

Copied to clipboard

Yuji Zhang, Sha Li, Cheng Qian, Jiateng Liu, Pengfei Yu, Chi Han, Yi R. Fung, Kathleen McKeown, ChengXiang Zhai, Manling Li, Heng Ji

Generative Models for Product Attribute Extraction (2023.emnlp-industry)

Challenge:	Hallucination is a persistent challenge in large language models where even with rigorous quality control, models often generate distorted facts.
Approach:	They propose a new framework to quantify factual hallucinations by modeling knowledge overshadowing.
Outcome:	The proposed framework improves model factuality on Overshadow (27.9%), MemoTrap (13.1%) and NQ-Swap (18.3%).

Copied to clipboard

Ansel Blume, Nasser Zalmout, Heng Ji, Xian Li

Tagging

Challenge:	generative models are used for product attribute extraction, a new field in information extraction and e-commerce.
Approach:	They analyze generative models for product attribute extraction and demonstrate their utility . they perform experiments on Amazon and MAVE product attribute datasets .
Outcome:	The proposed model can generate implicit attribute values, which state-of-the-art models are unable to extract.

Scaling Behaviors of LLM Reinforcement Learning Post-Training: An Empirical Study in Mathematical Reasoning (2026.acl-long)

Copied to clipboard

Zelin Tan, Hejia Geng, Xiaohang Yu, Mulei Zhang, Guancheng Wan, Yifan Zhou, Qiang He, Xiangyuan Xue, Heng Zhou, Yutao Fan, Zhong-Zhi Li, Zaibin Zhang, Guibin Zhang, Chen Zhang, Zhenfei Yin, Philip Torr, Lei Bai

Non-Sequential Graph Script Induction via Multimedia Grounding (2023.acl-long)

Challenge:	elucidating scaling laws for large language models (LLMs) during pre-training remains unexplored.
Approach:	They characterize how model scale, data, and compute interact during pre-training . they find that large models consistently demonstrate superior compute and data efficiency .
Outcome:	The proposed scaling laws offer practical guidance for scaling reasoning capabilities through reinforcement learning post-training.

Copied to clipboard

Yu Zhou, Sha Li, Manling Li, Xudong Lin, Shih-Fu Chang, Mohit Bansal, Heng Ji

Enhancing Knowledge Selection for Grounded Dialogues via Document Semantic Graphs (2022.naacl-main)

Challenge:	Existing scripts for everyday tasks are presented in a linear manner, which does not reflect the flexibility displayed by people executing tasks in real life.
Approach:	They propose to use loosely aligned videos to train a non-sequential graph script induction task by using a multimodal framework to ground procedural videos to WikiHow textual steps.
Outcome:	The proposed model outperforms the WikiHow linear baseline by 48.76% . it can predict future steps given a partial step sequence and generate explicit graph scripts .

Copied to clipboard

Sha Li, Mahdi Namazifar, Di Jin, Mohit Bansal, Heng Ji, Yang Liu, Dilek Hakkani-Tur

AGRaME: Any-Granularity Ranking with Multi-Vector Embeddings (2024.emnlp-main)

Challenge:	Existing conversation models treat knowledge selection as a sentence ranking problem where each sentence is handled individually, ignoring the internal semantic connection between sentences.
Approach:	They propose to automatically convert background knowledge documents into document semantic graphs and perform knowledge selection over such graphs.
Outcome:	The proposed model improves on the knowledge selection task and the response generation task on HollE and generalizes on unseen topics in WoW.

Copied to clipboard

Revanth Gangi Reddy, Omar Attia, Yunyao Li, Heng Ji, Saloni Potdar

Representation Learning

Challenge:	Existing ranking algorithms restrict granularity to full passages or require a specific dense index for each desired level of granules.
Approach:	They propose a multi-vector ranking approach that leverages multi-vctor embeddings to rank at varying levels of granularity while maintaining encoding at a single (coarser) level of grail.
Outcome:	The proposed method surpasses prompt-driven citation generation by incorporating proposition-level ranking to post-hoc citation addition.

Multilingual Entity, Relation, Event and Human Value Extraction (N19-4)

Copied to clipboard

Manling Li, Ying Lin, Joseph Hoover, Spencer Whitehead, Clare Voss, Morteza Dehghani, Heng Ji

RADAR: Enhancing Radiology Report Generation with Supplementary Knowledge Injection (2025.acl-long)

Challenge:	Existing systems that extract knowledge elements from multiple languages and documents do not aggregate knowledge from multiple documents and languages.
Approach:	They propose a multilingual knowledge extraction system that performs entity discovery and linking, relation extraction, event extraction, and coreference.
Outcome:	The proposed system performs entity discovery and linking, relation extraction, event extraction, and coreference.

Copied to clipboard

Wenjun Hou, Yi Cheng, Kaishuai Xu, Heng Li, Yan Hu, Wenjie Li, Jiang Liu

Open-Vocabulary Argument Role Prediction For Event Extraction (2022.findings-emnlp)

Challenge:	Existing approaches to enhance radiology report generation overlook the knowledge already embedded within the models, leading to redundant information integration.
Approach:	They propose a framework for enhancing radiology report generation with supplementary knowledge injection that leverages both internal and external knowledge.
Outcome:	Extensive experiments on MIMIC-CXR, CheXpert-Plus, and IU X-ray show that the proposed model outperforms state-of-the-art LLMs in both language quality and clinical accuracy.

Copied to clipboard

Yizhu Jiao, Sha Li, Yiqing Xie, Ming Zhong, Heng Ji, Jiawei Han