Papers by Wang Heng

GL-GAN: Perceiving and Integrating Global and Local Styles for Handwritten Text Generation with Mamba (2025.coling-main)

Challenge:	Recent studies show that supervised fine-tuning (SFT) is a common approach for reasoning in large language models.
Approach:	They propose to use supervised fine-tuning (SFT) on chain-of-thought trajectories demonstrations . they find that incorporating negative traxories yields substantial OOD generalization gains .
Outcome:	The proposed scheme yields 5.51% OOD gain over positive-only training.

Copied to clipboard

Yiming Wang, Hongxi Wei, Heng Wang, Shiwen Sun, Chao He

From Insight to Action: A Novel Framework for Interpretability-Guided Data Selection in Large Language Models (2026.acl-long)

Challenge:	Existing models lack the ability to perceive and integrate handwriting styles, which affects the realism of the synthesized samples.
Approach:	They propose a Hybrid Style Encoder that captures global and local styles and integrates them into a Dynamic Feature Enhancement Module (DFEM).
Outcome:	The proposed model outperforms state-of-the-art models on two widely used handwriting datasets and can provide training data for handwritten text recognition and signature verification.

Copied to clipboard

Ling Shi, Xinwei Wu, Xiaohu Zhao, Hao Wang, Heng Liu, Yangyang Liu, Linlong Xu, Longyue Wang, Deyi Xiong, Weihua Luo

LM-Infinite: Zero-Shot Extreme Length Generalization for Large Language Models (2024.naacl-long)

Challenge:	Recent research in mechanistic interpretability has revealed that Large Language models contain disentangled, human-understandable components.
Approach:	They propose a framework that first identifies causal task features through frequency recall and interventional filtering, then selects “Feature-Resonant Data” that maximally activates task features for fine-tuning.
Outcome:	The proposed framework outperforms existing models on mathematical reasoning, summarization, and translation tasks while using only 50% of the data.

Copied to clipboard

Chi Han, Qifan Wang, Hao Peng, Wenhan Xiong, Yu Chen, Heng Ji, Sinong Wang

LiveCANNBench: Benchmark SWE AI Coding for Ascend CANN (2026.findings-acl)

Challenge:	Currently, large language models (LLMs) train on short text segments due to the computational overhead quadratic in the input lengths of their Transformer architectures.
Approach:	They propose a method that allows LLMs pre-trained with 2K or 4K-long segments to generalize to up to 200M length inputs while retaining perplexity.
Outcome:	The proposed method achieves 2.7 decoding speed up and 7.5 memory saving over the original model.

Copied to clipboard

Sijie Wang, Kai Zhao, Wee Peng Tay, Shuo Zhang, Chengwen Liu, Quanjiang Guo, Ren Junhao, Xin Li, Heng Lian, Jingdi Lei, Rui She, Huacan Wang, Ronghao Chen

Challenge:	Recent advances in agents have enabled multi-file, multi-language, and dependency-aware AI coding.
Approach:	They propose an SWE-level benchmark for AI coding in the Huawei Ascend CANN software stack.
Outcome:	The proposed benchmark is constructed from real-world CANN repositories and consists of over 400 task instances spanning multiple file, multi-language, and execution-aware coding challenges.

KIA: Knowledge-Guided Implicit Vision-Language Alignment for Chest X-Ray Report Generation (2025.coling-main)

Copied to clipboard

Heng Yin, Shanlin Zhou, Pandong Wang, Zirui Wu, Yongtao Hao

Entailment-Preserving First-order Logic Representations in Natural Language Entailment (2025.acl-long)

Challenge:	Existing reports on medical images and reports lack fine-grained cross-modal interaction, leading to insufficient understanding of detailed information.
Approach:	They propose a framework for establishing cross-modal semantic alignment in radiology report pairs using knowledge-guided implicit vision-language alignment.
Outcome:	KIA improves understanding of medical images and reports by incorporating medical knowledge to enhance pathological observation and anatomical landm.

Copied to clipboard

Jinu Lee, Qi Liu, Runzhi Ma, Vincent Han, Ziqi Wang, Heng Ji, Julia Hockenmaier

Representation Learning

Challenge:	First-order logic (FOL) is often used to represent logical entailment, but determining natural language (NL) enanglement using FOL remains a challenge.
Approach:	They propose an Entailment-Preserving FOL representations task and a method which trains an NL-to-FOL translator by using the natural language entailment labels as verifiable rewards.
Outcome:	The proposed method achieves 1.8–2.7% improvement in EPR and 17.4–20.6% increase in E PR@16 compared to baselines in three datasets.

The Future is not One-dimensional: Complex Event Schema Induction by Graph Modeling for Event Prediction (2021.emnlp-main)

Copied to clipboard

Manling Li, Sha Li, Zhenhailong Wang, Lifu Huang, Kyunghyun Cho, Heng Ji, Jiawei Han, Clare Voss

Democratizing LLMs: An Exploration of Cost-Performance Trade-offs in Self-Refined Open-Source Models (2023.findings-emnlp)

Challenge:	Event schemas encode knowledge of stereotypical structures of events and their connections . previous work on event schema induction focuses on atomic events or linear temporal sequences .
Approach:	They propose a Temporal Complex Event Schema: a graph-based schema representation that encompasses events, arguments, temporal connections and argument relations.
Outcome:	The proposed model outperforms existing models on HITS@1 by 17.8%.

Copied to clipboard

Sumuk Shashidhar, Abhinav Chinta, Vaibhav Sahai, Zhenhailong Wang, Heng Ji

Language Model Pre-Training with Sparse Latent Typing (2022.emnlp-main)

Challenge:	Large Language Models (LLMs) have been experiencing seismic growth in size and capabilities, radically transforming the field of NLP.
Approach:	They propose a generalized variant of iterative self-critique and self-refinement devoid of external influence and a ranking metric to find the optimal model for a given task considering refined performance and cost.
Outcome:	The proposed model improves 8.2% from baseline and even with extremely small memory footprints, outperforms ChatGPT post-refinement.

Copied to clipboard

Liliang Ren, Zixuan Zhang, Han Wang, Clare Voss, ChengXiang Zhai, Heng Ji

Efficient Attentions for Long Document Summarization (2021.naacl-main)

Challenge:	Modern large-scale Pre-trained Language Models focus on text reconstruction, but have not sought to learn latent-level interpretable representations of sentences.
Approach:	They propose a new pre-training objective that enables the model to learn latent types . the objective allows the model a self-supervised way to extract sentence-level keywords .
Outcome:	The proposed model learns interpretable latent type categories without external knowledge and improves downstream tasks.

Copied to clipboard

Luyang Huang, Shuyang Cao, Nikolaus Parulian, Heng Ji, Lu Wang

From Word to World: Can Large Language Models be Implicit Text-based World Models? (2026.acl-long)

Challenge:	Existing models that use full attentions have quadratic computational and memory complexities, and are too costly for long documents.
Approach:	They propose an efficient encoder-decoder attention with head-wise positional strides to effectively pinpoint salient information from the source.
Outcome:	The proposed model can process ten times more tokens than current models that use full attentions.

Copied to clipboard

Yixia Li, Hongru Wang, Jiahao Qiu, Zhenfei Yin, Dongdong Zhang, Cheng Qian, Zeping Li, Xiaoteng Ma, Guanhua Chen, Heng Ji

Making Pre-trained Language Models both Task-solvers and Self-calibrators (2023.findings-acl)

Challenge:	Agentic learning increasingly hinges on interaction, yet real-world experience is expensive, limited, and often irreversible at inference time.
Approach:	They propose a framework that reframes language modeling as next-state prediction under interaction.
Outcome:	The proposed framework evaluates world models in text-based environments . it shows that sufficiently trained models capture coherent environment dynamics .

Copied to clipboard

Yangyi Chen, Xingyao Wang, Heng Ji

Reaction Miner: An Integrated System for Chemical Reaction Extraction from Textual Data (2023.emnlp-demo)

Challenge:	Existing work shows that pre-trained language models can be effective for high-stake applications, but they become overconfident in their wrong predictions.
Approach:	They propose to use extra data to train pre-trained language models to effectively utilize training samples to make them both task-solvers and self-calibrators.
Outcome:	The proposed method can be used in three downstream applications, including selective classification, adversarial defense, and model cascading.

Copied to clipboard

Ming Zhong, Siru Ouyang, Yizhu Jiao, Priyanka Kargupta, Leo Luo, Yanzhen Shen, Bobby Zhou, Xianrui Zhong, Xuan Liu, Hongxiang Li, Jinfeng Xiao, Minhao Jiang, Vivian Hu, Xuan Wang, Heng Ji, Martin Burke, Huimin Zhao, Jiawei Han

LORE: Continual Logit Rewriting Fosters Faithful Generation (2025.findings-emnlp)

Challenge:	Reaction Miner is a system designed to extract chemical reactions from raw scientific PDFs.
Approach:	They propose a system that extracts chemical reactions directly from raw scientific PDFs.
Outcome:	The proposed system can extract chemical reactions from raw scientific PDFs.

Copied to clipboard

Charles Yu, Qingyun Wang, Yuting Hu, Jinjun Xiong, Heng Ji

Multimodal Causal Reasoning Benchmark: Challenging Multimodal Large Language Models to Discern Causal Links Across Modalities (2025.findings-acl)

Challenge:	Using aspect-oriented summarization as a case study, we propose LOgit REwriting, a new controlled generation paradigm which can be faithful to external knowledge and to the LLM’s intentions.
Approach:	They propose a controlled generation paradigm which can be faithful to external knowledge and to the LLM's intentions.
Outcome:	The proposed paradigm can be faithful to external knowledge and to the LLM's intentions while balancing that with accuracy.

Copied to clipboard

Zhiyuan Li, Heng Wang, Dongnan Liu, Chaoyi Zhang, Ao Ma, Jieting Long, Weidong Cai

NewsClaims: A New Benchmark for Claim Detection from News with Attribute Knowledge (2022.emnlp-main)

Challenge:	Existing MLLMs lack robustness in multimodal causal reasoning compared to their performance in textual settings.
Approach:	They propose a novel multimodal chain-of-thought (CoT) reasoning benchmark that leverages siamese images and text pairs to challenge MLLMs.
Outcome:	The proposed benchmark leverages siamese images and text pairs to challenge MLLMs.

Copied to clipboard

Revanth Gangi Reddy, Sai Chetan Chinthakindi, Zhenhailong Wang, Yi Fung, Kathryn Conger, Ahmed ELsayed, Martha Palmer, Preslav Nakov, Eduard Hovy, Kevin Small, Heng Ji

Detecting Spoilers in Movie Reviews with External Movie Knowledge and User Networks (2023.emnlp-main)

Challenge:	Current claims detection methods focus on sentence analysis, ignoring other attributes . a key element of identifying misinformation is detecting the claims and the arguments that have been presented.
Approach:	They propose a benchmark for attribute-aware claim detection in the news domain . they extend the problem to include extraction of additional attributes related to each claim .
Outcome:	The proposed system performs well on the test, but human performance is still poor.

Copied to clipboard

Heng Wang, Wenqian Zhang, Yuyang Bai, Zhaoxuan Tan, Shangbin Feng, Qinghua Zheng, Minnan Luo

Knowledge Representation

Challenge:	Existing models focus on the textual content of the review, while spoiler detection requires putting the review into the context of facts and knowledge regarding movies.
Approach:	They propose a network-based spoiler detection model that takes into account external knowledge about movies and user activities on movie review platforms.
Outcome:	The proposed model takes into account external knowledge about movies and user activities on movie review platforms while incorporating user networks.

Mitigating the Alignment Tax of RLHF (2024.emnlp-main)

Copied to clipboard

Yong Lin, Hangyu Lin, Wei Xiong, Shizhe Diao, Jianmeng Liu, Jipeng Zhang, Rui Pan, Haoxiang Wang, Wenbin Hu, Hanning Zhang, Hanze Dong, Renjie Pi, Han Zhao, Nan Jiang, Heng Ji, Yuan Yao, Tong Zhang

Zero- and Few-Shot Event Detection via Prompt-Based Meta Learning (2023.acl-long)

Challenge:	Large Language Models (LLMs) acquire a wide range of abilities during pre-training, but aligning LLMs under Reinforcement Learning with Human Feedback (RLHF) can lead to forgetting pretrained abilities, which is also known as the alignment tax.
Approach:	They propose to use a model averaging technique to find the most powerful alignment-forging Pareto front among RLHF algorithms.
Outcome:	The proposed method achieves the strongest alignment-forging Pareto front among competing methods.

Copied to clipboard

Zhenrui Yue, Huimin Zeng, Mengfei Lan, Heng Ji, Dong Wang

DeepMaven: Deep Question Answering on Long-Distance Movie/TV Show Videos with Multimedia Knowledge Extraction and Synthesis (2023.eacl-main)

Challenge:	Existing methods for event detection often fail to detect unseen or rare events due to the lack of domain knowledge.
Approach:	They propose a meta learning-based framework for zero-shot event detection that uses a prompt-based prompt and a trigger-aware soft verbalizer to efficiently project output to unseen tasks.
Outcome:	The proposed framework performs state-of-the-art in zero-shot and few-shot scenarios on benchmark datasets FewEvent and MAVEN.

Copied to clipboard

Yi Fung, Han Wang, Tong Wang, Ali Kebarighotbi, Mohit Bansal, Heng Ji, Prem Natarajan

SciMON: Scientific Inspiration Machines Optimized for Novelty (2024.acl-long)

Challenge:	Long video content understanding poses a challenging set of research questions as it involves long-distance, cross-media reasoning and knowledge awareness.
Approach:	They propose a framework which extracts events, entities, and relations from the rich multimedia content in long videos to pre-construct movie knowledge graphs.
Outcome:	The proposed framework performs competitively for both the new DeepMovieQA and the pre-existing MovieQA dataset.

Copied to clipboard

Qingyun Wang, Doug Downey, Heng Ji, Tom Hope

MIMIR: A Customizable Agent Tuning Platform for Enhanced Scientific Applications (2024.emnlp-demo)

Challenge:	Existing literature-based hypothesis generation models focus on binary link prediction, limiting expressivity of hypotheses.
Approach:	They propose a framework that uses literature-based hypothesis generation as input . they use literature-derived literature as background and output natural language ideas .
Outcome:	The proposed model improves the ability of language models to generate new scientific directions grounded in literature.

Copied to clipboard

Xiangru Tang, Chunyuan Deng, Hanminwang Hanminwang, Haoran Wang, Yilun Zhao, Wenqi Shi, Yi Fung, Wangchunshu Zhou, Jiannan Cao, Heng Ji, Arman Cohan, Mark Gerstein

The Shifted and The Overlooked: A Task-oriented Investigation of User-GPT Interactions (2023.emnlp-main)

Challenge:	Large language models (LLMs) have evolved into interactive agents capable of planning, tool use, and task execution across various tasks.
Approach:	They propose a platform that leverages large language models to generate agent-tuning data for fine-tuneing smaller, specialized models.
Outcome:	MIMIR enables large models to simulate various roles and create interaction data, which can then be used to fine-tune open-source models like LLaMA2.

Copied to clipboard

Siru Ouyang, Shuohang Wang, Yang Liu, Ming Zhong, Yizhu Jiao, Dan Iter, Reid Pryzant, Chenguang Zhu, Heng Ji, Jiawei Han

SYNTHIA: Novel Concept Design with Affordance Composition (2025.acl-long)

Challenge:	Recent advances in large language models (LLMs) have produced models that exhibit remarkable performance across a variety of NLP tasks.
Approach:	They analyze a large-scale collection of user-GPT conversations to identify a significant gap between academic research in NLP and the needs of real-world NLP applications.
Outcome:	The proposed model outperforms existing models in a large-scale collection of user-GPT conversations and identifies a significant gap between the tasks that users frequently request from LLMs and the tasks commonly studied in academic research.

Copied to clipboard

Hyeonjeong Ha, Xiaomeng Jin, Jeonghwan Kim, Jiateng Liu, Zhenhailong Wang, Khanh Duy Nguyen, Ansel Blume, Nanyun Peng, Kai-Wei Chang, Heng Ji

M2PO: Multi-Perspective Multi-Pair Preference Optimization for Machine Translation (2026.acl-long)

Challenge:	Existing studies on concept design using text-to-image models have enabled rapid ideation of novel visual concepts.
Approach:	They propose a framework for generating novel, functionally coherent designs based on desired affordances by decomposing concepts into parts and affordance . they also develop a curriculum learning scheme that fine-tunes T2I models to progressively learn affordance composition while maintaining visual novelty.
Outcome:	The proposed framework outperforms state-of-the-art models for novelty and functional coherence in human evaluation.

Copied to clipboard

Hao Wang, Linlong Xu, Heng Liu, Yangyang Liu, Xiaohu Zhao, Bo Zeng, Liangying Shao, Yichen Dong, Xinwei Wu, Jiang Zhou, Tianyu Dong, Xiangxiang Zeng, Longyue Wang, Weihua Luo

Machine Translation

Challenge:	prevailing methods for machine translation are often hindered by misleading reward signals.
Approach:	They propose a framework that aligns large language models to human preferences . they propose 'M2PO' to correct the bias towards partial errors .
Outcome:	The proposed framework outperforms open-source models and achieves parity with proprietary models.

Can Language Models Follow Multiple Turns of Entangled Instructions? (2025.findings-emnlp)

Copied to clipboard

Chi Han, Xin Liu, Haodong Wang, Shiyang Li, Jingfeng Yang, Haoming Jiang, Zhengyang Wang, Qingyu Yin, Liang Qiu, Changlong Yu, Yifan Gao, Zheng Li, Bing Yin, Jingbo Shang, Heng Ji

Knowledge-Enriched Natural Language Generation (2021.emnlp-tutorials)

Challenge:	Despite of significant achievements in improving instruction-following capabilities of large language models, the ability to process multiple potentially entangled or conflicting instructions remains a considerable challenge.
Approach:	They construct multi-turn instruction with 1.1K high-quality multi-turned conversations using the human-in-the-loop approach and examine their capabilities.
Outcome:	The proposed model shows that it is difficult to integrate multiple turns and balance competing objectives when instructions intersect or conflict.

Copied to clipboard

Wenhao Yu, Meng Jiang, Zhiting Hu, Qingyun Wang, Heng Ji, Nazneen Rajani

EMCompress: Video-LLMs with Endomorphic Multimodal Compression (2026.findings-acl)

Challenge:	Knowledge-enriched text generation poses unique challenges in modeling and learning . a roadmap will outline the state-of-the-art methods to tackle these challenges .
Approach:	They propose a roadmap to tackle the challenges of knowledge-enriched text generation . they will dive deep into various technical components to illustrate how to represent knowledge .
Outcome:	This tutorial outlines the state-of-the-art methods to tackle the problem . it aims to show how to represent knowledge, feed knowledge into a generation model, evaluate results .

Copied to clipboard

Zheyu Fan, Jiateng Liu, Yuji Zhang, Zihan Wang, Yi R. Fung, Manling Li, Heng Ji

PALT: Parameter-Lite Transfer of Language Models for Knowledge Graph Completion (2022.findings-emnlp)

Challenge:	Static, sparse frame sampling either dilutes evidence across task-irrelevant segments at significant cost or misses fine-grained temporal semantics altogether.
Approach:	They propose a novel task that compresses multimodal input while preserving answer invariance across reasonable downstream models.
Outcome:	The proposed task surpasses prior methods by 0.40 F-1 with competitive query rewriting.

Copied to clipboard

Jianhao Shen, Chenguang Wang, Ye Yuan, Jiawei Han, Heng Ji, Koushik Sen, Ming Zhang, Dawn Song

FlashAudio: Rectified Flow for Fast and High-Fidelity Text-to-Audio Generation (2025.acl-long)

Challenge:	Pretrained language models (LMs) are a powerful transfer learning approach for knowledge graph (KG) completion.
Approach:	They propose a parameter-lite transfer learning approach for pretrained language models for knowledge graph (KG) completion.
Outcome:	The proposed model outperforms the state-of-the-art models on a knowledge graph completion benchmark by tuning 1% of the parameters.

Copied to clipboard

Huadai Liu, Jialei Wang, Rongjie Huang, Yang Liu, Heng Lu, Zhou Zhao, Wei Xue

Code4Struct: Code Generation for Few-Shot Event Structure Prediction (2023.acl-long)

Challenge:	Recent advances in latent diffusion models (LDMs) have markedly enhanced text-to-audio generation, yet their iterative sampling processes impose substantial computational demands, limiting practical deployment.
Approach:	They propose to learn straight flow for fast simulation by using flashAudio with rectified flows and immiscible flow to minimize the total distance of data-noise pairs in a batch vias assignment.
Outcome:	The proposed method can learn straight flow for fast simulations and reduce noise distribution.

Copied to clipboard

Xingyao Wang, Sha Li, Heng Ji

Enhancing Advanced Visual Reasoning Ability of Large Language Models (2024.emnlp-main)

Challenge:	Large Language Models (LLMs) trained on a mixture of text and code have demonstrated impressive capability in translating natural language (NL) into structured code.
Approach:	They propose to use programming language (PL) inheritance and type annotations to translate text into code to tackle structured prediction tasks.
Outcome:	The proposed model outperforms existing models on 20-shot data by 29.5% absolute F1.

Copied to clipboard

Zhiyuan Li, Dongnan Liu, Chaoyi Zhang, Heng Wang, Tengfei Xue, Weidong Cai

RESIN: A Dockerized Schema-Guided Cross-document Cross-lingual Cross-media Information Extraction and Event Tracking System (2021.naacl-demos)

Challenge:	Recent advances in Vision-Language (VL) research have sparked new benchmarks for complex visual reasoning, challenging models’ advanced reasoning ability.
Approach:	They propose a novel multi-modal in-context learning methodology to enhance LLMs’ contextual understanding and reasoning.
Outcome:	The proposed model achieves SOTA performance among all visual reasoning tasks and achieves a 'higher level of accuracy' than previous models.

Copied to clipboard

Haoyang Wen, Ying Lin, Tuan Lai, Xiaoman Pan, Sha Li, Xudong Lin, Ben Zhou, Manling Li, Haoyu Wang, Hongming Zhang, Xiaodong Yu, Alexander Dong, Zhenhailong Wang, Yi Fung, Piyush Mishra, Qing Lyu, Dídac Surís, Brian Chen, Susan Windisch Brown, Martha Palmer, Chris Callison-Burch, Carl Vondrick, Jiawei Han, Dan Roth, Shih-Fu Chang, Heng Ji

Visual Data in NLP

Challenge:	We present a new information extraction system that can construct temporal event graphs from news documents.
Approach:	They propose a temporal event graph extraction system that can extract news documents . they extend the system from sentence-level event extraction to cross-document cross-media event extraction .
Outcome:	The proposed system can extract temporal event graphs from news documents in multiple languages and multiple data modalities.

Schema-Guided Culture-Aware Complex Event Simulation with Multi-Agent Role-Play (2024.emnlp-demo)

Copied to clipboard

Sha Li, Revanth Gangi Reddy, Khanh Nguyen, Qingyun Wang, Yi Fung, Chi Han, Jiawei Han, Kartik Natarajan, Clare Voss, Heng Ji

Challenge:	Complex news events require swift responses from government and society, authors say . relying on historical events to project the future is insufficient, they say - a simulator for complex news events is needed .
Approach:	They propose a controllable complex news event simulator guided by event schema and user-provided assumptions . they incorporate a geo-diverse commonsense and cultural norm-aware knowledge enhancement component .
Outcome:	The proposed simulator achieves higher coherence and appropriateness than existing models.

Specialization without Sparsity: Efficient and Expressive Split-Path Experts for LLM Fine-Tuning (2026.findings-acl)

Copied to clipboard

Ganghao Liu, Qin Zhou, Zhe Wang, Xuehan Lu, Haihua Huang, Yunfei Tong, Heng Tian

RESIN-11: Schema-guided Event Prediction for 11 Newsworthy Scenarios (2022.naacl-demo)

Challenge:	Parameter-efficient fine-tuning (PEFT) is a low-cost alternative to full fine-timing due to the massive overhead.
Approach:	They propose a Mixture-of-Experts approach that enhances specialization while maintaining low resource overhead.
Outcome:	The proposed approach outperforms or matches state-of-the-art methods on GLUE, GSM8K, MBPP, and a text rewriting task from SmolTalk.

Copied to clipboard

Xinya Du, Zixuan Zhang, Sha Li, Pengfei Yu, Hongwei Wang, Tuan Lai, Xudong Lin, Ziqi Wang, Iris Liu, Ben Zhou, Haoyang Wen, Manling Li, Darryl Hannan, Jie Lei, Hyounghun Kim, Rotem Dror, Haoyu Wang, Michael Regan, Qi Zeng, Qing Lyu, Charles Yu, Carl Edwards, Xiaomeng Jin, Yizhu Jiao, Ghazaleh Kazeminejad, Zhenhailong Wang, Chris Callison-Burch, Mohit Bansal, Carl Vondrick, Jiawei Han, Dan Roth, Shih-Fu Chang, Martha Palmer, Heng Ji

Improving Consistency for Text Summarization with Energy Functions (2023.findings-emnlp)

Challenge:	Existing methods for event prediction are incomplete and noisy.
Approach:	They propose to use news-related event schemas to extract newsworthy events . they build a demo website and include a video demonstrating the framework .
Outcome:	The proposed framework can be applied to a wide variety of newsworthy scenarios.

Copied to clipboard

Qi Zeng, Qingyu Yin, Zheng Li, Yifan Gao, Sreyashi Nag, Zhengyang Wang, Bing Yin, Heng Ji, Chao Zhang

Summarization

Challenge:	Current abstractive summarization models generate inconsistent content due to the inherently noisy dataset and the discrepancy between maximum likelihood estimation based training objectives and consistency measurements.
Approach:	They propose a new consistency taxonomy that categorizes inconsistent content into faithfulness, factuality, and self-supportiveness.
Outcome:	Experiments on XSUM and CNN/DM datasets show that EnergySum mitigates the trade-off between accuracy and consistency.

From Specific-MLLMs to Omni-MLLMs: A Survey on MLLMs Aligned with Multi-modalities (2025.findings-acl)

Copied to clipboard

Shixin Jiang, Jiafeng Liang, Jiyuan Wang, Xuan Dong, Heng Chang, Weijiang Yu, Jinhua Du, Ming Liu, Bing Qin

SMART: Self-Aware Agent for Tool Overuse Mitigation (2025.findings-acl)

Challenge:	MLLMs are able to integrate multiple modalities into a single model to tackle complex tasks in real-world scenarios.
Approach:	They propose a comprehensive survey of Omni-MLLMs to address the challenges and opportunities of multimodal modeling.
Outcome:	The proposed model can integrate multiple modalities into a single model and provide novel perspectives.

Copied to clipboard

Cheng Qian, Emre Can Acikgoz, Hongru Wang, Xiusi Chen, Avirup Sil, Dilek Hakkani-Tür, Gokhan Tur, Heng Ji

Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration (2024.naacl-long)

Challenge:	Current Large Language Models (LLMs) lack self-awareness to balance reasoning and tool use, increasing computational overhead.
Approach:	They propose a paradigm that enhances an agent’s self-awareness to optimize task handling and reduce tool overuse.
Outcome:	The proposed model reduces tool use by 24% while improving performance by over 37%.

Copied to clipboard

Zhenhailong Wang, Shaoguang Mao, Wenshan Wu, Tao Ge, Furu Wei, Heng Ji

Infogent: An Agent-Based Framework for Web Information Aggregation (2025.findings-naacl)

Challenge:	Existing work on LLMs that only enhance reasoning abilities, but which lack factual hallucination and slow-thinking capabilities, argues that SPP is a cognitive synergist.
Approach:	They propose a Solo Performance Prompting (SPP) that transforms a single LLM into a cognitive synergist by engaging in multi-turn self-collaboration with multiple personas.
Outcome:	The proposed model reduces factual hallucination and maintains strong reasoning abilities on three challenging tasks .

Copied to clipboard

Revanth Gangi Reddy, Sagnik Mukherjee, Jeonghwan Kim, Zhenhailong Wang, Dilek Hakkani-Tür, Heng Ji

COVID-19 Literature Knowledge Graph Construction and Drug Repurposing Report Generation (2021.naacl-demos)

Challenge:	Existing web navigation tasks evaluate web agents on task completion basis . however, information aggregation tasks have received relatively little attention .
Approach:	They propose a web navigation framework that uses three components for web information aggregation.
Outcome:	The proposed framework beats existing SOTA search framework by 7% under Direct API-Driven Access on FRAMES and improves over an existing information-seeking web agent by 4.3% under Interactive Visual Access on AssistantBench.

Copied to clipboard

Qingyun Wang, Manling Li, Xuan Wang, Nikolaus Parulian, Guangxing Han, Jiawei Ma, Jingxuan Tu, Ying Lin, Ranran Haoran Zhang, Weili Liu, Aabhas Chauhan, Yingjun Guan, Bangzheng Li, Ruisong Li, Xiangchen Song, Yi Fung, Heng Ji, Jiawei Han, Shih-Fu Chang, James Pustejovsky, Jasmine Rah, David Liem, Ahmed ELsayed, Martha Palmer, Clare Voss, Cynthia Schneider, Boyan Onyshkevych

HyperAdaLoRA: Accelerating LoRA Rank Allocation During Training via Hypernetworks without Sacrificing Performance (2026.findings-acl)

Challenge:	a new framework to digest relevant biomedical knowledge is needed to combat COVID-19 . quantity of research results is a bottleneck, and false information promoted in publications .
Approach:	a team of researchers has developed a framework to extract multimedia knowledge elements from scientific literature to combat COVID-19.
Outcome:	a new framework extracts fine-grained multimedia knowledge elements from scientific literature . it provides detailed contextual sentences, subfigures, and knowledge subgraphs as evidence . the framework is based on a case study of drug repurposing .

Copied to clipboard

Hao Zhang, Zhenjia Li, Yifan Gao, Xi Xiao, Heng Zhang, Shuyang Zhang, null Xiaoxincc, Bo Huang, Yuhang Wu, Tianyang Wang, Hao Xu

Zemi: Learning Zero-Shot Semi-Parametric Language Models from Multiple Tasks (2023.findings-acl)

Challenge:	Low-Rank Adaptation (LoRA) assumes a uniform rank r for each incremental matrix, not accounting for the varying significance of weight matrices across modules and layers.
Approach:	They propose a framework that allows for faster convergence of low-rank adaptive models . they use a hypernetwork to prune the outputs of the hypernetworks to generate parameters .
Outcome:	The proposed framework accelerates convergence of AdaLoRA by leveraging a hypernetwork.

Copied to clipboard

Zhenhailong Wang, Xiaoman Pan, Dian Yu, Dong Yu, Jianshu Chen, Heng Ji

COVID-19 Claim Radar: A Structured Claim Extraction and Tracking System (2022.acl-demo)

Challenge:	Existing semi-parametric language models lack the capacity to perform zero-shot tasks . large language models have shown impressive zero-shoot ability, but huge model size costs . semi-parametric language model can be used to augment a smaller language model with retrieved background knowledge .
Approach:	They propose a semi-parametric language model for zero-shot task generalization that augments a smaller language model with retrieved related background knowledge.
Outcome:	The proposed model outperforms T0-3B by 16% across seven diverse evaluation tasks while being 3.8x smaller in scale.

Copied to clipboard

Manling Li, Revanth Gangi Reddy, Ziqi Wang, Yi-shyuan Chiang, Tuan Lai, Pengfei Yu, Zixuan Zhang, Heng Ji

EventKE: Event-Enhanced Knowledge Graph Embedding (2021.findings-emnlp)

Challenge:	a new system extracts supporting and refuting claims from COVID-19 related news . the system is publicly available at GitHub and DockerHub, with complete documentation.
Approach:	They propose a COVID-19 Claim Radar system that extracts supporting and refuting claims . the system leverages Wikidata as the hub to consolidate coreferential knowledge elements .
Outcome:	The system extracts supporting and refuting claims from COVID-19 pandemic information . it leverages Wikidata as the hub to merge coreferential knowledge elements .

Copied to clipboard

Zixuan Zhang, Hongwei Wang, Han Zhao, Hanghang Tong, Heng Ji

SciVerse: Unveiling the Knowledge Comprehension and Visual Reasoning of LMMs on Multi-modal Scientific Problems (2025.findings-acl)

Challenge:	Experimental results show that events can greatly improve the quality of KG embeddings on multiple downstream tasks.
Approach:	They propose an event-enhanced KG embedding model that incorporates events into KGs . they first incorporate event nodes by building a heterogeneous network with event argument links .
Outcome:	The proposed model incorporates event nodes into the original knowledge graphs . it can be used to fuse event information into the KG embeddings on multiple tasks .

Copied to clipboard

Ziyu Guo, Renrui Zhang, Hao Chen, Jialin Gao, Dongzhi Jiang, Jiaze Wang, Pheng-Ann Heng

R-Tuning: Instructing Large Language Models to Say ‘I Don’t Know’ (2024.naacl-long)

Challenge:	SciVerse is a multi-modal scientific evaluation benchmark to assess large multi-models . it examines the scientific knowledge comprehension, multi-mod content interpretation and Chain-of-Thought reasoning . authors examine the scientific proficiency of LMMs in scientific domains based on their work .
Approach:	They propose a multi-modal scientific evaluation benchmark to thoroughly assess Large Multi-modal Models across 5,735 test instances in five different versions.
Outcome:	The proposed evaluation reveals critical limitations in LMMs' scientific proficiency and provides new insights into future developments.

Copied to clipboard

Hanning Zhang, Shizhe Diao, Yong Lin, Yi Fung, Qing Lian, Xingyao Wang, Yangyi Chen, Heng Ji, Tong Zhang

VReST: Enhancing Reasoning in Large Vision-Language Models through Tree Search and Self-Reward Mechanism (2025.acl-long)

Challenge:	Existing methods for instruction tuning force the model to complete a sentence no matter whether it knows the knowledge or not.
Approach:	They propose a new approach to tuning large language models to refrain from answering questions beyond its parametric knowledge by identifying the disparity in parametric and parametric information.
Outcome:	The proposed approach improves a model’s ability to answer known questions and refrain from answering unknown questions.

Copied to clipboard

Congzhi Zhang, Jiawei Peng, Zhenglin Wang, Yilong Lai, Haowen Sun, Heng Chang, Fei Ma, Weijiang Yu

Paper Abstract Writing through Editing Mechanism (P18-2)

Challenge:	Large Vision-Language Models (LVLMs) have shown exceptional performance in multimodal tasks, but their effectiveness in complex visual reasoning is constrained.
Approach:	They propose a training-free approach that enhances Reasoning in Large Vision-Language Models . they propose integrating Monte Carlo Tree Search and Self-Reward mechanisms into the reasoning tree .
Outcome:	The proposed approach surpasses current prompting methods and secures state-of-the-art performance across three multimodal reasoning benchmarks.

Copied to clipboard

Qingyun Wang, Zhihao Zhou, Lifu Huang, Spencer Whitehead, Boliang Zhang, Heng Ji, Kevin Knight

Challenge:	a paper abstract writing system can automatically generate an abstract from a title . a typical recurrent neural network (RNN) based approach easily loses focus.
Approach:	They propose a paper abstract writing system that automatically generates an abstract from a title.
Outcome:	The proposed system passes Turing tests by junior domain experts and non-experts at a rate up to 80%.

MultiAgentBench : Evaluating the Collaboration and Competition of LLM agents (2025.acl-long)

Copied to clipboard

Kunlun Zhu, Hongyi Du, Zhaochen Hong, Xiaocheng Yang, Shuyi Guo, Zhe Wang, Zhenhailong Wang, Cheng Qian, Robert Tang, Heng Ji, Jiaxuan You

Challenge:	Large Language Models (LLMs) have shown remarkable capabilities as autonomous agents, yet existing benchmarks focus on single-agent tasks or are confined to narrow domains, failing to capture the dynamics of multi-agent coordination and competition.
Approach:	They propose a benchmark to evaluate LLM-based multi-agent systems across diverse, interactive scenarios.
Outcome:	The proposed framework measures task completion and quality of collaboration and competition using novel, milestone-based key performance indicators.

DELL: Generating Reactions and Explanations for LLM-Based Misinformation Detection (2024.findings-acl)

Copied to clipboard

Herun Wan, Shangbin Feng, Zhaoxuan Tan, Heng Wang, Yulia Tsvetkov, Minnan Luo

Non-Autoregressive Math Word Problem Solver with Unified Tree Structure (2023.emnlp-main)

Challenge:	Large language models are limited by challenges in factuality and hallucinations to be directly employed off-the-shelf for judging the veracity of news articles.
Approach:	They propose to integrate large language models into the news pipeline by generating news reactions and generating proxy tasks.
Outcome:	The proposed model outperforms state-of-the-art baselines by 16.8% in macro f1-score on seven datasets with three LLMs.

Copied to clipboard

Yi Bin, Mengqun Han, Wenhao Shi, Lei Wang, Yang Yang, See-Kiong Ng, Heng Shen

Numerical Reasoning

Challenge:	Existing MWP solvers do not handle variants that can be derived via mathematical manipulation.
Approach:	They propose a non-autoregressive solver to present a solution expression and decode it from a given problem description.
Outcome:	The proposed solver is able to decode multiple expression variants and correct them . it is based on a unified tree structure and is available on Math23K and MAWPS.

Chem-FINESE: Validating Fine-Grained Few-shot Entity Extraction through Text Reconstruction (2024.findings-eacl)

Copied to clipboard

Qingyun Wang, Zixuan Zhang, Hongxiang Li, Xuan Liu, Jiawei Han, Huimin Zhao, Heng Ji

How do Language Models Reshape Entity Alignment? A Survey of LM-Driven EA Methods: Advances, Benchmarks, and Future (2025.emnlp-main)

Challenge:	Existing frameworks for fine-grained few-shot entity extraction are difficult to implement in the chemical domain due to the information overload of scientific papers.
Approach:	They propose a sequence-to-sequence based few-shot entity extraction approach . it uses a seq2seq entity extractor and a self-validation module to reconstruct original input sentence .
Outcome:	The proposed framework achieves 8.26% and 6.84% performance gains on two datasets.

Copied to clipboard

Zerui Chen, Huiming Fan, Qianyu Wang, Tao He, Ming Liu, Heng Chang, Weijiang Yu, Ze Li, Bing Qin

A Language-First Approach for Procedure Planning (2023.findings-acl)

Challenge:	Entity alignment (EA) is critical for knowledge graph (KG) integration.
Approach:	They propose a taxonomy that categorizes methods in three stages: data preparation, feature embedding, and alignment.
Outcome:	The proposed taxonomy categorizes methods in three key stages: data preparation, feature embedding, and alignment.

Copied to clipboard

Jiateng Liu, Sha Li, Zhenhailong Wang, Manling Li, Heng Ji

Incorporating Graph Attention Mechanism into Knowledge Graph Reasoning Based on Deep Reinforcement Learning (D19-1)

Challenge:	Developing intelligent agents requires the ability to produce plans on the fly based on visual observations.
Approach:	They propose a language-first procedure planning framework with a modularized design . they first align current and goal observations with corresponding steps and then use a pre-trained LM to predict intermediate steps.
Outcome:	The proposed framework matches state-of-the-art procedures on COIN and CrossTask benchmarks.

Copied to clipboard

Heng Wang, Shuangyin Li, Rong Pan, Mingzhi Mao

Defining a New NLP Playground (2023.findings-emnlp)

Challenge:	Existing methods for learning knowledge Graphs are incomplete and therefore need well-pretraining.
Approach:	They propose a deep reinforcement learning based model which incorporates LSTM and Graph Attention Mechanism as the memory components.
Outcome:	The proposed model can get rid of the pretraining process and achieve state-of-the-art performance compared with the other models.

Copied to clipboard

Sha Li, Chi Han, Pengfei Yu, Carl Edwards, Manling Li, Xingyao Wang, Yi Fung, Charles Yu, Joel Tetreault, Eduard Hovy, Heng Ji

ReSo: A Reward-driven Self-organizing LLM-based Multi-Agent System for Reasoning Tasks (2025.emnlp-main)

Challenge:	Recent explosion of performance of large language models (LLMs) has changed the field more abruptly and seismically than any other shift in the field’s 80 year history.
Approach:	They propose 20+ PhD-dissertation-worthy research directions to define a new NLP playground by combining theoretical analysis, new and challenging problems, learning paradigms and interdisciplinary applications.
Outcome:	The proposed research will cover theoretical analysis, new and challenging problems, learning paradigms and interdisciplinary applications.

Copied to clipboard

Heng Zhou, Hejia Geng, Xiangyuan Xue, Li Kang, Yiran Qin, Zhiyong Wang, Zhenfei Yin, Lei Bai

Self-Correction is More than Refinement: A Learning Framework for Visual and Language Reasoning Tasks (2025.findings-acl)

Challenge:	Multi-agent systems (MAS) are limited by poor flexibility and scalability, with underdeveloped optimization strategies.
Approach:	They propose a task graph generation and a reward-driven two-stage agent selection process to integrate multi-agent systems to improve their reasoning capabilities.
Outcome:	The proposed model outperforms existing methods on Math-MAS and SciBench-MAS SciBech, while other methods completely fail.

Copied to clipboard

Jiayi He, Hehai Lin, Qingyun Wang, Yi R. Fung, Heng Ji

DecisionFlow: Advancing Large Language Model as Principled Decision Maker (2025.findings-emnlp)

Challenge:	Large Language Models (LLMs) have shown remarkable abilities, but they invariably generate flawed responses.
Approach:	They propose a self-correction approach that instructs VLMs to refine their outputs by allowing them to learn from their self-generated self-reference data without external feedback.
Outcome:	The proposed approach enables VLMs to learn from their self-generated self-correction data without relying on external feedback, facilitating self-improvement.

Copied to clipboard

Xiusi Chen, Shanyong Wang, Cheng Qian, Hongru Wang, Peixuan Han, Heng Ji

Analyzing and Internalizing Complex Policy Documents for LLM Agents (2026.acl-long)

Challenge:	Current language models lack the structured deliberation needed for high-stakes tasks such as healthcare and finance.
Approach:	They propose a decision-making framework that guides models to reason over structured representations of actions, attributes, and constraints.
Outcome:	The proposed framework achieves up to 30% accuracy gains over strong prompting baselines and enhances alignment in outcomes.

Copied to clipboard

Jiateng Liu, Zhenhailong Wang, Xiaojiang Huang, Yingjie Li, Xiang Li, Chenlei Guo, Xing Fan, Ruhi Sarikaya, Heng Ji

Can LLM Graph Reasoning Generalize beyond Pattern Memorization? (2024.findings-emnlp)

Challenge:	Large language model agents rely on in-context policy documents to act as effective user assistants.
Approach:	They propose an agentic benchmark generator with Controllable Complexity in agent policy across four levels to evaluate agents under increasing complexity.
Outcome:	The proposed method outperforms the baseline in data-sparse and high-complexity settings.

Copied to clipboard

Yizhuo Zhang, Heng Wang, Shangbin Feng, Zhaoxuan Tan, Xiaochuang Han, Tianxing He, Yulia Tsvetkov

ModelingAgent: Bridging LLMs and Mathematical Modeling for Real-World Challenges (2025.findings-emnlp)

Challenge:	Existing studies seek to enhance the graph reasoning capabilities of Large Language Models (LLMs) by specialized instruction tuning.
Approach:	They propose to evaluate LLM graph reasoning generalization using in-distribution settings . they propose to use three strategies to improve LLM generalization .
Outcome:	The proposed benchmark evaluates LLM graph reasoning generalization with in-distribution settings only . it shows that LLMs struggle to generalize across reasoning and real-world patterns .

Copied to clipboard

Cheng Qian, Hongyi Du, Hongru Wang, Xiusi Chen, Yuji Zhang, Avirup Sil, ChengXiang Zhai, Kathleen McKeown, Heng Ji

Numerical Reasoning

Challenge:	Existing benchmarks for large language models fail to reflect real-world complexity . existing benchmarks often fail to capture real-life problems .
Approach:	They propose a benchmark that features real-world-inspired, open-ended problems from competitions . they propose 'ModelingBench' that supports multiple valid solutions .
Outcome:	The proposed framework outperforms baselines and produces well-grounded, creative solutions.

LETI: Learning to Generate from Textual Interactions (2024.findings-naacl)

Copied to clipboard

Xingyao Wang, Hao Peng, Reyhaneh Jabbarvand, Heng Ji

Less Is More: Domain Adaptation with Lottery Ticket for Reading Comprehension (2021.findings-emnlp)

Challenge:	Existing techniques fine-tune on input-output pairs or with numerical rewards that gauge the output quality are not effective.
Approach:	They propose to fine-tune pre-trained language models with binary labels and a Python interpreter to get textual feedback from the inputs.
Outcome:	The proposed model outperforms the base model on unseen problems and achieves comparable or better performance on humanEval.

Copied to clipboard

Haichao Zhu, Zekun Wang, Heng Zhang, Ming Liu, Sendong Zhao, Bing Qin

Reusing Transferable Weight Increments for Low-resource Style Generation (2024.emnlp-main)

Challenge:	Existing domain adaptation paradigms for reading comprehension require large amounts of annotation data to achieve the desired task performance.
Approach:	They propose a few-shot domain adaptation paradigm for reading comprehension . they introduce self-attention attribution to weigh parameters and refine the lottery subnetwork .
Outcome:	The proposed model outperforms the full model fine-tuning adaptation on four out of five domains with a small amount of data available for adaptation.

Copied to clipboard

Chunzhen Jin, Eliot Huang, Heng Chang, Yaqi Wang, Peng Cao, Osmar Zaiane

Low-Resource NLP

Challenge:	Text style transfer (TST) is crucial in natural language processing, aiming to endow text with a new style without altering its meaning.
Approach:	They propose a framework to use style features in weight increments to transfer low-resource styles effectively.
Outcome:	The proposed framework achieves remarkable performance across different backbones, achieving particularly effective results in low-resource scenarios.

ROMA: Real-time Omni-Multimodal Assistant with Interactive Streaming Understanding (2026.findings-acl)

Copied to clipboard

Tian Xueyun, Wei Li, Bingbing Xu, Heng Dong, Yuanzhuo Wang, Huawei Shen

CALM: Unleashing the Cross-Lingual Self-Aligning Ability of Language Model Question Answering (2025.findings-naacl)

Challenge:	Existing omni-multimodal large language models lack incomplete modality support or lack autonomous proactive monitoring.
Approach:	They propose a real-time omni-multimodal assistant for unified reactive and proactive interaction that decouples response initiation from generation to ensure precise triggering without task conflict.
Outcome:	The proposed model achieves state-of-the-art performance on proactive tasks while competing in reactive settings.

Copied to clipboard

Yumeng Wang, Zhiyuan Fan, Qingyun Wang, Yi R. Fung, Heng Ji

Natural Language Interfaces

Challenge:	Large Language Models (LLMs) are pre-trained on extensive multilingual corpora to acquire both language-specific cultural knowledge and general knowledge.
Approach:	They propose to use the Cross-Lingual Self-Aligning ability of Language M**odels to align knowledge across languages.
Outcome:	The proposed model performs well in both zero-shot and retrieval-augmented settings.

Thinking Traps in Long Chain-of-Thought: A Measurable Study and Trap-Aware Adaptive Restart (2026.findings-acl)

Copied to clipboard

null Chenkang, Fan Yu, Junjie Nian, Sihan Zhao, Zhuoka Feng, Zijun Yao, Wang Heng, Yu Minshen, Yixin Cao

ManCC: A Task-Anchored Benchmark for Manchu–Classical Chinese Cross-Lingual Modeling (2026.findings-acl)

Challenge:	Experiments show that extended generation does not guarantee correctness . a recurring pattern in Long-CoT failures is a problem for large reasoning models .
Approach:	They propose a test-time control framework that truncates the trajectory before the trap segment and adaptively restarts decoding.
Outcome:	Experiments show that TAAR improves reasoning performance without fine-tuning model parameters.

Copied to clipboard

Meiqi Wang, Xiaoxin Sun, Dongjie Wang, Ruixin Yu, Xiantao Heng, Shuo Wang, Zhen Huang, Peng Zhao, Suhua Wang, Minghao Yin

Multilinguality

Challenge:	Mainstream research in natural language processing has focused on high-resource and modern languages.
Approach:	They propose a task-anchored benchmark for Manchu–Classical Chinese translation . they use a parallel corpus of 16,627 sentence pairs to evaluate the model .
Outcome:	The proposed benchmarks show that linguistic differences influence performance and broader language coverage facilitate low-resource transfer.

ViStruct: Visual Structural Knowledge Extraction via Curriculum Guided Code-Vision Representation (2023.emnlp-main)

Copied to clipboard

Yangyi Chen, Xingyao Wang, Manling Li, Derek Hoiem, Heng Ji

Named Entity Recognition Under Domain Shift via Metric Learning for Life Sciences (2024.naacl-long)

Challenge:	State-of-the-art vision-language models have limited performance in structural knowledge extraction, such as relations between objects.
Approach:	They propose to leverage the inherent structure of programming language to depict visual structural information in a well-organized structured format.
Outcome:	The proposed framework improves visual structural knowledge extraction on visual structure prediction tasks.

Copied to clipboard

Hongyi Liu, Qingyun Wang, Payam Karisani, Heng Ji

Closing the Modality Reasoning Gap for Speech Large Language Models (2026.acl-long)

Challenge:	Existing models for named entity recognition fail in scientific domains such as biomedicine and chemistry.
Approach:	They propose a model to transfer knowledge from the biomedical domain to the target domain . they use pseudo labeling and contrastive learning to enhance discrimination .
Outcome:	The proposed model outperforms baseline models by up to 5% . the proposed model is based on a biomedical domain model and a chemical domain model .

Copied to clipboard

Chaoren Wang, Heng Lu, Xueyao Zhang, Shujie Liu, Yan Lu, Jinyu Li, Zhizheng Wu

Mirror in the Model: Ad Banner Image Generation via Reflective Multi-LLM and Multi-modal Agents (2025.emnlp-industry)

Challenge:	Recent advances in Speech Large Language Models have a modality reasoning gap that is not addressed by prior work.
Approach:	They propose a reinforcement-learning framework that aligns text-conditioned and speech-conditioned trajectories through an asymmetric reward design.
Outcome:	Experiments on MMSU and OBQA show that the proposed framework narrows the modality reasoning gap and achieves state-of-the-art performance among 7B-scale Speech LLMs.

Copied to clipboard

Zhao Wang, Bowen Chen, Yotaro Shimose, Sota Moriyama, Heng Wang, Shingo Takamatsu

Incentivizing Parametric Knowledge via Reinforcement Learning with Verifiable Rewards for Cross-Cultural Entity Translation (2026.acl-long)

Challenge:	Recent advances in generative modeling have greatly improved image synthesis quality.
Approach:	They propose an agentic refinement framework for automatic ad banner generation that integrates a hierarchical multimodal agent system with a coordination loop.
Outcome:	The proposed model outperforms existing models in real-world banner design scenarios.

Copied to clipboard

Jiang Zhou, Xiaohu Zhao, Xinwei Wu, Tianyu Dong, Hao Wang, Yangyang Liu, Heng Liu, Linlong Xu, Longyue Wang, Weihua Luo, Deyi Xiong

Machine Translation

Challenge:	Current systems often fall short of this goal in settings where translation hinges on culturally grounded entities such as books, films, places, songs and idioms.
Approach:	They propose a framework that anchors supervision on a verifiable, entity-level reward signal and incorporates lightweight structural gates to stabilize optimization.
Outcome:	The proposed framework improves on XC-Translate and shows that it can learn a robust reasoning process rather than imitating reference translations.

Learning Shared Semantic Space for Speech-to-Text Translation (2021.findings-acl)

Copied to clipboard

Chi Han, Mingxuan Wang, Heng Ji, Lei Li

Generalizable LLM Learning of Graph Synthetic Data with Post-training Alignment (2026.findings-acl)

Challenge:	End-to-end speech translation (ST) has been treated as an independent task . however, the modality gap has rendered MT data and its end-to end models incompatible with their ST counterparts.
Approach:	They propose to bridge the representation gap between text and audio inputs by projecting audio and text features to a common semantic representation.
Outcome:	The proposed model improves performance on two ST benchmarks and achieves 27.1 BLEU on MuST-C EN-DE.

Copied to clipboard

Yizhuo Zhang, Heng Wang, Shangbin Feng, Zhaoxuan Tan, Xinyun Liu, Yulia Tsvetkov

Multimedia Generative Script Learning for Task Planning (2023.findings-acl)

Challenge:	Existing research has focused on enhancing graph reasoning capabilities of LLMs by supervised fine-tuning on synthetic graph data.
Approach:	They propose to unlock generalizable learning of graph with post-training alignment with synthetic graph data by aligning off-the-shelf LLMs and LLM fine-tuned on synthetic graphs.
Outcome:	The proposed algorithm improves on synthetic graph problems and out-of-domain tasks with implicit graph structures.

Copied to clipboard

Qingyun Wang, Manling Li, Hou Pong Chan, Lifu Huang, Julia Hockenmaier, Girish Chowdhary, Heng Ji

SARA: Unlocking Multilingual Knowledge in Mixture-of-Experts via Semantically Anchored Routing Alignment (2026.findings-acl)

Challenge:	Goal-oriented generative script learning aims to generate subsequent steps to reach a specific goal . ability to capture historical states in visual modalities provides detailed information not covered by text .
Approach:	They propose a goal-oriented generative script learning task to generate subsequent steps by tracking historical states in both text and vision modalities.
Outcome:	The proposed task outperforms baselines in three aspects of the current task.

Copied to clipboard

Tianyu Dong, Yangyang Liu, Jiang Zhou, Xinwei Wu, Xiaohu Zhao, Hao Wang, Heng Liu, Linlong Xu, Longyue Wang, Weihua Luo, Shaolin Zhu, Deyi Xiong

Multilinguality

Challenge:	Low-resource language tokens are often routed to different experts than those activated by high-resourced inputs, which hinders their efficacy in multilingual contexts.
Approach:	They propose a framework to transfer specialized capabilities from high-resource languages as anchors to low-resourced languages by using a symmetric Jensen-Shannon constraint.
Outcome:	The proposed framework outperforms standard instruction tuning on 5 low-resource languages and 3 benchmarks.

PaperRobot: Incremental Draft Generation of Scientific Ideas (P19-1)

Copied to clipboard

Qingyun Wang, Lifu Huang, Zhiying Jiang, Kevin Knight, Heng Ji, Mohit Bansal, Yi Luan

Code-Switching for Enhancing NMT with Pre-Specified Translation (N19-1)

Challenge:	a paper robot can read existing papers and create new nodes or links in the knowledge graphs.
Approach:	They propose to automate the creation of new ideas by predicting links from the background KGs.
Outcome:	The proposed paper automates three tasks: read existing papers, create new ideas, predict links . the paper generated abstracts, conclusion and future work sections, and new titles are chosen over human-written ones up to 30%, 24% and 12% of the time.

Copied to clipboard

Kai Song, Yue Zhang, Heng Yu, Weihua Luo, Kun Wang, Min Zhang

Multilinguality

Challenge:	Existing methods to constrain NMT use placeholder tags for lexicon words and hard constraints during decoding.
Approach:	They propose to use placeholder tags to replace lexicon words with target translations . they use a data augmentation method to make code-switched training data .
Outcome:	The proposed method improves translation quality without hurting unconstrained words.

BannerAgency: Advertising Banner Design with Multimodal LLM Agents (2025.emnlp-main)

Copied to clipboard

Heng Wang, Yotaro Shimose, Shingo Takamatsu

Compositional Reasoning via Joint Image and Language Decomposition (2026.findings-eacl)

Challenge:	Advertising banners are an instrumental medium in digital marketing campaigns.
Approach:	They propose a training-free framework for fully automated banner ad design creation that enables frontier multimodal large language models to streamline the production of effective banners with minimal manual effort.
Outcome:	The proposed framework is based on a training-free model that can be used to create fully automated banner ad design creations with minimal manual effort across diverse marketing contexts.

Copied to clipboard

Dwip Dalal, Madhav Kanda, Zhenhailong Wang, Heng Ji, Unnat Jain

Rescorla-Wagner Steering of LLMs for Undesired Behaviors over Disproportionate Inappropriate Context (2025.emnlp-main)

Challenge:	Existing approaches typically decompose only language queries, treating images as monolithic inputs.
Approach:	They propose a framework that decomposes both images and questions into visual sub-domains with corresponding sub-questions.
Outcome:	REDI achieves absolute accuracy improvements of 8.9%, 8.2%, and 16.0% over existing models.

Copied to clipboard

Rushi Wang, Jiateng Liu, Cheng Qian, Yifan Shen, Yanzhou Pan, Zhaozhuo Xu, Ahmed Abbasi, Heng Ji, Denghui Zhang

Robustness in NLP

Challenge:	Incorporating external context can enhance the response quality of Large Language Models (LLMs). however, real-world contexts often mix relevant information with disproportionate inappropriate content.
Approach:	They propose a Poisoned Context Testbed to pair queries with real-world contexts . they propose 'rw-Steering' to internalize inappropriate signals .
Outcome:	The proposed model improves response quality by 39.8% and reverses undesirable behavior curve.

Towards a Human-Computer Collaborative Scientific Paper Lifecycle: A Pilot Study and Hands-On Tutorial (2024.lrec-tutorials)

Copied to clipboard

Qingyun Wang, Carl Edwards, Heng Ji, Tom Hope

Do LVLMs Understand Charts? Analyzing and Correcting Factual Errors in Chart Captioning (2024.findings-acl)

Challenge:	a tutorial aims to provide an overview of the scientific paper lifecycle . large language models (LLMs) have increasingly played an important role in academic writing .
Approach:	They propose to provide an overview of the scientific paper lifecycle using large language models.
Outcome:	The tutorial will provide an overview of the scientific paper lifecycle, including scientific literature understanding, experiment development, manuscript draft writing, and finally draft evaluation.

Copied to clipboard

Kung-Hsiang Huang, Mingyang Zhou, Hou Pong Chan, Yi Fung, Zhenhailong Wang, Lingyu Zhang, Shih-Fu Chang, Heng Ji

Distantly-Supervised Named Entity Recognition with Noise-Robust Learning and Language Model Augmented Self-Training (2021.emnlp-main)

Challenge:	LVLMs are known for producing text that is factually inconsistent with visual input . factuality of generated captions for structured visuals has not been studied as much .
Approach:	They propose a typology of factual errors in captions generated by large vision-language models . they propose CHOCOLATE, a visual entailment model that outperforms current models based on this analysis .
Outcome:	The proposed model outperforms current models in evaluating caption factuality.

Copied to clipboard

Yu Meng, Yunyi Zhang, Jiaxin Huang, Xuan Wang, Yu Zhang, Heng Ji, Jiawei Han

Named Entity Recognition

Challenge:	Named entity recognition models require abundant high-quality annotations to train . distant supervision may induce incomplete and noisy labels, making supervised learning ineffective.
Approach:	They propose a noise-robust learning scheme for training named entity recognition models using only distantly-labeled data and a self-training method that uses contextualized augmentations created by pre-trained language models.
Outcome:	The proposed method outperforms existing supervised NER models on three datasets by significant margins.

Language + Molecules (2024.eacl-tutorials)

Copied to clipboard

Carl Edwards, Qingyun Wang, Heng Ji

Stage-wise Fine-tuning for Graph-to-Text Generation (2021.acl-srw)

Challenge:	In the last year, instruction-following language models have surged in popularity.
Approach:	This tutorial will provide an introduction to applying natural language-driven solutions to chemistry problems.
Outcome:	This tutorial will provide an introduction to this area of research. it requires no knowledge outside mainstream NLP, and it will enable participants to begin exploring relevant research.

Copied to clipboard

Qingyun Wang, Semih Yavuz, Xi Victoria Lin, Heng Ji, Nazneen Rajani