Papers by Zilong Wang

ReflectEvo: Improving Meta Introspection of Small LLMs by Learning Self-Reflection (2025.findings-acl)

Challenge:	Recent advances in Large Language Models (LLMs) have shown inspiring achievements in constructing autonomous agents that rely on language descriptions as inputs.
Approach:	They propose a flexible and simulation-free testbed that simulates 6 representative embodied tasks in textual embodies.
Outcome:	The proposed testbed offers adaptability to diverse environments without multiple simulation engines and allows easy customization of communication and action strategies.

Copied to clipboard

Jiaqi Li, Xinyi Dong, Yang Liu, Zhizhuo Yang, Quansen Wang, Xiaobo Wang, Song-Chun Zhu, Zixia Jia, Zilong Zheng

Reasoning

Challenge:	ReflectEvo-460k is a large-scale, comprehensive, self-generated reflection dataset with broadened instructions and diverse multi-domain tasks.
Approach:	They propose a pipeline that iteratively generates self-reflection for self-training and a large-scale reflection dataset with broadened instructions and diverse multi-domain tasks.
Outcome:	The proposed pipeline improves Llama-3 reasoning ability by up to 71.2% and Mistral by upto 44.4%.

Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step by Step (2024.findings-acl)

Copied to clipboard

Li Zhong, Zilong Wang, Jingbo Shang

Formulating Few-shot Fine-tuning Towards Language Model Pre-training: A Pilot Study on Named Entity Recognition (2022.findings-emnlp)

Challenge:	Large language models (LLMs) are leading progress in code generation, but they are underutilized in the literature.
Approach:	They propose a debugging framework that allows LLMs to refine their generated programs with the runtime execution information.
Outcome:	The proposed framework improves the baseline performance by 9.8% across the HumanEval, MBPP, and TransCoder benchmarks.

Copied to clipboard

Zihan Wang, Kewen Zhao, Zilong Wang, Jingbo Shang

Varying Sentence Representations via Condition-Specified Routers (2024.emnlp-main)

Challenge:	Existing methods for fine-tuning pre-trained language models are limited . we propose a few-shot fine-uning framework for NER .
Approach:	They propose a few-shot fine-tuning framework for named entity recognition (NER) they propose three new types of tokens, "is-entity", "which-type" and "bracket"
Outcome:	The proposed framework improves on pre-trained language models on several benchmark datasets.

Copied to clipboard

Ziyong Lin, Quansen Wang, Zixia Jia, Zilong Zheng

Natural-Language Policies to Executable Decisions: An Interpretable Large Language Model Framework (2026.acl-industry)

Challenge:	Existing sentences cannot account for different aspects of semantic similarity between two sentences.
Approach:	They propose a transformer-style framework that generates conditioned sentences . they propose 'conditional' STS, which measures similarity between two sentences based on condition sentences - a task that requires a sentence embedding model capable of generating distinct representations for the same sentence under different conditions.
Outcome:	The proposed framework is superior to existing models on two condition sentences . it can generate conditioned sentences while maintaining model parameters and computational efficiency .

Copied to clipboard

Ziqiang Zhang, Jing Ma, Zilong Wang, Jiayuan Chen, Yi Qiao, Yu He, Wei Zhang, Dai Cheng, Xiaoyu Shen

Are the Values of LLMs Structurally Aligned with Humans? A Causal Perspective (2025.findings-acl)

Challenge:	a production-grade pricing system for tourism is challenging due to unstructured nature of travel orders and ever-evolving pricing policies.
Approach:	They propose a production-grade pricing system with a strict decision boundary . they propose to combine structured extraction and bounded policy/path selection with interpretable condition trees .
Outcome:	The proposed system processed 3,960 orders in six months and reduced the order management team from 15-20 to 3 . the system reduced the per-order handling time from 10 minutes to 2 minutes.

Copied to clipboard

Yipeng Kang, Junqi Wang, Yexin Li, Mengmeng Wang, Wenming Tu, Quansen Wang, Hengli Li, Tingjun Wu, Xue Feng, Fangwei Zhong, Zilong Zheng

Structured Data in NLP

Challenge:	Current approaches to value alignment focus on a few core values, such as helpfulness, harmlessness, and honesty.
Approach:	They propose to use latent causal value graphs to guide two lightweight value-steering methods . role-based prompting and sparse autoencoder (SAE) steering are also used .
Outcome:	Experiments on Gemma-2B-IT and Llama3-8B- IT show that the proposed methods are effective and controllable.

LayoutReader: Pre-training of Text and Layout for Reading Order Detection (2021.emnlp-main)

Copied to clipboard

Zilong Wang, Yiheng Xu, Lei Cui, Jingbo Shang, Furu Wei

Visual Data in NLP

Challenge:	Existing methods for reading order detection are too laborious to annotate large datasets.
Approach:	They propose to use a large-scale dataset to annotate reading order information for document images . they use XML metadata to capture the reading order of WORD documents .
Outcome:	The proposed model performs almost perfectly in reading order detection and improves both open-source and commercial OCR engines in ordering text lines in their results.

Towards More Realistic Chinese Spell Checking with New Benchmark and Specialized Expert Model (2024.lrec-main)

Copied to clipboard

Yue Wang, Zilong Zheng, Juntao Li, Zhihui Liu, Jinxiong Chang, Qishen Zhang, Zhongyi Liu, Guannan Zhang, Min Zhang

Syntactic Text Processing

Challenge:	Large Language Models (LLMs) have been gaining attention for their ability to perform a wide range of open-domain tasks . however, the performance of LLMs has yet to be comprehensively evaluated in realistic scenarios .
Approach:	They propose a task to evaluate the performance of Large Language Models (LLMs) they propose RCSC task to convert Chinese text into correct text .
Outcome:	The proposed task evaluates the performance of existing methods in Chinese text . the realistic Chinese spell checker can achieve state-of-the-art performance on the task .

From Static Inference to Dynamic Interaction: A Survey of Streaming Large Language Models (2026.findings-acl)

Copied to clipboard

Junlong Tong, Zilong Wang, YuJie Ren, Peiran Yin, Hao Wu, Wei Zhang, Xiaoyu Shen

Exploring Semantic Capacity of Terms (2020.emnlp-main)

Challenge:	Existing definitions of streaming LLMs are fragmented and lack a systematic taxonomy . large language models are pre-trained on static and full-context corpora .
Approach:	They propose a systematic taxonomy of current streaming Large Language Models and propose underlying methodologies for streaming LLMs.
Outcome:	The proposed model is based on data flow and dynamic interaction to clarify existing ambiguities.

Copied to clipboard

Jie Huang, Zilong Wang, Kevin Chang, Wen-mei Hwu, JinJun Xiong

Challenge:	Existing models that measure semantic capacity of terms are not all considered equal . a good command of semantic capacity will give us more insight into the granularity of terms .
Approach:	They propose a model that evaluates semantic capacity of terms if text corpus can provide enough co-occurrence information of terms.
Outcome:	The proposed model can evaluate semantic capacity of terms if the corpus can provide enough co-occurrence information of terms.

Position Engineering: Boosting Large Language Models through Positional Information Manipulation (2024.emnlp-main)

Copied to clipboard

Zhiyuan He, Huiqiang Jiang, Zilong Wang, Yuqing Yang, Luna Qiu, Lili Qiu

Efficient Temporal Extrapolation of Multimodal Large Language Models with Temporal Grounding Bridge (2024.emnlp-main)

Challenge:	Recent advances in Large Language Models (LLMs) have demonstrated significant strides towards achieving artificial general intelligence.
Approach:	They propose a technique termed position engineering which alters the positional information in the prompt without modifying the text itself.
Outcome:	The proposed technique significantly improves on the baseline in retrieval-augmented generation and in-context learning scenarios.

Copied to clipboard

Yuxuan Wang, Yueqian Wang, Pengfei Wu, Jianxin Liang, Dongyan Zhao, Yang Liu, Zilong Zheng

DocStruct: A Multimodal Method to Extract Hierarchy Structure in Document for General Form Understanding (2020.findings-emnlp)

Challenge:	despite advances in multimodal large language models, the challenge of interpreting long-form videos remains a challenge . despite advancements in video-language benchmarks, the inefficiency in temporal grounding and limited pre-trained context window size remains .
Approach:	They propose a framework that bootstraps MLLMs with advanced temporal grounding capabilities and broadens their contextual scope.
Outcome:	The proposed framework significantly enhances the temporal capabilities of existing MLLMs.

Copied to clipboard

Zilong Wang, Mingjie Zhan, Xuebo Liu, Ding Liang

Rethinking Dictionaries and Glyphs for Chinese Language Pre-training (2023.findings-acl)

Challenge:	Form understanding is a complex task because of the textual contents and organizational structure of forms.
Approach:	They propose to use multimodal methods to extract key-value pairs from forms . they validate their method on two benchmarks and demonstrate their effectiveness .
Outcome:	The proposed method is validated on two benchmarks, MedForm and FUNSD.

Copied to clipboard

Yuxuan Wang, Jack Wang, Dongyan Zhao, Zilong Zheng

MGDoc: Pre-training with Multi-granular Hierarchy for Document Image Understanding (2022.emnlp-main)

Challenge:	Large-scale pre-trained language models (PLMs) such as BERT and GPT have revolutionized various research fields in natural language processing (NLP)
Approach:	They propose a new learning paradigm that enhances the semantics understanding ability of Chinese PLMs with dictionary knowledge and structure of Chinese characters.
Outcome:	The proposed model improves on both modern Chinese understanding benchmark CLUE and ancient Chinese understanding.

Copied to clipboard

Zilong Wang, Jiuxiang Gu, Chris Tensmeyer, Nikolaos Barmpalios, Ani Nenkova, Tong Sun, Jingbo Shang, Vlad Morariu

The Price of Format: Diversity Collapse in LLMs (2025.findings-emnlp)

Challenge:	Existing methods learn features from word-level or region-level but fail to consider both simultaneously.
Approach:	They propose a multi-modal multi-granular pre-training framework that encodes page-level, region-level and word-level information at the same time.
Outcome:	The proposed model learns features from word-level and region-level but fails to consider both simultaneously.

Copied to clipboard

Longfei Yun, Chenyang An, Zilong Wang, Letian Peng, Jingbo Shang

Towards Zero-shot Relation Extraction in Web Mining: A Multimodal Approach with Relative XML Path (2023.findings-emnlp)

Challenge:	Instruction-tuned large language models employ structured templates to enforce format consistency during inference.
Approach:	They fine-tune instruction-tuning large language models with structured templates and evaluate their results across three axes: downstream task performance, alignment behavior, and output diversity.
Outcome:	The proposed model generates semantically similar outputs even under high temperature sampling and structural tokens in templates significantly constrain the model’s output space.

Copied to clipboard

Zilong Wang, Jingbo Shang

Incubating Text Classifiers Following User Instruction with Nothing but LLM (2024.emnlp-main)

Challenge:	Existing methods for zero-shot relation extraction do not take into account relationships between text nodes within and across web pages.
Approach:	They propose a new approach for zero-shot relation extraction in web mining that encodes the shortest relative paths in the Document Object Model tree of the web page.
Outcome:	The proposed method outperforms the state-of-the-art methods on public benchmarks on semi-structured web pages.

Copied to clipboard

Letian Peng, Zilong Wang, Jingbo Shang

Boosting LLM Agents with Recursive Contemplation for Effective Deception Handling (2024.findings-acl)

Challenge:	In this paper, we aim to generate text classification data given arbitrary class definitions . Traditional supervised text classification fine-tunes models on expensive human annotation .
Approach:	They propose a framework that can generate text classification data given arbitrary class definitions . they use instruction-to-data mappings and in-context augmentation to refine the framework .
Outcome:	The proposed framework outperforms existing methods on benchmarks and training data generation by prompt engineering.

Copied to clipboard

Shenzhi Wang, Chang Liu, Zilong Zheng, Siyuan Qi, Shuo Chen, Qisen Yang, Andrew Zhao, Chaofei Wang, Shiji Song, Gao Huang

Challenge:	Recent advances in large language models (LLMs) have led to significant success in using LLMs as agents.
Approach:	They propose a cognitive framework that incorporates first-order and second-order perspective transitions into LLMs to enhance their ability to identify and counteract deceptive information.
Outcome:	The proposed framework enhances LLMs’ ability to identify and counteract deceptive information without extra fine-tuning and data.

Cuckoo: An IE Free Rider Hatched by Massive Nutrition in LLM’s Nest (2025.acl-long)

Copied to clipboard

Letian Peng, Zilong Wang, Feng Yao, Jingbo Shang

Towards Few-shot Entity Recognition in Document Images: A Graph Neural Network Approach Robust to Image Manipulation (2024.lrec-main)

Challenge:	Massive high-quality data, both pre-training raw texts and post-training annotations, have been carefully prepared to incubate advanced large language models (LLMs).
Approach:	They propose to reframe next-token prediction into extraction for tokens already present in the context of LLMs by reframing next-tongue prediction into IE models.
Outcome:	The proposed model learns 102.6M extractive data converted from pre-training and post-training data with better performance than existing pre-trained IE models.

Copied to clipboard

Prashant Krishnan, Zilong Wang, Yangkun Wang, Jingbo Shang

VSTAR: A Video-grounded Dialogue Dataset for Situated Semantic Understanding with Scene and Topic Transitions (2023.acl-long)

Challenge:	Existing methods for named entity recognition from document images are limited in few-shot settings.
Approach:	They propose a framework which leverages the topological adjacency relationship among tokens by learning layout information with graph neural networks.
Outcome:	The proposed framework outperforms baselines under different few-shot settings and shows better performance to image manipulations.

Copied to clipboard

Yuxuan Wang, Zilong Zheng, Xueliang Zhao, Jinpeng Li, Yueqian Wang, Dongyan Zhao

Dialogue Systems & Conversational Agents

Challenge:	Existing benchmarks for video-grounded dialogues neglect the intrinsic attributes of multimodal dialogues, such as scene and topic transitions.
Approach:	They propose to use a large scale video-grounded scene&topic AwaRe dialogue dataset to study video-based dialogue understanding.
Outcome:	The proposed dataset shows that multimodal information and segments are important in video-grounded dialogue understanding and generation.

PAR: Political Actor Representation Learning with Social Context and Expert Knowledge (2022.emnlp-main)

Copied to clipboard

Shangbin Feng, Zhaoxuan Tan, Zilong Chen, Ningnan Wang, Peisheng Yu, Qinghua Zheng, Xiaojun Chang, Minnan Luo

Representation Learning

Challenge:	Existing approaches focus on textual data and voting records to induce political actors' stances.
Approach:	They propose a Political Actor Representation learning framework that leverages social context and expert knowledge to model ideological stances.
Outcome:	The proposed framework improves political text understanding and improves roll call vote prediction and political perspective detection.

DOCMASTER: A Unified Platform for Annotation, Training, & Inference in Document Question-Answering (2024.naacl-demo)

Copied to clipboard

Alex Nguyen, Zilong Wang, Jingbo Shang, Dheeraj Mekala

Question Answering

Challenge:	DOCMASTER is a platform for annotating PDF documents, model training, and inference, tailored to document question-answering.
Approach:	They propose to integrate layout information into a unified platform for annotating PDF documents, model training, and inference tailored to document question-answering.
Outcome:	The proposed platform is designed for annotating PDF documents, model training, and inference, tailored to document question-answering.

Look Both Ways and No Sink: Converting LLMs into Text Encoders without Training (2025.acl-long)

Copied to clipboard

Ziyong Lin, Haoyi Wu, Shu Wang, Kewei Tu, Zilong Zheng, Zixia Jia

Annotate Chinese Aspect with UMR——a Case Study on the Liitle Prince (2024.lrec-main)

Challenge:	Existing methods for converting large language models into powerful text encoders require extensive training on large datasets.
Approach:	They propose a training-free approach that enables bidirectional attention and suppresses the attention sink phenomenon, resulting in superior performance.
Outcome:	The proposed approach enables bidirectional attention and suppresses the attention sink phenomenon, resulting in superior performance.

Copied to clipboard

Sijia Ge, Zilong Li, Alvin Po-Chun Chen, Guanchao Wang

Challenge:	Uniform Meaning Representation (UMR) is a graphbased cross-linguistically applicable semantic representation that allows for deep semantic analysis.
Approach:	They propose to use an aspectual lattice to adapt to different languages and design values that encompass both viewpoint aspect and situation aspect.
Outcome:	The proposed representations are based on the Chinese version of The Little Prince and are compared with other representations.

Answer is All You Need: Instruction-following Text Embedding via Answering the Question (2024.acl-long)

Copied to clipboard

Letian Peng, Yuwei Zhang, Zilong Wang, Jayanth Srinivasa, Gaowen Liu, Zihan Wang, Jingbo Shang

LMDX: Language Model-based Document Information Extraction and Localization (2024.findings-acl)

Challenge:	Existing methods for encoding instruction information fail to be sensitive to clearer criteria like “evaluate similarity based on emotion” . instead, we propose a different approach, which treats the instruction as a “question” about the input text and encodes the expected answers to obtain the representation accordingly.
Approach:	They propose a text embedder that captures characteristics of texts specified by user instructions clarifying the similarity criterion.
Outcome:	The proposed model improves instruction-following capabilities when applied to large language models and encoder-based LMs.

Copied to clipboard

Vincent Perot, Kai Kang, Florian Luisier, Guolong Su, Xiaoyu Sun, Ramya Sree Boppana, Zilong Wang, Zifeng Wang, Jiaqi Mu, Hao Zhang, Chen-Yu Lee, Nan Hua

Towards Few-shot Entity Recognition in Document Images: A Label-aware Sequence-to-Sequence Framework (2022.findings-acl)

Challenge:	Large Language Models have revolutionized Natural Language Processing but their application in extracting information from visually rich documents has not been successful.
Approach:	They propose a language model-based document information extraction and localization methodology to reframe the document information extract task for a LLM.
Outcome:	The proposed method enables extraction of singular, repeated, and hierarchical entities with and without training data.

Copied to clipboard

Zilong Wang, Jingbo Shang