Papers by Zhipeng Yang

AnyTrans: Translate AnyText in the Image with Large Scale Models (2024.findings-emnlp)

Copied to clipboard

Zhipeng Qian, Pei Zhang, Baosong Yang, Kai Fan, Yiwei Ma, Derek Wong, Xiaoshuai Sun, Rongrong Ji

Challenge:	Recent advances in natural language processing and computer vision have made it possible to translate images with text in one language into equivalent images displaying that text translated into another language.
Approach:	They propose an all-encompassing framework for the task–In-Image Machine Translation (IIMT) that incorporates contextual cues from both textual and visual elements during translation.
Outcome:	The proposed framework can be constructed using open-source models and requires no training, making it highly accessible and expandable.

A Sentence Cloze Dataset for Chinese Machine Reading Comprehension (2020.coling-main)

Copied to clipboard

Yiming Cui, Ting Liu, Ziqing Yang, Zhipeng Chen, Wentao Ma, Wanxiang Che, Shijin Wang, Guoping Hu

Reasoning

Challenge:	Using cloze-style reading comprehension, Chinese machine reading comprehension datasets are becoming more and more popular . a new task is proposed to fill the right candidate sentence into the passage with several blanks .
Approach:	They propose a Chinese task to fill the right candidate sentence into a passage with blanks . they build a dataset to evaluate the difficulty of the task and make fake candidates .
Outcome:	The proposed task fills the right candidate sentence into the passage with blanks . the proposed dataset contains over 100K blanks within over 10K passages based on Chinese narrative stories .

TextBrewer: An Open-Source Knowledge Distillation Toolkit for Natural Language Processing (2020.acl-demos)

Copied to clipboard

Ziqing Yang, Yiming Cui, Zhipeng Chen, Wanxiang Che, Ting Liu, Shijin Wang, Guoping Hu

Language Models

Challenge:	Large pre-trained language models have hundreds of millions of parameters and take several gigabytes of memory to train and inference.
Approach:	They propose an open-source knowledge distillation toolkit designed for natural language processing that provides a set of predefined distillation methods and can be extended with custom code.
Outcome:	The proposed method is comparable with or even higher than the public distilled BERT models with similar numbers of parameters.

LaMemo: Language Modeling with Look-Ahead Memory (2022.naacl-main)

Copied to clipboard

Haozhe Ji, Rongsheng Zhang, Zhenyu Yang, Zhipeng Hu, Minlie Huang

Language Models

Challenge:	Existing approaches to model long-term dependencies are limited to long texts with thousands of words.
Approach:	They propose a look-ahead memory that augments the recurrence memory by attending to the right-side tokens and interpolating with the old memory states to maintain long-term information in the history.
Outcome:	Experiments on widely used language modeling benchmarks show that LaMemo outperforms baseline models with recurrence memory.

DMN: A Compositional Framework for Jailbreaking Multimodal LLMs with Multi-Image Inputs (2026.acl-long)

Copied to clipboard

Wenzhuo Xu, Zhipeng Wei, Zonghao Ying, Deyue Zhang, Dongdong Yang, Xiangzheng Zhang, Quanchen Zou

Responsible & Trustworthy NLP

Challenge:	Existing jailbreak methods only use a single image, restricting the attack space . Existing frameworks only use single image to distribute harmful requests across multiple images .
Approach:	They propose a compositional jailbreak framework that leverages Distributed instruction, Multimodal evidence and a Number chain task to fully enhance the jailbreak performance.
Outcome:	The proposed framework achieves attack success rates of over 90% on GPT-4o, Gemini-2.5-pro and Claude Sonnet 4 .

UNIKIE-BENCH: Benchmarking Large Multimodal Models for Key Information Extraction in Visual Documents (2026.acl-long)

Copied to clipboard

Yifan Ji, Zhipeng Xu, Zhenghao Liu, Zulong Chen, Qian Zhang, Zhibo Yang, Junyang Lin, Yu Gu, Ge Yu, Maosong Sun

Information Extraction & Text Mining

Challenge:	Recent Large Multimodal Models (LMMs) have shown promising potential for performing end-to-end KIE directly from document images.
Approach:	They propose a benchmark to evaluate the performance of Large Multimodal Models (LMMs) using a constrained-category KIE track and an open-categorical KIE Track.
Outcome:	Experiments on 15 state-of-the-art LMMs show performance degradation under diverse schema definitions, long-tail key fields, and complex layouts, along with pronounced performance disparities across different document types and scenarios.

Internal Chain-of-Thought: Empirical Evidence for Layer‐wise Subtask Scheduling in LLMs (2025.emnlp-main)

Copied to clipboard

Zhipeng Yang, Junzhuo Li, Siyu Xia, Xuming Hu

Challenge:	et al., 2024) show large language models exhibit internal chain-of-thought, meaning they decompose and execute composite tasks layer-by-layer.
Approach:	They propose a method to decode hidden states using LogitLens . they also propose 'chain of thought' to decompose and execute composite tasks .
Outcome:	The proposed method decodes hidden states and shows consistent execution pattern . it opens avenues for fine-grained, instruction-level activation steering.

Jiuge: A Human-Machine Collaborative Chinese Classical Poetry Generation System (P19-3)

Copied to clipboard

Guo Zhipeng, Xiaoyuan Yi, Maosong Sun, Wenhao Li, Cheng Yang, Jiannan Liang, Huimin Chen, Yuhui Zhang, Ruoyu Li

Text Generation

Challenge:	Existing systems for automatic poetry generation are model-oriented, resulting in poor user participation.
Approach:	They propose a human-machine collaborative Chinese classical poetry generation system called Jiuge . Jiuge allows users to revise unsatisfied parts of a generated poem draft repeatedly .
Outcome:	The proposed system allows users to revise unsatisfied parts of a generated poem draft repeatedly.

Human-Agent Collaborative Paper-to-Page Crafting (2026.findings-acl)

Copied to clipboard

Qianli Ma, Siyu Wang, Chen Yilin, Yinhao Tang, Yixiang Yang, Chang Guo, Bingjie Gao, Zhening Xing, Yanan Sun, Zhipeng Zhang

Challenge:	Existing approaches to create project pages from academic papers have focused on static slides and posters, but the dynamic nature of webpages remains an unaddressed challenge.
Approach:	They propose a novel multi-agent system that deconstructs paper-to-page creation into a coarse-to fine pipeline from narrative planning to multimodal content generation and interactive rendering.
Outcome:	The proposed system generates high-quality, visually appealing pages in under 15 minutes for less than $0.1 .

ElitePLM: An Empirical Study on General Language Ability Evaluation of Pretrained Language Models (2022.naacl-main)

Copied to clipboard

Junyi Li, Tianyi Tang, Zheng Gong, Lixin Yang, Zhuohao Yu, Zhipeng Chen, Jingyuan Wang, Xin Zhao, Ji-Rong Wen

Language Models

Challenge:	Recent years have featured a trend towards Transformer based pretrained language models (PLMs) in natural language processing systems.
Approach:	They propose to use four evaluation dimensions to evaluate ten widely-used PLMs . they find that pretrained language models are good at different ability tests .
Outcome:	The results show that pretrained language models are good at different ability tests and have excellent transferability between tasks.

KuiLeiXi: a Chinese Open-Ended Text Adventure Game (2021.acl-demo)

Copied to clipboard

Yadong Xi, Xiaoxi Mao, Le Li, Lei Lin, Yanjiang Chen, Shuhan Yang, Xuhan Chen, Kailun Tao, Zhi Li, Gongzheng Li, Lin Jiang, Siyan Liu, Zeng Zhao, Minlie Huang, Changjie Fan, Zhipeng Hu

Language Models

Challenge:	Recent advances in pre-trained language models have made it possible to generate human-like text.
Approach:	They propose to integrate an open-ended text adventure game in Chinese, named KuiLeiXi, where players interact with the AI until the plot goals are reached.
Outcome:	The proposed game lacks incentives and relies on players to explore on their own.