Papers by Zhipeng Hu

Dataset for the First Evaluation on Chinese Machine Reading Comprehension (L18-1)

Copied to clipboard

Yiming Cui, Ting Liu, Zhipeng Chen, Wentao Ma, Shijin Wang, Guoping Hu

Challenge:	Existing reading comprehension datasets are mostly in English .
Approach:	They propose a Chinese reading comprehension dataset to add diversity to existing reading comprehension data . proposed dataset contains cloze-style reading comprehension and user query reading comprehension .
Outcome:	The proposed dataset is based on a Chinese reading comprehension dataset . it includes two types of cloze-style and user query reading comprehension . the proposed dataset hosted the 1st Evaluation on Chinese Machine Reading Comprehension (CMRC-2017)

A Sentence Cloze Dataset for Chinese Machine Reading Comprehension (2020.coling-main)

Copied to clipboard

Yiming Cui, Ting Liu, Ziqing Yang, Zhipeng Chen, Wentao Ma, Wanxiang Che, Shijin Wang, Guoping Hu

Reasoning

Challenge:	Using cloze-style reading comprehension, Chinese machine reading comprehension datasets are becoming more and more popular . a new task is proposed to fill the right candidate sentence into the passage with several blanks .
Approach:	They propose a Chinese task to fill the right candidate sentence into a passage with blanks . they build a dataset to evaluate the difficulty of the task and make fake candidates .
Outcome:	The proposed task fills the right candidate sentence into the passage with blanks . the proposed dataset contains over 100K blanks within over 10K passages based on Chinese narrative stories .

TextBrewer: An Open-Source Knowledge Distillation Toolkit for Natural Language Processing (2020.acl-demos)

Copied to clipboard

Ziqing Yang, Yiming Cui, Zhipeng Chen, Wanxiang Che, Ting Liu, Shijin Wang, Guoping Hu

Language Models

Challenge:	Large pre-trained language models have hundreds of millions of parameters and take several gigabytes of memory to train and inference.
Approach:	They propose an open-source knowledge distillation toolkit designed for natural language processing that provides a set of predefined distillation methods and can be extended with custom code.
Outcome:	The proposed method is comparable with or even higher than the public distilled BERT models with similar numbers of parameters.

LaMemo: Language Modeling with Look-Ahead Memory (2022.naacl-main)

Copied to clipboard

Haozhe Ji, Rongsheng Zhang, Zhenyu Yang, Zhipeng Hu, Minlie Huang

Language Models

Challenge:	Existing approaches to model long-term dependencies are limited to long texts with thousands of words.
Approach:	They propose a look-ahead memory that augments the recurrence memory by attending to the right-side tokens and interpolating with the old memory states to maintain long-term information in the history.
Outcome:	Experiments on widely used language modeling benchmarks show that LaMemo outperforms baseline models with recurrence memory.

A Span-Extraction Dataset for Chinese Machine Reading Comprehension (D19-1)

Copied to clipboard

Yiming Cui, Ting Liu, Wanxiang Che, Li Xiao, Zhipeng Chen, Wentao Ma, Shijin Wang, Guoping Hu

Machine Reading Comprehension

Challenge:	Existing reading comprehension datasets are mostly in English . MRC is a new field of research that aims to comprehend the context of articles and answer the questions based on them.
Approach:	They propose a Span-Extraction dataset for Chinese machine reading comprehension to add language diversities to existing reading comprehension datasets.
Outcome:	The proposed dataset is composed of 20,000 real questions annotated on Wikipedia paragraphs by human experts.

Towards Effective and Efficient Continual Pre-training of Large Language Models (2025.acl-long)

Copied to clipboard

Jie Chen, Zhipeng Chen, Jiapeng Wang, Kun Zhou, Yutao Zhu, Jinhao Jiang, Yingqian Min, Xin Zhao, Zhicheng Dou, Jiaxin Mao, Yankai Lin, Ruihua Song, Jun Xu, Xu Chen, Rui Yan, Zhewei Wei, Di Hu, Wenbing Huang, Ji-Rong Wen

Language Models

Challenge:	Continual pre-training (CPT) has been an important approach for adapting language models to specific domains or tasks.
Approach:	They propose a Continual pre-training method that can greatly improve Chinese language ability and scientific reasoning ability of LLMs.
Outcome:	The proposed method can greatly improve Chinese language ability and scientific reasoning ability of LLMs.

Internal Chain-of-Thought: Empirical Evidence for Layer‐wise Subtask Scheduling in LLMs (2025.emnlp-main)

Copied to clipboard

Zhipeng Yang, Junzhuo Li, Siyu Xia, Xuming Hu

Challenge:	et al., 2024) show large language models exhibit internal chain-of-thought, meaning they decompose and execute composite tasks layer-by-layer.
Approach:	They propose a method to decode hidden states using LogitLens . they also propose 'chain of thought' to decompose and execute composite tasks .
Outcome:	The proposed method decodes hidden states and shows consistent execution pattern . it opens avenues for fine-grained, instruction-level activation steering.

TextBox 2.0: A Text Generation Library with Pre-trained Language Models (2022.emnlp-demos)

Copied to clipboard

Tianyi Tang, Junyi Li, Zhipeng Chen, Yiwen Hu, Zhuohao Yu, Wenxun Dai, Wayne Xin Zhao, Jian-yun Nie, Ji-rong Wen

Language Models

Challenge:	TextBox 2.0 focuses on the use of pre-trained language models (PLMs) to generate text.
Approach:	They propose a library that integrates pre-trained language models into 13 common text generation tasks and 83 datasets.
Outcome:	The proposed library covers 13 common text generation tasks and their corresponding datasets and incorporates 45 PLMs covering general, translation, Chinese, dialogue, controllable, distilled, prompting, and lightweight PLM.

Crisp: Cognitive Restructuring of Negative Thoughts through Multi-turn Supportive Dialogues (2025.emnlp-main)

Copied to clipboard

Jinfeng Zhou, Yuxuan Chen, Jianing Yin, Yongkang Huang, Yihan Shi, Xikun Zhang, Libiao Peng, Rongsheng Zhang, Tangjie Lv, Zhipeng Hu, Hongning Wang, Minlie Huang

Dialogue Systems & Conversational Agents

Challenge:	Existing approaches to cognitive restructuring (CR) are limited by entrenched cognitive distortions, emotional resistance, and individual differences.
Approach:	They propose a framework that structures CR as theory-grounded multi-stage multi-turn dialogue and a multi-channel loop mechanism to account for diverse individual distortions.
Outcome:	The proposed framework integrates supportive strategies for emotional management and a multi-channel loop mechanism to account for diverse individual distortions.

TextBox: A Unified, Modularized, and Extensible Framework for Text Generation (2021.acl-demo)

Copied to clipboard

Junyi Li, Tianyi Tang, Gaole He, Jinhao Jiang, Xiaoxuan Hu, Puzhao Xie, Zhipeng Chen, Zhuohao Yu, Wayne Xin Zhao, Ji-Rong Wen

Text Generation

Challenge:	TextBox is an open-source text generation framework that is modularized and extensible.
Approach:	They propose to provide a unified, modularized, and extensible text generation framework that implements 21 text generation models on 9 benchmark datasets.
Outcome:	The proposed framework implements 21 models on 9 benchmark datasets and is available under the Apache License 2.0 license.

KuiLeiXi: a Chinese Open-Ended Text Adventure Game (2021.acl-demo)

Copied to clipboard

Yadong Xi, Xiaoxi Mao, Le Li, Lei Lin, Yanjiang Chen, Shuhan Yang, Xuhan Chen, Kailun Tao, Zhi Li, Gongzheng Li, Lin Jiang, Siyan Liu, Zeng Zhao, Minlie Huang, Changjie Fan, Zhipeng Hu

Language Models

Challenge:	Recent advances in pre-trained language models have made it possible to generate human-like text.
Approach:	They propose to integrate an open-ended text adventure game in Chinese, named KuiLeiXi, where players interact with the AI until the plot goals are reached.
Outcome:	The proposed game lacks incentives and relies on players to explore on their own.