Papers by Ziqing Yang

Gradient-based Intra-attention Pruning on Pre-trained Language Models (2023.acl-long)

Copied to clipboard

Ziqing Yang, Yiming Cui, Xin Yao, Shijin Wang

Challenge:	Pre-trained language models are computationally expensive and slow in inference due to their large sizes.
Approach:	They propose a structured pruning method which combines pruning with knowledge distillation to yield highly effective models.
Outcome:	The proposed method outperforms other pruning methods in sparsity regimes while maintaining 93% 99% performance.

TextPruner: A Model Pruning Toolkit for Pre-Trained Language Models (2022.acl-demo)

Copied to clipboard

Ziqing Yang, Yiming Cui, Zhigang Chen

Language Models

Challenge:	Large pre-trained language models have been used for many NLP tasks but computational resources are limited.
Approach:	They propose an open-source model pruning toolkit for pre-trained language models . they propose a self-supervised pruning method that can be applied without labeled data.
Outcome:	The proposed pruning method reduces model size without retraining the model and speeds up inference speed on the common CPU and GPU devices.

JailbreakRadar: Comprehensive Assessment of Jailbreak Attacks Against LLMs (2025.acl-long)

Copied to clipboard

Junjie Chu, Yugeng Liu, Ziqing Yang, Xinyue Shen, Michael Backes, Yang Zhang

Robustness in NLP

Challenge:	Large language models (LLMs) have been used to mitigate misuse and to align with human values.
Approach:	They propose to use large-scale evaluations of various jailbreak attacks to identify key patterns and test them under eight advanced defenses.
Outcome:	The proposed attacks achieve high success rates but are easy to mitigate by defenses.

IDOL: Indicator-oriented Logic Pre-training for Logical Reasoning (2023.findings-acl)

Copied to clipboard

Zihang Xu, Ziqing Yang, Yiming Cui, Shijin Wang

Language Models

Challenge:	Existing systems for logical reasoning have surpassed the average performance of humans in many tasks like SQuAD but there is still a long way to go when it comes to logical reasoning.
Approach:	They propose an InDicator-Oriented Logic Pre-training task which logically strengthens pre-trained models with the help of 6 types of logical indicators and a logicalally rich dataset.
Outcome:	The proposed task achieves state-of-the-art on ReClor and LogiQA, the two most representative benchmarks in logical reasoning MRC.

UniKER: A Unified Framework for Combining Embedding and Definite Horn Rule Reasoning for Knowledge Graph Inference (2021.emnlp-main)

Copied to clipboard

Kewei Cheng, Ziqing Yang, Ming Zhang, Yizhou Sun

Structured Data in NLP

Challenge:	Knowledge graph inference has been studied extensively due to its wide applications.
Approach:	They propose a framework that restricts logical rules to be definite Horn rules and can exploit the knowledge in logical rule-based reasoning and KGE in an extremely efficient way.
Outcome:	The proposed framework can exploit the knowledge in logical rules and improve KGE in an extremely efficient way.

Adversarial Training for Machine Reading Comprehension with Virtual Embeddings (2021.starsem-1)

Copied to clipboard

Ziqing Yang, Yiming Cui, Chenglei Si, Wanxiang Che, Ting Liu, Shijin Wang, Guoping Hu

Reasoning

Challenge:	Neural networks are vulnerable to adversarial examples that have been mixed with certain perturbations.
Approach:	They propose a novel adversarial training method that perturbs the embedding matrix instead of word vectors to differentiate the roles of passages and questions.
Outcome:	The proposed method is effective universally and further improves the performance of MRC tasks.

CINO: A Chinese Minority Pre-trained Language Model (2022.coling-1)

Copied to clipboard

Ziqing Yang, Zihang Xu, Yiming Cui, Baoxin Wang, Min Lin, Dayong Wu, Zhigang Chen

Multilinguality

Challenge:	Existing multilingual pre-trained language models do not perform well on some low-resource languages.
Approach:	They propose a multilingual pre-trained language model for Chinese minority languages . they collect documents from Wikipedia and construct two classification datasets .
Outcome:	The proposed model outperforms baseline models on various classification tasks.

A Sentence Cloze Dataset for Chinese Machine Reading Comprehension (2020.coling-main)

Copied to clipboard

Yiming Cui, Ting Liu, Ziqing Yang, Zhipeng Chen, Wentao Ma, Wanxiang Che, Shijin Wang, Guoping Hu

Reasoning

Challenge:	Using cloze-style reading comprehension, Chinese machine reading comprehension datasets are becoming more and more popular . a new task is proposed to fill the right candidate sentence into the passage with several blanks .
Approach:	They propose a Chinese task to fill the right candidate sentence into a passage with blanks . they build a dataset to evaluate the difficulty of the task and make fake candidates .
Outcome:	The proposed task fills the right candidate sentence into the passage with blanks . the proposed dataset contains over 100K blanks within over 10K passages based on Chinese narrative stories .

Consultant Decoding: Yet Another Synergistic Mechanism (2025.findings-acl)

Copied to clipboard

Chuanghao Ding, Jiaping Wang, Ziqing Yang, Xiaoliang Wang, Dahua Lin, Nguyen Cam-Tu, Fei Tan

Language Models

Challenge:	Large language models (LLMs) have attracted widespread attention and adoption across diverse domains due to their exceptional performance and robust generalization abilities.
Approach:	They propose a synergetic mechanism for Consultant Decoding (CD) that achieves a 2.5-fold increase in inference speed compared to the target model while maintaining comparable generation quality.
Outcome:	The proposed mechanism achieves 2.5-fold increase in inference speed while maintaining comparable generation quality (100% of the target model’s performance).

TextBrewer: An Open-Source Knowledge Distillation Toolkit for Natural Language Processing (2020.acl-demos)

Copied to clipboard

Ziqing Yang, Yiming Cui, Zhipeng Chen, Wanxiang Che, Ting Liu, Shijin Wang, Guoping Hu

Language Models

Challenge:	Large pre-trained language models have hundreds of millions of parameters and take several gigabytes of memory to train and inference.
Approach:	They propose an open-source knowledge distillation toolkit designed for natural language processing that provides a set of predefined distillation methods and can be extended with custom code.
Outcome:	The proposed method is comparable with or even higher than the public distilled BERT models with similar numbers of parameters.

Peering Behind the Shield: Guardrail Identification in Large Language Models (2026.findings-acl)

Copied to clipboard

Ziqing Yang, Yixin Wu, Rui Wen, Michael Backes, Yang Zhang

Language Models

Challenge:	Identifying guardrails in conversational AI agents is critical for identifying malicious content . identifying guardrail components in black-box AI agents poses security challenges .
Approach:	They propose a method that leverages guard-specific adversarial prompts to detect guardrails in black-box AI agents.
Outcome:	The proposed method achieves perfect classification accuracy in multiple scenarios.

Benchmarking Robustness of Machine Reading Comprehension Models (2021.findings-acl)

Copied to clipboard

Chenglei Si, Ziqing Yang, Yiming Cui, Wentao Ma, Ting Liu, Shijin Wang

Robustness in NLP

Challenge:	Existing benchmarks only evaluate models' robustness under test-time perturbations or adversarial attacks.
Approach:	They propose a model-agnostic benchmark to evaluate models' robustness under adversarial attacks.
Outcome:	The proposed model-agnostic benchmark evaluates models under four different types of adversarial attacks.

More Data or Better Data? A Critical Analysis of Data Selection and Synthesis for Mathematical Reasoning (2025.emnlp-industry)

Copied to clipboard

Yike Zhao, Simin Guo, Ziqing Yang, Shifan Han, Dahua Lin, Fei Tan

Reasoning

Challenge:	Despite various proposed data construction methods, their practical utility in real-world pipelines remains underexplored.
Approach:	They conduct a comprehensive analysis of open-source datasets and data synthesis techniques for mathematical reasoning under a unified pipeline designed to mirror training and deployment scenarios.
Outcome:	The proposed pipelines mirror training and deployment scenarios and are suitable for industrial applications.

The ELCo Dataset: Bridging Emoji and Lexical Composition (2024.lrec-main)

Copied to clipboard

Zi Yun Yang, Ziqing Zhang, Yisong Miao

Visual Data in NLP

Challenge:	Emoji-Lexical Composition dataset provides parallel annotations of emoji sequences corresponding to English phrases.
Approach:	They propose a dataset that offers parallel annotations of emoji sequences corresponding to English phrases.
Outcome:	The Emoji-Lexical Composition (ELCo) dataset offers parallel annotations of emoji sequences corresponding to English phrases.

PeerCheck: Enhancing LLM-Generated Academic Reviews Towards Human-Level Quality (2026.findings-acl)

Copied to clipboard

Zeyuan Chen, Ziqing Yang, Yihan Ma, Michael Backes, Yang Zhang

Challenge:	Increasing use of large language models (LLMs) in academic review has raised concerns about quality and fairness.
Approach:	They propose a framework to improve the quality of LLM-generated reviews by using retrieval-augmented generation.
Outcome:	The proposed framework improves the human-level quality of LLM-generated reviews by adopting prompt engineering and retrieval-augmented generation.