Papers by Zhiyuan Ding
FLIQA-AD: a Fusion Model with Large Language Model for Better Diagnose and MMSE Prediction of Alzheimer’s Disease (2025.naacl-short)
Copied to clipboard
| Challenge: | Existing classification and regression models that only extract finer-grained information from magnetic resonance imaging (MRI) may not be effective for Alzheimer's disease (AD). |
| Approach: | They propose to use a 3D Adapter in a Vision Transformer to extract the patient's EHR information and questions related to the disease as text prompts. |
| Outcome: | The proposed model can discriminate and predict the corresponding MMSE score based on the extracted brain structural information and textual content . |
Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification (2022.acl-long)
Copied to clipboard
| Challenge: | Recent studies suggest that pre-trained language models have gained rich knowledge during pre-training. |
| Approach: | They propose to tune pre-trained language models with task-specific prompts to improve and stabilize prompttuning. |
| Outcome: | Extensive experiments on zero and few-shot text classification tasks show that prompt-tuning improves and stabilizes prompttun-ing. |
Sparse Low-rank Adaptation of Pre-trained Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning pre-trained large language models in a parameter-efficient manner are gaining traction within the research community. |
| Approach: | They propose a method of low-rank adaptation that enables dynamic adjustments to the intrinsic rank during the adaptation process. |
| Outcome: | The proposed approach outperforms the current method with a fixed and unalterable intrinsic rank and a low-rank adaptation process. |
OpenPrompt: An Open-source Framework for Prompt-learning (2022.acl-demo)
Copied to clipboard
| Challenge: | Prompt-learning is a new paradigm in natural language processing, adapting pre-trained language models to cloze-style prediction, autoregressive modeling, or sequence to sequence generation. |
| Approach: | They propose a framework for prompt-learning that integrates pre-trained language models with a unified framework. |
| Outcome: | The proposed framework is easy to use and flexible enough to integrate with other frameworks. |
ProQA: Structural Prompt-based Pre-training for Unified Question Answering (2022.naacl-main)
Copied to clipboard
Wanjun Zhong, Yifan Gao, Ning Ding, Yujia Qin, Zhiyuan Liu, Ming Zhou, Jiahai Wang, Jian Yin, Nan Duan
| Challenge: | Existing QA research on question answering is focused on specific question types, knowledge domains, or reasoning skills. |
| Approach: | They propose a unified QA paradigm that solves various tasks through a single model. |
| Outcome: | The proposed model improves QA-centric ability on 11 QA benchmarks. |
OpenDelta: A Plug-and-play Library for Parameter-efficient Adaptation of Pre-trained Models (2023.acl-demo)
Copied to clipboard
| Challenge: | Existing implementations that modify the code of the backbone PTMs and hard-code specific delta tuning methods for each PTM have limited the practicality and flexibility of delta tuning. |
| Approach: | They propose an open-source library that provides a plug-and-play implementation of delta tuning methods for pre-trained models. |
| Outcome: | The proposed methods eliminate the need to modify the backbone PTMs’ code, making OpenDelta compatible with different, even novel PTM. |
Few-shot Classification with Hypersphere Modeling of Prototypes (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for fewshot learning use embeddings in space, but they lack expressivity and are difficult to perform statistically. |
| Approach: | They propose a method where class information is represented by hyperspheres with dynamic sizes with two sets of learnable parameters: the hypersphere’s center and the radius. |
| Outcome: | The proposed method is much more expressive than embeddings and performs better than statistical modeling. |
Exploring Lottery Prompts for Pre-trained Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to optimize pre-trained language models are expensive and slow to scale. |
| Approach: | They propose to search for instance-level lottery prompts and generalize them to unseen data . they validate the assumption that for every instance, there is almost always a lottery prompt that induces the correct prediction from the PLM . |
| Outcome: | The proposed method can achieve comparable results with other gradient-free and optimization-free baselines. |
HyKGE: A Hypothesis Knowledge Graph Enhanced RAG Framework for Accurate and Reliable Medical LLMs Responses (2025.acl-long)
Copied to clipboard
Xinke Jiang, Ruizhe Zhang, Yongxin Xu, Rihong Qiu, Yue Fang, Zhiyuan Wang, Jinyi Tang, Hongxin Ding, Xu Chu, Junfeng Zhao, Yasha Wang
| Challenge: | Recent approaches suffer from insufficient and repetitive knowledge retrieval, tedious and time-consuming query parsing, and monotonous knowledge utilization. |
| Approach: | They propose a retrieval-augmented generation framework which leverages LLMs’ powerful reasoning capacity to compensate for the incompleteness of user queries. |
| Outcome: | The proposed framework improves the accuracy and reliability of Large Language Models (LLMs) by combining the rich knowledge of LLMs with Hypothesis Outputs. |
Few-NERD: A Few-shot Named Entity Recognition Dataset (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to few-shot named entity recognition (NER) focus on coarse-grained entities with few examples, while most unseen entities are fine-grounded. |
| Approach: | They present a human-annotated few-shot named entity recognition dataset . they construct benchmark tasks to assess the generalization capability of models . |
| Outcome: | The proposed model is the first few-shot NER dataset and the largest human-crafted NER data set. |
Controllable Preference Optimization: Toward Controllable Multi-Objective Alignment (2024.emnlp-main)
Copied to clipboard
Yiju Guo, Ganqu Cui, Lifan Yuan, Ning Ding, Zexu Sun, Bowen Sun, Huimin Chen, Ruobing Xie, Jie Zhou, Yankai Lin, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing algorithms for achieving optimal alignment are mostly unidirectional . a recent study suggests that large language models can be ground with evident preferences . |
| Approach: | They propose to ground large language models with evident preferences . they propose to use controllable preference optimization to specify different objectives . |
| Outcome: | The proposed models can provide responses that match various preferences among the ”3H” desiderata. |
ITAKE: Interactive Unstructured Text Annotation and Knowledge Extraction System with LLMs and ModelOps (2024.acl-demos)
Copied to clipboard
| Challenge: | Unstructured text data contains a large amount of valuable knowledge, but there are many tools that do not meet the needs of actual business. |
| Approach: | They propose an unstructured text annotation and knowledge extraction system that integrates Large Language Models and ModelOps to improve model supervision and performance. |
| Outcome: | The proposed system integrates large language models and ModelOps to improve performance in low-resource contexts. |
Fusing Highly Specialized Language Models for Comprehensive Expertise (2025.acl-long)
Copied to clipboard
Ning Ding, Yulin Chen, Ganqu Cui, Xingtai Lv, Weilin Zhao, Kaiyan Zhang, Ruobing Xie, Bowen Zhou, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing models that focus on language, programming code, and mathematical symbols are not able to achieve mastery of all three domains simultaneously. |
| Approach: | They propose to fuse highly-specialized models that are already sufficiently trained on different domains to achieve a highly-specific model. |
| Outcome: | The proposed model could achieve mastery of the three crucial domains simultaneously. |
Decoder Tuning: Efficient Language Understanding as Decoding (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to adapt pre-trained models with parameters frozen are based on input-side adaptation, which requires thousands of API queries. |
| Approach: | They propose to train a model-as-a-service (MaaS) setting to provide only the inference APIs for users . they argue that input-side adaptation could be arduous due to the lack of gradient signals . |
| Outcome: | The proposed model outperforms state-of-the-art algorithms with a 200x speed-up. |
Exploring the Impact of Model Scaling on Parameter-Efficient Tuning (2023.emnlp-main)
Copied to clipboard
Yusheng Su, Chi-Min Chan, Jiali Cheng, Yujia Qin, Yankai Lin, Shengding Hu, Zonghan Yang, Ning Ding, Xingzhi Sun, Guotong Xie, Zhiyuan Liu, Maosong Sun
| Challenge: | Parameter-efficient tuning (PET) methods can drive large pre-trained language models by training only minimal parameters. |
| Approach: | They propose a parameter-efficient tuning method that is compatible with a tunable module and uses a random number generator to optimize fewer table parameters. |
| Outcome: | The proposed method is compatible with a tunable module and tested on 11 NLP tasks. |
INTERVENOR: Prompting the Coding Ability of Large Language Models with the Interactive Chain of Repair (2024.findings-acl)
Copied to clipboard
| Challenge: | Experimental results show that INTERVENOR surpasses baseline models, exhibiting improvements of approximately 18% and 4.3% over GPT-3.5 in code generation and code translation tasks. |
| Approach: | They propose a system that prompts Large Language Models to play distinct roles during the code repair process, functioning as both a Code Learner and a code teacher. |
| Outcome: | The proposed system surpasses baseline models in code generation and code translation tasks and improves on syntax errors and assertion errors. |
MAVEN-ERE: A Unified Large-scale Dataset for Event Coreference, Temporal, Causal, and Subevent Relation Extraction (2022.emnlp-main)
Copied to clipboard
Xiaozhi Wang, Yulin Chen, Ning Ding, Hao Peng, Zimu Wang, Yankai Lin, Xu Han, Lei Hou, Juanzi Li, Zhiyuan Liu, Peng Li, Jie Zhou
| Challenge: | Existing datasets only cover limited relation types at once, which prevents models from taking full advantage of relation interactions. |
| Approach: | They construct a large-scale human-annotated ERE dataset with improved annotation schemes to address these drawbacks. |
| Outcome: | The proposed dataset is larger than existing datasets of all the ERE tasks by at least an order of magnitude. |
Parameter-efficient Weight Ensembling Facilitates Task-level Knowledge Transfer (2023.acl-short)
Copied to clipboard
| Challenge: | Recent studies show that large pre-trained language models can be adapted to particular tasks in a parameter-efficient manner. |
| Approach: | They propose to use lightweight parameters to transfer them between tasks to obtain similarity between tasks. |
| Outcome: | The proposed methods show an improvement of 5%8% over baselines and could largely facilitate task-level knowledge transfer. |
The Right Time Matters: Data Arrangement Affects Zero-Shot Generalization in Instruction Tuning (2025.findings-acl)
Copied to clipboard
Bingxiang He, Ning Ding, Cheng Qian, Jia Deng, Ganqu Cui, Lifan Yuan, Haiwen Hong, Huan-ang Gao, Longtao Huang, Hui Xue, Huimin Chen, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing work on instruction tuning has focused on task level, without considering that tasks are artificially defined and, to LLMs, merely consist of tokens and representations. |
| Approach: | They propose a training data arrangement framework that allows for continual learning and loss reduction. |
| Outcome: | The proposed framework promotes continual learning and loss reduction on unseen tasks. |
Prototypical Verbalizer for Prompt-based Few-shot Tuning (2022.acl-long)
Copied to clipboard
| Challenge: | Prompt-based tuning for pre-trained language models has shown its effectiveness in few-shot learning. |
| Approach: | They propose a prototypical verbalizer which learns prototype vectors as verbalizes by contrastive learning. |
| Outcome: | The proposed verbalizer outperforms existing verbalizing methods on topic classification and entity typing tasks. |
Chinese Relation Extraction with Multi-Grained Information and External Linguistic Knowledge (P19-1)
Copied to clipboard
| Challenge: | Existing methods for Chinese relation extraction suffer from segmentation errors and ambiguity of polysemy. |
| Approach: | They propose a multi-grained lattice framework for Chinese relation extraction . they incorporate word-level information into character sequence inputs to avoid segmentation errors . |
| Outcome: | The proposed model outperforms existing models on three real-world datasets in distinct domains. |
WebCPM: Interactive Web Search for Chinese Long-form Question Answering (2023.acl-long)
Copied to clipboard
Yujia Qin, Zihan Cai, Dian Jin, Lan Yan, Shihao Liang, Kunlun Zhu, Yankai Lin, Xu Han, Ning Ding, Huadong Wang, Ruobing Xie, Fanchao Qi, Zhiyuan Liu, Maosong Sun, Jie Zhou
| Challenge: | Long-form question answering requires two procedures: information retrieval and information synthesis. |
| Approach: | They propose a Chinese long-form question answering dataset called WebCPM . the dataset is based on a web search interface that engages with a search engine in real time . |
| Outcome: | The proposed dataset generates answers that are no worse than human-written ones . the dataset is the first Chinese LFQA dataset . |
AutoFigure-Edit: Generating Editable Scientific Illustrations via Reference-Guided Styling (2026.acl-demo)
Copied to clipboard
Zhen Lin, Qiujie Xie, Minjun Zhu, Shichen Li, QiYao Sun, Enhao Gu, Yiran Ding, Ke Sun, Fang Guo, Panzhong Lu, Zhiyuan Ning, Yixuan Weng, Yue Zhang
| Challenge: | Existing automated systems for scientific illustrations are limited in editability, stylistic controllability, and efficiency. |
| Approach: | They propose an end-to-end system that generates fully editable scientific illustrations from long-form scientific text while enabling flexible style adaptation through user-provided reference images. |
| Outcome: | The proposed system generates fully editable scientific illustrations from long-form scientific texts while enabling flexible style adaptation through user-provided reference images. |
Prompt-learning for Fine-grained Entity Typing (2022.findings-emnlp)
Copied to clipboard
Ning Ding, Yulin Chen, Xu Han, Guangwei Xu, Xiaobin Wang, Pengjun Xie, Haitao Zheng, Zhiyuan Liu, Juanzi Li, Hong-Gee Kim
| Challenge: | Extensive experiments on fine-grained entity typing under fully supervised, few-shot, and zero-shot settings show the effectiveness of prompt-learning. |
| Approach: | They propose a prompt-learning pipeline that stimulates versatile knowledge of pre-trained language models (PLMs) by constructing entity-oriented verbalizers and templates and conducting masked language modeling. |
| Outcome: | The proposed approach can be applied to fine-grained entity typing in fully supervised, few-shot, and zero-shot scenarios. |
UltraLink: An Open-Source Knowledge-Enhanced Multilingual Supervised Fine-tuning Dataset (2024.acl-long)
Copied to clipboard
Haoyu Wang, Shuo Wang, Yukun Yan, Xujia Wang, Zhiyu Yang, Yuzhuang Xu, Zhenghao Liu, Liner Yang, Ning Ding, Xu Han, Zhiyuan Liu, Maosong Sun
| Challenge: | Open-source large language models (LLMs) have gained strength across diverse fields, but the majority of studies focus on English. |
| Approach: | They propose a knowledge-grounded data augmentation approach to elicit more language-specific knowledge of LLMs by enhancing their ability to serve users from different countries. |
| Outcome: | The proposed method can prune the language-agnostic supervised fine-tuning dataset without any performance degradation. |
Enhancing Chat Language Models by Scaling High-quality Instructional Conversations (2023.emnlp-main)
Copied to clipboard
| Challenge: | a recent study validates the effectiveness of chat language models by fine-tuning instruction data. |
| Approach: | They propose to use a large-scale dataset of instructional conversations to fine-tune a conversational model on instruction data. |
| Outcome: | The proposed model outperforms open-source models in key metrics including scale, average length, diversity, coherence, etc. |
Event Detection with Trigger-Aware Lattice Neural Network (D19-1)
Copied to clipboard
| Challenge: | Event detection is a key part of event extraction, but there are two issues with word-based models in languages without natural delimiters, such as Chinese. |
| Approach: | They propose a framework that can solve the problem of word- trigger mismatch . they also use an external knowledge base to model polysemous characters and words . |
| Outcome: | The proposed model outperforms state-of-the-art methods on two benchmark datasets and outperformed previous state- of-the art methods significantly. |
Large Language Models Are Still Misled by Simple Bias Ensembles (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks for large language models are constrained to datasets where each sample is manually injected with only one type of bias. |
| Approach: | They propose a multi-bias benchmark where each sample contains multiple types of biases. |
| Outcome: | The proposed benchmark shows that existing LLMs and debiasing methods perform poorly on this benchmark, highlighting the challenge of eliminating compounded biases. |
Different Tunes Played with Equal Skill: Exploring a Unified Optimization Subspace for Parameter-Efficient Tuning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing delta tuning algorithms freeze most of the parameters and only optimize minimal adaptive parameters. |
| Approach: | They propose to decompose DETs into a unified optimization subspace and conduct optimization within the subspace. |
| Outcome: | The proposed DETs achieve comparable performance to the original DET and can be transferred to another DET with non-trivial performance. |
Aggregation of Reasoning: A Hierarchical Framework for Enhancing Answer Selection in Large Language Models (2024.lrec-main)
Copied to clipboard
Zhangyue Yin, Qiushi Sun, Qipeng Guo, Zhiyuan Zeng, Xiaonan Li, Tianxiang Sun, Cheng Chang, Qinyuan Cheng, Ding Wang, Xiaofeng Mou, Xipeng Qiu, Xuanjing Huang
| Challenge: | Recent advances in Chain-of-Thought prompting have facilitated significant breakthroughs for Large Language Models (LLMs) in complex reasoning tasks. |
| Approach: | They propose a hierarchical reasoning aggregation framework to address this problem . they propose dynamic sampling to adjust the number of reasoning chains . |
| Outcome: | The proposed framework outperforms existing ensemble methods on complex reasoning tasks. |