Papers by Zijing Liu

Why Do More Experts Fail? A Theoretical Analysis of Model Merging (2026.acl-long)

Copied to clipboard

Zijing Wang, Xingle Xu, YongKang Liu, Yiqun Zhang, Peiqin Lin, Shi Feng, Daling Wang, Xiaocui Yang, Hinrich Schuetze

Challenge:	Existing methods for model merging struggle to maintain performance gains as the number of merged models increases.
Approach:	They propose a Reparameterized Heavy-Tailed method to extend the merged model’s coverage and enhance performance.
Outcome:	The proposed method extends the merged model’s coverage and enhances performance on 19 benchmarks, including knowledge-intensive and general-purpose tasks.

Integrating Semantics and Neighborhood Information with Graph-Driven Generative Models for Document Retrieval (2021.acl-long)

Copied to clipboard

Zijing Ou, Qinliang Su, Jianxing Yu, Bang Liu, Jingwen Wang, Ruihui Zhao, Changyou Chen, Yefeng Zheng

Information Retrieval

Challenge:	Existing methods for document hashing combine only one of semantics and neighborhood information, lacking a theoretical principle to guide the integration process.
Approach:	They propose to encode neighborhood information with a graph-induced Gaussian distribution and integrate it with generative models.
Outcome:	The proposed model can be trained as efficiently as state-of-the-art methods on benchmark datasets.

Imperfect also Deserves Reward: Multi-Level and Sequential Reward Modeling for Better Dialog Management (2021.naacl-main)

Copied to clipboard

Zhengxu Hou, Bang Liu, Ruihui Zhao, Zijing Ou, Yafei Liu, Xi Chen, Yefeng Zheng

Dialogue Systems & Conversational Agents

Challenge:	Existing research on taskoriented dialog systems mainly includes pipeline and end-to-end methods due to its non-differentiable nature.
Approach:	They propose a multi-level reward modeling approach that factorizes a reward into a three-level hierarchy: domain, act, and slot.
Outcome:	The proposed approach significantly improves performance and speed of training in a wide range of dialog systems.

Rethinking Text-based Protein Understanding: Retrieval or LLM? (2025.emnlp-main)

Copied to clipboard

Juntong Wu, Zijing Liu, He Cao, Li Hao, Bin Feng, Zishan Shu, Ke Yu, Li Yuan, Yu Li

Information Retrieval

Challenge:	Recent studies have focused on integrating protein-related knowledge into large language models through continued pretraining and multi-modal alignment.
Approach:	They propose a retrieval-enhanced method which significantly outperforms fine-tuned LLMs for protein-to-text generation and shows accuracy and efficiency in training-free scenarios.
Outcome:	The proposed method significantly outperforms fine-tuned LLMs for protein-to-text generation and shows accuracy and efficiency in training-free scenarios.

SAMP: A Model Inference Toolkit of Post-Training Quantization for Text Processing via Self-Adaptive Mixed-Precision (2023.emnlp-industry)

Copied to clipboard

Rong Tian, Zijing Zhao, Weijie Liu, Haoyan Liu, Weiquan Mao, Zhe Zhao, Kan Zhou

Challenge:	Existing methods for quantization of models are too complicated and can cause performance damage.
Approach:	They propose a self-adaptive mixed-precision (SAMP) toolkit to automatically control quantization rate by a mixed-presence architecture to balance model accuracy and efficiency.
Outcome:	The proposed toolkit has a higher speedup than PyTorch and FasterTransformer while ensuring the required accuracy.

MoleculeQA: A Dataset to Evaluate Factual Accuracy in Molecular Comprehension (2024.findings-emnlp)

Copied to clipboard

Xingyu Lu, He Cao, Zijing Liu, Shengyuan Bai, Leqing Chen, Yuan Yao, Hai-Tao Zheng, Yu Li

Question Answering

Challenge:	Existing models generate erroneous information and evaluations fail to assess factual correctness of models.
Approach:	They propose to use MoleculeQA to evaluate molecular factual correctness in large language models by organizing molecules into a taxonomy and building QA pairs through human and LLM efforts.
Outcome:	The proposed model improves the factual correctness of generated information and enables the development of new models.

Refining BERT Embeddings for Document Hashing via Mutual Information Maximization (2021.findings-emnlp)

Copied to clipboard

Zijing Ou, Qinliang Su, Jianxing Yu, Ruihui Zhao, Yefeng Zheng, Bang Liu

Semantic Text Processing

Challenge:	Existing unsupervised document hashing methods are mostly established on generative models . due to the difficulties of capturing long dependency structures, these methods rarely model the raw documents directly .
Approach:	They propose to learn hash codes from BERT embeddings by modifying existing models . they use mutual information maximization principle to maximize mutual information .
Outcome:	The proposed method outperforms existing methods learned from BERT embeddings on three benchmark datasets.

Entropy-Based Decoding for Retrieval-Augmented Large Language Models (2025.naacl-long)

Copied to clipboard

Zexuan Qiu, Zijing Ou, Bin Wu, Jingjing Li, Aiwei Liu, Irwin King

Semantic Text Processing

Challenge:	Despite their success, retrieval-augmented LLMs still face the distractibility issue, where the generated responses are negatively influenced by noise from both external and intrinsic knowledge sources.
Approach:	They propose a entropy-based document-parallel ensemble decoding method that prioritizes low-entropies from retrieved documents and incorporates a contrastive decoding mechanism that contrasts the obtained low- and high-entropic ensemble distributions with the high-end internal knowledge across layers.
Outcome:	The proposed method improves on open-domain question answering datasets and shows that it is highly efficient.

Look Within or Beyond? A Theoretical Comparison Between Parameter-Efficient and Full Fine-Tuning (2026.acl-long)

Copied to clipboard

YongKang Liu, Xingle Xu, Ercong Nie, Zijing Wang, Shi Feng, Daling Wang, Qian Li, Hinrich Schuetze

Responsible & Trustworthy NLP

Challenge:	Parameter-Efficient Fine-Tuning (PEFT) is an alternative to Full-Parameter Fine-tuning, but its effectiveness on complex tasks such as reasoning and instruction-following remains unclear.
Approach:	They propose to use PEFT to reduce the number of trainable parameters while freezing the weights of LLMs.
Outcome:	The proposed methods perform well on standard tasks, but weaknesses on complex and adversarial settings call for new directions beyond current paradigms.

PlaM: Training-Free Plateau-Guided Model Merging for Better Visual Grounding in MLLMs (2026.findings-acl)

Copied to clipboard

Zijing Wang, YongKang Liu, Mingyang Wang, Ercong Nie, Deyuan Chen, Zhengjie Zhao, Shi Feng, Daling Wang, Xiaocui Yang, Yifei Zhang, Hinrich Schuetze

Multimodality

Challenge:	Multimodal instruction fine-tuning degrades textual reasoning capability, undermining multimodal performance.
Approach:	They propose a plateau-guided model merging method that selectively injects base language model parameters into MLLMs to mitigate this degradation.
Outcome:	The proposed framework reduces multimodal instruction fine-tuning degradation by incorporating a plateau-guided model merging method into MLLMs.

PRESTO: Progressive Pretraining Enhances Synthetic Chemistry Outcomes (2024.findings-emnlp)

Copied to clipboard

He Cao, Yanjun Shao, Zhiyuan Liu, Zijing Liu, Xiangru Tang, Yuan Yao, Yu Li

Multimodality

Challenge:	Multimodal Large Language Models (MLLMs) have seen growing adoption across various scientific domains.
Approach:	They propose a framework that bridges the molecule-text modality gap by integrating a comprehensive benchmark of pretraining strategies and dataset configurations.
Outcome:	The proposed framework improves multimodal LLMs through cross-modal alignment and multi-graph understanding.

Dynamic Guided and Domain Applicable Safeguards for Enhanced Security in Large Language Models (2025.findings-naacl)

Copied to clipboard

Weidi Luo, He Cao, Zijing Liu, Yu Wang, Aidan Wong, Bin Feng, Yuan Yao, Yu Li

Language Models

Challenge:	Existing defense methods struggle with two key issues: inadequate defense capabilities and over-defensiveness.
Approach:	They propose a multi-agents-based framework that leverages accurate external information to provide an unbiased summary of user intentions and safety response guidance.
Outcome:	Experiments on popular jailbreak attacks and benign datasets show that the proposed framework can enhance LLM's robustness against jailbreaks without compromising its general functionality.

InstructMol: Multi-Modal Integration for Building a Versatile and Reliable Molecular Assistant in Drug Discovery (2025.coling-main)

Copied to clipboard

He Cao, Zijing Liu, Xingyu Lu, Yuan Yao, Yu Li

Multimodality

Challenge:	Large Language Models (LLMs) can attain professional-level proficiency in specific domains through fine-tuning.
Approach:	They propose a multi-modal LLM that aligns molecular structures with natural language via an instruction-tuning approach.
Outcome:	InstructMol surpasses existing models and reduces the gap with specialists in drug discovery tasks.

Parameter-Efficient Fine-Tuning via Circular Convolution (2025.findings-acl)

Copied to clipboard

Aochuan Chen, Jiashun Cheng, Zijing Liu, Ziqi Gao, Fugee Tsung, Yu Li, Jia Li

Language Models

Challenge:	Low-Rank Adaptation (LoRA) has gained popularity for fine-tuning large foundation models, but its intrinsic low-rank characteristic may limit its performance.
Approach:	They propose a low-rank adaptive method that uses low-ranked matrices to represent weight changes.
Outcome:	The proposed method reduces trainable parameters and mitigates heavy memory consumption associated with full delta matrices by sequentially multiplying mathbf A and mathbb B with the activation.