Papers by Zhen Zheng
OceanGPT: A Large Language Model for Ocean Science Tasks (2024.acl-long)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) have transformed the paradigm in ocean science. |
| Approach: | They propose a framework to automatically obtain large volume of ocean domain instruction data, which generates instructions based on multi-agent collaboration. |
| Outcome: | The proposed framework shows a higher level of knowledge expertise for ocean science tasks and gains preliminary embodied intelligence capabilities in ocean technology. |
Enhancing LLM-as-a-Judge through Active-Sampling-based Prompt Optimization (2025.acl-industry)
Copied to clipboard
| Challenge: | Suboptimal prompts can introduce biases, inconsistencies, and unreliable evaluations. |
| Approach: | They propose an active-sampling-based framework for automatic prompt optimization . they use a small, diverse subset of samples to guide prompt refinement . |
| Outcome: | The proposed framework outperforms baselines on four popular LLMs and three real-world datasets. |
AdaZeta: Adaptive Zeroth-Order Tensor-Train Adaption for Memory-Efficient Large Language Models Fine-Tuning (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in memory-efficient zeroth-order methods have limited their widespread adoption due to performance drops and a high risk of divergence. |
| Approach: | They propose a memory-efficient zeroth-order framework to improve performance and convergence of the MeZO methods by using only forward passes. |
| Outcome: | The proposed framework improves performance and convergence of the proposed methods on Roberta-Large and Llama-2-7B models. |
Wanda++: Pruning Large Language Models via Regional Gradients (2025.findings-acl)
Copied to clipboard
Yifan Yang, Kai Zhen, Bhavana Ganesh, Aram Galstyan, Goeric Huybrechts, Markus Müller, Jonas M. Kübler, Rupak Vignesh Swaminathan, Athanasios Mouchtaris, Sravan Babu Bodapati, Nathan Susanj, Zheng Zhang, Jack FitzGerald, Abhishek Kumar
| Challenge: | Existing pruning methods suffer from accuracy degradation without full-model sparsity-aware fine-tuning. |
| Approach: | They propose a pruning framework that uses decoder-block-level regional gradients to improve pruning accuracy. |
| Outcome: | The proposed pruning framework outperforms the state-of-the-art pruning frameworks by utilizing decoder-block-level regional gradients. |
QuZO: Quantized Zeroth-Order Fine-Tuning for Large Language Models (2025.emnlp-main)
Copied to clipboard
Jiajun Zhou, Yifan Yang, Kai Zhen, Ziyue Liu, Yequan Zhao, Ershad Banijamali, Athanasios Mouchtaris, Ngai Wong, Zheng Zhang
| Challenge: | Large Language Models (LLMs) are quantized to lower precision to reduce memory cost and latency in inference. |
| Approach: | They propose a quantized zeroth-order framework for fine-tuning Large Language Models (LLMs) using low-precision forward passes. |
| Outcome: | The proposed method achieves comparable results to first-order methods in FP8 and superior accuracy in INT8 and INT4 training. |
RoBSA: RoPE-based Blockwise Sparse Multi-head Latent Attention (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have advanced in recent years, scaling up in both parameter count and context length. |
| Approach: | They propose a method to compute attention over a subset of context tokens and to implement token selection in a blockwise manner. |
| Outcome: | The proposed method reduces end-to-end inference latency by up to 2.55x with minimal accuracy loss compared to full attention in long-context scenarios for very large models. |
Incremental Sequence Labeling: A Tale of Two Shifts (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to incremental sequence labeling have focused on the E2O and O2E issues, but neglect the O2e issue. |
| Approach: | They propose a framework for incremental sequence labeling without semantic shifts that mitigate catastrophic forgetting in models by using knowledge distillation to maintain the model’s discriminative ability for old entities. |
| Outcome: | The proposed framework mitigates catastrophic forgetting in models while maintaining discriminative ability for old entities while minimizing the model’s bias towards new entities. |
POMP: Probability-driven Meta-graph Prompter for LLMs in Low-resource Unsupervised Neural Machine Translation (2024.acl-long)
Copied to clipboard
| Challenge: | Low-resource languages (LRLs) face challenges in supervised neural machine translation due to limited parallel data. |
| Approach: | They propose a method that uses a dynamic graph to organize auxiliary languages in prompts to improve LRL translations. |
| Outcome: | The proposed method improves translation accuracy in low-resource languages (LRLs) using auxiliary language pairs and synthetic pseudo-parallel data. |
AIGuard: A Benchmark and Lightweight Detection for E-commerce AIGC Risks (2025.findings-acl)
Copied to clipboard
Wenhua Zhang, Weicheng Li, Xuanrong Rao, Lixin Zou, Xiangyang Luo, Chubin Zhuang, Yongjie Hong, Zhen Qin, Hengyu Chang, Chenliang Li, Bo Zheng
| Challenge: | Existing detection methods lack real-world scenarios and corresponding risk datasets . current MLLMs lack knowledge and have limited capability to detect the risk of AIGC content. |
| Approach: | They propose a benchmark for AIGC risk detection in real-world e-commerce . it includes 253,420 image-text pairs across four critical categories . |
| Outcome: | The proposed method achieves 9.68% higher recall than leading multimodal models while using only 25% of training resources. |
CDConv: A Benchmark for Contradiction Detection in Chinese Conversations (2022.emnlp-main)
Copied to clipboard
Chujie Zheng, Jinfeng Zhou, Yinhe Zheng, Libiao Peng, Zhen Guo, Wenquan Wu, Zheng-Yu Niu, Hua Wu, Minlie Huang
| Challenge: | Existing methods for detecting dialogue contradictions are difficult due to contextualization nature of conversations. |
| Approach: | They propose a benchmark for Contradiction Detection in Chinese Conversations . they use automatic conversation generation to simulate common user behaviors . |
| Outcome: | The proposed benchmark simulated the user behaviors that trigger chatbots to make contradictions . the results show that the current state-of-the-art chatbot can be easily goaded into making contradictions. |
EasyInstruct: An Easy-to-use Instruction Processing Framework for Large Language Models (2024.acl-demos)
Copied to clipboard
Yixin Ou, Ningyu Zhang, Honghao Gui, Ziwen Xu, Shuofei Qiao, Runnan Fang, Lei Li, Zhen Bi, Guozhou Zheng, Huajun Chen
| Challenge: | Large Language Models (LLMs) have improved performance across tasks and domains . instruction tuning is a crucial technique to enhance the capabilities of LLMs - but there is no standard open-source instruction processing framework available for the community . |
| Approach: | They propose an open-source instruction tuning framework for Large Language Models that modularizes instruction generation, selection, prompting and their combination and interaction. |
| Outcome: | The proposed framework is open-source and available on Github. |
Saten: Sparse Augmented Tensor Networks for Post-Training Compression of Large Language Models (2025.findings-emnlp)
Copied to clipboard
Ryan Solgi, Kai Zhen, Rupak Vignesh Swaminathan, Nathan Susanj, Athanasios Mouchtaris, Siegfried Kunzmann, Zheng Zhang
| Challenge: | Low-rank tensor compression techniques are used for over-parameterized neural networks, but their applications to compress pre-trained LLMs for downstream tasks remain challenging due to the high-rank nature of pre-training data. |
| Approach: | They propose sparse augmented tensor networks to enhance low-rank tenorized LLMs . they also propose a framework that enables full model compression . |
| Outcome: | The proposed framework improves accuracy and efficiency in tensorized language models. |
MaZO: Masked Zeroth-Order Optimization for Multi-Task Fine-Tuning of Large Language Models (2025.emnlp-main)
Copied to clipboard
Zhen Zhang, Yifan Yang, Kai Zhen, Nathan Susanj, Athanasios Mouchtaris, Siegfried Kunzmann, Zheng Zhang
| Challenge: | Large language models (LLMs) have demonstrated exceptional capabilities across diverse tasks, but their fine-tuning requires significant memory, posing challenges for resource-constrained environments. |
| Approach: | They propose a ZO-based framework that eliminates the need for backpropagation and provides a memory-efficient alternative to backprograming. |
| Outcome: | The proposed framework surpasses first-order methods in performance and accuracy. |
CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark (2022.acl-long)
Copied to clipboard
Ningyu Zhang, Mosha Chen, Zhen Bi, Xiaozhuan Liang, Lei Li, Xin Shang, Kangping Yin, Chuanqi Tan, Jian Xu, Fei Huang, Luo Si, Yuan Ni, Guotong Xie, Zhifang Sui, Baobao Chang, Hui Zong, Zheng Yuan, Linfeng Li, Jun Yan, Hongying Zan, Kunli Zhang, Buzhou Tang, Qingcai Chen
| Challenge: | a new benchmark for biomedical language understanding is being developed in Chinese . most benchmarks are limited to English, which makes it difficult to replicate success in other languages. |
| Approach: | They propose to use Chinese biomedical language understanding evaluation benchmarks to evaluate Chinese models. |
| Outcome: | The proposed benchmarks show that the current models perform worse than the human ceiling. |
GraphNarrator: Generating Textual Explanations for Graph Neural Networks (2025.acl-long)
Copied to clipboard
| Challenge: | Graph representation learning has garnered significant attention due to its broad applications in various domains, such as recommendation systems and social network analysis. |
| Approach: | They propose to use a generative language model to map input-output pairs to explanations reflecting the model’s decision-making process to generate a model that generates pseudo-labels that capture the model's decisions from saliency-based explanations. |
| Outcome: | Extensive experiments show that GraphNarrator produces human-preferred explanations that are faithful, concise, and human-like. |