Papers by Jipeng Zhang

Paper2Rebuttal: A Multi-Agent Framework for Transparent Author Response Assistance (2026.acl-long)

Challenge:	Recent advances in the field of computer vision have enabled more effective and sophisticated interactions between humans and machines.
Approach:	They propose a reasoning-based object detection paradigm that leverages state-of-the-art multi-modal models and open-vocabulary object detectors to perform reasoning within the context of the user’s instructions and the visual scene.
Outcome:	The proposed method enables users to interact with the system using natural language instructions, allowing for a higher level of interactivity.

Copied to clipboard

Qianli Ma, Chang Guo, Zhiheng Tian, Siyu Wang, Jipeng Xiao, Yuanhao Yue, Zhipeng Zhang

Challenge:	Current approaches to writing effective rebuttals are limited by the direct-to-text generation problem . authors must accurately decipher reviewer intent while ensuring every response is firmly anchored in verifiable manuscript details.
Approach:	They propose a framework that reframes rebuttal generation as an evidence-centric planning task.
Outcome:	The proposed framework outperforms baselines in coverage, faithfulness, and strategic coherence.

UniMath: A Foundational and Multimodal Mathematical Reasoner (2023.emnlp-main)

Copied to clipboard

Zhenwen Liang, Tianyu Yang, Jipeng Zhang, Xiangliang Zhang

The Instinctive Bias: Spurious Images lead to Illusion in MLLMs (2024.emnlp-main)

Challenge:	Existing methods for interpreting and processing diverse mathematical modalities are limited . existing systems are limited in interpreting complex mathematical tasks and implementing them in a multimodal manner.
Approach:	They propose a multimodal mathematical reasoning system that utilizes a fine-tuned T5 model augmented with a variational autoencoder (VAE)-based image tokenizer.
Outcome:	The proposed model achieves state-of-the-art performance on SVAMP, GeoQA, and TableMWP datasets and is generalized on two additional datasets.

Copied to clipboard

Tianyang Han, Qing Lian, Rui Pan, Renjie Pi, Jipeng Zhang, Shizhe Diao, Yong Lin, Tong Zhang

Mitigating the Alignment Tax of RLHF (2024.emnlp-main)

Challenge:	Existing multi-modal large language models (MLLMs) are able to process visual inputs by converting them into visual tokens that share the same latent space as language tokens in LLMs.
Approach:	They propose a benchmark that assesses the visual illusion level given spurious images and a pipeline that converts visual inputs into visual tokens.
Outcome:	The proposed benchmark shows that MLLMs suffer from an instinctive bias to varying degrees when presented with spurious images.

Copied to clipboard

Yong Lin, Hangyu Lin, Wei Xiong, Shizhe Diao, Jianmeng Liu, Jipeng Zhang, Rui Pan, Haoxiang Wang, Wenbin Hu, Hanning Zhang, Hanze Dong, Renjie Pi, Han Zhao, Nan Jiang, Heng Ji, Yuan Yao, Tong Zhang

SceMQA: A Scientific College Entrance Level Multimodal Question Answering Benchmark (2024.acl-short)

Challenge:	Large Language Models (LLMs) acquire a wide range of abilities during pre-training, but aligning LLMs under Reinforcement Learning with Human Feedback (RLHF) can lead to forgetting pretrained abilities, which is also known as the alignment tax.
Approach:	They propose to use a model averaging technique to find the most powerful alignment-forging Pareto front among RLHF algorithms.
Outcome:	The proposed method achieves the strongest alignment-forging Pareto front among competing methods.

Copied to clipboard

Zhenwen Liang, Kehan Guo, Gang Liu, Taicheng Guo, Yujun Zhou, Tianyu Yang, Jiajun Jiao, Renjie Pi, Jipeng Zhang, Xiangliang Zhang

Natural Language Interfaces

Challenge:	SceMQA focuses on core science subjects including Mathematics, Physics, Chemistry, and Biology.
Approach:	They propose to use SceMQA to evaluate multimodal question answering at college entrance level.
Outcome:	The proposed model provides specific knowledge points for each problem and detailed explanations for each answer.

DIDS: Domain Impact-aware Data Sampling for Large Language Model Training (2025.emnlp-main)

Copied to clipboard

Weijie Shi, Jipeng Zhang, Yaguang Wu, Jingzhi Fang, Shibo Zhang, Yao Zhao, Hao Chen, Ruiyuan Zhang, Yue Cui, Jia Zhu, Sirui Han, Jiajie Xu, Xiaofang Zhou

PipeNet: Question Answering with Semantic Pruning over Knowledge Graphs (2024.starsem-1)

Challenge:	Existing approaches for optimizing domain-level sampling strategies struggle with maintaining intra-domain consistency and accurately measuring domain impact.
Approach:	They propose to use a Fisher-Information Matrix-guided metric to measure domain impact to ensure intra-domain consistency and accuracy.
Outcome:	The proposed model achieves 3.4% higher average performance while maintaining comparable training efficiency.

Copied to clipboard

Ying Su, Jipeng Zhang, Yangqiu Song, Tong Zhang

Structured Data in NLP

Challenge:	Existing approaches to utilizing explicit knowledge graphs (KGs) are limited by the number of nodes in the subgraph.
Approach:	They propose a grounding-pruning-reasoning pipeline to prune noisy nodes in subgraphs to improve the efficiency of graph reasoning with KG.
Outcome:	The proposed method reduces computation cost and memory usage while obtaining decent representation of pruned subgraphs.

Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks (2023.findings-emnlp)

Copied to clipboard

Xinsong Zhang, Yan Zeng, Jipeng Zhang, Hang Li

LongVideoAgent: Multi-Agent Reasoning with Long Videos (2026.acl-long)

Challenge:	Existing foundation models can only perform the best in one type of understanding tasks.
Approach:	They propose a method for training a general foundation model, X-FM, using text, image, and image-text data.
Outcome:	The proposed method outperforms existing foundation models on language, vision, and vision-language understanding tasks.

Copied to clipboard

Runtao Liu, Ziyi Liu, Jiaqi Tang, Yue Ma, Renjie Pi, Jipeng Zhang, Qifeng Chen

MWP-BERT: Numeracy-Augmented Pre-training for Math Word Problem Solving (2022.findings-naacl)

Challenge:	a key emerging challenge is robust long video understanding, authors say . current methods compress content into lossy summaries or rely on limited toolsets .
Approach:	They propose a multi-agent framework where a master LLM coordinates a grounding agent and a vision agent to extract targeted textual observations.
Outcome:	The proposed model outperforms strong non-agent baselines on episode-level datasets . the proposed model significantly outperformed existing models on other datasets.

Copied to clipboard

Zhenwen Liang, Jipeng Zhang, Lei Wang, Wei Qin, Yunshi Lan, Jie Shao, Xiangliang Zhang

Semantic Text Processing

Challenge:	Existing work on math word problem solvers replace real numbers with symbolic placeholders to focus on logic reasoning.
Approach:	They propose to inject numerical properties into symbolic placeholders with contextualized representation learning schema to solve number representation dilemma.
Outcome:	The proposed model can solve MWP problems on English and Chinese benchmarks.

Plum: Prompt Learning using Metaheuristics (2024.findings-acl)

Copied to clipboard

Rui Pan, Shuo Xing, Shizhe Diao, Wenhe Sun, Xiang Liu, KaShun Shum, Jipeng Zhang, Renjie Pi, Tong Zhang

Low-Resource NLP

Challenge:	Recent advances in prompt learning have led to a need for general prompt optimization methods.
Approach:	They propose a branch of discrete non-convex optimization methods with over 100 options as a promising approach to prompt learning.
Outcome:	The proposed methods can be used to discover more human-understandable prompts that were previously unknown in reasoning and image generation tasks.

TheoremLlama: Transforming General-Purpose LLMs into Lean4 Experts (2024.emnlp-main)

Copied to clipboard

Ruida Wang, Jipeng Zhang, Yizhen Jia, Rui Pan, Shizhe Diao, Renjie Pi, Tong Zhang

LMFlow: An Extensible Toolkit for Finetuning and Inference of Large Foundation Models (2024.naacl-demo)

Challenge:	a framework for formal proof writing using formal languages like Lean4 is needed to prove mathematical theorems using formal language.
Approach:	They propose a framework that trains a general-purpose LLM to be a Lean4 expert.
Outcome:	The proposed framework achieves cumulative accuracies of 36.48% and 33.61% on MiniF2F-Valid and Test datasets.

Copied to clipboard

Shizhe Diao, Rui Pan, Hanze Dong, KaShun Shum, Jipeng Zhang, Wei Xiong, Tong Zhang

Non-Autoregressive Sentence Ordering (2023.findings-emnlp)

Challenge:	Foundation models have demonstrated a great ability to achieve general human-level intelligence far beyond traditional approaches.
Approach:	They propose a toolkit to simplify the finetuning of general foundation models.
Outcome:	The proposed toolkit simplifies the domain- and task-aware finetuning of general foundation models with limited computing resources.

Copied to clipboard

Yi Bin, Wenhao Shi, Bin Ji, Jipeng Zhang, Yujuan Ding, Yang Yang

Pointing to a Llama and Call it a Camel: On the Sycophancy of Multimodal Large Language Models (2025.emnlp-main)

Challenge:	Existing sentence ordering approaches only leverage unilateral dependencies during decoding and cannot fully explore the semantic dependency between sentences.
Approach:	They propose a non-autoregressive ordering network that explores bilateral dependencies between sentences and predicts sentences for each position in parallel.
Outcome:	The proposed model outperforms existing autoregressive sentence ordering approaches and yields competitive performance compared with the state-of-the-arts.

Copied to clipboard

Renjie Pi, Kehao Miao, Li Peihang, Runtao Liu, Jiahui Gao, Jipeng Zhang, Xiaofang Zhou

Modeling Intra-Relation in Math Word Problems with Different Functional Multi-Head Attentions (P19-1)

Challenge:	Multimodal large language models exhibit a pronounced form of visual sycophantic behavior when they process image inputs.
Approach:	They propose a technique that allows multimodal large language models to engage in reflective reasoning and determine whether a user’s instruction is misleading or corrective.
Outcome:	The proposed model resists misleading instructions but is stubborn even if it is wrong.

Copied to clipboard

Jierui Li, Lei Wang, Jipeng Zhang, Yan Wang, Bing Tian Dai, Dongxiang Zhang

Numerical Reasoning

Challenge:	Several deep learning models have been proposed for solving math word problems (MWPs) but their approaches to capturing features are not specifically designed for MWP.
Approach:	They propose to use a group attention mechanism to extract global features, quantity-related features, quantities-pair features and question-related feature in MWPs.
Outcome:	The proposed approach performs significantly better than previous state-of-the-art methods and boosts performance from 66.9% to 69.5% on Math23K with training-test split, from 65.8% to 66.99% on Math 23K with 5-fold cross-validation and from 69.99% to 76.1% on MAWPS.

Bridge-Coder: Transferring Model Capabilities from High-Resource to Low-Resource Programming Language (2025.findings-acl)

Copied to clipboard

Jipeng Zhang, Jianshu Zhang, Yuanzhe Li, Renjie Pi, Rui Pan, Runtao Liu, Zheng Ziqiang, Tong Zhang

TAGCOS: Task-agnostic Gradient Clustered Coreset Selection for Instruction Tuning Data (2025.findings-naacl)

Challenge:	Large Language Models (LLMs) excel at generating code for high-resource programming languages (HRPLs) however, they struggle significantly with low-resourced programming languages such as D, exacerbating the digital divide.
Approach:	They propose a method to generate LRPL data using LLM's general knowledge, HRPL proficiency, and in-context learning capabilities.
Outcome:	The proposed method improves on R, D, Racket, and Bash, while maintaining the same quality.

Copied to clipboard

Jipeng Zhang, Yaxuan Qin, Renjie Pi, Weizhong Zhang, Rui Pan, Tong Zhang

Semantic Text Processing

Challenge:	Existing methods for instruction tuning are limited due to the increasing volume of instruction datasets and the increased computational costs.
Approach:	They propose to extract a small and highly informative subset of training samples from a large dataset that achieves comparable performance to the full dataset.
Outcome:	The proposed algorithm outperforms other unsupervised methods and achieves comparable performance to the full dataset.

Chinese Idiom Paraphrasing (2023.tacl-1)

Copied to clipboard

Jipeng Qiang, Yang Li, Chaowei Zhang, Yun Li, Yi Zhu, Yunhao Yuan, Xindong Wu

Text Generation

Challenge:	Chinese idioms are hard to understand by children and non-native speakers due to their non-compositionality and metaphorical meaning.
Approach:	They propose a task to rephrase idiom-containing sentences to non-idiomatic ones under the premise of preserving the original sentence’s meaning.
Outcome:	The proposed method has better performance than baselines based on the established dataset.

Compositional Mathematical Encoding for Math Word Problems (2023.findings-acl)

Copied to clipboard

Zhenwen Liang, Jipeng Zhang, Kehan Guo, Xiaodong Wu, Jie Shao, Xiangliang Zhang

Numerical Reasoning

Challenge:	Existing MWP encoders work in a unimodal setting and map problem description to latent representation, then for decoding.
Approach:	They propose a Compositional Math Word Problem Solver which maps problem description to latent representation and decodes it in an interactive way.
Outcome:	Extensive experiments show that the proposed model outperforms state-of-the-art models on public benchmarks.

Analogical Math Word Problems Solving with Enhanced Problem-Solution Association (2022.emnlp-main)

Copied to clipboard

Zhenwen Liang, Jipeng Zhang, Xiangliang Zhang

ScaleBiO: Scalable Bilevel Optimization for LLM Data Reweighting (2025.acl-long)

Challenge:	Analogical reasoning has long been used in mathematical education, as it enables students to apply common relational structures of mathematical situations to solve new problems.
Approach:	They propose to leverage analogical MWPs to advance the solver’s generalization ability across different kinds of MWps.
Outcome:	The proposed model has a stronger generalization ability in solving difficult MWPs due to the analogical learning from easy MWPS.

Copied to clipboard

Rui Pan, Dylan Zhang, Hanning Zhang, Xingyuan Pan, Minrui Xu, Jipeng Zhang, Renjie Pi, Xiaoyu Wang, Tong Zhang

Green & Sustainable NLP

Challenge:	Existing paradigms for bilevel optimization require second-order information, making it difficult to scale them up.
Approach:	They propose a scalable instantiation of a bilevel optimization paradigm for large-scale LLMs by using a memory-efficient training technique.
Outcome:	The proposed paradigm scales to 30B-sized LLMs on 8H100 GPUs.

FIRST: Teach A Reliable Large Language Model Through Efficient Trustworthy Distillation (2024.emnlp-main)

Copied to clipboard

KaShun Shum, Minrui Xu, Jianshu Zhang, Zixin Chen, Shizhe Diao, Hanze Dong, Jipeng Zhang, Muhammad Raza

Responsible & Trustworthy NLP

Challenge:	Experimental results show that a well-calibrated model is more reliable than a fine-tuned model due to “tuning-induced mis-calibration”.
Approach:	They propose a method which utilizes a small portion of teacher’s knowledge to obtain a reliable language model in a cost-efficient way.
Outcome:	The proposed method reduces the computational burden by utilizing teacher's knowledge to obtain a reliable language model in a cost-efficient way.

Graph-to-Tree Learning for Solving Math Word Problems (2020.acl-main)

Copied to clipboard

Jipeng Zhang, Lei Wang, Roy Ka-Wei Lee, Yi Bin, Yan Wang, Jie Shao, Ee-Peng Lim

Numerical Reasoning

Challenge:	Existing tree-based neural models do not capture the relationships and order information among the quantities well.
Approach:	They propose a novel deep learning architecture that combines the merits of the graph-based encoder and tree-based decoder to generate better solution expressions.
Outcome:	The proposed framework outperforms the state-of-the-art on two available datasets significantly.

LegalReasoner: Step-wised Verification-Correction for Legal Judgment Reasoning (2025.acl-long)

Copied to clipboard

Weijie Shi, Han Zhu, Jiaming Ji, Mengze Li, Jipeng Zhang, Ruiyuan Zhang, Jia Zhu, Jiajie Xu, Sirui Han, Yike Guo

ExeSQL: Self-Taught Text-to-SQL Models with Execution-Driven Bootstrapping for SQL Dialects (2025.findings-emnlp)

Challenge:	Existing legal judgment prediction methods struggle with logical errors when conducting complex legal reasoning.
Approach:	They propose a method which enhances LJP reliability through step-wise verification and correction of the reasoning process.
Outcome:	The proposed model significantly improves concordance with court decisions from 72.37 to 80.27 on LLAMA-3.1-70B.

Copied to clipboard

Jipeng Zhang, Haolin Yang, Kehao Miao, Ruiyuan Zhang, Renjie Pi, Jiahui Gao, Xiaofang Zhou

Programming Languages in NLP

Challenge:	Existing text-to-SQL models are limited to SQLite due to dataset limitations . data generated through static prompting is noisy and unreliable, authors say .
Approach:	They propose a text-to-SQL framework with execution-driven, agentic bootstrapping . ExeSQl bridges the dialect gap in text- to-Sql, achieving average improvements .
Outcome:	ExeSQL bridges the dialect gap in text-to-SQl, with average improvements of 15.2%, 10.38%, and 4.49% over GPT-4o on PostgreSQLE, MySQL, and Oracle.

Making RALM Robust to Irrelevant Contexts via Layer Knowledge Guided Attention (2025.findings-acl)

Copied to clipboard

Weijie Shi, Hao Chen, Jiaming Li, Yao Zhao, Yazhong Zhang, Qijin Chen, Jipeng Zhang, Ruiyuan Zhang, Jia Zhu, Jiajie Xu, Xiaofang Zhou

Robustness in NLP

Challenge:	Large language models (LLMs) face factual hallucination and knowledge obsolescence when tackling knowledge-intensive tasks.
Approach:	They propose a layer-knowledge guided attention method which harnesses the layer-wise knowledge of large language models to optimize per-layer attention on useful passages.
Outcome:	The proposed method outperforms existing methods on RALM benchmarks.

Benchmarking Multi-National Value Alignment for Large Language Models (2025.findings-acl)

Copied to clipboard

Chengyi Ju, Weijie Shi, Chengzhong Liu, Jiaming Ji, Jipeng Zhang, Ruiyuan Zhang, Jiajie Xu, Yaodong Yang, Sirui Han, Yike Guo

EAGLE: Expert-Guided Self-Enhancement for Preference Alignment in Pathology Large Vision-Language Model (2025.acl-long)

Challenge:	Existing studies on large language models focus on ethical reviews, failing to capture the diversity of national values.
Approach:	They propose a national value extraction pipeline to efficiently construct value assessment datasets and a model-based model with instruction tagging to process raw data sources.
Outcome:	The proposed benchmark evaluates the alignment of LLMs with the values of five major nations: China, the United States, the UK, France, and Germany.

Copied to clipboard

Meidan Ding, Jipeng Zhang, Wenxuan Wang, Haiqin Zhong, Xiaoqin Wang, Xinheng Lyu, Wenting Chen, Linlin Shen

AI4Reading: Chinese Audiobook Interpretation System Based on Multi-Agent Collaboration (2025.acl-demo)

Challenge:	Recent advances in Large Vision Language Models (LVLMs) show promise for pathological diagnosis, yet their application in clinical settings faces critical challenges of multimodal hallucination and biased responses.
Approach:	They propose a framework that integrates medical expertise into preference alignment.
Outcome:	The proposed framework outperforms existing pathological LVLMs while maintaining pathological accuracy.

Copied to clipboard

Minjiang Huang, Jipeng Qiang, Yi Zhu, Chaowei Zhang, Xiangyu Zhao, Kui Yu