Papers by Zhiyuan Shi

On LLM-Based Scientific Inductive Reasoning Beyond Equations (2025.emnlp-main)

Challenge:	Triton is a high-level Python-like programming language for building efficient GPU kernels.
Approach:	They propose a TritonBench benchmark that provides a comprehensive evaluation of Tritonic operators on widely deployed GPUs.
Outcome:	The proposed benchmarks show that current LLMs struggle to generate efficient Triton operators on widely deployed GPUs aligned with industry applications.

Copied to clipboard

Brian S. Lin, Jiaxin Yuan, Zihan Zhou, Shouli Wang, Shuo Wang, Cunliang Kong, Qi Shi, Yuxuan Li, Liner Yang, Zhiyuan Liu, Maosong Sun

Reasoning

Challenge:	Existing research on inductive reasoning models emphasizes rule design without grounding them in specific scenarios.
Approach:	They propose to use LLMs to learn underlying patterns from limited examples in entirely new environments.
Outcome:	The proposed benchmark evaluates the inductive reasoning abilities of large language models in scientific settings.

Enhancing Agent Learning through World Dynamics Modeling (2024.findings-emnlp)

Copied to clipboard

Zhiyuan Sun, Haochen Shi, Marc-Alexandre Côté, Glen Berseth, Xingdi Yuan, Bang Liu

Challenge:	Existing methods assume that large language models have a complete understanding of their environment, overlooking potential gaps in their grasp of actual world dynamics.
Approach:	They propose a framework that discovers world dynamics from a small number of demonstrations, verifies the correctness of these dynamics, and evolves new, advanced dynamics tailored to the current situation.
Outcome:	The proposed framework discovers, verifies, and evolves world dynamics from a small number of demonstrations, and compares the automatically generated dynamics with human-annotated world dynamics.

Program Transfer for Answering Complex Questions over Knowledge Bases (2022.acl-long)

Copied to clipboard

Shulin Cao, Jiaxin Shi, Zijun Yao, Xin Lv, Jifan Yu, Lei Hou, Juanzi Li, Zhiyuan Liu, Jinghui Xiao

Knowledge Representation

Challenge:	Program induction for complex questions over knowledge bases relies on a large number of parallel question-program pairs for the given KB, but the gold program annotations are usually lacking, making learning difficult.
Approach:	They propose an approach to leverage program annotations on rich KBs as external supervision signals to aid program induction for low-resourced KB.
Outcome:	The proposed approach outperforms SOTA methods on ComplexWebQuestions and WebQuestionSP.

Seq1F1B: Efficient Sequence-Level Pipeline Parallelism for Large Language Model Training (2025.naacl-long)

Copied to clipboard

Sun Ao, Weilin Zhao, Xu Han, Cheng Yang, Xinrong Zhang, Zhiyuan Liu, Chuan Shi, Maosong Sun

UltraEval-Audio: A Unified Framework for Comprehensive Evaluation of Audio Foundation Models (2026.acl-demo)

Challenge:	Current PP methods face severe bottlenecks, including pipeline bubbles and memory footprint.
Approach:	They propose a sequence-level one-forward-one-backward (1F1B) PP method for training LLMs on long sequences with high throughput and memory efficiency.
Outcome:	The proposed method achieves 1.14X training throughput with half memory footprint compared to baseline methods . it trains an LLM with 30B parameters on sequences up to 64k tokens using 64X NVIDIA A100 GPUs .

Copied to clipboard

Qundong Shi, Jie Zhou, Biyuan Lin, Junbo Cui, Guoyang Zeng, Yixuan Zhou, Ziyang Wang, Xin Liu, Zhen Luo, Yudong Wang, Zhiyuan Liu

ChartCoder: Advancing Multimodal Large Language Model for Chart-to-Code Generation (2025.acl-long)

Challenge:	Existing evaluation frameworks for audio foundation models are heavily reliant on English, making it difficult to objectively assess models’ performance on Chinese.
Approach:	They propose a unified framework that supports 10 languages, 14 task categories, 24 models, and 36 benchmarks with one-command evaluation and real-time leaderboards.
Outcome:	The proposed framework supports 10 languages, 14 task categories, 24 models, and 36 benchmarks with one-command evaluation and real-time leaderboards.

Copied to clipboard

Xuanle Zhao, Xianzhen Luo, Qi Shi, Chi Chen, Shuo Wang, Zhiyuan Liu, Maosong Sun

HeteroCache: A Dynamic Retrieval Approach to Heterogeneous KV Cache Compression for Long-Context LLM Inference (2026.acl-long)

Challenge:	Existing open-source MLLMs fail to fully capture dense information embedded in charts . current models still face significant challenges in understanding and analyzing visual tasks such as captioning and question answering.
Approach:	They propose a chart-to-code MLLM which leverages Code LLMs as the language backbone to enhance the executability of the generated code.
Outcome:	The proposed model surpasses existing open-source models on chart-to-code benchmarks with only 7B parameters and provides lossless representations that contain all critical details.

Copied to clipboard

Zhiyuan Shi, Qibo Qiu, null Xuefeng, Zhonglin Jiang, Li Yu, Jian Jiang, Xiaofei He, Wenxiao Wang

Information Retrieval

Challenge:	Existing static compression methods suffer from coarse-grained caching and high I/O overhead.
Approach:	They propose a training-free dynamic compression framework that uses a sparse attention mechanism to categorize attention heads based on stability and similarity.
Outcome:	The proposed framework achieves state-of-the-art performance on long-context benchmarks and accelerates decoding by up to 3 compared to the original model with a 224K context.

KBAlign: Efficient Self Adaptation on Specific Textual Knowledge Bases (2025.findings-emnlp)

Copied to clipboard

Zheni Zeng, Yuxuan Chen, Shi Yu, Ruobing Wang, Yukun Yan, Zhenghao Liu, Shuo Wang, Xu Han, Zhiyuan Liu, Maosong Sun

ReactXT: Understanding Molecular “Reaction-ship” via Reaction-Contextualized Molecule-Text Pretraining (2024.findings-acl)

Challenge:	Existing methods for retrieval-augmented generation (RAG) are limited and fine-tuning incurs prohibitive costs of external signals.
Approach:	They propose a self-supervised framework that enhances RAG systems through efficient model adaptation.
Outcome:	The proposed framework achieves 90% of the performance gain obtained through GPT-4-supervised adaptation while relying entirely on self-annotation of much smaller models.

Copied to clipboard

Zhiyuan Liu, Yaorui Shi, An Zhang, Sihang Li, Enzhi Zhang, Xiang Wang, Kenji Kawaguchi, Tat-Seng Chua

Deciphering Digital Detectives: Understanding LLM Behaviors and Capabilities in Multi-Agent Mystery Games (2024.findings-acl)

Challenge:	Molecular-text modeling is an emerging research field that aims to facilitate molecule-relevant tasks with a textual interface and textual knowledge.
Approach:	They propose a new method for reaction-text modeling that uses three types of input contexts to incrementally pretrain LMs.
Outcome:	The proposed method improves experimental procedure prediction and molecule captioning and offers competitive results in retrosynthesis.

Copied to clipboard

Dekun Wu, Haochen Shi, Zhiyuan Sun, Bang Liu

Rethinking Jailbreak Detection of Large Vision Language Models with Representational Contrastive Scoring (2026.acl-long)

Challenge:	In this study, we explore the application of Large Language Models (LLMs) in Jubensha, a Chinese detective role-playing game and a novel area in Artificial Intelligence (AI) driven gaming.
Approach:	They propose to use large language models to foster AI agent development in Jubensha, a Chinese detective role-playing game.
Outcome:	The proposed framework enables AI agents to engage in Jubensha games autonomously.

Copied to clipboard

Peichun Hua, Hao Li, Shanghao Shi, Zhiyuan Yu, Ning Zhang

ReLM: Leveraging Language Models for Enhanced Chemical Reaction Prediction (2023.findings-emnlp)

Challenge:	Large Vision-Language Models (LVLMs) are vulnerable to a growing array of multimodal jailbreak attacks, necessitating a generalizable defense that is efficient for practical deployment.
Approach:	They propose a framework that uses a lightweight projection to separate benign and malicious inputs in safety-critical layers.
Outcome:	The proposed framework enables a simple yet powerful contrastive score that differentiates true malicious intent from mere distribution shift.

Copied to clipboard

Yaorui Shi, An Zhang, Enzhi Zhang, Zhiyuan Liu, Xiang Wang

DeepNote: Note-Centric Deep Retrieval-Augmented Generation (2025.findings-emnlp)

Challenge:	Existing methods for predicting chemical reactions are limited by insufficient training data and inability to utilize textual information.
Approach:	They propose a framework that leverages chemical knowledge encoded in language models to assist GNNs, thereby enhancing the accuracy of real-world chemical reaction predictions.
Outcome:	The proposed framework improves state-of-the-art GNN-based methods across chemical reaction datasets especially in out-of distribution settings.

Copied to clipboard

Ruobing Wang, Qingfei Zhao, Yukun Yan, Daren Zha, Yuxuan Chen, Shi Yu, Zhenghao Liu, Yixuan Wang, Shuo Wang, Xu Han, Zhiyuan Liu, Maosong Sun

Information Retrieval

Challenge:	. - (EN)
Approach:	. - (EN)
Outcome:	. - (EN)

MatPlotAgent: Method and Evaluation for LLM-Based Agentic Scientific Data Visualization (2024.findings-acl)

Copied to clipboard

Zhiyu Yang, Zihan Zhou, Shuo Wang, Xin Cong, Xu Han, Yukun Yan, Zhenghao Liu, Zhixing Tan, Pengyuan Liu, Dong Yu, Zhiyuan Liu, Xiaodong Shi, Maosong Sun

Structure-Aware Language Model Pretraining Improves Dense Retrieval on Structured Data (2023.findings-acl)

Challenge:	Scientific data visualization is an essential process in research, but its use of large language models remains unexplored.
Approach:	They propose a model-agnostic LLM agent framework to automate scientific data visualization tasks.
Outcome:	The proposed framework improves performance of commercial and open-source models.

Copied to clipboard

Xinze Li, Zhenghao Liu, Chenyan Xiong, Shi Yu, Yu Gu, Zhiyuan Liu, Ge Yu

Craw4LLM: Efficient Web Crawling for LLM Pretraining (2025.findings-acl)

Challenge:	Structure Aware Dense Retrieval (SANTA) model encodes user queries and structured data in one universal embedding space for retrieving structured data.
Approach:	They propose to use structured data and unstructured data to encode queries and structured data in one universal embedding space for retrieving structured data.
Outcome:	The proposed model achieves state-of-the-art on code search and product search and conducts convincing results in the zero-shot setting.

Copied to clipboard

Shi Yu, Zhiyuan Liu, Chenyan Xiong

Green & Sustainable NLP

Challenge:	Existing work discards over 90% of the raw data collected from web crawls, highlighting the inefficiency of current web crawlers in collecting LLM pretraining data.
Approach:	They propose a web crawling method that leverages the preference of LLMs as the priority score of the web crawler’s scheduler to obtain high-quality pretraining data.
Outcome:	The proposed method achieves high-quality pretraining data on a web graph containing 900 million webpages from a commercial search engine's index with just 21% URLs crawled.

ClueAnchor: Clue-Anchored Knowledge Reasoning Exploration and Optimization for Retrieval-Augmented Generation (2025.findings-emnlp)

Copied to clipboard

Hao Chen, Yukun Yan, Sen Mei, Wanxiang Che, Zhenghao Liu, Qi Shi, Xinze Li, Yuchun Fan, Pengcheng Huang, Qiushi Xiong, Zhiyuan Liu, Maosong Sun

Reasoning

Challenge:	Existing RAG systems often underutilize the retrieved documents, authors say . they fail to extract and integrate key clues needed to support faithful and interpretable reasoning .
Approach:	a new framework extracts key clues from retrieved content and generates multiple reasoning paths . the framework optimizes the model by selecting the most appropriate reasoning path .
Outcome:	Experiments show that ClueAnchor outperforms baseline RAG frameworks in completeness and robustness.

AHVE-CNER: Aligned Hanzi Visual Encoding Enhance Chinese Named Entity Recognition with Multi-Information (2025.coling-main)

Copied to clipboard

Xuhui Zheng, Zhiyuan Min, Bin Shi, Hao Wang

Information Extraction & Text Mining

Challenge:	Existing glyph-based models neglect the relationship between pictorial elements and radicals for Named Entity Recognition (NER) tasks.
Approach:	They propose a model that integrates multi-source visual and phonetic information of Hanzi . they propose combining pictographic features with radicals to facilitate integration .
Outcome:	The proposed model improves performance on benchmark datasets.

ThinkNote: Enhancing Knowledge Integration and Utilization of Large Language Models via Constructivist Cognition Modeling (2026.findings-eacl)

Copied to clipboard

Zhipeng Xu, Zhenghao Liu, Yukun Yan, Shuo Wang, Shi Yu, Zheni Zeng, Chaojun Xiao, Zhiyuan Liu, Ge Yu, Chenyan Xiong

AutoReproduce: Automatic AI Experiment Reproduction with Paper Lineage (2026.acl-long)

Challenge:	Large Language Models (LLMs) exhibit suboptimal behaviors and inconsistencies when exposed to unfamiliar external information, underscoring their limitations in effectively leveraging such knowledge.
Approach:	They propose a framework that enhances the external knowledge utilization of Large Language Models through a two-stage constructivist cognitive modeling process.
Outcome:	The proposed framework achieves a 10% improvement over baseline methods on various question-answering benchmarks.

Copied to clipboard

Xuanle Zhao, Zilin Sang, Yuxuan Li, Qi Shi, Weilun Zhao, Shuo Wang, Duzhen Zhang, Xu Han, Zhiyuan Liu, Maosong Sun

Challenge:	Efficient reproduction of research papers requires deep domain expertise.
Approach:	They propose a framework that systematically mines implicit knowledge from the cited literature to reproduce experimental code in a complete, end-to-end manner.
Outcome:	The proposed framework surpasses baselines across all metrics and reproduces experimental code in a complete, end-to-end manner.

Fusion-in-T5: Unifying Variant Signals for Simple and Effective Document Ranking with Attention Fusion (2024.lrec-main)

Copied to clipboard

Shi Yu, Chenghao Fan, Chenyan Xiong, David Jin, Zhiyuan Liu, Zhenghao Liu

Information Retrieval

Challenge:	Current document ranking pipelines involve multiple ranking layers to integrate different information step-by-step.
Approach:	They propose a novel re-ranker Fusion-in-T5 which integrates text matching information, ranking features, and global document information into one single unified model via templated-based input and global attention.
Outcome:	The proposed model significantly improves ranking performance over complex cascade pipelines.

ReflectRM: Boosting Generative Reward Models via Self-Reflection within a Unified Judgment Framework (2026.acl-long)

Copied to clipboard

Kai Qin, Liangxin Liu, Yu Liang, Longzheng Wang, null Wangyan, Zhang Yueyang, Long Xia, Zhiyuan Sun, Houde Liu, Daiting Shi

Challenge:	Existing methods for generating reward models focus on outcome-level supervision, neglecting analytical process quality, which constrains their potential.
Approach:	They propose a novel reward model that leverages self-reflection to assess analytical quality and enhance preference modeling.
Outcome:	The proposed model improves performance on four benchmarks and significantly mitigates positional bias.

Augmentation-Adapted Retriever Improves Generalization of Language Models as Generic Plug-In (2023.acl-long)

Copied to clipboard

Zichun Yu, Chenyan Xiong, Shi Yu, Zhiyuan Liu

ConsistRM: Improving Generative Reward Models via Consistency-Aware Self-Training (2026.acl-long)

Challenge:	Prior work on retrieval augmentation fine-tuned the retriever and the LM, making them closely coupled.
Approach:	They propose a generic retrieval plug-in that can be used to fine-tune retrieval augmentation and a LM to learn a user's preferences.
Outcome:	The proposed retriever improves the generalization of large language models on the MMLU and PopQA datasets by learning LM’s preferences from a known source LM .

Copied to clipboard

Yu Liang, Liangxin Liu, Longzheng Wang, null Wangyan, Zhang Yueyang, Long Xia, Zhiyuan Sun, Daiting Shi

Challenge:	ConsistRM is a self-training framework that enables effective and stable GRM training without human annotations.
Approach:	They propose a self-training framework that enables effective and stable GRM training without human annotations.
Outcome:	The proposed framework outperforms vanilla Reinforcement Fine-Tuning (RFT) by 1.5% on five benchmark datasets.

SConU: Selective Conformal Uncertainty in Large Language Models (2025.acl-long)

Copied to clipboard

Zhiyuan Wang, Qingni Wang, Yue Zhang, Tianlong Chen, Xiaofeng Zhu, Xiaoshuang Shi, Kaidi Xu

Partially-Aligned Data-to-Text Generation with Distant Supervision (2020.emnlp-main)

Challenge:	Existing frameworks fail to identify outliers that violate the exchangeability assumption, leading to unbounded miscoverage rates and unactionable prediction sets.
Approach:	They propose a method that implements significance tests to determine whether a given sample deviates from the uncertainty distribution of the calibration set.
Outcome:	The proposed approach facilitates rigorous management of miscoverage rates across single-domain and interdisciplinary contexts, and enhances the efficiency of predictions.

Copied to clipboard

Zihao Fu, Bei Shi, Wai Lam, Lidong Bing, Zhiyuan Liu

Text Generation

Challenge:	Using partially-aligned data is an alternative way of solving the dataset scarcity problem.
Approach:	They propose a task to generate human-readable text for describing some given structured data enabling more interpretability.
Outcome:	The proposed framework outperforms baseline models and validates the feasibility of using partially-aligned data.

ConU: Conformal Uncertainty in Large Language Models with Correctness Coverage Guarantees (2024.findings-emnlp)

Copied to clipboard

Zhiyuan Wang, Jinhao Duan, Lu Cheng, Yue Zhang, Qingni Wang, Xiaoshuang Shi, Kaidi Xu, Heng Tao Shen, Xiaofeng Zhu

Multi-Modal Multi-Granularity Tokenizer for Chu Bamboo Slips (2025.coling-main)

Challenge:	Uncertainty quantification (UQ) in natural language generation tasks remains an open challenge . however, black-box uncertainty measures require investigating with the proliferation of LLMs served via APIs.
Approach:	They propose a conformal uncertainty measure and a method to transform heuristic uncertainty notions into rigorous prediction sets.
Outcome:	Empirical results show that the proposed method outperforms state-of-the-art methods and can provide reliable guarantees for open-ended NLG tasks.

Copied to clipboard

Yingfa Chen, Chenlong Hu, Cong Feng, Chenyang Song, Shi Yu, Xu Han, Zhiyuan Liu, Maosong Sun

Syntactic Text Processing

Challenge:	Using a multi-modal multi-granularity tokenizer, we analyze ancient Chinese scripts . a large proportion of the characters in ancient Chinese are rare or undeciphered .
Approach:	They propose a multi-modal multi-granularity tokenizer specifically designed for ancient Chinese scripts.
Outcome:	The proposed tokenizer improves on the part-of-speech tagging task on the Chu bamboo slip script.

LLM×MapReduce: Simplified Long-Sequence Processing using Large Language Models (2025.acl-long)

Copied to clipboard

Zihan Zhou, Chong Li, Xinyi Chen, Shuo Wang, Yu Chao, Zhili Li, Haoyu Wang, Qi Shi, Zhixing Tan, Xu Han, Xiaodong Shi, Zhiyuan Liu, Maosong Sun

OPEx: A Component-Wise Analysis of LLM-Centric Agents in Embodied Instruction Following (2024.acl-long)

Challenge:	Existing studies have focused on extending the context length of large language models (LLMs) due to their quadratic computational complexity and a lack of high-quality long training examples, most LLMs are trained with a limited window size.
Approach:	They propose a training-free framework that enables large language models to effectively process long texts using a divide-and-conquer strategy for comprehensive document understanding.
Outcome:	The proposed framework outperforms open-source and commercial long-context LLMs and is compatible with several models.

Copied to clipboard

Haochen Shi, Zhiyuan Sun, Xingdi Yuan, Marc-Alexandre Côté, Bang Liu

Visual Data in NLP

Challenge:	Embodied Instruction Following (EIF) is a crucial task in embodied learning . however, there is n'a unified understanding regarding the impact of various components on task performance .
Approach:	They propose a framework that delineates the core components essential for embodied learning tasks . they integrate a multi-agent design into the Planner component of their LLM-centric architecture .
Outcome:	OPEx delineates the core components essential for solving embodied learning tasks . integrating a multi-agent design into the Planner component of the LLM-centric architecture further elevates performance.

ProSparse: Introducing and Enhancing Intrinsic Activation Sparsity within Large Language Models (2025.coling-main)

Copied to clipboard

Chenyang Song, Xu Han, Zhengyan Zhang, Shengding Hu, Xiyu Shi, Kuai Li, Chen Chen, Zhiyuan Liu, Guangli Li, Tao Yang, Maosong Sun

LEGENT: Open Platform for Embodied Agents (2024.acl-demos)

Challenge:	Activation sparsity is a promising paradigm for accelerating model inference . few large language models achieve high activation spar and comparable performance .
Approach:	They propose a method to achieve activation sparsity and acceleration in large language models . they introduce ReLU activation and adopt progressive sparse regularization .
Outcome:	The proposed method achieves high activation sparsity and comparable model performance.

Copied to clipboard

Zhili Cheng, Zhitong Wang, Jinyi Hu, Shengding Hu, An Liu, Yuge Tu, Pengkai Li, Lei Shi, Zhiyuan Liu, Maosong Sun