Papers by Conghui He
REST: Stress Testing Large Reasoning Models by Asking Multiple Problems at Once (2026.acl-long)
Copied to clipboard
| Challenge: | Recent Large Reasoning Models (LRMs) lack a narrow evaluation paradigm . a single-question evaluation setup suffers from two major limitations . |
| Approach: | They propose a stress-testing framework that exposes LRMs to multiple problems simultaneously. |
| Outcome: | The proposed framework outperforms existing models on reasoning benchmarks and state-of-the-art models. |
Tracing the Roots: A Multi-Agent Framework for Uncovering Data Lineage in Post-Training LLMs (2026.acl-long)
Copied to clipboard
Yu Li, Xiaoran Shang, Qizhi Pei, Yun Zhu, Xin Gao, Honglin Lin, Zhanping Zhong, Zhuoshi Pan, Zheng Liu, Xiaoyang Wang, Conghui He, Dahua Lin, Feng Zhao, Lijun Wu
| Challenge: | High-quality post-training data is the primary engine driving LLM capabilities . datasets are often treated as isolated artifacts, overlooking their true developmental context . |
| Approach: | They propose a framework to reconstruct the evolutionary graph of dataset development using data lineage. |
| Outcome: | The proposed framework characterizes domain-specific structural patterns in Math-oriented datasets and general-domain corpora. |
Stop Looking for “Important Tokens” in Multimodal Language Models: Duplication Matters More (2025.emnlp-main)
Copied to clipboard
Zichen Wen, Yifeng Gao, Shaobo Wang, Junyuan Zhang, Qintong Zhang, Weijia Li, Conghui He, Linfeng Zhang
| Challenge: | Vision tokens in multimodal large language models often dominate computational overhead due to excessive length compared to linguistic modality. |
| Approach: | They propose a token pruning method which defines an importance criterion for vision tokens and prunes the unimportant vision token during inference. |
| Outcome: | The proposed method can prune 88.9% of vision tokens while maintaining comparable performance. |
Meta-rater: A Multi-dimensional Data Selection Method for Pre-training Language Models (2025.acl-long)
Copied to clipboard
Xinlin Zhuang, Jiahui Peng, Ren Ma, Yinfan Wang, Tianyi Bai, Xingjian Wei, Qiu Jiantao, Chi Zhang, Ying Qian, Conghui He
| Challenge: | composition of pre-training datasets for large language models remains undisclosed . current methods for evaluating data quality are limited by single-dimensional evaluation or redundancy-focused strategies. |
| Approach: | They propose a multi-dimensional data selection method that integrates dimensions with existing quality metrics through learned optimal weightings. |
| Outcome: | The proposed method doubles convergence speed for 1.3B model models and improves downstream task performance by 3.23%. |
Token Pruning in Multimodal Large Language Models: Are We Solving the Right Problem? (2025.findings-acl)
Copied to clipboard
| Challenge: | Multimodal large language models have shown remarkable performance for cross-modal understanding and generation, yet suffer from severe inference costs. |
| Approach: | They propose to prune redundant tokens in MLLMs to reduce computation and storage costs. |
| Outcome: | The proposed method reduces the computational and storage costs of MLLMs by identifying redundant tokens and pruning them. |
OpenHuEval: Evaluating Large Language Model on Hungarian Specifics (2025.findings-acl)
Copied to clipboard
Haote Yang, Xingjian Wei, Jiang Wu, Noémi Ligeti-Nagy, Jiaxing Sun, Yinfan Wang, Győző Zijian Yang, Junyuan Gao, Jingchao Wang, Bowen Jiang, Shasha Wang, Nanjun Yu, Zihao Zhang, Shixin Hong, Hongwei Liu, Wei Li, Songyang Zhang, Dahua Lin, Lijun Wu, Gábor Prószéky, Conghui He
| Challenge: | Recent advances in Large Language Models (LLMs) represent significant strides toward artificial general intelligence (AGI). |
| Approach: | They introduce OpenHuEval, the first benchmark for LLMs focusing on the Hungarian language and specifics. |
| Outcome: | The framework reveals intrinsic patterns and mechanisms of LLMs in non-English languages, with Hungarian serving as an example. |
ChartVerse: Scaling Chart Reasoning via Reliable Programmatic Synthesis from Scratch (2026.acl-long)
Copied to clipboard
Zheng Liu, Honglin Lin, Xiaoyang Wang, Xin Gao, Yu Li, Mengzhang Cai, Yun Zhu, Zhanping Zhong, Qizhi Pei, Zhuoshi Pan, Xiaoran Shang, Conghui He, Bin Cui, Wentao Zhang, Lijun Wu
| Challenge: | Existing open-source vision language models lack high-quality training data for chart reasoning . current models are simplistic and repetitive, while associated QA pairs are prone to hallucinations . |
| Approach: | They propose a framework to synthesize complex charts and reliable reasoning data from scratch. |
| Outcome: | Experimental results show that ChartVerse-8B surpasses existing models in QA and difficulty . lack of high-quality training data hampers development of open-source models . |
CipherBank: Exploring the Boundary of LLM Reasoning Capabilities through Cryptography Challenge (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated remarkable capabilities, but their capabilities in cryptographic decryption tasks remain underexplored. |
| Approach: | They propose a benchmark to evaluate the reasoning capabilities of large language models in cryptographic decryption tasks. |
| Outcome: | The proposed benchmark examines the reasoning capabilities of large language models in cryptographic decryption tasks. |
Data Whisperer: Efficient Data Selection for Task-Specific LLM Fine-Tuning via Few-Shot In-Context Learning (2025.acl-long)
Copied to clipboard
Shaobo Wang, Xiangqi Jin, Ziming Wang, Jize Wang, Jiajun Zhang, Kaixin Li, Zichen Wen, Zhong Li, Conghui He, Xuming Hu, Linfeng Zhang
| Challenge: | Using fine-tuning on task-specific data is essential for large language models to be effective in specialized tasks. |
| Approach: | They propose a method that leverages few-shot in-context learning with the model to be fine-tuned. |
| Outcome: | The proposed method outperforms existing methods with a 3.1-point improvement and a 7.4 speedup on the Llama-3-8B-Instruct model using just 10% of the dataset. |
LEMMA: Learning from Errors for MatheMatical Advancement in LLMs (2025.findings-acl)
Copied to clipboard
Zhuoshi Pan, Yu Li, Honglin Lin, Qizhi Pei, Zinan Tang, Wei Wu, Chenlin Ming, H. Vicky Zhao, Conghui He, Lijun Wu
| Challenge: | Existing approaches focus on improving the quality of correct training data, neglecting the value contained in error data, thereby hindering the model’s reflective ability. |
| Approach: | They propose to improve LLM's reasoning ability by learning from error data and a grounded mistake augmentation method to collect representative errors. |
| Outcome: | The proposed model achieves significant performance improvements over other strong models with less than 90k data. |
GRAIT: Gradient-Driven Refusal-Aware Instruction Tuning for Effective Hallucination Mitigation (2025.findings-naacl)
Copied to clipboard
Runchuan Zhu, Xinke Jiang, Jiang Wu, Zhipeng Ma, Jiahe Song, Fengshuo Bai, Dahua Lin, Lijun Wu, Conghui He
| Challenge: | Experimental evaluations on open-ended and multiple-choice questions demonstrate GRAIT significantly outperforms existing RAIT methods in the overall performance. |
| Approach: | They propose a framework to reduce the risk of over-refusal and reduce hallucinations by rejecting unknown questions to minimize hallucinism and ensuring correct answers are not rejected. |
| Outcome: | The proposed framework outperforms existing methods on open-ended and multiple-choice questions. |
MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing (2026.acl-industry)
Copied to clipboard
Junbo Niu, Zheng Liu, Zhuangcheng Gu, Bin Wang, Linke Ouyang, Zhiyuan Zhao, Tao Chu, Tianyao He, Fan Wu, Qintong Zhang, Zhenjiang Jin, Guang Liang, Rui Zhang, Wenzheng Zhang, Yuan Qu, Zhifei Ren, Yuefeng Sun, Zirui Tang, Boyu Niu, Yuanhong Zheng, Dongsheng Ma, Ziyang Miao, Hejun Dong, Siyi Qian, Junyuan Zhang, Fangdong Wang, Jingzhou Chen, Xiaomeng Zhao, Liqun Wei, Wei Li, Shasha Wang, RuiLiang Xu, Yuanyuan Cao, Lu Chen, Qianqian Wu, Huaiyu Gu, Lindong Lu, Dechen Lin, null Shenguanlin, Xuanhe Zhou, Linfeng Zhang, Yuhang Zang, Xiaoyi Dong, Jiaqi Wang, Bo Zhang, Lei Bai, Pei Chu, Weijia Li, Jiang Wu, Lijun Wu, Zhenxiang Li, Guangyu Wang, Zhongying Tu, Chao Xu, Kai Chen, Bowen Zhou, Dahua Lin, Wentao Zhang, Conghui He
| Challenge: | Document images are characterized by higher resolutions, denser content, and more complex structural layouts. |
| Approach: | They propose a 1.2B-parameter document parsing vision-language model that decouples layout analysis from local content recognition. |
| Outcome: | The proposed model surpasses general-purpose and domain-specific models on multiple benchmarks while maintaining significantly lower computational overhead. |
SongComposer: A Large Language Model for Lyric and Melody Generation in Song Composition (2025.acl-long)
Copied to clipboard
Shuangrui Ding, Zihan Liu, Xiaoyi Dong, Pan Zhang, Rui Qian, Junhao Huang, Conghui He, Dahua Lin, Jiaqi Wang
| Challenge: | Creating lyrics and melodies in symbolic format requires expert knowledge of melody and an advanced understanding of lyrics. |
| Approach: | They introduce SongComposer, a music-specialized large language model that can create symbolic lyrics and melodies following instructions. |
| Outcome: | The proposed model outperforms existing models in symbolic song composition tasks. |
ProtLLM: An Interleaved Protein-Language LLM with Protein-as-Word Pre-Training (2024.acl-long)
Copied to clipboard
Le Zhuo, Zewen Chi, Minghao Xu, Heyan Huang, Jianan Zhao, Heqi Zheng, Conghui He, Xian-Ling Mao, Wentao Zhang
| Challenge: | Experimental results demonstrate that ProtLLM achieves superior performance against protein-specialized baselines on protein-centric tasks and induces zero-shot and in-context learning capabilities on protein language tasks. |
| Approach: | They propose a cross-modal large language model (LLM) that can handle protein-centric and protein-language tasks by using a dynamic protein mounting mechanism. |
| Outcome: | The proposed model can predict proteins from a vast pool of candidates and can also predict natural language and biological papers. |
Heterogeneous Adaptive Policy Optimization: Tailoring Optimization to Every Token’s Nature (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods that use entropy as a discrete filter or post-hoc regulator are limited in their ability to optimize for reasoning tasks. |
| Approach: | They propose a token-aware algorithm that continuously adapts optimization dynamics based on token-level entropy throughout the entire training process. |
| Outcome: | Extensive experiments on mathematical reasoning, code, and logic tasks across multiple models demonstrate HAPO’s consistent superiority over DAPO. |
Efficient Pretraining Data Selection for Language Models via Multi-Actor Collaboration (2025.acl-long)
Copied to clipboard
Tianyi Bai, Ling Yang, Zhen Hao Wong, Fupeng Sun, Xinlin Zhuang, Jiahui Peng, Chi Zhang, Lijun Wu, Qiu Jiantao, Wentao Zhang, Binhang Yuan, Conghui He
| Challenge: | Efficient data selection is crucial to accelerate the pretraining of language models . limited research has addressed the inherent conflicts between data selection methods . |
| Approach: | They propose a multi-actor collaborative data selection mechanism that prioritizes data based on its specific criterion and updates prioritization rules using the current state of the model. |
| Outcome: | The proposed model accelerates convergence in LM pretraining and achieves an average relative performance gain of 10.5% across multiple language model benchmarks. |
LongWanjuan: Towards Systematic Measurement for Long Text Quality (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing efforts to improve data quality have focused on deduplication and the evaluation of data diversity and difficulty. |
| Approach: | They propose a set of metrics to evaluate the quality of long texts by evaluating three fundamental linguistic dimensions: coherence, cohesion, and complexity. |
| Outcome: | The proposed model improves on long-text tasks with over 160B tokens and categorizes long texts into holistic, aggregated, and chaotic types. |
The Data Frontier for Large Language Models: Selection, Synthesis, and Tools (2026.acl-tutorials)
Copied to clipboard
| Challenge: | acquiring and curating high-quality training data remains a significant bottleneck . acquiring such high-quality data is a key challenge for researchers and practitioners . |
| Approach: | This tutorial provides a comprehensive and practical guide to the state-of-the-art in data research directions for LLMs. |
| Outcome: | The tutorial covers methods for curating the most valuable information from vast, noisy datasets and the synthetic data revolution. |
MuSC: Improving Complex Instruction Following with Multi-granularity Self-Contrastive Training (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for complex instruction-following with elaborate constraints rely on a weaker model, especially GPT-4, limiting their application. |
| Approach: | They propose a Multi-granularity Self-Contrastive Training framework to improve instruction alignment without relying on a stronger model. |
| Outcome: | The proposed framework improves instruction-following with elaborate constraints without external supervision on coarse and fine granularity. |
A Strategic Coordination Framework of Small LMs Matches Large LMs in Data Synthesis (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models suffer from high computational costs and environmental inefficiency . smaller LMs are more accessible and sustainable, but their individual capabilities often fall short . a collaborative framework for small LM combines specialized roles to iterative refinement and quality control . |
| Approach: | They propose a framework that aggregates specialized roles across small LMs to iterative refinement and quality control typically achieved by a single large LM. |
| Outcome: | The proposed framework aggregates specialized roles across small LMs to iterative refinement and quality control typically achieved by large LM. |
Middo: Model-Informed Dynamic Data Optimization for Enhanced LLM Fine-Tuning via Closed-Loop Learning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to improve data quality face limitations in static dataset curation that fail to adapt to evolving model capabilities. |
| Approach: | They propose a self-evolving framework that uses model-aware data selection and context-preserving data refinement to improve LLM performance. |
| Outcome: | The proposed framework improves the quality of seed data and boosts LLM’s performance with improving accuracy by 7.15% on average while maintaining the original dataset scale. |
Benchmarking Chinese Commonsense Reasoning of LLMs: From Chinese-Specifics to Reasoning-Memorization Correlations (2024.acl-long)
Copied to clipboard
| Challenge: | Currently, many benchmarks evaluate the commonsense reasoning of large language models (LLMs), but most are English-based, limiting non-English evaluations. |
| Approach: | They propose to use Chinese commonsense reasoning to evaluate LLMs' commonsensing ability. |
| Outcome: | The proposed benchmark covers both globally known and Chinese-specific commonsense reasoning abilities and can be used as a reference for future research. |
MathFusion: Enhancing Mathematical Problem-solving of LLM through Instruction Fusion (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown impressive progress in mathematical problem-solving . current approaches to enhance mathematical reasoning focus on instance-level modifications . |
| Approach: | They propose a framework that enhances mathematical reasoning through cross-problem instruction synthesis. |
| Outcome: | The proposed framework boosts mathematical reasoning by 18.0 points while maintaining high data efficiency. |
LOCR: Location-Guided Transformer for Optical Character Recognition (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Academic documents are packed with texts, equations, tables, and figures, posing challenges for accurate OCR results. |
| Approach: | They propose a model that integrates location guiding into the transformer architecture during autoregression. |
| Outcome: | The proposed model outperforms existing methods on an original large-scale dataset comprising 53M text-location pairs from 89K academic document pages. |
Condor: Enhance LLM Alignment with Knowledge-Driven Data Synthesis and Refinement (2025.acl-long)
Copied to clipboard
Maosongcao Maosongcao, Taolin Zhang, Mo Li, Chuyu Zhang, Yunxin Liu, Conghui He, Haodong Duan, Songyang Zhang, Kai Chen
| Challenge: | Existing high-quality human-annotated SFT data is a bottleneck for Large Language Models (LLMs). |
| Approach: | They propose a two-stage synthetic data generation framework that incorporates World Knowledge Trees and Self-Reflection Refinement to produce high-quality SFT data at scale. |
| Outcome: | The proposed model fine-tuned on 20K condor-generated samples achieves superior performance compared to instruct model trained with RLHF. |
MetaLadder: Ascending Mathematical Solution Quality via Analogical-Problem Reasoning Transfer (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Current paradigms generate CoT and answers directly for a given problem, diverging from human problem-solving strategies to some extent. |
| Approach: | They propose a framework that explicitly prompts LLMs to recall and reflect on meta-problems alongside their CoT solutions before addressing the target problem. |
| Outcome: | The proposed framework outperforms standard CoT-based methods on mathematical benchmarks and significantly improves their reasoning accuracy. |
LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-Training (2024.emnlp-main)
Copied to clipboard
| Challenge: | Mixture-of-Experts (MoE) has gained increasing popularity as a framework for scaling up large language models. |
| Approach: | They investigate how to build Mixture-of-Experts (MoE) models from existing large language models . they use expert construction, Continual pre-training and data sampling strategies . |
| Outcome: | The proposed model outperforms existing models with similar parameters on a wide range of tasks. |
BenchMAX: A Comprehensive Multilingual Evaluation Suite for Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing multilingual benchmarks focus primarily on language understanding tasks. |
| Approach: | They develop a multi-way multilingual benchmark that measures critical capabilities of large language models across languages. |
| Outcome: | Extensive experiments on BenchMAX reveal uneven utilization of core capabilities across languages, emphasizing the performance gaps that scaling model size alone does not resolve. |