Papers by Pei Yu
REST: Stress Testing Large Reasoning Models by Asking Multiple Problems at Once (2026.acl-long)
Copied to clipboard
| Challenge: | Recent Large Reasoning Models (LRMs) lack a narrow evaluation paradigm . a single-question evaluation setup suffers from two major limitations . |
| Approach: | They propose a stress-testing framework that exposes LRMs to multiple problems simultaneously. |
| Outcome: | The proposed framework outperforms existing models on reasoning benchmarks and state-of-the-art models. |
Tracing the Roots: A Multi-Agent Framework for Uncovering Data Lineage in Post-Training LLMs (2026.acl-long)
Copied to clipboard
Yu Li, Xiaoran Shang, Qizhi Pei, Yun Zhu, Xin Gao, Honglin Lin, Zhanping Zhong, Zhuoshi Pan, Zheng Liu, Xiaoyang Wang, Conghui He, Dahua Lin, Feng Zhao, Lijun Wu
| Challenge: | High-quality post-training data is the primary engine driving LLM capabilities . datasets are often treated as isolated artifacts, overlooking their true developmental context . |
| Approach: | They propose a framework to reconstruct the evolutionary graph of dataset development using data lineage. |
| Outcome: | The proposed framework characterizes domain-specific structural patterns in Math-oriented datasets and general-domain corpora. |
Analytical FFN-to-MoE Restructuring via Activation Pattern Analysis (2026.acl-long)
Copied to clipboard
Zehua Pei, Hui-Ling Zhen, Lancheng Zou, Xianzhi Yu, Wulong Liu, Sinno Jialin Pan, Mingxuan Yuan, Bei Yu
| Challenge: | Large language models (LLMs) are fast but require expensive pre-training . a new approach to scale large language models into MoEs reduces inference costs . |
| Approach: | They propose an analytical post-training framework that rapidly restructures FFNs into sparse MoE architectures using only a small calibration dataset. |
| Outcome: | The proposed framework outperforms existing methods on a small calibration dataset. |
ChartVerse: Scaling Chart Reasoning via Reliable Programmatic Synthesis from Scratch (2026.acl-long)
Copied to clipboard
Zheng Liu, Honglin Lin, Xiaoyang Wang, Xin Gao, Yu Li, Mengzhang Cai, Yun Zhu, Zhanping Zhong, Qizhi Pei, Zhuoshi Pan, Xiaoran Shang, Conghui He, Bin Cui, Wentao Zhang, Lijun Wu
| Challenge: | Existing open-source vision language models lack high-quality training data for chart reasoning . current models are simplistic and repetitive, while associated QA pairs are prone to hallucinations . |
| Approach: | They propose a framework to synthesize complex charts and reliable reasoning data from scratch. |
| Outcome: | Experimental results show that ChartVerse-8B surpasses existing models in QA and difficulty . lack of high-quality training data hampers development of open-source models . |
CipherBank: Exploring the Boundary of LLM Reasoning Capabilities through Cryptography Challenge (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated remarkable capabilities, but their capabilities in cryptographic decryption tasks remain underexplored. |
| Approach: | They propose a benchmark to evaluate the reasoning capabilities of large language models in cryptographic decryption tasks. |
| Outcome: | The proposed benchmark examines the reasoning capabilities of large language models in cryptographic decryption tasks. |
LEMMA: Learning from Errors for MatheMatical Advancement in LLMs (2025.findings-acl)
Copied to clipboard
Zhuoshi Pan, Yu Li, Honglin Lin, Qizhi Pei, Zinan Tang, Wei Wu, Chenlin Ming, H. Vicky Zhao, Conghui He, Lijun Wu
| Challenge: | Existing approaches focus on improving the quality of correct training data, neglecting the value contained in error data, thereby hindering the model’s reflective ability. |
| Approach: | They propose to improve LLM's reasoning ability by learning from error data and a grounded mistake augmentation method to collect representative errors. |
| Outcome: | The proposed model achieves significant performance improvements over other strong models with less than 90k data. |
ThinkAnswer Loss: Balancing Semantic Similarity and Exact Matching for LLM Reasoning Enhancement (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for knowledge distillation use Chain-of-Thought (CoT) and answer pairs, but they lack appropriate supervision signals. |
| Approach: | They propose a framework that decouples CoT and answer supervision . the framework applies semantic similarity constraints while maintaining strict literal matching for the answer . |
| Outcome: | The proposed framework decouples CoT and answer supervision while maintaining strict literal matching for the answer. |
CultureSynth: A Hierarchical Taxonomy-Guided and Retrieval-Augmented Framework for Cultural Question-Answer Synthesis (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Cultural competence is defined as the ability to understand and adapt to multicultural contexts. |
| Approach: | They propose a framework that uses a hierarchical multilingual taxonomy and a Retrieval-Augmented Generation to synthesize culturally relevant question-answer pairs. |
| Outcome: | The proposed framework contains a hierarchical multilingual taxonomy covering 12 primary and 130 secondary topics and a Retrieval-Augmented Generation (RAG)-based methodology leveraging factual knowledge to synthesize culturally relevant question-answer pairs. |
Training Language Model to Critique for Better Refinement (2025.findings-acl)
Copied to clipboard
Tianshu Yu, Chao Xiang, Mingchuan Yang, Pei Ke, Bosi Wen, Cunxiang Wang, Jiale Cheng, Li Zhang, Xinyu Mu, Chuxiong Sun, Minlie Huang
| Challenge: | Large language models (LLMs) have remarkable evaluation and critique capabilities, providing insightful feedback and identifying flaws in various tasks. |
| Approach: | They propose a framework to train critic models using refinement signals to generate feedback loops where critiques guide the model in refining its responses. |
| Outcome: | The proposed framework outperforms traditional methods and open-source models in terms of critique quality and refinement outcomes. |
SynthFix: Adaptive Neuro-Symbolic Code Vulnerability Repair (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) struggle with complex semantic and structural correctness required for automated code repair. |
| Approach: | They propose a hybrid neural-symbolic framework that unifies code synthesis with compiler-informed symbolic feedback to improve LLM-based vulnerability repair. |
| Outcome: | The proposed framework improves code repair accuracy and efficiency over strong SFT and RFT training strategies on the FixJS and CodeFlaws benchmarks. |
Improving Multilingual Sign Language Translation with Automatically Clustered Language Family Information (2025.coling-main)
Copied to clipboard
| Challenge: | Recent research has focused on bilingual translation models, but multilingual sign language translation presents unique challenges due to the diversity of sign languages across nations. |
| Approach: | They propose a method that leverages sign language families to improve MSLT performance. |
| Outcome: | The proposed approach can achieve balance between translation accuracy and computational cost by regulating the number of language families. |
A Strategic Coordination Framework of Small LMs Matches Large LMs in Data Synthesis (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models suffer from high computational costs and environmental inefficiency . smaller LMs are more accessible and sustainable, but their individual capabilities often fall short . a collaborative framework for small LM combines specialized roles to iterative refinement and quality control . |
| Approach: | They propose a framework that aggregates specialized roles across small LMs to iterative refinement and quality control typically achieved by a single large LM. |
| Outcome: | The proposed framework aggregates specialized roles across small LMs to iterative refinement and quality control typically achieved by large LM. |
SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualization (2023.emnlp-main)
Copied to clipboard
Hyunwoo Kim, Jack Hessel, Liwei Jiang, Peter West, Ximing Lu, Youngjae Yu, Pei Zhou, Ronan Bras, Malihe Alikhani, Gunhee Kim, Maarten Sap, Yejin Choi
| Challenge: | a dataset of 1.5 million conversations distilled from everyday spoken situations is limited in scale due to its associated costs. |
| Approach: | They propose to make SODA a publicly available, million-scale high-quality social dialogue dataset . they contextualize social commonsense knowledge from a knowledge graph to distill broad spectrum of social interactions . |
| Outcome: | The proposed dataset is the first publicly available, million-scale high-quality social dialogue dataset. |
Long-context Language Models Fail in Basic Retrieval Tasks Without Sufficient Reasoning Steps (2025.findings-emnlp)
Copied to clipboard
| Challenge: | despite their extensive context window, long-context language models fail in some basic cases . a recent study shows that long-cot methods are not necessary for long-constituency tasks . |
| Approach: | a new study evaluates long-context language models with a large context window . the authors propose a method that can be well addressed with arbitrary reasoning steps . |
| Outcome: | The proposed methods are well addressed with a sufficient number of reasoning steps, guided by specific CoT prompts. |
ItD: Large Language Models Can Teach Themselves Induction through Deduction (2024.acl-long)
Copied to clipboard
| Challenge: | Recent studies have shown that Large Language Models (LLMs) have limited ability to conduct induction. |
| Approach: | They propose a framework to enable LLMs to teach themselves induction through deduction. |
| Outcome: | The proposed framework improves performance on two induction benchmarks and shows that it can be used to teach induction through deduction. |
JointGT: Graph-Text Joint Representation Learning for Text Generation from Knowledge Graphs (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing pre-trained models for knowledgegraph-to-text generation ignore graph structure during encoding and lack elaborate pre-training tasks to explicitly model graph-text alignments. |
| Approach: | They propose a graph-text joint representation learning model called JointGT which incorporates a structure-aware semantic aggregation module into each Transformer layer to preserve the graph structure. |
| Outcome: | The proposed model achieves state-of-the-art performance on various KG-to-text datasets. |
MathFusion: Enhancing Mathematical Problem-solving of LLM through Instruction Fusion (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown impressive progress in mathematical problem-solving . current approaches to enhance mathematical reasoning focus on instance-level modifications . |
| Approach: | They propose a framework that enhances mathematical reasoning through cross-problem instruction synthesis. |
| Outcome: | The proposed framework boosts mathematical reasoning by 18.0 points while maintaining high data efficiency. |
CharacterGLM: Customizing Social Characters with Large Language Models (2024.emnlp-industry)
Copied to clipboard
Jinfeng Zhou, Zhuang Chen, Dazhen Wan, Bosi Wen, Yi Song, Jifan Yu, Yongkang Huang, Pei Ke, Guanqun Bi, Libiao Peng, JiaMing Yang, Xiyao Xiao, Sahand Sabour, Xiaohan Zhang, Wenjing Hou, Yijia Zhang, Yuxiao Dong, Hongning Wang, Jie Tang, Minlie Huang
| Challenge: | Character-based dialogue systems (CharacterDial) allow users to customize social characters for social interactions. |
| Approach: | They will collect a large-scale Chinese corpus of characters with diverse categories and behaviors and develop CharacterGLM models to address these challenges. |
| Outcome: | Experiments show that CharacterGLM outperforms most popular open- and closed-source LLMs and performs comparable to GPT-4. |
MetaLadder: Ascending Mathematical Solution Quality via Analogical-Problem Reasoning Transfer (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Current paradigms generate CoT and answers directly for a given problem, diverging from human problem-solving strategies to some extent. |
| Approach: | They propose a framework that explicitly prompts LLMs to recall and reflect on meta-problems alongside their CoT solutions before addressing the target problem. |
| Outcome: | The proposed framework outperforms standard CoT-based methods on mathematical benchmarks and significantly improves their reasoning accuracy. |
Pre-train and Plug-in: Flexible Conditional Text Generation with Variational Auto-Encoders (2020.acl-main)
Copied to clipboard
| Challenge: | Existing conditional generation models cannot handle emerging conditions due to their joint end-to-end learning fashion. |
| Approach: | They propose a framework for conditional text generation that decouples the text generation module from the condition representation module to allow "one-to-many" conditional generation. |
| Outcome: | The proposed framework decouples the text generation module from the condition representation module to allow “one-to-many” conditional generation. |
Accelerate Parallelizable Reasoning via Parallel Decoding within One Sequence (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for parallelizable reasoning tasks are inefficient, says a new study . generating lengthy reasoning sequences is computationally expensive and time-consuming, says the study authors . |
| Approach: | They propose a method that decodes multiple tokens per forward pass using a tree-like attention mask . their method achieves nearly 100% speedup in decoding while basically maintaining the answer quality . |
| Outcome: | Experimental results show that the method achieves nearly 100% speedup in decoding while maintaining the answer quality. |