Papers by Guangyue Peng
Two Pathways to Truthfulness: On the Intrinsic Encoding of LLM Hallucinations (2026.acl-long)
Copied to clipboard
Wen Luo, Guangyue Peng, Wei Li, Shaohang Wei, Feifan Song, Liang Wang, Nan Yang, Xingxing Zhang, Jing Jin, Furu Wei, Houfeng Wang
| Challenge: | Previous work shows that large language models generate hallucinations, yet the origins and mechanisms of these signals remain unclear. |
| Approach: | They propose to validate and disentangle two different pathways for truthfulness cues . they also propose to use the same mechanism to derive self-contained evidence from the generated answer . |
| Outcome: | The proposed applications improve hallucination detection performance by integrating two different inputs. |
Encode Errors: Representational Retrieval of In-Context Demonstrations for Multilingual Grammatical Error Correction (2025.findings-acl)
Copied to clipboard
| Challenge: | a novel method for encoding fine-grained error patterns improves performance on GEC. |
| Approach: | They propose a method for encoding grammatical errors from LLMs' internal states using a GER method. |
| Outcome: | The proposed method significantly boosts performance in ICL settings on multilingual GEC datasets. |
Explanation based In-Context Demonstrations Retrieval for Multilingual Grammatical Error Correction (2025.naacl-long)
Copied to clipboard
| Challenge: | Grammatical error correction (GEC) aims to correct grammatical, spelling, and semantic errors in natural language text. |
| Approach: | They propose a retrieval method based on natural language grammatical error explanations to match inputs with pre-constructed databases where explanations for erroneous samples are generated by LLMs. |
| Outcome: | The proposed method outperforms existing semantic and BM25-based retrieval techniques without additional training or language adaptation. |
PACE: Prefix-Protected and Difficulty-Aware Compression for Efficient Reasoning (2026.findings-acl)
Copied to clipboard
Ruixiang Feng, Yuntao Wen, Silin Zhou, Ke Shi, Yifan Wang, Ran Le, Zhenwei An, Zongchao Chen, Chen Yang, Guangyue Peng, Yiming Jia, Dongsheng Wang, Tao Zhang, Lisi Chen, Yang Song, Shen Gao, Shuo Shang
| Challenge: | Existing LRMs often suffer from "overthinking" and excessively long reasoning traces . a dual-level framework for length compression of LRM is proposed . |
| Approach: | They propose a framework for prefix-protected and difficulty-aware compression under hierarchical supervision. |
| Outcome: | The proposed framework reduces token usage while improving accuracy on math benchmarks. |
Odysseus Navigates the Sirens’ Song: Dynamic Focus Decoding for Factual and Diverse Open-Ended Text Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing decoding methods struggle to balance factuality and diversity . Deterministic decoding approaches suffer from degeneration and lack of diversity - a problem that is not addressed by the current literature. |
| Approach: | They propose a plug-and-play stochastic approach that adjusts decoding focus based on distributional differences across layers, leveraging the modular nature of factual knowledge within LLMs. |
| Outcome: | Extensive experiments on seven datasets show that DFD significantly improves performance. |
FEA-Bench: A Benchmark for Evaluating Repository-Level Code Generation for Feature Implementation (2025.acl-long)
Copied to clipboard
Wei Li, Xin Zhang, Zhongxin Guo, Shaoguang Mao, Wen Luo, Guangyue Peng, Yangyu Huang, Houfeng Wang, Scarlett Li
| Challenge: | Existing benchmarks focus on standalone programming problems, such as HumanEval, MBPP, and LiveCodeBench. |
| Approach: | They propose to use large language models to evaluate their ability to perform incremental development within code repositories by collecting pull requests from 83 GitHub repositorias and using rule-based and intent-based filtering to construct task instances focused on new feature development. |
| Outcome: | The proposed benchmarks show that large language models perform significantly worse in the FEA-Bench, highlighting considerable challenges in repository-level incremental code development. |
Learn to Memorize: Scalable Continual Learning in Semiparametric Models with Mixture-of-Neighbors Induction Memory (2025.acl-long)
Copied to clipboard
| Challenge: | Semiparametric language models (LMs) use static storage, which lacks learning capability and is disconnected from the internal information flow of the parametric models. |
| Approach: | They reconceptualize the non-parametric memory represented by kNN-LM as a learnable Mixture-of-Neighbors Induction Memory (MoNIM) this synergizes the induction capabilities of attention heads with the memorization strength of feed-forward networks . |
| Outcome: | The proposed model is a learnable Mixture-of-neighbors induction memory (MoNIM) it synergizes the induction capabilities of attention heads with the memorization strength of feed-forward networks (FFNs). |