Papers by Siheng Xiong
CausalEval: Towards Better Causal Reasoning in Language Models (2025.naacl-long)
Copied to clipboard
Longxuan Yu, Delin Chen, Siheng Xiong, Qingyang Wu, Dawei Li, Zhikai Chen, Xiaoze Liu, Liangming Pan
| Challenge: | Large language models (LLMs) have been used for a variety of tasks, including problem-solving, decision-making, and understanding of the world. |
| Approach: | They propose a review of existing methods aimed at enhancing LMs for causal reasoning . they categorize existing methods as reasoning engines or as helpers providing knowledge or data to traditional methods . |
| Outcome: | The proposed methods perform better than existing methods on a range of tasks. |
Harnessing the Power of Large Language Models for Natural Language to First-Order Logic Translation (2024.acl-long)
Copied to clipboard
| Challenge: | Logic-based approaches to reasoning have lost popularity due to limited scalability and coverage. |
| Approach: | They present a dataset of 28K sentence-level NL-FOL pairs from GPT4 and a LogicLLaMA2-7B/13B fine-tuned on MALLS for NL translation. |
| Outcome: | The proposed model can be used standalone or to correct previously generated rules by GPT3.5. |
Reinforcement Learning on Pre-Training Data (2026.acl-long)
Copied to clipboard
Siheng Li, Kejiao Li, Zenan Xu, Guanhua Huang, Kun Li, Haoyuan Wu, null Wujiajia, Zihao Zheng, Chenchen Zhang, Kun Shi, Xue Gong, Qi Yi, Ruibin Xiong, Tingqiang Xu, Yuhao Jiang, Jianfeng Yan, Yuyuan Zeng, Guanghui Xu, Jinbao Xue, Zhijiang xu, Zheng Fang, Shuai LI, Qibin Liu, Xiaoxue Li, Zhuoyu Li, Yangyu Tao, Fei Gao, Cheng Jiang, Bochao Wang, Kai Liu, Jianchen Zhu, Wai Lam, Bo Zhou, Di Wang
| Challenge: | Recent progress in large language models is driven by scaling of training compute through pre-training with nexttoken prediction (NTP) or post-training (RL) Pre-training using NTP enables models to acquire extensive knowledge and skills from general data, but it suffers from data inefficiency and catastrophic forgetting in continual learning settings. |
| Approach: | They propose to scale training compute through pre-training with next-token prediction (NTP) or post-training by scaling reinforcement learning (RL) to improve learning from general data. |
| Outcome: | Experiments on multiple benchmarks and models show that the proposed approach improves continual pre-training and provides a strong foundation for post-training on Qwen3-8B-Base. |
Large Language Models Can Learn Temporal Reasoning (2024.acl-long)
Copied to clipboard
| Challenge: | Temporal reasoning (TR) is a fundamental ability of large language models (LLMs) however, there is neo-standard methods to perform TR, which are not suitable for large language model applications. |
| Approach: | They propose a framework to enhance temporal reasoning by using a latent representation, temporal graph (TG) instead of reasoning over the original context, they adopt a temporal representation that enhances TR learning. |
| Outcome: | The proposed framework improves the learning of language-based TR by incorporating a latent representation, temporal graph (TG) a synthetic dataset is constructed for fine-tuning LLMs on text-to-TG translation tasks and benchmarks. |
Low-probability Tokens Sustain Exploration in Reinforcement Learning with Verifiable Reward (2026.findings-acl)
Copied to clipboard
Guanhua Huang, Tingqiang Xu, Mingze Wang, Qi Yi, Xue Gong, Siheng Li, Ruibin Xiong, Kejiao Li, Yuhao Jiang, Bo Zhou
| Challenge: | Recent studies show that RLVR training is slow and results plateau as policy entropy collapses . low-probability regularization (Lp-Reg) reduces the number of low-quality exploratory tokens induced by RL training . |
| Approach: | They propose a method to reduce RLVR over-penalization by eliminating low-probability exploratory tokens . they propose 'Low-provability Regularization' to reduce the gradual elimination of low-quality exploratory entropy tokens. |
| Outcome: | The proposed method eliminates low-probability exploratory tokens and prevents suppression of potentially valuable low-property candidates. |
Text2Mem: A Unified Memory Operation Language for Memory Operating System (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing memory frameworks lack a formal, executable specification for memory control. |
| Approach: | They propose a unified memory operation language that standardizes translation of natural-language instructions into reliable execution. |
| Outcome: | The proposed language standardizes translation of natural-language instructions into reliable execution. |
Can LLMs Reason in the Wild with Programs? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models have shown superior capability to solve reasoning problems with programs. |
| Approach: | They propose a task where an LLM is tasked to solve a reasoning problem of unknown type by identifying the sub-problems and their corresponding formalisms. |
| Outcome: | The proposed model can be fine tuned to achieve better performance on ambiguous and mixed scope problems. |
Deliberate Reasoning in Language Models as Structure-Aware Planning with an Accurate World Model (2025.acl-long)
Copied to clipboard
| Challenge: | Existing Chain-of-Thought (CoT) methods struggle with consistency and verification in complex reasoning tasks. |
| Approach: | They propose a framework that integrates structured knowledge representation with learned planning. |
| Outcome: | The proposed framework outperforms existing Chain-of-Thought (CoT) methods on math reasoning, logical reasoning, and coding tasks. |