Papers by Xiaozhe Li
Timely Machine: Awareness of Time Makes Test-Time Scaling Agentic (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) tackle complex reasoning tasks, but test-time scaling is becoming expensive. |
| Approach: | They propose to redefine test-time as wall-clock time, where models dynamically adjust strategies based on time budgets. |
| Outcome: | The proposed model improves time budget awareness and boosts performance across Timely-Eval. |
OPT-BENCH: Evaluating the Iterative Self-Optimization of LLM Agents in Large-Scale Search Spaces (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable capabilities in reasoning and tool use, but their ability to continuously refine solutions in response to dynamic environmental feedback remains underexplored. |
| Approach: | They propose a benchmark to evaluate self-improvement capabilities in large-scale search spaces by combining 20 machine learning tasks with 10 classic NP-hard problems. |
| Outcome: | The proposed framework emulates human-like cognitive adaptation and operates via a general perception–memory–reasoning loop, iteratively refining solutions based on environmental feedback. |
Forge: Quality-Aware Reinforcement Learning for NP-Hard Optimization in LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks focus on correctness, overlooking optimality . large language models excel at math, coding, logic and puzzles . |
| Approach: | They propose a framework for training and evaluating Large Language Models on NP-hard optimization problems through quality-aware RLVR. |
| Outcome: | The proposed framework outperforms existing benchmarks on math, coding, logic and puzzles. |
Self-Adjust Softmax (2025.emnlp-main)
Copied to clipboard
Chuanyang Zheng, Yihang Gao, Guoxuan Chen, Han Shi, Jing Xiong, Xiaozhe Ren, Chao Huang, Zhenguo Li, Yu Li
| Challenge: | Usually, tokens with larger attention scores are important for the final prediction. |
| Approach: | They propose to modify softmax(z) to z softmax and its normalized variant to improve the Transformer attention mechanism by making minor adjustments to the softmax function. |
| Outcome: | The proposed model provides enhanced gradient properties compared to the vanilla softmax function. |
DAPE V2: Process Attention Score as Feature Map for Length Extrapolation (2025.acl-long)
Copied to clipboard
Chuanyang Zheng, Yihang Gao, Han Shi, Jing Xiong, Jiankai Sun, Jingyao Li, Minbin Huang, Xiaozhe Ren, Michael Ng, Xin Jiang, Zhenguo Li, Yu Li
| Challenge: | Extensive experiments demonstrate that treating attention as a feature map and applying convolution as . a processing method significantly enhances Transformer performance. |
| Approach: | They propose to use the convolution operator to mimic the processing methods in computer vision to treat attention as a feature map and apply it to neighboring attention scores across different heads. |
| Outcome: | The proposed model can be adapted to various attention-related models and achieves high performance. |