Papers by Yuyang Shi
Unleashing LLM Reasoning Capability via Scalable Question Synthesis from Scratch (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods to improve the mathematical reasoning capabilities of Large Language Models (LLMs) are limited due to the proprietary nature of the data. |
| Approach: | They propose a data synthesis method that generates large-scale mathematical reasoning datasets using lightweight 7B-scale models. |
| Outcome: | The proposed method outperforms existing open-source datasets in both in-domain and out-of-domain evaluations and shows improvements in code reasoning tasks. |
GLIMPSE: Do Large Vision-Language Models Truly Think With Videos or Just Glimpse at Them? (2025.emnlp-main)
Copied to clipboard
Yiyang Zhou, Linjie Li, Shi Qiu, Zhengyuan Yang, Yuyang Zhao, Siwei Han, Yangfan He, Kangqi Li, Haonian Ji, Zihao Zhao, Haibo Tong, Lijuan Wang, Huaxiu Yao
| Challenge: | Existing video benchmarks often resemble image-based questions with scans of only a few key frames, without deep temporal reasoning. |
| Approach: | They propose a video benchmark to assess whether large vision-language models can genuinely think with videos rather than perform superficial frame-level analysis. |
| Outcome: | The proposed benchmark consists of 3,269 videos and over 4,342 highly visual-centric questions across 11 categories, including Trajectory Analysis, Temporal Reasoning, and Forensics Detection. |
Large Language Models for Predictive Analysis: How Far Are They? (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on LLMs do not evaluate their capability in predictive analysis. |
| Approach: | They propose a benchmark to evaluate Large Language Models (LLMs) they integrate 1130 queries from 44 real-world datasets of 8 different fields to evaluate their capability . |
| Outcome: | The proposed benchmark evaluates 12 renowned LLMs from 44 real-world datasets . results offer insights into their practical use in predictive analysis . |
AJ-Bench: Benchmarking Agent-as-a-Judge for Environment-Aware Evaluation (2026.findings-acl)
Copied to clipboard
Wentao Shi, Yu Wang, Yuyang Zhao, Yuxin Chen, Fuli Feng, Xueyuan Hao, Xi Su, Qi GU, Hui Su, Xunliang Cai, Xiangnan He
| Challenge: | Existing approaches to verify agent behaviors in complex environments rely on rule-based verifiers or LLM-as-a-Judge models. |
| Approach: | They propose a benchmark to evaluate Agent-as-a-Judge across three domains . the benchmark covers search, data systems, and graphical user interfaces - with 155 tasks and 516 trajectories . |
| Outcome: | The proposed benchmark outperforms existing benchmarks in search, data systems, and GUI domains while revealing open challenges in agent-based verification. |
Can Language Model Moderators Improve the Health of Online Discourse? (2024.naacl-long)
Copied to clipboard
Hyundong Cho, Shuai Liu, Taiwei Shi, Darpan Jain, Basem Rizk, Yuyang Huang, Zixun Lu, Nuan Wen, Jonathan Gratch, Emilio Ferrara, Jonathan May
| Challenge: | Existing efforts to automate conversational moderation have focused on banning harmful comments or deleting them, but such efforts can inadvertently push users towards echo chambers that exacerbate polarization. |
| Approach: | They propose a framework to assess models’ moderation capabilities independently of human intervention and propose 'conversational moderation' they propose to use language models as conversational moderators to provide specific feedback on toxic behavior but struggle to influence users to increase their levels of respect and cooperation. |
| Outcome: | The proposed framework assesses models’ moderation capabilities independently of human intervention and shows that appropriately prompted models provide specific and fair feedback on toxic behavior but struggle to influence users to increase their levels of respect and cooperation. |
Escaping the Echo Trap: On Credit Assignment Failure in Multi-turn LLM Self-Reflection (2026.acl-long)
Copied to clipboard
Linxuan Du, Guangquan Xue, Xiaobo Liang, Qipeng Huang, Yuyang Ding, Xinyu Shi, Zhang Yijun, Ji Qi, Wenpeng Zhu, Juntao Li, Min Zhang
| Challenge: | Existing methods for multi-turn self-reflection are limited by the Echo Trap problem . the model is limited by its inherent capabilities and repeats earlier reflections to preserve reward signals . |
| Approach: | They propose a tree-structured extension of GRPO for multi-turn self-reflection which enables more accurate advantage estimation. |
| Outcome: | The proposed method mitigates behavior collapse and improves performance across benchmarks. |