Papers by Xingcheng Ruan
Breaking the Attention Trap in Code LLMs: A Rejection Sampling Approach to Enhance Code Execution Prediction (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing research reveals significant performance bottlenecks in Code Execution tasks, which requires models to predict the execution results of given code snippets. |
| Approach: | They propose to use attention cracking with rejection sampling to eliminate attention traps in training data and to conduct secondary training on the outputs generated by the fine-tuned model to mitigate negative impacts from manual data intervention. |
| Outcome: | The proposed method eliminates the Attention Trap phenomenon in training data and conducts secondary training on the outputs generated by the fine-tuned model to mitigate negative impacts from manual data intervention. |