Papers by Qiushi Sun
Evaluating and Enhancing the Robustness of Code Pre-trained Models through Structure-Aware Adversarial Samples Generation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained code models have made significant strides in the field of neural code intelligence, but they are susceptible to adversarial attacks that subtly modify the input sequence and can impair generalization. |
| Approach: | They propose a set of novel robustness evaluation methods based on the intrinsic structure of the code to explore the impact of imperceptible perturbation. |
| Outcome: | The proposed methods have demonstrated their effectiveness across a wide range of models and tasks, and are able to predict the performance of perturbed models. |
Make Prompt-based Black-Box Tuning Colorful: Boosting Model Generalization from Three Orthogonal Perspectives (2024.lrec-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown increasing power on NLP tasks. however, tuning these models for downstream tasks usually requires exorbitant costs. |
| Approach: | They propose a black-box tuning technique that optimizes task-specific prompts without accessing gradients and hidden representations. |
| Outcome: | The proposed method improves performance under few-shot learning scenarios. |
Explicit Memory Learning with Expectation Maximization (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models lack reliable learning mechanisms for updating information across interactions. |
| Approach: | They propose a framework that enhances explicit memory updates via the Expectation-Maximization algorithm. |
| Outcome: | The proposed framework outperforms existing methods without memory or with static external memory on streaming inference tasks. |
SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents (2024.acl-long)
Copied to clipboard
| Challenge: | Existing GUI agents interact with the environment through extracted structured data, which can be notably lengthy (e.g., HTML) and occasionally inaccessible (e-book). |
| Approach: | They propose to enhance SeeClick with GUI grounding pre-training and devise a method to automate curation of GUI ground data. |
| Outcome: | The proposed agent improves ScreenSpot, the first realistic GUI grounding benchmark that encompasses mobile, desktop, and web environments. |
Automated Peer Reviewing in Paper SEA: Standardization, Evaluation, and Analysis (2024.findings-emnlp)
Copied to clipboard
Jianxiang Yu, Zichen Ding, Jiaqi Tan, Kangyang Luo, Zhenmin Weng, Chenghua Gong, Long Zeng, RenJing Cui, Chengcheng Han, Qiushi Sun, Zhiyong Wu, Yunshi Lan, Xiang Li
| Challenge: | Existing approaches to review scientific papers are limited by their content or quality . SEA is a framework for automated scientific review, but its contents are generic or partial. |
| Approach: | They propose a framework for automated scientific review using large language models . they propose to use a standardized review dataset to fine-tune an LLM to generate high-quality reviews. |
| Outcome: | The proposed framework can generate high-quality reviews from standardized datasets and improves on the existing feedback mechanisms. |
Do Large Language Models Know What They Don’t Know? (2023.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have vast knowledge that allows them to excel in various NLP tasks. |
| Approach: | They propose an automated method to detect uncertainty in the responses of large language models and a dataset to measure their self-knowledge. |
| Outcome: | The proposed method detects uncertainty in the responses of large language models and provides a novel measure of their self-knowledge. |
KS-Lottery: Finding Certified Lottery Tickets for Multilingual Transfer in Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing studies have shown that a small subset of parameters is highly effective in fine-tuning . prior work shows that there are a few additional parameters corresponding to an intrinsic dimension in a well-trained Large Language Model. |
| Approach: | They propose a method to identify a small subset of LLM parameters highly effective in multilingual fine-tuning. |
| Outcome: | The proposed method can find the certified winning tickets in the embedding layer, and fine-tuning on the found parameters is guaranteed to perform as well as full fine- tuning. |
Reasoning in Flux: Enhancing Large Language Models Reasoning through Uncertainty-aware Adaptive Guidance (2024.acl-long)
Copied to clipboard
Zhangyue Yin, Qiushi Sun, Qipeng Guo, Zhiyuan Zeng, Xiaonan Li, Junqi Dai, Qinyuan Cheng, Xuanjing Huang, Xipeng Qiu
| Challenge: | Extensive experiments across various reasoning tasks demonstrate that UAG not only enhances the reasoning abilities of LLMs but consistently outperforms several strong baselines with minimal computational overhead. |
| Approach: | They propose an approach to guide LLMs onto an accurate and reliable trajectory by identifying and adjusting uncertainty signals within each step of the reasoning chain. |
| Outcome: | The proposed approach outperforms strong baselines and outperformed strong models with minimal computational overhead. |
Genius: A Generalizable and Purely Unsupervised Self-Training Framework For Advanced Reasoning (2025.acl-long)
Copied to clipboard
Fangzhi Xu, Hang Yan, Chang Ma, Haiteng Zhao, Qiushi Sun, Kanzhi Cheng, Junxian He, Jun Liu, Zhiyong Wu
| Challenge: | Existing methods for enhancing LLM reasoning rely on supervisory signals . current methods rely heavily on outcome supervision and auxiliary reward models . |
| Approach: | They propose a gen-eralizable and purely unsupervised self-training framework to enhance LLM reasoning without supervision. |
| Outcome: | The proposed framework improves LLM reasoning without supervision without external supervision. |
Exchange-of-Thought: Enhancing Large Language Model Capabilities through Cross-Model Communication (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have made significant strides in complex reasoning tasks, but their reasoning is often constrained by their intrinsic understanding, lacking external insights. |
| Approach: | They propose a framework that enables cross-model communication during problem-solving. |
| Outcome: | The proposed framework surpasses established baselines in complex reasoning tasks and is cost-effective. |
TransCoder: Towards Unified Transferable Code Representation Learning Inspired by Human Skills (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods to fine-tune code intelligence models to individual tasks are costly and require large data sets. |
| Approach: | They propose a Transferable fine-tuning strategy for Code representation learning that uses a tunable prefix encoder to capture cross-task and cross-language transferable knowledge and apply it to downstream adaptation. |
| Outcome: | The proposed method can lead to superior performance on code-related tasks and encourage mutual reinforcement. |
ClueAnchor: Clue-Anchored Knowledge Reasoning Exploration and Optimization for Retrieval-Augmented Generation (2025.findings-emnlp)
Copied to clipboard
Hao Chen, Yukun Yan, Sen Mei, Wanxiang Che, Zhenghao Liu, Qi Shi, Xinze Li, Yuchun Fan, Pengcheng Huang, Qiushi Xiong, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing RAG systems often underutilize the retrieved documents, authors say . they fail to extract and integrate key clues needed to support faithful and interpretable reasoning . |
| Approach: | a new framework extracts key clues from retrieved content and generates multiple reasoning paths . the framework optimizes the model by selecting the most appropriate reasoning path . |
| Outcome: | Experiments show that ClueAnchor outperforms baseline RAG frameworks in completeness and robustness. |
Dynamic and Generalizable Process Reward Modeling (2025.acl-long)
Copied to clipboard
| Challenge: | Existing Process Reward Models lack cross-domain generalization and focus on feedback results. |
| Approach: | They propose a process reward model that uses a reward tree to capture and store fine-grained, multi-dimensional reward criteria. |
| Outcome: | The proposed model performs on prevailing benchmarks and out-of-distribution scenarios. |
CapArena: Benchmarking and Analyzing Detailed Image Captioning in the LLM Era (2025.findings-acl)
Copied to clipboard
Kanzhi Cheng, Wenpo Song, Jiaxin Fan, Zheng Ma, Qiushi Sun, Fangzhi Xu, Chenyang Yan, Nuo Chen, Jianbing Zhang, Jiajun Chen
| Challenge: | Image captioning has been a challenge for vision-language researchers for decades . current VLMs focus on tasks like visual question answering (YA) but image captioning is not as advanced as expected. |
| Approach: | They evaluate VLMs' performance on image captioning using human annotations . they find that some metrics show high caption-level agreement with humans . |
| Outcome: | The proposed model outperforms open-source models on image captioning . it achieves 93.4% correlation with human rankings at $4 per test . |
OS-Symphony: A Holistic Framework for Robust and Generalist Computer-Using Agents (2026.acl-long)
Copied to clipboard
Bowen Yang, Kaiming Jin, Zhenyu Wu, Zhaoyang Liu, Qiushi Sun, Zehao Li, JingJing Xie, Zhoumianze Liu, Fangzhi Xu, Kanzhi Cheng, Yian Wang, Qingyun Li, Yu Qiao, Zun Wang, Zichen Ding
| Challenge: | Vision-Language Models (VLMs) lack visual-aware tutorial retrieval and historical visual context curation and pruning. |
| Approach: | They propose a framework that integrates an orchestrator and a Reflection-Memory Agent for robust automation. |
| Outcome: | Experimental results show that OS-Symphony delivers substantial performance gains across model scales. |
Boosting Language Models Reasoning with Chain-of-Knowledge Prompting (2024.acl-long)
Copied to clipboard
| Challenge: | Recent studies have shown that Chain-of-Thought (CoT) prompting can be effective on complex reasoning tasks but generates unfaithful and unfactual reasoning chains. |
| Approach: | They propose a chain-of-knowledge prompting that elicits Large Language Models to generate explicit pieces of knowledge evidence in the form of structure triple. |
| Outcome: | The proposed method improves commonsense, factual, symbolic, and arithmetic reasoning tasks by estimating the reliability of the reasoning chains in terms of factuality and faithfulness. |
Uncertainty-aware Parameter-Efficient Self-training for Semi-supervised Language Understanding (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for pre-trained language models rely on noisy data, which can be expensive if all parameters are updated. |
| Approach: | They propose a self-training framework that incorporates Monte Carlo dropouts into the model and judiciously selects reliable pseudo-labeled examples based on confidence and certainty. |
| Outcome: | The proposed framework improves performance and efficiency over multiple tasks over multiple datasets. |
ARISE: An Adaptive Resolution-Aware Metric for Test-Time Scaling Evaluation in Large Reasoning Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluation methods for test-time scaling are limited. |
| Approach: | They propose an adaptive resolution-aware scaling evaluation metric specifically designed to assess the test-time scaling effectiveness of large reasoning models. |
| Outcome: | The proposed metric provides a reliable and fine-grained measurement of test-time scaling capabilities, revealing significant variations in scaling efficiency across models. |
OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis (2025.acl-long)
Copied to clipboard
Qiushi Sun, Kanzhi Cheng, Zichen Ding, Chuanyang Jin, Yian Wang, Fangzhi Xu, Zhenyu Wu, Chengyou Jia, Liheng Chen, Zhoumianze Liu, Ben Kao, Guohao Li, Junxian He, Yu Qiao, Zhiyong Wu
| Challenge: | Graphical User Interface (GUI) agents powered by Vision-Language Models (VLMs) have demonstrated human-like computer control capability. |
| Approach: | They propose a GUI data synthesis pipeline that reverse engineers GUI trajectory construction process by executing pre-defined tasks. |
| Outcome: | The proposed GUI data synthesis pipeline overcomes the bottlenecks of previous methods that rely on pre-defined tasks and limited data diversity. |
Symbol-LLM: Towards Foundational Symbol-centric Interface For Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have limitations when it comes to comprehending and expressing world knowledge that extends beyond the boundaries of natural language. |
| Approach: | They propose a model that integrates symbolic data into LLM training without loss of generality ability. |
| Outcome: | The proposed model performs better on symbol- and NL-centric tasks. |
Interactive Evolution: A Neural-Symbolic Self-Training Framework For Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods to fine-tune Large Language Models without human annotations are lacking in the field of natural language training. |
| Approach: | They propose an environment-guided neural-symbolic self-training framework to overcome two main challenges: the scarcity of symbolic data and the limited proficiency of LLMs in processing symbolic language. |
| Outcome: | The proposed framework overcomes two main challenges: the scarcity of symbolic data, and the limited proficiency of LLMs in processing symbolic language. |
OS-Sentinel: Towards Safety-Enhanced Mobile GUI Agents via Hybrid Validation in Realistic Workflows (2026.acl-long)
Copied to clipboard
Qiushi Sun, Mukai Li, Zhoumianze Liu, Zhihui Xie, Fangzhi Xu, Zhangyue Yin, Kanzhi Cheng, Zehao Li, Zichen Ding, Qi Liu, Zhiyong Wu, Zhuosheng Zhang, Ben Kao, Lingpeng Kong
| Challenge: | Existing methods for detecting unsafe mobile GUI agents are underexplored. |
| Approach: | They propose a mobile agent safety detection framework that integrates a formal verifier and a VLM-based contextual judge to detect system-level violations. |
| Outcome: | The proposed framework achieves 10%–30% improvements over existing approaches across multiple metrics. |
Structure-aware Fine-tuning for Code Pre-trained Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing CodePTMs are mainly structure-free and structurebased, but how to fine-tune them remains a challenge. |
| Approach: | They propose a plug-and-play fine-tuning method that incorporates structural knowledge into pre-trained code models. |
| Outcome: | The proposed method can benefit CodePTMs more with limited training data. |
Pass-Tuning: Towards Structure-Aware Parameter-Efficient Tuning for Code Representation Learning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Code pre-trained models have been proposed and widely applied in the domain of code intelligence. |
| Approach: | They propose a method that uses a plug-and-play graph neural network module as a tunable prefix to exploit structural information of source code. |
| Outcome: | The proposed method exploits structural information of source code and could replace full fine-tuning. |
CAT-probing: A Metric-based Approach to Interpret How Pre-trained Models for Programming Language Attend Code Structure (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing code pre-trained models fail to consider inherent characteristics of codes . Existing methods to interpret code pretrained model fail to take into account inherent characteristics . |
| Approach: | They propose a probing method to quantitatively interpret how CodePTMs attend code structure. |
| Outcome: | The proposed method denoises input code sequences and measures commonality between token-level attention scores and pair-wise distances between corresponding AST nodes. |
AgentStore: Scalable Integration of Heterogeneous Agents As Specialized Generalist Computer Assistant (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing agents lack generalization and specialization capabilities for open-ended tasks . specialized generalists are often underdeveloped in real-world environments . |
| Approach: | They propose a platform to dynamically integrate heterogeneous agents for automating computer tasks . they propose specialized generalist agent MetaAgent with the AgentToken strategy . |
| Outcome: | The proposed platform expands capabilities of existing agents in generalization and specialization . it can be used to automate open-ended tasks in real-world environments . |
When Gradient Descent Meets Derivative-Free Optimization: A Match Made in Black-Box Scenario (2023.findings-acl)
Copied to clipboard
| Challenge: | Large pre-trained language models (PLMs) are expensive and may not be open-sourced due to commercial considerations and potential risks of misuse. |
| Approach: | They propose to introduce gradient descent into black-box tuning scenario . they propose a method which integrates gradient descent and derivative-free optimization . |
| Outcome: | The proposed method achieves significant performance gains over previous state-of-the-art methods. |
CodeEvo: Interaction-Driven Synthesis of Code-centric Data through Hybrid and Iterative Feedback (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for generating instruction-code pairs rely on rigid heuristics and are labor-intensive. |
| Approach: | They propose a dual-agent architecture that integrates a Coder and a Reviewer to orchestrate the generation trajectory. |
| Outcome: | The proposed architecture outperforms baselines on a large-scale dataset of instruction-code pairs with stepped difficulty levels. |
Aggregation of Reasoning: A Hierarchical Framework for Enhancing Answer Selection in Large Language Models (2024.lrec-main)
Copied to clipboard
Zhangyue Yin, Qiushi Sun, Qipeng Guo, Zhiyuan Zeng, Xiaonan Li, Tianxiang Sun, Cheng Chang, Qinyuan Cheng, Ding Wang, Xiaofeng Mou, Xipeng Qiu, Xuanjing Huang
| Challenge: | Recent advances in Chain-of-Thought prompting have facilitated significant breakthroughs for Large Language Models (LLMs) in complex reasoning tasks. |
| Approach: | They propose a hierarchical reasoning aggregation framework to address this problem . they propose dynamic sampling to adjust the number of reasoning chains . |
| Outcome: | The proposed framework outperforms existing ensemble methods on complex reasoning tasks. |