Papers by Junjie Gao
AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments (2026.acl-long)
Copied to clipboard
Zhiheng Xi, Dingwen Yang, Jiaqi Liu, Jixuan Huang, Honglin Guo, Baodai Huang, Tinggang Chen, Qi Zhang, Zhonghang Lu, Chenyu Liu, Jiajun Sun, Jiazheng Zhang, Dingwei Zhu, Xin Guo, Junzhe Wang, Zhihao Zhang, Yuming Yang, Junjie Ye, Minghe Gao, Dongrui Liu, Jiaming Ji, Guohao Li, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Existing benchmarks evaluate agents in simplified, idealized settings, relying on pre-packaged tool interfaces, overlooking critical steps, and assume inputs are clean and fully specified. |
| Approach: | They propose a framework that evaluates language agents in simplified, idealized settings . they show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 . |
| Outcome: | Experiments on 15 proprietary and open-source models show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 . |
Decorrelate Irrelevant, Purify Relevant: Overcome Textual Spurious Correlations from a Feature Perspective (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods to debiase samples with biased features obstructs the model in learning from non-biased parts of the samples. |
| Approach: | They propose to eliminate spurious correlations in a fine-grained manner from a feature space perspective by using Random Fourier Features and weighted re-sampling to decorrelate dependencies between features. |
| Outcome: | The proposed method eliminates spurious correlations in a fine-grained manner from a feature space perspective. |
CodeExp: Explanatory Code Document Generation (2022.findings-emnlp)
Copied to clipboard
Haotian Cui, Chenglong Wang, Junjie Huang, Jeevana Priya Inala, Todd Mytkowicz, Bo Wang, Jianfeng Gao, Nan Duan
| Challenge: | Existing code-to-text generation models produce only high-level code summaries that do not capture implementation-level choices essential for these scenarios. |
| Approach: | They propose a code explanation generation task that uses code docstrings to refine models. |
| Outcome: | The proposed model can generate well-structured long docstrings comparable to human-written ones. |
Probing LLM World Models: Enhancing Guesstimation with Wisdom of Crowds Decoding (2025.emnlp-main)
Copied to clipboard
Yun-Shiuan Chuang, Sameer Narendran, Nikunj Harlalka, Alexander Cheung, Sizhe Gao, Siddharth Suresh, Junjie Hu, Timothy T. Rogers
| Challenge: | a common real-world skill of guesstimation is underexplored in large language model research . a recent study suggests that LLMs encode a world model that supports approximate reasoning . |
| Approach: | They propose to decode a guesstimation dataset using MARBLES, FUTURE, and ELECPRED . they replicate WOC effects in human participants and find similar benefits . |
| Outcome: | The proposed model improves accuracy over greedy, self-consistency, and mean decoding in human participants. |
A Survey of Post-Training Scaling in Large Language Models (2025.acl-long)
Copied to clipboard
Hanyu Lai, Xiao Liu, Junjie Gao, Jiale Cheng, Zehan Qi, Yifan Xu, Shuntian Yao, Dan Zhang, Jinhua Du, Zhenyu Hou, Xin Lv, Minlie Huang, Yuxiao Dong, Jie Tang
| Challenge: | Large language models (LLMs) have demonstrated proficiency in understanding and generating human natural languages. |
| Approach: | They propose a framework for scaling large language models using supervised fine-tuning, RLxF and test-time compute methodologies. |
| Outcome: | The proposed model can be used to understand and generate human natural languages. |
Zero-Shot Learners for Natural Language Understanding via a Unified Multiple Choice Perspective (2022.emnlp-main)
Copied to clipboard
Ping Yang, Junjie Wang, Ruyi Gan, Xinyu Zhu, Lin Zhang, Ziwei Wu, Xinyu Gao, Jiaxing Zhang, Tetsuya Sakai
| Challenge: | Existing approaches to zero-shot learning are format-agnostic and can address new learning tasks without additional training. |
| Approach: | They propose a new paradigm for zero-shot learning that is format agnostic and compatible with any format and applicable to a list of language tasks. |
| Outcome: | The proposed model shows state-of-the-art performance on several benchmarks and produces satisfactory results on tasks such as text classification and commonsense reasoning. |
QFinZero: A Unified Financial Toolchain for LLM-Based Trading Agents (2026.acl-demo)
Copied to clipboard
Haochen Luo, Yifan LI, Ho Tin Ko, An Binh Minh, Junjie Xu, Tang Pok Hin, Wang Chak Wong, Gao Yuan, Zhengzhao Lai, Yuan Zhang, Chen Liu
| Challenge: | Existing trading systems rely on fragmented and task-specific APIs, resulting in inconsistent schemas and limited reproducibility. |
| Approach: | They propose a unified trading environment for large language model (LLM) agents that standardizes three core capabilities . they argue that such a standardized trading environment is essential for scalable research on LLM-based financial agents. |
| Outcome: | The proposed trading environment reduces engineering overhead and supports reproducible evaluation through comprehensive logging and deterministic replay. |
ToolSword: Unveiling Safety Issues of Large Language Models in Tool Learning Across Three Stages (2024.acl-long)
Copied to clipboard
Junjie Ye, Sixian Li, Guanyu Li, Caishuang Huang, Songyang Gao, Yilong Wu, Qi Zhang, Tao Gui, Xuanjing Huang
| Challenge: | Existing research focuses on enhancing LLMs capabilities through tool utilization. |
| Approach: | They propose a framework to investigate safety issues in large language models in tool learning . they propose malicious queries and jailbreak attacks in the input stage . |
| Outcome: | The proposed framework investigates six safety scenarios for LLMs in tool learning . the data will be released upon acceptance of the proposed framework . |
ToolEyes: Fine-Grained Evaluation for Tool Learning Capabilities of Large Language Models in Real-world Scenarios (2025.coling-main)
Copied to clipboard
Junjie Ye, Guanyu Li, SongYang Gao, Caishuang Huang, Yilong Wu, Sixian Li, Xiaoran Fan, Shihan Dou, Tao Ji, Qi Zhang, Tao Gui, Xuanjing Huang
| Challenge: | Existing evaluations of tool learning focus on validation of tools for large language models with expected outcomes, but this focus ignores the complex capabilities required for LLMs to effectively use tools. |
| Approach: | They propose a fine-grained system for evaluation of large language models’ tool learning capabilities in authentic scenarios. |
| Outcome: | The proposed system examines seven real-world scenarios, analyzing five dimensions crucial to LLMs in tool learning: format alignment, intent comprehension, behavior planning, tool selection, and answer organization. |
RoTBench: A Multi-Level Benchmark for Evaluating the Robustness of Large Language Models in Tool Learning (2024.emnlp-main)
Copied to clipboard
Junjie Ye, Yilong Wu, Songyang Gao, Caishuang Huang, Sixian Li, Guanyu Li, Xiaoran Fan, Qi Zhang, Tao Gui, Xuanjing Huang
| Challenge: | Current research emphasizes LLMs’ capacity to utilize tools in well-structured environments while overlooking their stability when confronted with the inevitable noise of the real world. |
| Approach: | They propose a multi-level benchmark to evaluate the robustness of large language models in tool learning by establishing five external environments with varying levels of noise. |
| Outcome: | The proposed model outperforms the GPT-4 model in tool learning in three critical phases: tool selection, parameter identification, and content filling. |
AndroidGen: Building an Android Language Agent under Data Scarcity (2025.acl-long)
Copied to clipboard
| Challenge: | Existing LLMs lack high-quality data sources and lack robust data filtration strategies. |
| Approach: | They develop a framework to enhance the capabilities of LLM-based agents under data scarcity. |
| Outcome: | The proposed framework improves the capabilities of LLM-based agents under data scarcity. |
REO-Relevance, Extraness, Omission: A Fine-grained Evaluation for Image Captioning (D19-1)
Copied to clipboard
| Challenge: | Existing metrics for image captioning evaluation provide an overall quality score, which is difficult to infer specific description errors. |
| Approach: | They propose a fine-grained evaluation method REO for automatically measuring the performance of image captioning systems. |
| Outcome: | The proposed method achieves higher consistency with human judgments and provides more intuitive evaluation results than other metrics. |
Any Information Is Just Worth One Single Screenshot: Unifying Search With Visualized Information Retrieval (2025.acl-long)
Copied to clipboard
| Challenge: | Existing multimodal retrieval models are lacking in visual representations of multimodal data. |
| Approach: | They propose a visualized information retrieval paradigm where multimodal information is represented by a unified visual format called Screenshots for various retrieval applications. |
| Outcome: | The proposed model is based on a large dataset of screenshots from diverse sources . it is compared with existing models and lays a solid foundation for the new model . |