Papers by Juntao Tan
ActionStudio: A Lightweight Framework for Data and Training of Large Action Models (2025.emnlp-main)
Copied to clipboard
Jianguo Zhang, Thai Quoc Hoang, Ming Zhu, Zuxin Liu, Shiyu Wang, Tulika Manoj Awalgaonkar, Akshara Prabhakar, Haolin Chen, Weiran Yao, Zhiwei Liu, Juntao Tan, Juan Carlos Niebles, Shelby Heinecke, Huan Wang, Silvio Savarese, Caiming Xiong
| Challenge: | Existing infrastructure for efficient agentic data processing and model training remains underdeveloped. |
| Approach: | They propose a lightweight and extensible data and training framework for large action models . they propose to unify diverse agent trajectories using Unified Format 2.0 . |
| Outcome: | The proposed framework shows 9 higher throughput than existing frameworks and performs well across public and realistic agent benchmarks. |
PersonaBench: Evaluating AI Models on Understanding Personal Information through Accessing (Synthetic) Private User Data (2025.findings-acl)
Copied to clipboard
Juntao Tan, Liangwei Yang, Zuxin Liu, Zhiwei Liu, Rithesh R N, Tulika Manoj Awalgaonkar, Jianguo Zhang, Weiran Yao, Ming Zhu, Shirley Kokane, Silvio Savarese, Huan Wang, Caiming Xiong, Shelby Heinecke
| Challenge: | Existing research lacks direct access to such data, making benchmarking difficult due to privacy concerns. |
| Approach: | They propose a synthetic data pipeline that generates realistic user profiles and private documents and a benchmark to evaluate models' ability to understand personal information. |
| Outcome: | The proposed pipeline generates realistic user profiles and private documents, enabling PersonaBench, a benchmark for evaluating models’ ability to understand personal information. |
LATTE: Learning to Think with Vision Specialists (2025.emnlp-main)
Copied to clipboard
Zixian Ma, Jianguo Zhang, Zhiwei Liu, Jieyu Zhang, Juntao Tan, Manli Shu, Juan Carlos Niebles, Shelby Heinecke, Huan Wang, Caiming Xiong, Ranjay Krishna, Silvio Savarese
| Challenge: | Open-source vision-language models excel on simple question-answering tasks, but struggle with complex questions that require both perception and reasoning. |
| Approach: | They propose a family of vision-language models that have LeArned to Think wiTh vision spEcialists by offloading perception to state-of-the-art vision models. |
| Outcome: | The proposed model achieves 4-5% gains over baselines across 6 benchmarks covering both perception and reasoning abilities. |
VIP5: Towards Multimodal Foundation Models for Recommendation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in foundation models have impeded the ability for these fields to benefit from each other’s advancements. |
| Approach: | They propose to use a multimodal foundation model to unify various modalities and recommendation tasks under the P5 recommendation paradigm to implement personalized prompts. |
| Outcome: | The proposed model will unify visual, textual, and personalization modalities under the P5 recommendation paradigm and will improve recommendation performance and efficiency. |
Feature Adaptation of Pre-Trained Language Models across Languages and Domains with Robust Self-Training (2020.emnlp-main)
Copied to clipboard
| Challenge: | Adapting pre-trained language models (PrLMs) to new domains has gained much attention . Adaptation of PrLMs to newdomains is important, but requires fine-tuning . |
| Approach: | They propose to use PrLMs to adapt to new domains without fine-tuning . they use class-aware feature self-distillation to learn discriminative features . |
| Outcome: | The proposed model can learn discriminative features from pre-trained language models without fine-tuning. |
xLAM: A Family of Large Action Models to Empower AI Agent Systems (2025.naacl-long)
Copied to clipboard
Jianguo Zhang, Tian Lan, Ming Zhu, Zuxin Liu, Thai Quoc Hoang, Shirley Kokane, Weiran Yao, Juntao Tan, Akshara Prabhakar, Haolin Chen, Zhiwei Liu, Yihao Feng, Tulika Manoj Awalgaonkar, Rithesh R N, Zeyuan Chen, Ran Xu, Juan Carlos Niebles, Shelby Heinecke, Huan Wang, Silvio Savarese, Caiming Xiong
| Challenge: | Autonomous agents powered by large language models (LLMs) have attracted significant research interest, but there are few standards for developing specialized models for agent tasks. |
| Approach: | They propose a series of large action models with dense and mixture-of-expert architectures that unifies, augments, and synthesizes diverse datasets to enhance agent generalizability and performance. |
| Outcome: | The proposed models outperform GPT-4, Claude-3, and many other models in terms of tool use and outperformed GPT-based models on multiple agent ability benchmarks. |
SlackAgents: Scalable Collaboration of AI Agents in Workspaces (2025.emnlp-demos)
Copied to clipboard
Zhiwei Liu, Weiran Yao, Zuxin Liu, Juntao Tan, Jianguo Zhang, Frank Wang, Sukhandeep Nahal, Huan Wang, Shelby Heinecke, Silvio Savarese, Caiming Xiong
| Challenge: | Existing open-source frameworks like LangChain and LlamaIndex fail to integrate into daily workflows, resulting in limited daily usage for work. |
| Approach: | They propose a multi-agent library for scalable management and collaboration of AI agents on Slack. |
| Outcome: | The proposed framework offers instant AI integration into organizational workflows and facilitates scalable collaboration, allowing for effective communication and task orchestration. |