Agent-Ops: A Multi-Agent Orchestration Framework for End-to-End SOP Automation in E-Commerce Operations (2026.acl-industry)
Copied to clipboard
| Challenge: | Existing Large Language Models fail to execute multistep operational workflows requiring precise procedural adherence. |
| Approach: | They propose an end-to-end multi-agent framework automating Standard Operating Procedures in e-commerce. |
| Outcome: | The proposed framework achieves 85-97% accuracy and a 94.2% execution consistency in e-commerce . it is based on a human-AI framework that transforms ambiguous documentation into automation-ready specifications . |
Similar Papers
<SYNTACT>: Structuring Your Natural Language SOPs into Tailored Ambiguity-Resolved Code Templates (2025.emnlp-industry)
Copied to clipboard
Sachin Kumar Giroh, Pushpendu Ghosh, Aryan Jain, Harshal Giridhari Paunikar, Aditi Rastogi, Promod Yenigalla, Anish Nediyanchath
| Challenge: | Unstructured and ambiguous Standard Operating Procedures suffer from ambiguity, missing information, and inconsistency, all of which hinder automation. |
| Approach: | They propose a three-stage LLM framework that transforms unstructured SOPs into a structured plan and an executable code template. |
| Outcome: | The proposed framework shows an 88.4% accuracy and significant reduction in inconsistency on real-world SOPs and synthetic variants. |
A Practical Approach for Building Production-Grade Conversational Agents with Workflow Graphs (2025.acl-industry)
Copied to clipboard
Chiwan Park, Wonjun Jang, Daeryong Kim, Aelim Ahn, Kichang Yang, Woosung Hwang, Jihyeon Roh, Hyerin Park, Hyosun Wang, Min Seok Kim, Jihoon Kang
| Challenge: | Large Language Models (LLMs) have led to significant improvements in various service domains, including search, recommendation, and chatbot applications. |
| Approach: | They propose a framework for developing scalable, controllable, and reliable AI-driven agents that can be applied to real-world applications. |
| Outcome: | The proposed framework bridges the gap between academic research and real-world application, and enables scalable, controllable, and reliable AI-driven agents. |
ReAct Meets Industrial IoT: Language Agents for Data Access (2025.emnlp-industry)
Copied to clipboard
| Challenge: | a framework for domain-specific language agents is being developed for industrial automation . a novel approach to adapting these systems to domain-based applications poses new challenges . |
| Approach: | They propose a framework for deploying domain-specific language agents that can query industrial sensor data using natural language. |
| Outcome: | The proposed framework outperforms standard prompting baselines across multiple LLMs including smaller models. |
WorkTeam: Constructing Workflows from Natural Language with Multi-Agents (2025.naacl-industry)
Copied to clipboard
| Challenge: | Existing workflow construction methods require specialized knowledge and task-switching skills. |
| Approach: | They propose a multi-agent workflow framework that incorporates a supervisor, orchestrator, and filler agent. |
| Outcome: | The proposed framework significantly increases the success rate of workflow construction . the proposed framework is based on a dataset of 3,695 real-world business samples . |
Towards large language model-based personal agents in the enterprise: Current trends and open problems (2023.findings-emnlp)
Copied to clipboard
Vinod Muthusamy, Yara Rizk, Kiran Kate, Praveen Venkateswaran, Vatche Isahagian, Ashu Gulati, Parijat Dube
| Challenge: | Existing large language models (LLMs) are brittle to input changes and can produce inconsistent results for the same inputs. |
| Approach: | They propose to use large language models to reason about complex goals and orchestrate a set of pluggable tools or APIs to accomplish a goal. |
| Outcome: | The proposed use cases have many open problems in an exciting area of NLP research, such as trust and explainability, consistency and reproducibility, and the need for new metrics and benchmarks. |
Auto-SLURP: A Benchmark Dataset for Evaluating Multi-Agent Frameworks in Smart Personal Assistant (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Auto-SLURP is a benchmark dataset for evaluating multi-agent frameworks powered by large language models. |
| Approach: | Auto-SLURP is a benchmark dataset aimed at evaluating LLM-based multi-agent frameworks . authors propose it extends original SLURP dataset by relabeling data and integrating simulated servers and external services. |
| Outcome: | The proposed dataset extends the original SLURP dataset for natural language understanding tasks. |
TPTU-v2: Boosting Task Planning and Tool Usage of Large Language Model-based Agents in Real-world Industry Systems (2024.emnlp-industry)
Copied to clipboard
Yilun Kong, Jingqing Ruan, YiHong Chen, Bin Zhang, Tianpeng Bao, Shi Shiwei, Du Qing, Xiaoru Hu, Hangyu Mao, Ziyue Li, Xingyu Zeng, Rui Zhao, Xueqian Wang
| Challenge: | Large language models have demonstrated proficiency in addressing tasks that necessitate a combination of task planning and the usage of external tools. |
| Approach: | They propose a framework to enhance the task planning and tool usage abilities of LLMs in industrial systems. |
| Outcome: | The proposed framework enhances the task planning and tool usage abilities of LLM-based agents in industrial systems. |
AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments (2026.acl-long)
Copied to clipboard
Zhiheng Xi, Dingwen Yang, Jiaqi Liu, Jixuan Huang, Honglin Guo, Baodai Huang, Tinggang Chen, Qi Zhang, Zhonghang Lu, Chenyu Liu, Jiajun Sun, Jiazheng Zhang, Dingwei Zhu, Xin Guo, Junzhe Wang, Zhihao Zhang, Yuming Yang, Junjie Ye, Minghe Gao, Dongrui Liu, Jiaming Ji, Guohao Li, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Existing benchmarks evaluate agents in simplified, idealized settings, relying on pre-packaged tool interfaces, overlooking critical steps, and assume inputs are clean and fully specified. |
| Approach: | They propose a framework that evaluates language agents in simplified, idealized settings . they show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 . |
| Outcome: | Experiments on 15 proprietary and open-source models show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 . |
A Functionality-Grounded Benchmark for Evaluating Web Agents in E-commerce Domains (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks focus on product search tasks, but ignore potential risks. |
| Approach: | They propose a data generation pipeline that leverages webpage content and interactive elements to create diverse, functionality-grounded user queries. |
| Outcome: | The proposed framework assesses the performance and safety of web agents under dynamic, real-world e-commerce environments. |
Unifying Language Agent Algorithms with Graph-based Orchestration Engine for Reproducible Agent Research (2025.acl-demo)
Copied to clipboard
Qianqian Zhang, Jiajia Liao, Heting Ying, Yibo Ma, Haozhan Shen, Jingcheng Li, Peng Liu, Lu Zhang, Chunxin Fang, Kyusong Lee, Ruochen Xu, Tiancheng Zhao
| Challenge: | Language agents powered by large language models (LLMs) have demonstrated remarkable capabilities in understanding, reasoning, and executing complex tasks. |
| Approach: | They propose a flexible framework that addresses engineering overhead and insufficient evaluation frameworks for fair comparison. |
| Outcome: | The proposed framework simplifies language agent development and establishes a foundation for reproducible agent research. |