Papers by Bartosz Przybył
Evaluating Conversational Agents with Persona-driven User Simulations based on Large Language Models: A Sales Bot Case Study (2025.emnlp-industry)
Copied to clipboard
Justyna Gromada, Alicja Kasicka, Ewa Komkowska, Lukasz Krajewski, Natalia Krawczyk, Morgan Veyret, Bartosz Przybył, Lina M. Rojas-Barahona, Michał K. Szczerbak
| Challenge: | Recent advances in LLMs enable sophisticated user simulations that can replace traditional rule-based evaluations. |
| Approach: | They propose a persona-driven approach to conversational agent evaluation using Large Language Models (LLMs) they introduce a dataset of customer personas, which are then used to configure a single LLM-based user simulator. |
| Outcome: | The proposed model emulates nuanced customer roles and can implement cross-selling strategies with minimal impact on customer satisfaction, varying by customer type. |