Papers by Yara Rizk
NESTFUL: A Benchmark for Evaluating LLMs on Nested Sequences of API Calls (2025.emnlp-main)
Copied to clipboard
Kinjal Basu, Ibrahim Abdelaziz, Kiran Kate, Mayank Agarwal, Maxwell Crouse, Yara Rizk, Kelsey Bradford, Asim Munawar, Sadhana Kumaravel, Saurabh Goyal, Xin Wang, Luis A. Lastras, Pavan Kapanipathi
| Challenge: | Existing benchmarks and datasets for tool calling have lagged behind . nested sequencing is a common problem in LLMs, but it is not enough to evaluate them. |
| Approach: | They propose a benchmark to evaluate LLMs on nested sequences of API calls, i.e. sequences where the output of one API call is passed as input to a subsequent call. |
| Outcome: | The proposed model achieves a full sequence match accuracy of 28% and a win-rate of 60% on nested sequences of API calls. |
How Good Are LLMs at Processing Tool Outputs? (2026.eacl-long)
Copied to clipboard
Kiran Kate, Yara Rizk, Poulami Ghosh, Ashu Gulati, Tathagata Chakraborti, Zidane Wright, Mayank Agarwal
| Challenge: | Real-world task automation tasks require large language models to call tools, which often return complex JSON responses. |
| Approach: | They evaluated 15 open and closed weight models using multiple prompting approaches to evaluate their tool response processing task and their ability to process structured (JSON) responses. |
| Outcome: | The proposed model can process structured (JSON) responses with 3% to 50% performance differences. |
Granite-Function Calling Model: Introducing Function Calling Abilities via Multi-task Learning of Granular Tasks (2024.emnlp-industry)
Copied to clipboard
Ibrahim Abdelaziz, Kinjal Basu, Mayank Agarwal, Sadhana Kumaravel, Matthew Stallone, Rameswar Panda, Yara Rizk, G P Shrivatsa Bhargav, Maxwell Crouse, Chulaka Gunasekara, Shajith Ikbal, Sachindra Joshi, Hima Karanam, Vineet Kumar, Asim Munawar, Sumit Neelam, Dinesh Raghu, Udit Sharma, Adriana Soria, Dheeraj Sreedhar, Praveen Venkateswaran, Merve Unuvar, David Cox, Salim Roukos, Luis Lastras, Pavan Kapanipathi
| Challenge: | Existing research explores the use of Large Language Models (LLMs) as the backbone of agentic systems. |
| Approach: | They propose a model trained using a multi-task training approach on seven fundamental tasks encompassed in function calling that has better generalizability on multiple tasks across seven evaluation benchmarks. |
| Outcome: | The proposed model outperforms more than 15 other models on out-of-domain datasets and ranks among the top on the Berkeley Function Calling Leaderboard (BFCL). |
Towards large language model-based personal agents in the enterprise: Current trends and open problems (2023.findings-emnlp)
Copied to clipboard
Vinod Muthusamy, Yara Rizk, Kiran Kate, Praveen Venkateswaran, Vatche Isahagian, Ashu Gulati, Parijat Dube
| Challenge: | Existing large language models (LLMs) are brittle to input changes and can produce inconsistent results for the same inputs. |
| Approach: | They propose to use large language models to reason about complex goals and orchestrate a set of pluggable tools or APIs to accomplish a goal. |
| Outcome: | The proposed use cases have many open problems in an exciting area of NLP research, such as trust and explainability, consistency and reproducibility, and the need for new metrics and benchmarks. |
TaskDiff: A Similarity Metric for Task-Oriented Conversations (2023.emnlp-main)
Copied to clipboard
| Challenge: | Popularity of chatGPT and Llama 2 has led to a race to build custom task-oriented conversational assistants in enterprise domains like finance and retail. |
| Approach: | They propose a conversational similarity metric that uses different dialogue components to compute similarity. |
| Outcome: | Experiments on a benchmark dataset show that the proposed metric outperforms existing approaches and is more robust than previous approaches. |
Towards LLMs Robustness to Changes in Prompt Format Styles (2025.naacl-srw)
Copied to clipboard
| Challenge: | Existing prompt engineering techniques for adapting pre-trained LLMs to specific tasks are zero-shot prompting and few-shot supervised fine-tuning. |
| Approach: | They propose a technique for addressing prompt brittleness by diversifying the styles used in the prompt few-shot examples by using computer vision techniques. |
| Outcome: | Empirical results show that the proposed technique reduces style-induced prompt brittleness while improving overall performance across prompt variations and different datasets. |