Papers by Sumit Agarwal
Granite-Function Calling Model: Introducing Function Calling Abilities via Multi-task Learning of Granular Tasks (2024.emnlp-industry)
Copied to clipboard
Ibrahim Abdelaziz, Kinjal Basu, Mayank Agarwal, Sadhana Kumaravel, Matthew Stallone, Rameswar Panda, Yara Rizk, G P Shrivatsa Bhargav, Maxwell Crouse, Chulaka Gunasekara, Shajith Ikbal, Sachindra Joshi, Hima Karanam, Vineet Kumar, Asim Munawar, Sumit Neelam, Dinesh Raghu, Udit Sharma, Adriana Soria, Dheeraj Sreedhar, Praveen Venkateswaran, Merve Unuvar, David Cox, Salim Roukos, Luis Lastras, Pavan Kapanipathi
| Challenge: | Existing research explores the use of Large Language Models (LLMs) as the backbone of agentic systems. |
| Approach: | They propose a model trained using a multi-task training approach on seven fundamental tasks encompassed in function calling that has better generalizability on multiple tasks across seven evaluation benchmarks. |
| Outcome: | The proposed model outperforms more than 15 other models on out-of-domain datasets and ranks among the top on the Berkeley Function Calling Leaderboard (BFCL). |
PRO-CS : An Instance-Based Prompt Composition Technique for Code-Switched Tasks (2022.emnlp-main)
Copied to clipboard
| Challenge: | Code-switching (CS) data is ubiquitous in today’s globalized world, but the dearth of annotated datasets in code-switch tasks poses a significant challenge for transfer learning in limited-resource setups. |
| Approach: | They propose a prompt composition technique that outperforms prompt-tuning and fine-tuned prompt-based prompt composition techniques for CS tasks that combine language and task knowledge. |
| Outcome: | The proposed approach outperforms prompt-tuning and fine-tuned approaches on 10 datasets across 4 languages and achieves competitive results in low-resource cross-lingual and cross-task setting. |
PEFTDebias : Capturing debiasing information using PEFTs (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent research suggests that biases acquired during pretraining can propagate to downstream models, resulting in superficial text dependencies and potential implicit bias. |
| Approach: | They propose a parameter-efficient fine-tuning approach to mitigate implicit biases within foundation models by incorporating parameters into the model and freezing them during the fine-uning process. |
| Outcome: | The proposed method reduces biases in foundation models by incorporating parameters and freezing them during fine-tuning. |
CodeBERTScore: Evaluating Code Generation with Pretrained Models of Code (2023.emnlp-main)
Copied to clipboard
| Challenge: | NLCode generates long expressions and statements rather than a single next-token . evaluating and comparing different models has remained a challenge . |
| Approach: | They propose a code-generating evaluation metric built on BERTScore . they use five language-specific pretrained models to evaluate their code . |
| Outcome: | The proposed evaluation metric achieves higher correlation with human preference and functional correctness than existing metrics across four programming languages. |