Papers by Vineet Kumar
Pointwise Mutual Information Based Metric and Decoding Strategy for Faithful Generation in Document Grounded Dialogs (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing metrics for faithfulness of response are not aligned with human judgments. |
| Approach: | They propose a new metric that utilizes (Conditional) Point-wise Mutual Information (PMI) between the generated response and the source document, conditioned on the dialogue. |
| Outcome: | The proposed metric improves on BEGIN benchmarks and shows that it generates more faithful responses than standard decoding techniques. |
Exemplar Encoder-Decoder for Neural Conversation Generation (P18-1)
Copied to clipboard
| Challenge: | Existing approaches to generate conversational systems suffer from lack of diversity in responses and generation of short, repetitive and uninteresting responses. |
| Approach: | They propose a novel conversation model that uses similar examples from training data to generate responses. |
| Outcome: | The proposed model outperforms state-of-the-art sequence to sequence learning on several evaluation metrics on two large data sets. |
Granite-Function Calling Model: Introducing Function Calling Abilities via Multi-task Learning of Granular Tasks (2024.emnlp-industry)
Copied to clipboard
Ibrahim Abdelaziz, Kinjal Basu, Mayank Agarwal, Sadhana Kumaravel, Matthew Stallone, Rameswar Panda, Yara Rizk, G P Shrivatsa Bhargav, Maxwell Crouse, Chulaka Gunasekara, Shajith Ikbal, Sachindra Joshi, Hima Karanam, Vineet Kumar, Asim Munawar, Sumit Neelam, Dinesh Raghu, Udit Sharma, Adriana Soria, Dheeraj Sreedhar, Praveen Venkateswaran, Merve Unuvar, David Cox, Salim Roukos, Luis Lastras, Pavan Kapanipathi
| Challenge: | Existing research explores the use of Large Language Models (LLMs) as the backbone of agentic systems. |
| Approach: | They propose a model trained using a multi-task training approach on seven fundamental tasks encompassed in function calling that has better generalizability on multiple tasks across seven evaluation benchmarks. |
| Outcome: | The proposed model outperforms more than 15 other models on out-of-domain datasets and ranks among the top on the Berkeley Function Calling Leaderboard (BFCL). |
Gaining Insights into Unrecognized User Utterances in Task-Oriented Dialog Systems (2022.emnlp-industry)
Copied to clipboard
| Challenge: | Goal-oriented dialog systems fail to recognize the intent of natural language requests due to system errors, incomplete service coverage, or insufficient training. |
| Approach: | They propose an end-to-end pipeline for processing unrecognized user utterances, deployed in a commercial task-oriented dialog system, including a specifically-tailored clustering algorithm, a novel approach to cluster representative extraction, and cluster naming. |
| Outcome: | The proposed components show that they improve the performance of the proposed system in the analysis of unrecognized user requests. |
Large-Scale Differentially Private BERT (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that scaling up the batch size to millions improves the utility of a DP-SGD step for BERT. |
| Approach: | They propose to use differentially private SGD to pretrain BERT-Large with a batch size of millions to improve the utility of the DP-SGD step. |
| Outcome: | The proposed approach achieves a masked language model accuracy of 60.5% at a batch size of 2M, which is a reasonable privacy setting. |