Papers by Dinesh Raghu
Pointwise Mutual Information Based Metric and Decoding Strategy for Faithful Generation in Document Grounded Dialogs (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing metrics for faithfulness of response are not aligned with human judgments. |
| Approach: | They propose a new metric that utilizes (Conditional) Point-wise Mutual Information (PMI) between the generated response and the source document, conditioned on the dialogue. |
| Outcome: | The proposed metric improves on BEGIN benchmarks and shows that it generates more faithful responses than standard decoding techniques. |
Systematic Knowledge Injection into Large Language Models via Diverse Augmentation for Domain-Specific RAG (2025.findings-naacl)
Copied to clipboard
Kushagra Bhushan, Yatin Nandwani, Dinesh Khandelwal, Sonam Gupta, Gaurav Pandey, Dinesh Raghu, Sachindra Joshi
| Challenge: | Retrieval-Augmented Generation (RAG) enhances response relevance by incorporating retrieved domain knowledge in the context, retrieval errors can still lead to hallucinations and incorrect answers. |
| Approach: | They propose a framework that augments the learning process by context augmentation and knowledge paraphrasing by incorporating retrieved domain knowledge into the context. |
| Outcome: | The proposed framework achieves 10% relative gain in token-level recall while preserving the LLM’s generalization capabilities. |
Granite-Function Calling Model: Introducing Function Calling Abilities via Multi-task Learning of Granular Tasks (2024.emnlp-industry)
Copied to clipboard
Ibrahim Abdelaziz, Kinjal Basu, Mayank Agarwal, Sadhana Kumaravel, Matthew Stallone, Rameswar Panda, Yara Rizk, G P Shrivatsa Bhargav, Maxwell Crouse, Chulaka Gunasekara, Shajith Ikbal, Sachindra Joshi, Hima Karanam, Vineet Kumar, Asim Munawar, Sumit Neelam, Dinesh Raghu, Udit Sharma, Adriana Soria, Dheeraj Sreedhar, Praveen Venkateswaran, Merve Unuvar, David Cox, Salim Roukos, Luis Lastras, Pavan Kapanipathi
| Challenge: | Existing research explores the use of Large Language Models (LLMs) as the backbone of agentic systems. |
| Approach: | They propose a model trained using a multi-task training approach on seven fundamental tasks encompassed in function calling that has better generalizability on multiple tasks across seven evaluation benchmarks. |
| Outcome: | The proposed model outperforms more than 15 other models on out-of-domain datasets and ranks among the top on the Berkeley Function Calling Leaderboard (BFCL). |
Selective Self-to-Supervised Fine-Tuning for Generalization in Large Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) can be fine-tuned on task-specific data to improve performance on target tasks but can be overfitted resulting in a loss of generalization. |
| Approach: | They propose a method that uses the correct model responses from a training set to fine-tune the model using the correct response and the gold response for the remaining samples. |
| Outcome: | The proposed approach reduces model specialization during the fine-tuning stage while improving generalization. |
Synergizing In-context Learning with Hints for End-to-end Task-oriented Dialog Systems (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing end-to-end task-oriented dialogue systems require extensive training datasets to perform well. |
| Approach: | They propose a system that synergizes LLMs with task-specific hints to improve alignment in low-data settings. |
| Outcome: | The proposed model improves alignment in low-data settings while retaining competitive performance in full-data environments. |
Few shot chain-of-thought driven reasoning to prompt LLMs for open-ended medical question answering (2024.findings-emnlp)
Copied to clipboard
Saeel Nachane, Ojas Gramopadhye, Prateek Chanda, Ganesh Ramakrishnan, Kshitij Jadhav, Yatin Nandwani, Dinesh Raghu, Sachindra Joshi
| Challenge: | Large Language models (LLMs) are increasingly utilized in the healthcare sector for query-related tasks. |
| Approach: | They propose a modified version of the MedQA-USMLE dataset, named MEDQA-OPEN, which contains open-ended medical questions without options to mimic clinical scenarios. |
| Outcome: | The proposed approach outperforms the state-of-the-art 5-shot CoT-based prompt by exploring multiple differential diagnoses and narrowing down to a final diagnosis using MCQ-ELIMINATIVE. |
Disentangling Language and Knowledge in Task-Oriented Dialogs (N19-1)
Copied to clipboard
| Challenge: | Existing approaches to handle task-oriented dialogs break when asked to handle such changes. |
| Approach: | They propose an encoder-decoder architecture with a novel Bag-of-Sequences memory which facilitates the disentangled learning of the response’s language model and its knowledge incorporation. |
| Outcome: | The proposed architecture outperforms state-of-the-art models on bAbI OOV test sets and other human-human datasets and shows that it is robust to KB modifications. |
Multi-Level Memory for Task Oriented Dialogs (N19-1)
Copied to clipboard
| Challenge: | Recent task oriented dialog systems use memory architectures to incorporate external knowledge in their dialogs. |
| Approach: | They propose a novel multi-level memory architecture that separates dialog context and knowledge base results . they use cells for each query and their corresponding results to address queries . |
| Outcome: | The proposed architecture outperforms current state-of-the-art models on three publicly available data sets. |
Joint Reasoning on Hybrid-knowledge sources for Task-Oriented Dialog (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing systems for task oriented dialog use knowledge present only in structured knowledge sources to generate responses. |
| Approach: | They propose a model that assumes that information is always present in a structured knowledge base . they also refine the model to take into account the fact that it can fuse information from structured and unstructured knowledge sources. |
| Outcome: | The proposed model is robust to perturbations to knowledge modality and can fuse information from structured and unstructured knowledge to generate responses. |
Constraint based Knowledge Base Distillation in End-to-End Task Oriented Dialogs (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for learning task-oriented dialog systems filter irrelevant KB information over a large KB. |
| Approach: | They propose a pairwise similarity filter that respects the n-ary structure in a KB record and an auxiliary loss that helps in separating contextually unrelated KB information. |
| Outcome: | The proposed method outperforms existing state-of-the-art models on three publicly available datasets. |
End-to-End Learning of Flowchart Grounded Task-Oriented Dialogs (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing systems that use human-to-human dialogs to help users with specific tasks are still unexplored. |
| Approach: | They propose a problem in which a dialog system mimics a troubleshooting agent . they use a dataset grounded on 12 different troubleshooking flowcharts to train the agent a neural model . |
| Outcome: | The proposed model can do zero-shot transfer to unseen flowcharts and sets a strong baseline for future research. |
Structural Constraints and Natural Language Inference for End-to-End Flowchart Grounded Dialog Response Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to learn flowchart grounded dialogs have two limitations . Flowchart-based systems require only the chat transcripts and no additional annotations . |
| Approach: | They propose a structure-aware approach to learn flowchart grounded dialogs . it uses structural constraints derived from connectivity structure of flowchartes into a RAG framework . |
| Outcome: | The proposed approach outperforms existing approaches with a success rate of 68% and 123%. |
Unsupervised Learning of KB Queries in Task-Oriented Dialogs (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing approaches require dialog datasets to explicitly annotate knowledge base (KB) queries. |
| Approach: | They propose a pipelined approach to predict when to make a KB query and train the dialog agent without explicit annotation. |
| Outcome: | The proposed approach predicts when to make a KB query, then predicts a query at the predicted position and uses the results in subsequent dialog. |
DKAF: KB Arbitration for Learning Task-Oriented Dialog Systems with Dialog-KB Inconsistencies (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches for learning task-oriented dialog agents assume the KB snapshot is current during training. |
| Approach: | They propose a dialog-KB arbitration framework which predicts the contemporary KB snapshot for each train dialog. |
| Outcome: | The proposed model performs better on two publicly available dialog datasets than baselines on both datasets. |
MediTOD: An English Dialogue Dataset for Medical History Taking with Comprehensive Annotations (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing datasets lacking comprehensive annotations for medical history-taking are non-English . existing datasets lack comprehensive annotation for medical slots and their attributes . |
| Approach: | They propose a dataset of doctor-patient dialogues in English for medical history-taking task. |
| Outcome: | The proposed datasets are available in English and are compared with existing datasets. |
HealthAlignSumm : Utilizing Alignment for Multimodal Summarization of Code-Mixed Healthcare Dialogues (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Collaboration between doctors and AI scientists is leading to personalized models to stream-line healthcare tasks and improve productivity. |
| Approach: | They propose to use alignment techniques to combine a doctor-patient dialogue with a visual component of the BART model. |
| Outcome: | The proposed model in-tegrates visual components with the BART ar-chitecture. |