Papers by Rocktim Das
Synergizing In-context Learning with Hints for End-to-end Task-oriented Dialog Systems (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing end-to-end task-oriented dialogue systems require extensive training datasets to perform well. |
| Approach: | They propose a system that synergizes LLMs with task-specific hints to improve alignment in low-data settings. |
| Outcome: | The proposed model improves alignment in low-data settings while retaining competitive performance in full-data environments. |
Exploring Distributional Shifts in Large Language Models for Code Analysis (2023.emnlp-main)
Copied to clipboard
| Challenge: | Since the late 2000s, researchers have been reporting poor generalization of statistical learning models to new software systems, such as GitHub Copilot, Amazon CodeWhisperer, Replit, etc. |
| Approach: | They systematically study how three large language models with code capabilities generalize to out-of-domain data. |
| Outcome: | The proposed model outperforms the existing model for code generation on multiple domains at once. |
EXAMS-V: A Multi-Discipline Multilingual Multimodal Exam Benchmark for Evaluating Vision Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for vision language models are outdated and unable to accurately assess their performance. |
| Approach: | They propose a multi-discipline multimodal multilingual exam benchmark for vision language models . they collect multiple-choice questions across 20 disciplines across 11 languages from 7 language families . |
| Outcome: | The EXAMS-V exam includes 20,932 multiple-choice questions across 20 disciplines . the questions come in 11 languages from 7 language families and require advanced reasoning skills . |
Factuality of Large Language Models: A Survey (2024.emnlp-main)
Copied to clipboard
Yuxia Wang, Minghan Wang, Muhammad Arslan Manzoor, Fei Liu, Georgi Georgiev, Rocktim Das, Preslav Nakov
| Challenge: | Large language models (LLMs) are factually incorrect, which limits their applicability in real-world scenarios. |
| Approach: | They analyze existing work to identify major challenges and their associated causes . they propose to evaluate LLMs using a variety of measures to mitigate factual errors . |
| Outcome: | The proposed methods are based on a variety of datasets and proposed strategies to mitigate factual errors. |
DKAF: KB Arbitration for Learning Task-Oriented Dialog Systems with Dialog-KB Inconsistencies (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches for learning task-oriented dialog agents assume the KB snapshot is current during training. |
| Approach: | They propose a dialog-KB arbitration framework which predicts the contemporary KB snapshot for each train dialog. |
| Outcome: | The proposed model performs better on two publicly available dialog datasets than baselines on both datasets. |
MediTOD: An English Dialogue Dataset for Medical History Taking with Comprehensive Annotations (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing datasets lacking comprehensive annotations for medical history-taking are non-English . existing datasets lack comprehensive annotation for medical slots and their attributes . |
| Approach: | They propose a dataset of doctor-patient dialogues in English for medical history-taking task. |
| Outcome: | The proposed datasets are available in English and are compared with existing datasets. |