Papers by Deepanway Ghosal
NLKI: A Lightweight Natural Language Knowledge Integration Framework for Improving Small VLMs in Commonsense VQA Tasks (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Small vision-language models lag behind their larger generative counterparts due to lack of knowledge. |
| Approach: | They propose a framework that integrates commonsense knowledge into small vision-language models . the framework retrieves natural language facts and prompts an LLM to craft natural language explanations . |
| Outcome: | The proposed framework retrieves natural language facts and prompts an LLM to craft natural language explanations. |
PuzzleVQA: Diagnosing Multimodal Reasoning Challenges of Language Models with Abstract Visual Patterns (2024.findings-acl)
Copied to clipboard
| Challenge: | recognizing patterns and abstracting concepts are key to general intelligence, we show . state-of-the-art large multimodal models struggle to generalize well to simple abstract patterns . |
| Approach: | They evaluate large multimodal models with abstract patterns based on colors, numbers, sizes, and shapes. |
| Outcome: | The proposed model fails to generalize well to simple abstract patterns, the study shows . the model fails on single-concept puzzles, despite its sophistication . |
COSMIC: COmmonSense knowledge for eMotion Identification in Conversations (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Current methods for emotion recognition in conversations often face difficulties in context propagation, emotion shift detection, and differentiating between related emotion classes. |
| Approach: | They propose a framework that incorporates mental states, events, and causal relations to learn interactions between interlocutors participating in a conversation. |
| Outcome: | The proposed framework improves on four conversational benchmark datasets. |
Exploring the Role of Context in Utterance-level Emotion, Act and Intent Classification in Conversations: An Empirical Study (2021.findings-acl)
Copied to clipboard
| Challenge: | utterance-level dialogue understanding tasks are often performed at utterrance level and are often conjoined together under the umbrella of utterence-level dialog understanding. |
| Approach: | They propose to use a contextual utterance-level dialogue understanding baseline as a strong framework for six dialogue-understanding tasks. |
| Outcome: | The proposed framework can be easily adapted for other tasks for similar purposes. |
Language Guided Visual Question Answering: Elevate Your Multimodal Language Model Using Knowledge-Enriched Prompts (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Visual question answering (VQA) is a task that requires an understanding of both the image and the question to provide a natural language answer. |
| Approach: | They propose a multimodal framework that leverages language guidance to answer questions more accurately. |
| Outcome: | The proposed framework improves on the multi-choice question-answering task using CLIP and BLIP models. |
MIME: MIMicking Emotions for Empathetic Response Generation (2020.emnlp-main)
Copied to clipboard
Navonil Majumder, Pengfei Hong, Shanshan Peng, Jiankun Lu, Deepanway Ghosal, Alexander Gelbukh, Rada Mihalcea, Soujanya Poria
| Challenge: | Empathy is a fundamental human trait that reflects our ability to understand and reflect the thoughts and feelings of the people we interact with. |
| Approach: | They propose to use polarity-based emotion clusters to generate empathetic responses . they also introduce stochasticity into the emotion mixture that yields emotionally more varied responses compared to the previous work . |
| Outcome: | The proposed methods improve empathy and contextual relevance of the response, and introduce stochasticity into the emotion mixture that yields emotionally more varied responses than the previous work. |
Vector Space Interpolation for Query Expansion (2022.aacl-short)
Copied to clipboard
| Challenge: | Topic-sensitive query set expansion is crucial for queries related to sensitive and emerging topics. |
| Approach: | They propose a method for topic-sensitive query set expansion using vector space interpolation. |
| Outcome: | The proposed method generates new queries about the sensitive topic by incorporating set diversity, which is not captured by traditional sentence-level augmentation methods such as paraphrasing or back-translation. |
Contextual Inter-modal Attention for Multi-modal Sentiment Analysis (D18-1)
Copied to clipboard
Deepanway Ghosal, Md Shad Akhtar, Dushyant Chauhan, Soujanya Poria, Asif Ekbal, Pushpak Bhattacharyya
| Challenge: | Existing methods for multi-modal sentiment analysis are limited due to the use of text, visual and acoustic inputs. |
| Approach: | They propose a recurrent neural network based multi-modal attention framework that leverages contextual information for utterance-level sentiment prediction. |
| Outcome: | The proposed framework performs better on two multi-modal sentiment analysis benchmark datasets with accuracies of 82.31% and 79.80% for the MOSI and MOSEI datasets. |
ReTAG: Reasoning Aware Table to Analytic Text Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Table to text models generate descriptive summaries that repeat information contained within a table in sentences. |
| Approach: | They propose a table-aware table-to-text model that uses vector-quantization to infuse different types of analytical reasoning into the output. |
| Outcome: | The proposed model achieves 2.2%, 2.9% improvement on PARENT metric over state-of-the-art models. |
CICERO: A Dataset for Contextualized Commonsense Inference in Dialogues (2022.acl-long)
Copied to clipboard
| Challenge: | Fig. 1a shows an example where commonsense knowledge is crucial in sifting relevant information from the context. |
| Approach: | They curate a dataset of dyadic conversations with five types of utterance-level reasoning-based inferences: cause, subsequent event, prerequisite, motivation, and emotional reaction. |
| Outcome: | The dataset contains 53,105 of such inferences from 5,672 dialogues. |
Emma-X: An Embodied Multimodal Action Model with Grounded Chain of Thought and Look-ahead Spatial Reasoning (2025.acl-long)
Copied to clipboard
| Challenge: | Visual-Language-Action models lack the ability to generate actionable policies tailored to specific robotic embodiments. |
| Approach: | They propose an embodied multimodal action model with Grounded Chain of Thought and Look-ahead Spatial Reasoning that enhances spatial reasoning and task planning. |
| Outcome: | The proposed model improves on existing baselines in tasks requiring spatial reasoning and grounding reasoning. |
STaCK: Sentence Ordering with Temporal Commonsense Knowledge (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to sentence order prediction ignore the importance of document level global information, i.e., while predicting relative order of two sentences (s i , s j) other sentences sk from the same document do not play any role. |
| Approach: | They propose a framework based on graph neural networks and temporal commonsense knowledge to model global information and predict relative order of sentences. |
| Outcome: | The proposed method is naturally suitable for order prediction on five different datasets and has potential applications in the evaluation of the quality of machinegenerated documents. |
Evaluating LLMs’ Mathematical and Coding Competency through Ontology-guided Interventions (2025.findings-acl)
Copied to clipboard
| Challenge: | Current large language models have shown impressive performance on logical reasoning benchmarks . however, the true depth of their competencies and robustness in reasoning tasks remains an open question . |
| Approach: | They propose a general ontology of perturbations and a semi-automatic method to apply perturbations to arithmetic reasoning and code generation datasets to test their LLMs' capabilities. |
| Outcome: | The proposed model outperforms existing models on arithmetic reasoning and code generation tasks. |
DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation (D19-1)
Copied to clipboard
| Challenge: | Emotion recognition in conversation (ERC) has received much attention lately due to its potential widespread applications in diverse areas, such as health-care, education, and human resources. |
| Approach: | They propose a graph neural network-based approach to emotion recognition in conversation that leverages self and inter-speaker dependency of the interlocutors to model conversational context. |
| Outcome: | The proposed method outperforms the current state-of-the-art on a number of benchmark emotion classification datasets while minimizing context propagation issues. |
AlgoPuzzleVQA: Diagnosing Multimodal Reasoning Challenges of Language Models with Algorithmic Multimodal Puzzles (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing datasets focused on visual question-answering focus on visual, language, and algorithmic knowledge . a new study examines the performance of multimodal language models in solving algorithmic puzzles . |
| Approach: | They propose a dataset to test the capabilities of multimodal language models in solving algorithmic puzzles. |
| Outcome: | The proposed dataset is generated automatically from human code. |
Mustango: Toward Controllable Text-to-Music Generation (2024.naacl-long)
Copied to clipboard
| Challenge: | Mustango is a text-to-music system that allows music-domain-knowledge-informed text-based music generation. |
| Approach: | They propose a music-domain-knowledge-inspired text-to-music system based on diffusion that generates music with captions that include specific instructions related to chords, beats, key and tempo. |
| Outcome: | The proposed system outperforms existing models in music generation tasks. |
KinGDOM: Knowledge-Guided DOMain Adaptation for Sentiment Analysis (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to cross-domain sentiment analysis cannot be reliably deployed due to the distributional mismatch between training and evaluation domains. |
| Approach: | They propose a framework that uses ConceptNet to enrich semantics of documents by providing domain-specific and domain-general background concepts. |
| Outcome: | The proposed framework improves on a domain-adversarial baseline method and can be used in domain adaptation. |
Two is Better than Many? Binary Classification as an Effective Approach to Multi-Choice Question Answering (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to multi-choice question answering are based on binary classifications instead of scoring each answer as a single class. |
| Approach: | They propose a simple refactoring of multi-choice question answering tasks as a series of binary classifications and propose re-framing to make them more efficient. |
| Outcome: | The proposed approach is significantly more effective across different tasks and models. |
Multi-task Learning for Multi-modal Emotion Recognition and Sentiment Analysis (N19-1)
Copied to clipboard
Md Shad Akhtar, Dushyant Chauhan, Deepanway Ghosal, Soujanya Poria, Asif Ekbal, Pushpak Bhattacharyya
| Challenge: | Existing frameworks for sentiment and emotion analysis are not efficient for inter-task learning. |
| Approach: | They propose a multi-task learning framework that performs sentiment and emotion analysis together. |
| Outcome: | The proposed framework improves on a CMU-MOSEI dataset for sentiment and emotion analysis. |