Papers by Abhisek Tiwari
Action and Reaction Go Hand in Hand! a Multi-modal Dialogue Act Aided Sarcasm Identification (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies have shown that sarcasm is reflected by the intended meaning of the speaker's utterance. |
| Approach: | They propose to extend the MUStARD dataset to enclose dialogue acts for each dialogue . they propose a dialogue act-aided multi-modal transformer network for sarcasm identification model . |
| Outcome: | The proposed model improves performance in dialogue act-aided sarcasm identification compared to sardasmatic identification alone. |
Persona or Context? Towards Building Context adaptive Personalized Persuasive Virtual Sales Assistant (2022.aacl-main)
Copied to clipboard
Abhisek Tiwari, Sriparna Saha, Shubhashis Sengupta, Anutosh Maitra, Roshni Ramnani, Pushpak Bhattacharyya
| Challenge: | Existing task-oriented conversational agents assume that end-users will always have a pre-determined and servable task goal, which results in dialogue failure in hostile scenarios, such as goal unavailability. |
| Approach: | They propose to build an end-to-end multi-modal persuasive dialogue system incorporating a personalized persuasive module aided goal controller and goal persuader. |
| Outcome: | The proposed system achieves user tasks even in goal unavailability scenarios by persuading them towards a similar and servable goal. |
From Sights to Insights: Towards Summarization of Multimodal Clinical Documents (2024.acl-long)
Copied to clipboard
| Challenge: | a recent WHO report highlights a drastic doctor-to-patient ratio . telehealth is one of the most impactful sectors where AI advances can bring a significant revolution . |
| Approach: | They propose an image-guided encoder-decoder model that uses contextual attention to create detailed visual-guides for multimodal documents. |
| Outcome: | The proposed model outperforms state-of-the-art models on multimodal question and dialogue summarization tasks. |
Seeing Is Believing! towards Knowledge-Infused Multi-modal Medical Dialogue Generation (2024.lrec-main)
Copied to clipboard
Abhisek Tiwari, Shreyangshu Bera, Preeti Verma, Jaithra Varma Manthena, Sriparna Saha, Pushpak Bhattacharyya, Minakshi Dhar, Sarbajeet Tiwari
| Challenge: | Existing models of disease diagnosis using AI do not use knowledge infusion. |
| Approach: | They propose a transformer-based, knowledge-infused multi-modal medical dialogue generation framework . they propose 'discourse-aware' image identifier that recognizes signs and their severity . |
| Outcome: | The proposed model outperforms state-of-the-art models by 7.84% in the english language. |
I know you are different! Towards Persona Driven Knowledge-infused Dialogue Assistant (2026.eacl-long)
Copied to clipboard
| Challenge: | Task-Oriented Dialogue (TOD) systems often fall short in delivering personalized, context-rich responses, especially in low-resource, code-mixed, and multimodal settings like Hinglish. |
| Approach: | They propose a Hinglish multimodal, multidomain, persona-based TOD dataset that captures user-agent interactions across text and visual modalities. |
| Outcome: | The proposed framework outperforms standard and ablated models in Hinglish and Hinglanish. |