Papers by Soujanya Poria
Modeling Inter-Aspect Dependencies for Aspect-Based Sentiment Analysis (N18-2)
Copied to clipboard
Devamanyu Hazarika, Soujanya Poria, Prateek Vij, Gangeshwar Krishnamurthy, Erik Cambria, Roger Zimmermann
| Challenge: | Present neural-based models exploit aspect and its contextual information in the sentence but ignore inter-aspect dependencies. |
| Approach: | They propose to combine aspect-based sentiment analysis with temporal dependency processing to incorporate this pattern into a sentence. |
| Outcome: | The proposed approach is based on the SemEval 2014 dataset and shows that it is effective for predicting sentiments of aspects in sentences with multiple aspects. |
Vector-Quantized Input-Contextualized Soft Prompts for Natural Language Understanding (2022.emnlp-main)
Copied to clipboard
| Challenge: | Prompt Tuning has been successful as a parameter-efficient method of conditioning large-scale pre-trained language models to perform downstream tasks. |
| Approach: | They propose to use a vector-quantized input-contextualized prompt as an extension to the soft prompt tuning framework to learn contextualization of soft prompt tokens. |
| Outcome: | The proposed prompt outperforms soft prompt tuning by an average margin of 1.19% on various language understanding tasks like SuperGLUE, QA, Relation classification, NER and NLI. |
A Comprehensive Survey of Sentence Representations: From the BERT Epoch to the CHATGPT Era and Beyond (2024.eacl-long)
Copied to clipboard
Abhinav Ramesh Kashyap, Thanh-Tung Nguyen, Viktor Schlegel, Stefan Winkler, See-Kiong Ng, Soujanya Poria
| Challenge: | Sentence representations are a critical component in NLP applications such as retrieval, question answering, and text classification. |
| Approach: | They present a systematic review of the literature on sentence representations focusing mostly on deep learning models. |
| Outcome: | The proposed methods highlight the key contributions and challenges in this area and suggest potential avenues for improving the quality and efficiency of sentence representations. |
Adapter Pruning using Tropical Characterization (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on adapter pruning have not examined the optimal number of adapter parameters needed for downstream applications. |
| Approach: | They propose an adapter pruning approach that prunes adapter parameters without changing the orientation of underlying tropical hypersurfaces. |
| Outcome: | The proposed approach prunes adapter layers without changing the orientation of underlying tropical hypersurfaces. |
So Different Yet So Alike! Constrained Unsupervised Text Style Transfer (2022.acl-long)
Copied to clipboard
| Challenge: | Automated transfer of text between domains does not maintain other attributes between the source and translated text. |
| Approach: | They propose a method for automatic transfer of text between domains that preserves semantic content but changes other attributes. |
| Outcome: | The proposed method retains lexical, syntactic and domain-specific constraints between domains for multiple benchmark datasets, including ones where more than one attribute change. |
Uncertainty Guided Label Denoising for Document-level Distant Relation Extraction (2023.acl-long)
Copied to clipboard
| Challenge: | Document-level relation extraction (DocRE) aims to extract semantic relations between entities in a document. |
| Approach: | They propose a Document-level distant relation extraction framework with unreliable pseudo labels to denoise DS data. |
| Outcome: | The proposed framework outperforms strong baselines on two public datasets. |
MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations (P19-1)
Copied to clipboard
| Challenge: | Emotion recognition in conversations has gained popularity due to its potential applications. Until now, a large multimodal multi-party emotional conversational database containing more than two speakers per dialogue was missing. |
| Approach: | They propose to extend and enhance EmotionLines by combining 13,000 utterances from Friends dialogues with emotion and sentiment labels. |
| Outcome: | The proposed dataset contains about 13,000 utterances from 1,433 dialogues from the TV-series Friends. |
CMU-MOSEAS: A Multimodal Language Dataset for Spanish, Portuguese, German and French (2020.emnlp-main)
Copied to clipboard
AmirAli Bagher Zadeh, Yansheng Cao, Simon Hessner, Paul Pu Liang, Soujanya Poria, Louis-Philippe Morency
| Challenge: | Existing datasets in multimodal language are limited and disproportionately affect native speakers of other languages . authors propose a large-scale dataset for Spanish, Portuguese, German and French . |
| Approach: | They propose a large-scale multimodal language dataset for Spanish, Portuguese, German and French. |
| Outcome: | The proposed dataset is the largest of its kind with 40,000 total labelled sentences . it covers a diverse set topics and speakers and carries supervision of 20 labels including sentiment, emotions, and attributes. |
Large Language Models for Automated Open-domain Scientific Hypotheses Discovery (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing research on hypothetical induction is limited by the observation annotations in the dataset and the ground truth hypotheses are mostly commonsense knowledge. |
| Approach: | They propose a first dataset for social science academic hypotheses discovery using raw web corpus as observations and propose valid, useful scientific hypothese . they propose 'a multi-module framework' that includes feedback mechanisms to boost performance. |
| Outcome: | The proposed dataset generates valid, novel, and helpful scientific hypotheses, even new to humanity, using open-domain data and a web corpus as observations. |
KNOT: Knowledge Distillation Using Optimal Transport for Solving NLP Tasks (2022.coling-1)
Copied to clipboard
| Challenge: | Knowledge Distillation using Optimal Transport (KNOT) aims to distill the natural language semantic knowledge from multiple teacher networks to a student network. |
| Approach: | They propose to distill natural language semantic knowledge from multiple teacher networks to a student network by learning to minimize the optimal transport cost of its assigned probability distribution over the labels to the weighted sum of probabilities predicted by the (local) teacher models. |
| Outcome: | The proposed method shows improvements in the global model’s SD performance over the baseline across three NLP tasks while performing on par with Entropy-based distillation on standard accuracy and F1 metrics. |
Knowledge Enhanced Reflection Generation for Counseling Dialogues (2022.acl-long)
Copied to clipboard
| Challenge: | Using retrieval and generative methods, we generate responses using commonsense and domain knowledge. |
| Approach: | They propose a pipeline that collects domain knowledge through web mining and a model that incorporates knowledge generated by COMET using soft positional encoding and masked self-attention. |
| Outcome: | The proposed pipeline collects domain knowledge through web mining and incorporates knowledge generated by COMET using soft positional encoding and masked self-attention. |
Lightweight Spatial Modeling for Combinatorial Information Extraction From Documents (2023.findings-eacl)
Copied to clipboard
Yanfei Dong, Lambert Deng, Jiazheng Zhang, Xiaodong Yu, Ting Lin, Francesco Gelli, Soujanya Poria, Wee Sun Lee
| Challenge: | Existing datasets do not cover documents with complex spatial structures and a lack of spatial information for document entity classification. |
| Approach: | They propose a new spatial bias in attention calculation based on the K-nearest-neighbor graph of document entities that limits entities’ attention to their local radius. |
| Outcome: | The proposed model outperforms baselines in most entity types and is highly parameter-efficient compared to existing methods. |
Error Typing for Smarter Rewards: Improving Process Reward Models with Error-Aware Hierarchical Supervision (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are prone to hallucination, especially during multihop tasks. |
| Approach: | They propose a hierarchical, erroraware discriminative PRM that classifies math errors at each step and combines finegrained signals to estimate step correctness. |
| Outcome: | The proposed model outperforms the prior best in a new stateof-theart PRMScore of 67.7 on a 400Ksample dataset . |
PuzzleVQA: Diagnosing Multimodal Reasoning Challenges of Language Models with Abstract Visual Patterns (2024.findings-acl)
Copied to clipboard
| Challenge: | recognizing patterns and abstracting concepts are key to general intelligence, we show . state-of-the-art large multimodal models struggle to generalize well to simple abstract patterns . |
| Approach: | They evaluate large multimodal models with abstract patterns based on colors, numbers, sizes, and shapes. |
| Outcome: | The proposed model fails to generalize well to simple abstract patterns, the study shows . the model fails on single-concept puzzles, despite its sophistication . |
COSMIC: COmmonSense knowledge for eMotion Identification in Conversations (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Current methods for emotion recognition in conversations often face difficulties in context propagation, emotion shift detection, and differentiating between related emotion classes. |
| Approach: | They propose a framework that incorporates mental states, events, and causal relations to learn interactions between interlocutors participating in a conversation. |
| Outcome: | The proposed framework improves on four conversational benchmark datasets. |
UDAPTER - Efficient Domain Adaptation Using Adapters (2023.eacl-main)
Copied to clipboard
| Challenge: | Using adapters, unsupervised domain adaptation (UDA) is more parameter efficient and requires large-scale data to be effective. |
| Approach: | They propose to add small bottleneck layers to each layer of a pre-trained language model to make it more parameter efficient by adding adapters. |
| Outcome: | The proposed methods outperform unsupervised domain adaptation methods such as DANN and DSN in natural language inference and sentiment classification tasks. |
Exploring the Role of Context in Utterance-level Emotion, Act and Intent Classification in Conversations: An Empirical Study (2021.findings-acl)
Copied to clipboard
| Challenge: | utterance-level dialogue understanding tasks are often performed at utterrance level and are often conjoined together under the umbrella of utterence-level dialog understanding. |
| Approach: | They propose to use a contextual utterance-level dialogue understanding baseline as a strong framework for six dialogue-understanding tasks. |
| Outcome: | The proposed framework can be easily adapted for other tasks for similar purposes. |
Language Guided Visual Question Answering: Elevate Your Multimodal Language Model Using Knowledge-Enriched Prompts (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Visual question answering (VQA) is a task that requires an understanding of both the image and the question to provide a natural language answer. |
| Approach: | They propose a multimodal framework that leverages language guidance to answer questions more accurately. |
| Outcome: | The proposed framework improves on the multi-choice question-answering task using CLIP and BLIP models. |
MIME: MIMicking Emotions for Empathetic Response Generation (2020.emnlp-main)
Copied to clipboard
Navonil Majumder, Pengfei Hong, Shanshan Peng, Jiankun Lu, Deepanway Ghosal, Alexander Gelbukh, Rada Mihalcea, Soujanya Poria
| Challenge: | Empathy is a fundamental human trait that reflects our ability to understand and reflect the thoughts and feelings of the people we interact with. |
| Approach: | They propose to use polarity-based emotion clusters to generate empathetic responses . they also introduce stochasticity into the emotion mixture that yields emotionally more varied responses compared to the previous work . |
| Outcome: | The proposed methods improve empathy and contextual relevance of the response, and introduce stochasticity into the emotion mixture that yields emotionally more varied responses than the previous work. |
ICON: Interactive Conversational Memory Network for Multimodal Emotion Detection (D18-1)
Copied to clipboard
| Challenge: | Existing studies do not explicitly consider inter-personal influences that thrive in the emotional dynamics of dialogues. |
| Approach: | They propose a multimodal emotion detection framework that extracts multimodal features from conversational videos and hierarchically models the self- and inter-speaker emotional influences into global memories. |
| Outcome: | The proposed model outperforms state-of-the-art networks on multiple classification and regression tasks in two benchmark datasets. |
IARM: Inter-Aspect Relation Modeling with Memory Networks in Aspect-Based Sentiment Analysis (D18-1)
Copied to clipboard
| Challenge: | Aspect-based sentiment analysis is a new approach to extract aspect specific sentimental information from user feedback. |
| Approach: | They propose a method that incorporates neighboring aspects related information into the sentiment classification of a target aspect using memory networks. |
| Outcome: | The proposed method outperforms the state-of-the-art by 1.6% on average in restaurant and laptop domains. |
Towards Multimodal Sarcasm Detection (An _Obviously_ Perfect Paper) (P19-1)
Copied to clipboard
Santiago Castro, Devamanyu Hazarika, Verónica Pérez-Rosas, Roger Zimmermann, Rada Mihalcea, Soujanya Poria
| Challenge: | sarcasm is often expressed through multiple verbal and non-verbal cues, such as a change of tone, overemphasis, drawn-out syllables, or a straight looking face. |
| Approach: | They propose to use multimodal cues to improve sarcasm detection using audiovisual utterances annotated with sarcasm labels to improve the accuracy. |
| Outcome: | The proposed dataset reduces the error rate of sarcasm detection by 12.9% . it is based on audiovisual utterances annotated with sarcasm labels . |
RelationPrompt: Leveraging Prompts to Generate Synthetic Data for Zero-Shot Relation Triplet Extraction (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to extract relation triplets require large datasets and a fixed set of relations. |
| Approach: | They propose to use a sentence-based task setting to generalize relation extraction methods to unseen relation sets. |
| Outcome: | The proposed method can extract multiple relation triplets in a sentence using language model prompts and structured text approaches. |
A Dataset for Hyper-Relational Extraction and a Cube-Filling Approach (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods do not consider qualifier attributes for each relation triplet, such as time, quantity or location. |
| Approach: | They propose a hyper-relational extraction task to extract more specific facts from text using qualifiers. |
| Outcome: | The proposed model outperforms baselines and reveal possible directions for future research. |
MM-Align: Learning Optimal Transport-based Alignment Dynamics for Fast and Accurate Inference on Missing Modality Sequences (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to multimodal learning assume a complete input modality setting, i.e., each modality is either complete or completely missing in both training and test sets. |
| Approach: | They propose an alignment dynamics learning module based on the theory of optimal transport for missing data imputation and a denoising training algorithm to enhance the quality of iputation and accuracy of model predictions. |
| Outcome: | The proposed method performs faster and more accurate inferences under different missing conditions and alleviates the overfitting issue. |
LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models (2023.emnlp-main)
Copied to clipboard
Zhiqiang Hu, Lei Wang, Yihuai Lan, Wanyu Xu, Ee-Peng Lim, Lidong Bing, Xing Xu, Soujanya Poria, Roy Lee
| Challenge: | Large language models (LLMs) have shown unprecedented performance across various tasks. |
| Approach: | They propose an easy-to-use framework that integrates adapters into LLMs . they evaluate adapters on 14 datasets from two different reasoning tasks . |
| Outcome: | The proposed framework can be used to fine-tune open-access language models with task-specific data and instruction data. |
Evaluating AI for Finance: Is AI Credible at Assessing Investment Risk Appetite? (2025.emnlp-industry)
Copied to clipboard
Divij Chawla, Ashita Bhutada, Duc Anh Do, Abhinav Raghunathan, Vinod Sp, Cathy Guo, Dar Win Liew, Prannaya Gupta, Rishabh Bhardwaj, Rajat Bhardwaj, Soujanya Poria
| Challenge: | Our analysis was conducted on proprietary systems and open-weight models . FINRISKEVAL analyzed 1,720 profiles spanning a broad spectrum of possible risk categories . |
| Approach: | They evaluated proprietary AI systems and open-weight models to assess investment risk appetite using carefully curated user profiles. |
| Outcome: | The proposed models exhibit significant variance when user attributes that should not influence risk computation are changed. |
Contextual Inter-modal Attention for Multi-modal Sentiment Analysis (D18-1)
Copied to clipboard
Deepanway Ghosal, Md Shad Akhtar, Dushyant Chauhan, Soujanya Poria, Asif Ekbal, Pushpak Bhattacharyya
| Challenge: | Existing methods for multi-modal sentiment analysis are limited due to the use of text, visual and acoustic inputs. |
| Approach: | They propose a recurrent neural network based multi-modal attention framework that leverages contextual information for utterance-level sentiment prediction. |
| Outcome: | The proposed framework performs better on two multi-modal sentiment analysis benchmark datasets with accuracies of 82.31% and 79.80% for the MOSI and MOSEI datasets. |
MTAG: Modal-Temporal Attention Graph for Unaligned Human Multimodal Language Sequences (2021.naacl-main)
Copied to clipboard
Jianing Yang, Yongxin Wang, Ruitao Yi, Yuying Zhu, Azaan Rehman, Amir Zadeh, Soujanya Poria, Louis-Philippe Morency
| Challenge: | a novel graph-based neural model for multimodal sequential data is proposed . fusion is the process of blending information from multiple modalities, usually preceded by alignment . |
| Approach: | They propose a graph-based neural model that converts unaligned data into a modal-temporal graph . they use a dynamic pruning and read-out technique to efficiently process the graph fusion operation . |
| Outcome: | The proposed model performs state-of-the-art on multimodal sentiment analysis and emotion recognition benchmarks while utilizing significantly fewer model parameters. |
More Identifiable yet Equally Performant Transformers for Text Classification (2021.acl-long)
Copied to clipboard
| Challenge: | Current studies prove that attention weights are not unique and therefore unfit for interpretation. |
| Approach: | They propose a transformer encoder layer that decouples the relationship between key and value vector and provides identifiable weights up to the desired length of the input. |
| Outcome: | The proposed model is more identifiable than previously thought but still prone to be non-unique attentions that make them unfit for interpretation. |
DoubleMix: Simple Interpolation-Based Data Augmentation for Text Classification (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods to improve the robustness of text classification models are token-, sentence-, and hiddenlevel augmentation. |
| Approach: | They propose an interpolation-based data augmentation approach called DoubleMix to improve the robustness of text classification models by learning the “shifted” features in hidden space. |
| Outcome: | The proposed approach outperforms several popular methods on six text classification benchmark datasets and visual analysis shows that the model features are highly interpretable. |
M-LongDoc: A Benchmark For Multimodal Super-Long Document Understanding And A Retrieval-Aware Tuning Framework (2025.emnlp-main)
Copied to clipboard
Yew Ken Chia, Liying Cheng, Hou Pong Chan, Maojia Song, Chaoqun Liu, Mahani Aljunied, Soujanya Poria, Lidong Bing
| Challenge: | Existing benchmarks for large multimodal models focus on short documents with less than 50 pages and are limited to extraction-based questions. |
| Approach: | They propose a retrieval-aware tuning approach to improve the accuracy of multimodal document reading by 4.6%. |
| Outcome: | The proposed framework improves the accuracy of model responses by 4.6% compared to existing benchmarks on documents with hundreds of pages and longer documents with more complex content. |
Understanding the Capabilities and Limitations of Large Language Models for Cultural Commonsense (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated substantial commonsense understanding through numerous benchmark evaluations. |
| Approach: | They conduct a comprehensive examination of the capabilities and limitations of several state-of-the-art LLMs in the context of cultural commonsense tasks. |
| Outcome: | The language used to query the LLMs can impact their performance on cultural-related tasks. |
Pixel-Level Reasoning Segmentation via Multi-turn Conversations (2025.acl-long)
Copied to clipboard
| Challenge: | Existing visual perception systems focus on region-level segmentation in single-turn dialogues . existing systems cannot reason at the pixel level and comprehend dynamic user intent . |
| Approach: | They propose a task that tracks evolving user intent via multi-turn interactions for fine-grained segmentation. |
| Outcome: | The proposed method outperforms existing baselines in segmentation and reasoning metrics. |
CICERO: A Dataset for Contextualized Commonsense Inference in Dialogues (2022.acl-long)
Copied to clipboard
| Challenge: | Fig. 1a shows an example where commonsense knowledge is crucial in sifting relevant information from the context. |
| Approach: | They curate a dataset of dyadic conversations with five types of utterance-level reasoning-based inferences: cause, subsequent event, prerequisite, motivation, and emotional reaction. |
| Outcome: | The dataset contains 53,105 of such inferences from 5,672 dialogues. |
Emma-X: An Embodied Multimodal Action Model with Grounded Chain of Thought and Look-ahead Spatial Reasoning (2025.acl-long)
Copied to clipboard
| Challenge: | Visual-Language-Action models lack the ability to generate actionable policies tailored to specific robotic embodiments. |
| Approach: | They propose an embodied multimodal action model with Grounded Chain of Thought and Look-ahead Spatial Reasoning that enhances spatial reasoning and task planning. |
| Outcome: | The proposed model improves on existing baselines in tasks requiring spatial reasoning and grounding reasoning. |
Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task Arithmetic (2024.acl-long)
Copied to clipboard
| Challenge: | RESTA is a tool to perform LLM realignment towards safety, which gets compromised due to downstream task fine-tuning. |
| Approach: | They propose to add a safety vector to the weights of a compromised model by arithmetic and demonstrate its generalizability on three existing safety evaluation benchmarks and a multilingual benchmark dataset. |
| Outcome: | The proposed model reduces harmfulness of the model from 18.6% to 5.1% and from 9.2% to 1.5% in parameter-efficient and full fine-tuning while maintaining most of the system's performance on the task. |
PREMISE: Matching-based Prediction for Accurate Review Recommendation (2025.findings-naacl)
Copied to clipboard
| Challenge: | Experimental results show that PREMISE achieves promising performance with less computational cost. |
| Approach: | They propose a new architecture for matching-based learning in multimodal fields for the MRHP task. |
| Outcome: | The proposed architecture significantly boosts performance on multimodal tasks with less computational cost compared to the state-of-the-art fusion-based methods. |
Ferret: Faster and Effective Automated Red Teaming with Reward-Based Scoring Technique (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing red-teaming methods generate adversarial attacks to identify vulnerabilities, but they face slow performance, limited categorical diversity, and high resource demands. |
| Approach: | They propose a method that generates multiple adversarial prompt mutations per iteration and ranks them using scoring functions. |
| Outcome: | The proposed method achieves a 95% attack success rate and reduces time to a 90% ASR by 15.2%. |
STaCK: Sentence Ordering with Temporal Commonsense Knowledge (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to sentence order prediction ignore the importance of document level global information, i.e., while predicting relative order of two sentences (s i , s j) other sentences sk from the same document do not play any role. |
| Approach: | They propose a framework based on graph neural networks and temporal commonsense knowledge to model global information and predict relative order of sentences. |
| Outcome: | The proposed method is naturally suitable for order prediction on five different datasets and has potential applications in the evaluation of the quality of machinegenerated documents. |
Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on multimodal sentiment analysis relies on back-propagated task loss or geometric property of feature spaces to produce favorable fusion results. |
| Approach: | They propose a framework which hierarchically maximizes the Mutual Information (MI) in unimodal input pairs and between multimodal fusion result and unimod input to maintain task-related information through multimodal integration. |
| Outcome: | The proposed framework maximizes the Mutual Information (MI) in unimodal input pairs and between multimodal fusion result and unimodulated input to maintain task-related information through multimodal integration. |
SAT: Improving Semi-Supervised Text Classification with Simple Instance-Adaptive Self-Training (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for semi-supervised text classification have shown great performance in few-shot scenarios, where both labeled and unlabeled data are utilized. |
| Approach: | They propose a simple instance-adaptive self-training method for semi-supervised text classification that generates two augmented views for each unlabeled data and trains a meta learner to identify relative strength of augmentations based on the similarity between the original view and the augmented view. |
| Outcome: | The proposed method consistently shows competitive performance with varying sizes of labeled training data compared to existing semi-supervised learning methods. |
Evaluating LLMs’ Mathematical and Coding Competency through Ontology-guided Interventions (2025.findings-acl)
Copied to clipboard
| Challenge: | Current large language models have shown impressive performance on logical reasoning benchmarks . however, the true depth of their competencies and robustness in reasoning tasks remains an open question . |
| Approach: | They propose a general ontology of perturbations and a semi-automatic method to apply perturbations to arithmetic reasoning and code generation datasets to test their LLMs' capabilities. |
| Outcome: | The proposed model outperforms existing models on arithmetic reasoning and code generation tasks. |
WalledEval: A Comprehensive Safety Evaluation Toolkit for Large Language Models (2024.emnlp-demo)
Copied to clipboard
Prannaya Gupta, Le Yau, Hao Low, I-Shiang Lee, Hugo Lim, Yu Teoh, Koh Hng, Dar Liew, Rishabh Bhardwaj, Rajat Bhardwaj, Soujanya Poria
| Challenge: | Potential harms include training data leakage, biases in responses and decision-making, and unauthorized use for purposes such as terrorism and the generation of sexually explicit content. |
| Approach: | WalledEval is a comprehensive AI safety testing toolkit designed to evaluate large language models. |
| Outcome: | The framework supports both LLM and judge benchmarking and incorporates custom mutators to test safety against various text-style mutations such as future tense and paraphrasing. |
Analyzing Modality Robustness in Multimodal Sentiment Analysis (2022.naacl-main)
Copied to clipboard
| Challenge: | despite its importance, little attention has been paid to improving the robustness of multimodal models. |
| Approach: | They propose simple diagnostic checks for modality robustness in a trained multimodal model . they find MSA models highly sensitive to a single modality, which creates issues . |
| Outcome: | The proposed checks show that models are highly sensitive to a single modality, which creates issues in their robustness. |
SANCL: Multimodal Review Helpfulness Prediction with Selective Attention and Natural Contrastive Learning (2022.coling-1)
Copied to clipboard
| Challenge: | e-commerce has become a research hotspot for review helpfulness prediction . a new approach to help predict helpfulness of multimodal product reviews is proposed . |
| Approach: | They propose a machine learning task to identify helpfulness of multimodal product reviews . they use a probe-based strategy to enforce high attention weights on regions of greater significance . |
| Outcome: | The proposed model achieves state-of-the-art performance with lower memory consumption on two benchmark datasets with three categories. |
A Robust Information-Masking Approach for Domain Counterfactual Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Domain shift is a big challenge in NLP, but many approaches fail to leverage domain-specific nuances relevant to the task at hand. |
| Approach: | They propose a method that uses frequency-based masking to transform a text from the source domain to a target domain. |
| Outcome: | The proposed method outperforms baselines on 10 out of 12 domain-counterfactual classification settings with an average of 1.7% improvement in accuracy metric. |
Reasoning Paths Optimization: Learning to Reason and Explore From Diverse Paths (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Advanced models such as OpenAI o1 exhibit impressive problem-solving capabilities, but they may still falter on more complex problems, making errors that disrupt their reasoning paths. |
| Approach: | They propose a framework that encourages favorable branches at each reasoning step while penalizing unfavorable ones, enhancing the model’s overall problem-solving performance. |
| Outcome: | The proposed framework improves reasoning performance on multi-step reasoning tasks such as math word problems and science-based exam questions. |
Sowing the Wind, Reaping the Whirlwind: The Impact of Editing Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) face challenges in maintaining accuracy due to the dynamic nature of world knowledge. |
| Approach: | They propose to use a benchmark dataset to investigate the effects of model edits on model safety metrics and guardrails. |
| Outcome: | The proposed dataset sheds light on how the edits, impact the model’s safety metrics and guardrails. |
DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation (D19-1)
Copied to clipboard
| Challenge: | Emotion recognition in conversation (ERC) has received much attention lately due to its potential widespread applications in diverse areas, such as health-care, education, and human resources. |
| Approach: | They propose a graph neural network-based approach to emotion recognition in conversation that leverages self and inter-speaker dependency of the interlocutors to model conversational context. |
| Outcome: | The proposed method outperforms the current state-of-the-art on a number of benchmark emotion classification datasets while minimizing context propagation issues. |
AlgoPuzzleVQA: Diagnosing Multimodal Reasoning Challenges of Language Models with Algorithmic Multimodal Puzzles (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing datasets focused on visual question-answering focus on visual, language, and algorithmic knowledge . a new study examines the performance of multimodal language models in solving algorithmic puzzles . |
| Approach: | They propose a dataset to test the capabilities of multimodal language models in solving algorithmic puzzles. |
| Outcome: | The proposed dataset is generated automatically from human code. |
Mustango: Toward Controllable Text-to-Music Generation (2024.naacl-long)
Copied to clipboard
| Challenge: | Mustango is a text-to-music system that allows music-domain-knowledge-informed text-based music generation. |
| Approach: | They propose a music-domain-knowledge-inspired text-to-music system based on diffusion that generates music with captions that include specific instructions related to chords, beats, key and tempo. |
| Outcome: | The proposed system outperforms existing models in music generation tasks. |
Safety Arithmetic: A Framework for Test-time Safety Alignment of Language Models by Steering Parameters and Activations (2024.emnlp-main)
Copied to clipboard
| Challenge: | Current alignment methods struggle with dynamic user intentions and complex objectives, making models vulnerable to harmful content. |
| Approach: | They propose a training-free framework that enhances LLM safety across different scenarios. |
| Outcome: | The proposed framework significantly improves safety measures, reduces over-safety, and maintains model utility, outperforming existing methods in ensuring safe content generation. |
HYPERTTS: Parameter Efficient Adaptation in Text to Speech Using Hypernetworks (2024.lrec-main)
Copied to clipboard
| Challenge: | Neural text-to-speech (TTS) systems limited to predefined speaker styles or specific sets of speaker IDs. |
| Approach: | They propose a network that can adapt adapter parameters to new speakers . they compare two domain adaptation settings and find it to be very efficient . |
| Outcome: | The proposed Adapters improve speech synthesis performance on two domains and compare them with baselines. |
From Grounding to Manipulation: Case Studies of Foundation Model Integration in Embodied Robotic Systems (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new study examines the operational characteristics of different integration strategies for robotics . end-to-end vision-language-action models implicitly unify perception and planning . |
| Approach: | They propose end-to-end vision-language-action models that implicitly unify perception and planning . they also propose modular pipelines using either vision-linguistic models or MLLMs . |
| Outcome: | The proposed frameworks implicitly unify perception and planning, and modular pipelines using either vision-language models or multimodal large language models. |
DiffPO: Diffusion-styled Preference Optimization for Inference Time Alignment of Large Language Models (2025.acl-long)
Copied to clipboard
Ruizhe Chen, Wenhao Chai, Zhifei Yang, Xiaotian Zhang, Ziyang Wang, Tony Quek, Joey Tianyi Zhou, Soujanya Poria, Zuozhu Liu
| Challenge: | Inference-time alignment approaches still face limitations due to policy-specific value functions and latency during the inference phase. |
| Approach: | They propose an efficient and policy-agnostic preference optimization method that avoids time latency associated with token generation. |
| Outcome: | The proposed method achieves a favorable trade-off between alignment quality and inference-time latency. |
KinGDOM: Knowledge-Guided DOMain Adaptation for Sentiment Analysis (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to cross-domain sentiment analysis cannot be reliably deployed due to the distributional mismatch between training and evaluation domains. |
| Approach: | They propose a framework that uses ConceptNet to enrich semantics of documents by providing domain-specific and domain-general background concepts. |
| Outcome: | The proposed framework improves on a domain-adversarial baseline method and can be used in domain adaptation. |
Two is Better than Many? Binary Classification as an Effective Approach to Multi-Choice Question Answering (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to multi-choice question answering are based on binary classifications instead of scoring each answer as a single class. |
| Approach: | They propose a simple refactoring of multi-choice question answering tasks as a series of binary classifications and propose re-framing to make them more efficient. |
| Outcome: | The proposed approach is significantly more effective across different tasks and models. |
CM-TTS: Enhancing Real Time Text-to-Speech Synthesis Efficiency through Weighted Samplers and Consistency Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Neural Text-to-Speech systems are a promising approach for high-fidelity speech synthesis . but the efficiency of multi-step sampling in Diffusion Models presents challenges . |
| Approach: | They propose a novel architecture grounded in consistency models to improve model convergence. |
| Outcome: | The proposed architecture achieves top-quality speech synthesis in fewer steps without adversarial training or pre-trained model dependencies. |
CASCADE: Contextual Sarcasm Detection in Online Discussion Forums (C18-1)
Copied to clipboard
| Challenge: | Existing studies on sarcasm detection focus on lexical, syntactic and semantic cues, but sarcasm can be expressed implicitly without such cue. |
| Approach: | They propose a ContextuAl SarCasm DEtector which extracts contextual information from the discourse of a discussion thread. |
| Outcome: | The proposed model improves on a large Reddit corpus. |
kNN-CM: A Non-parametric Inference-Phase Adaptation of Parametric Text Classifiers (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on text-discriminating properties of semi-parametric models have not been done on non-parameter models. |
| Approach: | They propose an inference-phase approach that incorporates a neighborhood search into a model to enhance the capacity of a pre-trained parametric text classifier. |
| Outcome: | The proposed model improves performance on eight SuperGLUE tasks, three adversarial natural language inference datasets, 11 question-answering (QA) datasets and two sentiment classification datasets. |
Multi-task Learning for Multi-modal Emotion Recognition and Sentiment Analysis (N19-1)
Copied to clipboard
Md Shad Akhtar, Dushyant Chauhan, Deepanway Ghosal, Soujanya Poria, Asif Ekbal, Pushpak Bhattacharyya
| Challenge: | Existing frameworks for sentiment and emotion analysis are not efficient for inter-task learning. |
| Approach: | They propose a multi-task learning framework that performs sentiment and emotion analysis together. |
| Outcome: | The proposed framework improves on a CMU-MOSEI dataset for sentiment and emotion analysis. |
Multimodal Language Analysis in the Wild: CMU-MOSEI Dataset and Interpretable Dynamic Fusion Graph (P18-1)
Copied to clipboard
| Challenge: | Analyzing human multimodal language is emerging area of research in NLP. |
| Approach: | They propose a multimodal fusion technique to exploit how modalities interact in multimodal language. |
| Outcome: | The proposed technique exploits how modalities interact with each other in human multimodal language. |
Few-shot Joint Multimodal Aspect-Sentiment Analysis Based on Generative Multimodal Prompt (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies require massive labeled data to train models for multimodal data analysis. |
| Approach: | They propose a novel multimodal prompt model that captures specific aspect terms in a few-shot scenario. |
| Outcome: | The proposed model outperforms baselines on two MABSA-related tasks on a few-shot dataset. |
Self-Adaptive Sampling for Accurate Video Question Answering on Image Text Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Image–text models (ITMs) are the prevalent architecture to solve video question–answering tasks, which requires only a few input frames to save huge computational cost compared to video–language models. |
| Approach: | They propose a sampling method based on question–frame correlation that is efficient for the few-frame situations. |
| Outcome: | The proposed method can boost the performance of image–text pretrained models and have a wide application scenario in terms of model architectures and dataset types. |
Conversational Memory Network for Emotion Recognition in Dyadic Dialogue Videos (N18-1)
Copied to clipboard
Devamanyu Hazarika, Soujanya Poria, Amir Zadeh, Erik Cambria, Louis-Philippe Morency, Roger Zimmermann
| Challenge: | Existing methods for recognizing emotions in conversations ignore inter-speaker dependency relations . dyadic conversations are a form of dialogue between two entities . |
| Approach: | They propose a deep neural framework which leverages contextual information from the conversation history to model past utterances of each speaker into memories. |
| Outcome: | The proposed framework improves by 3 4% over the state-of-the-art in recognizing emotions in dyadic conversational videos. |
Reward-Guided Tree Search for Inference Time Alignment of Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Inference-time computation methods enhance performance of Large Language Models by leveraging additional computational resources. |
| Approach: | They propose an inference-time alignment method that leverages a reward model to achieve alignment through reward-guided tree search. |
| Outcome: | The proposed method outperforms other inference-time alignment methods on two benchmarks . it achieves comparable performance to preference-tuned models on both benchmarks, authors show . |