Papers by Amit Sheth
FACTIFY3M: A benchmark for multimodal fact verification with explainability through 5W Question-Answering (2023.emnlp-main)
Copied to clipboard
Megha Chakraborty, Khushbu Pahwa, Anku Rani, Shreyas Chatterjee, Dwip Dalal, Harshit Dave, Ritvik G, Preethi Gurumurthy, Adarsh Mahor, Samahriti Mukherjee, Aditya Pakala, Ishan Paul, Janvita Reddy, Arghya Sarkar, Kinjal Sensharma, Aman Chadha, Amit Sheth, Amitava Das
| Challenge: | Disinformation can cause disruption in the share market, panic and anxiety in society, and even death during crises. |
| Approach: | a new dataset is being developed to help combat disinformation . the dataset is a multimodal fake news dataset with 5W question-answering . |
| Outcome: | FACTIFY 3M is the largest dataset and benchmark for multimodal fact verification. |
Counter Turing Test (CT2): Investigating AI-Generated Text Detection for Hindi - Ranking LLMs based on Hindi AI Detectability Index (ADI_hi) (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a growing number of large language models are being used to detect AI-generated text . a recent study has found that some techniques to bypass detection are fragile . |
| Approach: | They propose to use 26 LLMs to evaluate their proficiency in generating Hindi text . they propose to introduce a Hindi AI Detectability Index to assess and rank LLM models based on their detectability levels. |
| Outcome: | The proposed methods are effective in English, but struggle in Hindi . the proposed methods show that they are susceptible to fragility . |
KnowledgePrompts: Exploring the Abilities of Large Language Models to Solve Proportional Analogies via Knowledge-Enhanced Prompting (2025.coling-main)
Copied to clipboard
Thilini Wijesiriwardene, Ruwan Wickramarachchi, Sreeram Reddy Vennam, Vinija Jain, Aman Chadha, Amitava Das, Ponnurangam Kumaraguru, Amit Sheth
| Challenge: | Proportional analogies are used to assess linguistic and cognitive abilities. |
| Approach: | They propose a dataset for proportional analogy completion and evaluate its performance in large-scale learning environments. |
| Outcome: | The proposed model achieves 55% accuracy in knowledge-enhanced prompts. |
Multi-Task Learning Framework for Mining Crowd Intelligence towards Clinical Treatment (N18-2)
Copied to clipboard
| Challenge: | In recent past, social media has emerged as an active platform in the context of healthcare and medicine. |
| Approach: | They propose to use a novel adversarial learning approach to capture medical sentiments expressed in a medical blog to analyze the user's opinions on health-related issues. |
| Outcome: | The proposed framework can capture the user's opinions on health-related issues at a medical blog level. |
A Practical Incremental Learning Framework For Sparse Entity Extraction (C18-1)
Copied to clipboard
| Challenge: | Existing approaches to extract entities from textual data are expensive and unattractive due to the high cost of training. |
| Approach: | They propose a framework that integrates Entity Set Expansion and Active Learning to reduce the cost of data annotation. |
| Outcome: | The proposed framework reduces the cost of sparse entity annotation by 85% and 45% while maintaining high accuracy. |
YinYang-Align: A new Benchmark for Competing Objectives and Introducing Multi-Objective Preference based Text-to-Image Alignment (2025.findings-acl)
Copied to clipboard
Amitava Das, Yaswanth Narsupalli, Gurpreet Singh, Vinija Jain, Vasu Sharma, Suranjana Trivedy, Aman Chadha, Amit Sheth
| Challenge: | Recent controversies highlight the need for robust alignment mechanisms in text-to-image systems. |
| Approach: | They propose a framework to evaluate T2I systems across six contradictory alignment objectives . objectives highlight key trade-offs such as artistic freedom and cultural sensitivity . |
| Outcome: | The proposed framework achieves superior alignment across all objectives. |
Counter Turing Test (CT2): AI-Generated Text Detection is Not as Easy as You May Think - Introducing AI Detectability Index (ADI) (2023.emnlp-main)
Copied to clipboard
Megha Chakraborty, S.M Towhidul Islam Tonmoy, S M Mehedi Zaman, Shreya Gautam, Tanay Kumar, Krish Sharma, Niyar Barman, Chandan Gupta, Vinija Jain, Aman Chadha, Amit Sheth, Amitava Das
| Challenge: | a number of issues have arisen regarding the risk and consequences of AI-generated text detection. |
| Approach: | They propose a counter-turing test to evaluate the robustness of existing AGTD methods . they propose ADI, a quantifiable spectrum to assess detectability of LLMs . |
| Outcome: | The proposed method evaluates the robustness of existing AGTD methods . it shows that larger LLMs tend to have lower ADI, indicating they are less detectable . |
On the Relationship between Sentence Analogy Identification and Sentence Structure Encoding in Large Language Models (2024.findings-eacl)
Copied to clipboard
Thilini Wijesiriwardene, Ruwan Wickramarachchi, Aishwarya Naresh Reganti, Vinija Jain, Aman Chadha, Amit Sheth, Amitava Das
| Challenge: | Analogies facilitate the transfer of meaning and knowledge from one domain to another. |
| Approach: | They propose to use large language models to encode syntactic and semantic structures of sentences to identify sentence analogies. |
| Outcome: | The LLMs which capture syntactic structures better, also have higher abilities in identifying sentence analogies. |
IndustryAssetEQA: A Neurosymbolic Operational Intelligence System for Embodied Question Answering in Industrial Asset Maintenance (2026.acl-industry)
Copied to clipboard
| Challenge: | Industrial maintenance assistants produce generic explanations that are weakly grounded in telemetry and omit verifiable provenance. |
| Approach: | They propose a neurosymbolic operational intelligence system that combines episode-centric telemetry representations with a Failure Mode and Effects Analysis Knowledge Graph to enable Embodied Question Answering over industrial assets. |
| Outcome: | The proposed system improves structural validity by up to +0.51, counterfactual accuracy by up . to +0.47, and explanation entailment by +0.64, while reducing severe expert-rated overclaims from 28% to 2%. |
Tutorial Proposal: Hallucination in Large Language Models (2024.lrec-tutorials)
Copied to clipboard
| Challenge: | Grasping the intricacies of hallucination in LLMs can be daunting, especially for those new to the field. |
| Approach: | This tutorial aims to bridge the gap between the field and the field of hallucination . it will explore the key aspects of hallucinonation, including benchmarking, detection, and mitigation techniques . |
| Outcome: | This tutorial will explore the key aspects of hallucination in LLMs . it will also explore the specific constraints and shortcomings of current approaches . |
The Troubling Emergence of Hallucination in Large Language Models - An Extensive Definition, Quantification, and Prescriptive Remediations (2023.emnlp-main)
Copied to clipboard
Vipula Rawte, Swagata Chakraborty, Agnibh Pathak, Anubhav Sarkar, S.M Towhidul Islam Tonmoy, Aman Chadha, Amit Sheth, Amitava Das
| Challenge: | Recent advances in Large Language Models have generated widespread acclaim, but hallucination has also emerged as a by-product. |
| Approach: | They propose a fine-grained discourse on profiling hallucination based on its degree, orientation, and category . they categorize hallucines into six types: acronym ambiguity, generated golem, virtual voice, geographic erratum, time wrap . |
| Outcome: | The proposed method categorizes hallucination into six types based on their degree, orientation, and category . |
Medical Knowledge-enriched Textual Entailment Framework (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to achieving robust medical question answering systems lack a textual entailment framework that can capture the con-text beyond the sentence. |
| Approach: | They propose a medical knowledge-enriched textual entailment framework that can acquire a semantic and global representation of the input medical text with the help of a relevant domain-specific knowledge graph. |
| Outcome: | The proposed framework achieves 8.27% improvement over existing language models on MEDIQA-RQE dataset. |
Identifying Depressive Symptoms from Tweets: Figurative Language Enabled Multitask Learning Framework (2020.coling-main)
Copied to clipboard
Shweta Yadav, Jainish Chauhan, Joy Prakash Sain, Krishnaprasad Thirunarayan, Amit Sheth, Jeremiah Schumm
| Challenge: | Existing studies on social media for deriving mental health status of users focus on the depression detection task. |
| Approach: | They propose to use a BERT based robust multi-task learning framework to accurately identify the depressive symptoms using the auxiliary task of figurative usage detection. |
| Outcome: | The proposed model improves its robustness and reliability for distinguishing the depression symptoms. |
SEPSIS: I Can Catch Your Lies – A New Paradigm for Deception Detection (2025.acl-srw)
Copied to clipboard
Anku Rani, Dwip Dalal, Shreya Gautam, Pankaj Gupta, Vinija Jain, Aman Chadha, Amit Sheth, Amitava Das
| Challenge: | a new framework categorizes deception into three forms: lies of omission, lies of commission, and lies of influence . a novel framework for deception detection leveraging NLP techniques is proposed . |
| Approach: | They propose a framework that categorizes deception into three forms: lies of omission, lies of commission, and lies of influence. |
| Outcome: | The proposed framework achieves an impressive F1 score of 0.87 across all layers . it can be used to investigate lies of omission, lies of commission and lies of influence . |
Ask Me Again Differently: GRAS for Measuring Bias in Vision Language Models on Gender, Race, Age, and Skin Tone (2026.findings-eacl)
Copied to clipboard
| Challenge: | Using vision language models, we examine demographic biases in VLMs across gender, race, age, and skin tone. |
| Approach: | They propose a benchmark for uncovering demographic biases in Vision Language Models . they propose 'Gras Bias Score' to quantify bias in VLMs based on gender, race, age and skin tone . |
| Outcome: | The proposed model achieves 98, far from the unbiased ideal of 0. |
FACTIFY-5WQA: 5W Aspect-based Fact Verification through Question Answering (2023.acl-long)
Copied to clipboard
Anku Rani, S.M Towhidul Islam Tonmoy, Dwip Dalal, Shreya Gautam, Megha Chakraborty, Aman Chadha, Amit Sheth, Amitava Das
| Challenge: | Contemporary fact-checking systems focus on estimating truthfulness using numerical scores which are not human-interpretable. |
| Approach: | They propose a 5W framework for question-answer-based fact explainability that can assist human fact-checkers in asking relevant questions . they propose masked language model which generates QA pairs for claims and a baseline QA system that automatically locates those answers from evidence documents. |
| Outcome: | The proposed framework can assist human fact-checkers in asking relevant questions related to a fact, which can then be validated separately to reach a final verdict. |
Location Name Extraction from Targeted Text Streams using Gazetteer-based Statistical Language Models (C18-1)
Copied to clipboard
| Challenge: | Location name extraction tool (LNEx) is a statistical language for extracting location names from informal and unstructured social media data. |
| Approach: | They propose a location name extraction tool that extracts location names from social media data . they use n-gram statistics and location-related dictionaries to evaluate an observed n in targeted text . |
| Outcome: | The proposed tool outperforms state-of-the-art taggers on 4,500 event-specific tweets . it improves the average F-Score by 33-179%, outperforming all tagger . |
ANALOGICAL - A Novel Benchmark for Long Text Analogy Evaluation in Large Language Models (2023.findings-acl)
Copied to clipboard
Thilini Wijesiriwardene, Ruwan Wickramarachchi, Bimal Gajera, Shreeyash Gowaikar, Chandan Gupta, Aman Chadha, Aishwarya Naresh Reganti, Amit Sheth, Amitava Das
| Challenge: | Modern large language models are evaluated on extrinsic measures based on benchmarks such as GLUE and SuperGLUE. |
| Approach: | They propose a benchmark to intrinsically evaluate large language models across a taxonomy of analogies of long text with six levels of complexity. |
| Outcome: | The proposed benchmark evaluates LLMs across a taxonomy of analogies of long text with six levels of complexity. |