Papers by Vasu Sharma
MiSCHiEF: A Benchmark in Minimal-Pairs of Safety and Culture for Holistic Evaluation of Fine-Grained Image-Caption Alignment (2026.eacl-short)
Copied to clipboard
Sagarika Banerjee, Tangatar Madi, Advait Swaminathan, Jolie Nguyen, Shivank Garg, Kevin Zhu, Vasu Sharma
| Challenge: | Fine-grained image-caption alignment is crucial for vision-language models in socially critical contexts. |
| Approach: | They present a benchmarking dataset for fine-grained image-caption alignment in safety and culture contexts. |
| Outcome: | The proposed benchmarks show that models perform better at confirming correct pairs than rejecting incorrect ones on dual alignment tasks. |
YinYang-Align: A new Benchmark for Competing Objectives and Introducing Multi-Objective Preference based Text-to-Image Alignment (2025.findings-acl)
Copied to clipboard
Amitava Das, Yaswanth Narsupalli, Gurpreet Singh, Vinija Jain, Vasu Sharma, Suranjana Trivedy, Aman Chadha, Amit Sheth
| Challenge: | Recent controversies highlight the need for robust alignment mechanisms in text-to-image systems. |
| Approach: | They propose a framework to evaluate T2I systems across six contradictory alignment objectives . objectives highlight key trade-offs such as artistic freedom and cultural sensitivity . |
| Outcome: | The proposed framework achieves superior alignment across all objectives. |
DPO Kernels: A Semantically-Aware, Kernel-Enhanced, and Divergence-Rich Paradigm for Direct Preference Optimization (2025.findings-acl)
Copied to clipboard
Amitava Das, Suranjana Trivedy, Danush Khanna, Yaswanth Narsupalli, Basab Ghosh, Rajarshi Roy, Gurpreet Singh, Vinija Jain, Vasu Sharma, Aishwarya Naresh Reganti, Aman Chadha
| Challenge: | Direct Preference Optimization (DPO) is a cornerstone for preference alignment but is constrained by fixed divergence measures and limited feature transformations. |
| Approach: | They propose a new enhancement of Direct Preference Optimization that integrates kernel methods to overcome these challenges. |
| Outcome: | The proposed model improves divergence measures and features by using kernels . the proposed model achieves state-of-the-art generalization in factuality, safety, reasoning, and instruction following . |
Alignment Quality Index (AQI) : Beyond Refusals: AQI as an Intrinsic Alignment Diagnostic via Latent Geometry, Cluster Divergence, and Layer wise Pooled Representations (2025.emnlp-main)
Copied to clipboard
Abhilekh Borah, Chhavi Sharma, Danush Khanna, Utkarsh Bhatt, Gurpreet Singh, Hasnat Md Abdullah, Raghav Kaushik Ravi, Vinija Jain, Jyoti Patel, Shubham Singh, Vasu Sharma, Arpita Vats, Rahul Raja, Aman Chadha, Amitava Das
| Challenge: | a new metric measures the quality of large language models (LLMs) that detects hidden misalignments and jailbreak risks. |
| Approach: | They propose a decoding-invariant metric that measures latent safety failures . they propose 'Alignment Quality Index' to measure latent activations in latent space . |
| Outcome: | The proposed metric detects latent safety failures overlooked by behavioral benchmarks and jailbreaks. |
NovelHopQA: Diagnosing Multi-Hop Reasoning Failures in Long Narrative Contexts (2025.emnlp-main)
Copied to clipboard
| Challenge: | Current large language models struggle to answer questions that span tens of thousands of tokens. |
| Approach: | They evaluate 1–4 hop QA over 64k–128k-token excerpts from 83 novels . they find consistent accuracy drops with increased hops and context length . |
| Outcome: | The novelhopqa benchmark evaluates 1–4 hop QA over 64k–128k-token excerpts from 83 public-domain novels. |
Tweet Based Reach Aware Temporal Attention Network for NFT Valuation (2022.findings-emnlp)
Copied to clipboard
Ramit Sawhney, Megh Thakkar, Ritesh Soun, Atula Neerkaje, Vasu Sharma, Dipanwita Guhathakurta, Sudheer Chava
| Challenge: | Non-Fungible Tokens (NFTs) are a relatively unexplored class of assets due to their extremely volatile nature. |
| Approach: | They propose a reach-aware temporal learning approach to predict future NFT trends from a dataset consisting of over 1.3 million tweets and 180 thousand NFT transactions . |
| Outcome: | The proposed model outperforms state-of-the-art models by an average of 36% on a dataset consisting of over 1.3 million tweets and 180 thousand NFT transactions spanning over 15 NFT collections. |
Rosetta-PL: Propositional Logic as a Benchmark for Large Language Model Reasoning (2025.naacl-srw)
Copied to clipboard
Shaun Lee Baek, Shaun Esua-Mensah, Cyrus Tsui, Sejan Vigneswaralingam, Abdullah Alali, Michael Lu, Vasu Sharma, Kevin Zhu
| Challenge: | Large Language Models (LLMs) are primarily trained on high-resource natural languages, limiting their effectiveness in low-resourced settings and in tasks requiring deep logical reasoning. |
| Approach: | They propose to use a dataset of logical propositions from Lean into a custom logical language to evaluate LLMs' logical reasoning and generalization capabilities in a controlled environment. |
| Outcome: | The proposed model improves accuracy and accuracy beyond 20,000 training samples. |