Papers by Tejas Gokhale
Self-Supervised Test-Time Learning for Reading Comprehension (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work on unsupervised question answering shows that models can be trained with procedurally generated question-answer pairs and achieve performance competitive with supervised methods. |
| Approach: | They propose a method that performs "test-time learning" on a given context . they use self-supervision to train models on synthetically generated question-answer pairs . |
| Outcome: | The proposed method outperforms current unsupervised methods and outperformed supervised methods. |
Video2Commonsense: Generating Commonsense Descriptions to Enrich Video Captioning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Observable changes in the scene are reflected in captions, but actions are also linked to social aspects such as intentions, effects, and attributes that describe the agent. |
| Approach: | They propose to generate captions from videos that describe latent aspects of the human agent's actions. |
| Outcome: | The proposed model can be used to describe latent aspects of human actions in video clips and answer questions about videos. |
MUTANT: A Training Paradigm for Out-of-Distribution Generalization in Visual Question Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | Availability of large-scale datasets has enabled statistical machine learning in vision and language understanding. |
| Approach: | They propose a training paradigm that exposes models to perceptually similar mutations of input . they show a 10.57% improvement in the VQA-CP challenge . |
| Outcome: | The proposed training paradigm improves on the visual question answering challenge with 10.57% accuracy. |
End-to-end Knowledge Retrieval with Multi-modal Queries (2023.acl-long)
Copied to clipboard
| Challenge: | a new task is proposed to learn knowledge retrieval with multimodal queries . a vision-language model can retrieve knowledge using images and text inputs . |
| Approach: | They propose a task for vision-language models to retrieve knowledge with multi-modal queries . they propose reViz, a model that integrates content from both text and image queries based on a multimodal query task . |
| Outcome: | The proposed task performs better under zero-shot settings than previous work on cross-modal retrieval. |
Side Effects of Erasing Concepts from Diffusion Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Concept erasure techniques (CETs) are used to erade text-to-image (T2I) generative models infringing on privacy, copyright and safety. |
| Approach: | They propose a side effect evaluation benchmark to measure the robustness of CETs. |
| Outcome: | The proposed techniques can be easily circumvented by hierarchy, prompts, and compositional variants of the target. |
Semantically Distributed Robust Optimization for Vision-and-Language Inference (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to integrate linguistic knowledge into training pipelines are under-explored. |
| Approach: | They propose a model-agnostic method that leverages linguistic transformations to infer a set of linguistic phenomena. |
| Outcome: | The proposed method improves on benchmark datasets with images and video and is generalizable to other V&L tasks. |
Generalized but not Robust? Comparing the Effects of Data Modification Methods on Out-of-Domain Generalization and Adversarial Robustness (2022.findings-acl)
Copied to clipboard
| Challenge: | Data modification has been proposed as an effective solution for generalizing to out-of-domain (OOD) inputs. |
| Approach: | They propose to use data modification to generalize to out-of-domain inputs . they also analyze their adversarial robustness using a synthetic dataset . |
| Outcome: | The proposed data modification strategies improve OOD accuracy and AR, but data filtering hurts OOD on other tasks. |
CRIPP-VQA: Counterfactual Reasoning about Implicit Physical Properties via Video Question Answering (2022.emnlp-main)
Copied to clipboard
| Challenge: | Videos often capture objects, their visible properties, their motion, and the interactions between different objects. |
| Approach: | They propose a video question answering dataset for reasoning about the implicit physical properties of objects in a scene. |
| Outcome: | The proposed dataset enables evaluation under several out-of-distribution settings – videos with objects with masses, coefficients of friction, and initial velocities that are not observed in the training distribution. |
Zero-Shot Multimodal Retrieval with Multi-Scale Contextual Representations (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to multimodal information retrieval (MMIR) lack generalization across different modalities and require annotated training data. |
| Approach: | They propose a fine-tuning-free, two-stage MMIR approach that couples efficient candidate filtering with fine-grained multimodal re-ranking. |
| Outcome: | The proposed approach outperforms supervised methods on 23 datasets. |
WeaQA: Weak Supervision via Captions for Visual Question Answering (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for training visual question answering models rely on datasets with human-annotated image-quest-answer triplets. |
| Approach: | They propose a method to train models with synthetic Q-A pairs generated procedurally from captions. |
| Outcome: | The proposed method trains models with synthetic Q-A pairs generated from captions on three VQA benchmarks. |
Unsupervised Natural Language Inference Using PHL Triplet Generation (2022.findings-acl)
Copied to clipboard
| Challenge: | In some cases, training samples may not be available or collecting them could be time-consuming and resource-intensive. |
| Approach: | They propose a procedural approach that leverages sentence transformations to collect PHL triplets for training NLI models. |
| Outcome: | The proposed model outperforms existing models on several NLI benchmarks with a set of sentence transformations. |
To Find Waldo You Need Contextual Cues: Debiasing Who’s Waldo (2022.acl-short)
Copied to clipboard
| Challenge: | Existing datasets for Person-centric visual grounding (PCVG) are biased and over-estimated . |
| Approach: | They propose to use a dataset for the Person-centric Visual Grounding task that requires pairing up a person’s name mentioned in a caption with a bounding box that points to the person in the image. |
| Outcome: | The proposed dataset offers a more practical benchmark for future improvements and better performance than the original dataset. |