| Challenge: | a new question answering task on instructional videos is needed due to their verbose nature . factoid questions are only a small part of what people actually want to ask on video contents . |
| Approach: | They propose a question answering task on instructional videos based on video transcripts . they use a dataset consisting of 6,000 manually collected triples of (video, question, answer span) |
| Outcome: | The proposed task focuses on screencast tutorial videos pertaining to an image editing program. |
Similar Papers
Video Question Answering: Datasets, Algorithms and Challenges (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in video question answering have led to a surge in popularity . despite the popularity, VideoQA remains one of the greatest challenges . |
| Approach: | They categorize the video question-answer datasets into normal VideoQA, multi-modal VideoQA and knowledge-based VideoQA according to the modalities invoked in the question-announcement pairs. |
| Outcome: | The proposed methods are mainly designed for Factoid QA and inference VideoQA . the proposed methods have been compared with other methods and are robust and interpretable. |
LifeQA: A Real-life Dataset for Video Question Answering (2020.lrec-1)
Copied to clipboard
Santiago Castro, Mahmoud Azab, Jonathan Stroud, Cristina Noujaim, Ruoyao Wang, Jia Deng, Rada Mihalcea
| Challenge: | Existing video question answering datasets consist of movies and TV shows, but they are not representative of our day-to-day lives. |
| Approach: | They propose a benchmark dataset for video question answering that focuses on day-to-day situations. |
| Outcome: | The proposed dataset analyzes the challenging but realistic aspects of LifeQA . it consists of video clips and over 2.3k multiple-choice questions . |
TVQA: Localized, Compositional Video Question Answering (D18-1)
Copied to clipboard
| Challenge: | Recent studies have focused on image-based question-answering (QA) tasks, but little has been done on video-based QA. |
| Approach: | They present a large-scale video QA dataset based on 6 popular TV shows . they provide analysis of the new dataset and trainable neural network framework . |
| Outcome: | The proposed dataset includes 152,545 QA pairs from 21,793 clips spanning over 460 hours of video. |
Mulan: A Multi-Level Alignment Model for Video Question Answering (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods focus on visual-language alignment at the video level, but they do not account for fine-grained semantic interaction between video and text. |
| Approach: | They propose a multi-level Alignment Model for Video Question Answering that establishes alignment between visual and textual modalities at the object-level, frame-level and video-level. |
| Outcome: | The proposed model outperforms state-of-the-art methods even with a small amount of extra visual-language pre-training data and a reduced number of trainable parameters. |
Towards Answering Health-related Questions from Medical Videos: Datasets and Approaches (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing systems that can provide visual answers from medical videos to natural language questions are limited by the availability of large datasets. |
| Approach: | They propose to use large-scale medical video datasets to provide visual answers to questions . they propose to combine multimodal and monomodal approaches to provide answers . |
| Outcome: | The proposed approach can provide visual answers from medical videos to natural language questions. |
MultiSpanQA: A Dataset for Multi-Span Question Answering (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing reading comprehension datasets focus on single-span answers, but multi-spread questions are less studied. |
| Approach: | They propose a new reading comprehension dataset that focuses on multi-span questions . they introduce new metrics for the purposes of multi--spontaneous question answering evaluation . |
| Outcome: | The proposed model beats baselines and achieves state-of-the-art on the existing dataset. |
In-the-Wild Video Question Answering (2022.coling-1)
Copied to clipboard
| Challenge: | Existing video understanding datasets focus on human interactions with little attention being paid to the “in the wild” settings. |
| Approach: | They propose a video understanding dataset of videos recorded outdoors . they propose identifying visual support for a given question and answer . |
| Outcome: | The proposed dataset examines the ability of models to understand videos, including video question answering, video captioning, and fill-inthe-blank tasks. |
CodeQA: A Question Answering Dataset for Source Code Comprehension (2021.findings-emnlp)
Copied to clipboard
| Challenge: | False. a free-form question answering dataset can serve as a useful research benchmark for source code comprehension. |
| Approach: | They propose a free-form question answering dataset for source code comprehension . they implement syntactic rules and semantic analysis to transform code comments into question-answer pairs. |
| Outcome: | The proposed dataset can serve as a useful research benchmark for source code comprehension. |
AraVQA: Building a New Arabic Factoid Visual Question Answering Dataset from Wikipedia (2026.acl-long)
Copied to clipboard
| Challenge: | Existing Arabic VQA datasets focus on culturally-specific and dialect-aware domains. |
| Approach: | They propose a pipeline that leverages Wikipedia template tags to extract relevant information for each image and utilize it to generate a new visual question answering dataset. |
| Outcome: | The proposed pipeline can enhance existing VLMs on Arabic VQA tasks by leveraging Wikipedia template tags. |
GazeVQA: A Video Question Answering Dataset for Multiview Eye-Gaze Task-Oriented Collaborations (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on the use of exocentric and egocentric videos in video question answering are focusing on eye-gaze information. |
| Approach: | They propose a task-oriented VQA dataset that captures eye-gaze information . they propose assisting models that ground the perceptual input into semantic information based on three different answer types . |
| Outcome: | The proposed model can ground the perceptual input into semantic information while reducing ambiguities. |