| Challenge: | Existing video QA datasets only contain QA pairs without labels for key clips or regions needed to answer the question. |
| Approach: | They propose a framework that grounds evidence in both spatial and temporal domains to answer questions about videos using bounding boxes. |
| Outcome: | The proposed framework can produce interpretable spatio-temporal attention visualizations. |
Similar Papers
VideoQA-TA: Temporal-Aware Multi-Modal Video Question Answering (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for video question answering align visual or textual features directly with large language models, limiting the deep semantic association between modalities and hindering a comprehensive understanding of interactions within spatial and temporal contexts. |
| Approach: | They propose a temporal-aware framework for multi-modal video question answering that aligns videos and questions at fine-grained levels. |
| Outcome: | The proposed framework improves reasoning ability and accuracy of videoQA by aligning videos and questions at fine-grained levels. |
Dense-Caption Matching and Frame-Selection Gating for Temporal Localization in VideoQA (2020.acl-main)
Copied to clipboard
| Challenge: | Recent years have witnessed a paradigm shift in the way we get our information, and a lot of it. |
| Approach: | They propose a video question answering model which integrates multi-modal input sources and finds temporally relevant information to answer questions. |
| Outcome: | The proposed model outperforms the state-of-the-art on a TVQA dataset. |
TVQA: Localized, Compositional Video Question Answering (D18-1)
Copied to clipboard
| Challenge: | Recent studies have focused on image-based question-answering (QA) tasks, but little has been done on video-based QA. |
| Approach: | They present a large-scale video QA dataset based on 6 popular TV shows . they provide analysis of the new dataset and trainable neural network framework . |
| Outcome: | The proposed dataset includes 152,545 QA pairs from 21,793 clips spanning over 460 hours of video. |
Video Question Answering: Datasets, Algorithms and Challenges (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in video question answering have led to a surge in popularity . despite the popularity, VideoQA remains one of the greatest challenges . |
| Approach: | They categorize the video question-answer datasets into normal VideoQA, multi-modal VideoQA and knowledge-based VideoQA according to the modalities invoked in the question-announcement pairs. |
| Outcome: | The proposed methods are mainly designed for Factoid QA and inference VideoQA . the proposed methods have been compared with other methods and are robust and interpretable. |
Target-Aware Spatio-Temporal Reasoning via Answering Questions in Dynamic Audio-Visual Scenarios (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Audio-visual question answering requires multistep spatio-temporal reasoning over multimodal contexts. |
| Approach: | They propose a new target-aware joint spatio-temporal grounding network for audio-visual question answering . the proposed system integrates audio-vision fusion and question-awful temporal grounding into one module . |
| Outcome: | The proposed method over existing state-of-the-art methods is effective over existing methods . it can focus on audio-visual cues relevant to the query subject by utilizing explicit semantics from the question . |
Weakly-Supervised Spatio-Temporally Grounding Natural Sentence in Video (P19-1)
Copied to clipboard
| Challenge: | Existing techniques for weakly-supervised spatio-temporally grounding natural sentence in video are lacking . |
| Approach: | They propose a weakly-supervised task for spatially grounding sentences in video . they extract instances from video and encode them using attentive interactor . results demonstrate superiority of their proposed task over baseline approaches . |
| Outcome: | The proposed model outperforms baseline approaches in a weakly-supervised task . it can characterize reliable instance-sentence pairs and penalize unreliable ones . |
BiST: Bi-directional Spatio-Temporal Reasoning for Video-Grounded Dialogues (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to video-grounded dialogues focus on superficial temporal-level visual cues, but neglect more fine-grained spatial signals from videos. |
| Approach: | They propose a vision-language neural framework for high-resolution queries in videos based on textual cues that exploits both spatial and temporal-level information. |
| Outcome: | The proposed approach outperforms previous approaches on the TGIF-QA benchmark and significantly outperformed previous approaches. |
Time-MQA: Time Series Multi-Task Question Answering with Context Enhancement (2025.acl-long)
Copied to clipboard
Yaxuan Kong, Yiyuan Yang, Yoontae Hwang, Wenjie Du, Stefan Zohren, Zhangyang Wang, Ming Jin, Qingsong Wen
| Challenge: | Existing time series models focus on a narrow spectrum of tasks, such as forecasting or anomaly detection. |
| Approach: | They propose a framework that enables natural language queries across multiple time series tasks such as numerical analytical tasks and open-ended question answering with reasoning. |
| Outcome: | The proposed framework enables natural language queries across multiple time series tasks and allows for more advanced and intuitive interactions with temporal data. |
ComplexTempQA: A 100m Dataset for Complex Temporal Question Answering (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing datasets that focus on temporal knowledge are limited in size and lack comprehensive coverage of temporal information. |
| Approach: | They introduce a large-scale temporal question-answer-matching dataset . the new taxonomy categorizes questions as attributes, comparisons, and counting questions . |
| Outcome: | The proposed dataset surpasses existing benchmarks in scale and scope. |
MESAQA: A Dataset for Multi-Span Contextual and Evidence-Grounded Question Answering (2025.coling-main)
Copied to clipboard
| Challenge: | Existing question answering systems focus on extracting answers from single spans, but real-world scenarios require synthesizing information from multiple spans. |
| Approach: | They propose a dataset that leverages the MASH-QA dataset and large language models (LLMs) to ensure that each Q/A pair requires considering all selected spans. |
| Outcome: | The proposed method enables the model to answer multiple Q/A pairs in a single span, while ensuring that all selected spans are considered. |