VideoQA-TA: Temporal-Aware Multi-Modal Video Question Answering (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for video question answering align visual or textual features directly with large language models, limiting the deep semantic association between modalities and hindering a comprehensive understanding of interactions within spatial and temporal contexts. |
| Approach: | They propose a temporal-aware framework for multi-modal video question answering that aligns videos and questions at fine-grained levels. |
| Outcome: | The proposed framework improves reasoning ability and accuracy of videoQA by aligning videos and questions at fine-grained levels. |
Similar Papers
TVQA+: Spatio-Temporal Grounding for Video Question Answering (2020.acl-main)
Copied to clipboard
| Challenge: | Existing video QA datasets only contain QA pairs without labels for key clips or regions needed to answer the question. |
| Approach: | They propose a framework that grounds evidence in both spatial and temporal domains to answer questions about videos using bounding boxes. |
| Outcome: | The proposed framework can produce interpretable spatio-temporal attention visualizations. |
Dense-Caption Matching and Frame-Selection Gating for Temporal Localization in VideoQA (2020.acl-main)
Copied to clipboard
| Challenge: | Recent years have witnessed a paradigm shift in the way we get our information, and a lot of it. |
| Approach: | They propose a video question answering model which integrates multi-modal input sources and finds temporally relevant information to answer questions. |
| Outcome: | The proposed model outperforms the state-of-the-art on a TVQA dataset. |
Rethinking Multi-Modal Alignment in Multi-Choice VideoQA from Feature and Sample Perspectives (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to VideoQA focus on utilizing frame- or object-level visual representations, but they neglect visual-language interactions. |
| Approach: | They propose to break down video into trajectories and first leverage trajectory feature in VideoQA to enhance alignment between two modalities. |
| Outcome: | The proposed method outperforms all the state-of-the-art models on the NExT-QA benchmark. |
Video Question Answering: Datasets, Algorithms and Challenges (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in video question answering have led to a surge in popularity . despite the popularity, VideoQA remains one of the greatest challenges . |
| Approach: | They categorize the video question-answer datasets into normal VideoQA, multi-modal VideoQA and knowledge-based VideoQA according to the modalities invoked in the question-announcement pairs. |
| Outcome: | The proposed methods are mainly designed for Factoid QA and inference VideoQA . the proposed methods have been compared with other methods and are robust and interpretable. |
Mulan: A Multi-Level Alignment Model for Video Question Answering (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods focus on visual-language alignment at the video level, but they do not account for fine-grained semantic interaction between video and text. |
| Approach: | They propose a multi-level Alignment Model for Video Question Answering that establishes alignment between visual and textual modalities at the object-level, frame-level and video-level. |
| Outcome: | The proposed model outperforms state-of-the-art methods even with a small amount of extra visual-language pre-training data and a reduced number of trainable parameters. |
TVQA: Localized, Compositional Video Question Answering (D18-1)
Copied to clipboard
| Challenge: | Recent studies have focused on image-based question-answering (QA) tasks, but little has been done on video-based QA. |
| Approach: | They present a large-scale video QA dataset based on 6 popular TV shows . they provide analysis of the new dataset and trainable neural network framework . |
| Outcome: | The proposed dataset includes 152,545 QA pairs from 21,793 clips spanning over 460 hours of video. |
ReasVQA: Advancing VideoQA with Imperfect Reasoning Process (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing approaches to VideoQA often fail when complex reasoning or temporal relationships are involved. |
| Approach: | They propose a method that leverages reasoning processes generated by Multimodal Large Language Models to improve VideoQA models. |
| Outcome: | The proposed method improves VideoQA models on three benchmarks. |
Enhancing Temporal Sensitivity and Reasoning for Time-Sensitive Question Answering (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing language models have limited sensitivity to temporal information and inadequate temporal reasoning capabilities. |
| Approach: | They propose a framework that enhances temporal awareness and reasoning . they propose to use Temporal Information-Aware Embedding and Granular Contrastive Reinforcement Learning . |
| Outcome: | The proposed framework outperforms existing LLMs on time-sensitive question answering tasks. |
Target-Aware Spatio-Temporal Reasoning via Answering Questions in Dynamic Audio-Visual Scenarios (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Audio-visual question answering requires multistep spatio-temporal reasoning over multimodal contexts. |
| Approach: | They propose a new target-aware joint spatio-temporal grounding network for audio-visual question answering . the proposed system integrates audio-vision fusion and question-awful temporal grounding into one module . |
| Outcome: | The proposed method over existing state-of-the-art methods is effective over existing methods . it can focus on audio-visual cues relevant to the query subject by utilizing explicit semantics from the question . |
Empowering Large Language Model for Continual Video Question Answering with Collaborative Prompting (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing VideoQA models struggle to adapt to new questions or tasks posed by newly available content. |
| Approach: | They propose a continual learning framework that fine-tunes a large language model for a sequence of tasks and integrates specific question constraint prompting, knowledge acquisition prompting and visual temporal awareness prompting. |
| Outcome: | The proposed model achieves 55.14% accuracy on both NExT-QA and DramaQA datasets and 71.24% accuracy for DramaQA. |