Multimodal Neural Graph Memory Networks for Visual Question Answering (2020.acl-main)
Copied to clipboard
| Challenge: | Visual question answering (VQA) is a new challenge for AI. |
| Approach: | They propose a graph neural network architecture based on the recently proposed Graph Network (GN) . they generate visual features and encoded captions for an image to generate two GNs . |
| Outcome: | The proposed model rivals the state-of-the-art models on Visual7W, VQA-v2.0, and CLEVR datasets. |
Similar Papers
Think Visually: Question Answering through Virtual Imagery (P18-1)
Copied to clipboard
| Challenge: | Existing models of geometric reasoning are based on visual representations of objects and objects, but they are not based in symbols or words. |
| Approach: | They propose a new deep network architecture that specializes in answering questions that admit latent visual representations and learns to generate and reason over such representations. |
| Outcome: | The proposed model can generate and reason over latent visual representations and is validated by two synthetic benchmarks. |
Multimodal Graph Transformer for Multimodal Question Answering (2023.eacl-main)
Copied to clipboard
| Challenge: | a myriad of complex tasks require both prior knowledge and reasoning intelligence. |
| Approach: | They propose a plug-and-play quasi-attention mechanism to integrate multimodal graph information to vanilla self-attention as effective prior. |
| Outcome: | The proposed model is able to perform reasoning across multiple modalities. |
Aligned Dual Channel Graph Convolutional Network for Visual Question Answering (2020.acl-main)
Copied to clipboard
| Challenge: | Existing graph-based methods focus only on relations between objects in an image and neglect the importance of syntactic dependency relations between words. |
| Approach: | They propose a dual channel graph convolutional network to capture relations between objects in an image and syntactic dependency relations between words in a question. |
| Outcome: | The proposed model achieves comparable performance with the state-of-the-art approaches. |
Question Answering by Reasoning Across Documents with Graph Convolutional Networks (N19-1)
Copied to clipboard
| Challenge: | Recent research in reading comprehension has focused on answering questions based on individual documents or even single paragraphs. |
| Approach: | They propose a neural model which integrates and reasons relying on information spread within documents and across multiple documents. |
| Outcome: | The proposed model achieves state-of-the-art on a multi-document question answering dataset, WikiHop. |
Enhancing Key-Value Memory Neural Networks for Knowledge Based Question Answering (N19-1)
Copied to clipboard
| Challenge: | Existing Key-value Memory Neural Networks are effective for shallow reasoning over documents . but extending them to Knowledge Based Question Answering is not trivial . |
| Approach: | They propose a mechanism to enable conventional KV-MemNNs models to perform interpretable reasoning for complex questions. |
| Outcome: | The proposed solution provides better reasoning abilities on complex questions and achieves state-of-the-art performance. |
Modeling Semantics with Gated Graph Neural Networks for Knowledge Base Question Answering (C18-1)
Copied to clipboard
| Challenge: | Existing approaches to Knowledge Base Question Answering focus on semantic parsing . previous work focused on selecting the correct semantic relations and not on the structure of the semantic parses . |
| Approach: | They propose to use Gated Graph Neural Networks to encode the graph structure of the semantic parse. |
| Outcome: | The proposed approach outperforms baseline models that do not explicitly model the structure. |
Dynamic Relevance Graph Network for Knowledge-Aware Question Answering (2022.coling-1)
Copied to clipboard
| Challenge: | Existing approaches to solve commonsense question answering problems often miss some edges between entities, which breaks the reasoning chain. |
| Approach: | They propose a graph neural network architecture that uses relevance as graph edges to establish new edges dynamically for learning node representations in the graph network. |
| Outcome: | The proposed approach shows competitive performance on two QA benchmarks, CommonsenseQA and OpenbookQA, compared to the state-of-the-art published results. |
Towards Knowledge-Augmented Visual Question Answering (2020.coling-main)
Copied to clipboard
| Challenge: | Visual Question Answering (VQA) is a challenging task for humans, but it is effortless for visual-based approaches. |
| Approach: | They propose a visual-based approach that captures interactions between visual scenes and external knowledge sources and exploits ConceptNet as the source of general knowledge. |
| Outcome: | The proposed model learns a question-adaptive graph representation of related knowledge instances. |
Propagate-Selector: Detecting Supporting Sentences for Question Answering via Graph Neural Networks (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing question-answering models do not require reasoning across sentences in the given context (passage). |
| Approach: | They propose a graph neural network that propagates information over sentences to understand information that cannot be inferred when considering sentences in isolation. |
| Outcome: | The proposed approach obtains the best performance compared to the widely used answer-selection models that do not consider the intersentential relationship. |
VGNMN: Video-grounded Neural Module Networks for Video-Grounded Dialogue Systems (2022.naacl-main)
Copied to clipboard
| Challenge: | Neural module networks (NMN) have been used in image-grounded tasks such as Visual Question Answering (VQA) however, very limited work on NMN has been studied in the video-ground dialogue tasks. |
| Approach: | They propose to use video as the grounding feature in video-grounded dialogues to model the information retrieval process in videogrounded language tasks as a pipeline of neural modules. |
| Outcome: | The proposed model can achieve promising performance on video-grounded dialogue and QA benchmarks. |