Papers with MMKGs
Noise-powered Multi-modal Knowledge Graph Representation Framework (2025.coling-main)
Copied to clipboard
| Challenge: | Current efforts to integrate MMKG with pretraining are scarce. |
| Approach: | They propose a method that integrates multi-modal entity features into MMKGs using a Transformer-based architecture equipped with modality-level noise masking. |
| Outcome: | The proposed method achieves SOTA performance across ten datasets. |
Multi-Modal Knowledge Graph Transformer Framework for Multi-Modal Entity Alignment (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Multi-modal entity alignment (MMEA) is a critical task that aims to identify equivalent entity pairs across multi-modal knowledge graphs (MMKGs). |
| Approach: | They propose a novel MMEA transformer that hierarchically introduces neighbor features, multi-modal attributes, and entity types to enhance alignment task. |
| Outcome: | The proposed transformer hierarchically introduces neighbor features, multi-modal attributes, and entity types to enhance the alignment task. |
SGMEA: Structure-Guided Multimodal Entity Alignment (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods focus on interactions between neighboring entities in the structural modality while neglecting interactions between entities in visual and attribute modalities. |
| Approach: | They propose a structure-guided multimodal entity alignment method which prioritizes structural information from knowledge graphs to enhance the visual and attribute modalities. |
| Outcome: | The proposed method achieves state-of-the-art performance across multiple datasets, validating its effectiveness and superiority in practical applications. |
Multimodal Reasoning with Multimodal Knowledge Graph (2024.acl-long)
Copied to clipboard
| Challenge: | Multimodal reasoning with large language models (LLMs) often suffers from hallucinations and the presence of deficient or outdated knowledge within LLMs. |
| Approach: | They propose a multimodal reasoning method that leverages multimodal knowledge graphs to learn rich and semantic knowledge across modalities. |
| Outcome: | The proposed method outperforms state-of-the-art models on multimodal question answering and multimodal analogy reasoning tasks while training on only a small fraction of parameters. |
Multi-Modal Entities Matter: Benchmarking Multi-Modal Entity Alignment (2025.coling-main)
Copied to clipboard
| Challenge: | Existing MMEA datasets consider multi-modal data as attributes of textual entities, neglecting correlations between the multi-modal data. |
| Approach: | They propose a multi-modal entity alignment dataset that models multi-dimensional data as textual entities in the MMKG. |
| Outcome: | The proposed dataset can learn the structural information of entities by considering both intra-modal and cross-modal relations and infer the similarity of different types of entity pairs. |
Light Up the Shadows: Enhance Long-Tailed Entity Grounding with Concept-Guided Vision-Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Multi-Modal Knowledge Graphs (MMKGs) are knowledge graphs that integrate and align information from diverse modalities (e.g., text and images). |
| Approach: | They propose a framework that integrates image-text pairs of long-tailed entities and a concept guidance module that offers explainability and enables human verification. |
| Outcome: | The proposed framework improves the accuracy of recognizing long-tailed image-text pairs compared to baselines and also offers flexibility and explainability. |
Differentiated Vision: Unveiling Entity-Specific Visual Modality Requirements for Multimodal Knowledge Graph (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to extract features from images of entities overlook varying relevance of visual information across entities. |
| Approach: | a new model integrates structural and multimodal information of entities into a multimodal knowledge graph . a model evaluates the necessity of visual modality for each entity based on its attributes . |
| Outcome: | The proposed model improves on existing methods by adjusting visual data to different entity types. |
Collaboration of Fusion and Independence: Hypercomplex-driven Robust Multi-Modal Knowledge Graph Completion (2026.acl-long)
Copied to clipboard
| Challenge: | Existing multi-modal knowledge graphs lack modality-specific information and are limited in their ability to capture nuanced semantic interplay between modalities. |
| Approach: | They propose a multi-modal knowledge graph completion method which integrates both paradigms . they use a fine-grained Entity Representation Factorization module and a Robust Relation-aware Modality Fusion module to obtain robust representations for three independent modalities and one fused modality. |
| Outcome: | The proposed method achieves coexistence and collaboration of fused and independent modality representations while maintaining modality-specific information. |
Breaking the Noise Barrier: LLM-Guided Semantic Filtering and Enhancement for Multi-Modal Entity Alignment (2025.emnlp-main)
Copied to clipboard
| Challenge: | Multi-modal entity alignment (MMEA) aims to identify equivalent entities between two multimodal knowledge graphs. |
| Approach: | They propose a novel LLMguided MMEA framework that prioritizes noise reduction before fusion. |
| Outcome: | The proposed framework prioritizes noise reduction before fusion and improves semantics on the noisy FB YG dataset. |