Information Screening whilst Exploiting! Multimodal Relation Extraction with Feature Denoising and Multimodal Topic Modeling (2023.acl-long)
Copied to clipboard
| Challenge: | Existing research on multimodal relation extraction (MRE) faces internal-information over-utilization and external-information under-exploitation. |
| Approach: | They propose a framework that implements internal-information screening and external-information exploiting to address these challenges. |
| Outcome: | The proposed framework outperforms the current best model on the benchmark dataset. |
Similar Papers
Multimodal Relation Extraction with Cross-Modal Retrieval and Synthesis (2023.acl-short)
Copied to clipboard
| Challenge: | Existing retrieval-augmented approaches focus on modeling the retrieved textual knowledge but this may not be able to accurately identify complex relations. |
| Approach: | They propose to retrieve multimodal relation extraction information based on object, sentence, and whole image . they propose to synthesize the object-level, image-level and sentence-level information . |
| Outcome: | The proposed method outperforms state-of-the-art models on multimodal relation extraction. |
MRE-MI: A Multi-image Dataset for Multimodal Relation Extraction in Social Media Posts (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to Multimodal Relation Extraction focus on single image scenarios . current approaches focus on text paired with a single image, ignoring valuable insights provided by remaining images. |
| Approach: | They propose a human-annotated dataset that includes multi-image and single-image instances for relation extraction. |
| Outcome: | The proposed model significantly improves relation extraction in multi-image scenarios. |
Retrieval over Classification: Integrating Relation Semantics for Multimodal Relation Extraction (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to multimodal relation extraction ignore structural constraints and lack semantic expressiveness for fine-grained relation understanding. |
| Approach: | They propose a framework that reformulates multimodal relation extraction as a retrieval task driven by relation semantics. |
| Outcome: | The proposed framework achieves state-of-the-art performance on the benchmark datasets MNRE and MORE and exhibits stronger robustness and interpretability. |
Enhancing multi-modal Relation Extraction with Reinforcement Learning Guided Graph Diffusion Framework (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for cross-modal relation extraction focus on single-modal data, which limits their use in real-world situations. |
| Approach: | They propose a framework that leverages pre-trained models to encode multi-modal data into scene graphs and combine them into a cross-modal graph. |
| Outcome: | The proposed model outperforms existing methods on multi-modal relation extraction tasks. |
Multimodal Document-level Triple Extraction via Dynamic Graph Enhancement and Relation-Aware Reflection (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for extracting structured triples knowledge from multimodal documents face limitations in simultaneously processing long textual content and multiple associated images for triple extraction. |
| Approach: | They propose a multimodal document-level triple extraction framework that integrates multimodal text and visual content into a large language model and injects the global information and external knowledge into the model. |
| Outcome: | The proposed framework outperforms the state-of-the-art methods and fills the gap in multimodal document extraction. |
Multimodal Graph-based Transformer Framework for Biomedical Relation Extraction (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing models based on textual data do not capture context beyond the sentence. |
| Approach: | They propose a framework that enables the model to learn multi-omnics biological information about entities (proteins) with the help of additional multi-modal cues like molecular structure. |
| Outcome: | The proposed model is generalized and optimized for protein-protein interaction task and benefited from additional domain-specific cues. |
Rethinking Multimodal Entity and Relation Extraction from a Translation Point of View (2023.acl-long)
Copied to clipboard
| Challenge: | Special attention is paid to the cross-modal misalignment in text-image datasets which may mislead the learning. |
| Approach: | They propose a multimodal back-translation method which uses diffusion-based generative models for pseudo-paralleled pairs and a divergence estimator to construct a high-resource corpora as a bridge for low-ressource learners. |
| Outcome: | The proposed method outperforms 14 state-of-the-art methods in both entity and relation extraction tasks. |
TopicDiff: A Topic-enriched Diffusion Approach for Multimodal Conversational Emotion Detection (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies focus on learning contextual information in conversations, neglecting acoustic and vision topic information. |
| Approach: | They propose a model-agnostic Topic-enriched Diffusion approach for capturing multimodal topic information in MCE tasks. |
| Outcome: | The proposed approach improves over the state-of-the-art MCE models and the existing models. |
Exploring Multimodal Relation Extraction of Hierarchical Tabular Data with Multi-task Learning (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies overlook the need of mining relations among multiple columns rather than just the semantic relation between two specific columns in real-world practice. |
| Approach: | They propose a Chain-of-Thought distillation framework with self-correction mechanism to enhance MLLMs’ reasoning capabilities without increasing parameter scale. |
| Outcome: | The proposed method significantly outperforms baselines on wide datasets. |
Improving Distantly-Supervised Relation Extraction with Joint Label Embedding (D19-1)
Copied to clipboard
| Challenge: | Existing methods for relation extraction treat labels as independent and meaningless one-hot vectors, which cause a loss of potential label information for selecting valid instances. |
| Approach: | They propose a multi-layer attention-based model to improve relation extraction with joint label embedding by gating integration and using the embeddable entities as an atten- tion. |
| Outcome: | The proposed model significantly outperforms state-of-the-art methods in relation extraction with joint label embedding. |