Papers with MEL
MELOV: Multimodal Entity Linking with Optimized Visual Features in Latent Space (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to multimodal entity linking focus on textual contexts but lack in social media vision modality. |
| Approach: | They propose a latent space vision feature optimization framework MELOV to address these challenges . they exploit variational autoencoder to mine shared information and generate text-based visual features . |
| Outcome: | The proposed framework is superior to existing methods on three benchmark datasets. |
Optimal Transport Guided Correlation Assignment for Multimodal Entity Linking (2024.findings-acl)
Copied to clipboard
Zefeng Zhang, Jiawei Sheng, Zhang Chuang, Liangyunzhi Liangyunzhi, Wenyuan Zhang, Siqi Wang, Tingwen Liu
| Challenge: | Existing methods to link ambiguous mentions to entities in multimodal knowledge graphs rely on partial correlations. |
| Approach: | They propose a framework that leverages multi-element correlations to bridge modality gap and enable fine-grained semantic matching by exploiting correlation between multimodal features and entities. |
| Outcome: | The proposed framework outperforms state-of-the-art models and confirms the effectiveness of the proposed method. |
WikiDiverse: A Multimodal Entity Linking Dataset with Diversified Contextual Topics and Entity Types (2022.acl-long)
Copied to clipboard
| Challenge: | Multimodal Entity Linking (MEL) is an essential task for many multimodal applications. |
| Approach: | They propose to use a human-annotated Wikipedia-based multimodal entity linking dataset to improve the quality of existing MEL models. |
| Outcome: | The proposed model uses the visual information of images more effectively than existing models. |
Self-Alignment Pretraining for Biomedical Entity Representations (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to self-supervised learning of biomedical entities are limited in the biomedic domain. |
| Approach: | They propose a pretraining scheme that self-aligns the representation space of biomedical entities. |
| Outcome: | The proposed framework achieves state-of-the-art on six MEL benchmarking datasets. |
Enhancing Multimodal Entity Linking with Jaccard Distance-based Conditional Contrastive Learning and Contextual Visual Augmentation (2025.naacl-long)
Copied to clipboard
Cong-Duy T Nguyen, Xiaobao Wu, Thong Thanh Nguyen, Shuai Zhao, Khoi M. Le, Nguyen Viet Anh, Feng Yichao, Anh Tuan Luu
| Challenge: | Existing approaches to multimodal entity linking use contrastive learning to align input sentences and entities, but are limited by their random negative sampling. |
| Approach: | They propose a method to match negative samples with similar attributes using JD-CCL . they also propose 'contextual visual-aid controllable patch transform' experimental results demonstrate the strong effectiveness of their method . |
| Outcome: | The proposed method is able to match negative samples with similar attributes on a multimodal knowledge graph. |
Trust in Internal or External Knowledge? Generative Multi-Modal Entity Linking with Knowledge Retriever (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing generative approaches struggle with the knowledge gap between visual entity information and the intrinsic parametric knowledge of LLMs. |
| Approach: | They propose a knowledge retrieval method that leverages external sources to enhance visual entity information and a prioritization scheme that handles noisy retrieval results. |
| Outcome: | The proposed method shows improvements of 3.0% to 6.5% across all evaluation metrics compared to baselines. |
OVEL: Online Video Entity Linking (2025.coling-main)
Copied to clipboard
| Challenge: | Existing studies on Multi-modal Entity Linking focus on linking textual and visual mentions or offline videos’ mentions to entities in multi-modal knowledge bases. |
| Approach: | They propose a task called Online Video Entity Linking to establish connections between online videos and a knowledge base with high accuracy and timeliness. |
| Outcome: | The proposed method can establish connections between mentions in online videos and a knowledge base with high accuracy and timeliness. |
Generative Multimodal Entity Linking (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing Entity Linking methods focus on designing complex multimodal interaction mechanisms and require fine-tuning all model parameters. |
| Approach: | They propose a framework for multimodal entity linking based on Large Language Models (LLMs) that trains a feature mapper to enable cross-modal interactions. |
| Outcome: | The proposed framework achieves state-of-the-art on two well-established datasets with a performance gain of 7.7% on WikiDiverse and 8.8% on Wikileaks. |
VP-MEL: Visual Prompts Guided Multimodal Entity Linking (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for multimodal entity linking rely on mention words as retrieval cues, which limits their ability to effectively utilize information from both images and text. |
| Approach: | They propose a visual prompt-guided multimodal entity linking task for a text-image pair . they propose VPWiki to facilitate this task and a framework to capture latent information. |
| Outcome: | The proposed framework outperforms baseline methods on a VPWiki dataset. |
mReFinED: An Efficient End-to-End Multilingual Entity Linking System (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work assumed that entity mentions were given and skipped the entity mention detection step due to a lack of high-quality multilingual training corpora. |
| Approach: | They propose a bootstrapping mention detection framework that enhances the quality of training corpora. |
| Outcome: | The proposed framework outperforms existing work in the end-to-end MEL task while being 44 times faster. |
ThinkLinker: From Low-Rank Interaction to Knowledge-Aware Verification for Multimodal Entity Linking (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for multimodal entity linking rely on textual context for disambiguation . textual contextual information alone fails to resolve ambiguity, leading to unreliable disambiguations in weak contexts. |
| Approach: | They propose a two-stage multimodal entity linking framework called ThinkLinker . they propose fusion mechanism to model joint dependencies among features . |
| Outcome: | The proposed framework outperforms state-of-the-art models on public benchmark datasets. |