Multimodal Coreference Resolution for Chinese Social Media Dialogues: Dataset and Benchmark Approach (2025.acl-long)
Copied to clipboard
| Challenge: | Multimodal coreference resolution (MCR) aims to identify mentions referring to the same entity across different modalities, such as text and visuals. |
| Approach: | They propose a Chinese multimodal coreference dataset based on Douyin short-video platform to help researchers understand multimodal content. |
| Outcome: | The proposed dataset pairs short videos with corresponding textual dialogues from user comments and includes manually annotated coreference clusters for person mentions in the text and the coreferential person head regions in the corresponding video frames. |
Similar Papers
Investigating Multilingual Coreference Resolution by Universal Annotations (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing systems for multilingual coreference resolution have been challenging due to linguistic diversity and complexity of different languages. |
| Approach: | They propose a multilingual coreference dataset with universal morphosyntactic and coreference annotations. |
| Outcome: | The proposed dataset improves the baseline system by 0.9% . the proposed dataset is based on the framework of Universal Dependencies 2 . |
GRAVL-BERT: Graphical Visual-Linguistic Representations for Multimodal Coreference Resolution (2022.coling-1)
Copied to clipboard
Danfeng Guo, Arpit Gupta, Sanchit Agarwal, Jiun-Yu Kao, Shuyang Gao, Arijit Biswas, Chien-Wei Lin, Tagyoung Chung, Mohit Bansal
| Challenge: | Multimodal coreference resolution (MCR) is a crucial capability for building next-generation conversational agents. |
| Approach: | They propose a multimodal coreference resolution model that resolves coreferences made in multi-turn dialogues with scene images. |
| Outcome: | The proposed model resolves coreferences made in multi-turn dialogues with scene images. |
Multilingual Coreference Resolution in Multiparty Dialogue (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing datasets for entity coreference resolution are limited to English and other languages are rare. |
| Approach: | They propose to use TV transcripts to create multilingual multiparty coreference datasets that leverage existing subtitles in Chinese and Farsi. |
| Outcome: | The proposed dataset re-annotates for coreference on TV transcripts and then leverages existing subtitle translations to create a multilingual corpus. |
MCECR: A Novel Dataset for Multilingual Cross-Document Event Coreference Resolution (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing datasets for event coreference resolution focus on within-document event coreference and English text, lacking cross-document ECR datasets beyond English. |
| Approach: | They propose a multiligual dataset that manually annotates documents for event mentions and coreference in five languages. |
| Outcome: | The proposed dataset annotates documents for event mentions and coreference in five languages . the dataset fetches related news articles from the google search engine to increase the number of non-singleton clusters . |
A Large-Scale Chinese Multimodal NER Dataset with Speech Clues (2021.acl-long)
Copied to clipboard
| Challenge: | Using a large-scale dataset, we explore Chinese named entity recognition (NER) with both textual and acoustic contents. |
| Approach: | They propose a Chinese multimodal named entity recognition dataset . their corpus contains 42,987 annotated sentences and 71 hours of speech data . |
| Outcome: | The proposed model yields state-of-the-art (SoTA) results on Chinese multimodal named entity recognition (NER) based on 42,987 annotated sentences and 71 hours of speech data. |
Joint Coreference Resolution and Character Linking for Multiparty Conversation (2021.eacl-main)
Copied to clipboard
| Challenge: | Character linking is the task of linking mentioned people in conversations to the real world . human use of pronouns or normal entities makes it difficult to link mentioned people to real people . a critical step towards understanding conversations is grounding mentioned people - a goal of the natural language processing community . |
| Approach: | They propose to integrate richer context from the coreference relations among different mentions to help the linking task. |
| Outcome: | The proposed model outperforms all previous models on both tasks. |
Multimodal Cross-Document Event Coreference Resolution Using Linear Semantic Transfer and Mixed-Modality Ensembles (2024.lrec-main)
Copied to clipboard
Abhijnan Nath, Huma Jamil, Shafiuddin Rehan Ahmed, George Arthur Baker, Rahul Ghosh, James H. Martin, Nathaniel Blanchard, Nikhil Krishnaswamy
| Challenge: | Existing methods for cross-document coreference resolution do not provide images for all mentions of events. |
| Approach: | They propose a multimodal cross-document event coreference resolution method that integrates visual and textual cues with a simple linear map between vision and language models. |
| Outcome: | The proposed method improves on a popular ECB+ and AIDA datasets. |
Disambiguating Reference in Visually Grounded Dialogues through Joint Modeling of Textual and Multimodal Semantic Structures (2025.acl-long)
Copied to clipboard
| Challenge: | incorporating textual and multimodal reference resolution improves performance in visual-based reference resolution . Phrase grounding is a well-established task for understanding semantic relations between mentions and objects . ambiguities caused by pronouns and ellipses can arise in visually grounded dialogues . |
| Approach: | They propose a framework that unifies textual and multimodal reference resolution by mapping mention embeddings to object embeddements and selecting mentions or objects based on their similarity. |
| Outcome: | The proposed framework performs better in phrase grounding than other models for this task. |
NovelCR: A Large-Scale Bilingual Dataset Tailored for Long-Span Coreference Resolution (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing coreference resolution datasets are either small in scale or restrict coreference to a limited text span. |
| Approach: | They present a large-scale bilingual benchmark for long-span coreference resolution . they find that NovelCR is notably rich in long-spanning coreference pairs . |
| Outcome: | The proposed benchmark is rich in long-span coreference pairs and notably low baselines. |
Semi-supervised multimodal coreference resolution in image narrations (2023.emnlp-main)
Copied to clipboard
| Challenge: | a semi-supervised approach is used to resolve multimodal coreferences and narrative grounding in a multimodal context. |
| Approach: | They propose a semi-supervised approach that utilizes image-narration pairs to resolve coreferences and narrative grounding in a multimodal context. |
| Outcome: | The proposed approach outperforms baselines quantitatively and qualitatively for coreference resolution and narrative grounding in a multimodal context. |