Change Entity-guided Heterogeneous Representation Disentangling for Change Captioning (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to describe differences between two images are highly challenging due to distractors such as illumination and viewpoint changes. |
| Approach: | They propose a change-entity-guided disentanglement network that explicitly learns difference representations while mitigating the impact of distractors. |
| Outcome: | The proposed method outperforms existing methods on CLEVR-Change, CLE VR-DC and Spot-the-Diff datasets and achieves state-of-the art performance. |
Similar Papers
Semantic Relation-aware Difference Representation Learning for Change Captioning (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to describe semantic change in images with distractors are difficult to learn . |
| Approach: | They propose a semantic relation-aware difference representation learning network to explicitly learn the difference representation in the existence of distractors. |
| Outcome: | The proposed network achieves state-of-the-art performance on CLEVR-Change and Spot-the -Diff datasets. |
Rˆ3Net:Relation-embedded Representation Reconstruction Network for Change Captioning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on change captioning uses a natural language sentence to describe disagreement between two images. |
| Approach: | They propose a Relation-embedded Representation Reconstruction Network to distinguish real change from clutter and irrelevant changes. |
| Outcome: | The proposed method achieves state-of-the-art on two public datasets. |
CLIP4IDC: CLIP for Image Difference Captioning (2022.aacl-short)
Copied to clipboard
| Challenge: | Conventional approaches learn an IDC model with a pre-trained and usually frozen visual feature extractor. |
| Approach: | They propose to transfer a CLIP model to the downstream IDC task to address two major issues: (1) a large domain gap exists between the pre-training datasets used for training such a visual feature extractor; (2) the visual feature extraction often does not effectively encode the visual changes between two images. |
| Outcome: | Experiments on three IDC benchmark datasets show the proposed model performs well. |
Transfer Learning for Entity Recognition of Novel Classes (C18-1)
Copied to clipboard
| Challenge: | Existing approaches to entity recognition are based on class labels in source and target domains, and many NER corpora only annotate a small number of categories. |
| Approach: | They replicate and extend several past studies on transfer learning for entity recognition. |
| Outcome: | The proposed methods perform better when there is more labeled target data. |
Learning to Describe Implicit Changes: Noise-robust Pre-training for Image Difference Captioning (2025.findings-emnlp)
Copied to clipboard
Zixin Guo, Jiayang Sun, Tzu-Jui Julius Wang, Abduljalil Radman, Selen Pehlivan, Min Cao, Jorma Laaksonen
| Challenge: | Large Multimodal Models (LMMs) are used to capture subtle differences between images but are noisy and coarse summaries. |
| Approach: | They propose a noise-robust approach to image difference capture using large multimodal models . they use LMMs with structured prompts to generate fine-grained change descriptions . |
| Outcome: | The proposed model outperforms streamlined architectures and improves inference efficiency. |
Entity Disambiguation via Fusion Entity Decoding (2024.naacl-long)
Copied to clipboard
Junxiong Wang, Ali Mousavi, Omar Attia, Ronak Pradeep, Saloni Potdar, Alexander Rush, Umar Farooq Minhas, Yunyao Li
| Challenge: | Existing generative approaches demonstrate improved accuracy compared to classification approaches under the standardized ZELDA benchmark. |
| Approach: | They propose an encoder-decoder model to disambiguate entities with more detailed entity descriptions. |
| Outcome: | The proposed model outperforms existing classification models on the ZELDA benchmark and on retrieval/reader frameworks. |
Contrastive Representation Learning for Cross-Document Coreference Resolution of Events and Entities (2022.naacl-main)
Copied to clipboard
| Challenge: | Identifying related entities and events within and across documents is fundamental to natural language understanding. |
| Approach: | They propose an approach to entity and event coreference resolution using contrastive representation learning. |
| Outcome: | The proposed method achieves state-of-the-art results on key metrics on the ECB+ corpus and is competitive on others. |
Rethinking Multimodal Entity and Relation Extraction from a Translation Point of View (2023.acl-long)
Copied to clipboard
| Challenge: | Special attention is paid to the cross-modal misalignment in text-image datasets which may mislead the learning. |
| Approach: | They propose a multimodal back-translation method which uses diffusion-based generative models for pseudo-paralleled pairs and a divergence estimator to construct a high-resource corpora as a bridge for low-ressource learners. |
| Outcome: | The proposed method outperforms 14 state-of-the-art methods in both entity and relation extraction tasks. |
Polarized-VAE: Proximity Based Disentangled Representation Learning for Text Generation (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for learning disentangled representations of real-world data focus on attribute labels or unsupervised methods that manipulate factorization in the latent space of models such as the variational autoencoder (VAE). |
| Approach: | They propose an approach that disentangles select attributes in the latent space based on proximity measures reflecting the similarity between data points with respect to these attributes. |
| Outcome: | The proposed method outperforms the VAE baseline and is competitive with state-of-the-art approaches while being more a general framework applicable to other attribute disentanglement tasks. |
Cross-Domain NER using Cross-Domain Language Modeling (P19-1)
Copied to clipboard
| Challenge: | Existing methods for named entity recognition (NER) use labeled data for both source and target domains. |
| Approach: | They propose to use language modeling as a bridge between NER domains to perform cross-domain and cross-task knowledge transfer. |
| Outcome: | The proposed method extracts domain differences from cross-domain LM contrast, allowing unsupervised domain adaptation while giving state-of-the-art results among supervised domain adapters. |