Papers by Hadar Averbuch-Elor
ICC : Quantifying Image Caption Concreteness for Multimodal Dataset Curation (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to curation text-image data are noisy and lack the fine-grained ability to isolate the most concrete samples that provide the strongest signal for learning in a noisy dataset. |
| Approach: | They propose a metric that evaluates caption text without an image reference to measure its concreteness and relevancy. |
| Outcome: | The proposed method detects the concreteness of captions without an image reference and correlates with human evaluation of concreteness in both single-word and caption-level texts. |
What is Learned in Visually Grounded Neural Syntax Acquisition (2020.acl-main)
Copied to clipboard
| Challenge: | Visual features are promising for learning bootstrap textual models, but blackbox learning models make it difficult to isolate the specific contribution of visual components. |
| Approach: | They propose to use alignments between phrases and images as a learning signal for syntax acquisition. |
| Outcome: | The proposed model performs better than the previous model, but it is significantly less expressive. |
Mitigating Open-Vocabulary Caption Hallucinations (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for image captioning ignore the long-tailed nature of hallucinations . a new framework is proposed to address hallucines in image captions in the open-vocabulary setting . |
| Approach: | They propose a framework to address hallucinations in image captioning in the open-vocabulary setting. |
| Outcome: | The proposed framework surpasses the CHAIR benchmark in diversity and accuracy in open-vocabulary captioning. |