Papers by Julia Kruk
Integrating Text and Image: Determining Multimodal Document Intent in Instagram Posts (D19-1)
Copied to clipboard
| Challenge: | Existing studies on text-image content have focused on image as primary content, and text as secondary content. |
| Approach: | They propose a multimodal dataset of 1299 Instagram posts labeled for three orthogonal taxonomies . they show that employing both text and image improves intent detection by 9.6 . |
| Outcome: | The proposed model shows that using both text and image improves intent detection by 9.6 compared to using only the image modality. |
Silent Signals, Loud Impact: LLMs for Word-Sense Disambiguation of Coded Dog Whistles (2024.acl-long)
Copied to clipboard
| Challenge: | a dog whistle is a coded communication that carries a secondary meaning to specific audiences and is often weaponized for racial and socioeconomic discrimination. |
| Approach: | They propose an approach for word-sense disambiguation of dog whistles from standard speech using Large Language Models. |
| Outcome: | The proposed method allows disambiguation of dog whistles from standard speech using large language models. |
Impressions: Visual Semiotics and Aesthetic Impact Understanding (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing image captioning and conditional generation models struggle to simulate plausible human responses to images. |
| Approach: | They propose a dataset to investigate the semiotics of images and how visual features and design choices can elicit specific emotions, thoughts and beliefs. |
| Outcome: | The proposed dataset improves existing models for image captioning and conditional generation. |