Papers by Adriana Kovashka
Quantifying the Gaps Between Translation and Native Perception in Training for Multimodal, Multilingual Retrieval (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing models that account for perceptual differences in image captions are limited to use in English . culture-based tasks such as recognition, detection, and image retrieval are hindered by relying on English supervision. |
| Approach: | They propose and evaluate caption augmentation strategies to address these gaps . they use captions from german perception and captions that have been machine-translated or human-transcribed from English into german . |
| Outcome: | The proposed models achieve a mean recall improvement of +1.3, but still lack flexibility . cultural differences present in language with respect to object specificity and importance . |
Probing Logical Reasoning of MLLMs in Scientific Diagrams (2025.emnlp-main)
Copied to clipboard
| Challenge: | logical reasoning is key to real-world applications like science education, environmental monitoring, and medical diagnostics. |
| Approach: | They construct visual questions that follow seven structured templates with progressively more complex reasoning involved. |
| Outcome: | The proposed models perform logical inferences based on visual information. |
Leveraging Large Models to Evaluate Novel Content: A Case Study on Advertisement Creativity (2025.emnlp-main)
Copied to clipboard
| Challenge: | Evaluating creativity is challenging, even for humans, because of its subjectivity and complex cognitive processes. |
| Approach: | They propose a set of tasks to break down visual advertisement creativity into atypicality and originality with fine-grained annotations by humans. |
| Outcome: | The proposed tasks demonstrate the promise and challenges of using VLMs for automated creativity assessment. |
Synonym relations affect object detection learned on vision-language data (2024.findings-naacl)
Copied to clipboard
| Challenge: | a recent study shows that vision-language models that accept textual input are not robust to variations in how input is provided. |
| Approach: | They propose two approaches to improve vision-language object detectors' performance . they use back-translation and class embedding enrichment to improve their models . |
| Outcome: | The proposed approaches improve performance on synonyms from mAP@0.3=33.87% to 37.93%. |
The Face of Persuasion: Analyzing Bias and Generating Culture-Aware Ads (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Text-to-image models are appealing for customizing visual ads and targeting specific populations. |
| Approach: | We examine the disparate level of persuasiveness of ads that are identical except for gender/race of the people portrayed. |
| Outcome: | The proposed technique is based on a demographic bias analysis of ads for different topics and a disparate level of persuasiveness of ads that are identical except for gender/race of the people portrayed. |
VEIL: Vetting Extracted Image Labels from In-the-Wild Captions for Weakly-Supervised Object Detection (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing methods to “vet” labels from noisy captions for weakly-supervised object detection are limited for object detection. |
| Approach: | They propose a technique to “vet” labels extracted from noisy captions and use them for weakly-supervised object detection without any bounding boxes. |
| Outcome: | The proposed method improves WSOD without label vetting by 30% on five datasets. |
Decoding Symbolism in Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Existing language models can be used to decode symbolism, but they are biased in pre-trained corpora. |
| Approach: | They propose to use language models to decode symbols by re-ranking pre-trained models. |
| Outcome: | The proposed framework shows that pre-trained models can mitigate the bias and improve performance to be on par with human models. |