Papers by Elisa Kreiss
CommVQA: Situating Visual Question Answering in Communicative Contexts (2024.emnlp-main)
Copied to clipboard
| Challenge: | Current visual question answering models are trained on image-question pairs in isolation, but the questions people ask are dependent on their informational needs and prior knowledge about the image content. |
| Approach: | They propose a visual question-answer-as-question dataset that contains 1000 images and 8,949 question-announcer pairs to evaluate how situating images within naturalistic contexts shapes visual questions. |
| Outcome: | The proposed dataset contains 1000 images and 8,949 question-answer pairs. |
Updating CLIP to Prefer Descriptions Over Captions (2024.emnlp-main)
Copied to clipboard
| Challenge: | Current metrics for imagetext similarity tend to be insensitive to the text's purpose. |
| Approach: | They propose to use a model that assigns higher scores to descriptions than captions . they use parameter efficient fine-tuning and a loss objective to shed light on the distinction . |
| Outcome: | The proposed model correlates with the judgements of blind and low-vision people while preserving transfer capabilities and sheds light on the caption–description distinction. |
Measuring Bias or Measuring the Task: Understanding the Brittle Nature of LLM Gender Biases (2025.emnlp-main)
Copied to clipboard
| Challenge: | a growing number of efforts to measure and mitigate gender bias have focused on task prompts that overtly or covertly signal the presence of gender bias-related content. |
| Approach: | They examine how signaling the evaluative purpose of a task impacts measured gender bias in LLMs. |
| Outcome: | The proposed models show that prompts that align with (gender bias) evaluation framing elicit distinct gender output distributions compared to less evaluation-framed prompts. |
MOSAIC: Modeling Social AI for Content Dissemination and Regulation in Multi-Agent Simulations (2025.emnlp-main)
Copied to clipboard
| Challenge: | generative language agents predict user behaviors such as liking, sharing, and flagging content. |
| Approach: | They propose a framework where generative language agents predict user behaviors such as liking, sharing, and flagging content. |
| Outcome: | The proposed framework analyzes content moderation strategies and user engagement dynamics at scale and demonstrates that agents’ articulated reasoning for their social interactions aligns with their collective engagement patterns. |
When More Words Say Less: Decoupling Length and Specificity in Image Description Evaluation (2026.acl-short)
Copied to clipboard
| Challenge: | Vision-language models are increasingly used to produce textual descriptions of visual content. |
| Approach: | They propose to disentangle description specificity from description length . they find people prefer more specific descriptions regardless of length based on their own subjective preferences . |
| Outcome: | The proposed model shows that people prefer more specific descriptions regardless of length. |
Causal Distillation for Language Models (2022.naacl-main)
Copied to clipboard
Zhengxuan Wu, Atticus Geiger, Joshua Rozner, Elisa Kreiss, Hanson Lu, Thomas Icard, Christopher Potts, Noah Goodman
| Challenge: | Distillation efforts have led to language models that are more compact and efficient without serious drops in performance. |
| Approach: | They propose to augment distillation with a third objective that encourages the student model to imitate the causal dynamics of the teacher through a distillation interchange intervention training objective (DIITO). |
| Outcome: | The proposed method lowers perplexity on the WikiText-103M corpus and improves on the GLUE benchmark, SQuAD, and CoNLL-2003. |
Context Matters for Image Descriptions for Accessibility: Challenges for Referenceless Evaluation Metrics (2022.emnlp-main)
Copied to clipboard
Elisa Kreiss, Cynthia Bennett, Shayan Hooshmand, Eric Zelikman, Meredith Ringel Morris, Christopher Potts
| Challenge: | Existing referenceless metrics do not take context into account, whereas contextual information is highly valued by BLV users. |
| Approach: | They propose a contextual version of the referenceless metric CLIPScore which addresses the disconnect to the BLV data. |
| Outcome: | The proposed evaluation metrics are based on a proof-of-concept with blind and low vision (BLV) participants. |
Concadia: Towards Image-Based Text Generation with a Purpose (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing models fail to generate fluent, truthful text, despite excellent results on benchmark datasets . current systems fail to produce texts that are useful in practice, authors argue . |
| Approach: | They propose to distinguish descriptions from captions based on their communicative roles . descriptions focus on visual features and are meant to replace an image . authors characterize commonalities and differences between descriptions and captions in a Wikipedia corpus . |
| Outcome: | The proposed model can generate fluent, truthful texts in a wide range of scenarios . it can also generate captions that are used to make an image accessible to users who can't see them . |