Papers by Alireza Zareian
Cross-media Structured Common Space for Multimedia Event Extraction (2020.acl-main)
Copied to clipboard
| Challenge: | We propose a new task to extract events and their arguments from multimedia documents . traditional methods target text, images or videos, but multimedia content is distributed via multimedia . |
| Approach: | They propose a method that encodes structured representations of semantic information from textual and visual data into a common embedding space. |
| Outcome: | The proposed method achieves 4.0% and 9.8% absolute gains on text event argument role labeling and visual event extraction. |
Red-Teaming NSFW Image Classifiers as Text-to-Image Safeguards (2026.findings-acl)
Copied to clipboard
Tinghao Xie, Yueqi Xie, Alireza Zareian, Shuming Hu, Felix Juefei-Xu, Xiaowen Lin, Ankit Jain, Prateek Mittal, Li Chen
| Challenge: | Not Safe for Work (NSFW) image classifiers play a critical role in safeguarding text-to-image systems. |
| Approach: | They propose an automated red-teaming framework that leverages a set of generative AI tools to uncover NSFW image failures. |
| Outcome: | The proposed framework uncovers and interprets failure modes and enables it to be applied to real-world T2I and T2V systems. |
Unsupervised Vision-and-Language Pre-training Without Parallel Images and Captions (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing models require large amounts of image-caption data for pre-training . existing models require expensive data collection and curation . |
| Approach: | They propose to conduct "mask-and-predict" pre-training on text-only and image-only corpora and introduce the object tags detected by an object recognition model as anchor points to bridge two modalities. |
| Outcome: | The proposed approach achieves performance close to a model pre-trained with aligned data, on four English benchmarks. |
GAIA: A Fine-grained Multimedia Knowledge Extraction System (2020.acl-demos)
Copied to clipboard
Manling Li, Alireza Zareian, Ying Lin, Xiaoman Pan, Spencer Whitehead, Brian Chen, Bo Wu, Heng Ji, Shih-Fu Chang, Clare Voss, Daniel Napierski, Marjorie Freedman
| Challenge: | Open source knowledge extraction tools are used for many real-world applications, but there is no comprehensive system for KE. |
| Approach: | They propose a multimedia knowledge extraction system that takes multimedia data from various sources and languages as input and creates a coherent, structured knowledge base. |
| Outcome: | The system achieves top performance at the recent NIST TAC SM-KBP2019 evaluation. |
Learning from Children: Improving Image-Caption Pretraining via Curriculum (2023.findings-acl)
Copied to clipboard
| Challenge: | Image-caption pretraining is a difficult problem as it requires multiple concepts (nouns) from captions to be aligned to multiple objects in images. |
| Approach: | They propose a curriculum learning framework that uses images to align multiple concepts to multiple objects in an image. |
| Outcome: | The proposed learning framework improves over pretraining from scratch, using a pretrained image or/and text encoder, low data regime etc. |