Papers by Ajay Divakaran
Demonstrations Are All You Need: Advancing Offensive Content Paraphrasing using In-Context Learning (2024.findings-acl)
Copied to clipboard
| Challenge: | Paraphrasing of offensive content is a better alternative to content removal, but supervised methods often retain a large portion of the offensiveness of the original content. |
| Approach: | They propose to use In-Context Learning (ICL) to generate usable offensive paraphrases by using large language models. |
| Outcome: | The proposed framework is better than supervised methods on human evaluation and lower toxicity by 76%. |
Integrating Text and Image: Determining Multimodal Document Intent in Instagram Posts (D19-1)
Copied to clipboard
| Challenge: | Existing studies on text-image content have focused on image as primary content, and text as secondary content. |
| Approach: | They propose a multimodal dataset of 1299 Instagram posts labeled for three orthogonal taxonomies . they show that employing both text and image improves intent detection by 9.6 . |
| Outcome: | The proposed model shows that using both text and image improves intent detection by 9.6 compared to using only the image modality. |
Multilingual Content Moderation: A Case Study on Reddit (2023.eacl-main)
Copied to clipboard
| Challenge: | a growing need for AI moderators to safeguard users and protect mental health of human moderator from traumatic content. |
| Approach: | They propose to use a multilingual dataset to study the challenges of content moderation . they propose to analyze 1.8 million Reddit comments in English, german, spanish and french . |
| Outcome: | The proposed dataset highlights the challenges and suggests related research problems . it shows that the proposed model can be used to predict the violated rule . |
Sunny and Dark Outside?! Improving Answer Consistency in VQA through Entailed Question Generation (D19-1)
Copied to clipboard
| Challenge: | interacting with a model for Visual Question Answering (VQA) quickly reveals that these models lack consistency. |
| Approach: | They propose a dataset, ConVQA, and metrics that enable quantitative evaluation of consistency in VQA. |
| Outcome: | The proposed data augmentation module improves the consistency of VQA models on the Con-VQA dataset and is a strong baseline for future research. |
Measuring and Improving Chain-of-Thought Reasoning in Vision-Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Vision-language models have demonstrated strong efficacy as visual assistants . however, evaluation of their reasoning capabilities requires a costly benchmark . |
| Approach: | They propose a pipeline to measure the reasoning consistency of vision-language models . they propose supervised fine-tuning of VLMs and feedback from LLMs . |
| Outcome: | The proposed framework reduces cost while ensuring the generation of a high-quality dataset. |
BloomVQA: Assessing Hierarchical Multi-modal Comprehension (2024.findings-acl)
Copied to clipboard
Yunye Gong, Robik Shrestha, Jared Claypoole, Michael Cogswell, Arijit Ray, Christopher Kanan, Ajay Divakaran
| Challenge: | Recent advances of machine intelligence solutions have demonstrated tremendous success in a wide range of language and multi-modal tasks over diverse domains. |
| Approach: | They propose a VQA dataset to facilitate comprehensive evaluation of large vision-language models on comprehension tasks. |
| Outcome: | The proposed dataset shows improved accuracy over all comprehension levels and a tendency to bypass visual inputs especially for higher-level tasks. |
Pelican: Correcting Hallucination in Vision-LLMs via Claim Decomposition and Program of Thought Verification (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Visual Language Models (LVLMs) suffer from hallucinations due to limited training data, lack of * Equal contribution precise grounding, and over-reliance on language priors. |
| Approach: | They propose a framework to detect and mitigate hallucinations through claim verification using program-of-thought prompting and Python code to generate a graph. |
| Outcome: | The proposed framework improves over baseline LVLMs and existing methods across several benchmarks. |