Papers by Shravan Nayak
Grammar Search for Multi-Agent Systems (2026.acl-long)
Copied to clipboard
Mayank Singh, Vikas Yadav, Shiva Krishna Reddy Malay, Shravan Nayak, Sai Rajeswar, Sathwik Tejaswi Madhusudhan, Eduardo Blanco
| Challenge: | Several prior approaches have relied on LLM-based free-form search over the code space. |
| Approach: | They propose a more structured framework that explores the same space through a fixed set of composable components. |
| Outcome: | The proposed framework outperforms existing approaches on most benchmarks across two backbone LLMs and two domains: mathematics and question answering. |
Benchmarking Vision Language Models for Cultural Understanding (2024.emnlp-main)
Copied to clipboard
Shravan Nayak, Kanishk Jain, Rabiul Awal, Siva Reddy, Sjoerd Steenkiste, Lisa Hendricks, Karolina Stanczak, Aishwarya Agrawal
| Challenge: | Recent multimodal vision-language models have shown impressive performance in tasks such as image-to-text generation, visual question answering, and image captioning. |
| Approach: | They propose a visual question-answering benchmark to assess VLMs' cultural understanding of various facets of culture from 11 countries across 5 continents. |
| Outcome: | The visual question-answering benchmark aims to assess VLMs' cultural understanding across regions. |
CulturalFrames: Assessing Cultural Expectation Alignment in Text-to-Image Models and Evaluation Metrics (2025.findings-emnlp)
Copied to clipboard
Shravan Nayak, Mehar Bhatia, Xiaofeng Zhang, Verena Rieser, Lisa Anne Hendricks, Sjoerd Van Steenkiste, Yash Goyal, Karolina Stanczak, Aishwarya Agrawal
| Challenge: | CulturalFrames is a benchmark designed for rigorous human evaluation of cultural representation in visual generations. |
| Approach: | They propose to quantify the alignment of T2I models and evaluation metrics with respect to both explicit (stated) and implicit (unstated, implied by the prompt’s cultural context) cultural expectations. |
| Outcome: | The proposed model is based on 983 prompts, 3637 images and 10k human annotations from 10 countries and 5 socio-cultural domains. |
Pre-Trained Multilingual Sequence-to-Sequence Models: A Hope for Low-Resource Language Translation? (2022.findings-acl)
Copied to clipboard
En-Shiun Lee, Sarubi Thillainathan, Shravan Nayak, Surangika Ranathunga, David Adelani, Ruisi Su, Arya McCarthy
| Challenge: | Pre-trained multilingual sequence-to-sequence models like mBART and mT5 can be used to translate low-resource languages, but their practical application is unclear. |
| Approach: | They conduct an empirical experiment in 10 languages to determine what can pre-trained multilingual sequence-to-sequence models like mBART do to translate low-resource languages? |
| Outcome: | The proposed models are robust to domain differences, but translations for unseen and typologically distant languages remain below 3.0 BLEU. |
The Two Shades of Dubbing in Neural Machine Translation (2020.coling-main)
Copied to clipboard
| Challenge: | Dubbing has two shades; synchronisation constraints are applied only when the actor’s mouth is visible on screen, while the translation is unconstrained for off-screen dubbing. |
| Approach: | They annotate an existing dubbing corpus for this dichotomy and find that on-screen dubbing is more difficult for MT than off-screen. |
| Outcome: | The results show that on-screen dubbing is more difficult for MT than off-screen translation, and that synchronisation constraints dramatically decrease translation quality for off- screen dubbing. |
Improving Adversarial Robustness in Vision-Language Models with Architecture and Prompt Design (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Vision-Language Models (VLMs) have seen a significant increase in research interest and real-world applications, including healthcare, autonomous systems, and security. |
| Approach: | They propose novel approaches to enhance model robustness through prompt engineering by suggesting adversarial perturbations or rephrasing questions. |
| Outcome: | The proposed approaches improve model robustness against strong image-based attacks such as Auto-PGD. |
Merkel Podcast Corpus: A Multimodal Dataset Compiled from 16 Years of Angela Merkel’s Weekly Video Podcasts (2022.lrec-1)
Copied to clipboard
| Challenge: | a dataset of 16 years of (almost) weekly Internet podcasts of former german chancellor Angela Merkel is presented. |
| Approach: | They propose to curate a German podcast corpus from 16 years of podcasts of former german chancellor Angela Merkel using audio-visual-text methods. |
| Outcome: | The proposed pipeline can be used to curate other datasets of similar nature, such as talk show contents. |