Papers by Sravan Jayanthi
Synthetic Multimodal Question Generation (2024.findings-emnlp)
Copied to clipboard
Ian Wu, Sravan Jayanthi, Vijay Viswanathan, Simon Rosenberg, Sina Pakazad, Tongshuang Wu, Graham Neubig
| Challenge: | Multimodal Retrieval Augmented Generation (MMRAG) is a powerful approach to question-answering over multimodal documents. |
| Approach: | They propose a synthetic data generation framework that leverages interplay between a retriever, large language model and large multimodal model to generate question and answer pairs directly from multimodal documents. |
| Outcome: | The proposed framework generates question and answer pairs from 1024 questions over Wikipedia documents and evaluates state-of-the-art models using it. |
Retrieve and Copy: Scaling ASR Personalization to Large Catalogs (2023.emnlp-industry)
Copied to clipboard
| Challenge: | End-to-end ASR models struggle to recognize uncommon domain-specific words due to limited audio context. |
| Approach: | They propose a "Retrieve and Copy" mechanism to improve latency while retaining the accuracy even when scaled to a large catalog. |
| Outcome: | The proposed method achieves 6% more word error rate reduction and 3.6% improvement in F1 when scaled to a large catalog size while retaining the accuracy. |