Papers by Prashant Bannulmath
LaRA: Large Rank Adaptation for Speech and Text Cross-Modal Learning in Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to integrate speech and text capabilities into large language models (LLMs) require significantly larger ranks comparable to the pretrained weights to accommodate the complexities of speech-text cross-modality learning. |
| Approach: | They propose a large-rank adaptive approach for cross-modal integration of speech and text into large language models (LLMs) it uses a Hi-Fi vocoder to synthesize speech waveforms from the generated speech units. |
| Outcome: | The proposed model can be extended to other cross-modal applications. |