Papers by Ozlem Kalinli
AudioChatLlama: Towards General-Purpose Speech Abilities for LLMs (2024.naacl-long)
Copied to clipboard
Yassir Fathullah, Chunyang Wu, Egor Lakomkin, Ke Li, Junteng Jia, Yuan Shangguan, Jay Mahadeokar, Ozlem Kalinli, Christian Fuegen, Mike Seltzer
| Challenge: | a new model for speech processing and reasoning uses curated data instead of text. |
| Approach: | They extend the instruction-tuned Llama-2 model with end-to-end speech processing and reasoning abilities without using any carefully curated paired data. |
| Outcome: | The proposed model outperforms or outperfects existing models on synthesized and recorded speech QA tests. |
Joint Audio/Text Training for Transformer Rescorer of Streaming Speech Recognition (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have shown that streaming end-to-end speech recognition models suffer from higher word error rates (WER) compared to non-streaming models, streaming endto-ended ASR models are limited to short audio context or not use future context to satisfy low latency constraints. |
| Approach: | They propose a 2nd-pass rescoring model on top of the 1st-pass streaming model to improve recognition accuracy while keeping latency low. |
| Outcome: | The proposed method improves word error rate significantly compared to the existing model without adding any additional parameters or latency. |