Papers by Santiago Cuervo
Scaling Properties of Speech Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Speech Language Models (SLMs) aim to learn language from raw audio without textual resources. |
| Approach: | They propose to use scaling properties of neural language models to estimate scale at which SLMs will be trained . they establish a strong correlation between pre-training loss and downstream syntactic and semantic performance . |
| Outcome: | The proposed model will have the English proficiency of text-based Large Language Models. |