Papers by Itai Gat
SpiRit-LM: Interleaved Spoken and Written Language Model (2025.tacl-1)
Copied to clipboard
Tu Anh Nguyen, Benjamin Muller, Bokai Yu, Marta R. Costa-jussa, Maha Elbayad, Sravya Popuri, Christophe Ropers, Paul-Ambroise Duquenne, Robin Algayres, Ruslan Mavlyutov, Itai Gat, Mary Williamson, Gabriel Synnaeve, Juan Pino, Benoît Sagot, Emmanuel Dupoux
| Challenge: | SpiRit-LM is a foundation multimodal language model that freely mixes text and speech. |
| Approach: | They propose a multimodal language model that freely mixes text and speech . they extend the model to the speech modality by continuously training it on text and language units. |
| Outcome: | The proposed model can learn new tasks in a few-shot fashion across modalities. |
Are VQA Systems RAD? Measuring Robustness to Augmented Data with Focused Interventions (2021.acl-short)
Copied to clipboard
| Challenge: | Existing systems for visual question answering are overfitted to training data and are sensitive to small perturbations. |
| Approach: | They propose a robustness measure to augment visual question answering datasets to measure generalization capabilities. |
| Outcome: | The proposed model can quantify failure cases which reveal that current systems are still brittle. |