Papers by Jihuai Zhao
Bridging the Temporal Gap in Multimodal LLMs: Deeply Stacking Temporal Tokens for Audio-Visual Speech Recognition (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing audio-visual speech recognition systems suffer from a temporal gap . visual speech patterns captured from lip movements provide complementary information that remains inherently robust to acoustic noise. |
| Approach: | They propose a framework that deeply stacks temporal tokens across both encoding and decoding stages to bridge this temporal gap. |
| Outcome: | The proposed framework outperforms existing supervised, self-supervised, and LLM-based methods by 6.1% on LRS2 and 7.8% on LLS3. |