Papers by Zejun Ma
CIF-PT: Bridging Speech and Text Representations for Spoken Language Understanding via Continuous Integrate-and-Fire Pre-Training (2023.findings-acl)
Copied to clipboard
| Challenge: | Speech-to-text training and language model distillation are used to bridge the representations between speech and text. |
| Approach: | They propose a pre-training paradigm that integrates speech and text into a single frame-to-token alignment. |
| Outcome: | The proposed paradigm outperforms the state-of-the-art model on intent classification and slot filling tasks. |
Audio-centric Video Understanding Benchmark without Text Shortcut (2025.emnlp-main)
Copied to clipboard
Yudong Yang, Jimin Zhuang, Guangzhi Sun, Changli Tang, Yixuan Li, Peihan Li, Yifan Jiang, Wei Li, Zejun Ma, Chao Zhang
| Challenge: | Recent advances in multimodal large language models (MLLMs) focus on visual abilities, but audio is essential for video understanding. |
| Approach: | They propose an audio-centric video understanding benchmark to evaluate video comprehension capabilities of multimodal LLMs with a particular focus on auditory information. |
| Outcome: | The proposed video understanding benchmarks evaluate video comprehension capabilities of multimodal models with a particular focus on auditory information. |
Improving Contextual Representation with Gloss Regularized Pre-training (2022.findings-naacl)
Copied to clipboard
| Challenge: | Experimental results show that the gloss regularizer module enhances word semantic similarity in pre-training. |
| Approach: | They propose an auxiliary gloss regularizer module to BERT pre-training to enhance word semantic similarity. |
| Outcome: | The proposed model improves word similarity in word-level and sentence-level representation. |