Papers by Jinglun Cai
Masked Audio Text Encoders are Effective Multi-Modal Rescorers (2023.findings-acl)
Copied to clipboard
| Challenge: | Masked Language Models (MLMs) have proven to be effective for second-pass rescoring in Automatic Speech Recognition systems. |
| Approach: | They propose a multi-modal masked language model rescorer which integrates acoustic representations into the input space of MLM. |
| Outcome: | The proposed model reduces word error rate (WER) by 4%-16% on in-domain and 3%-7% on out-of-domain datasets over the text-only baseline. |
Balancing Multi-Domain Corpora Learning for Open-Domain Response Generation (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies on open-domain conversational systems are limited to single corpus training and evaluation. |
| Approach: | They propose a method which encodes each corpus through a unique corpus embedding and a new word-level importance weighting method that integrates DF to the loss function. |
| Outcome: | The proposed methods gain significant improvements on both automatic and human evaluation. |