Papers by Chengqi Zhao
NeurST: Neural Speech Translation Toolkit (2021.acl-demo)
Copied to clipboard
| Challenge: | a toolkit for speech translation is available for free and provides step-by-step recipes for feature extraction, data preprocessing, distributed training, and evaluation. |
| Approach: | They propose to use NeurST to facilitate speech translation research for NLP researchers . they show experimental results for different benchmark datasets which can be regarded as reliable baselines . |
| Outcome: | The proposed framework provides reliable benchmarks for speech translation research. |
DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models (2024.acl-long)
Copied to clipboard
Damai Dai, Chengqi Deng, Chenggang Zhao, R.x. Xu, Huazuo Gao, Deli Chen, Jiashi Li, Wangding Zeng, Xingkai Yu, Y. Wu, Zhenda Xie, Y.k. Li, Panpan Huang, Fuli Luo, Chong Ruan, Zhifang Sui, Wenfeng Liang
| Challenge: | Mixture-of-Experts (MoE) architectures face challenges in ensuring expert specialization . despite the promising performance, scaling language models to an extremely large scale is associated with exceedingly high computational costs. |
| Approach: | They propose an architecture that allows for ultimate expert specialization by segmenting experts into mN ones and activating mK from them. |
| Outcome: | The proposed architecture achieves comparable performance with GShard with 2B parameters and computation. |
Rethinking Document-level Neural Machine Translation (2022.findings-acl)
Copied to clipboard
| Challenge: | Neural machine translation models are weak enough for document-level translation . current models only translate sentences individually, resulting in poor document coherence . |
| Approach: | They propose to use the original Transformer model to test document-level neural machine translation . they find that the original transformer models can achieve strong results for document translation if trained properly . |
| Outcome: | The proposed model outperforms sentence-level models on nine datasets and two sentence- level datasets across six languages. |
Autocorrect in the Process of Translation — Multi-task Learning Improves Dialogue Machine Translation (2021.naacl-industry)
Copied to clipboard
| Challenge: | Existing neural machine translation models are not able to translate dialogues in real life scenarios. |
| Approach: | They propose a joint learning method to identify omission and typos and utilize context to translate dialogue utterances. |
| Outcome: | The proposed method improves translation quality by 3.2 BLEU over baselines and recovers omitted pronouns by 47.16%. |
Secoco: Self-Correcting Encoding for Neural Machine Translation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) is a challenging field due to the wide variety of noises in real-world scenarios. |
| Approach: | They propose a framework that explicitly deals with noisy inputs for robust neural machine translation by introducing self-correcting predictors. |
| Outcome: | The proposed framework can correct noisy inputs and delete specific errors with the translation decoding process. |
BLEURT Has Universal Translations: An Analysis of Automatic Metrics by Minimum Risk Training (2023.acl-long)
Copied to clipboard
| Challenge: | Automated metrics are used for machine translation, but they are often considered to be black boxes with potential biases that are difficult to detect. |
| Approach: | They analyze automatic metrics from the perspective of their guidance for machine translation training. |
| Outcome: | The proposed measures improve the performance of machine translation models. |
Counter-Interference Adapter for Multilingual Machine Translation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to multilingual machine translation suffer from performance degradation, resulting in a single model being inferior to separately trained bilingual models on resource-rich languages. |
| Approach: | They propose a transformer-based model with a small parameter overhead for multilingual machine translation that outperforms strong multilingual baselines on 64 of 66 language directions. |
| Outcome: | The proposed model outperforms strong multilingual baselines on 64 of 66 language directions, 42 of which have above 0.5 BLEU improvement. |
Improving Speech Translation by Fusing Speech and Text (2023.findings-emnlp)
Copied to clipboard
| Challenge: | In speech translation, multimodal data to address limitations of individual modalities has shown significant effectiveness. |
| Approach: | They propose a cross-modal model which supports three input modalities for speech, text and fused speech-text. |
| Outcome: | The proposed model achieves an average of 34.0 BLEU on MuST-C, GigaST and newstest benchmark. |