Papers by Wenliang Dai
Automatic Speech Recognition Datasets in Cantonese: A Survey and New Dataset (2022.lrec-1)
Copied to clipboard
Tiezheng Yu, Rita Frieske, Peng Xu, Samuel Cahyawijaya, Cheuk Tung Yiu, Holy Lovenia, Wenliang Dai, Elham J. Barezi, Qifeng Chen, Xiaojuan Ma, Bertram Shi, Pascale Fung
| Challenge: | In this paper, we address the problem of data scarcity for the Hong Kong Cantonese language . due to the popularization of deep learning, ASR technology has led to a significant improvement in recognizing many languages. |
| Approach: | They propose to use a dataset to analyze the data available for the Hong Kong Cantonese language . they use zh-HK as a source and a state-of-the-art ASR model to build a powerful model . |
| Outcome: | The proposed model improves on the biggest existing dataset, Common Voice zh-HK. |
mCLIP: Multilingual CLIP via Cross-lingual Transfer (2023.acl-long)
Copied to clipboard
Guanhua Chen, Lu Hou, Yun Chen, Wenliang Dai, Lifeng Shang, Xin Jiang, Qun Liu, Jia Pan, Wenping Wang
| Challenge: | Existing multilingual vision-language pretrained models are biased towards English due to the lack of sufficient non-English image-text pairs. |
| Approach: | They propose to train a retrieval-efficient dual-stream multilingual VLP model by aligning CLIP model and a multilingual text encoder through a novel Triangle Cross-modal Knowledge Distillation method. |
| Outcome: | Empirical results show that mCLIP achieves new state-of-the-art performance for both zero-shot and finetuned multilingual image-text retrieval tasks. |
Multi-hop Question Generation with Graph Convolutional Network (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on text-based QG focus on generating SQuAD-style questions. |
| Approach: | They propose a multi-hop question generation model that does context encoding in multiple hops with Graph Convolutional Network and encoder fusion via an Encoder Reasoning Gate. |
| Outcome: | Empirical results show that the proposed model generates fluent questions with high completeness and outperforms baselines on automatic evaluation metrics. |
Vision Guided Generative Pre-trained Language Models for Multimodal Abstractive Summarization (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing MAS models cannot leverage GPLMs’ powerful generation ability. |
| Approach: | They propose a method to construct vision guided (VG) GPLMs that incorporate visual information while maintaining their original text generation ability. |
| Outcome: | The proposed model outperforms the previous state-of-the-art model by 5.7 ROUGE-1, 5.3 ROUGe-2, and 5.1 ROUGEL-L scores on the How2 dataset and contributes 83.6% of the overall improvement. |
SelfMix: Robust Learning against Textual Label Noise with Self-Mixup Training (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods to handle label noise in text classification tasks are limited to visual data. |
| Approach: | They propose a method to handle label noise in text classification tasks using a Gaussian Mixture Model. |
| Outcome: | The proposed method outperforms baselines on three types of text classification tasks on visual and textual data. |
NusaCrowd: Open Source Initiative for Indonesian NLP Resources (2023.findings-acl)
Copied to clipboard
Samuel Cahyawijaya, Holy Lovenia, Alham Fikri Aji, Genta Winata, Bryan Wilie, Fajri Koto, Rahmad Mahendra, Christian Wibisono, Ade Romadhony, Karissa Vincentio, Jennifer Santoso, David Moeljadi, Cahya Wirawan, Frederikus Hudi, Muhammad Satrio Wicaksono, Ivan Parmonangan, Ika Alfina, Ilham Firdausi Putra, Samsul Rahmadani, Yulianti Oenang, Ali Septiandri, James Jaya, Kaustubh Dhole, Arie Suryani, Rifki Afina Putri, Dan Su, Keith Stevens, Made Nindyatama Nityasya, Muhammad Adilazuarda, Ryan Hadiwijaya, Ryandito Diandaru, Tiezheng Yu, Vito Ghifari, Wenliang Dai, Yan Xu, Dyah Damapuspita, Haryo Wibowo, Cuk Tho, Ichwanul Karo Karo, Tirana Fatyanosa, Ziwei Ji, Graham Neubig, Timothy Baldwin, Sebastian Ruder, Pascale Fung, Herry Sujaini, Sakriani Sakti, Ayu Purwarianti
| Challenge: | Existing NLP research in Indonesian languages has been held back by factors such as language diversity, orthographic variation, resource limitation and other societal challenges. |
| Approach: | They present a collaborative initiative to collect and unify existing resources for Indonesian languages and open access to previously non-public resources. |
| Outcome: | The results show that the datasets are highly reliable and can be used to generate the first zero-shot benchmarks for natural language understanding and generation in Indonesian and the local languages of Indonesia. |
Multimodal End-to-End Sparse Model for Emotion Recognition (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing work in emotion recognition uses a two-phase pipeline, but the extracted features are fixed and cannot be fine-tuned on different tasks. |
| Approach: | They propose a two-phase pipeline for emotion recognition and personality recognition . they propose restructured datasets to enable fully end-to-end training . |
| Outcome: | The proposed model outperforms the current state-of-the-art models on emotion recognition and personality recognition tasks with half less computation in the feature extraction part. |
CI-AVSR: A Cantonese Audio-Visual Speech Datasetfor In-car Command Recognition (2022.lrec-1)
Copied to clipboard
Wenliang Dai, Samuel Cahyawijaya, Tiezheng Yu, Elham J. Barezi, Peng Xu, Cheuk Tung Yiu, Rita Frieske, Holy Lovenia, Genta Winata, Qifeng Chen, Xiaojuan Ma, Bertram Shi, Pascale Fung
| Challenge: | In-car smart assistants should be able to process general as well as car-related commands and perform corresponding actions, which eases driving and improves safety. |
| Approach: | They propose a dataset for in-car command recognition in the cantonese language with both video and audio data. |
| Outcome: | The proposed model can achieve a considerable quality on the clean test set, but the speech recognition quality on noisy data is still inferior. |
ASCEND: A Spontaneous Chinese-English Dataset for Code-switching in Multi-turn Conversation (2022.lrec-1)
Copied to clipboard
Holy Lovenia, Samuel Cahyawijaya, Genta Winata, Peng Xu, Yan Xu, Zihan Liu, Rita Frieske, Tiezheng Yu, Wenliang Dai, Elham J. Barezi, Qifeng Chen, Xiaojuan Ma, Bertram Shi, Pascale Fung
| Challenge: | Code-switching is a speech phenomenon occurring when a speaker switches language during a conversation. |
| Approach: | They propose to collect Mandarin Chinese-English code-switching corpus from read speech rather than spontaneous speech to address this phenomenon. |
| Outcome: | ASCEND consists of 10.62 hours of clean speech, collected from 23 bilingual speakers of Chinese and English. |
Enabling Multimodal Generation on CLIP via Vision-Language Knowledge Distillation (2022.findings-acl)
Copied to clipboard
| Challenge: | Recent large-scale vision-language pre-training models are powerful in multimodal classification and retrieval tasks. |
| Approach: | They propose to augment a vision-language pre-training model with a textual pre-trained language model . the model achieves 44.5% zero-shot accuracy on multimodal generation tasks . |
| Outcome: | The proposed model achieves 44.5% zero-shot accuracy on open-ended visual question answering and image captioning tasks. |
Plausible May Not Be Faithful: Probing Object Hallucination in Vision-Language Pre-training (2023.eacl-main)
Copied to clipboard
| Challenge: | Large-scale vision-language pre-trained (VLP) models generate unfaithful or nonsensical texts given the source input, which is called hallucination. |
| Approach: | They propose a VLP loss-based model to mitigate object hallucination by decoupling VLP objectives and a token-level image-text alignment. |
| Outcome: | The proposed model reduces object hallucination by 17.4% on two benchmarks. |
Modality-Transferable Emotion Embeddings for Low-Resource Multimodal Emotion Recognition (2020.aacl-main)
Copied to clipboard
| Challenge: | despite recent advances in multimodal emotion recognition, two problems still exist: sub-optimal performance and low-resource emotions. |
| Approach: | They propose a modality-transferable model with emotion embeddings to solve these problems . they use pre-trained word embedders to represent emotion categories for textual data . |
| Outcome: | The proposed model outperforms baselines in zero-shot and few-shot scenarios for unseen emotions. |