Papers by Quynh Do
Sharing Encoder Representations across Languages, Domains and Tasks in Large-Scale Spoken Language Understanding (2023.acl-industry)
Copied to clipboard
Jonathan Hueser, Judith Gaspers, Thomas Gueudre, Chandana Prakash, Jin Cao, Daniil Sorokin, Quynh Do, Nicolas Anastassacos, Tobias Falke, Turan Gojayev
| Challenge: | Larger encoders can improve accuracy for spoken language understanding (SLU) but are difficult to use given the inference latency constraints of online systems. |
| Approach: | They propose to use a larger 170M parameter BERT encoder that shares representations across languages, domains and tasks for SLU. |
| Outcome: | The proposed encoders achieve state-of-the-art performance on numerous NLP tasks. |
Distributionally Robust Finetuning BERT for Covariate Drift in Spoken Language Understanding (2022.acl-long)
Copied to clipboard
| Challenge: | Covariate drift can occur when there is a drift between training and testing regarding what users request or how they request it. |
| Approach: | They propose a method that exploits natural variations in data to create a covariate drift in spoken language understanding datasets. |
| Outcome: | The proposed method improves robustness against covariate drift in spoken language understanding (SLU) it shows that a state-of-the-art model suffers performance loss under this drift. |
Towards Need-Based Spoken Language Understanding Model Updates: What Have We Learned? (2022.emnlp-industry)
Copied to clipboard
| Challenge: | In productionized machine learning systems, online model performance deteriorates when there is a distributional drift between offline training and online data. |
| Approach: | They propose a need-based retraining strategy guided by an efficient drift detector . they propose overlapping model releases, observation limitation and lack of annotated resources at runtime . |
| Outcome: | The proposed strategy reduces the cost of retraining models at fixed intervals . the proposed strategy can detect drifts when the model is applied on a new data set . |
To What Degree Can Language Borders Be Blurred In BERT-based Multilingual Spoken Language Understanding? (2020.coling-main)
Copied to clipboard
| Challenge: | Existing models for multilingual SLU are mostly DNN-based joint models of intent classification and slot filling. |
| Approach: | They propose a BERT-based adversarial model architecture to learn language-shared and language-specific representations for multilingual SLU. |
| Outcome: | The proposed model narrows the gap to the ideal multilingual performance. |
Cross-lingual Transfer Learning with Data Selection for Large-Scale Spoken Language Understanding (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to improve cross-lingual transfer learning on spoken language are pre-train on all available supervised data from another language. |
| Approach: | They propose a language model based source-language data selection method for cross-lingual transfer learning in spoken language understanding. |
| Outcome: | The proposed method reduces training time and improves model performance on spoken language understanding. |