Papers by Turan Gojayev
Sharing Encoder Representations across Languages, Domains and Tasks in Large-Scale Spoken Language Understanding (2023.acl-industry)
Copied to clipboard
Jonathan Hueser, Judith Gaspers, Thomas Gueudre, Chandana Prakash, Jin Cao, Daniil Sorokin, Quynh Do, Nicolas Anastassacos, Tobias Falke, Turan Gojayev
| Challenge: | Larger encoders can improve accuracy for spoken language understanding (SLU) but are difficult to use given the inference latency constraints of online systems. |
| Approach: | They propose to use a larger 170M parameter BERT encoder that shares representations across languages, domains and tasks for SLU. |
| Outcome: | The proposed encoders achieve state-of-the-art performance on numerous NLP tasks. |
Continuous Model Improvement for Language Understanding with Machine Translation (2021.naacl-industry)
Copied to clipboard
Abdalghani Abujabal, Claudio Delli Bovi, Sungho Ryu, Turan Gojayev, Fabian Triefenbach, Yannick Versley
| Challenge: | a simple translation-test approach would fail the latency requirements of a live environment. |
| Approach: | They show that annotating unlabeled utterances offline can improve performance . they demonstrate that an extrinsic evaluation can improve the performance if manual data is available . |
| Outcome: | The proposed method improves performance in an extrinsic evaluation setting with real-world commercial dialog system in german. |
Knowledge Distillation Transfer Sets and their Impact on Downstream NLU Tasks (2022.emnlp-industry)
Copied to clipboard
| Challenge: | Domain Classification (DC) and Intent Classification/Named Entity Recognition (ICNER) are the most common methods for reducing teacher-student knowledge into manageable sizes for low-latency downstream applications. |
| Approach: | They investigate whether distillation from a generic LM benefits downstream tasks . a domain classification and a task-specific data set are used to fine tune the model . |
| Outcome: | The proposed model improves across tasks and test sets when only task-specific data is used. |