Papers by Daniil Sorokin
Interactive Instance-based Evaluation of Knowledge Base Question Answering (D18-2)
Copied to clipboard
| Challenge: | Existing approaches to Knowledge Base Question Answering are based on semantic parsing. |
| Approach: | They propose a tool that aids in debugging of question answering systems that construct a structured semantic representation for the input question. |
| Outcome: | The proposed system allows debugging of model predictions on individual instances and simplifies manual error analysis. |
Sharing Encoder Representations across Languages, Domains and Tasks in Large-Scale Spoken Language Understanding (2023.acl-industry)
Copied to clipboard
Jonathan Hueser, Judith Gaspers, Thomas Gueudre, Chandana Prakash, Jin Cao, Daniil Sorokin, Quynh Do, Nicolas Anastassacos, Tobias Falke, Turan Gojayev
| Challenge: | Larger encoders can improve accuracy for spoken language understanding (SLU) but are difficult to use given the inference latency constraints of online systems. |
| Approach: | They propose to use a larger 170M parameter BERT encoder that shares representations across languages, domains and tasks for SLU. |
| Outcome: | The proposed encoders achieve state-of-the-art performance on numerous NLP tasks. |
Modeling Semantics with Gated Graph Neural Networks for Knowledge Base Question Answering (C18-1)
Copied to clipboard
| Challenge: | Existing approaches to Knowledge Base Question Answering focus on semantic parsing . previous work focused on selecting the correct semantic relations and not on the structure of the semantic parses . |
| Approach: | They propose to use Gated Graph Neural Networks to encode the graph structure of the semantic parse. |
| Outcome: | The proposed approach outperforms baseline models that do not explicitly model the structure. |
Towards Need-Based Spoken Language Understanding Model Updates: What Have We Learned? (2022.emnlp-industry)
Copied to clipboard
| Challenge: | In productionized machine learning systems, online model performance deteriorates when there is a distributional drift between offline training and online data. |
| Approach: | They propose a need-based retraining strategy guided by an efficient drift detector . they propose overlapping model releases, observation limitation and lack of annotated resources at runtime . |
| Outcome: | The proposed strategy reduces the cost of retraining models at fixed intervals . the proposed strategy can detect drifts when the model is applied on a new data set . |
Data-Efficient Paraphrase Generation to Bootstrap Intent Classification and Slot Labeling for New Features in Task-Oriented Dialog Systems (2020.coling-industry)
Copied to clipboard
| Challenge: | a number of dialog systems have been developed to perform tasks with high accuracy on benchmarks, but there is a problem with annotated seed data. |
| Approach: | They propose a model that augments initial seed data by paraphrasing existing utterances automatically. |
| Outcome: | The proposed approach improves intent classification and slot labeling on a public dataset and with a real-world dialog system. |
Local-to-global learning for iterative training of production SLU models on new features (2022.naacl-industry)
Copied to clipboard
| Challenge: | In many real-world NLP systems, new data becomes available with time and there is a need to refresh the model. |
| Approach: | They propose to adapt a local-to-global learning schedule to production settings where full data is not available at initial training iterations. |
| Outcome: | The proposed model improves model error rates by 7.3% and saves up to 25% training time for individual iterations. |
Leveraging User Paraphrasing Behavior In Dialog Systems To Automatically Collect Annotations For Long-Tail Utterances (2020.coling-industry)
Copied to clipboard
| Challenge: | In large-scale commercial dialog systems, users express the same request in a wide variety of alternative ways with a long tail of less frequent alternatives. |
| Approach: | They propose a method to leverage this feedback by creating annotated training examples from it. |
| Outcome: | The proposed method can be used in a commercial dialog system across various domains and three languages. |