Contextual Domain Classification with Temporal Representations (2021.naacl-industry)
Copied to clipboard
| Challenge: | Existing studies that incorporate context in SLU have focused on domains where context is limited to a few minutes. |
| Approach: | They propose temporal representations that combine wall-clock second difference and turn order offset information to utilize both recent and distant context in a novel large-scale setup. |
| Outcome: | The proposed model reduces 13.04% of classification errors compared to baseline . previous studies have focused on domains where context is limited to a few minutes . |
Similar Papers
Temporal Generalization for Spoken Language Understanding (2022.naacl-industry)
Copied to clipboard
| Challenge: | Spoken Language Understanding models are usually trained offline on historical data, but must perform well on incoming user requests after deployment. |
| Approach: | They propose different strategies for achieving good temporal generalization . they focus on temporal drift, where the distribution of utterances may change . |
| Outcome: | The proposed model can perform well on unseen domains, e.g., upcoming data. |
Decay-Function-Free Time-Aware Attention to Context and Speaker Indicator for Spoken Language Understanding (N19-1)
Copied to clipboard
| Challenge: | Existing models that use contextual information of dialogues to improve spoken language understanding (SLU) select the wrong history when the histories are similar in content. |
| Approach: | They propose time-aware models that automatically learn the latent time-decay function of the history without a manual time- decay. |
| Outcome: | The proposed models achieve higher F1 scores than state-of-the-art models on a benchmark dataset . |
How Time Matters: Learning Time-Decay Attention for Contextual Spoken Language Understanding in Dialogues (N18-1)
Copied to clipboard
| Challenge: | Spoken language understanding (SLU) is an essential component in conversational systems. |
| Approach: | They propose a universal time-decay attention mechanism that can be used to decay utterances on the sentence-level and speaker-level. |
| Outcome: | The proposed model significantly improves the state-of-the-art model for contextual understanding performance on the benchmark Dialogue State Tracking Challenge (DSTC4) dataset. |
Contextualize Knowledge Bases with Transformer for End-to-end Task-Oriented Dialogue Systems (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to integrate knowledge bases into end-to-end task-oriented dialogue systems are limited in their ability to properly represent the entity of KB. |
| Approach: | They propose a framework that dynamically perceives all relevant entities and dialogue history . it uses a Memory Mask to enforce the entity to focus on its relevant entities . |
| Outcome: | The proposed framework can achieve superior performance over the state of the arts. |
Exploring Contextualized Neural Language Models for Temporal Dependency Parsing (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that deep contextualized language models (LMs) can extract temporal relations between events and time expressions. |
| Approach: | They propose a temporal relation extraction technique which extracts temporal relations between events and time expressions. |
| Outcome: | The proposed method significantly improves temporal dependency parsing, the authors show . their work compares the proposed method to other methods and shows where they may fail . |
Efficient Context and Schema Fusion Networks for Multi-Domain Dialogue State Tracking (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to track dialogue state are limited due to data sparsity and long dialogues. |
| Approach: | They propose to use the previous dialogue state and current dialogue utterance as input for DST. |
| Outcome: | The proposed approach outperforms existing methods and improves existing ones. |
How much do contextualized representations encode long-range context? (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies of contextualized representations focus on short sequences of tens to hundreds of tokens, whereas modern language models handle hundreds of thousands of token in a single context window. |
| Approach: | They use a perturbation setup and a metric to capture contextualization of long-range patterns from the perspective of representation geometry. |
| Outcome: | The proposed model can encode long-range contexts, but it's not fully recurrent, the authors say . their results suggest improvements in existing language models . |
Leveraging Contextual Embeddings for Detecting Diachronic Semantic Shift (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing methods for word embeddings have been used to model semantic relations with word embeds. |
| Approach: | They propose a method that leverages contextual embeddings for diachronic semantic shift detection by generating time specific word representations from BERT embedds. |
| Outcome: | The proposed method performs comparable to the current state-of-the-art without time consuming domain adaptation on large corpora. |
Sharing Encoder Representations across Languages, Domains and Tasks in Large-Scale Spoken Language Understanding (2023.acl-industry)
Copied to clipboard
Jonathan Hueser, Judith Gaspers, Thomas Gueudre, Chandana Prakash, Jin Cao, Daniil Sorokin, Quynh Do, Nicolas Anastassacos, Tobias Falke, Turan Gojayev
| Challenge: | Larger encoders can improve accuracy for spoken language understanding (SLU) but are difficult to use given the inference latency constraints of online systems. |
| Approach: | They propose to use a larger 170M parameter BERT encoder that shares representations across languages, domains and tasks for SLU. |
| Outcome: | The proposed encoders achieve state-of-the-art performance on numerous NLP tasks. |
Neural Temporality Adaptation for Document Classification: Diachronic Word Embeddings and Domain Adaptation Models (P19-1)
Copied to clipboard
| Challenge: | Recent studies show that document classifiers can become more stable over time when trained in ways that account for temporal variations. |
| Approach: | They propose a method for embedding diachronic word embedds into document classification models . they propose 'time-driven neural classification model' that accounts for temporal variations . |
| Outcome: | The proposed model can be trained on six corpora and make it more robust over time. |