| Challenge: | Existing deep neural networks have a tiny memory footprint and low computational capacity compared to high performance computing systems such as CPUs, GPUs and TPUs on the cloud. |
| Approach: | They propose on-device self-governing neural networks which learn compact projection vectors with local sensitive hashing. |
| Outcome: | The proposed models perform better on dialog act classification tasks while maintaining high accuracy. |
Similar Papers
Self-Governing Neural Networks for On-Device Short Text Classification (D18-1)
Copied to clipboard
| Challenge: | Existing deep neural networks have a tiny memory footprint and low computational capacity compared to high performance computing systems such as CPUs, GPUs and TPUs on the cloud. |
| Approach: | They propose on-device self-governing neural networks which learn compact projection vectors with local sensitive hashing. |
| Outcome: | The proposed models perform better on dialog act classification tasks while maintaining high accuracy. |
On-device Structured and Context Partitioned Projection Networks (P19-1)
Copied to clipboard
| Challenge: | A challenge in on-device text classification is to build highly accurate models that fit in small memory footprint and have low latency. |
| Approach: | They propose an on-device neural network which learns compact projection vectors from raw text using structured and context-dependent partition projections. |
| Outcome: | The proposed model outperforms baseline models and surpasses RNN, CNN and BiLSTM models on dialog act and intent prediction. |
PRADO: Projection Attention Networks for Document Classification On-Device (D19-1)
Copied to clipboard
| Challenge: | Recent advances in deep learning have improved the performance of on-device neural networks for long text classification. |
| Approach: | They propose a projection attention neural network PRADO that combines trainable projections with attention and convolutions to train tiny neural networks that achieve high performance on multiple long document classification tasks. |
| Outcome: | The proposed model achieves high performance on multiple long document classification tasks while maintaining compact size. |
ProSeqo: Projection Sequence Networks for On-Device Text Classification (D19-1)
Copied to clipboard
| Challenge: | ProSeqo is a novel on-device sequence model for text classification . it uses dynamic recurrent projections without the need to store or look up pre-trained embeddings. |
| Approach: | They propose a novel on-device sequence model for text classification using recurrent projections that uses dynamic recursion projections without the need to store or look up any pre-trained embeddings. |
| Outcome: | The proposed model outperforms state-of-the-art neural and on-device approaches for short and long text classification tasks while maintaining low memory footprint and high accuracy. |
Fast Word Predictor for On-Device Application (2020.coling-demos)
Copied to clipboard
| Challenge: | a proposed word prediction model is developed for a chat application serving more than 100 million users. |
| Approach: | They propose a fast word predictor that reduces memory size and inference time on mobile devices. |
| Outcome: | The proposed model reduces memory size and inference time on a mobile device compared with a standard neural network . it achieves robust performance by learning on large text corpora and is available on microsoft's chat app . |
On-Device Text Representations Robust To Misspellings via Projections (2021.eacl-main)
Copied to clipboard
| Challenge: | Recent advances in Locality-Sensitive Hashing (LSH)-based projection networks have demonstrated state-of-the-art performance in various classification tasks without explicit word embedding lookup tables by computing on-the fly text representations. |
| Approach: | They propose to use locality-sensitive hashing to compute on-the-fly text representations without explicit word embedding tables. |
| Outcome: | The proposed classifiers are more robust to common misspellings and perturbations of the input text compared to biLSTMs and fine-tuned BERT based methods. |
Dialogue Act Classification with Context-Aware Self-Attention (N19-1)
Copied to clipboard
| Challenge: | Recent work in Dialogue Act classification has treated the task as a sequence labeling problem using hierarchical deep neural networks. |
| Approach: | They propose a hierarchical deep neural network to model different levels of utterance and dialogue act semantics and use contextual dependencies to improve performance. |
| Outcome: | The proposed model improves on the Switchboard Dialogue Act Corpus while maintaining high accuracy. |
Handling Normalization Issues for Part-of-Speech Tagging of Online Conversational Text (L18-1)
Copied to clipboard
Géraldine Damnati, Jeremy Auguste, Alexis Nasr, Delphine Charlet, Johannes Heinecke, Frédéric Béchet
| Challenge: | a new approach to POS tagging noisy user generated text is proposed . word embeddings are trained on a noisy corpus to address both normalization and POS. |
| Approach: | They propose to use word embeddings to normalize text before tagging it, while a gated neural network based tagger handles the remaining errors. |
| Outcome: | The proposed approach normalizes some errors before tagging, while a gated neural network handles the remaining errors. |
On-Device Neural Language Model Based Word Prediction (C18-2)
Copied to clipboard
| Challenge: | Currently, on-device keyboards have limited memory and response time for word prediction . a proposed on-device neural language model based word prediction method is available for mobile devices . |
| Approach: | They propose an on-device neural language model based word prediction method that optimizes run-time memory and provides a real-time prediction environment. |
| Outcome: | The proposed model outperforms existing methods for word prediction in keystroke savings and word prediction rate and has been commercialized. |
Joint Learning of Pre-Trained and Random Units for Domain Adaptation in Part-of-Speech Tagging (N19-1)
Copied to clipboard
| Challenge: | Pre-trained neural networks struggle with learning uncommon target-specific patterns. |
| Approach: | They propose to augment the target-network with normalised, weighted and randomly initialised units that beget a better adaptation while maintaining valuable source knowledge. |
| Outcome: | The proposed method achieves state-of-the-art on 3 commonly used datasets. |