| Challenge: | ProSeqo is a novel on-device sequence model for text classification . it uses dynamic recurrent projections without the need to store or look up pre-trained embeddings. |
| Approach: | They propose a novel on-device sequence model for text classification using recurrent projections that uses dynamic recursion projections without the need to store or look up any pre-trained embeddings. |
| Outcome: | The proposed model outperforms state-of-the-art neural and on-device approaches for short and long text classification tasks while maintaining low memory footprint and high accuracy. |
Similar Papers
On-device Structured and Context Partitioned Projection Networks (P19-1)
Copied to clipboard
| Challenge: | A challenge in on-device text classification is to build highly accurate models that fit in small memory footprint and have low latency. |
| Approach: | They propose an on-device neural network which learns compact projection vectors from raw text using structured and context-dependent partition projections. |
| Outcome: | The proposed model outperforms baseline models and surpasses RNN, CNN and BiLSTM models on dialog act and intent prediction. |
Self-Governing Neural Networks for On-Device Short Text Classification (D18-1)
Copied to clipboard
| Challenge: | Existing deep neural networks have a tiny memory footprint and low computational capacity compared to high performance computing systems such as CPUs, GPUs and TPUs on the cloud. |
| Approach: | They propose on-device self-governing neural networks which learn compact projection vectors with local sensitive hashing. |
| Outcome: | The proposed models perform better on dialog act classification tasks while maintaining high accuracy. |
Self-Governing Neural Networks for On-Device Short Text Classification (D18-1)
Copied to clipboard
| Challenge: | Existing deep neural networks have a tiny memory footprint and low computational capacity compared to high performance computing systems such as CPUs, GPUs and TPUs on the cloud. |
| Approach: | They propose on-device self-governing neural networks which learn compact projection vectors with local sensitive hashing. |
| Outcome: | The proposed models perform better on dialog act classification tasks while maintaining high accuracy. |
Fast Word Predictor for On-Device Application (2020.coling-demos)
Copied to clipboard
| Challenge: | a proposed word prediction model is developed for a chat application serving more than 100 million users. |
| Approach: | They propose a fast word predictor that reduces memory size and inference time on mobile devices. |
| Outcome: | The proposed model reduces memory size and inference time on a mobile device compared with a standard neural network . it achieves robust performance by learning on large text corpora and is available on microsoft's chat app . |
PRADO: Projection Attention Networks for Document Classification On-Device (D19-1)
Copied to clipboard
| Challenge: | Recent advances in deep learning have improved the performance of on-device neural networks for long text classification. |
| Approach: | They propose a projection attention neural network PRADO that combines trainable projections with attention and convolutions to train tiny neural networks that achieve high performance on multiple long document classification tasks. |
| Outcome: | The proposed model achieves high performance on multiple long document classification tasks while maintaining compact size. |
On-Device Neural Language Model Based Word Prediction (C18-2)
Copied to clipboard
| Challenge: | Currently, on-device keyboards have limited memory and response time for word prediction . a proposed on-device neural language model based word prediction method is available for mobile devices . |
| Approach: | They propose an on-device neural language model based word prediction method that optimizes run-time memory and provides a real-time prediction environment. |
| Outcome: | The proposed model outperforms existing methods for word prediction in keystroke savings and word prediction rate and has been commercialized. |
Denoising based Sequence-to-Sequence Pre-training for Text Generation (D19-1)
Copied to clipboard
| Challenge: | PoDA pre-trains encoders and decoders by denoising noise-corrupted text . Unlike encoder-only or decode-only methods, it can be used for text generation tasks without using any task-specific techniques. |
| Approach: | They propose a sequence-to-sequence (seq2sequ) pre-training method PoDA which denoises autoencoders by denoising noise-corrupted text. |
| Outcome: | The proposed method improves model performance over strong baselines without using any task-specific techniques and significantly speed up convergence. |
Text Generation with Text-Editing Models (2022.naacl-tutorials)
Copied to clipboard
Eric Malmi, Yue Dong, Jonathan Mallinson, Aleksandr Chuklin, Jakub Adamek, Daniil Mirylenka, Felix Stahlberg, Sebastian Krause, Shankar Kumar, Aliaksei Severyn
| Challenge: | Text-editing models are a popular alternative to seq2seq for monolingual text generation tasks such as text summarization and style transfer. |
| Approach: | They propose to use text-editing models to predict edit operations applied to the source sequence and to generate outputs word-by-word from scratch. |
| Outcome: | This paper provides an overview of the text-edit based models and their current state-of-the-art approaches. |
Seq2SeqPy: A Lightweight and Customizable Toolkit for Neural Sequence-to-Sequence Modeling (2020.lrec-1)
Copied to clipboard
| Challenge: | Neural models have attracted a lot of attention in the past few years due to their complexity and need to be customized to meet specific needs. |
| Approach: | They propose a lightweight toolkit for sequence-to-sequence modeling that prioritizes simplicity and ability to customize the standard architectures easily. |
| Outcome: | The proposed tool performs similarly or even better than a very widely used sequence-to-sequence toolkit. |
RecGPT: Generative Pre-training for Text-based Recommendation (2024.acl-short)
Copied to clipboard
| Challenge: | Existing models for text-based recommendation lack data sparsity and flexibility to capture fluctuations in user preferences over time. |
| Approach: | They present the first domain-adapted and fully-trained large language model for text-based recommendation. |
| Outcome: | The proposed model outperforms baseline models on rating prediction and sequential recommendation tasks. |