Papers by Rustem Takhanov
Reusing Weights in Subword-Aware Neural Language Models (N18-1)
Copied to clipboard
| Challenge: | a statistical language model assigns a probability to a sequence of words . data sparsity is a major problem in building traditional n-gram language models . |
| Approach: | They propose several ways to reuse subword embeddings and other weights in subword-aware neural language models. |
| Outcome: | The proposed techniques do not benefit a competitive character-aware model . but they show significant reductions in model sizes and performance. |
Reproducing and Regularizing the SCRN Model (C18-1)
Copied to clipboard
| Challenge: | Recurrent neural networks (RNNs) have demonstrated tremendous success in sequence modeling . naive dropout, variational dropout and weight tying are common techniques used to regularize the SCRN model . |
| Approach: | They propose a Structurally Constrained Recurrent Network (SCRN) model and regularize it using existing techniques. |
| Outcome: | The proposed model outperforms the LSTM model on non-English data while being much simpler. |