Papers by Sergey Troshin
Representation Collapse in Machine Translation Through the Lens of Angular Dispersion (2026.findings-eacl)
Copied to clipboard
| Challenge: | Effective training of Transformer models for sequential language tasks is difficult due to various forms of collapse of the internal representations learned. |
| Approach: | They propose to use angular dispersion to analyze representation collapse at different levels of discrete and continuous transformers throughout training. |
| Outcome: | The proposed method mitigates collapse and improves translation quality. |
A Simple Approach for Handling Out-of-Vocabulary Identifiers in Deep Learning for Source Code (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to handle out-of-vocabulary identifiers are not suitable for source code processing. |
| Approach: | They propose a method to handle out-of-vocabulary identifiers by identifies anonymization . they show that the method significantly improves the performance of the Transformer . |
| Outcome: | The proposed method significantly improves the performance of the Transformer in two code processing tasks. |
Angular Dispersion Accelerates k-Nearest Neighbors Machine Translation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | k-nearest neighbors machine translation is a well-established strategy for increasing translation performance . high computational cost and memory requirements make it difficult to perform exhaustive searches in large data stores . |
| Approach: | They propose to augment neural machine translation with external memory at decoding time . they propose to encourage angular dispersion of the neural hidden representations of contexts . |
| Outcome: | The proposed method is expensive and high memory-intensive . it is based on the performance properties of k-NN lookup data structures. |