Papers by Vlad Niculae
Representation Collapse in Machine Translation Through the Lens of Angular Dispersion (2026.findings-eacl)
Copied to clipboard
| Challenge: | Effective training of Transformer models for sequential language tasks is difficult due to various forms of collapse of the internal representations learned. |
| Approach: | They propose to use angular dispersion to analyze representation collapse at different levels of discrete and continuous transformers throughout training. |
| Outcome: | The proposed method mitigates collapse and improves translation quality. |
Sparse Sequence-to-Sequence Models (P19-1)
Copied to clipboard
| Challenge: | Sequence-to-sequence models are dense and assigning nonzero probability to implausible outputs. |
| Approach: | They propose a new family of -entmax transformations that includes softmax and sparsemax as particular cases and is sparser for any > 1 . they provide fast algorithms to evaluate these transformations and their gradients, which scale well for large vocabulary sizes. |
| Outcome: | The proposed models are able to produce sparse alignments and assign nonzero probability to short list of plausible outputs, sometimes rendering beam search exact. |
Understanding the Mechanics of SPIGOT: Surrogate Gradients for Latent Structure Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Latent structure models can mitigate the error propagation and annotation bottleneck in pipeline systems, while uncovering linguistic insights about the data. |
| Approach: | They propose a latent structure model with a pullback of the downstream learning objective. |
| Outcome: | The proposed model outperforms the known and proposed model in the same family and yields new insights for practitioners and revealing intriguing failure cases. |
Unlocking Latent Discourse Translation in LLMs Through Quality-Aware Decoding (2026.eacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) struggle to adequately handle discourse phenomena, such as pronoun resolution and lexical cohesion at the document level. |
| Approach: | They propose to use minimum Bayes risk decoding to extract discourse knowledge from LLMs and propose to apply QAD to enhance the semantic richness of translations. |
| Outcome: | The proposed method outperforms other methods and enhances the semantic richness of translations and aligns them more closely with human preferences. |
Latent Structure Models for Natural Language Processing (P19-4)
Copied to clipboard
| Challenge: | Latent structure models are a powerful tool for compositional data modeling and pipelines. |
| Approach: | This tutorial will cover recent advances in discrete latent structure models . it will discuss their motivation, potential, and limitations . |
| Outcome: | This tutorial will cover recent advances in discrete latent structure models . it will discuss their motivation, potential, and limitations . |
Towards Dynamic Computation Graphs via Sparse Latent Structure (D18-1)
Copied to clipboard
| Challenge: | Existing approaches to learn latent structure are limited by factorization assumptions or end-to-end differentiability. |
| Approach: | They propose a method that allows for end-to-end learning of latent structure predictors jointly with a downstream predictor. |
| Outcome: | The proposed method allows for unrestricted dynamic graph construction from the global latent structure while maintaining differentiability. |
On Measuring Context Utilization in Document-Level MT Systems (2024.findings-eacl)
Copied to clipboard
| Challenge: | Current studies on document-level translation evaluation focus on sentence-level models which are inadequate for capturing improvements in discourse phenomena. |
| Approach: | They propose to complement accuracy-based evaluation with measures of context utilization. |
| Outcome: | The proposed model can be used to handle context-dependent discourse phenomena using an automatic annotation tool. |
Viewing Knowledge Transfer in Multilingual Machine Translation Through a Representational Lens (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies show that translation quality alone is not sufficient for measuring knowledge transfer in multilingual neural machine translation. |
| Approach: | They propose a method that measures representational similarities between languages to measure knowledge transfer. |
| Outcome: | The proposed method improves translation quality for low- and mid-resource languages across multiple datasets and models. |
Angular Dispersion Accelerates k-Nearest Neighbors Machine Translation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | k-nearest neighbors machine translation is a well-established strategy for increasing translation performance . high computational cost and memory requirements make it difficult to perform exhaustive searches in large data stores . |
| Approach: | They propose to augment neural machine translation with external memory at decoding time . they propose to encourage angular dispersion of the neural hidden representations of contexts . |
| Outcome: | The proposed method is expensive and high memory-intensive . it is based on the performance properties of k-NN lookup data structures. |
The Unreasonable Effectiveness of Random Target Embeddings for Continuous-Output Neural Machine Translation (2024.naacl-short)
Copied to clipboard
| Challenge: | Continuous-output neural machine translation models are trained to predict the continuous representation based on distances between vectors. |
| Approach: | They propose a continuous-output neural machine translation (CoNMT) approach that uses random output embeddings to outperform laboriously pre-trained models. |
| Outcome: | The proposed strategy outperforms pre-trained embeddings on large datasets and is strongest for rare words due to the geometry of their embedders. |
kNN For Whisper And Its Effect On Bias And Speaker Adaptation (2025.findings-naacl)
Copied to clipboard
| Challenge: | Speech recognition performance varies by language, domain, speaker characteristics such as accent . fine-tuning models on any of these categories may lead to catastrophic forgetting . |
| Approach: | They propose a nonparametric method that adapts to any category without training the model. |
| Outcome: | The proposed method improves the performance of a transformer end-to-end speech model on transcribed audio in over 680,000 hours of training data on a wide variety of languages. |
Adaptively Sparse Transformers (D19-1)
Copied to clipboard
| Challenge: | Recent architectures, notably the Transformer, learn powerful context-aware word representations through layered, multi-headed attention. |
| Approach: | They propose an adaptively sparse Transformer where attention heads have flexible, context-dependent sparsity patterns. |
| Outcome: | The proposed model improves interpretability and head diversity when compared to softmax-based models on machine translation datasets. |