Papers by Luke Vilnis
Probabilistic Embedding of Knowledge Graphs with Box Lattice Measures (P18-1)
Copied to clipboard
| Challenge: | Structured embeddings based on regions, densities, and orderings have gained popularity for their inductive bias towards the essential asymmetries inherent in problems such as image captioning. |
| Approach: | They propose a box lattice and accompanying probability measure to capture negative correlations over arbitrary concepts. |
| Outcome: | The proposed model can capture anti-correlation and even disjoint concepts while learning from and predicting calibrated uncertainty. |
Distributional Inclusion Vector Embedding for Unsupervised Hypernymy Detection (N18-1)
Copied to clipboard
| Challenge: | Existing unsupervised methods for learning hypernyms from unlabeled text are not scaled to large vocabularies or yield unacceptably poor accuracy. |
| Approach: | They propose an unsupervised method of hypernym discovery using word contexts . they use word2vec to embed word context distributions without supervision . |
| Outcome: | The proposed method provides double the precision and highest average performance on 11 datasets. |
MEMORY-VQ: Compression for Tractable Internet-Scale Memory (2024.naacl-short)
Copied to clipboard
Yury Zemlyanskiy, Michiel de Jong, Luke Vilnis, Santiago Ontanon, William Cohen, Sumit Sanghai, Joshua Ainslie
| Challenge: | Memory-based methods like LUMEN pre-compute token representations for retrieved passages to speed up inference. |
| Approach: | They propose a method to reduce storage requirements of memory-augmented models . they use a vector quantization variational autoencoder to compress token representations . |
| Outcome: | The proposed method achieves 16x compression rate with comparable performance on KILT benchmark. |
Hierarchical Losses and New Resources for Fine-grained Entity Typing and Linking (P18-1)
Copied to clipboard
| Challenge: | Existing methods to incorporate hierarchical information into knowledge bases have yielded little benefit. |
| Approach: | They propose methods to integrate hierarchical information using real bilinear mappings . they also propose two new datasets containing wide and deep hierarchies . |
| Outcome: | The proposed methods improve on flat predictions and fine-grained entity typing on FIGER dataset. |