| Challenge: | Recent training-free methods suggest that accuracy can be improved without fine-tuning. |
| Approach: | They propose a method for correcting errors in trained contrastive image-text retrieval models with no additional training, called Nearest Neighbor Normalization. |
| Outcome: | The proposed method improves retrieval metrics for all contrastive models and datasets and does not require training on the reference database. |
Similar Papers
Normalized Contrastive Learning for Text-Video Retrieval (2022.emnlp-main)
Copied to clipboard
| Challenge: | Cross-modal contrastive learning suffers from incorrect normalization of the sum retrieval probabilities of each text or video instance. |
| Approach: | They propose a normalized contrastive learning algorithm that normalizes the sum retrieval probabilities of each instance so that every text and video instance is fairly represented. |
| Outcome: | Empirical results show that NCL brings significant gains in text-video retrieval on different model architectures without any architecture engineering. |
Contrastive Learning-Enhanced Nearest Neighbor Mechanism for Multi-Label Text Classification (2022.acl-short)
Copied to clipboard
| Challenge: | Existing methods for multi-label text classification neglect the knowledge from the existing similar instances when predicting labels of a specific text. |
| Approach: | They propose a k nearest neighbor mechanism which retrieves several neighbor instances and interpolates the model output with their labels. |
| Outcome: | Extensive experiments show that the proposed method can bring significant performance improvements to multiple MLTC models including state-of-the-art pretrained and non-pretrained ones. |
On Retrieval Augmentation and the Limitations of Language Model Training (2024.naacl-short)
Copied to clipboard
| Challenge: | Recent efforts to improve the performance of language models (LMs) have focused on scaling up model and training data size, though with steep accompanying energy and compute resource costs. |
| Approach: | They propose to augment a language model with k-nearest neighbors retrieval on its training data to reduce its perplexity. |
| Outcome: | The proposed model reduces storage costs by over 25x compared to traditional retrieval methods for GPT-2 and Mistral 7B . |
Contrastive Fine-tuning Improves Robustness for Neural Rankers (2021.findings-acl)
Copied to clipboard
| Challenge: | Current state-of-the-art neural rankers can deteriorate when exposed to noisy inputs or applied to a new domain. |
| Approach: | They propose a contrastive loss and ranking loss method for fine-tuning rankers that combine ranking loss and rank loss to improve their robustness to query reformulations and noise perturbations. |
| Outcome: | The proposed method outperforms data augmentation for robustifying rankers on four passage ranking datasets. |
Learning Decoupled Retrieval Representation for Nearest Neighbour Neural Machine Translation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods to integrate external corpus are sparse in practical applications, and noises in low similarity retrieval could lead to severe performance degradation. |
| Approach: | They propose a method to integrate external corpus into k-nearest neighbor machine translation (kNNMT) instead of storing discrete word sequence, kNN-MT uses a pre-trained NMT model to force decoding the external corpi. |
| Outcome: | The proposed approach improves retrieval accuracy and BLEU score on five domains compared to vanilla kNNMT. |
Neighborhood Contrastive Learning for Scientific Document Representations with Citation Embeddings (2022.emnlp-main)
Copied to clipboard
| Challenge: | Prior work relies on discrete citation relations to generate contrast samples, but discrete ones enforce a hard cut-off to similarity. |
| Approach: | They propose to use nearest neighbor sampling to learn continuous similarity and to sample hard-to-learn negatives and positives by controlling the sampling margin between them. |
| Outcome: | The proposed method outperforms the state-of-the-art on the SciDocs benchmark and can train (or tune) language models sample-efficiently. |
Contrastive Data and Learning for Natural Language Processing (2022.naacl-tutorials)
Copied to clipboard
| Challenge: | Current NLP models heavily rely on effective representation learning algorithms. |
| Approach: | This tutorial introduces contrastive learning and provides an introduction to the techniques. |
| Outcome: | This tutorial provides an introduction to the fundamentals of contrastive learning approaches and the theory behind them. |
Exploiting Pseudo Image Captions for Multimodal Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to multimodal summarization with multimodal output (MSMO) lack reference images for training, and exposure of image captions during training is inconsistent with MSMO’s task settings. |
| Approach: | They propose a coarse-to-fine image-text alignment mechanism to identify the most relevant sentence of each image in a document, resembling the role of image captions in capturing visual knowledge. |
| Outcome: | The proposed method sets up state-of-the-art on all intermodality and intramodality metrics and improves on image recommendation precision. |
Nearest Neighbour Few-Shot Learning for Cross-lingual Classification (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing pre-trained models can cause over-fitting when limited data are available. |
| Approach: | They propose to use a nearest-neighbor few-shot technique to improve cross-lingual adaptation using 16 distinct languages across two NLP tasks. |
| Outcome: | The proposed approach improves fine-tuning using only a handful of labeled samples in target locales and also generalizes across tasks. |
Efficient Nearest Neighbor Language Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Non-parametric neural language models (NLMs) learn text distributions by memorizing training data points. |
| Approach: | They propose to use an external datastore to learn from a non-parametric language model. |
| Outcome: | The proposed methods achieve up to a 6x speed-up in inference speed while retaining comparable performance. |