Papers with bi-encoder
Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Tasks (2021.naacl-main)
Copied to clipboard
| Challenge: | Cross-encoders perform full-attention over the input pair, while bi-encoding requires substantial training data and fine-tuning over the target task to achieve competitive performance. |
| Approach: | They propose a data augmentation strategy that uses cross-encoders to label larger set of input pairs to augment training data for bi-encoding. |
| Outcome: | The proposed approach improves on multiple tasks and domain adaptation tasks by up to 37 points compared to the original bi-encoder performance. |
Entity Linking in the Job Market Domain (2024.findings-eacl)
Copied to clipboard
| Challenge: | In Natural Language Processing, entity linking (EL) has centered around Wikipedia, but yet remains underexplored for the job market domain. |
| Approach: | They propose to use a bi-encoder and an autoregressive model to link fine-grained span-level skill mentions to a specific taxonomy entry to quantify labor market demands. |
| Outcome: | The proposed model outperforms GENRE in strict evaluation, but performs better in loose evaluation. |
Retrieve Fast, Rerank Smart: Cooperative and Joint Approaches for Improved Cross-Modal Retrieval (2022.tacl-1)
Copied to clipboard
| Challenge: | Current approaches to cross-modal retrieval process text and visual input jointly . current approaches are pretrained from scratch and suffer from huge retrieval latency and inefficiency issues . |
| Approach: | They propose a cooperative retrieve-and-rerank framework that turns pretrained text-image multi-modal models into efficient retrieval models. |
| Outcome: | The proposed framework improves retrieval performance over current approaches . it uses twin networks to encode all items of a corpus and a cross-encoder component for a more nuanced ranking . |
Backtracing: Retrieving the Cause of the Query (2024.findings-eacl)
Copied to clipboard
| Challenge: | a number of online content portals allow users to ask questions to supplement their understanding. |
| Approach: | They propose a task of backtracing to retrieve the text segment that most likely caused a user query. |
| Outcome: | The proposed method improves on the backtracing task in three domains . the results show that there is room for improvement and new retrieval approaches . |
Question Answering Infused Pre-training of General-Purpose Contextualized Representations (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing pretraining objectives for question answering (QA) are not optimized for being immediately useful without fine-tuning. |
| Approach: | They propose a pre-training objective based on question answering (QA) that is based more directly on context. |
| Outcome: | The proposed model matches predictions of a more accurate cross-encoder model on 80 million synthesized QA pairs and achieves large improvements over previous state-of-the-art models on paraphrase detection and fewshot named entity recognition. |
Evaluating Token-Level and Passage-Level Dense Retrieval Models for Math Information Retrieval (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study has shown that dense retrieval methods are suboptimal for capturing contextual similarities in complex data. |
| Approach: | They propose to combine a structure search method and efficient bi-encoder dense retrieval models to capture contextual similarities. |
| Outcome: | The proposed model improves on token-level and passage-level dense retrieval tasks. |
Learning to Retrieve In-Context Examples for Large Language Models (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing approaches to improve in-context learning performance are highly sensitive to the quality of the incontext examples provided. |
| Approach: | They propose a framework to iteratively train dense retrievers that can identify high-quality in-context examples for large language models. |
| Outcome: | The proposed model improves performance by retrieving examples with similar patterns, and the gains are consistent across LLMs of varying sizes. |
Fact Checking Beyond Training Set (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing fact checking systems are unsuitable for evaluating the veracity of everyday claims due to the availability of evidence resources. |
| Approach: | They propose an adversarial algorithm to make the retriever component robust against distribution shift. |
| Outcome: | The proposed method is insensitive to the order of claims and evidence documents. |
NevIR: Negation in Neural Information Retrieval (2024.eacl-long)
Copied to clipboard
| Challenge: | Negation is a common everyday phenomenon and has been a consistent area of weakness for language models. |
| Approach: | They ask IR models to rank two documents that differ only by negation . they find that most current information retrieval models do not consider negation. |
| Outcome: | The proposed benchmarks show that most current models do not consider negation . the results are similar to those found in the literature, but are poorer than random ranking . |
Task-Aware Specialization for Efficient and Robust Dense Retrieval for Open-Domain Question Answering (2023.acl-short)
Copied to clipboard
| Challenge: | Existing dense retrieval models are parameter-inefficient and underperform sparse counterparts. |
| Approach: | They propose a task-aware specialization for dEnse Retrieval architecture that enables parameter sharing by interleaving shared and specialized blocks in a single encoder. |
| Outcome: | The proposed architecture surpasses BM25 on questions and passages using 60% of the parameters as bi-encoder dense retrievers. |
GeAR: Generation Augmented Retrieval (2025.findings-acl)
Copied to clipboard
Haoyu Liu, Shaohan Huang, Jianfeng Liu, Yuefeng Zhan, Hao Sun, Weiwei Deng, Feng Sun, Furu Wei, Qi Zhang
| Challenge: | Document retrieval techniques are used to compute semantic similarity between a query and documents, but the scalar similarity fails to reflect enough information, hindering the interpretation of retrieval results. |
| Approach: | They propose a method which improves the global document-query similarity through contrastive learning and integrates well-designed fusion and decoding modules. |
| Outcome: | The proposed method improves the global document-query similarity through contrastive learning and integrates well-designed fusion and decoding modules. |
CORE: Cooperative Training of Retriever-Reranker for Effective Dialogue Response Selection (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods to train retrieval-based dialogue systems are suboptimal . existing methods to optimize retrieval and rerank modules are sub-optimal, causing sub-optimum performance. |
| Approach: | They propose a retrieval-based dialogue system with a fast retriever and a smart response reranker that combine the best of both worlds. |
| Outcome: | The proposed method can learn from each other and evolve together . it can be used in industrial applications and has powered industrial applications. |
CORE: A Retrieve-then-Edit Framework for Counterfactual Data Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work on counterfactual data augmentation only considered restricted classes of perturbations, limiting their effectiveness. |
| Approach: | They propose a retrieval-augmented framework for creating diverse counterfactual perturbations for CDA. |
| Outcome: | Experiments on natural language inference and sentiment analysis show that the proposed framework can be used to encourage diversity in manually authored perturbations. |
Triple-Encoders: Representations That Fire Together, Wire Together (2024.acl-long)
Copied to clipboard
| Challenge: | Existing search-based dialog models re-encode the dialog history at every turn, causing high cost. |
| Approach: | They propose a method that encodes relative distances between utterances via a bi-encoder . they use a hebbian-inspired co-occurrence learning objective to compute distributed utterrance mixtures . |
| Outcome: | The proposed method improves on bi-encoded utterances and even better generalization . it can encode 2020 and Erker et al. (2023) with the same results. |
Conditional Semantic Textual Similarity via Conditional Contrastive Learning (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods to assess similarity between sentences encounter over-estimation problem . compared to fuzzy representations, similarity is comparatively lower in terms of "The person's age". |
| Approach: | They propose a conditional contrastive learning framework that constructs positive and negative samples from two perspectives. |
| Outcome: | The proposed method achieves state-of-the-art performance with five models based on bi-encoder and tri-encoding architectures. |
Scalable Zero-shot Entity Linking with Dense Entity Retrieval (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for entity linking use manually curated mention tables and incoming Wikipedia link popularity. |
| Approach: | They propose a BERT-based entity linking model with a bi-encoder that embeds the mention context and the entity descriptions and then re-ranked the candidate with . they also evaluate the accuracy-speed trade-off inherent to large pre-trained models. |
| Outcome: | The proposed model is state-of-the-art on recent zero-shot benchmarks and established non-zero-shot evaluations. |
DataFinder: Scientific Dataset Recommendation from Natural Language Descriptions (2023.acl-long)
Copied to clipboard
| Challenge: | Modern machine learning relies on datasets to develop and validate research ideas. |
| Approach: | They propose a dataset recommendation system that uses a training set and an evaluation set to help people find relevant datasets. |
| Outcome: | The proposed model finds more relevant search results than existing third-party search engines. |
LogiCoL: Logically-Informed Contrastive Learning for Set-based Dense Retrieval (2025.emnlp-main)
Copied to clipboard
| Challenge: | Current dense retrievers struggle with queries with logical connectives, a use case that is often overlooked but important in downstream applications. |
| Approach: | They propose a logically-informed contrastive learning objective for dense retrievers that learns to respect the subset and mutually exclusive set relation between query results. |
| Outcome: | The proposed model improves retrieval performance and consistency on entity retrieval tasks. |
RUIE: Retrieval-based Unified Information Extraction using Large Language Model (2025.coling-main)
Copied to clipboard
| Challenge: | Unified information extraction (UIE) aims to extract diverse structured information from unstructured text using a single model or framework. |
| Approach: | They propose a framework that leverages in-context learning for efficient task generalization by combining LLM preferences with a keyword-enhanced reward model. |
| Outcome: | The proposed framework performs better on eight held-out datasets than existing methods and instruction-tuning methods. |
Learn to Not Link: Exploring NIL Prediction in Entity Linking (2023.findings-acl)
Copied to clipboard
| Challenge: | Entity linking models have been successful in capturing semantic features, but the NIL prediction problem has not been addressed. |
| Approach: | They propose an entity linking dataset that categorizes mentions linking to NIL into Missing Entity and Non-Entity Phrases. |
| Outcome: | The proposed dataset categorizes mentions linking to NIL into Missing Entity and Non-Entity Phrase categories and ensures the presence of mentions by human annotation and entity masking. |
Pseudo-Relevance for Enhancing Document Representation (2022.emnlp-main)
Copied to clipboard
| Challenge: | a novel approach to document retrieval can be used to encode documents as vectors . a few query-relevant terms can be pruned out to reduce index overhead . |
| Approach: | They propose to enhance the document representation for the bi-encoder approach in dense document retrieval. |
| Outcome: | The proposed solution reduces latency and memory footprint up to 8- and 3-fold . it is validated on MSMARCO and real-world search query logs . |
ACORD: An Expert-Annotated Retrieval Dataset for Legal Contract Drafting (2025.acl-long)
Copied to clipboard
Steven H Wang, Maksim Zubkov, Kexin Fan, Sarah Harrell, Yuyang Sun, Wei Chen, Andreas Plesner, Roger Wattenhofer
| Challenge: | Contract clause retrieval is critical to contract drafting because of its high quality and complexity. |
| Approach: | They propose the first expert-annotated benchmark specifically designed for contract clause retrieval . ACORD focuses on complex contract clauses such as Limitation of Liability, Indemnification, Change of Control . |
| Outcome: | The atticus clause retrieval dataset shows promising results but needs improvement . the benchmark can be used as an IR benchmark for the NLP community . |