BoolQuestions: Does Dense Retrieval Understand Boolean Logic in Language? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Dense retrieval systems focus on optimizing text embedding space while overlooking Boolean logic in language. |
| Approach: | They propose a task to investigate whether retrieval systems can comprehend Boolean logic in language. |
| Outcome: | The proposed method is based on a benchmark dataset covering complex queries containing basic Boolean logic and corresponding annotated passages. |
Similar Papers
SURE or Not? Investigating Semantic Understanding in Dense Retrieval Models (2026.acl-long)
Copied to clipboard
| Challenge: | Dense retrieval models have been successful in a number of applications but it is unclear whether they truly understand semantics. |
| Approach: | They propose a benchmark for semantic understanding in dense retrieval that characterizes semantic precision, semantic abstraction and semantic equivalence along three dimensions. |
| Outcome: | The proposed model characterizes semantic understanding in dense retrieval along three dimensions: semantic precision, semantic abstraction, and semantic equivalence. |
Dense X Retrieval: What Retrieval Granularity Should We Use? (2024.emnlp-main)
Copied to clipboard
| Challenge: | a learned dense retrieval model is often overlooked when using a corpus for inference, resulting in a design choice of retrieval unit . granularity of retrievals is important for both retrieval and downstream tasks . |
| Approach: | They propose a retrieval unit for dense retrieval that uses propositions to index corpus . propositions are defined as atomic expressions within text, each encapsulating a distinct factoid . |
| Outcome: | The proposed retrieval unit outperforms passage-level units on retrieval and downstream tasks. |
Bridging the Training-Inference Gap for Dense Phrase Retrieval (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for building dense retrievers are often misaligned and do not reflect retrieval scenario at inference time. |
| Approach: | They propose a way to validate dense retrievers using a small subset of the entire corpus. |
| Outcome: | The proposed model improves top-1 phrase retrieval accuracy by 2 3 points and top-20 passage retrieval by 2 4 points for open-domain question answering. |
BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions (N19-1)
Copied to clipboard
| Challenge: | In this paper we build a reading comprehension dataset of yes/no questions that are naturally occurring . they often query for complex, non-factoid information, and require difficult entailment-like inference to solve. |
| Approach: | They build a reading comprehension dataset of yes/no questions that are naturally occurring . they find they are unexpectedly challenging and require difficult inferences to solve . |
| Outcome: | The proposed method achieves 80.4% accuracy compared to 90% accuracy of human annotators and 62% majority-baseline. |
Phrase Retrieval Learns Passage Retrieval, Too (2021.emnlp-main)
Copied to clipboard
| Challenge: | Dense retrieval methods have shown great promise over sparse methods in a range of NLP problems. |
| Approach: | They propose to use dense phrase retrieval to learn coarse-level retrieval including passages . they show phrase retrievals can be fine-tuned for more coarse-grained retrieval units . |
| Outcome: | The proposed method improves passage retrieval accuracy and QA performance with fewer passages. |
Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One? (2022.findings-emnlp)
Copied to clipboard
Xilun Chen, Kushal Lakhotia, Barlas Oguz, Anchit Gupta, Patrick Lewis, Stan Peshterliev, Yashar Mehdad, Sonal Gupta, Wen-tau Yih
| Challenge: | Existing sparse retrievers lack the ability to match salient phrases and rare entities in the query. |
| Approach: | They introduce a dense Lexical Model that can be trained to imitate a sparse one. |
| Outcome: | The proposed model outperforms sparse retrievers on a range of tasks including five question answering datasets and the MS MARCO passage retrieval. |
Simple Entity-Centric Questions Challenge Dense Retrievers (2021.emnlp-main)
Copied to clipboard
| Challenge: | Open-domain question answering has exploded in popularity due to the success of dense retrieval models. |
| Approach: | They construct a set of simple, entity-rich questions based on facts from Wikidata and test their models against supervised datasets. |
| Outcome: | The proposed model outperforms sparse retrieval methods on open-domain question answering datasets by a large margin. |
LogiCoL: Logically-Informed Contrastive Learning for Set-based Dense Retrieval (2025.emnlp-main)
Copied to clipboard
| Challenge: | Current dense retrievers struggle with queries with logical connectives, a use case that is often overlooked but important in downstream applications. |
| Approach: | They propose a logically-informed contrastive learning objective for dense retrievers that learns to respect the subset and mutually exclusive set relation between query results. |
| Outcome: | The proposed model improves retrieval performance and consistency on entity retrieval tasks. |
Open Domain Question Answering over Tables via Dense Retrieval (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent advances in open-domain QA focus on retrieving textual passages . a retriever designed to handle tabular context can improve retrieval quality . |
| Approach: | They propose a tabular-based retrieval model that improves retrieval quality over a BERT-based retriever. |
| Outcome: | The proposed retriever improves retrieval quality with mined hard negatives over a BERT-based retriever. |
Logic Haystacks: Probing LLMs’ Long-Context Logical Reasoning (Without Easily Identifiable Unrelated Padding) (2026.eacl-short)
Copied to clipboard
| Challenge: | Recent large language models claim long context windows, but evaluations often involve simple retrieval tasks or synthetic tasks padded with irrelevant text. |
| Approach: | They use grammars to generate simplified English with logical representations to create long input text while controlling its semantics. |
| Outcome: | The proposed model performs better with realistic distractors than with standard models. |