Papers by Jinhyuk Lee
When Should Dense Retrievers Be Updated in Evolving Corpora? Detecting Out-of-Distribution Corpora Using GradNormIR (2025.findings-acl)
Copied to clipboard
Dayoon Ko, Jinyoung Kim, Sohyeon Kim, Jinhyuk Kim, Jaehoon Lee, Seonghak Song, Minyoung Lee, Gunhee Kim
| Challenge: | Dense retrievers encode text into embeddings to retrieve relevant documents . however, real-world corpora evolve, resulting in degraded retrieval performance . identifying when a dense retriever requires an update is critical for robust retrieval systems . |
| Approach: | They propose a task of predicting whether a corpus is out-of-distribution (OOD) relative to a dense retriever before indexing. |
| Outcome: | The proposed method detects whether a corpus is out-of-distribution (OOD) relative to a dense retriever before indexing. |
Ranking Paragraphs for Improving Answer Recall in Open-Domain Question Answering (D18-1)
Copied to clipboard
| Challenge: | Recent work has combined open-domain question answering with machine comprehension models to find answers in a large knowledge source. |
| Approach: | They propose a machine comprehension model that ranks paragraphs of retrieved documents for a higher answer recall with less noise. |
| Outcome: | The proposed model improves on four open-domain QA datasets by 7.8% on average. |
Learning Dense Representations of Phrases at Scale (2021.acl-long)
Copied to clipboard
| Challenge: | Existing phrase retrieval models rely on sparse representations and still underperform retriever-reader approaches. |
| Approach: | They propose a method to learn phrase representations from reading comprehension tasks using negative sampling methods. |
| Outcome: | The proposed model improves over previous models by 15%-25% absolute accuracy and matches the performance of state-of-the-art retrieval models. |
Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Index (P19-1)
Copied to clipboard
| Challenge: | Existing open-domain question answering models require multiple documents on-demand for every input query. |
| Approach: | They propose query-agnostic indexable representations of document phrases that can drastically speed up open-domain question answering. |
| Outcome: | The proposed model can be trained and deployed even in a single 4-GPU server. |
Simple Entity-Centric Questions Challenge Dense Retrievers (2021.emnlp-main)
Copied to clipboard
| Challenge: | Open-domain question answering has exploded in popularity due to the success of dense retrieval models. |
| Approach: | They construct a set of simple, entity-rich questions based on facts from Wikidata and test their models against supervised datasets. |
| Outcome: | The proposed model outperforms sparse retrieval methods on open-domain question answering datasets by a large margin. |
LOFT: Scalable and More Realistic Long-Context Evaluation (2025.findings-naacl)
Copied to clipboard
Jinhyuk Lee, Anthony Chen, Zhuyun Dai, Dheeru Dua, Devendra Singh Sachan, Michael Boratko, Yi Luan, Séb Arnold, Vincent Perot, Siddharth Dalmia, Hexiang Hu, Xudong Lin, Panupong Pasupat, Aida Amini, Jeremy R. Cole, Sebastian Riedel, Iftekhar Naim, Ming-Wei Chang, Kelvin Guu
| Challenge: | Long-context language models (LCLMs) can be used to perform tasks traditionally reliant on external tools like retrieval systems or databases. |
| Approach: | They propose a benchmark to evaluate LCLMs' performance on in-context retrieval and reasoning tasks using a set of tokens. |
| Outcome: | The proposed model outperforms state-of-the-art retrieval and RAG systems on in-context retrieval tasks while still requiring prompting strategies. |
Bridging the Training-Inference Gap for Dense Phrase Retrieval (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for building dense retrievers are often misaligned and do not reflect retrieval scenario at inference time. |
| Approach: | They propose a way to validate dense retrievers using a small subset of the entire corpus. |
| Outcome: | The proposed model improves top-1 phrase retrieval accuracy by 2 3 points and top-20 passage retrieval by 2 4 points for open-domain question answering. |
Adversarial Subword Regularization for Robust Neural Machine Translation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for segmenting words into subword units are not robust enough to handle multiple subword candidates. |
| Approach: | They propose to regularize subword segmentations that maximize the translation loss by using gradient signals during training to prevent erroneous segmentations of unseen words. |
| Outcome: | The proposed method improves the performance of NMT models on low-resource and out-domain datasets. |
Simple Questions Generate Named Entity Recognition Datasets (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent named entity recognition models rely on human-annotated datasets . however, in-domain dictionaries and sentences are often unavailable or expensive to construct for many entity types. |
| Approach: | They propose an ask-to-generate approach which automatically generates NER datasets by asking natural language questions to an open-domain question answering system. |
| Outcome: | The proposed model outperforms the previous best model by 19.5 F1 score on six benchmarks and achieves state-of-the-art performance. |
Phrase Retrieval Learns Passage Retrieval, Too (2021.emnlp-main)
Copied to clipboard
| Challenge: | Dense retrieval methods have shown great promise over sparse methods in a range of NLP problems. |
| Approach: | They propose to use dense phrase retrieval to learn coarse-level retrieval including passages . they show phrase retrievals can be fine-tuned for more coarse-grained retrieval units . |
| Outcome: | The proposed method improves passage retrieval accuracy and QA performance with fewer passages. |
Contextualized Sparse Representations for Real-Time Open-Domain Question Answering (2020.acl-main)
Copied to clipboard
| Challenge: | Existing phrase retrieval models suffer from low accuracy due to limited scalability and speed . 'Open-domain question answering' is a task of answering generic factoid questions by looking up a large knowledge source, typically unstructured text corpora such as Wikipedia. |
| Approach: | They aim to augment existing phrase retrieval models with contextualized sparse representations to improve the quality of each phrase embedding. |
| Outcome: | The proposed model improves CuratedTREC and SQuAD-Open by 4% and 45x faster inference speeds over the existing model. |
Can Language Models be Biomedical Knowledge Bases? (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have focused on probing LMs in the general domain but little attention has been given to whether they can be used as domain knowledge bases. |
| Approach: | They propose to use 49K biomedical factual knowledge triples to probe LMs for biomedically . they find that biomedic LM can achieve up to 18.51% Acc@5 on retrieving biomedcial knowledge. |
| Outcome: | The proposed biomedical factual knowledge probing benchmark achieves 18.51% Acc@5 on biomedically-relevant knowledge retrieval. |
Optimizing Test-Time Query Representations for Dense Retrieval (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent developments of dense retrieval rely on quality representations of queries and contexts from pre-trained query and context encoders. |
| Approach: | They propose a test-time optimization of query representations that provides fine-grained pseudo labels over retrieval results. |
| Outcome: | The proposed algorithm improves open-domain question answering accuracy and direct re-ranking by up to 2.0% while running 1.3–2.4x faster with an efficient implementation. |
Biomedical Entity Representations with Synonym Marginalization (2020.acl-main)
Copied to clipboard
| Challenge: | Biomedical named entities are often used as key features in biomedical text mining. |
| Approach: | They propose to use a model-based candidate selection to maximize the marginal likelihood of the synonyms present in top candidates. |
| Outcome: | The proposed model outperforms previous state-of-the-art models on four biomedical entity normalization datasets with three different entity types. |
Look at the First Sentence: Position Bias in Question Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | Extractive question answering models are trained to predict start and end positions of answers . recent QA models outperform humans in some datasets due to their simplicity and effectiveness. |
| Approach: | They propose to use prior distribution of answer positions as a bias model to reduce position bias. |
| Outcome: | The proposed model outperforms BERT from 37.48% to 81.64% when trained on a biased SQUAD dataset. |