Papers by Yaron Fairstein
Extremely efficient online query encoding for dense retrieval (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing dense retrieval systems use the same model architecture for encoding both passages and queries, even though queries are much shorter and simpler than passages. |
| Approach: | They propose a small efficient RNN query encoder that can reduce latency by 12 with only a minor decrease in quality. |
| Outcome: | The proposed solution reduces latency by up to 12 while achieving 35.5 MRR@10 score. |
ChaI-TeA: A Benchmark for Evaluating Autocompletion of Interactions with LLM-based Chatbots (2025.naacl-short)
Copied to clipboard
Shani Goren, Oren Kalinsky, Tomer Stav, Yuri Rapoport, Yaron Fairstein, Ram Yazdi, Nachshon Cohen, Alexander Libov, Guy Kushilevitz
| Challenge: | Large Language Models (LLMs) have revolutionized many NLP applications. |
| Approach: | They propose an autocomplete evaluation framework for LLM-based chatbot interactions that includes a formal definition of the task and suitable metrics. |
| Outcome: | The proposed framework evaluates 11 models on a task that performs fairly but still lacks the ranking of the generated suggestions. |
Evaluating D-MERIT of Partial-annotation on Information Retrieval (2024.emnlp-main)
Copied to clipboard
Royi Rassin, Yaron Fairstein, Oren Kalinsky, Guy Kushilevitz, Nachshon Cohen, Alexander Libov, Yoav Goldberg
| Challenge: | Using partially-annotated datasets for evaluation can lead to false conclusions . a dataset containing only a subset of relevant passages might result in misleading rankings . |
| Approach: | They propose to use a Wikipedia passage retrieval evaluation set to contain all relevant passages for each query. |
| Outcome: | The proposed dataset can be downloaded from https://d-merit.github.io. |