Papers by Guy Kushilevitz
Extremely efficient online query encoding for dense retrieval (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing dense retrieval systems use the same model architecture for encoding both passages and queries, even though queries are much shorter and simpler than passages. |
| Approach: | They propose a small efficient RNN query encoder that can reduce latency by 12 with only a minor decrease in quality. |
| Outcome: | The proposed solution reduces latency by up to 12 while achieving 35.5 MRR@10 score. |
Simple and Effective Multi-Token Completion from Masked Language Models (2023.findings-eacl)
Copied to clipboard
| Challenge: | Pre-trained neural masked language models are limited to predicting a single token . recent pre-tried LMs like T5 do allow predicting multi-token completions, but are more expensive to train and run. |
| Approach: | They propose two ways to adapt pre-trained masked language models to produce multi-token completions. |
| Outcome: | The proposed method surpasses current state-of-the-art models while being more parameter efficient. |
ChaI-TeA: A Benchmark for Evaluating Autocompletion of Interactions with LLM-based Chatbots (2025.naacl-short)
Copied to clipboard
Shani Goren, Oren Kalinsky, Tomer Stav, Yuri Rapoport, Yaron Fairstein, Ram Yazdi, Nachshon Cohen, Alexander Libov, Guy Kushilevitz
| Challenge: | Large Language Models (LLMs) have revolutionized many NLP applications. |
| Approach: | They propose an autocomplete evaluation framework for LLM-based chatbot interactions that includes a formal definition of the task and suitable metrics. |
| Outcome: | The proposed framework evaluates 11 models on a task that performs fairly but still lacks the ranking of the generated suggestions. |
Evaluating D-MERIT of Partial-annotation on Information Retrieval (2024.emnlp-main)
Copied to clipboard
Royi Rassin, Yaron Fairstein, Oren Kalinsky, Guy Kushilevitz, Nachshon Cohen, Alexander Libov, Yoav Goldberg
| Challenge: | Using partially-annotated datasets for evaluation can lead to false conclusions . a dataset containing only a subset of relevant passages might result in misleading rankings . |
| Approach: | They propose to use a Wikipedia passage retrieval evaluation set to contain all relevant passages for each query. |
| Outcome: | The proposed dataset can be downloaded from https://d-merit.github.io. |
A Two-Stage Masked LM Method for Term Set Expansion (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for Term Set Expansion are either distributional or pattern-based . Term set expansion is a task of expanding a small seed set of example terms into a larger set of terms that belong to the same semantic category. |
| Approach: | They propose a method which uses neural masked language models to expand a small seed set of terms into a larger set of semantic terms. |
| Outcome: | The proposed method outperforms state-of-the-art methods due to the small seed set size . it uses neural masked language models to query large, pre-trained mlms . |