Papers by Elron Bandel
Efficient Benchmarking (of Language Models) (2024.naacl-long)
Copied to clipboard
Yotam Perlitz, Elron Bandel, Ariel Gera, Ofir Arviv, Liat Ein-Dor, Eyal Shnarch, Noam Slonim, Michal Shmueli-Scheuer, Leshem Choshen
| Challenge: | Efficient Benchmarking is a method for reducing computation costs of LM evaluation without compromising reliability. |
| Approach: | They propose to reduce the computation costs of LM evaluation without compromising reliability by using a new measure - Decision Impact on Reliability. |
| Outcome: | The proposed benchmarks reduce computation costs by x100 or more, while maintaining reliability. |
FastFit: Fast and Effective Few-Shot Text Classification with a Multitude of Classes (2024.naacl-demo)
Copied to clipboard
| Challenge: | Few-shot prompting of large language models (LLMs) via API calls presents a unique challenge when dealing with a multitude of classes that share similar semantic meanings. |
| Approach: | They present a Python package that integrates batch contrastive learning and token-level similarity score to provide fast few-shot classification. |
| Outcome: | The proposed method significantly improves multi-class classification speed and accuracy across English and Multilingual datasets. |
Unitxt: Flexible, Shareable and Reusable Data Preparation and Evaluation for Generative AI (2024.naacl-demo)
Copied to clipboard
Elron Bandel, Yotam Perlitz, Elad Venezian, Roni Friedman, Ofir Arviv, Matan Orbach, Shachar Don-Yehiya, Dafna Sheinwald, Ariel Gera, Leshem Choshen, Michal Shmueli-Scheuer, Yoav Katz
| Challenge: | Textual data processing pipelines are tailored to specific datasets, task and model combinations. |
| Approach: | They propose a library for customizable textual data preparation and evaluation tailored to generative language models. |
| Outcome: | Unitxt is a library for customizable textual data preparation and evaluation tailored to generative language models. |
AlephBERT: Language Model Pre-training and Evaluation from Sub-Word to Sentence Level (2022.acl-long)
Copied to clipboard
| Challenge: | a recent study shows that large pre-trained language models are not sufficient for Hebrew. |
| Approach: | They propose a large pre-trained language model for Hebrew that recovers morphological segments encoded in contextualized embedding vectors. |
| Outcome: | The proposed model obtains state-of-the-art on all tasks beyond contemporary Hebrew baselines. |
DOVE: A Large-Scale Multi-Dimensional Predictions Dataset Towards Meaningful LLM Evaluation (2025.findings-acl)
Copied to clipboard
Eliya Habba, Ofir Arviv, Itay Itzhak, Yotam Perlitz, Elron Bandel, Leshem Choshen, Michal Shmueli-Scheuer, Gabriel Stanovsky
| Challenge: | Recent work found that LLMs are sensitive to arbitrary prompt dimensions . this challenges traditional single-prompt evaluation practices . |
| Approach: | They present a large-scale dataset containing prompt perturbations of various evaluation benchmarks . they examine LLM sensitivity from an holistic perspective and assess the joint effects of perturbations along various dimensions . |
| Outcome: | The proposed dataset aims to democratize evaluation research and enable robust protocols . it includes more than 250M prompt perturbations and model outputs . |
Lexical Generalization Improves with Larger Models and Longer Training (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Excessive utilization of lexical overlap heuristics can lead to failure on challenging inputs. |
| Approach: | They analyze the use of lexical overlap heuristics in natural language inference, paraphrase detection, and reading comprehension using a contrastive dataset. |
| Outcome: | The proposed model can be used to improve performance on a wide range of tasks, but it is often wrong. |
Quality Controlled Paraphrase Generation (2022.acl-long)
Copied to clipboard
| Challenge: | Recent studies have shown that high quality paraphrases are difficult to generate because of their low flexibility and scalability. |
| Approach: | They propose a quality-guided controlled paraphrase generation model that allows directly controlling the quality dimensions of the generated paraphrase. |
| Outcome: | The proposed method generates paraphrases which maintain original meaning while achieving higher diversity than the uncontrolled baseline. |