Papers by Inbal Magar
Data Contamination: From Memorization to Exploitation (2022.acl-short)
Copied to clipboard
| Challenge: | Pretrained language models are typically trained on web-based datasets that are often "contaminated" with downstream test sets. |
| Approach: | They propose a method to pretrain BERT models on Wikipedia and labeled downstream datasets and fine-tune them on the relevant task. |
| Outcome: | The proposed method compares models on Wikipedia and labeled downstream datasets on two models and three downstream tasks. |
Generating Benchmarks for Factuality Evaluation of Language Models (2024.eacl-long)
Copied to clipboard
Dor Muhlgay, Ori Ram, Inbal Magar, Yoav Levine, Nir Ratner, Yonatan Belinkov, Omri Abend, Kevin Leyton-Brown, Amnon Shashua, Yoav Shoham
| Challenge: | Existing methods for factuality evaluation of LLM generation focus on facts sampled from the LM itself and might under-represent domain specific or rare facts. |
| Approach: | They propose a method that transforms a factual corpus into a benchmark evaluating an LM's propensity to generate true facts from the corpus . |
| Outcome: | The proposed framework transforms a factual corpus of interest into a benchmark evaluating an LM's propensity to generate true facts from the corpus vs. similar but incorrect statements. |
Parallel Context Windows for Large Language Models (2023.acl-long)
Copied to clipboard
Nir Ratner, Yoav Levine, Yonatan Belinkov, Ori Ram, Inbal Magar, Omri Abend, Ehud Karpas, Amnon Shashua, Kevin Leyton-Brown, Yoav Shoham
| Challenge: | Existing efforts to address context window limitation for off-the-shelf LLMs involve training specialized architectures. |
| Approach: | They propose a method that carves a long context into chunks and restricts attention to apply only within each window. |
| Outcome: | The proposed method shows significant improvements on in-context learning tasks with diverse input and output spaces. |