Papers by Dor Muhlgay
Value-based Search in Execution Space for Mapping Instructions to Programs (N19-1)
Copied to clipboard
| Challenge: | Existing methods to map instructions to programs require searching for good programs at training time. |
| Approach: | They propose a search algorithm that uses the target world state to train a critic network that predicts the expected reward of every search state. |
| Outcome: | The proposed algorithm significantly improves on all three domains compared to baselines on the SCONE dataset. |
Generating Benchmarks for Factuality Evaluation of Language Models (2024.eacl-long)
Copied to clipboard
Dor Muhlgay, Ori Ram, Inbal Magar, Yoav Levine, Nir Ratner, Yonatan Belinkov, Omri Abend, Kevin Leyton-Brown, Amnon Shashua, Yoav Shoham
| Challenge: | Existing methods for factuality evaluation of LLM generation focus on facts sampled from the LM itself and might under-represent domain specific or rare facts. |
| Approach: | They propose a method that transforms a factual corpus into a benchmark evaluating an LM's propensity to generate true facts from the corpus . |
| Outcome: | The proposed framework transforms a factual corpus of interest into a benchmark evaluating an LM's propensity to generate true facts from the corpus vs. similar but incorrect statements. |
In-Context Retrieval-Augmented Language Models (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing RALM methods focus on modifying the LM architecture to facilitate incorporation of external information, complicating deployment. |
| Approach: | They propose to condition a language model on relevant documents from a grounding corpus during generation by conditioning on external knowledge sources. |
| Outcome: | The proposed method significantly improves language modeling performance and provides natural source attribution mechanism. |