Papers by Amir Globerson
Incorporating Structured Representations into Pretrained Vision & Language Models Using Scene Graphs (2023.emnlp-main)
Copied to clipboard
Roei Herzig, Alon Mendelson, Leonid Karlinsky, Assaf Arbelle, Rogerio Feris, Trevor Darrell, Amir Globerson
| Challenge: | Vision and language models (VLMs) have demonstrated remarkable zero-shot (ZS) performance in a variety of tasks. |
| Approach: | They propose to integrate structured annotations into visual and textual representations to improve VLMs' understanding of compositional scenes. |
| Outcome: | The proposed method improves VLMs on multiple VL datasets with only a mild degradation in ZS capabilities. |
LM vs LM: Detecting Factual Errors via Cross Examination (2023.emnlp-main)
Copied to clipboard
| Challenge: | Modern language models (LMs) generate inconsistent, non-attributable or factually incorrect text, which hinders their usability. |
| Approach: | They propose a factuality evaluation framework for LMs that is based on cross-examination to detect inconsistencies between LM and examiner. |
| Outcome: | The proposed framework outperforms existing methods and baselines on factual claims on four benchmarks. |
Few-Shot Question Answering by Pretraining Span Selection (2021.acl-long)
Copied to clipboard
| Challenge: | Pretraining models with recurring span selection are effective, but perform poorly in a few-shot setting. |
| Approach: | They propose recurring span selection scheme that asks model to select correct span in passage with multiple sets of recurring recurrings. |
| Outcome: | The proposed model achieves 72.7 F1 on multiple benchmarks while maintaining competitive performance in the high-resource setting. |
Hopping Too Late: Exploring the Limitations of Large Language Models on Multi-Hop Queries (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) can solve complex multi-step problems, but little is known about how these computations are implemented internally. |
| Approach: | They propose a "back-patching" analysis method to solve multi-hop queries . they propose resolving the bridge entity into the bridge and the second hop into the target entity into latent steps. |
| Outcome: | The proposed method solves multi-hop queries that require two information extraction steps . it shows that the later layers lack the necessary knowledge to correctly generate the answer . |
Cross-Lingual Alignment of Contextual Word Embeddings, with Applications to Zero-shot Dependency Parsing (N19-1)
Copied to clipboard
| Challenge: | Existing methods for multilingual transfer are limited by their dynamic nature. |
| Approach: | They propose a method that utilizes deep contextual embeddings, pretrained in an unsupervised fashion. |
| Outcome: | The proposed method outperforms the state-of-the-art on 6 languages, yielding an improvement of 6.8 LAS points on average. |
Crawling The Internal Knowledge-Base of Language Models (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for representing factual knowledge in a language model are insufficient. |
| Approach: | They propose a procedure for “crawling” the internal knowledge-base of a language model by expanding a knowledge-graph around it. |
| Outcome: | The proposed method yields high precision graphs (82-92%) while emitting a reasonable number of facts per entity. |
Text-Only Training for Image Captioning using Noise-Injected CLIP (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a new approach to image captioning requires large datasets of captioned images and is difficult to collect. |
| Approach: | They propose to use a decoder to translate CLIP textual embeddings back into text . they show that this intuition is “almost correct” because of a gap between the embeddable spaces . |
| Outcome: | The proposed approach shows that the intuition is “almost correct” because of a gap between the embedding spaces, and rectifies this via noise injection during training. |
What Are You Token About? Dense Retrieval as Distributions Over the Vocabulary (2023.acl-long)
Copied to clipboard
| Challenge: | Dense retrieval models based on text representations have proven very effective, but when applied off-the-shelf they often experience a severe drop in performance. |
| Approach: | They propose to interpret the vector representations produced by dual encoders by projecting them into the model’s vocabulary space. |
| Outcome: | The proposed model significantly improves on the BEIR benchmark and in zero-shot settings. |
Coreference Resolution with Entity Equalization (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to coreference resolution capture the properties of entity clusters and use them in the resolution process. |
| Approach: | They propose an approach that captures entities and uses them in coreference resolution . they propose an "Entity Equalization" mechanism that represents each mention in a cluster . |
| Outcome: | The proposed approach improves the CoNLL-2012 coreference resolution task by 3.6%. |
Weakly Supervised Semantic Parsing with Abstract Examples (P18-1)
Copied to clipboard
| Challenge: | training semantic parsers from weak supervision complicates training in two ways . spurious programs that accidentally lead to a correct denotation add noise to training . |
| Approach: | They propose to use tokens in both language utterance and program to map denotations to executable programs. |
| Outcome: | The proposed method improves performance and reaches 82.5% accuracy compared to the best reported accuracy so far. |
Dissecting Recall of Factual Associations in Auto-Regressive Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have focused on identifying where factual knowledge is encoded in the network, but little is known about how it is extracted from the model parameters during inference. |
| Approach: | They examine how factual associations are stored and retrieved internally in LMs . they use attention edges to identify critical points where information propagates to the prediction . |
| Outcome: | The proposed model aggregates information about subject and relation to predict the correct attribute . the model “queries” the enriched subject to extract the attribute based on the proposed model . |
A Simple and Effective Model for Answering Multi-span Questions (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for reading comprehension restrict output space to a set of single contiguous spans . multi-span questions are problematic because they require multiple inputs - a task that requires a sequence tagging problem . |
| Approach: | They propose a simple architecture for answering multi-span questions by casting the task as a sequence tagging problem. |
| Outcome: | The proposed model significantly improves performance on span extraction questions from DROP and Quoref by 9.9 and 5.5 EM points respectively. |
Pre-training Mention Representations in Coreference Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to improve coreference resolution use labeled data. |
| Approach: | They propose two self-supervised tasks that are closely related to coreference resolution to improve mention representation. |
| Outcome: | The proposed models improve mention representations by learning them on a GAP dataset. |
Covering Uncommon Ground: Gap-Focused Question Generation for Answer Assessment (2023.acl-short)
Copied to clipboard
Roni Rabin, Alexandre Djerbetian, Roee Engelberg, Lidan Hackmon, Gal Elidan, Reut Tsarfaty, Amir Globerson
| Challenge: | Human communication often involves information gaps between the interlocutors. |
| Approach: | They propose a model that generates such gap-focused questions automatically . they propose an evaluation by human annotators of the generated questions . |
| Outcome: | The proposed model outperforms human generated questions in a competitive environment. |
Learning to Retrieve Passages without Supervision (2022.naacl-main)
Copied to clipboard
| Challenge: | Dense retrievers for open domain question answering have been shown to achieve impressive performance by training on large datasets of question-passage pairs. |
| Approach: | They propose to use recurring spans to create pseudo examples for contrastive learning. |
| Outcome: | The proposed model outperforms all pretrained baselines on a wide range of ODQA datasets and is competitive with BM25, a strong sparse baseline. |
In-Context Learning Creates Task Vectors (2023.findings-emnlp)
Copied to clipboard
| Challenge: | In-context learning (ICL) is a powerful new learning paradigm for Large Language Models (LLMs). |
| Approach: | They propose to use a model with a prompt and a query to learn a mapping based on two examples to produce the output. |
| Outcome: | The proposed model can learn functions from a simple structure based on a training set and a single task vector calculated from the training set. |
ConvApparel: A Benchmark Dataset and Validation Framework for User Simulators in Conversational Recommenders (2026.eacl-long)
Copied to clipboard
Ofer Meshi, Krisztian Balog, Sally Goldman, Avi Caciularu, Guy Tennenholtz, Jihwan Jeong, Amir Globerson, Craig Boutilier
| Challenge: | a "realism gap" exists between simulations and real-world user models . large language models (LLMs) are a key component of conversational AI . |
| Approach: | They propose a framework that combines statistical alignment, human-likeness score and counterfactual validation to test for generalization. |
| Outcome: | The proposed framework outperforms baselines in counterfactual validation, showing that data-driven simulators adapt more realistically to unseen behaviors. |
BERTese: Learning to Speak to BERT (2021.eacl-main)
Copied to clipboard
| Challenge: | Recent work shows that pre-trained language models encode large amounts of world knowledge in their parameters. |
| Approach: | They propose a method for automatically rewriting queries into a paraphrase query called "BERTese" they add auxiliary loss functions that encourage the query to correspond to actual language tokens . |
| Outcome: | The proposed method outperforms baselines and provides some insight into the type of language that helps language models perform knowledge extraction. |