Papers by Jacob Goldberger
Peek Across: Improving Multi-Document Modeling via Cross-Document Question-Answering (2023.acl-long)
Copied to clipboard
| Challenge: | Among recent NLP research, multi-document processing is gaining increasing attention due to the need to handle and process an increasing amount of textual data and available documents online. |
| Approach: | They propose to pre-train a generic multi-document model from a cross-document question answering pre-training objective by generating salient sentences from one document and challenging it to recover the sentence from which it was generated. |
| Outcome: | The proposed model outperforms zero-shot GPT-3.5 and GPT-4 in multiple document tasks and generates the correct answer and the salient sentence from a salient document. |
Multilingual word translation using auxiliary languages (D19-1)
Copied to clipboard
| Challenge: | Existing multilingual word translation methods focus on learning mappings from each language to a shared space. |
| Approach: | They propose a multilingual translation procedure that uses all the learned mappings to translate a word from one language to another. |
| Outcome: | Experiments on a standard multilingual word translation benchmark show that the proposed translation procedure outperforms state-of-the-art translation methods. |
Denoising Word Embeddings by Averaging in a Shared Space (2021.starsem-1)
Copied to clipboard
| Challenge: | Continuous word embeddings have been introduced several years ago as a standard building block for NLP tasks. |
| Approach: | They propose a method of fusing word embeddings that were trained on the same corpus but with different initializations. |
| Outcome: | The proposed method improves word embeddings on a range of tasks. |
A Multi-Pairwise Extension of Procrustes Analysis for Multilingual Word Translation (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to multilingual word embeddings require a k-way dictionary. |
| Approach: | They propose a novel approach to simultaneously representing multiple languages in a common space by using a pairwise bilingual dictionary. |
| Outcome: | The proposed approach requires only pairwise bilingual dictionaries that are much easier to construct. |
Aligning Vector-spaces with Noisy Supervised Lexicon (N19-1)
Copied to clipboard
| Challenge: | Current approaches to learning to translate between two vector spaces assume that the lexicon defines alignment pairs is noise-free. |
| Approach: | They propose a model that accounts for noisy pairs and propose supervised learning problems for this problem. |
| Outcome: | The proposed model significantly improves translation accuracy on bilingual word embedding translation and mapping between diachronic embeddable spaces. |
Conformal Nucleus Sampling (2023.findings-acl)
Copied to clipboard
| Challenge: | Modern language generation methods employ one of a handful of standard decoding strategies . greedy search and other methods generate dull text or degenerate text . |
| Approach: | They employ a calibration procedure to calibrate the parameter p as a function of the entropy of the next word distribution. |
| Outcome: | The proposed method overconfidently selects the word with the highest probability . the greedy method and its beam search variations tend to return dull text or degenerate text . |
A Locally Linear Procedure for Word Translation (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods to learn word embeddings of two languages are limited by the expressiveness of the translation model. |
| Approach: | They propose an algorithm that uses multiple orthogonal translation matrices to model the mapping and derive an algorithm to learn these multiple matric. |
| Outcome: | The proposed algorithm achieves better performance in bilingual and cross-lingual word translation tasks compared to the single matrix baseline. |
Proposition-Level Clustering for Multi-Document Summarization (2022.naacl-main)
Copied to clipboard
Ori Ernst, Avi Caciularu, Ori Shapira, Ramakanth Pasunuru, Mohit Bansal, Jacob Goldberger, Ido Dagan
| Challenge: | Existing methods focused on clustering sentences to indicate information saliency and avoid redundancy. |
| Approach: | They propose to group together sub-sentential propositions to generate a representative sentence for each cluster via text fusion. |
| Outcome: | The proposed method improves over the previous state-of-the-art method in the DUC 2004 and TAC 2011 datasets, both in automatic ROUGE scores and human preference. |
Long Context Question Answering via Supervised Contrastive Learning (2022.naacl-main)
Copied to clipboard
| Challenge: | Long-context question answering tasks often require identifying evidence spans (e.g., sentences) prior work showed that jointly training models to perform evidence extraction and question answering is important for achieving high performance. |
| Approach: | They propose a method for equipping long-context QA models with an additional sequence-level objective for better identification of the supporting evidence. |
| Outcome: | The proposed method exhibits consistent improvements on three different strong long-context transformer models, across two challenging question answering benchmarks – HotpotQA and QAsper. |
The Power of Summary-Source Alignments (2024.findings-acl)
Copied to clipboard
Ori Ernst, Ori Shapira, Aviv Slobodkin, Sharon Adar, Mohit Bansal, Jacob Goldberger, Ran Levy, Ido Dagan
| Challenge: | Multi-document summarization (MDS) is a challenging task, often decomposed to subtasks of salience and redundancy detection, followed by text generation. |
| Approach: | They propose to extend the summary-source alignment framework by applying it at the more fine-grained proposition span level and annotating alignment manually in a multi-document setup. |
| Outcome: | The proposed framework can yield several datasets for at least six different tasks. |
Self-Normalization Properties of Language Modeling (C18-1)
Copied to clipboard
| Challenge: | Existing methods to reduce run-times for language models with large word vocabularies are based on noise contrastive estimation (NCE) |
| Approach: | They propose to use noise-constrained noise-based models to approximate the normalized probability of a class without having to compute the partition function. |
| Outcome: | The proposed model outperforms softmax-based models in a variety of NLP tasks and is based on the noise-constrained noise-constant estimation properties. |