Papers by Idan Szpektor
Generating Coherent Sequences of Visual Illustrations for Real-World Manual Tasks (2024.acl-long)
Copied to clipboard
João Bordalo, Vasco Ramos, Rodrigo Valério, Diogo Glória-Silva, Yonatan Bitton, Michal Yarom, Idan Szpektor, Joao Magalhaes
| Challenge: | Large Vision/Language Models (LVLMs) are less capable of generating accompanying image sequences. |
| Approach: | They propose a method that integrates a Latent Diffusion Model (LDM) with an LLM to generate captions to maintain semantic coherence of the sequence. |
| Outcome: | The proposed method is preferred by humans in 46.6% of the cases against 26.6% for the second best method. |
A Joint Named-Entity Recognizer for Heterogeneous Tag-sets Using a Tag Hierarchy (P19-1)
Copied to clipboard
| Challenge: | Named-entity recognition (NER) has seen significant progress with the application of Neural Networks to the task. |
| Approach: | They propose to use a given tag hierarchy to jointly learn a neural network that shares its tagging layer among all tag-sets. |
| Outcome: | The proposed model outperforms models that combine independent models and multitasking approaches in a domain adaptation for named-entity recognition task. |
MaXM: Towards Multilingual Visual Question Answering (2023.findings-emnlp)
Copied to clipboard
Soravit Changpinyo, Linting Xue, Michal Yarom, Ashish Thapliyal, Idan Szpektor, Julien Amelot, Xi Chen, Radu Soricut
| Challenge: | Visual Question Answering (VQA) has been studied in the English language, but in other languages it would require a considerable amount of resources. |
| Approach: | They propose scalable solutions to multilingual visual question answering using an English language framework and an annotation protocol. |
| Outcome: | The proposed framework reduces human annotation efforts and creates a test-only VQA benchmark in 7 languages. |
DisentQA: Disentangling Parametric and Contextual Knowledge with Counterfactual Question Answering (2023.acl-long)
Copied to clipboard
| Challenge: | Question answering models have access to two sources of knowledge during inference time: parametric knowledge and contextual knowledge. |
| Approach: | They propose a new paradigm in which QA models are trained to disentangle the two sources of knowledge. |
| Outcome: | The proposed model generates two answers for a given question based on parametric and contextual knowledge. |
Multilingual Sequence-to-Sequence Models for Hebrew NLP (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work on pretrained language models for Hebrew is under-parameterized and under-trained . previous work on pretraining Hebrew LMs focused on encoder-only architectures . |
| Approach: | They propose to use sequence-to-sequence generative architectures to train large LMs in morphologically rich languages such as Hebrew. |
| Outcome: | The proposed model improves on all existing Hebrew NLP benchmarks. |
Bridging the Visual Gap: Fine-Tuning Multimodal Models with Knowledge-Adapted Captions (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent work focuses on training vision-language models with long, detailed image captions, but small-scale VLMs struggle to balance the richness of these captions with the risk of hallucinations. |
| Approach: | They propose an evaluation framework that breaks down generated captions into individual propositions, assessing each in isolation. |
| Outcome: | The proposed framework outperforms baselines in both automatic metrics and human evaluations on small-scale vision-language models with long, detailed captions. |
Q2: Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answering (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluation methods for factual consistency in knowledge-grounded dialogues are unreliable and limit their applicability. |
| Approach: | They propose an automatic evaluation metric for factual consistency in knowledge-grounded dialogue using automatic question generation and question answering. |
| Outcome: | The proposed evaluation metric consistently shows higher correlation with human judgements. |
Factually Consistent Summarization via Reinforcement Learning with Textual Entailment Feedback (2023.acl-long)
Copied to clipboard
Paul Roit, Johan Ferret, Lior Shani, Roee Aharoni, Geoffrey Cideron, Robert Dadashi, Matthieu Geist, Sertan Girgin, Leonard Hussenot, Orgad Keller, Nikola Momchev, Sabela Ramos Garea, Piotr Stanczyk, Nino Vieillard, Olivier Bachem, Gal Elidan, Avinatan Hassidim, Olivier Pietquin, Idan Szpektor
| Challenge: | Recent advances in abstractive summarization systems produce factually inconsistent text . this is emphasized in tasks like summarizing, which often produce inconsistent text with no input article . |
| Approach: | They use reinforcement learning to optimize for factual consistency and explore trade-offs . they use textual-entailment rewards to optimize the accuracy of the generated summaries . |
| Outcome: | The proposed method improves faithfulness, salience and conciseness of the generated summaries. |
All You May Need for VQA are Image Captions (2022.naacl-main)
Copied to clipboard
| Challenge: | Visual Question Answering (VQA) has benefited from increasingly sophisticated models, but has not enjoyed the same level of engagement in terms of data creation. |
| Approach: | They propose a method that automatically derives VQA examples at volume by leveraging existing image-caption annotations combined with neural models for textual question generation. |
| Outcome: | The proposed method improves state-of-the-art zero-shot accuracy by double digits and achieves robustness that lacks in the same model trained on human-annotated VQA data. |
Audio De-identification - a New Entity Recognition Task (N19-2)
Copied to clipboard
Ido Cohn, Itay Laish, Genady Beryozkin, Gang Li, Izhak Shafran, Idan Szpektor, Tzvika Hartman, Avinatan Hassidim, Yossi Matias
| Challenge: | Named Entity Recognition (NER) is an important step in de-identification (de-ID) of medical records, many of which are recorded conversations between a patient and a doctor. |
| Approach: | They propose to use Named Entity Recognition (NER) to detect audio spans with entity mentions in medical records and then use it to evaluate the results. |
| Outcome: | The proposed pipeline is based on a large labeled segment of the Switchboard and Fisher audio datasets and compares it with a benchmark. |
On the Robustness of Dialogue History Representation in Conversational Question Answering: A Comprehensive Study and a New Prompt-based Method (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing models show impressive results on a common CQA benchmark, but are they robust to domain, setting and domain? |
| Approach: | They propose a prompt-based history modeling approach that adds textual prompts directly to the text of a passage. |
| Outcome: | The proposed model is simple, easy to plug into practically any model and highly effective. |
DiscoFuse: A Large-Scale Dataset for Discourse-Based Sentence Fusion (N19-1)
Copied to clipboard
| Challenge: | Existing datasets for sentence fusion are small and insufficient for training modern neural models. |
| Approach: | They propose a method for automatically-generating fusion examples from raw text . they apply their method to Wikipedia and Sports articles to generate fusion models . |
| Outcome: | The proposed method improves performance on WebSplit when viewed as a sentence fusion task. |
MetaFaith: Faithful Natural Language Uncertainty Expression in LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for faithful calibration of large language models (LLMs) are insufficient and can harm faithful calibration. |
| Approach: | They propose a new prompt-based calibration approach inspired by human metacognition that measures faithfulness across diverse models and task domains and enables up to 61% improvement in faithfulness. |
| Outcome: | The proposed approach improves faithfulness across diverse models and task domains and achieves an 83% win rate over original generations as judged by humans. |
Are LLMs Better than Reported? Detecting Label Errors and Mitigating Their Effect on Model Performance (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) offer new opportunities to enhance the annotation process, particularly for detecting label errors in existing datasets. |
| Approach: | They propose to use an ensemble of large language models to flag mislabeled examples by using an LLM-as-a-judge approach to detect label errors in existing datasets. |
| Outcome: | The proposed method improves label accuracy and consistency in large language models. |
TRUE: Re-evaluating Factual Consistency Evaluation (2022.naacl-main)
Copied to clipboard
Or Honovich, Roee Aharoni, Jonathan Herzig, Hagai Taitelbaum, Doron Kukliansy, Vered Cohen, Thomas Scialom, Idan Szpektor, Avinatan Hassidim, Yossi Matias
| Challenge: | Grounded text generation systems often generate factual inconsistencies, hindering their real-world applicability. |
| Approach: | They propose a method to assess factual consistency metrics on standardized texts . they recommend NLI and question generation-and-answering-based methods as starting points . |
| Outcome: | The proposed method is more actionable and interpretable than previous methods. |
Localizing Factual Inconsistencies in Attributable Text Generation (2026.tacl-1)
Copied to clipboard
Arie Cattan, Paul Roit, Shiyue Zhang, David Wan, Roee Aharoni, Idan Szpektor, Mohit Bansal, Ido Dagan
| Challenge: | Existing methods for detecting hallucinations in model-generated texts fail to pinpoint errors. |
| Approach: | They propose a formalism for localizing factual inconsistencies in attributable text generation . they propose to decompose the generated text into simple question-answer pairs . |
| Outcome: | The proposed method achieves substantial inter-annotator agreement while achieving a substantial consistency score. |
Location Not Found: Exposing Implicit Local and Global Biases in Multilingual LLMs (2026.acl-long)
Copied to clipboard
Guy Mor-Lan, Omer Goldman, Matan Eyal, Adi Mayrav Gilady, Sivan Eiger, Idan Szpektor, Avinatan Hassidim, Yossi Matias, Reut Tsarfaty
| Challenge: | Multilingual large language models have minimized the fluency gap between languages, but they are exposed to the risk of biases as knowledge and norms may propagate across languages. |
| Approach: | They propose a test set with 2,156 questions in 12 languages to quantify models' biases . they show a global bias towards answers relevant to the US-locale . |
| Outcome: | The proposed model can answer locale-ambiguous questions in 12 languages. |
MDCure: A Scalable Pipeline for Multi-Document Instruction-Following (2025.acl-long)
Copied to clipboard
| Challenge: | Multi-document (MD) processing is crucial for LLMs to handle real-world tasks such as summarization and question-answering across large sets of documents. |
| Approach: | They propose a framework that generates high-quality synthetic MD instruction data over sets of articles via targeted prompts. |
| Outcome: | MDCure generates high-quality synthetic MD instruction data over sets of articles . evaluations show it improves over pre-trained models by up to 75.1% . |
TrueTeacher: Learning Factual Consistency Evaluation with Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for evaluating factual consistency are limited in their effectiveness. |
| Approach: | They propose a method for generating synthetic data by annotating diverse model-generated summaries using a LLM. |
| Outcome: | The proposed method outperforms state-of-the-art models and the LLM teacher on TRUE benchmarks. |
Semantically Driven Sentence Fusion: Modeling and Evaluation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Sentence fusion is the task of joining related sentences into coherent text. |
| Approach: | They propose a method where ground-truth solutions are automatically expanded into multiple references via curated equivalence classes of connective phrases. |
| Outcome: | The proposed approach improves on state-of-the-art models by expanding ground-truth solutions into multiple references. |
Unpacking Tokenization: Evaluating Text Compression and its Correlation with Model Performance (2024.findings-acl)
Copied to clipboard
| Challenge: | Despite being the cornerstone of BPE, the importance of compression in the tokenization process is still unclear. |
| Approach: | They argue for the theoretical importance of compression in the tokenization process . they also demonstrate the empirical importance of compressing tokenizers for downstream success of pre-trained language models. |
| Outcome: | The proposed method can be viewed as 0-gram language modeling where equal probability is assigned to all tokens. |
RefVNLI: Towards Scalable Evaluation of Subject-driven Text-to-image Generation (2025.findings-emnlp)
Copied to clipboard
Aviv Slobodkin, Hagai Taitelbaum, Yonatan Bitton, Brian Gordon, Michal Sokolik, Nitzan Bitton Guetta, Almog Gueta, Royi Rassin, Dani Lischinski, Idan Szpektor
| Challenge: | Existing methods assess only one aspect of the task, misalign with human judgments or rely on costly API-based evaluation. |
| Approach: | RefVNLI evaluates textual alignment and subject preservation in a single run. |
| Outcome: | RefVNLI outperforms or matches existing baselines across multiple benchmarks and subject categories. |
Multilingual Instruction Tuning With Just a Pinch of Multilinguality (2024.findings-acl)
Copied to clipboard
| Challenge: | Using multilingual instruction tuning, large language models can be used to follow instructions in multiple languages . a multilingual model can be tuned on a wide range of languages, yet most datasets are limited to English . |
| Approach: | They investigate how multilinguality during instruction tuning affects instruction-following across languages . they find that only 40 multilingual examples improve multilingual instruction- follow . |
| Outcome: | The results show that multilingual models perform better on multilingual mixtures compared to monolingual models . the results suggest that building multilingual instruction-tuned models can be done with only 2-4 languages . |