Papers by Avi Caciularu
Optimizing Retrieval-augmented Reader Models via Token Elimination (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for ODQA use a retrieval-augmented language model . a generative model can cause a significant bottleneck in decoding time . |
| Approach: | They propose to eliminate some of the retrieved information that might not contribute essential information to the answer generation process. |
| Outcome: | The proposed method reduces run-time by up to 62.2% with only 2% reduction in performance and improves performance. |
Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarks (2023.emnlp-main)
Copied to clipboard
| Challenge: | Common NLP models are trained on data crawled from the internet, and it is difficult to audit at scale. |
| Approach: | They propose three strategies to prevent data contamination by encrypting test data and preventing it from being released on the internet. |
| Outcome: | The proposed strategies can make a difference in preventing data contamination. |
Peek Across: Improving Multi-Document Modeling via Cross-Document Question-Answering (2023.acl-long)
Copied to clipboard
| Challenge: | Among recent NLP research, multi-document processing is gaining increasing attention due to the need to handle and process an increasing amount of textual data and available documents online. |
| Approach: | They propose to pre-train a generic multi-document model from a cross-document question answering pre-training objective by generating salient sentences from one document and challenging it to recover the sentence from which it was generated. |
| Outcome: | The proposed model outperforms zero-shot GPT-3.5 and GPT-4 in multiple document tasks and generates the correct answer and the salient sentence from a salient document. |
CDLM: Cross-Document Language Modeling (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing language models (LMs) provide powerful representations for internal text structure, but there are important applications for multi-text tasks. |
| Approach: | They propose a pretraining approach that incorporates two key ideas into the masked language modeling objective. |
| Outcome: | The proposed model improves over existing models and sets of long-range transformers and can be easily applied to multiple multi-text tasks. |
QASem Parsing: Text-to-text Modeling of QA-based Semantics (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing work suggests the appeals of incorporating explicit semantic representations into NLP . semi-structured natural language structures provide an intermediate meaning-capturing representation . |
| Approach: | They propose a semi-structured natural-language representation of textual information . they examine input and output linearization strategies and multitask learning . |
| Outcome: | The proposed model is based on pre-trained sequence-to-sequence language models . it is easy to use and can be used for downstream tasks that benefit from it . |
LM-Debugger: An Interactive Tool for Inspection and Intervention in Transformer-Based Language Models (2022.emnlp-demos)
Copied to clipboard
| Challenge: | Transformer-based language models (LMs) are opaque and unexplained, causing problems for endusers and developers who wish to debug or fix their behaviour. |
| Approach: | They propose an interactive debugger tool for transformer-based LMs that provides a fine-grained interpretation of the model's internal prediction process and a powerful framework for intervening in LM behavior. |
| Outcome: | The proposed tool provides a fine-grained interpretation of the model's internal prediction construction process, and a powerful framework for intervening in LM behavior. |
Bayesian Hierarchical Words Representation Learning (2020.acl-main)
Copied to clipboard
| Challenge: | Using hierarchical priors, BHWR improves word representations by incorporating word semantic taxonomy. |
| Approach: | They propose a Bayesian Hierarchical Words Representation (BHWR) learning algorithm that integrates hierarchical priors and word semantic taxonomy to improve representations. |
| Outcome: | The proposed model performs better on rare words and on linguistic datasets than other methods. |
ReliableEval: A Recipe for Stochastic LLM Evaluation via Method of Moments (2025.findings-emnlp)
Copied to clipboard
| Challenge: | LLMs are highly sensitive to prompt phrasing, yet standard benchmarks typically report performance using a single prompt, raising concerns about the reliability of such evaluations. |
| Approach: | They propose a stochastic method of moments evaluation over the space of meaning-preserving prompt perturbations and propose resamplings to estimate the number of prompt re-sampleds needed to obtain meaningful results. |
| Outcome: | The proposed method is model-, task-, and metric-agnostic, offering a recipe for meaningful and robust evaluation. |
The Curious Case of Hallucinatory (Un)answerability: Finding Truths in the Hidden States of Over-Confident Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have been shown to possess impressive capabilities, but they are not problem-free. |
| Approach: | They explore the behavior of large language models when presented with (un)answerable queries. |
| Outcome: | The proposed models encode the answerability of an input query, the authors show . they also show that the first decoded token is a strong indicator . |
Denoising Word Embeddings by Averaging in a Shared Space (2021.starsem-1)
Copied to clipboard
| Challenge: | Continuous word embeddings have been introduced several years ago as a standard building block for NLP tasks. |
| Approach: | They propose a method of fusing word embeddings that were trained on the same corpus but with different initializations. |
| Outcome: | The proposed method improves word embeddings on a range of tasks. |
Revisiting Sentence Union Generation as a Testbed for Text Consolidation (2023.findings-acl)
Copied to clipboard
| Challenge: | In order to acquire knowledge on a new subject, it is often necessary to consult multiple sources of written information. |
| Approach: | They propose to revisit the sentence union generation task as an effective well-defined testbed for assessing text consolidation capabilities. |
| Outcome: | The proposed evaluation protocol includes human and automatic evaluations. |
Self-Supervised Document Similarity Ranking via Contextualized Language Models and Hierarchical Inference (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to document-to-document similarity ranking are limited to relatively short documents or lack similarity labels. |
| Approach: | They propose a self-supervised method for document similarity ranking that can be applied to documents of arbitrary length. |
| Outcome: | The proposed model outperforms existing methods on large documents datasets. |
Cross-document Event Coreference Search: Task, Dataset and Modeling (2022.emnlp-main)
Copied to clipboard
| Challenge: | Cross-document Event Coreference resolution is the task of identifying clusters of text mentions that refer to the same event, whether within a single document or across a document collection. |
| Approach: | They propose a cross-document coreference search task that searches for all coreferring mentions for a query event in a large document collection. |
| Outcome: | The proposed model integrates a powerful coreference scoring scheme into the DPR architecture, yielding improved performance. |
Within-Between Lexical Relation Classification (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for recognizing lexical-semantic relations between words are path-based and distributional. |
| Approach: | They propose a novel Within-Between Relation model for recognizing lexical-semantic relations between words. |
| Outcome: | The proposed model outperforms baselines across various benchmarks and is competitive and competitive. |
A Comprehensive Evaluation of Tool-Assisted Generation Strategies (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Various few-shot tool-usage strategies have been proposed to overcome LMs' shortcomings. |
| Approach: | They propose to augment language models with tools to overcome their shortcomings . they find strong no-tool baselines are competitive to tool-assisted strategies . |
| Outcome: | The proposed strategies outperform those that refine incorrect outputs with tools in knowledge-retrieval tasks, the study finds . the findings suggest few-shot tool integration is still an open challenge . |
RecoBERT: A Catalog Language Model for Text-Based Recommendations (2020.findings-emnlp)
Copied to clipboard
| Challenge: | RecoBERT is a BERT-based approach for learning catalog-specialized language models for text-based item recommendations. |
| Approach: | They propose a BERT-based approach for learning catalog-specialized language models for text-based item recommendations that incorporates four scores during inference. |
| Outcome: | The proposed model can infer item-to-item similarities more accurately than other methods. |
Don’t Add, don’t Miss: Effective Content Preserving Generation from Pre-Selected Text Spans (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing CTR models are mediocre and lack reliable performance . authors propose an explicit decomposition of these two subtasks into a single task . |
| Approach: | They propose an isolated task that challenges models to generate coherent text conforming to pre-selected content within the input text ("highlights") authors propose a high-quality, open-source CTR model that tackles two prior key limitations: inadequate enforcement of the content-preservation constraint, and suboptimal silver training data. |
| Outcome: | The proposed model significantly improves silver training data quality over the existing model, with up to 30 ROUGE-L points. |
MetaFaith: Faithful Natural Language Uncertainty Expression in LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for faithful calibration of large language models (LLMs) are insufficient and can harm faithful calibration. |
| Approach: | They propose a new prompt-based calibration approach inspired by human metacognition that measures faithfulness across diverse models and task domains and enables up to 61% improvement in faithfulness. |
| Outcome: | The proposed approach improves faithfulness across diverse models and task domains and achieves an 83% win rate over original generations as judged by humans. |
Proposition-Level Clustering for Multi-Document Summarization (2022.naacl-main)
Copied to clipboard
Ori Ernst, Avi Caciularu, Ori Shapira, Ramakanth Pasunuru, Mohit Bansal, Jacob Goldberger, Ido Dagan
| Challenge: | Existing methods focused on clustering sentences to indicate information saliency and avoid redundancy. |
| Approach: | They propose to group together sub-sentential propositions to generate a representative sentence for each cluster via text fusion. |
| Outcome: | The proposed method improves over the previous state-of-the-art method in the DUC 2004 and TAC 2011 datasets, both in automatic ROUGE scores and human preference. |
Long Context Question Answering via Supervised Contrastive Learning (2022.naacl-main)
Copied to clipboard
| Challenge: | Long-context question answering tasks often require identifying evidence spans (e.g., sentences) prior work showed that jointly training models to perform evidence extraction and question answering is important for achieving high performance. |
| Approach: | They propose a method for equipping long-context QA models with an additional sequence-level objective for better identification of the supporting evidence. |
| Outcome: | The proposed method exhibits consistent improvements on three different strong long-context transformer models, across two challenging question answering benchmarks – HotpotQA and QAsper. |
MDCure: A Scalable Pipeline for Multi-Document Instruction-Following (2025.acl-long)
Copied to clipboard
| Challenge: | Multi-document (MD) processing is crucial for LLMs to handle real-world tasks such as summarization and question-answering across large sets of documents. |
| Approach: | They propose a framework that generates high-quality synthetic MD instruction data over sets of articles via targeted prompts. |
| Outcome: | MDCure generates high-quality synthetic MD instruction data over sets of articles . evaluations show it improves over pre-trained models by up to 75.1% . |
ConvApparel: A Benchmark Dataset and Validation Framework for User Simulators in Conversational Recommenders (2026.eacl-long)
Copied to clipboard
Ofer Meshi, Krisztian Balog, Sally Goldman, Avi Caciularu, Guy Tennenholtz, Jihwan Jeong, Amir Globerson, Craig Boutilier
| Challenge: | a "realism gap" exists between simulations and real-world user models . large language models (LLMs) are a key component of conversational AI . |
| Approach: | They propose a framework that combines statistical alignment, human-likeness score and counterfactual validation to test for generalization. |
| Outcome: | The proposed framework outperforms baselines in counterfactual validation, showing that data-driven simulators adapt more realistically to unseen behaviors. |
iFacetSum: Coreference-based Interactive Faceted Summarization for Multi-Document Exploration (2021.emnlp-demo)
Copied to clipboard
Eran Hirsch, Alon Eirew, Ori Shapira, Avi Caciularu, Arie Cattan, Ori Ernst, Ramakanth Pasunuru, Hadar Ronen, Mohit Bansal, Ido Dagan
| Challenge: | iFS provides a faceted navigation scheme that provides abstractive summaries for the user’s selections. |
| Approach: | They propose a web application that integrates interactive summarization and faceted search to provide a faceted navigation scheme that yields abstractive summaries for the user's selections. |
| Outcome: | The proposed system provides a comprehensive overview as well as particular details regard-ing subtopics of interest. |
Unpacking Tokenization: Evaluating Text Compression and its Correlation with Model Performance (2024.findings-acl)
Copied to clipboard
| Challenge: | Despite being the cornerstone of BPE, the importance of compression in the tokenization process is still unclear. |
| Approach: | They argue for the theoretical importance of compression in the tokenization process . they also demonstrate the empirical importance of compressing tokenizers for downstream success of pre-trained language models. |
| Outcome: | The proposed method can be viewed as 0-gram language modeling where equal probability is assigned to all tokens. |
Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space (2022.emnlp-main)
Copied to clipboard
| Challenge: | Fig. 1 shows how feed-forward network (FFN) layers are utilized to build LMs. |
| Approach: | They reverse-engineer the operation of feed-forward network layers to find out how they work . they show that each update can be decomposed to sub-updates corresponding to single parameter vectors . |
| Outcome: | The proposed model reduces the toxicity of GPT2 by almost 50% and improves computation efficiency with a simple early exit rule, saving 20% of computation on average. |
Paraphrasing vs Coreferring: Two Sides of the Same Coin (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Lexical resources such as WordNet capture synonyms and hypernyms, as well as antonyms, which can be used to refer to the same event when the arguments are reversed. |
| Approach: | They used annotations from an event coreference dataset as distant supervision to re-score heuristically-extracted predicate paraphrases. |
| Outcome: | The proposed model improved modestly but consistently in the two tasks. |