Papers by Ori Ernst
Re-Examining Summarization Evaluation across Multiple Quality Criteria (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a number of automated evaluation metrics are evaluated by multiple quality criteria, such as relevance, consistency, fluency and coherence. |
| Approach: | They propose a method that removes the confounding variable and detects unreliable correlations. |
| Outcome: | The proposed method detects unreliable correlations between QCs and human scores . it is based on a multi-QC setup, but it fails to detect summary corruptions . |
How “Multi” is Multi-Document Summarization? (2022.emnlp-main)
Copied to clipboard
| Challenge: | Multi-document summarization (MDS) aims at combining information spread across multiple documents . a single document often covers the full summary content . |
| Approach: | They propose a measure to evaluate the degree to which a summary is "disperse" they propose to combine information from multiple documents into a single document to generate a concise summary . |
| Outcome: | The proposed measure evaluates the degree to which a summary is "disperse" the measure is applied to several popular MDS datasets and state-of-the-art systems. |
QA-Align: Representing Cross-Text Content Overlap by Aligning Question-Answer Propositions (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to consolidate textual inputs are difficult to implement . a recent study aims to capture content overlap by combining multiple textual elements . |
| Approach: | They propose to align predicate-argument relations across texts to represent content overlap . their setting exploits QA-SRL, utilizing question-answer pairs to capture predicates . |
| Outcome: | The proposed task captures content overlap beyond lexical similarity and complements cross-document coreference with proposition-level links, offering potential use for downstream tasks. |
Controlled Text Reduction (2022.emnlp-main)
Copied to clipboard
| Challenge: | Abstractive text summarization models separate the salience detection phase from the text generation phase. |
| Approach: | They propose to formalize Controlled Text Reduction as a standalone task . they advocate the potential of such models for modular fully-automatic summarization . |
| Outcome: | The proposed model shows that it is possible to produce a reduced version of a source text using decomposed modeling. |
PreSumm: Predicting Summarization Performance Without Summarizing (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in summarization models do not produce all documents in the same way, despite their inherent design principles and operational mechanisms. |
| Approach: | They propose a task where a system predicts summarization performance based solely on the source document. |
| Outcome: | The proposed task identifies documents that require manual summarization and improves dataset quality by filtering outliers and noisy documents. |
Improving the Calibration of Confidence Scores in Text Generation Using the Output Distribution’s Characteristics (2025.acl-short)
Copied to clipboard
| Challenge: | Existing methods for estimating confidence in text generation do not account for many valid answers in generation tasks. |
| Approach: | They propose task-agnostic confidence metrics that rely solely on model probabilities without the need for further fine-tuning or heuristics. |
| Outcome: | The proposed models improve the accuracy of BART and Flan-T5 on summarization, translation, and question answering datasets. |
Where Did That Come From? Sentence-Level Error-Tolerant Attribution (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing task definitions exclude unsupported or hallucinated content leaving them unattributed . authors propose a new definition for sentence-level error-tolerant attribution . |
| Approach: | They propose a new definition for sentence-level error-tolerant attribution that extends attribution to include incorrect or hallucinated content. |
| Outcome: | The proposed approach reduces annotation time and facilitates hallucination fixing. |
Extending Multi-Text Sentence Fusion Resources via Pyramid Annotations (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing datasets for sentence fusion tasks are limited in size and scope . despite recent advances, cross-document tasks such as multi-document summarization have not progressed with the same pace. |
| Approach: | They propose to extend a sentence fusion dataset by almost four times its original size . they relabel the dataset and employ more data sources to improve model performance . |
| Outcome: | The proposed dataset triples the size of an earlier dataset and improves performance . it also includes more complex training instances better reflecting those found in "the wild" |
Proposition-Level Clustering for Multi-Document Summarization (2022.naacl-main)
Copied to clipboard
Ori Ernst, Avi Caciularu, Ori Shapira, Ramakanth Pasunuru, Mohit Bansal, Jacob Goldberger, Ido Dagan
| Challenge: | Existing methods focused on clustering sentences to indicate information saliency and avoid redundancy. |
| Approach: | They propose to group together sub-sentential propositions to generate a representative sentence for each cluster via text fusion. |
| Outcome: | The proposed method improves over the previous state-of-the-art method in the DUC 2004 and TAC 2011 datasets, both in automatic ROUGE scores and human preference. |
The Power of Summary-Source Alignments (2024.findings-acl)
Copied to clipboard
Ori Ernst, Ori Shapira, Aviv Slobodkin, Sharon Adar, Mohit Bansal, Jacob Goldberger, Ran Levy, Ido Dagan
| Challenge: | Multi-document summarization (MDS) is a challenging task, often decomposed to subtasks of salience and redundancy detection, followed by text generation. |
| Approach: | They propose to extend the summary-source alignment framework by applying it at the more fine-grained proposition span level and annotating alignment manually in a multi-document setup. |
| Outcome: | The proposed framework can yield several datasets for at least six different tasks. |
iFacetSum: Coreference-based Interactive Faceted Summarization for Multi-Document Exploration (2021.emnlp-demo)
Copied to clipboard
Eran Hirsch, Alon Eirew, Ori Shapira, Avi Caciularu, Arie Cattan, Ori Ernst, Ramakanth Pasunuru, Hadar Ronen, Mohit Bansal, Ido Dagan
| Challenge: | iFS provides a faceted navigation scheme that provides abstractive summaries for the user’s selections. |
| Approach: | They propose a web application that integrates interactive summarization and faceted search to provide a faceted navigation scheme that yields abstractive summaries for the user's selections. |
| Outcome: | The proposed system provides a comprehensive overview as well as particular details regard-ing subtopics of interest. |
OpenAsp: A Benchmark for Multi-document Open Aspect-based Summarization (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing models focus on a limited set of predefined aspects, resulting in a lack of realistic open aspect setting. |
| Approach: | They propose a benchmark for multi-document open aspect-based summarization using an annotation protocol. |
| Outcome: | The proposed benchmark satisfies the needs of users in real-world scenarios. |