Papers by Ran Levy
McPhraSy: Multi-Context Phrase Similarity and Clustering (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for estimating phrase similarity use the phrase context only during training, instead relying on the phrase itself. |
| Approach: | They propose a novel algorithm that leverages multiple contexts during inference to estimate the similarity of phrases based on multiple context. |
| Outcome: | The proposed method outperforms existing models on two phrase similarity datasets by 13.3% and a new task that relies on phrase similarities in the product reviews domain. |
Re-Examining Summarization Evaluation across Multiple Quality Criteria (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a number of automated evaluation metrics are evaluated by multiple quality criteria, such as relevance, consistency, fluency and coherence. |
| Approach: | They propose a method that removes the confounding variable and detects unreliable correlations. |
| Outcome: | The proposed method detects unreliable correlations between QCs and human scores . it is based on a multi-QC setup, but it fails to detect summary corruptions . |
PASS: Perturb-and-Select Summarizer for Product Reviews (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work on product reviews summarization focuses on generating concise, coherent and informative summaries, but this task is challenging. |
| Approach: | They propose a product reviews summarization task that employs a large pre-trained Transformer-based model and a method for ranking these summaries according to desired criteria. |
| Outcome: | The proposed system avoids the problem of self-contradiction by ranking the summaries according to desired criteria. |
Semantic Relatedness of Wikipedia Concepts – Benchmark Data and a Working Solution (L18-1)
Copied to clipboard
| Challenge: | Existing methods to measure relatedness between Wikipedia concepts are lacking. |
| Approach: | They propose a new type of concept relatedness dataset, WORD, which is annotated by a human . they use this dataset to assess relatedness between Wikipedia concepts using supervised methods. |
| Outcome: | The proposed dataset outperforms existing methods for measuring relatedness between Wikipedia concepts. |
Identifying Helpful Sentences in Product Reviews (2021.naacl-main)
Copied to clipboard
| Challenge: | a key advantage of online shopping is the ability to read what other customers are saying about products of interest. |
| Approach: | They propose a task to extract a representative helpful sentence from reviews . they collect a dataset in english and use crowd-sourcing to test their model . |
| Outcome: | The proposed model outperforms baselines in a crowd-sourced model of representative helpful sentences from product reviews. |
Towards an argumentative content search engine using weak supervision (C18-1)
Copied to clipboard
| Challenge: | Existing work focused on detecting claims within a small set of documents . however, pinpointing relevant claims within massive unstructured corpora, received little attention. |
| Approach: | They propose to use a weak signal to develop a query for claim–sentence detection using a large text corpus. |
| Outcome: | The proposed system outperforms previous results in terms of precision and coverage. |
HotelQuEST: Balancing Quality and Efficiency in Agentic Search (2026.eacl-industry)
Copied to clipboard
| Challenge: | Existing benchmarks for agentic search focus primarily on answer quality, overlooking efficiency factors that are critical for real-world deployment. |
| Approach: | They propose a benchmark for hotel search queries that includes 214 hotel query queries that range from simple factual requests to complex queries. |
| Outcome: | The proposed benchmarks show that LLM-based agents achieve higher accuracy than traditional retrievers, but at substantially higher costs due to redundant tool calls and suboptimal routing that fails to match query complexity to model capability. |
Multi-Review Fusion-in-Context (2024.findings-naacl)
Copied to clipboard
| Challenge: | Current methods for generating text are opaque and difficult to control and interpret due to their opaque nature. |
| Approach: | They propose a modular approach with separate components for each step . they formalize Fusion-in-Context as a standalone task, whose input consists of source texts with highlighted spans of targeted content. |
| Outcome: | The proposed approach is based on a curated dataset of 1000 instances in the reviews domain and a novel evaluation framework for assessing the faithfulness and coverage of highlights. |
The Power of Summary-Source Alignments (2024.findings-acl)
Copied to clipboard
Ori Ernst, Ori Shapira, Aviv Slobodkin, Sharon Adar, Mohit Bansal, Jacob Goldberger, Ran Levy, Ido Dagan
| Challenge: | Multi-document summarization (MDS) is a challenging task, often decomposed to subtasks of salience and redundancy detection, followed by text generation. |
| Approach: | They propose to extend the summary-source alignment framework by applying it at the more fine-grained proposition span level and annotating alignment manually in a multi-document setup. |
| Outcome: | The proposed framework can yield several datasets for at least six different tasks. |