Papers by Yoav Katz
Masked by Consensus: Disentangling Privileged Knowledge in LLM Correctness (2026.acl-long)
Copied to clipboard
| Challenge: | Recent research suggests large language models encode meta-information about their own outputs. |
| Approach: | They investigate whether large language models possess similar privileged knowledge about answer correctness . they train correctness classifiers on question representations from a model’s hidden states and external models . |
| Outcome: | The proposed model outperforms peer-model models in factual knowledge tasks, but shows no advantage in math reasoning. |
From Surrogacy to Adoption; From Bitcoin to Cryptocurrency: Debate Topic Expansion (P19-1)
Copied to clipboard
Roy Bar-Haim, Dalia Krieger, Orith Toledo-Ronen, Lilach Edelstein, Yonatan Bilu, Alon Halfon, Yoav Katz, Amir Menczel, Ranit Aharonov, Noam Slonim
| Challenge: | Recent advances in argumentation mining have left much of the relevant argumentative content out of reach. |
| Approach: | They propose a task of Debate Topic Expansion to find related topics for a given debate topic, along with an annotated dataset for the task. |
| Outcome: | The proposed algorithms differ from well-studied lexical-semantic relations and show they work well in argumentation mining. |
Welcome to the Real World: Efficient, Incremental and Scalable Key Point Analysis (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Key Point Analysis (KPA) extracts the main points from opinions and quantifies their prevalence. |
| Approach: | They propose a key point analysis framework that extracts the main points from opinions and quantifies their prevalence. |
| Outcome: | The proposed system is able to match sentences to key points over five datasets and demonstrate its performance. |
Knowledge is a Region in Weight Space for Fine-tuned Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Specifically, language models that have been finetuned on the same dataset form a tight cluster in the weight space, while models finetuning on different datasets from the same underlying task form s looser clusters. |
| Approach: | They analyze the weight space and the underlying loss landscape of different models to determine how they are interconnected and how they perform. |
| Outcome: | The proposed method improves accuracy on 11 out of 12 datasets by starting from the center of the region, if not more, than using the pretrained model. |
Financial Event Extraction Using Wikipedia-Based Weak Supervision (D19-51)
Copied to clipboard
Liat Ein-Dor, Ariel Gera, Orith Toledo-Ronen, Alon Halfon, Benjamin Sznajder, Lena Dankin, Yonatan Bilu, Yoav Katz, Noam Slonim
| Challenge: | Existing methods for detecting financial and economic events from text have relied on a knowledge-base of financial events, or corresponding financial figures. |
| Approach: | They propose to use Wikipedia sections to extract weak labels for sentences describing economic events from text. |
| Outcome: | The proposed method can extract weak labels for sentences describing economic events from Wikipedia sentences. |
Unitxt: Flexible, Shareable and Reusable Data Preparation and Evaluation for Generative AI (2024.naacl-demo)
Copied to clipboard
Elron Bandel, Yotam Perlitz, Elad Venezian, Roni Friedman, Ofir Arviv, Matan Orbach, Shachar Don-Yehiya, Dafna Sheinwald, Ariel Gera, Leshem Choshen, Michal Shmueli-Scheuer, Yoav Katz
| Challenge: | Textual data processing pipelines are tailored to specific datasets, task and model combinations. |
| Approach: | They propose a library for customizable textual data preparation and evaluation tailored to generative language models. |
| Outcome: | Unitxt is a library for customizable textual data preparation and evaluation tailored to generative language models. |
NERetrieve: Dataset for Next Generation Named Entity Recognition and Retrieval (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a widely adopted NLP task . authors present three variants of NER task, with dataset to support them . |
| Approach: | They propose three variants of the NER task, together with a dataset to support them . they propose a move towards more fine-grained entities and zero-shot recognition . |
| Outcome: | The proposed model matches or surpasses existing models in NER tasks . the proposed model is based on a large, silver-annotated corpus of 4 million paragraphs . |
NER Retriever: Zero-Shot Named Entity Retrieval with Type-Aware Embeddings (2025.findings-emnlp)
Copied to clipboard
| Challenge: | NER Retriever uses a user-defined type description to retrieve documents mentioning entities of that type. |
| Approach: | They propose a zero-shot retrieval framework for ad-hoc Named Entity Recognition . a user-defined type description is used to retrieve documents mentioning entities of that type . |
| Outcome: | The proposed framework outperforms lexical and dense retrieval baselines on three benchmarks. |
Zero-shot Topical Text Classification with LLMs - an Experimental Study (2023.findings-emnlp)
Copied to clipboard
Shai Gretz, Alon Halfon, Ilya Shnayderman, Orith Toledo-Ronen, Artem Spector, Lena Dankin, Yannis Katsis, Ofir Arviv, Yoav Katz, Noam Slonim, Liat Ein-Dor
| Challenge: | Topical text classification is an ancient, yet timely research area in natural language processing. |
| Approach: | They compare the zero-shot performance of a variety of LMs over a large dataset of 23 publicly available TTC datasets. |
| Outcome: | The proposed models outperform their counterparts over a large dataset and show that they perform better in a zero-shot scenario. |
Multi-Domain Targeted Sentiment Analysis (2022.naacl-main)
Copied to clipboard
| Challenge: | Targeted Sentiment Analysis (TSA) is a task for generating insights from consumer reviews. |
| Approach: | They propose a multi-domain TSA system that augments a given training set with diverse weak labels from assorted domains and augments it with Yelp reviews. |
| Outcome: | The proposed model outperforms manual methods on three evaluation datasets across different domains and shows that it performs well. |
Benchmark Data and Evaluation Framework for Intent Discovery Around COVID-19 Vaccine Hesitancy (2023.findings-eacl)
Copied to clipboard
Shai Gretz, Assaf Toledo, Roni Friedman, Dan Lahav, Rose Weeks, Naor Bar-Zeev, João Sedoc, Pooja Sangha, Yoav Katz, Noam Slonim
| Challenge: | As COVID-19 vaccines were rolled out, they were met with widespread hesitancy. |
| Approach: | They propose a new framework for intent discovery that leverages existing intent classifiers to provide a real-world conversational dataset of conversations conducted by actual users with VIRA. |
| Outcome: | The proposed framework enables users to find out what they are doing and why they are hesitant. |
Knowledge Navigator: LLM-guided Browsing Framework for Exploratory Search in Scientific Literature (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Knowledge Navigator organizes retrieved documents into a navigable, two-level hierarchy of named and descriptive topics and subtopics. |
| Approach: | They propose to organize retrieved scientific documents into a navigable, two-level hierarchy of named and descriptive topics and subtopics. |
| Outcome: | The proposed system provides an overall view of the research themes in a domain while also enabling iterative search and deeper knowledge discovery within specific subtopics. |
Where to start? Analyzing the potential value of intermediate models (2023.emnlp-main)
Copied to clipboard
| Challenge: | a finetuned model may be better base models than the vanilla pretrained model . this scheme, often referred to as intertraining, is the focus of the present work . |
| Approach: | They propose a scheme to analyze the potential intertraining gain independently for the target dataset and for a base model being considered as a starting point. |
| Outcome: | The proposed model is strong even if training data was not aligned with target dataset. |
YASO: A Targeted Sentiment Analysis Evaluation Dataset for Open-Domain Reviews (2021.emnlp-main)
Copied to clipboard
| Challenge: | YASO contains 2,215 English sentences from dozens of review domains, annotated with target terms and their sentiment. |
| Approach: | They propose a new TSA evaluation dataset of open-domain user reviews in English . YASO contains 2,215 English sentences annotated with target terms and their sentiment . |
| Outcome: | The proposed dataset verifies the reliability of the annotations and explores the characteristics of the collected data. |
Project Debater APIs: Decomposing the AI Grand Challenge (2021.emnlp-demo)
Copied to clipboard
| Challenge: | Project Debater is the first AI system that can debate human experts on complex topics. |
| Approach: | They describe Project Debater's architecture and evaluate its performance . they will focus on Key Point Analysis, a novel technology that identifies main points . |
| Outcome: | The proposed system can debate human experts on complex topics. |
Active Learning for BERT: An Empirical Study (2020.emnlp-main)
Copied to clipboard
Liat Ein-Dor, Alon Halfon, Ariel Gera, Eyal Shnarch, Lena Dankin, Leshem Choshen, Marina Danilevsky, Ranit Aharonov, Yoav Katz, Noam Slonim
| Challenge: | Existing approaches to deal with data scarcity are active learning (AL) and pre-trained models are not being considered. |
| Approach: | They propose to use active learning techniques to cope with data scarcity in binary text classification scenarios where the annotation budget is very small and the data is often skewed. |
| Outcome: | The proposed methods improve BERT performance in binary text classification scenarios where the annotation budget is very small and the data is often skewed. |