Papers by Eyal Shnarch
Efficient Benchmarking (of Language Models) (2024.naacl-long)
Copied to clipboard
Yotam Perlitz, Elron Bandel, Ariel Gera, Ofir Arviv, Liat Ein-Dor, Eyal Shnarch, Noam Slonim, Michal Shmueli-Scheuer, Leshem Choshen
| Challenge: | Efficient Benchmarking is a method for reducing computation costs of LM evaluation without compromising reliability. |
| Approach: | They propose to reduce the computation costs of LM evaluation without compromising reliability by using a new measure - Decision Impact on Reliability. |
| Outcome: | The proposed benchmarks reduce computation costs by x100 or more, while maintaining reliability. |
Cluster & Tune: Boost Cold Start Performance in Text Classification (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods to fine-tune pre-trained models for text classification are poor in practice. |
| Approach: | They propose to add an intermediate unsupervised classification task between pre-training and fine-tuning phases to boost performance of pre-trained models. |
| Outcome: | The proposed method improves performance on topical classification tasks when labeled data is scarce. |
Label Sleuth: From Unlabeled Text to a Classifier in a Few Hours (2022.emnlp-demos)
Copied to clipboard
Eyal Shnarch, Alon Halfon, Ariel Gera, Marina Danilevsky, Yannis Katsis, Leshem Choshen, Martin Santillan Cooper, Dina Epelboim, Zheng Zhang, Dakuo Wang
| Challenge: | Label Sleuth is an open source system for labeling and creating text classifiers which does not require coding skills nor machine learning knowledge. |
| Approach: | *Label Sleuth* is an open source system for labeling and creating text classifiers which does not require coding skills nor machine learning knowledge. |
| Outcome: | *Label Sleuth* is an open source system for labeling and creating text classifiers. |
GrASP: A Library for Extracting and Exploring Human-Interpretable Textual Patterns (2022.lrec-1)
Copied to clipboard
| Challenge: | a Python library is available for extracting patterns from textual data. |
| Approach: | They propose a Python library for extracting patterns from textual data . it integrates a public implementation of the existing GrASP algorithm . |
| Outcome: | The proposed library integrates a public implementation of the existing GrASP algorithm. |
Semantic Relatedness of Wikipedia Concepts – Benchmark Data and a Working Solution (L18-1)
Copied to clipboard
| Challenge: | Existing methods to measure relatedness between Wikipedia concepts are lacking. |
| Approach: | They propose a new type of concept relatedness dataset, WORD, which is annotated by a human . they use this dataset to assess relatedness between Wikipedia concepts using supervised methods. |
| Outcome: | The proposed dataset outperforms existing methods for measuring relatedness between Wikipedia concepts. |
Zero-Shot Text Classification with Self-Training (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large pretrained language models have increased attention to zero-shot text classification. |
| Approach: | They propose a plug-and-play method to bridge this gap by requiring only class names along with an unlabeled dataset. |
| Outcome: | The proposed model can be trained on a natural language inference dataset and performs on dozens of unseen tasks without the need for domain expertise or trial and error. |
Will it Blend? Blending Weak and Strong Labeled Data in a Neural Network for Argumentation Mining (P18-2)
Copied to clipboard
Eyal Shnarch, Carlos Alzate, Lena Dankin, Martin Gleize, Yufang Hou, Leshem Choshen, Ranit Aharonov, Noam Slonim
| Challenge: | Obtaining high quality labeled data for natural language understanding tasks is slow, error-prone, complicated and expensive. |
| Approach: | They propose a method to blend weak and strong labeled data during the training of neural networks using a topic-dependent evidence detection dataset. |
| Outcome: | The proposed method improves the training of neural networks when a small amount of labeled data is available. |
The Benefits of Bad Advice: Autocontrastive Decoding across Model Layers (2023.acl-long)
Copied to clipboard
Ariel Gera, Roni Friedman, Ofir Arviv, Chulaka Gunasekara, Benjamin Sznajder, Noam Slonim, Eyal Shnarch
| Challenge: | Existing approaches to apply language models to tasks that require intermediate representations are less informative. |
| Approach: | They propose a novel approach that utilizes the contrast between layers to improve text generation outputs. |
| Outcome: | The proposed approach mitigates degenerative behaviors of the model in open-ended generation, significantly improving the quality of generated texts. |
Label-Efficient Model Selection for Text Generation (2024.acl-long)
Copied to clipboard
| Challenge: | Model selection for a given task can entail extensive annotation of the quality of outputs of different models. |
| Approach: | They propose a model-agnostic method to make an informed decision between candidate text generation models based on preference annotations. |
| Outcome: | The proposed method reduces the required number of annotations by up to 75% while maintaining high evaluation reliability. |
Are You Convinced? Choosing the More Convincing Evidence with a Siamese Network (P19-1)
Copied to clipboard
Martin Gleize, Eyal Shnarch, Leshem Choshen, Lena Dankin, Guy Moshkowich, Ranit Aharonov, Noam Slonim
| Challenge: | Recent advances in argument detection have made it easier to identify the more convincing arguments. |
| Approach: | They propose a new data set of pairs of evidence labeled for convincingness that is more challenging than existing alternatives. |
| Outcome: | The proposed method outperforms baselines on convincingness data and its own. |
Active Learning for BERT: An Empirical Study (2020.emnlp-main)
Copied to clipboard
Liat Ein-Dor, Alon Halfon, Ariel Gera, Eyal Shnarch, Lena Dankin, Leshem Choshen, Marina Danilevsky, Ranit Aharonov, Yoav Katz, Noam Slonim
| Challenge: | Existing approaches to deal with data scarcity are active learning (AL) and pre-trained models are not being considered. |
| Approach: | They propose to use active learning techniques to cope with data scarcity in binary text classification scenarios where the annotation budget is very small and the data is often skewed. |
| Outcome: | The proposed methods improve BERT performance in binary text classification scenarios where the annotation budget is very small and the data is often skewed. |
Unsupervised Expressive Rules Provide Explainability and Assist Human Experts Grasping New Domains (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Understanding unexplored data is a slow process, and there is no labeled data at hand. |
| Approach: | They propose to use unsupervised methods to reveal rules which cluster unexplored corpus by its prominent categories to help domain experts understand their texts. |
| Outcome: | The proposed rules can be bootstrapped to identify target categories and deepen understanding of the data. |