Papers by Maor Ivgi
Achieving Model Robustness through Discrete Adversarial Training (2021.emnlp-main)
Copied to clipboard
| Challenge: | Discrete adversarial attacks are symbolic perturbations to a language input that preserve the output label but lead to predicting error. |
| Approach: | They propose a discrete adversarial attack based on best-first search and random sampling attacks that are not based upon expensive search procedures. |
| Outcome: | The proposed attack outperforms offline augmentation and speedups on three datasets. |
Scaling Laws Under the Microscope: Predicting Transformer Performance from Small Scale Experiments (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Neural scaling laws define a predictable relationship between a model’s parameter count and its performance after training in the form of a power law. |
| Approach: | They perform an empirical investigation of language understanding tasks and evaluate their results to determine whether scaling laws can be used to accelerate model development. |
| Outcome: | The proposed scaling laws can be exploited for debugging convergence when training large models, and can predict the performance of larger models. |
Efficient Long-Text Understanding with Short-Text Models (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing transformer-based pretrained language models cannot be applied to long sequences due to their quadratic complexity. |
| Approach: | They propose a simple approach to long sequences that re-uses battle-tested short-text pretrained LMs. |
| Outcome: | The proposed approach is competitive with specialized models that are up to 50x larger and require a dedicated and expensive pretraining step. |
In-Context Learning with Long-Context Models: An In-Depth Exploration (2025.naacl-long)
Copied to clipboard
| Challenge: | In-context learning is limited by context length, but it can be used for many tasks. |
| Approach: | They study the behavior of in-context learning at an extreme context length . example retrieval shows excellent performance at low context lengths but has diminished gains . |
| Outcome: | The proposed model can perform many tasks with reasonable accuracy when a few examples are provided in-context. |
SCROLLS: Standardized CompaRison Over Long Language Sequences (2022.emnlp-main)
Copied to clipboard
Uri Shaham, Elad Segal, Maor Ivgi, Avia Efrat, Ori Yoran, Adi Haviv, Ankit Gupta, Wenhan Xiong, Mor Geva, Jonathan Berant, Omer Levy
| Challenge: | Standard NLP benchmarks focus on short texts, but long texts are produced in the context of longer discourses. |
| Approach: | They propose a new benchmark that places models in context of long texts that require reasoning over long texts. |
| Outcome: | The proposed task sets are based on a set of long-text datasets and host a live leaderboard to facilitate research on model architecture and pretraining methods. |
ZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understanding (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing benchmarks for long text understanding focus on short sequences, such as BigBench and HELM. |
| Approach: | They propose a zero-shot benchmark for natural language understanding over long texts . they adapt six tasks from the SCROLLS benchmark and add four new datasets . |
| Outcome: | The proposed benchmark outperforms ChatGPT and GPT-4 in a number of open tasks. |