Papers by Zohar Karnin
TADPOLE: Task ADapted Pre-Training via AnOmaLy DEtection (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to solve domain shifts in NLP tasks require additional pre-training . current approaches focus on the downstream corpus when it is small, but are not effective . |
| Approach: | They propose a task-adapted pre-training framework that can be used when the downstream corpus is too small for additional pre-tuning. |
| Outcome: | The proposed framework outperforms baseline methods on biomedical, computer science, news, and movie reviews tasks. |
Representation Projection Invariance Mitigates Representation Collapse (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Fine-tuning contextualized representations by pre-trained models can lead to representation degradation, which can result in instability, sub-optimal performance, and weak generalization. |
| Approach: | They propose a regularization method to maintain the information content of representations and reduce representation collapse during fine-tuning. |
| Outcome: | The proposed method outperforms baselines on most tasks and improves out-of-distribution performance. |
The Distracting Effect: Understanding Irrelevant Passages in RAG (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods to detect and use hard distracting passages in RAG can cause problems . retrieved passages contain irrelevant but semantically related information that may mislead the LLM . |
| Approach: | They propose a method to identify and use hard distracting passages to improve RAG . they find that adding retrieved passages is found to ground the LLM response . |
| Outcome: | The proposed method achieves up to 7.5% increase in answering accuracy compared to fine-tuned datasets. |
schuBERT: Optimizing Elements of BERT (2020.acl-main)
Copied to clipboard
| Challenge: | Recent Transformer based models have achieved state-of-the-art performance for many natural language processing tasks including machine translation, question-answering tasks and semantic role labeling. |
| Approach: | They propose to reduce the number of parameters of BERT to obtain a much efficient light model. |
| Outcome: | The proposed model achieves 6.6% higher average accuracy on GLUE and SQuAD datasets than the previous model with three encoder layers while having the same number of parameters. |
Generating Q&A Benchmarks for RAG Evaluation in Enterprise Settings (2025.acl-industry)
Copied to clipboard
| Challenge: | DataMorgana generates synthetic Q&A benchmarks tailored to RAG applications . lexical, syntactic, and semantic diversity of generated benchmarks exceeds existing tools . |
| Approach: | They propose a tool for generating synthetic Q&A benchmarks tailored to RAG applications in enterprise settings. |
| Outcome: | The proposed tool surpasses existing tools in terms of lexical, syntactic, and semantic diversity while maintaining high quality. |
Pyramid-BERT: Reducing Complexity via Successive Core-set based Token Selection (2022.acl-long)
Copied to clipboard
| Challenge: | Existing models that use heuristics to shorten sequence lengths are computationally prohibitive. |
| Approach: | They propose a new method to shorten sequence lengths by transforming tokens through encoders and a core-set based token selection method that avoids expensive pre-training and fine tuning. |
| Outcome: | The proposed model outperforms existing models on GLUE benchmarks and Long Range Arena datasets and demonstrates that it is cost-effective and space-efficient. |
Quality Matters: Evaluating Synthetic Data for Tool-Using LLMs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to assess data quality for training and testing large language models are lacking. |
| Approach: | They propose two approaches to assess the reliability of data for training large language models for external tool usage. |
| Outcome: | The proposed approaches outperform models trained on high-quality data on two popular benchmarks and an extrinsic evaluation that showcases the impact of data quality on model performance. |