Papers by Peter Izsak
Optimizing Retrieval-augmented Reader Models via Token Elimination (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for ODQA use a retrieval-augmented language model . a generative model can cause a significant bottleneck in decoding time . |
| Approach: | They propose to eliminate some of the retrieved information that might not contribute essential information to the answer generation process. |
| Outcome: | The proposed method reduces run-time by up to 62.2% with only 2% reduction in performance and improves performance. |
Transformer Language Models without Positional Encodings Still Learn Positional Information (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Using positional embeddings, Causal transformer language models learn an implicit notion of absolute positions. |
| Approach: | They propose to use positional embeddings to encode positional information in transformer language models. |
| Outcome: | The proposed model learns an implicit notion of absolute positions across datasets, model sizes, and sequence lengths. |
CoTAR: Chain-of-Thought Attribution Reasoning with Multi-level Granularity (2024.findings-emnlp)
Copied to clipboard
| Challenge: | State-of-the-art QA systems employ Large Language Models (LLMs) however, these models tend to hallucinate information in their responses. |
| Approach: | They propose an attribution-oriented Chain-of-Thought reasoning method to enhance attributions. |
| Outcome: | The proposed method outperforms existing models on context enhanced question-answering datasets and shows that it can be used to improve accuracy. |
Term Set Expansion based NLP Architect by Intel AI Lab (D18-2)
Copied to clipboard
Jonathan Mamou, Oren Pereg, Moshe Wasserblat, Alon Eirew, Yael Green, Shira Guskin, Peter Izsak, Daniel Korat
| Challenge: | SetExpander is a corpus-based system for expanding a seed set of terms into a more complete set of words belonging to the same semantic class. |
| Approach: | They propose a corpus-based system for expanding a seed set of terms into a more complete set of words that belong to the same semantic class. |
| Outcome: | The proposed system can expand a seed set of terms into a more complete set of words belonging to the same semantic class. |
How to Train BERT with an Academic Budget (2021.emnlp-main)
Copied to clipboard
| Challenge: | Large language models such as BERT are used in many NLP tasks, but their pretraining phase can be prohibitively expensive for startups and academic research groups. |
| Approach: | They propose a recipe for pretraining a large language model in 24 hours using a low-end deep learning server. |
| Outcome: | The proposed model can be trained on GLUE tasks at fraction of the cost of pretraining. |
SetExpander: End-to-end Term Set Expansion Based on Multi-Context Term Embeddings (C18-2)
Copied to clipboard
Jonathan Mamou, Oren Pereg, Moshe Wasserblat, Ido Dagan, Yoav Goldberg, Alon Eirew, Yael Green, Shira Guskin, Peter Izsak, Daniel Korat
| Challenge: | SetExpander is a corpus-based system for expanding a seed set of terms into a more complete set of words belonging to the same semantic class. |
| Approach: | They propose to use a corpus-based system for expanding a seed set of terms into a more complete set of words that belong to the same semantic class. |
| Outcome: | The proposed system can expand a seed set of terms, validate it, re-expand the expanded set and store it, thus simplifying the extraction of domain-specific fine-grained semantic classes. |