Papers by Ehsan Doostmohammadi
Surface-Based Retrieval Reduces Perplexity of Retrieval-Augmented Language Models (2023.acl-short)
Copied to clipboard
| Challenge: | Augmenting language models with a retrieval mechanism has been shown to improve performance while keeping the number of parameters low. |
| Approach: | They propose to augment language models with a retrieval mechanism by replacing semantic retrieval with BM25 . they find that the model's performance is better explained by surface-level similarities, they say . |
| Outcome: | The proposed method reduces perplexity and lowers the number of parameters while keeping the number low. |
Studying the Role of Input-Neighbor Overlap in Retrieval-Augmented Language Models Training Efficiency (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown that retrieval-augmented language models can perform better than larger models, but the optimal degree of overlap remains unclear. |
| Approach: | They investigate how varying levels of query–context overlap affect model performance . they specifically generate synthetic context through paraphrasing queries . |
| Outcome: | The proposed model can improve test-time perplexity and training time by 40% without compromising performance. |
On the Generalization Ability of Retrieval-Enhanced Transformers (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent work on retrieval-augmented language models has shown impressive results . performance gains from retrieval to a large extent originate from overlapping tokens between the database and test data, suggesting less of non-trivial generalization than previously assumed. |
| Approach: | They propose to off-load memory from trainable weights to a retrieval database and compare it to larger models with a larger model. |
| Outcome: | The proposed model outperforms GPT-3 and Jurassic-1 on the Pile at 4% of the model parameters. |
Persian Ezafe Recognition Using Transformers and Its Role in Part-Of-Speech Tagging (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Ezafe is a grammatical particle in some Iranian languages that links two words together but is almost always not indicated in Persian script. |
| Approach: | They propose to use Persian ezafe to improve part-of-speech tagging by using transformer-based methods to achieve state-of the-art results. |
| Outcome: | The proposed methods achieve state-of-the-art in the task of ezafe recognition and show that they are not useful to transformer-based methods. |
How Reliable Are Automatic Evaluation Methods for Instruction-Tuned LLMs? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on instruction-tuned Large Language Models has used automatic methods based on text overlap and LLM judgments as cost-effective alternatives to human evaluation. |
| Approach: | They perform a meta-evaluation of automatic methods and assess their reliability across a broad range of tasks. |
| Outcome: | The proposed method is unreliable in free-form generation tasks and cross-lingual scenarios. |
Joint Persian Word Segmentation Correction and Zero-Width Non-Joiner Recognition Using BERT (2020.coling-main)
Copied to clipboard
| Challenge: | a set of rules are set by the regulatory body of the Persian language regarding the use of white space and ZWNJ . only a few people follow these rules in writing formal Persian, let alone the informal language. |
| Approach: | They address problems of word segmentation and zero-width non-joiner recognition in Persian . they use a macro-averaged F1 score of 92.40% on a carefully collected corpus of 500 sentences . |
| Outcome: | The proposed problem is a sequence labeling problem in Persian . it achieves a macro-averaged F1 score of 92.40% on a carefully collected corpus of 500 sentences . |