Papers by Mohammad Rostami
Learn Continually, Generalize Rapidly: Lifelong Knowledge Accumulation for Few-shot Learning (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models that pursue rapid generalization to new tasks are mostly trained in a single shot on fixed datasets, unable to dynamically expand their knowledge. |
| Approach: | They propose a new learning setup that assumes a model learns from a sequence of diverse NLP tasks arriving sequentially, accumulating knowledge for improved generalization to new tasks. |
| Outcome: | The proposed learning setup improves generalization ability while retaining performance on the tasks learned earlier. |
History repeats: Overcoming catastrophic forgetting for event-centric temporal knowledge graph completion (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for knowledge graph completion are incomplete and can lead to errors . retraining the model with the entire updated TKG can mitigate forgetting but is computationally burdensome. |
| Approach: | They propose a temporal regularization framework that allows repurposing of parameters . they propose 'clustering-based experience replay' that reinforces the past knowledge . |
| Outcome: | The proposed framework adapts to new events while reducing catastrophic forgetting. |
Domain Adaptation for Sentiment Analysis Using Robust Internal Representations (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Cross-domain sentiment analysis methods reduce the domain gap by training generalizable classifiers for each domain . large interclass margins in source domain help to reduce the effect of "domain shift" in the target domain. |
| Approach: | They propose a domain adaptation method which induces large margins between data representations that belong to different classes in an embedding space. |
| Outcome: | The proposed method reduces the domain gap by training cross-domain generalizable classifiers . large interclass margins in the source domain help reduce the effect of "domain shift" the proposed method is available in the u.s. |
Task-Attentive Transformer Architecture for Continual Learning of Vision-and-Language Tasks Using Knowledge Distillation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing algorithms for learning unimodal vision-only or language-only tasks are limited by the size and computational load of fine-tuning large-scale pre-trained neural networks. |
| Approach: | They propose a transformer-based CL architecture for learning bimodal vision-and-language tasks by increasing the number of the learnable parameters dynamically and using knowledge distillation. |
| Outcome: | The proposed model reaches state-of-the-art on vision-and-language tasks. |