Papers by Haytham Assem
Aligned Weight Regularizers for Pruning Pretrained Neural Networks (2022.findings-acl)
Copied to clipboard
| Challenge: | Pruning aims to reduce the number of parameters while maintaining performance close to the original network. |
| Approach: | They propose a self-distilled pruning strategy that maximizes representational similarity between pruned and unpruned networks. |
| Outcome: | The proposed pruning strategy outperforms smaller models and outperformed smaller ones with an equal number of parameters and is competitive against (6 times) larger distilled networks. |
Multi-Stage Framework with Refinement Based Point Set Registration for Unsupervised Bi-Lingual Word Alignment (2022.coling-1)
Copied to clipboard
| Challenge: | Existing unsupervised approaches to cross-lingual word embeddings suffer from instability and convergence issues. |
| Approach: | They propose a multi-stage framework for unsupervised mapping of bi-lingual word embeddings onto a shared vector space by combining adversarial initialization, refinement procedure and point set registration. |
| Outcome: | The proposed framework shows robustness against variable adversarial performance on diverse languages. |
Cross-lingual Sentence Embedding using Multi-Task Learning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing multilingual sentence embedding models require large parallel corpora to learn efficiently, limiting their scope. |
| Approach: | They propose a sentence embedding framework based on an unsupervised loss function . they capture semantic similarity and relatedness between sentences using a multi-task loss function. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on STS, BUCC and Tatoeba benchmarks and on a monolingual benchmark. |
Enhancing Contextual Understanding in Large Language Models through Contrastive Decoding (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models lack contextual knowledge, resulting in text with factual inconsistencies or contextually unfaithful content. |
| Approach: | They propose a method that integrates contrastive decoding with adversarial irrelevant passages as negative samples to enhance robust context grounding during generation. |
| Outcome: | The proposed method improves context grounding during generation without training. |
Improved Out-of-Scope Intent Classification with Dual Encoding and Threshold-based Re-Classification (2024.lrec-main)
Copied to clipboard
| Challenge: | Current methods for intent classification often rely on assumptions about data distributions and outliers are unpredictable . |
| Approach: | They propose a dual encoder for threshold-based re-classification that generates user utterance embeddings and incorporates out-of-scope phrases from open-domain datasets. |
| Outcome: | The proposed framework outperforms benchmarks on the CLINC-150, Stackoverflow, and Banking77 datasets and achieves an increase of up to 13% and 5% in F1 score for known and unknown intents. |