Papers by Wael Hamza
A Hybrid Approach to Cross-lingual Product Review Summarization (2022.emnlp-industry)
Copied to clipboard
| Challenge: | Existing methods for summarizing product reviews with thousands of reviews are inefficient and time consuming. |
| Approach: | They propose an unsupervised extractive step and a supervised abstractive step to generate a short summary in any language. |
| Outcome: | The proposed model is as good as human written summaries in coherence, informativeness, non-redundancy, and fluency as human summary summators. |
Zero-shot Generalization in Dialog State Tracking through Generative Question Answering (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for Dialog State Tracking do not generalize well to new domains and unseen slots. |
| Approach: | They propose an ontology-free framework that queries for unseen constraints and slots in multi-domain task-oriented dialogs using a conditional language model pre-trained on substantive English sentences. |
| Outcome: | The proposed framework improves goal accuracy in zero-shot domain adaptation settings by up to 9% over the previous state-of-the-art on the MultiWOZ 2.1 dataset. |
Low-Resource Compositional Semantic Parsing with Concept Pretraining (2023.eacl-main)
Copied to clipboard
| Challenge: | Semantic parsing is a key role in voice assistants by mapping natural language to structured meaning representations. |
| Approach: | They propose an architecture to perform domain adaptation automatically with only a small amount of metadata about the new domain and without any new training data. |
| Outcome: | The proposed architecture outperforms existing models in low-resource settings. |
Recipes for Sequential Pre-training of Multilingual Encoder and Seq2Seq Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained encoder-only and sequence-to-sequence models are computationally expensive. |
| Approach: | They propose a recipe to initialize one model from the other to improve pre-training efficiency. |
| Outcome: | The proposed method matches the performance of a from-scratch model with a multilingual encoder while reducing the total compute cost by 27%. |
Attention Fusion: a light yet efficient late fusion mechanism for task adaptation in NLU (2022.findings-naacl)
Copied to clipboard
| Challenge: | a recent study has shown that fine-tuning pre-trained models is parameter-inefficient and expensive. |
| Approach: | They propose a task-attuned token module which integrates pre-trained network representations into a pre-trainer. |
| Outcome: | The proposed model trains only 0.0009% of the parameters and is efficient during computation and scalable during deployment. |
Delexicalized Paraphrase Generation (2020.coling-industry)
Copied to clipboard
| Challenge: | Using convolutional neural networks, we generate delexicalized sentences . 1.29% accuracy is achieved with the generated paraphrases . |
| Approach: | They propose a neural paraphrasing model that generates delexicalized sentences . they use convolutional neural networks to pool on slot values and use pointers to locate them . |
| Outcome: | The proposed model generates delexicalized sentences with high quality . it can be used for intent classification and named entity recognition tasks . |
LINGUIST: Language Model Instruction Tuning to Generate Annotated Utterances for Intent Classification and Slot Tagging (2022.coling-1)
Copied to clipboard
| Challenge: | LINGUIST generates annotated data for Intent Classification and Slot Tagging (IC+ST) we demonstrate fine-tuning of a large-scale seq2seq model to control outputs of multilingual data generation. |
| Approach: | They propose a method for generating annotated data for Intent Classification and Slot Tagging (IC+ST) they use a 5-billion-parameter multilingual sequence-to-sequence model to fine-tune it on a flexible instruction prompt. |
| Outcome: | The proposed method outperforms state-of-the-art approaches on a SNIPS intent setting and shows significant improvement on IC+ST in a cross-lingual setting. |
CLASP: Few-Shot Cross-Lingual Data Augmentation for Semantic Parsing (2022.aacl-short)
Copied to clipboard
| Challenge: | Large Language Models excel at a low-resource level given limited data, but are unsuitable for runtime systems which require low latency. |
| Approach: | They propose a method to augment training data for a model 40x smaller (500M parameters) they use Alexa to generate synthetic data from Alexa 20B to augment the training set . |
| Outcome: | The proposed method improves low-resource SP on two datasets in low-source settings. |
Neural Cross-Lingual Coreference Resolution And Its Application To Entity Linking (P18-2)
Copied to clipboard
| Challenge: | a cross-lingual coreference model is based on multi-lingual embeddings and language independent features. |
| Approach: | They propose a crosslingual coreference model that builds on multi-lingual embeddings and language independent features. |
| Outcome: | The proposed model outperforms the existing models on Chinese and Spanish test sets. |
Leveraging Context Information for Natural Question Generation (N18-2)
Copied to clipboard
| Challenge: | Existing work for natural question generation ignores the input passage or hard-codes answer positions. |
| Approach: | They propose a model that matches the answer with the passage before generating a question. |
| Outcome: | The proposed model outperforms the state-of-the-art model using rich features. |
Contextual Domain Classification with Temporal Representations (2021.naacl-industry)
Copied to clipboard
| Challenge: | Existing studies that incorporate context in SLU have focused on domains where context is limited to a few minutes. |
| Approach: | They propose temporal representations that combine wall-clock second difference and turn order offset information to utilize both recent and distant context in a novel large-scale setup. |
| Outcome: | The proposed model reduces 13.04% of classification errors compared to baseline . previous studies have focused on domains where context is limited to a few minutes . |
Instilling Type Knowledge in Language Models via Multi-Task QA (2022.findings-naacl)
Copied to clipboard
| Challenge: | Current methods to learn entity types rely on coarse, noisy labels . current methods rely only on text-to-text pre-training on type-centric questions . |
| Approach: | They propose to instill fine-grained type knowledge in language models by pre-training on type-centric questions. |
| Outcome: | The proposed model achieves state-of-the-art in zero-shot dialog state tracking benchmarks and can accurately infer entity types in Wikipedia articles. |
Controlled Data Generation via Insertion Operations for NLU (2022.naacl-industry)
Copied to clipboard
| Challenge: | a new approach to annotate live traffic is emerging to be cost-effective and efficient . manual data annotation is expensive and not preferred for meeting customer privacy expectations . |
| Approach: | They propose a targeted synthetic data generation technique by inserting tokens into a given semantic signature. |
| Outcome: | The proposed approach achieves the same accuracy as training with all available data on a voice assistant dataset. |
Multi-task Learning of Spoken Language Understanding by Integrating N-Best Hypotheses with Hierarchical Attention (2020.coling-industry)
Copied to clipboard
| Challenge: | Existing methods to integrate hypotheses into speech recognition systems are noisy and can cause information loss. |
| Approach: | They propose to integrate hypotheses into multi-task learning and transfer learning to improve performance. |
| Outcome: | The proposed model improves domain and intent classification by 19% and 37% compared to current methods . the proposed model could recover transcription and rewrite the query for a better understanding . |