Papers by Amirhossein Kazemnejad
Paraphrase Generation by Learning How to Edit from Samples (2020.acl-main)
Copied to clipboard
| Challenge: | Experimental results show the superiority of our retrieval-based paraphrase generation model in terms of both automatic metrics and human evaluation of relevance, grammaticality, and diversity of generated paraphrases. |
| Approach: | They propose a retrieval-based method for paraphrase generation which uses a novel editor module to extract edits from paraphrase pairs. |
| Outcome: | The proposed model outperforms existing models in automatic metrics and human evaluation of relevance, grammaticality, and diversity of generated paraphrases. |
Measuring the Knowledge Acquisition-Utilization Gap in Pretrained Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent research has demonstrated that pre-trained language models acquire a broad range of knowledge about linguistic structures, encyclopedic relations, levels of commonsense, and even coding and reasoning rules. |
| Approach: | They propose a systematic framework to measure parametric knowledge utilization in pre-trained language models by extracting parametric information from a PLM and constructing a downstream task around this extracted knowledge. |
| Outcome: | The proposed framework extracts parametric knowledge from a PLM and constructs a downstream task around this extracted knowledge. |
The Curious Case of Absolute Position Embeddings (2022.findings-emnlp)
Copied to clipboard
| Challenge: | In natural language, it is not absolute position that matters, but relative position . et al., 2017) language models incorporate positional encodings that encode absolute (linear) word order. |
| Approach: | They find that Transformer language models encode word order using positional information . they also find that models that use absolute position embeddings over-rely on positional data . |
| Outcome: | The results raise questions about the efficacy of APEs to model the relativity of position information. |