Papers by Hamidreza Amirzadeh
Context-aware Biases for Length Extrapolation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for Relative Positional Encoding (RPE) lack the capacity to adapt to different input contexts. |
| Approach: | They propose an additive RPE method that learns token-specific, context-aware biases for each attention head in transformers by dynamically adjusting positional biase based on the input sequence. |
| Outcome: | The proposed method significantly improves the extrapolation performance of existing RPE methods on the fineWeb-Edu-10B and WikiText-103 datasets. |
data2lang2vec: Data Driven Typological Features Completion (2025.coling-main)
Copied to clipboard
| Challenge: | Language typology databases are useful for multilingual Natural Language Processing (NLP) but their coverage is limited, with only 28.9% of all possible combinations specified in the database. |
| Approach: | They propose to use textual data to improve feature prediction by using a multi-lingual Part-of-Speech tagger and a more realistic evaluation setup to focus on likely to be missing typology features. |
| Outcome: | The proposed model outperforms previous studies on missing features in 1,749 languages and with external statistical features and machine learning algorithms. |