Papers by Muhammad ElNokrashy
Investigating Cultural Alignment of Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are used to represent the diversity of human experience and culturally sensitive topics. |
| Approach: | They propose a method leveraging anthropological reasoning to enhance cultural alignment by prompting LLMs with different pretraining data mixtures in Arabic and English. |
| Outcome: | The proposed method enables users to better represent the diversity of human experience and the plurality of different cultures. |
Depth-Wise Attention (DWAtt): A Layer Fusion Method for Data-Efficient Classification (2024.lrec-main)
Copied to clipboard
| Challenge: | Language Models pretrained on large textual data can encode different types of knowledge simultaneously. |
| Approach: | They propose a method to re-surface intermediate layer features from non-final layers by combining them with a concatenation-based layer fusion method. |
| Outcome: | The proposed method outperforms the baseline model on large datasets and shows 3.68 9.73% gain. |