Papers by Fahim Dalvi
Analyzing Redundancy in Pretrained Transformer Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that transformer-based deep NLP models are over-parameterized and do not require all the representational power lent by the rich architectural choices during inference. |
| Approach: | They define a notion of Redundancy and propose a feature-based transfer learning procedure which maintains 97% performance while using at-most 10% of the original neurons. |
| Outcome: | The proposed model maintains 97% performance while using 10% of the original neurons. |
How transfer learning impacts linguistic knowledge in deep NLP models? (2021.findings-acl)
Copied to clipboard
| Challenge: | Several researchers have shown that deep NLP models learn non-trivial amount of linguistic knowledge, captured at different layers of the model. |
| Approach: | They propose to fine-tune pre-trained models towards downstream NLP tasks to capture linguistic knowledge. |
| Outcome: | The proposed model is adapted to GLUE tasks and retains linguistic information in the network while forgetting it. |
AraDiCE: Benchmarks for Dialectal and Cultural Capabilities in LLMs (2025.coling-main)
Copied to clipboard
Basel Mousi, Nadir Durrani, Fatema Ahmad, Md. Arid Hasan, Maram Hasanain, Tameem Kabbani, Fahim Dalvi, Shammur Absar Chowdhury, Firoj Alam
| Challenge: | a recent study has found that Arabic is underrepresented in Large Language Models, especially in dialectal variations. |
| Approach: | They propose a benchmark for Arabic Dialect and Cultural Evaluation that evaluates Arabic dialect comprehension and generation. |
| Outcome: | The proposed model outperforms multilingual models on dialect comprehension and generation, but significant challenges persist in dialect identification, generation, and translation. |
One Size Does Not Fit All: Comparing NMT Representations of Different Granularities (N19-1)
Copied to clipboard
| Challenge: | Recent work has shown that contextualized word representations are a viable alternative to simple word prediction tasks. |
| Approach: | They propose to use subword units and characters to model morphology, syntax, and semantics instead of word embeddings. |
| Outcome: | The proposed representations are better for modeling syntax and more robust to noisy input. |
Exploring Alignment in Shared Cross-lingual Spaces (2024.acl-long)
Copied to clipboard
| Challenge: | a new study examines the degree of alignment between languages in multilingual embeddings . cross-lingual embeds are designed to encode linguistic concepts that bridge equivalent semantic meaning . a comprehensive approach is needed to address these questions. |
| Approach: | They employ clustering to uncover latent concepts within multilingual models . they introduce two metrics to quantify alignment and overlap of these concepts . |
| Outcome: | The proposed model can capture linguistic nuances across languages, but is not language-agnostic? the proposed model is able to capture nuances in multiple languages, the authors say. |
Beyond the Leaderboard: Understanding Performance Disparities in Large Language Models via Model Diffing (2025.emnlp-main)
Copied to clipboard
| Challenge: | a recent study shows that benchmarking fails to explain why models outperform others . open-weight large language models have transformed the AI landscape . |
| Approach: | They use model diffing to analyze capability differences between Gemma-2-9b-it and SimPO-enhanced variants. |
| Outcome: | The proposed model diffing approach can provide fine-grained insights beyond leaderboard metrics . it can also help to identify model performance gaps, the authors say . |
Once Correct, Still Wrong: Counterfactual Hallucination in Multilingual Vision-Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing hallucination benchmarks rarely test this failure mode outside Western contexts and English. |
| Approach: | They propose a multimodal benchmark built from images spanning 17 MENA countries . they use a CFHR-based test to measure hallucination beyond raw accuracy . |
| Outcome: | The proposed model is based on images from 17 MENA countries . it measures counterfactual acceptance conditioned on correctly answering the true statement. |
LAraBench: Benchmarking Arabic AI with Large Language Models (2024.eacl-long)
Copied to clipboard
Ahmed Abdelali, Hamdy Mubarak, Shammur Chowdhury, Maram Hasanain, Basel Mousi, Sabri Boughorbel, Samir Abdaljalil, Yassine El Kheir, Daniel Izham, Fahim Dalvi, Majd Hawasly, Nizi Nazar, Youssef Elshahawy, Ahmed Ali, Nadir Durrani, Natasa Milic-Frayling, Firoj Alam
| Challenge: | Recent advances in Large Language Models (LLMs) have significantly influenced the landscape of language and speech research. |
| Approach: | They used GPT-3.5-turbo, GPT-4, BLOOMZ, Jais-13b-chat, Whisper, and USM to tackle 33 distinct tasks across 61 datasets. |
| Outcome: | The proposed model outperforms SOTA models in zero-shot learning, with a few exceptions. |
Analyzing Individual Neurons in Pre-trained Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that deep NLP models capture linguistic knowledge but little attention is paid to individual neurons. |
| Approach: | They conduct a neuron-level analysis of pre-trained neural language models to determine linguistic properties. |
| Outcome: | The proposed model is more localized and disjoint when predicting properties than BERT and others. |
Incremental Decoding and Training Methods for Simultaneous Translation in Neural Machine Translation (N18-2)
Copied to clipboard
| Challenge: | a tunable agent decides the best segmentation strategy for a user-defined BLEU loss and Average Proportion (AP) constraint. |
| Approach: | They propose a tunable agent which decides the best segmentation strategy for a user-defined BLEU loss and average proportion (AP) constraint. |
| Outcome: | The proposed agent outperforms existing Wait-if-diff and Wait-If-worse agents on BLEU with a lower latency. |
LLMeBench: A Flexible Framework for Accelerating LLMs Benchmarking (2024.eacl-demo)
Copied to clipboard
Fahim Dalvi, Maram Hasanain, Sabri Boughorbel, Basel Mousi, Samir Abdaljalil, Nizi Nazar, Ahmed Abdelali, Shammur Absar Chowdhury, Hamdy Mubarak, Ahmed Ali
| Challenge: | Recent development and success of Large Language Models necessitate evaluation of their performance across diverse NLP tasks in different languages. |
| Approach: | They propose a framework that can be customized to evaluate LLMs for any NLP task, regardless of language. |
| Outcome: | The LLMeBench framework can be customized to evaluate LLMs for any NLP task, regardless of language. |
Effect of Post-processing on Contextualized Word Representations (2022.coling-1)
Copied to clipboard
| Challenge: | Post-processing of static embeddings has been shown to improve their performance on both lexical and sequence-level tasks. |
| Approach: | They standardize individual neuron activations using z-score, min-max normalization, and remove top principal components using the all-but-the-top method. |
| Outcome: | The proposed method unwraps vital information present in the representations for both lexical and sequence classification tasks. |
Can LLMs Facilitate Interpretation of Pre-trained Language Models? (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to uncover knowledge encoded within pre-trained language models are limited in terms of scalability and scope of interpretation. |
| Approach: | They propose to use a large language model, ChatGPT, as an annotation tool . they demonstrate that ChatGPt produces accurate and semantically richer annotations . |
| Outcome: | The proposed method produces accurate and semantically richer annotations compared to human annotations. |
Editing Across Languages: A Survey of Multilingual Knowledge Editing (2025.emnlp-main)
Copied to clipboard
| Challenge: | Knowledge Editing is a growing subdomain of model editing focused on ensuring factual edits generalize across languages. |
| Approach: | They present a taxonomy of multilingual knowledge editing methods and benchmarks . authors summarize key findings on method effectiveness and transfer patterns . |
| Outcome: | The proposed methods are compared against available benchmarks and benchmark datasets. |
Similarity Analysis of Contextual Word Representation Models (2020.acl-main)
Copied to clipboard
| Challenge: | Existing and novel similarity measures are used to analyze contextual word representations . different architectures have rather similar representations, but different individual neurons. |
| Approach: | They propose a method to analyze contextual word representation models using similarity analysis. |
| Outcome: | The proposed approach can be used to analyze model similarity without external annotations. |
NxPlain: A Web-based Tool for Discovery of Latent Concepts (2023.eacl-demo)
Copied to clipboard
| Challenge: | Interpretability of deep neural networks has gained a lot of attention in recent years, especially in NLP, where state-of-the-art models are being widely deployed and used in practice. |
| Approach: | They propose to analyze what linguistic and non-linguistic knowledge is learned within deep neural networks and highlight the salient parts of the input. |
| Outcome: | The proposed tool is useful for debugging, unraveling model bias, and for highlighting spurious correlations in a model. |
Analyzing Encoded Concepts in Transformer Language Models (2022.naacl-main)
Copied to clipboard
| Challenge: | a new framework to analyze how latent concepts are encoded in representations learned in pre-trained lan-guage models is proposed . conceptX uses clustering to discover the encoded concepts and align them with a large set of human-defined concepts. |
| Approach: | They propose a framework to analyze how latent concepts are encoded in representations learned within pre-trained lan-guage models. |
| Outcome: | The proposed framework explains encoded concepts by aligning with human-defined concepts. |
On the Transformation of Latent Space in Fine-Tuned NLP Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | a large body of work analyzed the knowledge learned within representations of pre-trained models. |
| Approach: | They use hierarchical clustering to discover latent concepts in representational space . they compare pre-trained and fine-tuned models and perform a thorough analysis . |
| Outcome: | The results show that the model space evolves towards task-specific concepts whereas the lower layers retain generic concepts acquired in the pre-trained model. |
NeuroX Library for Neuron Analysis of Deep NLP Models (2023.acl-demo)
Copied to clipboard
| Challenge: | NeuroX is an open-source toolkit to conduct neuron analysis of natural language processing models. |
| Approach: | They propose a Python toolkit to conduct neuron analysis of natural language processing models. |
| Outcome: | a new open-source toolkit enables neuron analysis of natural language processing models . the framework provides a framework for data processing and evaluation, making it easier for researchers and practitioners to perform neuron analyses. |
AraBench: Benchmarking Dialectal Arabic-English Machine Translation (2020.coling-main)
Copied to clipboard
| Challenge: | Existing efforts to translate Arabic dialects to English are limited due to the lack of evaluation benchmarks. |
| Approach: | They propose an evaluation suite for Arabic to English machine translation using existing Arabic resources. |
| Outcome: | The evaluation suite for Arabic to English machine translation is based on existing evaluation benchmarks. |
Neuron-level Interpretation of Deep NLP Models: A Survey (2022.tacl-1)
Copied to clipboard
| Challenge: | Existing work on deep neural networks has focused on representation analysis, but recent work focused on analyzing neurons within these models. |
| Approach: | They propose to analyze neural networks to uncover linguistic concepts captured by the network . they propose to use a granular approach to analyze neurons within these models . |
| Outcome: | The proposed method combines methods to discover and understand neurons in a network with evaluation methods. |
Fine-grained Interpretation and Causation Analysis in Deep NLP Models (2021.naacl-tutorials)
Copied to clipboard
| Challenge: | Despite the proven efficacy of deep neural networks at-large, their opaqueness is a major cause of concern. |
| Approach: | They will present research work on interpreting fine-grained components of a neural network model from two perspectives, i) fine-grain interpretation, and ii) causation analysis. |
| Outcome: | This paper presents work on interpreting fine-grained components of a neural network model from two perspectives, i) fine-grain interpretation, and ii) causation analysis. |
Latent Concept-based Explanation of NLP Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing attempts to explain deep learning models rely on input features, such as the words . however, such explanations are often less informative due to the discrete nature of words and lack of contextual verbosity. |
| Approach: | They propose a method that generates explanations for predictions based on latent concepts . they map the representations of salient input words into the training latent space . |
| Outcome: | The proposed method generates explanations for predictions based on latent concepts . it maps representations of salient input words into training latent space . |
Fighting the COVID-19 Infodemic: Modeling the Perspective of Journalists, Fact-Checkers, Social Media Platforms, Policy Makers, and the Society (2021.findings-emnlp)
Copied to clipboard
Firoj Alam, Shaden Shaar, Fahim Dalvi, Hassan Sajjad, Alex Nikolov, Hamdy Mubarak, Giovanni Da San Martino, Ahmed Abdelali, Nadir Durrani, Kareem Darwish, Abdulaziz Al-Homaid, Wajdi Zaghouani, Tommaso Caselli, Gijs Danoe, Friso Stolk, Britt Bruntink, Preslav Nakov
| Challenge: | a dataset of 16K manually annotated tweets is used to analyze disinformation . the democratic nature of social media has raised questions about the quality and the factuality of the information that is shared on these platforms. |
| Approach: | They use a dataset of manually annotated tweets to analyze COVID-19 disinformation . they show that tweets contain fake cures, rumors, conspiracy theories and xenophobia . |
| Outcome: | The proposed dataset shows that it is useful in monolingual vs. multilingual settings. |
Scaling up Discovery of Latent Concepts in Deep NLP Models (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing studies on pre-trained language models focus on clustering latent spaces within pre-training models . authors propose metrics for assessing the quality of discovered latent concepts . |
| Approach: | They propose metrics to assess the quality of discovered latent concepts . they propose to scale latent concept discovery to larger datasets and models . |
| Outcome: | The proposed clustering algorithms improve performance while maintaining quality of the obtained concepts. |