Papers by Abbas Ghaddar
Context-aware Adversarial Training for Name Regularity Bias in Named Entity Recognition (2021.tacl-1)
Copied to clipboard
| Challenge: | Name Regularity Bias is a problem in NER models that use contextual information to predict the type of an ambiguous entity. |
| Approach: | They propose a model-agnostic training method that adds learnable adversarial noise to some entity mentions to improve their accuracy. |
| Outcome: | The proposed method outperforms feature-based models on name regularity bias . it adds learnable adversarial noise to some entity mentions, leading to gains . |
End-to-End Self-Debiasing Framework for Robust NLU Training (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing models incorporate dataset biases leading to strong performance on in-distribution test sets but poor performance on out-of-distortion (OOD) tests. |
| Approach: | They propose a debiasing framework where the shallow representations of the main model are used to derive a bias model and both models are trained simultaneously. |
| Outcome: | The proposed framework outperforms existing approaches on three well-studied NLU tasks while still delivering high in-distribution performance. |
RW-KD: Sample-wise Loss Terms Re-Weighting for Knowledge Distillation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Knowledge Distillation (KD) is used to compress the pre-training and task-specific fine-tuning phases of large neural language models. |
| Approach: | They propose a sample-wise loss weighting method that re-weights the two losses for each sample. |
| Outcome: | The proposed method outperforms existing methods on 7 datasets of the GLUE benchmark. |
OTTAWA: Optimal TransporT Adaptive Word Aligner for Hallucination and Omission Translation Errors Detection (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for detecting hallucinations and omissions in Machine Translation systems focus on analyzing the model’s internal states or relying on external tools. |
| Approach: | They propose an Optimal Transport-based word aligner specifically designed to enhance the detection of hallucinations and omissions in Machine Translation systems. |
| Outcome: | The proposed method is competitive with state-of-the-art methods across 18 language pairs on the HalOmi benchmark and shows promising features. |
CHARP: Conversation History AwaReness Probing for Knowledge-grounded Dialogue Systems (2024.findings-acl)
Copied to clipboard
Abbas Ghaddar, David Alfonso-Hermelo, Philippe Langlais, Mehdi Rezagholizadeh, Boxing Chen, Prasanna Parthasarathi
| Challenge: | CHARP is a testbed for knowledge-grounded dialogue evaluation of models trained on FaithDial data. |
| Approach: | They propose a testbed for evaluating models trained on FaithDial with annotation artifacts that may bias models towards completely ignoring the conversation history. |
| Outcome: | The proposed model fails to accurately evaluate the conversational history and lacks hallucination detection. |
Contextualized Word Representations from Distant Supervision with and for NER (D19-55)
Copied to clipboard
| Challenge: | Existing word embeddings for named entity recognition are stacked with traditional ones for downstream tasks. |
| Approach: | They propose a special type of contextualized word representation that is learned from distant supervision annotations and dedicated to named entity recognition. |
| Outcome: | The proposed representation surpasses the existing representations and is complementary to existing embeddings. |
SEDAR: a Large Scale French-English Financial Domain Parallel Corpus (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing approaches for neural machine translation use small amount of data or monolingual data. |
| Approach: | They describe acquisition, preprocessing and characteristics of a large English-French parallel corpus for the financial domain. |
| Outcome: | The proposed corpus contains 8.6 million high quality sentence pairs . the first release of the corpus is available on github. |
Universal-KD: Attention-based Output-Grounded Intermediate Layer Knowledge Distillation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for intermediate layer matching are limited due to huge over-parameterization . |
| Approach: | They propose to match intermediate layers of teacher and student in output space via attention-based layer projection. |
| Outcome: | The proposed method outperforms existing methods on GLUE tasks. |
Robust Lexical Features for Improved Neural Network Named-Entity Recognition (C18-1)
Copied to clipboard
| Challenge: | Named-Entity Recognition (NER) uses word embeddings to extend, rather than replace, hand-crafted features. |
| Approach: | They propose to embed words and entity types into a low-dimensional vector space and compute a feature vector representing each word offline. |
| Outcome: | The proposed representations outperform existing models and achieve state-of-the-art performance. |
Revisiting Pre-trained Language Models and their Evaluation for Arabic Natural Language Processing (2022.emnlp-main)
Copied to clipboard
Abbas Ghaddar, Yimeng Wu, Sunyam Bagga, Ahmad Rashid, Khalil Bibi, Mehdi Rezagholizadeh, Chao Xing, Yasheng Wang, Xinyu Duan, Zhefeng Wang, Baoxing Huai, Xin Jiang, Qun Liu, Phillippe Langlais
| Challenge: | Existing pre-trained language models are not well-explored and are not reproducible in the literature. |
| Approach: | They propose to improve existing Arabic language pre-trained language models using a more methodical approach. |
| Outcome: | The proposed models outperform existing models on ALUE, a leaderboard-powered benchmark for Arabic NLU and NLG tasks. |
Transforming Wikipedia into a Large-Scale Fine-Grained Entity Type Corpus (L18-1)
Copied to clipboard
| Challenge: | et al. (2017): WiFiNE annotated with fine-grained entity types . lack of a well-established training corpus makes it difficult to manually annotate the amount of data needed for training. |
| Approach: | They propose an English corpus annotated with fine-grained entity types based on Wikipedia . they use heuristics to build a large, high quality, annotating corpus using 2 manually annotized benchmarks . |
| Outcome: | The proposed system outperforms the existing systems with two datasets and gains a 2.8 macro F1 score. |
CILDA: Contrastive Data Augmentation Using Intermediate Layer Knowledge Distillation (2022.coling-1)
Copied to clipboard
Md Akmal Haidar, Mehdi Rezagholizadeh, Abbas Ghaddar, Khalil Bibi, Phillippe Langlais, Pascal Poupart
| Challenge: | Knowledge distillation (KD) is an efficient framework for compressing large-scale pre-trained language models. |
| Approach: | They propose a data augmentation technique tailored for knowledge distillation based on contrastive loss to improve masked adversarial data augmented by intermediate layer matching. |
| Outcome: | The proposed technique outperforms state-of-the-art methods on the GLUE benchmark and in an out-of domain evaluation. |
RAIL-KD: RAndom Intermediate Layer Mapping for Knowledge Distillation (2022.findings-naacl)
Copied to clipboard
Md Akmal Haidar, Nithin Anchuri, Mehdi Rezagholizadeh, Abbas Ghaddar, Philippe Langlais, Pascal Poupart
| Challenge: | Existing methods for intermediate layer knowledge distillation suffer from computational burdens and engineering efforts for setting up a proper layer mapping. |
| Approach: | They propose a method where intermediate layers from teacher and student models are randomly selected to be distilled into intermediate layers of student models. |
| Outcome: | The proposed method outperforms state-of-the-art intermediate layer knowledge distillation methods on GLUE tasks and out-of domain test sets. |
Towards Zero-Shot Knowledge Distillation for Natural Language Processing (2021.emnlp-main)
Copied to clipboard
| Challenge: | Knowledge distillation (KD) is a common knowledge transfer algorithm used for model compression across a variety of deep learning based natural language processing (NLP) solutions. |
| Approach: | They propose to use teacher training data for model compression . they investigate six tasks and find they can achieve between 75% and 92% of the teacher’s classification score while compressing the model 30 times. |
| Outcome: | The proposed solution achieves between 75% and 92% of the teacher’s classification score while compressing the model 30 times. |
CHIQ: Contextual History Enhancement for Improving Query Rewriting in Conversational Search (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in task-solving capabilities of Large Language Models (LLMs) have motivated researchers to integrate these models into existing conversational search systems. |
| Approach: | They propose a method that leverages the capabilities of large language models to resolve ambiguities in conversation history before query rewriting. |
| Outcome: | The proposed method leads to state-of-the-art results across most settings compared with closed-source LLMs. |
BOSCH: Black-Box Binary Optimization for Short-Context Attention-Head Selection in LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Existing hybridization schemes use sliding-window attention (SWA) to reduce KV cache usage and improve latency. |
| Approach: | They propose a training-free method that decomposes a large neighborhood search problem into three subproblems and a method that uses black-box binary optimization for short-context head selection. |
| Outcome: | Extensive experiments on 4 LLMs show that BOSCH outperforms layer-level heuristics and 6 strong static head-level methods with larger gains at higher SWA ratios. |
Integral Transformer: Denoising Attention, Not Too Much Not Too Little (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to reduce attention noise by integrating signals from logit distributions are prone to attention noise. |
| Approach: | They propose a self-attention mechanism that integrates signals from the logit distribution to denoise attention. |
| Outcome: | The proposed model outperforms vanilla, Cog, and Differential attention variants on knowledge and reasoning benchmarks. |
CHARPEVAL: Benchmarking Large Language Models’ Contextual Reasoning in Knowledge-Grounded Dialogue (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks that evaluate the ability of Large Language Models (LLMs) to perform contextualized reasoning in knowledge-grounded dialogue scenarios are lacking. |
| Approach: | They propose a benchmark to evaluate the ability of Large Language Models to perform contextualized reasoning in knowledge-grounded dialogue scenarios. |
| Outcome: | The proposed benchmark shows that open-weight LLMs are ineffective at reasoning over discontinuous chunks of text across the input. |
EWEK-QA : Enhanced Web and Efficient Knowledge Graph Retrieval for Citation-based Question Answering Systems (2024.acl-long)
Copied to clipboard
Mohammad Dehghan, Mohammad Alomrani, Sunyam Bagga, David Alfonso-Hermelo, Khalil Bibi, Abbas Ghaddar, Yingxue Zhang, Xiaoguang Li, Jianye Hao, Qun Liu, Jimmy Lin, Boxing Chen, Prasanna Parthasarathi, Mahdi Biparva, Mehdi Rezagholizadeh
| Challenge: | citation-based QA systems are suffering from two shortcomings . they usually rely only on web as a source of extracted knowledge and external knowledge sources can hamper the efficiency of the system. |
| Approach: | They propose to use a web-based knowledge graph retrieval solution to enrich extracted knowledge fed to a citation-based QA system. |
| Outcome: | The proposed model outperforms open-source state-of-the-art models in 7 quantitative and human evaluation tasks. |
AraMUS: Pushing the Limits of Data and Model Scale for Arabic Natural Language Processing (2023.findings-acl)
Copied to clipboard
Asaad Alghamdi, Xinyu Duan, Wei Jiang, Zhenhai Wang, Yimeng Wu, Qingrong Xia, Zhefeng Wang, Yi Zheng, Mehdi Rezagholizadeh, Baoxing Huai, Peilun Cheng, Abbas Ghaddar
| Challenge: | Developing monolingual large Pre-trained Language Models (PLMs) is shown to be very successful in handling different tasks in Natural Language Processing (NLP). |
| Approach: | They present AraMUS, the largest Arabic PLM with 11B parameters trained on 529GB of high-quality Arabic textual data. |
| Outcome: | The proposed model achieves state-of-the-art performance on a diverse set of Arabic classification and generative tasks. |