Papers by Amal Zouaq
MLMLM: Link Prediction with Mean Likelihood Masked Language Model (2021.findings-acl)
Copied to clipboard
| Challenge: | Knowledge Bases (KBs) are easy to query, verifiable, and interpretable. however, they scale with man-hours and high-quality data. |
| Approach: | They propose to commit the knowledge embedded in MLMs to a KB, making it interpretable . they propose to use a mean likelihood Masked Language Model to compare the likelihood of generating different entities to perform link prediction in a tractable manner. |
| Outcome: | The proposed approach compares the likelihood of generating different entities to perform link prediction in a tractable manner. |
Detecting Languages Unintelligible to Multilingual Models through Local Structure Probes (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in multilingual pretrained models have proven effective at zero-shot transfer to a wide variety of languages, but this transfer is not universal, with many languages not currently understood by multilingual approaches. |
| Approach: | They propose a general approach that requires only unlabelled text to detect which languages are not well understood by a cross-lingual model. |
| Outcome: | The proposed model can detect which languages are not well understood by a multilingual model on 350 low-resource languages. |
Reducing Hallucinations in Language Model-based SPARQL Query Generation Using Post-Generation Memory Retrieval (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are susceptible to hallucinations and out-of-distribution errors when generating KG elements, such as Uniform Resource Identifiers (URIs). |
| Approach: | They propose a SPARQL query-generating framework that uses natural language placeholders and a non-parametric memory module to retrieve and resolve the correct KG URIs. |
| Outcome: | The proposed framework significantly enhances query correctness across various LLMs, datasets, and distribution shifts while achieving the near-complete suppression of URI hallucinations. |
Local Structure Matters Most: Perturbation Study in NLU (2022.findings-acl)
Copied to clipboard
| Challenge: | Recent research shows that neural models are insensitive to word-order perturbations, but other studies suggest that models learn some abstract notion of syntax. |
| Approach: | They develop order-altering perturbations on the order of words, subwords, and characters to analyze their effect on neural models’ performance on language understanding tasks. |
| Outcome: | The proposed models are insensitive to word-order perturbations while the local ordering remains relatively unperturbed. |
TagDebias: Entity and Concept Tagging for Social Bias Mitigation in Pretrained Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to mitigate social bias in pre-trained language models are limited . researchers have found that these models inherit substantial social biases in their pre-training data . |
| Approach: | They propose a method which proposes debiasing a dataset using type tags and fine-tunes PLMs on this debiased dataset. |
| Outcome: | The proposed method improves bias scores on a ranking task . it is based on analyzing type tags and fine-tuning pre-trained models . |
MVP: Minimal Viable Phrase for Long Text Understanding (2024.lrec-main)
Copied to clipboard
| Challenge: | Renewed interest in understanding long texts has sparked interest in benchmarks based on length of input text . |
| Approach: | They propose a new metric that determines the shortest average text length that needs to be preserved to execute the task with limited performance degradation. |
| Outcome: | The proposed benchmarks show that models outperform the previous generation on the QuALITY task due to their limited understanding of long-range dependencies. |
A Copy Mechanism for Handling Knowledge Base Elements in SPARQL Neural Machine Translation (2022.findings-aacl)
Copied to clipboard
| Challenge: | Current architectures are unable to integrate knowledge base schema and handle questions unseen during training rendering them unusable outside the scope of topics covered in the training set. |
| Approach: | They propose to integrate a copy mechanism for neural SPARQL query generation by adding a knowledge base layer and a dynamic knowledge base vocabulary to two Seq2Seq architectures. |
| Outcome: | The proposed model outperforms existing models on state-of-the-art datasets and shows a significant increase in performance. |
Local Structure Matters Most in Most Languages (2022.aacl-short)
Copied to clipboard
| Challenge: | Recent perturbation studies have found unintuitive results on what does and does not matter when performing Natural Language Understanding (NLU) tasks in English. |
| Approach: | They replicate a study on the importance of local structure and relative unimportance of global structure in a multilingual setting. |
| Outcome: | The proposed model replicates a study on the importance of local structure and relative unimportance of global structure in a multilingual setting. |
Ontology Matching Using Convolutional Neural Networks (2020.lrec-1)
Copied to clipboard
| Challenge: | a growing number of ontologies require effective ways to align different ontology models . traditional methods to align ontological models are based on string metrics and structure analysis . but convolutional neural networks can be applied as-is to any domain, allowing for cross-domain applications . |
| Approach: | They propose a method to align ontologies automatically using machine learning techniques . they use convolutional neural networks to perform string matching between class labels . |
| Outcome: | The proposed method achieves state-of-the-art on ontologies from the Ontology Alignment Evaluation Initiative (OAEI) it maintains good performance when tested on a different domain, which could lead to cross-domain applications. |
Combining Domain and Alignment Vectors Provides Better Knowledge-Safety Trade-offs in LLMs (2025.acl-short)
Copied to clipboard
| Challenge: | Large language models (LLMs) excel in specific technical fields, but are not explicitly trained to be safe. |
| Approach: | They propose a model merging-based alignment method that allows for safer domain-specific models that preserve their utility. |
| Outcome: | The proposed method improves safety alignment on LLMs with minimal degradation on domain-specific benchmarks. |
GeoCoder: Solving Geometry Problems by Generating Modular Code through Vision-Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Various vision-language models (VLMs) have made significant progress in multimodal tasks, but they still struggle with geometry problems. |
| Approach: | They propose a vision-language model that leverages modular code-finetuning to generate and execute code using a predefined geometry function library. |
| Outcome: | The proposed model improves geometric reasoning abilities by 16% on a GeomVerse dataset compared to other methods. |
A Deep Dive into the Trade-Offs of Parameter-Efficient Preference Alignment Techniques (2024.acl-long)
Copied to clipboard
| Challenge: | Large language models are pre-trained on trillions of tokens and instruction-tuned or aligned to specific preferences. |
| Approach: | They propose guidelines to help researchers perform more effective parameter-efficient LLM alignment. |
| Outcome: | The proposed methods outperform preference optimization and outperformed pre-trained models on three key axes. |