Papers by Ali Modarressi

Do We Know What LLMs Don’t Know? A Study of Consistency in Knowledge Probing (2025.findings-emnlp)

Copied to clipboard

Raoyuan Zhao, Abdullatif Köksal, Ali Modarressi, Michael A. Hedderich, Hinrich Schuetze

Challenge:	Existing methods for probing knowledge gaps in large language models are inconsistent and inconsistent.
Approach:	They propose a process based on input variations and quantitative metrics to evaluate probing methods that are inconsistent on knowledge gaps.
Outcome:	The proposed process exposes two dimensions of inconsistency in knowledge gap probing.

Time Course MechInterp: Analyzing the Evolution of Components and Knowledge in Large Language Models (2025.findings-acl)

Copied to clipboard

Ahmad Dawar Hakimi, Ali Modarressi, Philipp Wicke, Hinrich Schuetze

Semantic Text Processing

Challenge:	Large language models acquire and store factual knowledge for interpretability, reliability, efficiency . prior work on factual recall focused on localizing knowledge within transformer parameters .
Approach:	They analyze the evolution of factual knowledge representation in a large language model by tracking its attention heads and feed forward networks over training.
Outcome:	The proposed model acquires and stores factual knowledge over time and is adaptively trained . the proposed model can be pruned, optimized, and transparent .

DecompX: Explaining Transformers Decisions by Propagating Token Decomposition (2023.acl-long)

Copied to clipboard

Ali Modarressi, Mohsen Fayyaz, Ehsan Aghazadeh, Yadollah Yaghoobzadeh, Mohammad Taher Pilehvar

Explainability & Interpretability in NLP

Challenge:	Existing vector-based explanation methods for Transformer-based models are limited in their ability to explain the decisions of multiple layers.
Approach:	They propose a vector-based explanation method based on the construction of decomposed token representations and their successive propagation throughout the model without mixing them in between layers.
Outcome:	The proposed method outperforms existing vector-based and gradient-based methods on transformer-based models by a wide margin.

Exploring the Role of BERT Token Representations to Explain Sentence Probing Results (2021.emnlp-main)

Copied to clipboard

Hosein Mohebbi, Ali Modarressi, Mohammad Taher Pilehvar

Semantic Text Processing

Challenge:	Recent studies have focused on enhancing existing models with the primary objective of improving downstream performance on various NLP tasks.
Approach:	They propose to use BERT to encode meaningful knowledge in token representations to explain probing results.
Outcome:	The proposed model can detect syntactic and semantic abnormalities and distinguish between grammatical number and tense subspaces.

Collapse of Dense Retrievers: Short, Early, and Literal Biases Outranking Factual Evidence (2025.acl-long)

Copied to clipboard

Mohsen Fayyaz, Ali Modarressi, Hinrich Schuetze, Nanyun Peng

Information Retrieval

Challenge:	Notably, when multiple biases combine, models exhibit catastrophic performance degradation, selecting the answer-containing document in less than 10% of cases over a synthetic biased document without the answer.
Approach:	They repurpose a relation extraction dataset to quantify the impact of heuristic biases on retrievers like Dragon+ and Contriever.
Outcome:	The proposed models exhibit catastrophic performance degradation when multiple biases combine, selecting the answer-containing document in less than 10% of cases over a synthetic biased document without the answer.

MEXA: Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignment (2025.findings-acl)

Copied to clipboard

Amir Hossein Kargaran, Ali Modarressi, Nafiseh Nikeghbal, Jana Diesner, François Yvon, Hinrich Schuetze

Multilinguality

Challenge:	Existing benchmarks for multilinguality for English-centric large language models focus on classic tasks or cover a minimal number of languages.
Approach:	They propose a method to assess multilingual capabilities of pre-trained LLMs using parallel sentences.
Outcome:	The proposed method evaluates the multilingual capabilities of pre-trained English-centric models using parallel sentences.

Consistent Document-level Relation Extraction via Counterfactuals (2024.findings-emnlp)

Copied to clipboard

Ali Modarressi, Abdullatif Köksal, Hinrich Schuetze

Information Extraction & Text Mining

Challenge:	Document-level relation extraction models trained on factual data exhibit inconsistent behavior, relying on spurious signals such as specific entities and external knowledge to extract triples.
Approach:	They propose a counterfactual data generation approach for document-level relation extraction datasets using entity replacement to generate triples from factual data.
Outcome:	The proposed approach extracts triples from factual data but fails on counterfactual modification.

ImpliRet: Benchmarking the Implicit Fact Retrieval Challenge (2025.emnlp-main)

Copied to clipboard

Zeinab Sadat Taghavi, Ali Modarressi, Yunpu Ma, Hinrich Schuetze

Information Retrieval

Challenge:	Retrieval systems rely on surface-level cues such as keyword overlap and semantic similarity to evaluate retrieval beyond these shallow signals.
Approach:	They propose a benchmark that shifts the reasoning challenge to query-side processing techniques that can help resolve complexity.
Outcome:	The proposed benchmarks show that document-side reasoning remains a challenge.

GlobEnc: Quantifying Global Token Attribution by Incorporating the Whole Encoder Layer in Transformers (2022.naacl-main)

Copied to clipboard

Ali Modarressi, Mohsen Fayyaz, Yadollah Yaghoobzadeh, Mohammad Taher Pilehvar

Language Models

Challenge:	Existing methods for interpreting the underlying dynamics of Transformers have been criticized for their lack of reliability.
Approach:	They propose a token attribution analysis method that incorporates all components in the encoder block and aggregates this across layers.
Outcome:	The proposed method significantly outperforms existing methods on saliency scores and correlation with gradient-based salience scores.

AdapLeR: Speeding up Inference by Adaptive Length Reduction (2022.acl-long)

Copied to clipboard

Ali Modarressi, Hosein Mohebbi, Mohammad Taher Pilehvar

Challenge:	Pre-trained language models have shown stellar performance in downstream tasks, but their excessive computational costs and high latency hinder their usage in resource-limited settings.
Approach:	They propose a method that dynamically eliminates less contributing tokens through layers, resulting in shorter lengths and consequently lower computational cost.
Outcome:	The proposed method shows speedups up to 22x during inference time without much sacrifice in performance.

Guide the Learner: Controlling Product of Experts Debiasing Method Based on Token Attribution Similarities (2023.eacl-main)

Copied to clipboard

Ali Modarressi, Hossein Amirkhani, Mohammad Taher Pilehvar

Robustness in NLP

Challenge:	Several proposals have been put forward for improving out-of-distribution performance by mitigating dataset biases.
Approach:	They propose a fine-tuning strategy that incorporates the similarity between the main and biased model attribution scores in a Product of Experts (PoE) loss function to further improve OOD performance.
Outcome:	The proposed method improves OOD performance while maintaining in-distribution performance.

Persistent Personas? Role-Playing, Instruction Following, and Safety in Extended Interactions (2026.eacl-long)

Copied to clipboard

Pedro Henrique Luz de Araujo, Michael A. Hedderich, Ali Modarressi, Hinrich Schuetze, Benjamin Roth

Dialogue Systems & Conversational Agents

Challenge:	Persona-assigned large language models are used in education, healthcare and sociodemographic simulations.
Approach:	They propose a protocol that combines long persona dialogues and evaluation datasets to create dialogue-conditioned benchmarks that can robustly measure long-context effects.
Outcome:	The proposed protocol can measure persona fidelity, instruction-following, and safety in long conversations.