Papers by Pepa Atanasova
Fact Checking with Insufficient Evidence (2022.tacl-1)
Copied to clipboard
| Challenge: | Existing work on how to automate fact checking relies on information obtained from external sources. |
| Approach: | They propose a fluency-preserving method for omitting information from the evidence at the constituent and sentence level and a diagnostic dataset for FC with omitted evidence. |
| Outcome: | The proposed method improves evidence sufficiency prediction by 17.8 F1 score and 2.6 F1 scores. |
DYNAMICQA: Tracing Internal Knowledge Conflicts in Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | LMs are useful in a variety of downstream applications from summarization to fact-checking, often relying on factual knowledge memorized during pre-training. |
| Approach: | They use two knowledge conflict measures and a novel dataset DYNAMICQA to examine the effect of intra-memory conflict on LMs' ability to accept contextual knowledge. |
| Outcome: | The proposed model can accept contextual knowledge with a higher degree of accuracy than models with fewer truth values. |
Explaining Interactions Between Text Spans (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing highlight-based explanations focus on identifying individual important features or interactions only between adjacent tokens or tuples of tokens. |
| Approach: | They propose a multi-annotator dataset of human span interaction explanations for NLU and FC. |
| Outcome: | The proposed method compares human reasoning processes to those of a fine-tuned large language model. |
Self-Critique and Refinement for Faithful Natural Language Explanations (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing work has demonstrated that Large Language Models (LLMs) can self-critique and refine their initial outputs, but this capability remains unexplored for improving explanation faithfulness. |
| Approach: | They propose a framework that enables models to improve the faithfulness of their own explanations through an iterative critique and refinement process without external supervision. |
| Outcome: | The proposed framework reduces unfaithfulness rates in three datasets and four state-of-the-art LLMs by 36% compared to 54.81% for baseline. |
A Diagnostic Study of Explainability Techniques for Text Classification (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing explainability techniques that can be produced post-hoc with already trained models are lacking a definitive guide on how to choose one given a particular task and model architecture. |
| Approach: | They propose to use a list of diagnostic properties to evaluate existing explainability techniques to compare them with human annotations of salient input regions. |
| Outcome: | The proposed list compares a set of explainability techniques on downstream text classification tasks and neural network architectures. |
SOLID: A Large-Scale Semi-Supervised Dataset for Offensive Language Identification (2021.findings-acl)
Copied to clipboard
| Challenge: | toxicity, hate speech, cyberbullying, and cyber-aggression are common themes in social media . authors present a dataset that is limited in size and biased towards offensive language . |
| Approach: | They present an expanded dataset that uses a taxonomy for offensive language identification . they show that using SOLID and OLID yields sizable performance gains . |
| Outcome: | The proposed dataset shows that it performs better than the OLID dataset for two different models. |
bgGLUE: A Bulgarian General Language Understanding Evaluation Benchmark (2023.acl-long)
Copied to clipboard
Momchil Hardalov, Pepa Atanasova, Todor Mihaylov, Galia Angelova, Kiril Simov, Petya Osenova, Veselin Stoyanov, Ivan Koychev, Preslav Nakov, Dragomir Radev
| Challenge: | bgGLUE is a benchmark for evaluating language models on natural language understanding (NLU) tasks in Bulgarian. |
| Approach: | They propose to use a benchmark to evaluate language models on NLU tasks in Bulgarian. |
| Outcome: | The proposed model performs well on sequence labeling tasks, but there is room for improvement for tasks that require more complex reasoning. |
Graph-Guided Textual Explanation Generation Framework (2025.emnlp-main)
Copied to clipboard
Shuzhou Yuan, Jingyi Sun, Ran Zhang, Michael Färber, Steffen Eger, Pepa Atanasova, Isabelle Augenstein
| Challenge: | Existing work has questioned their faithfulness, as they may not accurately reflect the model’s internal reasoning process regarding its predicted answer. |
| Approach: | They propose a Graph-Guided Textual Explanation Generation framework that generates a graph neural network layer that guides the NLE generation and generates explanations with greater semantic and lexical similarity to human-written ones. |
| Outcome: | The proposed framework improves NLE faithfulness by up to 12.12% compared to baseline methods on encoder-decoder and decoder-only models. |
Generating Label Cohesive and Well-Formed Adversarial Claims (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on adversarial triggers for fact checking models reveals weaknesses and flaws of models . universal adversarials often inadvertently invert the meaning of instances they are inserted in . |
| Approach: | They propose a method for automatically generating highly potent, well-formed, label cohesive claims for FC using universal adversarial triggers. |
| Outcome: | The proposed method maintains the directionality and semantic validity of the claim better than previous work on the FEVER dataset. |
Evaluating Input Feature Explanations through a Unified Diagnostic Evaluation Framework (2025.naacl-long)
Copied to clipboard
| Challenge: | Input feature explanations reveal how a model makes decisions based on a specific input. |
| Approach: | They propose a framework that facilitates an automated comparison between highlight and interactive explanations comprised of four diagnostic properties. |
| Outcome: | The proposed framework compares highlight and interactive explanations across two datasets and two models and shows that interactive span explanations outperform other explanation types across most diagnostic properties. |
A Reality Check on Context Utilisation for Retrieval-Augmented Generation (2025.acl-long)
Copied to clipboard
Lovisa Hagström, Sara Vera Marjanovic, Haeun Yu, Arnav Arora, Christina Lioma, Maria Maistro, Pepa Atanasova, Isabelle Augenstein
| Challenge: | Existing studies on LM context utilisation of retrieved information have focused on synthetic text. |
| Approach: | They propose a dataset of unreliable, insufficient and difficult-to-understand contexts with real-world queries and contexts manually annotated for stance to compare them to synthetic datasets. |
| Outcome: | The proposed model outperforms synthetic datasets and exaggerates rare context characteristics, leading to inflated context utilisation results. |
Generating Fact Checking Explanations (2020.acl-main)
Copied to clipboard
| Challenge: | Existing work on automated fact checking is concerned with predicting the veracity of claims based on metadata, social network spread, language used in claims, and, more recently, evidence supporting or denying claims. |
| Approach: | They propose to combine the generation of justifications for verdicts on claims with the multi-task model to optimize both objectives at the same time rather than training them separately. |
| Outcome: | The proposed model improves the informativeness, coverage and overall quality of the generated explanations, rather than training them separately. |
Faithfulness Tests for Natural Language Explanations (2023.acl-short)
Copied to clipboard
Pepa Atanasova, Oana-Maria Camburu, Christina Lioma, Thomas Lukasiewicz, Jakob Grue Simonsen, Isabelle Augenstein
| Challenge: | Existing methods for explaining neural models are misleading as they often present reasons that are unfaithful to the model’s inner workings. |
| Approach: | They propose a counterfactual input editor for inserting reasons that lead to counterfacts but are not reflected by the NLEs. |
| Outcome: | The proposed model can evaluate emerging NLE models, proving a fundamental tool in the development of faithful explanations. |
Revealing the Parametric Knowledge of Language Models: A Unified Framework for Attribution Methods (2024.acl-long)
Copied to clipboard
| Challenge: | Language Models acquire parametric knowledge from their training process, embedding it within their weights. |
| Approach: | They propose a new evaluation framework to quantify and compare the knowledge revealed by Instance Attribution and Neuron Attributions. |
| Outcome: | The proposed evaluation framework compares the knowledge revealed by IA and NA with that of neuron attribution methods. |