Papers by Antske Fokkens
DefVerify: Do Hate Speech Models Reflect Their Dataset’s Definition? (2025.coling-main)
Copied to clipboard
| Challenge: | DefVerify is a 3-step procedure that encodes a user-specified definition of hate speech, quantifies to what extent the model reflects the intended definition, and identifies the point of failure in the workflow. |
| Approach: | They propose a 3-step procedure that encodes a user-specified definition of hate speech and quantifies to what extent the model reflects intended definition. |
| Outcome: | The proposed procedure detects gaps between definition and model behavior when applied to six popular hate speech benchmark datasets. |
Story Trees: Representing Documents using Topological Persistence (2022.lrec-1)
Copied to clipboard
| Challenge: | Topological data analysis (TDA) focuses on the inherent shape of (spatial) data. |
| Approach: | They propose to use topological data analysis to represent document structure as story trees . story trees are hierarchical representations created from semantic vector representations of sentences . |
| Outcome: | The proposed methods can be used to extract summary summaries from news stories using story trees. |
Studying Muslim Stereotyping through Microportrait Extraction (L18-1)
Copied to clipboard
| Challenge: | Research shows that stereotypical ideas are often reflected in language use. |
| Approach: | They propose to use microportraits to investigate stereotyping in the media to explore various dimensions of stereotypation. |
| Outcome: | The proposed system allows social scientists to explore various dimensions of stereotyping compared to more basic models such as word clouds. |
Challenging distributional models with a conceptual network of philosophical terms (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for analyzing philosophical data are not accurate enough to support philosophers . comparative research on concepts should follow a conceptual model approach, authors argue . |
| Approach: | They propose a ground truth for evaluation created by philosophy experts and a blueprint for using DS models in a sound methodological setup. |
| Outcome: | The proposed model does not perform well enough to directly support philosophers yet, but it yields promising directions for future work. |
Methodological Insights in Detecting Subtle Semantic Shifts with Contextualized and Static Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a study comparing static and contextualized language models for subtle semantic shifts in Dutch and English shows that they can detect political connotations and associations. |
| Approach: | They propose a method for detecting subtle semantic shifts between political communities in Dutch and English using static and contextualized language models. |
| Outcome: | The proposed method outperforms static models on a Russian and Spanish task . it relies on behavioral information, specifically the most probable substitutions, instead of geometrical comparison of representations. |
Dynamic Top-k Estimation Consolidates Disagreement between Feature Attribution Methods (2023.emnlp-main)
Copied to clipboard
| Challenge: | Feature attribution scores are used to explain the prediction of a text classifier to users by highlighting a k number of tokens. |
| Approach: | They propose to determine the number of optimal k tokens that should be displayed from sequential properties of attribution scores. |
| Outcome: | The proposed method is dynamic across sentences, method-agnostic, and deals with sentence length bias. |
Better Hit the Nail on the Head than Beat around the Bush: Removing Protected Attributes with a Single Projection (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to remove specific information from embeddings are based on multiple iterations, but multiple iters increase the risk of negative effects. |
| Approach: | They propose two methods that find a single targeted projection: Mean Projection and Tukey Median Projection. |
| Outcome: | The proposed method removes biases by removing information from embedding spaces . it is cleaner than the previous method because it removes separability based on the target . |
Would you describe a leopard as yellow? Evaluating crowd-annotations with justified and informative disagreement (2020.coling-main)
Copied to clipboard
| Challenge: | Existing evaluation methods rely on agreement between annotators, which implies a single correct interpretation. |
| Approach: | They propose an agreement-independent quality metric based on answer-coherence to evaluate on expected disagreement. |
| Outcome: | The proposed model shows that agreement is the most important indicator of quality in semantic annotation tasks. |
The Role of Syntactic Span Preferences in Post-Hoc Explanation Disagreement (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for post-hoc explanations for transformer models disagree with each other . disagreement is often overlooked and the reasons for disagreement are not investigated . |
| Approach: | They propose to use a dynamic *k* approach to estimate syntactic spans to improve agreement between different methods. |
| Outcome: | The proposed method better agrees on syntactic span level, especially for the methods that agree the least with other methods. |
Improving Causal Interventions in Amnesic Probing with Mean Projection or LEACE (2025.findings-acl)
Copied to clipboard
| Challenge: | Amnesic probing examines the influence of specific linguistic information on the behaviour of a model by identifying and removing a property and then testing whether the model can still perform well on next-word prediction. |
| Approach: | They propose to use iterative nullspace projection to remove information by iterating on the target property and then assessing whether the model's performance changes. |
| Outcome: | The proposed methods remove information in a more targeted manner, thereby enhancing the potential for obtaining behavioural explanations through Amnesic Probing. |
Investigating the Robustness of Modelling Decisions for Few-Shot Cross-Topic Stance Detection: A Preregistered Study (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing models for stance detection are not robust enough to be used in a viewpoint-diverse news recommender because the news constantly has new discussion topics. |
| Approach: | They propose to use two stance task definitions (Pro/Con versus Same Side Stance) and two LLM architectures (bi-encoding versus cross-encode) to test model performance. |
| Outcome: | The proposed models outperform the same side-stance definition and other models on stance across different topics. |
Engagement-driven Persona Prompting for Rewriting News Tweets (2025.coling-main)
Copied to clipboard
| Challenge: | Text style transfer is a challenging research task which modifies the linguistic style of a text to meet pre-set objectives such as making the text simpler or more accessible. |
| Approach: | They propose to use large language models to rewrite Dutch news tweets to match specific linguistic styles to achieve a more accessible and accessible text. |
| Outcome: | The proposed prompting strategies perform best for rewriting Dutch news tweets in specific linguistic styles (formal, casual and factual). |
Dealing with Abbreviations in the Slovenian Biographical Lexicon (2022.emnlp-main)
Copied to clipboard
| Challenge: | Abbreviations are a significant challenge for NLP systems because they cause tokenization and out-of-vocabulary errors. |
| Approach: | They propose a method for identifying abbreviations in a Slovenian biographical lexicon . they use a newly developed dataset to evaluate the method against common ad-hoc solutions . |
| Outcome: | The proposed method performs better than ad-hoc solutions on a Slovenian biographical lexicon. |
Introducing Frege to Fillmore: A FrameNet Dataset that Captures both Sense and Reference (2022.lrec-1)
Copied to clipboard
| Challenge: | a widely supported claim in the fields of semantics and philosophy is that meaning arises from the combination of sense and reference. |
| Approach: | They propose a tool that facilitates both referential- and frame annotations of language-independent corpora. |
| Outcome: | The Dutch FrameNet annotation tool facilitates both referential- and frame annotations of language-independent corpora. |
Large-scale Cross-lingual Language Resources for Referencing and Framing (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing corpora that capture language but do not represent actual situations hinder development of systems to resolve cross-document coreference. |
| Approach: | They introduce the concept of cross-lingual referential corpora and propose a framework to analyze framing . they expect to capture larger variation in framation compared to traditional approaches . |
| Outcome: | The proposed project will analyze the framing of incidents in different languages and texts . it expects to capture larger variation in framation compared to traditional approaches . |
Neural Models of Selectional Preferences for Implicit Semantic Role Labeling (L18-1)
Copied to clipboard
| Challenge: | Existing studies on implicit semantic role labeling have been limited due to the lack of training data. |
| Approach: | They propose to use more complex machine learning models trained on a large amount of explicit roles to recover implicit roles. |
| Outcome: | The proposed models outperform baseline models on ON5V dataset, but have mostly negative results . they show that multi-way selectional preference improves results for predicting explicit semantic roles, but harms performance for implicit roles. |