Papers by Karin Verspoor
Noisy Label Regularisation for Textual Regression (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods to regularise noisy labels are ineffective in the face of noisy data. |
| Approach: | They propose a method that regularises noisy labels and prevents error propagation from the input layer. |
| Outcome: | The proposed method regularises noisy labels and improves generalisation performance over real-world human-disagreement annotations and randomly-corrupted and data-augmented labels. |
What does it take to bake a cake? The RecipeRef corpus and anaphora resolution in procedural text (2022.findings-acl)
Copied to clipboard
| Challenge: | Current research on anaphora resolution is mostly based on declarative text, such as chemical patents or instruction manuals. |
| Approach: | They propose a framework for anaphora annotation for the chemical domain for modeling anamorphic phenomena in recipes and chemical patents. |
| Outcome: | The proposed framework improves resolution of anaphora in recipes, suggesting transferability of general procedural knowledge. |
M3: Multi-level dataset for Multi-document summarisation of Medical studies (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing summarisation systems are not up to such complex tasks, yet limited tools exist to determine where and why they are failing. |
| Approach: | They propose to use a dataset to evaluate the quality of summarisation systems in the biomedical domain. |
| Outcome: | The proposed model can be used to evaluate the quality of summarisation systems in the biomedical domain. |
Principles from Clinical Research for NLP Model Generalization (2024.naacl-long)
Copied to clipboard
| Challenge: | In clinical research, generalizability depends on (a) internal validity of experiments and (b) external validity or transportability of the results to the wider population. |
| Approach: | They propose to ensure internal validity when building machine learning models in NLP by incorporating learning spurious correlations into their models. |
| Outcome: | The proposed model can perform well on data unseen during training, but drawn from the same distribution or population. |
The patient is more dead than alive: exploring the current state of the multi-document summarisation of the biomedical literature (2022.acl-long)
Copied to clipboard
| Challenge: | Existing evaluation approaches to multi-document summarization of biomedical literature lack consistency and transparency. |
| Approach: | They propose a systematic approach to human evaluation of biomedical summaries and apply it to analyze the summary generated by two current evaluation models. |
| Outcome: | The proposed evaluation framework is based on two state-of-the-art models and examines the summaries generated by the two models to understand the deficiencies of existing evaluation approaches. |
How Robust Are Large Language Models for Clinical Numeracy? An Empirical Study on Numerical Reasoning Abilities in Clinical Contexts (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluations of Large Language Models for clinical numerical reasoning provide limited operation-level coverage and limited robustness of numerical understanding across clinical note formats. |
| Approach: | They propose a benchmarking tool that evaluates four main types of clinical numeracy . they present longitudinal MIMIC-IV vital-sign records in three semantically equivalent representations . |
| Outcome: | The proposed benchmark evaluates four main types of clinical numeracy: value retrieval, arithmetic computation, relational comparison, and aggregation. |
Not another Negation Benchmark: The NaN-NLI Test Suite for Sub-clausal Negation (2022.aacl-main)
Copied to clipboard
| Challenge: | Negation is an important linguistic phenomenon which denotes non-existence, denial, or contradiction. |
| Approach: | They propose a natural language inference test suite to test models for negation . they use a linguistic framework to analyze negation types and constructions . |
| Outcome: | The proposed test suite is more challenging than existing benchmarks on negation . it includes annotation of negation types and constructions grounded in linguistic theory . |
Effects of Human Adversarial and Affable Samples on BERT Generalization (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Limiting quantities of training data is considered a key impediment to achieving generalizability in machine learning. |
| Approach: | They examine the impact of training data quality, not quantity, on a model’s generalizability by comparing human-adversarial and human-affable training samples. |
| Outcome: | The proposed model performance improves with 10-30% h-adversarial instances in text classification and relation extraction tasks. |
Memorization vs. Generalization : Quantifying Data Leakage in NLP Performance Evaluation (2021.eacl-main)
Copied to clipboard
| Challenge: | Public datasets are often used to evaluate the efficacy and generalizability of state-of-the-art methods for many tasks in natural language processing (NLP). |
| Approach: | They identify leakage of training data into test data on several publicly available datasets used to evaluate NLP tasks, including named entity recognition and relation extraction. |
| Outcome: | The proposed model can memorize and generalize data on several publicly available datasets and is compared against previously unseen data. |
WikiUMLS: Aligning UMLS to Wikipedia via Cross-lingual Neural Ranking (2020.coling-main)
Copied to clipboard
| Challenge: | Using a neural reranking model, we can match a UMLS concept with a Wikipedia page, enabling manual alignment with minimal effort. |
| Approach: | They propose a cross-lingual neural reranking model to match a UMLS concept with a Wikipedia page, which achieves a recall@1of 72%, a substantial improvement of 20% over word- and char-level BM25. |
| Outcome: | The proposed model achieves recall@1of 72%, 20% better than word- and char-level BM25, and will facilitate easier access to Wikipedia for health professionals, patients, and NLP systems, including in multilingual settings. |
Evaluating Numeracy of Language Models as a Natural Language Inference Task (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have enhanced their capabilities to solve mathematical problems, but other aspects of numeracy remain underexplored. |
| Approach: | They propose to frame numeracy as a Natural Language Inference task to assess the models’ ability to understand both numbers and language contexts. |
| Outcome: | The proposed model outperforms smaller models in arithmetic tasks, indicating that mathematical reasoning cannot be generalized to other numeracy skills such as number comparison and normalization. |
ChEMU-Ref: A Corpus for Modeling Anaphora Resolution in the Chemical Domain (2021.eacl-main)
Copied to clipboard
| Challenge: | Using a novel annotation scheme, we identify anaphoric references in chemical patents and determine the chemical relation between linked entities. |
| Approach: | They propose a neural approach to anaphora resolution based on coreference and bridging links in chemical patents. |
| Outcome: | The proposed framework can be used to identify anaphoric references in chemical patents and determine the chemical relation between linked entities. |
Revisiting subword tokenization: A case study on affixal negation in large language models (2024.naacl-long)
Copied to clipboard
| Challenge: | Negation is central to language understanding but is not properly captured by modern NLP methods. |
| Approach: | They propose to use subword tokenization methods to detect negation in large language models . they find that models can reliably recognize negation, despite mismatches in tokenization accuracy . |
| Outcome: | The proposed models can detect negation in English using subword tokenization methods despite some mismatches in tokenization accuracy and negation detection performance. |
RADS: Reinforcement Learning-Based Sample Selection Improves Transfer Learning in Low-resource and Imbalanced Clinical Settings (2026.findings-acl)
Copied to clipboard
| Challenge: | Experimental evaluations show that RL methods favor outliers rather than truly informative samples under low-resource and class-imbalanced conditions. |
| Approach: | They propose a robust sample selection strategy using reinforcement learning to identify the most informative samples using a class imbalance approach. |
| Outcome: | The proposed strategy improves model transferability while maintaining robust performance under extreme class imbalance compared to traditional methods. |
Collective Human Opinions in Semantic Textual Similarity (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing benchmarks for semantic textual similarity (STS) use averaged human ratings as gold standard. |
| Approach: | They propose to use a Chinese sentence-to-sentence dataset to study collective human opinions in semantic textual similarity (STS) neither a scalar nor a single Gaussian fits a set of observed judgments adequately, they argue . |
| Outcome: | The proposed dataset does not capture disagreements on individual instances, but rather the confidence over the aggregate dataset. |
Language models are not naysayers: an analysis of language models on negation benchmarks (2023.starsem-1)
Copied to clipboard
| Challenge: | Negation has been shown to be a major bottleneck for masked language models, such as BERT, but whether this finding still holds for larger-sized auto-regressive language models has not been studied comprehensively. |
| Approach: | They evaluate the ability of current-generation auto-regressive language models to handle negation using a wide range of benchmarks and models. |
| Outcome: | The proposed models are compared against a wide range of negation benchmarks and show that they are insensitive to negation, inability to capture the lexical semantics of negations, and failure to reason under negation. |
Improving negation detection with negation-focused pre-training (2022.naacl-main)
Copied to clipboard
| Challenge: | Negation is a common linguistic feature that is crucial in many language understanding tasks. |
| Approach: | They propose a new approach to detect negation in language models using data augmentation and negation masking. |
| Outcome: | The proposed approach improves negation detection performance and generalizability over the strong baseline NegBERT. |
Parallel Corpora for the Biomedical Domain (L18-1)
Copied to clipboard
| Challenge: | Existing corpora of parallel corporata are being used in the biomedical domain . MT is known to support readers' access to textual documents in a language other than their native language . |
| Approach: | They propose to leverage parallel corpora to implement cross-lingual information retrieval or machine translation tools. |
| Outcome: | The proposed corpus is being used in the biomedical task at the conference on machine translation (WMT'16 and WMT'17) it can be leveraged to provide access to health information in languages other than English. |
A Bag-of-concepts Model Improves Relation Extraction in a Narrow Knowledge Domain with Limited Data (N19-3)
Copied to clipboard
| Challenge: | Existing methods for relation extraction on small data sets are time-consuming and expensive. |
| Approach: | They propose an automatic relation extraction task with limited annotated data and a narrow knowledge domain. |
| Outcome: | The proposed method outperforms methods of higher complexity on a small clinical corpus. |
Uncertainty Estimation and Reduction of Pre-trained Models for Text Regression (2022.tacl-1)
Copied to clipboard
| Challenge: | State-of-the-art classification and regression models are often not well calibrated and can be inaccurate. |
| Approach: | They quantify calibration of pre- trained language models for text regression . they apply uncertainty estimates to augment training data in low-resource domains . |
| Outcome: | The proposed model calibrations improve performance and generalizability in low-resource settings. |