Papers by Thomas Lukasiewicz
Improving Language Models’ Meaning Understanding and Consistency by Learning Conceptual Roles from Dictionary (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have exploited data augmentation or implemented specialised loss functions to alleviate the inconsistent behaviour issue, but they consume expensive training resources and can only handle a certain consistency type. |
| Approach: | They propose a method that allows PLMs to capture accurate meaning by learning precise interrelationships between concepts from word-definition pairs in a dictionary. |
| Outcome: | The proposed method can improve multiple types of consistency and integrate pre-trained knowledge with PLMs’ pre-training knowledge. |
Does the Objective Matter? Comparing Training Objectives for Pronoun Resolution (2020.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models have been used to obtain state-of-the-art results on pronoun resolution. |
| Approach: | They propose to use pre-trained language models to train pronoun resolution models . they compare performance and seed-wise stability of four models that represent four objectives . |
| Outcome: | The proposed model performs best in-domain, while the other objectives perform poorly out-of-domain. |
WikiCREM: A Large Unsupervised Corpus for Coreference Resolution (D19-1)
Copied to clipboard
| Challenge: | Large-scale training sets for pronoun resolution are scarce, since manually labelling data is costly. |
| Approach: | They propose a language-model-based approach to solve pronoun disambiguation problems using a WikiCREM dataset. |
| Outcome: | The proposed model outperforms state-of-the-art approaches on 6 out of 7 datasets. |
GraphSynth: Resolving the Diversity-Reliability Trade-off with Probabilistic Factor Graphs (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models are a scaleable solution for the generation of synthetic data . however, the utility of such data is capped by a critical tension between diversity and factual reliability. |
| Approach: | They propose a framework which leverages a probabilistic factor graph modeling the universe of attributes. |
| Outcome: | The proposed framework outperforms state-of-the-art models with a high structural integrity and a boost in performance on downstream tasks. |
Learning to Model Multimodal Semantic Alignment for Story Visualization (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Story visualization aims to generate sequence of images to narrate each sentence in a multi-sentence story . current methods face semantic misalignment because of their fixed architecture and diversity of input modalities . |
| Approach: | They propose to use a GAN-based generative model to match semantic levels between text and image representations to solve the semantic misalignment problem. |
| Outcome: | Experiments show that the proposed approach improves image quality and story consistency compared with state-of-the-art methods. |
Few-Shot Out-of-Domain Transfer Learning of Natural Language Explanations in a Label-Abundant Setup (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to train models to provide natural language explanations (NLEs) require acquisition of task-specific NLEs, which is time- and resource-consuming. |
| Approach: | They propose a few-shot out-of-domain transfer of NLEs from a parent task to a child task . they propose four methods that cover possible fine-tuning combinations of NLESs and labels . |
| Outcome: | The proposed methods cover the possible fine-tuning combinations of labels and NLEs for the parent and child tasks. |
BECEL: Benchmark for Consistency Evaluation of Language Models (2022.coling-1)
Copied to clipboard
| Challenge: | Existing definitions of behavioural consistency are inconsistent across many studies. |
| Approach: | They propose a behavioural consistency model and propose behavioural taxonomy that classifies consistencies into several sub-categories. |
| Outcome: | The proposed model performs poorly on 19 test cases while exhibiting high inconsistency in many cases. |
KNOW How to Make Up Your Mind! Adversarially Detecting and Alleviating Inconsistencies in Natural Language Explanations (2023.acl-short)
Copied to clipboard
| Challenge: | eIA is an adversarial attack that generates inconsistent natural language explanations (NLEs) a model that generate In-NLE is undesirable, as it has a faulty decision-making process or is prone to inconsistencies. |
| Approach: | They propose an off-the-shelf mitigation method to alleviate inconsistencies by grounding the model into external background knowledge. |
| Outcome: | The proposed method reduces inconsistencies detected by previous models . it is based on external knowledge bases and a novel approach to mitigate inconsistent models based upon the proposed method . |
Controlling Text Edition by Changing Answers of Specific Questions (2021.findings-acl)
Copied to clipboard
| Challenge: | In many situations, we need to change specific content in a document. |
| Approach: | They propose a task where we take a long text, a question, and a target answer as input and output a minimally modified text so that it fits the target answer. |
| Outcome: | The proposed task is based on the existing dataset WIKIBIO and tests on a test set. |
An Empirical Analysis of Parameter-Efficient Methods for Debiasing Pre-Trained Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models inherit more human-like biases from the training corpora, causing computationally expensive problems. |
| Approach: | They propose parameter-efficient methods in combination with counterfactual data augmentation for bias mitigation. |
| Outcome: | The proposed methods are effective in mitigating gender bias, prompt tuning is more suitable for GPT-2 than BERT, and less effective when it comes to racial and religious bias. |
Fool Me Once? Contrasting Textual and Visual Explanations in a Clinical Decision-Support Setting (2024.emnlp-main)
Copied to clipboard
Maxime Kayser, Bayar Menzat, Cornelius Emde, Bogdan Bercean, Alex Novak, Abdalá Morgado, Bartlomiej Papiez, Susanne Gaube, Thomas Lukasiewicz, Oana-Maria Camburu
| Challenge: | XAI models are being used in safety-critical domains, but their use is limited due to their limited transparency and insufficient model robustness. |
| Approach: | They evaluated visual, natural language and a combination of both modalities to examine how users use them. |
| Outcome: | The proposed model is more robust and transparent than previous models. |
Syntactically Rich Discriminative Training: An Effective Method for Open Information Extraction (2022.emnlp-main)
Copied to clipboard
| Challenge: | Open information extraction (OIE) is the task of extracting facts from natural language text. |
| Approach: | They propose a method for computing syntactically rich text embeddings using the structure of dependency trees and a discriminative approach to OIE where tokens in the generated fact are classified as "real" and "fake" they propose to reduce repetitive tokens and improve models' ability to generate implicit facts by a factor of 23%. |
| Outcome: | The proposed method reduces repetitive tokens by a factor of 23% on the CaRB, OIE2016, and LSOIE datasets and improves on augmented datasets. |
Systematic Comparison of Neural Architectures and Training Approaches for Open Information Extraction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Open information extraction (OIE) is a method for extracting facts from text in structured format . alternative formulations allow for longer tuples, but most work focuses on binary predicates only. |
| Approach: | They propose to extract facts from natural language text and represent them as structured triples . they compare different neural network architectures and training approaches . |
| Outcome: | The proposed approach improves the currently best models on the OIE16 benchmark by 0.421 F1 score and 0.420 AUC-PR . |
A Surprisingly Robust Trick for the Winograd Schema Challenge (P19-1)
Copied to clipboard
| Challenge: | The Winograd Schema Challenge (WSC) dataset WSC273 and its inference counterpart WNLI are popular benchmarks for natural language understanding and commonsense reasoning. |
| Approach: | They propose to fine-tune language models on the Winograd Schema Challenge dataset WSC273 and its inference counterpart WNLI to achieve accuracies of 72.5% and 74.7%, respectively. |
| Outcome: | The proposed language models achieve 72.5% and 74.7% accuracy on the WSC273 and WNLI datasets, respectively. |
CircuitSynth: Reliable Synthetic Data Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches lack mechanisms to balance linguistic expressivity with formal guarantees regarding validity and coverage. |
| Approach: | They propose a neuro-symbolic framework that decouples semantic reasoning from surface realization. |
| Outcome: | The proposed framework achieves 100% Schema Validity even in complex logic puzzles where unconstrained baselines fail (12.4%) while outperforming state-of-the-art methods in rare-combination coverage. |
Counter-GAP: Counterfactual Bias Evaluation through Gendered Ambiguous Pronouns (2023.eacl-main)
Copied to clipboard
| Challenge: | a number of studies have focused on gender bias in language models, but these methods fail to detect it. |
| Approach: | They propose to use gender bias in coreference resolution to evaluate gender bias . they propose to construct an annotated quadruple-level dataset with 4008 instances . |
| Outcome: | The proposed method is able to detect gender bias in a quadruple dataset . previous methods failed to detect bias or cancel it, the authors argue . |
Consistency Analysis of ChatGPT (2023.emnlp-main)
Copied to clipboard
| Challenge: | ChatGPT and GPT-4 have been reported to be more reliable and trustworthy, provided they behave similarly to humans. |
| Approach: | They propose to compare ChatGPT and GPT-4 in terms of logically consistent behaviour and the properties of negation, symmetric, and transitive consistency. |
| Outcome: | The proposed models show that they can be more reliable and trustworthy provided they behave similarly to humans. |
Investigating Multi-layer Representations for Dense Passage Retrieval (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Dense retrieval models adopt vectors from the last hidden layer of the document encoder to represent a document, which is in contrast to the fact that representations in different layers of a pre-trained language model contain different kinds of linguistic knowledge and behave differently during fine-tuning. |
| Approach: | They propose to utilize representations from multiple encoder layers to make up the representation of a document, which they denote Multi-layer Representations (MLR). |
| Outcome: | The proposed model outperforms dual encoder, ME-BERT and ColBERT in the single-vector retrieval setting and with other advanced training techniques. |
Make Up Your Mind! Adversarial Generation of Inconsistent Natural Language Explanations (2020.acl-main)
Copied to clipboard
| Challenge: | a promising research direction consists of designing neural models capable of generating natural language explanations for their predictions. |
| Approach: | They propose a framework for sanity checking models against inconsistent explanations . they apply the framework to a state-of-the-art neural natural language inference model . |
| Outcome: | The proposed framework can generate inconsistent explanations on a state-of-the-art model . it also addresses the problem of adversarial attacks with full target sequences . |
Faithfulness Tests for Natural Language Explanations (2023.acl-short)
Copied to clipboard
Pepa Atanasova, Oana-Maria Camburu, Christina Lioma, Thomas Lukasiewicz, Jakob Grue Simonsen, Isabelle Augenstein
| Challenge: | Existing methods for explaining neural models are misleading as they often present reasons that are unfaithful to the model’s inner workings. |
| Approach: | They propose a counterfactual input editor for inserting reasons that lead to counterfacts but are not reflected by the NLEs. |
| Outcome: | The proposed model can evaluate emerging NLE models, proving a fundamental tool in the development of faithful explanations. |
Knowledge Base Completion Meets Transfer Learning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge bases are tedious and require a large amount of labor to build. |
| Approach: | They propose a method that allows for transfer of knowledge from one collection of facts to another without entity or relation matching. |
| Outcome: | The proposed method is the most impactful on small datasets, showing a 6% increase in rank and 65% decrease in rank over the previous best method. |
Beyond Distributional Hypothesis: Let Language Models Learn Meaning-Text Correspondence (2022.findings-naacl)
Copied to clipboard
| Challenge: | Recent evidence shows that large-size pre-trained language models do not satisfy the logical negation property (LNP) However, their reliability is being challenged due to faulty behaviours and incomprehension on number-related representations. |
| Approach: | They propose a new intermediate training task to directly learn meaning text correspondence instead of relying on the distributional hypothesis. |
| Outcome: | The proposed approach outperforms previous models on 7 GLUE tasks and outperformed previous models. |