Papers by Steven Schockaert
A Mixture-of-Experts Model for Learning Multi-Facet Entity Embeddings (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for learning entity embeddings from text descriptions leave it to downstream applications to identify these different facets and to select the most relevant ones. |
| Approach: | They propose a model that instead learns several vectors for each entity, each of which captures a different aspect of the considered domain. |
| Outcome: | The proposed model learns several vectors for each entity, each of which intuitively captures a different aspect of the considered domain. |
CONTOR: Benchmarking Strategies for Completing Ontologies with Plausible Missing Rules (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing evaluations focus on distinguishing held-out ontologies from randomly corrupted ones, which often makes the task unrealistically easy. |
| Approach: | They propose to use the common description logic syntax for encoding ontology rules to test their effectiveness on manually annotated hard negatives. |
| Outcome: | The proposed models are compared with existing models and have been evaluated on different ontologies. |
Extracting Conceptual Spaces from LLMs Using Prototype Embeddings (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Conceptual spaces represent entities and concepts using cognitively meaningful dimensions . practical methods for extracting conceptual spaces are currently lacking . |
| Approach: | They propose a strategy in which features are encoded by embedding a description of a corresponding prototype. |
| Outcome: | The proposed approach is highly effective. |
Grouping Entities with Shared Properties using Multi-Facet Prompting and Property Embeddings (2025.emnlp-main)
Copied to clipboard
| Challenge: | Methods for learning taxonomies from data are well-studied, but it is difficult to use them in large domains. |
| Approach: | They propose to use LLMs to describe the different properties that are satisfied by each entity individually and then use pre-trained embeddings to cluster these properties. |
| Outcome: | The proposed model can be used to describe the properties of the entities and group them into clusters. |
Improving Cross-Lingual Word Embeddings by Meeting in the Middle (D18-1)
Copied to clipboard
| Challenge: | Cross-lingual word embeddings are becoming increasingly important in multilingual NLP. |
| Approach: | They propose to apply an additional transformation after initial alignment to align two disjoint monolingual vector spaces. |
| Outcome: | The proposed approach outperforms state-of-the-art models in monolingual and cross-lingual evaluation tasks. |
Self-Supervised Intermediate Fine-Tuning of Biomedical Language Models for Interpreting Patient Case Descriptions (2022.coling-1)
Copied to clipboard
| Challenge: | Existing work has found that biomedical language models lack the knowledge needed for such tasks. |
| Approach: | They propose to fine-tune biomedical language models on the task of predicting masked medical concepts from PubMed abstracts to improve their performance. |
| Outcome: | The proposed strategy improves the performance of biomedical language models on the task of predicting masked medical concepts from patient case descriptions. |
On the Robustness of Unsupervised and Semi-supervised Cross-lingual Word Embedding Learning (2020.lrec-1)
Copied to clipboard
| Challenge: | Cross-lingual word embeddings are vector representations of words in different languages where words with similar meaning are represented by similar vectors, regardless of the language. |
| Approach: | They propose to evaluate multiple cross-lingual word embedding models and compare their strengths and limitations to evaluate their effectiveness. |
| Outcome: | The proposed models perform well with noisy text and language pairs with major differences. |
Modelling Commonsense Properties Using Pre-Trained Bi-Encoders (2022.coling-1)
Copied to clipboard
| Challenge: | Pre-trained language models can capture commonsense properties that are rarely expressed in text. |
| Approach: | They propose to fine-tune language models to explicitly model commonsense properties . they train separate concept and property encoders on extracted hyponym-hypernym pairs and generic sentences . |
| Outcome: | The proposed model can capture commonsense properties with higher accuracy than human models . a new study shows that the model can model commonsensence properties with much higher accuracy . |
Solving Hard Analogy Questions with Relation Embedding Chains (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to model how concepts are related are incomplete and noisy. |
| Approach: | They propose to model relations as paths but associate their edges with relation embeddings. |
| Outcome: | The proposed representations are useful for solving hard analogy questions. |
Interpretable Emoji Prediction via Label-Wise Attention LSTMs (D18-1)
Copied to clipboard
| Challenge: | Emojis are the evolution of characterbased emoticons and are used to express ideas about a myriad of topics. |
| Approach: | They propose a label-wise attention mechanism to better understand emoji prediction . they propose to model e-mails with eojis and then label them based on their meaning . |
| Outcome: | The proposed model improves over baselines and does particularly well when predicting infrequent emojis. |
Unsupervised Learning of Distributional Relation Vectors (P18-1)
Copied to clipboard
| Challenge: | Existing word embedding models rely on co-occurrence statistics to learn vector representations of word meaning. |
| Approach: | They propose a method which directly learns relation vectors from co-occurrence statistics. |
| Outcome: | The proposed method is based on a variant of GloVe, which has an explicit connection between word vectors and PMI weighted co-occurrence vectors. |
Pre-Training Language Models for Identifying Patronizing and Condescending Language: An Analysis (2022.lrec-1)
Copied to clipboard
| Challenge: | Patronizing and Condescending Language (PCL) is a subtle but harmful type of discourse. |
| Approach: | They propose to pre-train PCL detection models on other NLP tasks to improve their detection . they find that performance gains are possible when pre-training on sentiment, harmful language and commonsense morality. |
| Outcome: | The proposed models improve on pre-training on other NLP tasks focusing on sentiment, harmful language and commonsense morality, compared with tasks concentrating on political speech and social justice, the authors show . |
What’s the Meaning of Superhuman Performance in Today’s NLU? (2023.acl-long)
Copied to clipboard
Simone Tedeschi, Johan Bos, Thierry Declerck, Jan Hajič, Daniel Hershcovich, Eduard Hovy, Alexander Koller, Simon Krek, Steven Schockaert, Rico Sennrich, Ekaterina Shutova, Roberto Navigli
| Challenge: | Recent research has focused on developing larger pretrained language models and introducing benchmarks such as SuperGLUE and SQuAD to measure their abilities. |
| Approach: | They propose to use benchmarks such as SuperGLUE and SQUAD to evaluate PLMs' abilities in language understanding, reasoning, and reading comprehension to assess their performance. |
| Outcome: | The proposed benchmarks have serious limitations affecting comparison between humans and PLMs and provide recommendations for fairer and more transparent benchmarks. |
Sentence Selection Strategies for Distilling Word Embeddings from BERT (2022.lrec-1)
Copied to clipboard
| Challenge: | Using language models to learn word embeddings is a key feature of transformer-based language models. |
| Approach: | They propose to use language models to learn high-quality word vectors from as few as 5 to 10 sentences with a careful selection strategy. |
| Outcome: | The proposed strategies can learn high-quality word vectors from as few as 5 to 10 sentences. |
BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language Models Identify Analogies? (2021.acl-long)
Copied to clipboard
| Challenge: | Analogies play a central role in human commonsense reasoning. |
| Approach: | They analyze the capabilities of transformer-based language models on an unsupervised task . they find off-the-shelf language models can identify analogies to a certain extent . |
| Outcome: | The proposed language models outperform word embedding models on an unsupervised task . the best results were obtained with GPT-2 and RoBERTa . |
EnCore: Fine-Grained Entity Typing by Pre-Training Entity Encoders on Coreference Chains (2024.eacl-long)
Copied to clipboard
| Challenge: | Entity typing is the task of assigning semantic types to entities mentioned in text. |
| Approach: | They propose to pre-train an entity encoder such that embeddings of coreferring entities are more similar to each other. |
| Outcome: | The proposed method improves state-of-the-art on fine-grained entity typing and entity extraction. |
Cabbage Sweeter than Cake? Analysing the Potential of Large Language Models for Learning Conceptual Spaces (2023.emnlp-main)
Copied to clipboard
| Challenge: | Conceptual spaces are constructed from a set of quality dimensions, which are usually learned from human judgements, which means that applications of conceptual spaces are limited to narrow domains. |
| Approach: | They propose to use Large Language Models to learn perceptually grounded representations by comparing them to larger models of the BERT family. |
| Outcome: | The proposed models outperform the largest model, despite being 2 to 3 orders of magnitude smaller. |
Distilling Relation Embeddings from Pretrained Language Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models capture a surprisingly rich amount of lexical knowledge, but it is unclear to what extent relation embeddings can be used to encode relational knowledge. |
| Approach: | They found that word vector differences capture lexical relations . relationship embeddings can be used to encode relational knowledge . |
| Outcome: | The results are highly competitive on analogy (unsupervised) and relation classification (supervised) benchmarks, even without any task-specific fine-tuning. |
Collocation Classification with Unsupervised Relation Vectors (P19-1)
Copied to clipboard
| Challenge: | Existing methods for relation classification are based on word embeddings, but they pose a greater challenge than syntactic and semantic relations. |
| Approach: | They propose a distributional landscape based on word embeddings as a suitable basis for relation classification of collocations . they also conduct experiments on a subset of this benchmark . |
| Outcome: | The proposed dataset is compared to the well known DiffVec dataset and shows that it is more efficient than the standard methods. |
Entity or Relation Embeddings? An Analysis of Encoding Strategies for Relation Extraction (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to relation extraction use concatenating embeddings of head and tail entities . however, such representations capture the types of the entities involved, leading to false positives and confusion between relations involving entities of the same type. |
| Approach: | They propose a model which combines [MASK] embeddings with entity embedds to learn relation embeddations. |
| Outcome: | The proposed model outperforms the state-of-the-art on several benchmarks . it uses a self-supervised pre-training strategy which further improves the results. |
Probing Pre-Trained Language Models for Disease Knowledge (2021.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models perform medical reasoning tasks, but standard benchmarks lack examples that require such forms of reasoning. |
| Approach: | They propose a medical reasoning benchmark that uses pre-trained language models to analyze medical reasoning in the biomedical domain. |
| Outcome: | The proposed benchmarks are based on pre-trained language models that perform medical reasoning tasks. |
A RelEntLess Benchmark for Modelling Graded Relations between Named Entities (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing Knowledge Graphs do not cover graded relations, yet they are difficult to draw a line between those that satisfy them and those that do not. |
| Approach: | They propose a benchmark in which entity pairs have to be ranked according to how much they satisfy a given graded relation. |
| Outcome: | The proposed model outperforms several publicly available LLMs and closed conversational models. |
Can Language Models Learn Embeddings of Propositional Logic Assertions? (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for automating reasoning can no longer be used for natural language tasks. |
| Approach: | They propose to use transformer-based language models to reason about knowledge expressed in natural language rather than using LMs to perform reasoning directly. |
| Outcome: | The proposed approach is feasible to some extent, but lacks robustness. |
Ranking Entities along Conceptual Space Dimensions with LLMs: An Analysis of Fine-Tuning Strategies (2024.findings-acl)
Copied to clipboard
| Challenge: | Conceptual spaces represent entities in terms of their primitive semantic features. |
| Approach: | They argue that conceptual spaces should be used alongside knowledge graphs in many settings to model entities in terms of their primitive semantic features. |
| Outcome: | The proposed model can rank entities according to a given conceptual space dimension but ground truth rankings for conceptual space dimensions are rare. |
Word and Document Embedding with vMF-Mixture Priors on Context Word Vectors (P19-1)
Copied to clipboard
| Challenge: | Word embedding models typically learn two types of vectors: target word vectors and context word vector. |
| Approach: | They propose to explicitly impose a cluster structure on context word vectors to improve word embedding models. |
| Outcome: | The proposed model improves word embedding models qualitatively by imposing a cluster structure on the set of context word vectors. |
Relational Word Embeddings (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to learn word embeddings rely on external knowledge bases . however, they are limited by the amount of available relational knowledge . |
| Approach: | They propose to encode relational knowledge in a separate word embedding . this is complementary to a standard word embedded from co-occurrence statistics . |
| Outcome: | The proposed word embedding is complementary to a standard word embed. |
There’s No Such Thing as Simple Reasoning for LLMs (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing work has focused on relatively complex “many-hop” reasoning problems. |
| Approach: | They analyse the performance of fine-tuned LLMs on simple reasoning problems . they find the models remain highly brittle, being susceptible to seemingly innocent perturbations . |
| Outcome: | The proposed models fail on simple reasoning problems, but are highly brittle . they are susceptible to seemingly innocent perturbations, such as adding duplicates to the set of premises and shuffling the order in which the premises are presented. |
Large Language and Reasoning Models are Shallow Disjunctive Reasoners (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) struggle with systematic reasoning on out-of-distribution (OOD) tasks. |
| Approach: | They propose to use a set of constraints to measure OOD generalization to create large reasoning models that can be leveraged to solve real-world problems. |
| Outcome: | The proposed models outperform their LLM counterparts in single-path reasoning tasks but struggle in multi-path setting. |
Knowing the Facts but Choosing the Shortcut: Understanding How Large Language Models Compare Entities (2026.eacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly used for knowledge-based reasoning tasks, yet understanding when they rely on genuine knowledge versus superficial heuristics remains challenging. |
| Approach: | They propose to ask LLMs to compare numerical attributes to find out which country has the highest population, France or Germany. |
| Outcome: | The proposed model comparisons show that heuristics override principled reasoning for larger models, while smaller models show no discrimination. |
Modelling Commonsense Commonalities with Multi-Facet Concept Embeddings (2024.findings-acl)
Copied to clipboard
| Challenge: | Concept embeddings are a useful and efficient mechanism for injecting commonsense knowledge into downstream tasks. |
| Approach: | They propose to model commonalities in concepts by capturing a more diverse range of commonsense properties. |
| Outcome: | The proposed model captures a more diverse range of commonsense properties and improves ontology completion and ultra-fine entity typing tasks. |
Less Is MuRE: Revisiting Shallow Knowledge Graph Embeddings (2025.emnlp-main)
Copied to clipboard
| Challenge: | Knowledge graphs encode knowledge in the form of subject-predicate-object triples, which is notoriously incomplete. |
| Approach: | They propose a framework for analyzing existing shallow knowledge graph models and their extensions. |
| Outcome: | The proposed framework shows that MuRE and ExpressivE are highly competitive . it can capture the same class of rule bases as state-of-the-art region-based embedding models. |
WordNet under Scrutiny: Dictionary Examples in the Era of Large Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Lexical resources are a repository of knowledge and are used for many tasks, including word sense disambiguation and etymology. |
| Approach: | They compare WordNet, the most commonly used lexical resource in NLP, with a variety of dictionaries and examples that were generated by ChatGPT. |
| Outcome: | The most commonly used lexical resource in NLP, with a variety of dictionaries and examples that were generated by ChatGPT. |
What do Deck Chairs and Sun Hats Have in Common? Uncovering Shared Properties in Large Concept Vocabularies (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on decontextualised concept embeddings from language models has focused on capturing taxonomic structure in concepts. |
| Approach: | They propose a strategy for identifying what different concepts have in common with others and representing them in terms of their properties. |
| Outcome: | The proposed approach improves the performance of state-of-the-art models for a multi-label classification problem. |
Inductive Knowledge Graph Completion with GNNs and Rules: An Analysis (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for inductive knowledge graph completion are underperforming . implausible entities are not ranked and only the most informative path is taken into account . |
| Approach: | They propose to use a rule-based approach to find plausible triples missing from a given KG. |
| Outcome: | The proposed models outperform state-of-the-art methods on inductive knowledge graph completion. |
Relation Induction in Word Embeddings Revisited (C18-1)
Copied to clipboard
| Challenge: | Existing approaches to relation induction are based on vector translations, but they are often inadequate for knowledge base completion. |
| Approach: | They propose to use Gaussian to explicitly model the variability of translations and Bayesian linear regression to encode the assumption that there is a linear relationship between the vector representations of related words. |
| Outcome: | The proposed models are based on translations but use Gaussian to model the variability of translations and encode soft constraints on the source and target words that may be chosen. |
Don’t Patronize Me! An Annotated Dataset with Patronizing and Condescending Language towards Vulnerable Communities (2020.coling-main)
Copied to clipboard
| Challenge: | a new dataset is proposed to help develop NLP models to categorize language that is patronizing or condescending towards vulnerable communities. |
| Approach: | They propose to annotate a dataset to help develop NLP models to categorize language that is patronizing or condescending towards vulnerable communities. |
| Outcome: | The proposed dataset supports the development of NLP models to categorize language that is patronizing or condescending towards vulnerable communities. |
RAGAs: Automated Evaluation of Retrieval Augmented Generation (2024.eacl-demo)
Copied to clipboard
| Challenge: | RAGAs are a framework for reference-free evaluation of Retrieval Augmented Generation (RAG) pipelines. |
| Approach: | They propose a framework for reference-free evaluation of Retrieval Augmented Generation pipelines. |
| Outcome: | RAGAs can be used to evaluate RAG pipelines without human annotations . the framework can be useful for faster evaluation cycles given the fast adoption of LLMs based on human annotation. |
Syntactically Aware Neural Architectures for Definition Extraction (N18-2)
Copied to clipboard
| Challenge: | Existing approaches to identify definitional knowledge in text corpora are based on Wikipedia-like definitions. |
| Approach: | They propose to combine Convolutional and Recurrent Neural Networks to train definitional knowledge in text corpora. |
| Outcome: | The proposed models can be applied to more noisy domain-specific corpora. |
Ultra-Fine Entity Typing with Prior Knowledge about Labels: A Simple Clustering Based Strategy (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Ultra-fine entity typing is a task of inferring the semantic types from a large set of fine-grained candidates that apply to a given entity mention. |
| Approach: | They propose to use pre-trained label embeddings to cluster the labels into semantic domains and treat them as additional types. |
| Outcome: | The proposed method improves the performance of existing models with high quality embeddings. |
SeVeN: Augmenting Word Embeddings with Unsupervised Relation Vectors (C18-1)
Copied to clipboard
| Challenge: | Word embeddings use fixed-dimensional vectors to represent the meaning of words. |
| Approach: | They propose a pipeline for learning relation vectors based on word vector averaging and an ad hoc autoencoder. |
| Outcome: | The proposed pipeline can capture aspects of word meaning complementary to word embeddings. |
AMenDeD: Modelling Concepts by Aligning Mentions, Definitions and Decontextualised Embeddings (2024.lrec-main)
Copied to clipboard
| Challenge: | Contextualised Language Models (LMs) improve on word embeddings by encoding meaning of words in context. |
| Approach: | They propose to learn a unified embedding space in which all three types of representations can be integrated. |
| Outcome: | The proposed model outperforms existing approaches in ontology completion tasks. |