Papers by Leo Wanner
How much pretraining data do language models need to learn syntax? (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pretraining methods are convenient, but expensive in terms of time and resources. |
| Approach: | They investigate the impact of pretraining data size on the syntactic capabilities of RoBERTa by using syntaktic structural probes to determine whether models pretrained on more data encode a higher amount of syntastic information. |
| Outcome: | The proposed models perform better on part-of-speech tagging, dependency parsing and paraphrase identification. |
On the evolution of syntactic information encoded by BERT’s contextualized representations (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing studies have focused on how linguistic information is encoded in pretrained language models to solve supervised tasks. |
| Approach: | They analyze how the syntax trees are embedded in the geometry of pretrained models for six different tasks, covering all levels of the linguistic structure. |
| Outcome: | The proposed model is able to learn and improve on GLUE and SQUAD, but it lacks the ability to learn the linguistic information required to solve the tasks. |
The Second Multilingual Surface Realisation Shared Task (SR’19): Overview and Evaluation Results (D19-63)
Copied to clipboard
| Challenge: | EMNLP’19 Workshop on Multilingual Surface Realisation aims to stimulate the exploration of advanced neural networks for multilingual sentence generation from Universal Dependency (UD) structures. |
| Approach: | They present results from the SR'19 Shared Task, a multilingual surface realisation task organised as part of the EMNLP'19 Workshop on Multilingual Surface Realisation. |
| Outcome: | The SR'19 shared task was organised as part of the EMNLP'19 Workshop on Multilingual Surface Realisation . it consisted of two tracks with different levels of complexity . the shallow track was offered in eleven, and the deep track in three languages . |
ThemePro: A Toolkit for the Analysis of Thematic Progression (2020.lrec-1)
Copied to clipboard
| Challenge: | Thematic progression is relevant to natural language processing applications dealing with discourse structure, argumentation structure, natural language generation, summarization and topic detection. |
| Approach: | They propose a toolkit for automatic analysis of thematic progression using a web interface. |
| Outcome: | ThemePro provides a visualization of the results including syntactic trees, hierarchical thematicity over propositions and thematic progression over whole texts. |
What the #?*!: Disentangling Hate Across Target Identities (2025.naacl-long)
Copied to clipboard
| Challenge: | Hate speech classifiers do not perform equally well in detecting hateful expressions towards different target identities. |
| Approach: | They propose to use two recently proposed functionality test datasets to analyze the impact of different factors on HS prediction. |
| Outcome: | The proposed classifiers do not perform equally well across different datasets and different target identities. |
Collocation Classification with Unsupervised Relation Vectors (P19-1)
Copied to clipboard
| Challenge: | Existing methods for relation classification are based on word embeddings, but they pose a greater challenge than syntactic and semantic relations. |
| Approach: | They propose a distributional landscape based on word embeddings as a suitable basis for relation classification of collocations . they also conduct experiments on a subset of this benchmark . |
| Outcome: | The proposed dataset is compared to the well known DiffVec dataset and shows that it is more efficient than the standard methods. |
Multilingual Extraction and Categorization of Lexical Collocations with Graph-aware Transformers (2022.starsem-1)
Copied to clipboard
| Challenge: | lexical collocations exhibit varying degrees of frozenness due to their varying degree of frozenncy. |
| Approach: | They propose a sequence tagging BERT-based model enhanced with a graph-aware transformer architecture and evaluate the task of collocation recognition in context. |
| Outcome: | The proposed model encoding syntactic dependencies is useful, and provides insights on differences in collocation typification in English, Spanish and French. |
Assessing the Syntactic Capabilities of Transformer-based Multilingual Language Models (2021.findings-acl)
Copied to clipboard
| Challenge: | Multilingual Transformer-based language models have been shown to be excellent learners in crosslingual transfer tasks. |
| Approach: | They evaluate the syntactic generalization capabilities of BERT and RoBERTa models on English and Spanish tests. |
| Outcome: | The proposed models perform well on English and Spanish tests, and the proposed tests are compared against models on the same language and models on two different languages. |
GPT-HateCheck: Can LLMs Write Better Functional Tests for Hate Speech Detection? (2024.lrec-main)
Copied to clipboard
| Challenge: | HateCheck test cases are generic and have simplistic sentence structures that do not match the real-world data. |
| Approach: | They propose a framework to generate more diverse and realistic functional tests from scratch by instructing large language models. |
| Outcome: | The proposed framework generates more diverse and realistic functional tests from scratch by instructing large language models (LLMs). |
Toxic, Hateful, Offensive or Abusive? What Are We Really Classifying? An Empirical Analysis of Hate Speech Datasets (2020.lrec-1)
Copied to clipboard
| Challenge: | a recent study shows that many definitions are being used for equivalent concepts, making most datasets incompatible. |
| Approach: | They analyze six publicly available datasets to determine their similarity and compatibility . they propose to use Fast Text word vectors to analyze similarity between different datasets . |
| Outcome: | The proposed model performs better on similar datasets and worse on more non-offensive samples. |
Evaluating language models for the retrieval and categorization of lexical collocations (2021.eacl-main)
Copied to clipboard
| Challenge: | Lexical collocations are idiosyncratic combinations of two syntactically bound lexical items. |
| Approach: | They perform an exhaustive analysis of current language models for collocation understanding . they first construct a dataset of apparitions of lexical collocations in context . |
| Outcome: | The proposed models perform well in distinguishing light verb constructions, especially if the collocation’s first argument acts as subject, but often fail to distinguish, first, different syntactic structures within the same semantic category, and second, fine-grained semantic categories which restrict the use of small sets of valid collocates for a given base. |
Directions for NLP Practices Applied to Online Hate Speech Detection (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to address hate speech in online spaces have relied on conventions and practices from NLP. |
| Approach: | They argue that many conventions in NLP are poorly suited for the problem and encourage researchers to develop methods that are more appropriate for the task. |
| Outcome: | The proposed methods are poorly suited for the problem and should be adapted to address the propagation of online harms. |
Generation of a Spanish Artificial Collocation Error Corpus (L18-1)
Copied to clipboard
| Challenge: | collocations are combinations of two elements where one (the base) is freely chosen, despite the limitations of the other (collocate) current tools for collocation error detection and correction focus on collocation validation and identification of miscollocations . |
| Approach: | They propose an algorithm for automatic generation of an artificial collocation error corpus of american English learners of Spanish that includes 17 different types of collocation errors. |
| Outcome: | The proposed algorithm can detect and classify collocation errors in learners' writings . collocation error detection and correction has not received the attention it deserves . |
Compilation of Corpora for the Study of the Information Structure–Prosody Interface (L18-1)
Copied to clipboard
| Challenge: | empirical studies on the Information Structure-prosody interface are scarce . thematicity defines how content is packaged in terms of "what is being talked about" a different view on thematicality is advocated by I. Mel'uk in the context of the MTT. |
| Approach: | They propose a method for the compilation of annotated corpora to study the correspondence between Information Structure and prosody. |
| Outcome: | The proposed method is applied to a corpus of read speech in English annotated with hierarchical thematicity and automatically extracted prosodic parameters. |
Exploring morphology-aware tokenization: A case study on Spanish language modeling (2025.emnlp-main)
Copied to clipboard
| Challenge: | a recent study shows that subword tokenization improves performance of neural language models. |
| Approach: | They propose a linguistically grounded approach to train a tokenizer on morphologically segmented data. |
| Outcome: | The proposed tokenizer improves on a Spanish language model with morphological information. |