Papers by Alessandro Lenci
Comparing Static and Contextual Distributional Semantic Models on Intrinsic Tasks: An Evaluation on Mandarin Chinese Datasets (2024.lrec-main)
Copied to clipboard
| Challenge: | Distributional Semantics has undergone significant changes with the introduction of contextualized distributional models. |
| Approach: | They compare static and contextual distributional models for Mandarin Chinese . they find that static models are stronger for some of the classical tasks . |
| Outcome: | The proposed models perform better on some of the classical tasks that consider word meaning independent of context, while contextualized models excel in identifying semantic relations between word pairs and categorization of words into abstract semantic classes. |
Probing for the Usage of Grammatical Number (2022.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models can be used to perform a wide array of NLP tasks, but their encoding is still a mystery. |
| Approach: | They propose a usage-based probing setup to find an encoding that the model actually uses, and propose 'a use-based approach' they propose to use a behavioral task to remove the linguistic property, and to identify which encodes are used to transfer information from a noun to its head verb. |
| Outcome: | The proposed encodings are based on a behavioral task which cannot be solved without the linguistic property. |
“Voices of the Great War”: A Richly Annotated Corpus of Italian Texts on the First World War (2020.lrec-1)
Copied to clipboard
Federico Boschetti, Irene De Felice, Stefano Dei Rossi, Felice Dell’Orletta, Michele Di Giorgio, Martina Miliani, Lucia C. Passaro, Angelica Puddu, Giulia Venturi, Nicola Labanca, Alessandro Lenci, Simonetta Montemagni
| Challenge: | “Voices of the Great War” is the first large corpus of Italian historical texts dating back to the period of First World War. |
| Approach: | "Voices of the Great War" is the first large corpus of Italian historical texts dating back to the period of First World War. |
| Outcome: | The "Voices of the Great War" corpus is the first large corpus of Italian historical texts dating back to the period of First World War. |
Comparing Probabilistic, Distributional and Transformer-Based Models on Logical Metonymy Interpretation (2020.aacl-main)
Copied to clipboard
| Challenge: | Logical metonymies are type clashes between an event-selecting verb and an entity-denoting noun . they are typically interpreted by inferring a hidden event on the basis of contextual cues . |
| Approach: | They propose to use probabilistic and distributional models to model logical metonymy interpretation . they compare models with the best Transformer-based models and some traditional distributional ones . |
| Outcome: | The proposed models perform well on a complex scenario, but low performance on some datasets suggests that logical metonymy is still a challenging phenomenon for computational modeling. |
Did the Cat Drink the Coffee? Challenging Transformers with Generalized Event Knowledge (2021.starsem-1)
Copied to clipboard
Paolo Pedinotti, Giulia Rambelli, Emmanuele Chersoni, Enrico Santus, Alessandro Lenci, Philippe Blache
| Challenge: | Prior work has explored the ability of computational models to predict word semantic fit with a given predicate. |
| Approach: | They compare Transformers Language Models to SDM to assess their performance . they found that TLMs do not capture important aspects of event knowledge . people can discriminate between typical and atypical events, they say . |
| Outcome: | The proposed models can achieve comparable performance to SDM, but they lack important aspects of event knowledge. |
Does BERT Recognize an Agent? Modeling Dowty’s Proto-Roles with Contextual Embeddings (2022.coling-1)
Copied to clipboard
| Challenge: | Contextual embeddings build multidimensional representations of word tokens based on their context of occurrence. |
| Approach: | They propose to map the verb embeddings to an interpretable space of semantic properties built from a linguistic dataset and test their ability to model the semantic properties of the agent of the verbs participating in the alternation. |
| Outcome: | The proposed models can model the semantic properties of the verbs participating in the so-called causative alternation. |
Don’t Invite BERT to Drink a Bottle: Modeling the Interpretation of Metonymies Using BERT and Distributional Representations (2020.coling-main)
Copied to clipboard
| Challenge: | a recent study has shown that metonymy is a productive and systematic process . linguistic and psycholinguistic studies support the idea that metnomic interpretations are based on lexical ambiguity . |
| Approach: | They compare BERT to a generalized event knowledge model to capture the meaning shift associated with metonymy. |
| Outcome: | The proposed model is good at predicting the meaning of metonymic expressions, the authors say . they show that the model can capture the meaning shift associated with metonymy . |
Representing Verbs with Visual Argument Vectors (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing models for verb semantic similarities are based on linguistic data, but they do not register intuitive attributes. |
| Approach: | They evaluated two textual distributional semantic models and a visual one to explore verb semantic similarities. |
| Outcome: | The proposed models extract meaningful information and capture semantic similarity between verbs using visual distributional models. |
We Understand Elliptical Sentences, and Language Models should Too: A New Dataset for Studying Ellipsis and its Interaction with Thematic Fit (2023.acl-long)
Copied to clipboard
| Challenge: | ellipsis is a linguistic phenomenon characterized by the omission of one or more sentence elements. |
| Approach: | They investigated how prototypicality affects the ability of Language Models to handle elliptical sentences . they found that models were better suited to evaluating argument thematic fit . |
| Outcome: | The proposed dataset shows that the models perform better for typical events than for atypical ones in different elliptical contexts. |
Subject Verb Agreement Error Patterns in Meaningless Sentences: Humans vs. BERT (2022.coling-1)
Copied to clipboard
| Challenge: | Existing research shows that humans are prone to making agreement errors with specific constructions. |
| Approach: | They compare the performance of BERT-base and that of humans using crowdsourcing . they find that meaningfulness is stronger for BERT than for humans . |
| Outcome: | The proposed model performs better than humans on a crowdsourcing experiment . |
Word Order Matters When You Increase Masking (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that word order information is not always useful for pre-trained models to solve downstream tasks. |
| Approach: | They examine the effect of removing position encodings on the pre-training objective itself to see whether models can reconstruct position information from co-occurrences. |
| Outcome: | The proposed model can reconstruct position information from co-occurrences without explicit encoding, but not on some tasks. |
Are Word Embeddings Really a Bad Fit for the Estimation of Thematic Fit? (2020.lrec-1)
Copied to clipboard
| Challenge: | In recent years, vectors derived from neural network training have replaced count-based distributional semantic models as a de facto standard for word representation in NLP. |
| Approach: | They propose to evaluate count models and word embeddings on thematic fit estimation by taking into account a larger number of parameters and verb roles and introducing dependency-based embedders in the comparison. |
| Outcome: | The proposed model outperforms count models and word embeddings in thematic fit estimation tasks while introducing dependency-based embedders. |
All-in-one: Understanding and Generation in Multimodal Reasoning with the MAIA Benchmark (2025.findings-emnlp)
Copied to clipboard
Davide Testa, Giovanni Bonetta, Raffaella Bernardi, Alessandro Bondielli, Alessandro Lenci, Alessio Miaschi, Lucia Passaro, Bernardo Magnini
| Challenge: | MAIA evaluates visual language models on video-related tasks using reasoning categories that aim to disentangle language and vision relations. |
| Approach: | a native-italian benchmark is designed for fine-grained investigation of the reasoning abilities of visual language models on videos. |
| Outcome: | The benchmark evaluates visual language models on two aligned tasks and a visual question-answering task. |
PISA: A measure of Preference In Selection of Arguments to model verb argument recoverability (2020.starsem-1)
Copied to clipboard
| Challenge: | a computational model of the semantic recoverability of verb arguments is tested on direct objects and Instruments. |
| Approach: | They propose a computational model of the semantic recoverability of verb arguments . they use a selectional preference model to compute selectional preferences of verbs . |
| Outcome: | The proposed model can predict the recoverability of objects and Instruments at a much cheaper computational cost. |
ExpliCa: Evaluating Explicit Causal Reasoning in Large Language Models (2025.findings-acl)
Copied to clipboard
Martina Miliani, Serena Auriemma, Alessandro Bondielli, Emmanuele Chersoni, Lucia Passaro, Irene Sucameli, Alessandro Lenci
| Challenge: | Large Language Models (LLMs) are increasingly used in tasks requiring interpretive and inferential accuracy. |
| Approach: | They propose a dataset for evaluating Large Language Models in explicit causal reasoning that integrates causal and temporal relations presented in different linguistic orders and explicitly expressed by linguistic connectives. |
| Outcome: | The proposed model performs better than existing models in the domain of causal reasoning. |
Neural Readability Pairwise Ranking for Sentences in Italian Administrative Language (2022.aacl-main)
Copied to clipboard
| Challenge: | Automatic Readability Assessment (ARA) aims at assigning a complexity level to a text, which could help improve the accessibility to information in specific domains, such as the administrative one. |
| Approach: | They propose to use a Neural Pairwise Ranking Model (NPRM) to assess Italian administrative texts by rewriting sentences using three different rewrite strategies. |
| Outcome: | The proposed model is effective in zero-shot scenarios (0.78) and the best results are obtained by adding in-domain data (achieving perfect performance for such sentence pairs). |
Does BERT really agree ? Fine-grained Analysis of Lexical Dependence on a Syntactic Task (2022.findings-acl)
Copied to clipboard
| Challenge: | lexically-independent subject-verb number agreement (NA) is performed by transformer-based neural language models (NLMs) . but when as little as one attractor is present, the model fails to perform lexical generalization . |
| Approach: | They propose to disrupt lexical patterns found in naturally occurring stimuli for each targeted structure in a novel fine-grained analysis of BERT's behavior. |
| Outcome: | The proposed model generalizes well for simple templates, but fails to perform lexically-independent generalization when as little as one attractor is present. |
Are Language Models Sensitive to Semantic Attraction? A Study on Surprisal (2023.starsem-1)
Copied to clipboard
| Challenge: | In psycholinguistics, semantic attraction is a sentence processing phenomenon in which a given argument violates the selectional requirements of a verb but is not perceived by comprehenders due to its attraction to another noun in the same sentence. |
| Approach: | They used autoregressive language models to compute the sentence-level and target phrase-level Surprisal scores of a psycholinguistic dataset on semantic attraction. |
| Outcome: | The proposed models are sensitive to semantic attraction, leading to reduced Surprisal scores, although none perfectly matches the human behavioral pattern. |