Papers by Sara Tonelli
Do LLMs suffer from Multi-Party Hangover? A Diagnostic Approach to Addressee Recognition and Response Selection in Conversations (2024.emnlp-main)
Copied to clipboard
| Challenge: | Conventional evaluation methods often overlook variances in model behavior across different levels of structural complexity on interaction graphs. |
| Approach: | They propose a methodological pipeline to investigate model performance across structural attributes of conversations. |
| Outcome: | The proposed method analyzes the performance of an LLM to classify multi-party conversations . it shows that response selection relies more on the textual content of conversations compared to addressee recognition . |
Features or Spurious Artifacts? Data-centric Baselines for Fair and Robust Hate Speech Detection (2022.naacl-main)
Copied to clipboard
| Challenge: | lexical biases in hate speech detection are limited when applied to real-world data, exhibiting limited out-of-distribution robustness and perpetuating harmful social biase. |
| Approach: | They propose to disentangle spurious and authentic artifacts and analyze their impact on out-of-distribution fairness and robustness. |
| Outcome: | The proposed models show that spurious artifacts require different treatments to attain robustness and fairness in hate speech detection. |
ModaFact: Multi-paradigm Evaluation for Joint Event Modality and Factuality Detection (2025.coling-main)
Copied to clipboard
| Challenge: | NLP studies have mostly dealt with factuality and modality separately . linguistic modality conveys the relationship a situation is supposed to have with respect to wishes, norms, goals, authority, etc. |
| Approach: | They propose a resource with joint factuality and modality information for event-denoting expressions in Italian. |
| Outcome: | The proposed resource is consistent with existing ones and compares classification systems trained on italy's ModaFact dataset and best-performing model. |
Adding Gesture, Posture and Facial Displays to the PoliModal Corpus of Political Interviews (2020.lrec-1)
Copied to clipboard
| Challenge: | a corpus of face-to-face interviews is annotated with facial displays, hand gestures and body posture. |
| Approach: | They introduce a multimodal corpus on top of transcribed face-to-face interviews that presents the annotation of facial displays, hand gestures and body posture. |
| Outcome: | The proposed corpus is extracted from a larger corpus of 56 face-to-face interviews (14 hours) the annotations include facial displays, hand gestures and body posture. |
Building a Multilingual Taxonomy of Olfactory Terms with Timestamps (2022.lrec-1)
Copied to clipboard
| Challenge: | olfactory references play a crucial role in our memory and experiences . but only few works in NLP have attempted to capture this sensory dimension from a computational perspective. |
| Approach: | They describe a process that has led to the semi-automatic development of a taxonomy for olfactory information in four languages (English, French, German and Italian) |
| Outcome: | The proposed taxonomy can be extended using existing language models and n-grams to include olfactory terms in four languages. |
Fine-grained Fallacy Detection with Human Label Variation (2025.naacl-long)
Copied to clipboard
| Challenge: | Fallacy detection is an open challenge in NLP and has shown to be intrinsically difficult for both humans and machines. |
| Approach: | They propose a framework that minimizes annotation errors whilst keeping signals of human label variation. |
| Outcome: | The proposed framework minimizes annotation errors while keeping signals of human label variation. |
Job Unfair: An Investigation of Gender and Occupational Bias in Free-Form Text Completions by LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | a recent study has identified that LLMs are used in domains where they support or replace human decision-making . a systematic review of LLM outputs shows that many facets of social bias remain unaccounted for . |
| Approach: | They propose to disentangle gender and occupational biases in Italian and English as expressed by LLMs. |
| Outcome: | The proposed method captures gender and occupational biases in Italian and English . it also shows that models struggle with gender-neutral expressions, especially beyond English - the authors conclude . |
Hybrid Emoji-Based Masked Language Models for Zero-Shot Abusive Language Detection (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have demonstrated the effectiveness of cross-lingual language model pre-training on NLP tasks. |
| Approach: | They propose a hybrid emoji-based Masked Language Model to leverage eojis across languages to improve the learning of short text messages. |
| Outcome: | The proposed model performs better on German, Italian and Spanish. |
Agreeing to Disagree: Annotating Offensive Language Datasets with Annotators’ Disagreement (2021.emnlp-main)
Copied to clipboard
| Challenge: | supervised learning is a key component of offensive language detection, but there is little attention given to the quality of annotated data. |
| Approach: | They propose to examine the level of agreement among annotators while selecting data to create offensive language datasets, a task involving a high level of subjectivity. |
| Outcome: | The proposed datasets show that annotators' agreement has a strong effect on classifiers performance and robustness. |
Putting Context in Context: the Impact of Discussion Structure on Text Classification (2024.eacl-long)
Copied to clipboard
| Challenge: | Current text classification approaches focus on the content to be classified, but contextual information is neglected in many cases. |
| Approach: | They propose to integrate contextual information into a transformer-based model by feeding it as natural language input into . they also experiment with different amounts of training data and analyse local discussion networks in a privacy-compliant way. |
| Outcome: | The proposed model can be generalized to other datasets and is privacy-compliant. |
Generation-Based Data Augmentation for Offensive Language Detection: Is It Worth It? (2023.eacl-main)
Copied to clipboard
| Challenge: | generative data augmentation has been shown to be effective in offensive language detection but the potential for bias injection has not been investigated. |
| Approach: | They propose to investigate the robustness of models trained on generated data in a variety of data augmentation setups and analyze models using the HateCheck suite. |
| Outcome: | The proposed model training setups on four English offensive language datasets are robust and robust, while the generative DA setups do not present bias injection issues. |
Work Hard, Play Hard: Collecting Acceptability Annotations through a 3D Game (2022.lrec-1)
Copied to clipboard
| Challenge: | Corpus-based studies on acceptability judgements have always been popular thanks to the release of the CoLA corpus, a large-scale corpus of sentences extracted from linguistic handbooks as examples of acceptable/non acceptable phenomena in English. |
| Approach: | They present a 3D video game that was used to collect acceptability judgments on italian sentences and compare them with experts’ acceptability judgements. |
| Outcome: | The proposed game compares the annotations of Italian sentences with those of experts and shows that they are more reliable than crowd-sourced annotations. |
Monolingual and Cross-Lingual Acceptability Judgments with the Italian CoLA corpus (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Acceptability judgments are the most significant source of data in linguistics . however, there are still many open issues regarding methods for collecting and evaluating them. |
| Approach: | They propose to create a corpus of sentences with acceptability judgments using the same approach and the same steps as the English corpus. |
| Outcome: | The proposed corpus contains almost 10,000 sentences with acceptability judgments. |
Real Men are Tough: Evaluating Gender Bias and Sensitivity to Masculinity Norms in LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models exhibit gender bias, but most evaluations focus on downstream stereotypes . a recent study found that explicit endorsement of masculinity norms is low across models . |
| Approach: | They investigate whether large language models rely on traditional masculinity norms as latent priors in gender-biased inference. |
| Outcome: | The findings show that large language models rely on stereotypes as latent priors . the authors used the Male Role Norms Inventory (MRNI) to investigate gender bias . |
Multilingual vs Crosslingual Retrieval of Fact-Checked Claims: A Tale of Two Approaches (2025.emnlp-main)
Copied to clipboard
| Challenge: | Previous work has mostly tackled the task monolingually, i.e., having both the input and the retrieved claims in the same language. |
| Approach: | They examine strategies to improve multilingual and crosslingual performance by selecting negative examples and re-ranking. |
| Outcome: | The proposed methods improve performance on a multilingual and crosslingual dataset. |
Don’t Augment, Rewrite? Assessing Abusive Language Detection with Synthetic Data (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing datasets for abusive language detection and content moderation are limited by regulatory bodies and social media platforms. |
| Approach: | They propose to replace existing datasets in English with synthetic data by rewriting original texts with an instruction-based generative model. |
| Outcome: | The proposed model improves performance in cross-dataset training. |
Why Don’t You Do It Right? Analysing Annotators’ Disagreement in Subjective Tasks (2023.eacl-main)
Copied to clipboard
| Challenge: | Disagreement can reflect different aspects of linguistic annotation, from annotators’ subjectivity to sloppiness or lack of context to interpret a text. |
| Approach: | They propose a taxonomy of possible reasons leading to annotators' disagreement in subjective tasks and manually label part of a Twitter dataset for offensive language detection in english following this taxonomies. |
| Outcome: | The proposed taxonomy of disagreements in linguistic datasets can be used to assess how accurate tweets belonging to different disagreement categories can be classified as offensive or not. |
Delving into Qualitative Implications of Synthetic Data for Hate Speech Detection (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on synthetic data for training models for NLP tasks reports mixed results on subjective tasks such as hate speech detection. |
| Approach: | They propose to use synthetic data to train models for highly subjective tasks such as hate speech detection to investigate the potential and specific pitfalls of using it. |
| Outcome: | The proposed model outperforms models trained with real data on hate speech detection tasks, but it fails to accurately reflect real-world data on linguistic dimensions and results in different class distributions. |