Papers by Raquel Fernández

Word Representation Learning in Multimodal Pre-Trained Transformers: An Intrinsic Evaluation (2021.tacl-1)

Challenge:	gradable adjectives of size are relative, i.e., determined by the context.
Approach:	They propose to model how the meaning of gradable adjectives of size can be learned from visually-grounded contexts by using four tasks to determine whether an object is ‘big’ or ‘small’.
Outcome:	The proposed model can learn subtending the meaning of size adjectives, but their performance decreases while moving from simple to more complex tasks.

Copied to clipboard

Sandro Pezzelle, Ece Takmaz, Raquel Fernández

Interpreting Predictive Probabilities: Model Confidence or Human Label Variation? (2024.eacl-short)

Challenge:	Existing models for linguistic representations of words are based on information extracted from large text corpora, and the sensory-motor experiences humans have with the world play an important role in determining word meaning.
Approach:	They propose to use contextualized word representations to learn semantic representations of words that align with human semantic intuitions.
Outcome:	The proposed models are shown to be more efficient on concrete word pairs than on abstract ones.

Copied to clipboard

Joris Baan, Raquel Fernández, Barbara Plank, Wilker Aziz

Construction Repetition Reduces Information Rate in Dialogue (2022.aacl-main)

Challenge:	In modern NLP, neural networks are the de-facto standard to predict complex probability measures from available context.
Approach:	They propose to use a single predictive distribution to evaluate models with disentangled representations of uncertainty about predictions and uncertainty about human labels.
Outcome:	The proposed models are crucial for trustworthy and fair NLP systems, but exploiting a single distribution is limiting.

Copied to clipboard

Mario Giulianelli, Arabella Sinclair, Raquel Fernández

Dialogue Systems & Conversational Agents

Challenge:	We observe that construction usage lowers the information content of utterances.
Approach:	They propose to use construction repetition to mitigate information rate in English open-domain spoken dialogues.
Outcome:	The proposed method lowers the information content of utterances, while increasing the frequency and density of repetition.

Beyond task success: A closer look at jointly learning to see, ask, and GuessWhat (N19-1)

Copied to clipboard

Ravi Shekhar, Aashish Venkatesh, Tim Baumgärtner, Elia Bruni, Barbara Plank, Raffaella Bernardi, Raquel Fernández

Natural Language Interfaces

Challenge:	Existing systems that address the abilities that need to be put to work during conversations are lacking in terms of visual grounding.
Approach:	They propose a visually-grounded dialogue state encoder which integrates visual grounding with dialogue system components.
Outcome:	The proposed system improves the GuessWhat?! game by combining guessing and asking questions with multi-task learning.

Short-Term Meaning Shift: A Distributional Exploration (N19-1)

Copied to clipboard

Marco Del Tredici, Raquel Fernández, Gemma Boleda

Challenge:	a new study examines the phenomenon of short-term meaning shift in online communities . the authors use distributional representations to explore the phenomenon .
Approach:	They propose to use distributional representations to explore short-term meaning shift in online communities.
Outcome:	The proposed model has problems distinguishing meaning shift from referential phenomena, and measures contextual variability to remedy this.

Asking the Right Question at the Right Time: Human and Model Uncertainty Guidance to Ask Clarification Questions (2024.eacl-long)

Copied to clipboard

Alberto Testoni, Raquel Fernández

Natural Language Interfaces

Challenge:	Using model uncertainty as supervision for deciding when to ask may not be the most effective way to resolve model uncertainty.
Approach:	They propose to generate clarification questions based on model uncertainty estimation and compare it to several alternatives to generate questions .
Outcome:	The proposed approach improves the model uncertainty of a collaborative dialogue task and shows that it is more effective than other alternatives.

Words are the Window to the Soul: Language-based User Representations for Fake News Detection (2020.coling-main)

Copied to clipboard

Marco Del Tredici, Raquel Fernández

You Shall Know a User by the Company It Keeps: Dynamic Representations for Social Media Users in NLP (D19-1)

Challenge:	Existing studies on fake news classification focus on textual content, but also social context in which news are consumed.
Approach:	They propose a model that creates representations of individuals on social media based only on the language they produce and uses them to detect fake news.
Outcome:	The proposed model exploits the relationship between language use and connections in the social graph to assess the presence of the Echo Chamber effect in the data.

Copied to clipboard

Marco Del Tredici, Diego Marcheggiani, Sabine Schulte im Walde, Raquel Fernández

Representation Learning

Challenge:	Current approaches to social media modelling ignore the fact that an individual may be part of several communities which are not equally relevant in all communicative situations.
Approach:	They propose a model that captures the sociological phenomenon of homophily and combines it with linguistic information to make a prediction.
Outcome:	The proposed model significantly outperforms existing models on three different tasks and is compared with other models.

Reading Between the Prompts: How Stereotypes Shape LLM’s Implicit Personalization (2025.emnlp-main)

Copied to clipboard

Vera Neplenbroek, Arianna Bisazza, Raquel Fernández

AnaLog: Testing Analytical and Deductive Logic Learnability in Language Models (2022.starsem-1)

Challenge:	Prior work has shown that such inferences can lead to lower quality responses for users assumed to be from minority groups.
Approach:	They analyze LLMs' latent user representations through both model internals and generated answers to targeted user questions.
Outcome:	The proposed models infer demographic attributes based on stereotypical signals, which persists even when the user explicitly identifies with a different demographic group.

Copied to clipboard

Samuel Ryb, Mario Giulianelli, Arabella Sinclair, Raquel Fernández

Language Models

Challenge:	Existing approaches to NLP tasks rely on pre-trained language models, but some do not.
Approach:	They propose a natural language inference task to test pre-trained language models for logical reasoning capabilities.
Outcome:	The proposed language model performs better than other models across logical connectives and reasoning domains, but is sensitive to lexical and syntactic variations in the realisation of logical statements.

RAcQUEt: Unveiling the Dangers of Overlooked Referential Ambiguity in Visual LLMs (2025.emnlp-main)

Copied to clipboard

Alberto Testoni, Barbara Plank, Raquel Fernández

Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Models (2023.emnlp-main)

Challenge:	Existing language models that address ambiguity are limited in their ability to address it . ambiguities are an inherent feature of human language, according to research .
Approach:	They propose a dataset targeting referential ambiguity in image-based question answering . they find that current language models lack robust strategies to deal with ambiguities .
Outcome:	The proposed dataset shows that state-of-the-art models fail to address ambiguity . ambiguities are an inherent feature of human language, according to research .

Copied to clipboard

Jirui Qi, Raquel Fernández, Arianna Bisazza

Triangulating LLM Progress through Benchmarks, Games, and Cognitive Tests (2025.findings-emnlp)

Challenge:	Multilingual large-scale pretrained language models store factual knowledge, but large variations are observed across languages.
Approach:	They propose a ranking-based consistency metric to evaluate cross-lingual consistency of factual knowledge in multilingual PLMs.
Outcome:	The proposed metric evaluates cross-lingual consistency of factual knowledge across languages independently from accuracy.

Copied to clipboard

Filippo Momentè, Alessandro Suglia, Mario Giulianelli, Ambra Ferrari, Alexander Koller, Oliver Lemon, David Schlangen, Raquel Fernández, Raffaella Bernardi

Challenge:	MMLU and BBH are three evaluation paradigms for language learning models . interactive games are superior to standard benchmarks in discriminating models based on human cognitive assessments .
Approach:	They examine three evaluation paradigms: standard benchmarks, interactive games and cognitive tests . they examine whether interactive games are more effective at discriminating LLMs .
Outcome:	The results show that interactive games are superior to standard benchmarks in discriminating models.

Vision-Language Models Align with Human Neural Representations in Concept Processing (2026.eacl-long)

Copied to clipboard

Anna Bavaresco, Marianne De Heer Kloots, Sandro Pezzelle, Raquel Fernández

Not (yet) the whole story: Evaluating Visual Storytelling Requires More than Measuring Coherence, Grounding, and Repetition (2024.findings-emnlp)

Challenge:	Recent studies suggest that transformer-based vision-language models capture the multimodality of concept processing in the human brain.
Approach:	They analysed multiple VLMs employing different strategies to integrate visual and textual modalities, along with language-only counterparts.
Outcome:	The transformer-based vision-language models outperform language-only models in two experimental conditions, while only some outperformed the language-based models.

Copied to clipboard

Aditya Kaushik Surikuchi, Raquel Fernández, Sandro Pezzelle

Psycholinguistics Meets Continual Learning: Measuring Catastrophic Forgetting in Visual Question Answering (P19-1)

Challenge:	Visual storytelling is a task of generating a story for a sequence of several temporally-ordered images or video frames.
Approach:	They propose a method that measures story quality in terms of human likeness regarding three key aspects highlighted in previous work: visual grounding, coherence, and repetitiveness.
Outcome:	The proposed method improves on the foundation model LLaVA but only slightly compared to TAPM, a 50-times smaller visual storytelling model.

Copied to clipboard

Claudio Greco, Barbara Plank, Raquel Fernández, Raffaella Bernardi

Refer, Reuse, Reduce: Generating Subsequent References in Visual and Conversational Contexts (2020.emnlp-main)

Challenge:	Existing methods to overcome catastrophic forgetting in visual question answering models are inadequate, but have received little attention within natural language processing.
Approach:	They devise a set of linguistically-informed visual question answering tasks motivated by psycholinguistics and investigate impact of task difficulty on continual learning.
Outcome:	The proposed models differ in the types of questions they ask and show that task difficulty and order matter.

Copied to clipboard

Ece Takmaz, Mario Giulianelli, Sandro Pezzelle, Arabella Sinclair, Raquel Fernández

Generating Image Descriptions via Sequential Cross-Modal Alignment Guided by Human Gaze (2020.emnlp-main)

Challenge:	Subsequent references exploit the common ground accumulated by the interlocutors and tend to be shorter and reuse expressions that were effective in previous mentions.
Approach:	They propose a model that generates first and subsequent references in visually grounded dialogue . they also implement a reference resolution system to assess the referring effectiveness .
Outcome:	The proposed model produces better, more effective referring utterances than one not grounded in the dialogue context.

Copied to clipboard

Ece Takmaz, Sandro Pezzelle, Lisa Beinborn, Raquel Fernández

I see what you mean: Co-Speech Gestures for Reference Resolution in Multimodal Dialogue (2025.findings-acl)

Challenge:	a long tradition of cognitive studies shows that the interplay between language and vision is complex.
Approach:	They propose an approach to image description generation where visual processing is modelled sequentially.
Outcome:	The proposed model exploits gaze-driven attention to produce better descriptions . it sheds light on human cognitive processes by comparing different ways of aligning gaze with language production.

Copied to clipboard

Esam Ghaleb, Bulat Khaertdinov, Asli Ozyurek, Raquel Fernández

Multimodality

Challenge:	Using representational co-speech gestures, face-to-face interaction participants resolve references to objects using speech and gestures.
Approach:	They propose a multimodal reference resolution task centred on representational gestures . they propose 'self-supervised' pre-training approach to gesture representation learning that grounds body movements in spoken language.
Outcome:	The proposed approach aligns with expert annotations and has significant predictive power.

The PhotoBook Dataset: Building Common Ground through Visually-Grounded Dialogue (P19-1)

Copied to clipboard

Janosch Haber, Tim Baumgärtner, Ece Takmaz, Lieke Gelderloos, Elia Bruni, Raquel Fernández

Analysing Lexical Semantic Change with Contextualised Word Representations (2020.acl-main)

Challenge:	Using the PhotoBook dataset, we investigate shared dialogue history accumulating during conversation . human interlocutors are known to collaboratively establish a shared repository of mutual information during a conversation - this common ground is then used to optimise understanding and communication efficiency.
Approach:	They propose a data-collection task formulated as a collaborative game prompting two online participants to refer to images utilising both their visual context and previously established referring expressions.
Outcome:	The proposed model takes into account shared information accumulated in a reference chain and is important to resolve later descriptions.

Copied to clipboard

Mario Giulianelli, Marco Del Tredici, Raquel Fernández

GROOViST: A Metric for Grounding Objects in Visual Storytelling (2023.emnlp-main)

Challenge:	Existing studies on lexical semantic change have focused on detecting and characterising word meaning shifts using distributional semantic models.
Approach:	They propose a method that exploits the BERT neural language model to obtain representations of word usages, clusters these representations into usage types, and measures change along time with three proposed metrics.
Outcome:	The proposed method captures a variety of synchronic and diachronic linguistic phenomena and is highly reproducible and reproducible.

Copied to clipboard

Aditya Surikuchi, Sandro Pezzelle, Raquel Fernández

Multimodality

Challenge:	Several evaluation metrics for visual storytelling do not consider images at all . authors propose a novel evaluation tool that accounts for cross-modal dependencies and temporal misalignments .
Approach:	They propose a visual storytelling evaluation tool that evaluates visual grounding . they use cross-modal dependencies, temporal misalignments and human intuitions .
Outcome:	The proposed evaluation tool accounts for cross-modal dependencies, temporal misalignments and human intuitions on visual grounding.

Structural Persistence in Language Models: Priming as a Window into Abstract Language Representations (2022.tacl-1)

Copied to clipboard

Arabella Sinclair, Jaap Jumelet, Willem Zuidema, Raquel Fernández

Cross-Lingual Transfer of Debiasing and Detoxification in Multilingual LLMs: An Extensive Investigation (2025.findings-acl)

Challenge:	a rich literature has emerged in the last few years addressing these questions, including whether specific LMs have acquired specific linguistic constructions.
Approach:	They introduce a novel metric and release Prime-LM, a large corpus where they control for various linguistic factors that interact with priming strength.
Outcome:	The proposed model can learn abstract structural information independent of the structure of a sentence and is able to perform tasks that require natural language understanding skills.

Copied to clipboard

Vera Neplenbroek, Arianna Bisazza, Raquel Fernández

The BLA Benchmark: Investigating Basic Language Abilities of Pre-Trained Multimodal Models (2023.emnlp-main)

Challenge:	Prior work has shown that finetuning on specialized datasets can mitigate this behavior, and doing so in English can transfer to other languages.
Approach:	They propose to fine tune generative large language models to provide safe responses to harmful user input and to use direct preference optimization to mitigate toxicity.
Outcome:	The proposed models show that finetuning on specialized datasets reduces biases but also produces fluent and diverse text in non-English languages.

Copied to clipboard

Xinyi Chen, Raquel Fernández, Sandro Pezzelle

Language Models

Challenge:	Pre-trained language-and-vision models have impressive performance in downstream tasks, but it remains unclear whether this improves understanding of image-text interaction.
Approach:	They propose to use BLA to evaluate multimodal models on basic linguistic constructions that even preschool children can typically master.
Outcome:	The proposed model improves basic language skills in a zero-shot learning setting.

Information Value: Measuring Utterance Predictability as Distance from Plausible Alternatives (2023.emnlp-main)

Copied to clipboard

Mario Giulianelli, Sarenne Wallbridge, Raquel Fernández

Challenge:	'information value' quantifies the predictability of an utterance relative to a set of plausible alternatives.
Approach:	They propose a method to obtain interpretable estimates of information value using neural text generators and exploit their psychometric predictive power to investigate the dimensions of predictability that drive human comprehension behaviour.
Outcome:	The proposed method is able to obtain interpretable estimates of information value using neural text generators and exploits their psychometric predictive power to investigate the dimensions of predictability that drive human comprehension behaviour.

Semantic shift in social networks (2021.starsem-1)

Copied to clipboard

Bill Noble, Asad Sayeed, Raquel Fernández, Staffan Larsson

Text Clustering

Challenge:	lexical semantic change manifests differently across different communities, according to a new study . social network analysis is a tool of sociolinguists studying variation and change .
Approach:	They use distributional methods to quantify lexical semantic change and induce a social network on communities based on interactions between members.
Outcome:	The proposed method is based on interactions between members and the community.

Playpen: An Environment for Exploring Learning From Dialogue Game Feedback (2025.emnlp-main)

Copied to clipboard

Nicola Horst, Davide Mazzaccara, Antonia Schmidt, Michael Sullivan, Filippo Momentè, Luca Franceschetti, Philipp Sadler, Sherzod Hakimov, Alberto Testoni, Raffaella Bernardi, Raquel Fernández, Alexander Koller, Oliver Lemon, David Schlangen, Mario Giulianelli, Alessandro Suglia

Dialogue Systems & Conversational Agents

Challenge:	In this paper, we investigate whether Dialogue Games—goal-directed and rule-governed activities driven predominantly by verbal actions—can also serve as a source of feedback signals for learning.
Approach:	They introduce Playpen, an environment for off- and online learning through Dialogue Game self-play, and investigate a representative set of post-training methods: supervised fine-tuning, direct alignment and reinforcement learning with Group Relative Policy Optimization.
Outcome:	The proposed model improves performance on unseen instances, but negatively impacts other skills, while interactive learning shows balanced improvements without loss of skills.

When Models Reason in Your Language: Controlling Thinking Language Comes at the Cost of Accuracy (2025.findings-emnlp)

Copied to clipboard

Jirui Qi, Shan Chen, Zidi Xiong, Raquel Fernández, Danielle Bitterman, Arianna Bisazza

Reasoning

Challenge:	Recent Large Reasoning Models (LRMs) with thinking traces have shown strong performance on English reasoning tasks.
Approach:	They evaluate two leading LRMs with thinking traces on established benchmark XReasoning and propose directions for future research.
Outcome:	The proposed models often revert to English or produce fragmented reasoning in other languages, revealing a substantial gap in the capability of thinking in non-English languages.

Big Generalizations with Small Data: Exploring the Role of Training Samples in Learning Adjectives of Size (D19-64)

Copied to clipboard

Sandro Pezzelle, Raquel Fernández

Challenge:	In previous work, models have been shown to fail in generalizing to unseen adjective-noun combinations.
Approach:	They propose a visual reasoning task dealing with quantities that challenges models to learn the meaning of size adjectives from visually-grounded contexts.
Outcome:	The proposed task is based on a visual reasoning task dealing with quantities and shows that seeing some of the cases during training helps a model understand the rule subtending the task.

Describing Images Fast and Slow: Quantifying and Predicting the Variation in Human Signals during Visuo-Linguistic Processes (2024.eacl-long)

Copied to clipboard

Ece Takmaz, Sandro Pezzelle, Raquel Fernández