Papers by Albert Gatt
Is Shortest Always Best? The Role of Brevity in Logic-to-Text Generation (2023.starsem-1)
Copied to clipboard
| Challenge: | Logical formulae are essential for scholars in many fields, including linguistics and artificial intelligence. |
| Approach: | They propose to use a Quantified Boolean Formulae (QBFs) problem to find the shortest formulae as input for a "logic-to-text" generation system. |
| Outcome: | The proposed approach improves the comprehensibility and fluency of the generated texts. |
LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks (2025.acl-short)
Copied to clipboard
Anna Bavaresco, Raffaella Bernardi, Leonardo Bertolazzi, Desmond Elliott, Raquel Fernández, Albert Gatt, Esam Ghaleb, Mario Giulianelli, Michael Hanna, Alexander Koller, Andre Martins, Philipp Mondorf, Vera Neplenbroek, Sandro Pezzelle, Barbara Plank, David Schlangen, Alessandro Suglia, Aditya K Surikuchi, Ece Takmaz, Alberto Testoni
| Challenge: | Existing evaluations of NLP models with LLMs are based on human judgments . however, there are concerns about their validity and reproducibility in proprietary models . |
| Approach: | They evaluate 11 current LLMs for their ability to replicate annotations. they show substantial variance across models and datasets. |
| Outcome: | The proposed model can replicate human annotations on 20 NLP datasets and show substantial variance across models and datasets. |
Synthetic Eggs in Many Baskets: The Impact of Synthetic Data Diversity on LLM Fine-Tuning (2026.findings-acl)
Copied to clipboard
| Challenge: | Increasing demand for training data is causing language models to be trained on synthetic data, a new study finds . fine-tuning models on synthetic datasets reduces self-preference bias . |
| Approach: | They investigate the impact of diversity of synthetic data on fine-tuned large language models. |
| Outcome: | The proposed model can mitigate distribution collapse, maintain diversity of output distribution, and reduce self-preference bias. |
Burn After Reading: Do Multimodal Large Language Models Truly Capture Order of Events in Image Sequences? (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks focus on single image settings, but some focus on multi-image settings. |
| Approach: | They introduce the TempVS benchmark which focuses on temporal grounding and reasoning capabilities of Multimodal Large Language Models in image sequences. |
| Outcome: | The proposed model performs poorly compared to human models in vision and language tasks. |
VAQUUM: Are Vague Quantifiers Grounded in Visual Data? (2025.findings-acl)
Copied to clipboard
| Challenge: | a dataset containing 20,300 human ratings on quantified statements is used to evaluate the appropriateness of vague quantifiers in visual contexts. |
| Approach: | They use a visual-language-models-based dataset to evaluate the appropriateness of vague quantifiers. |
| Outcome: | The proposed model is based on a visual-visual-language-model-based dataset . it shows that the model is compatible with humans when producing or judging vague quantifiers . |
Grounded Textual Entailment (C18-1)
Copied to clipboard
Hoa Trong Vu, Claudio Greco, Aliia Erofeeva, Somayeh Jafaritazehjan, Guido Linders, Marc Tanti, Alberto Testoni, Raffaella Bernardi, Albert Gatt
| Challenge: | Existing models for entailment analysis are not performing well in visual information-based models. |
| Approach: | They propose to use a visual representation of the Textual Entailment task to compare visual-grounded models with a multimodal version of the SNLI dataset. |
| Outcome: | The proposed model performs better when there is an image of the “world” or “situation” . |
MASRI-HEADSET: A Maltese Corpus for Speech Recognition (2020.lrec-1)
Copied to clipboard
Carlos Daniel Hernandez Mena, Albert Gatt, Andrea DeMarco, Claudia Borg, Lonneke van der Plas, Amanda Muscat, Ian Padovani
| Challenge: | Maltese is the national language of Malta and is spoken by approximately 500,000 people. |
| Approach: | They present the first spoken Maltese corpus designed purposely for Automatic Speech Recognition (ASR) it consists of 8 hours of speech paired with text, recorded by using short text snippets in a laboratory environment. |
| Outcome: | The MASRI-HEADSET corpus was developed by the MASR project at the University of Malta. |
When Models Decide and When They Bind: A Two-Stage Computation for Multiple-Choice Question Answering (2026.findings-acl)
Copied to clipboard
| Challenge: | Multiple-choice question answering (MCQA) is easy to evaluate but adds a meta-task . prior work has shown that language models exhibit selection biases for particular option identifiers such as the label "A" |
| Approach: | They find that option-boundary residual states contain strong linearly decodable signals . winning content position becomes decoded after final option is processed . |
| Outcome: | The proposed model solves the problem and outputs the symbol that represents the answer. |
Disentangling the Roles of Representation and Selection in Data Pruning (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for data pruning involve many different design choices, which have not been systematically studied. |
| Approach: | They decompose data pruning into two key components: data representation and selection algorithm . theoretical and empirical results highlight crucial role of representations . |
| Outcome: | The proposed method can be used to train models with less data. |
A Systematic Analysis of Large Language Models as Soft Reasoners: The Case of Syllogistic Inferences (2024.emnlp-main)
Copied to clipboard
| Challenge: | syllogistic reasoning is a deductive reasoning skill that is crucial in everyday problem-solving and decision-making experiences. |
| Approach: | They propose to study the reasoning abilities of Large Language Models (LLMs) they propose to use supervised fine-tuning and chain-of-thought reasoning to investigate their results. |
| Outcome: | The proposed models exhibit reasoning biases, avoid answering that no conclusion follows, align with human difficulties, and struggle with multi-step reasoning. |
Annotating for Hate Speech: The MaNeCo Corpus and Some Input from Critical Discourse Analysis (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing methods for detecting hate speech are based on the problem of identification, but there is no clear definition of hate speech. |
| Approach: | They propose a multi-layer annotation scheme for the detection of hate speech in a web 2.0 corpus . they propose to use a binary hate speech classification to identify hate speech . |
| Outcome: | The proposed scheme is piloted against a binary hate speech classification and appears to yield higher inter-annotator agreement. |
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena (2022.acl-long)
Copied to clipboard
| Challenge: | Current models have difficulty addressing most phenomena, despite performance gains on many V&L tasks. |
| Approach: | They propose a benchmark for testing general-purpose pretrained vision and language models for their visio-linguistic grounding capabilities on specific linguistic phenomena. |
| Outcome: | The proposed model can be used to evaluate models on linguistic phenomena in the visual modality. |
BERT, are you paying attention? Attention regularization with human-annotated rationales (2026.eacl-long)
Copied to clipboard
| Challenge: | Attention regularisation aims to supervise the attention patterns in language models like BERT. |
| Approach: | They compare regularisation on human rationales with random tokens to find that human-annotated rationale is better at reducing model sensitivity to spurious correlations. |
| Outcome: | The proposed regularisation method improves model performance and model robustness, but not with human-annotated rationales. |
Face2Text: Collecting an Annotated Image Description Corpus for the Generation of Rich Face Descriptions (L18-1)
Copied to clipboard
Albert Gatt, Marc Tanti, Adrian Muscat, Patrizia Paggio, Reuben A Farrugia, Claudia Borg, Kenneth P Camilleri, Michael Rosner, Lonneke van der Plas
| Challenge: | a crowdsourcing study has been conducted to generate rich textual descriptions of human faces . the aim is to investigate how users describe images of human face images . |
| Approach: | They propose to extend the problem of automatically generating text from images to face description . they conducted an annotation study on a subset of the corpus to gain a better understanding of the variation they find in face descriptions . |
| Outcome: | The proposed corpus is based on images taken in the wild and is expected to be large enough to support non-trivial machine learning work on the automated description of faces. |
Entity-Based Semantic Adequacy for Data-to-Text Generation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing pre-trained language models have improved the fluency of text generation systems, but semantic adequacy remains an unsolved issue. |
| Approach: | They propose an automatic evaluation metric to assess to what extent models that verbalise RDF graphs produce text that contains mentions of entities occurring in the input. |
| Outcome: | The proposed metric can be used to assess to what extent generation models verbalise RDF graphs produce text that contains mentions of the entities occurring in the input. |
You Write like You Eat: Stylistic Variation as a Predictor of Social Stratification (P19-1)
Copied to clipboard
| Challenge: | In order to test whether and to what extent variations in writing style are influenced by socio-economic status, we used user-generated restaurant reviews on social media. |
| Approach: | They propose to use user-generated restaurant reviews to test whether and to what extent variations in writing style are influenced by socio-economic status. |
| Outcome: | The proposed model is based on user-generated restaurant reviews and user-created reviews. |
FTFT: Efficient and Robust Fine-Tuning by Transferring Training Dynamics (2025.coling-main)
Copied to clipboard
| Challenge: | Despite the success of fine-tuning Pre-trained Language Models, they remain susceptible to out-of-distribution input. |
| Approach: | They propose a novel approach that fine-tunes Pre-trained Language Models by transFerring Training dynamics (FTFT) FTFT uses more efficient reference models and aggressive early stopping . |
| Outcome: | The proposed approach improves the robustness of fine-tuned PLMs while reducing training costs. |