Papers by Begoña Altuna
This is not a Dataset: A Large Negation Benchmark to Challenge Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have grammatical knowledge but fail to interpret negation . a recent study shows that LLMs struggle with negative sentences . |
| Approach: | They propose to use a dataset to grasp LLMs' generalization and inference capability . they also fine-tuned models to assess whether the understanding of negation can be trained . |
| Outcome: | The proposed model is able to generalize and infer negation in 400,000 sentences . but it is suboptimal when it comes to negation, a key step in natural language processing . |
A Multi-layered Approach to Physical Commonsense Understanding: Creation and Evaluation of an Italian Dataset (2024.lrec-main)
Copied to clipboard
| Challenge: | Using a multilingual model, we examine the ability of large language models to perform reasoning tasks. |
| Approach: | They propose to use a multilingual model to analyze commonsense reasoning in large language models for Italian and to provide a semi-automated system to complete the annotation. |
| Outcome: | The proposed model performs at high-level classification tasks but its easoning is inconsistent and unverifiable, since it does not capture intermediate evidence. |