Papers by Jeremy Barnes
A Fine-grained Sentiment Dataset for Norwegian (2020.lrec-1)
Copied to clipboard
| Challenge: | Using a dataset for fine-grained sentiment analysis in Norwegian, we examine the annotation effort and provide an overview of the developed annotation guidelines. |
| Approach: | They propose a dataset for fine-grained sentiment analysis in Norwegian . they provide an overview of the developed annotation guidelines and analyze inter-annotator agreement . |
| Outcome: | The proposed dataset is the first of its kind for Norwegian and is available online. |
Truth Knows No Language: Evaluating Truthfulness Beyond English (2025.acl-long)
Copied to clipboard
Blanca Calvo Figueras, Eneko Sagarzazu, Julen Etxaniz, Jeremy Barnes, Pablo Gamallo, Iria de-Dios-Flores, Rodrigo Agerri
| Challenge: | a new benchmark evaluates the truthfulness of large language models (LLMs) based on imitative falsehoods. |
| Approach: | They propose a professionally translated extension of the TruthfulQA benchmark . it evaluates truthfulness in Basque, Catalan, Galician, and Spanish . |
| Outcome: | The proposed extension of the TruthfulQA benchmark evaluates truthfulness in Basque, Catalan, Galician, and Spanish. |
Projecting Embeddings for Domain Adaption: Joint Modeling of Sentiment Analysis in Diverse Domains (C18-1)
Copied to clipboard
| Challenge: | Existing domain adaptation methods for sentiment analysis are sensitive to domain differences, resulting in classifiers that perform poorly on new domains. |
| Approach: | They propose a domain adaptation problem as an embedding projection task using two mono-domain embeddable spaces and a bi-domain space to project across domains and predict sentiment. |
| Outcome: | The proposed model performs better on domains similar to state-of-the-art methods while requiring longer training times. |
MultiBooked: A Corpus of Basque and Catalan Hotel Reviews Annotated for Aspect-level Sentiment Classification (L18-1)
Copied to clipboard
| Challenge: | sentiment analysis research has focused on unsupervised or semi-supervised approaches, but these still require a large number of resources and do not reach the performance of supervised approaches. |
| Approach: | They propose two datasets for supervised aspect-level sentiment analysis in Basque and Catalan. |
| Outcome: | The proposed datasets are based on two under-resourced languages, basque and catalan. |
Direct parsing to sentiment graphs (2022.acl-short)
Copied to clipboard
| Challenge: | Existing methods for structured sentiment analysis (SSA) focus on subcomponents of sentiment graphs without explicitly expressing their relations or the polarity. |
| Approach: | They propose a graph-based semantic parser which directly predicts sentiment graphs from text without reliance on lossy conversions to intermediate dependency representations. |
| Outcome: | The proposed model performs on 4 out of 5 standard benchmark sets and compares with dependency-based models on the more structurally complex datasets. |
A Virtual Assistant for Architectural Design in a VR Environment (2026.eacl-demo)
Copied to clipboard
Ander Salaberria, Oier Ijurco, Markel Ferro, Jiayuan Wang, Iñigo Vilá Muñoz, Roberto de Ioris, Jeremy Barnes, Oier Lopez De Lacalle
| Challenge: | Existing interfaces lack intuitive, contextaware capabilities needed for easily navigating complex architectural components in 3D spaces. |
| Approach: | They propose a virtual assistant that allows users to interact with the BIM file in a VR environment. |
| Outcome: | The proposed assistant could allow users to interact with the BIM file in a virtual reality environment and perform complex actions. |
skweak: Weak Supervision Made Easy for NLP (2021.acl-demo)
Copied to clipboard
| Challenge: | skweak is a Python-based toolkit for NLP developers to use weak supervision . labelled data remains a scarce resource in many practical NLP scenarios . |
| Approach: | They present a Python-based toolkit that allows NLP developers to use weak supervision . skweak is designed to facilitate the use of weak supervision for NLP tasks . |
| Outcome: | skweak is a Python-based toolkit that facilitates weak supervision . the toolkit provides a simple interface to apply labels to a large corpus of text data . |
Named Entity Recognition without Labelled Data: A Weak Supervision Approach (2020.acl-main)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) performance often degrades when applied to target domains that differ from the texts observed during training. |
| Approach: | They propose a method to learn NER models in the absence of labelled data through weak supervision by using a broad spectrum of labelling functions to automatically annotate texts from the target domain. |
| Outcome: | The proposed approach improves on two English datasets and shows that it improves by 7 percentage points on entity-level F1 scores compared to an out-of-domain neural NER model. |
If you’ve got it, flaunt it: Making the most of fine-grained sentiment annotations (2021.eacl-main)
Copied to clipboard
| Challenge: | a recent study shows that annotating sentiments is difficult and difficult. |
| Approach: | They propose to integrate holder and expression information into sentiment analysis to improve target extraction . they perform experiments on eight English datasets to determine whether annotating expressions improves target extraction. |
| Outcome: | The proposed approach improves target extraction and classification on English datasets. |
Evaluating morphological typology in zero-shot cross-lingual transfer (2021.acl-long)
Copied to clipboard
| Challenge: | morphological typology has been used to improve cross-lingual transfer . however, some language families and typologies consistently perform worse . |
| Approach: | They examine effects of morphological typology on zero-shot cross-lingual transfer . they perform part-of-speech tagging and sentiment analysis on 19 languages . |
| Outcome: | The proposed model improves on fusional and introflexive languages, but some language families and typologies perform worse. |
Structured Sentiment Analysis as Dependency Graph Parsing (2021.acl-long)
Copied to clipboard
| Challenge: | Structured sentiment analysis attempts to extract full opinion tuples from a text, but has been subdivided into smaller and smaller sub-tasks, e.g., target extraction or targeted polarity classification. |
| Approach: | They propose a framework which jointly predicts all elements of an opinion tuple and their relations by using dependency graph parsing. |
| Outcome: | The proposed framework improves on five datasets in English, Norwegian, Basque, and Catalan and refining the sentiment graphs with syntactic dependency information further improves results. |
Bilingual Sentiment Embeddings: Joint Projection of Sentiment Across Languages (P18-1)
Copied to clipboard
| Challenge: | Existing approaches to sentiment analysis in low-resource languages lack annotated corpora or do not capture sentiment information. |
| Approach: | They propose a model that represents sentiment in a source and target language without annotated corpus. |
| Outcome: | The proposed model outperforms state-of-the-art methods on four out of six setups and captures complementary information to machine translation. |
The interplay between language similarity and script on a novel multi-layer Algerian dialect corpus (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have focused on cross-lingual transfer between languages with similar typology and languages of different scripts. |
| Approach: | They propose to annotate Algerian user-generated comments with parallel annotations . they also investigate the effect of script vs. language similarity in cross-lingual transfer . |
| Outcome: | The proposed model fine-tunes multi-lingual models on Algerian language and scripts . it shows that script vs. language similarity is important for part-of-speech tagging and sentiment analysis . |
XNLIeu: a dataset for cross-lingual NLI in Basque (2024.naacl-long)
Copied to clipboard
| Challenge: | XNLI is a popular benchmark used to evaluate cross-lingual Natural Language Understanding (NLU) in languages such as English, Basque and other low-resource languages. |
| Approach: | They expand XNLI to include Basque, a low-resource language that can benefit from transfer-learning approaches. |
| Outcome: | The proposed dataset includes Basque, a low-resource language that can benefit from transfer-learning approaches. |
Vision-Language Models Struggle to Align Entities across Modalities (2025.findings-acl)
Copied to clipboard
| Challenge: | Several real-world applications require the ability to perform cross-modal entity linking . cross-functional entity linking is a skill needed for multimodal code generation and scene understanding . |
| Approach: | They propose a task and benchmark to evaluate cross-modal entity linking performance . they use visual scenes aligned with their textual representations to evaluate performance a question-answering task . |
| Outcome: | The proposed task and benchmark aims to improve cross-modal entity linking performance . it evaluates state-of-the-art vision-language models and humans on the task . |
Multi-task Learning of Negation and Speculation for Targeted Sentiment Classification (2021.naacl-main)
Copied to clipboard
| Challenge: | Currently, most work on targeted sentiment analysis is focused on improving the overall results. |
| Approach: | They propose a multi-task learning method to incorporate information from syntactic and semantic auxiliary tasks to create English-language models that are more robust to linguistic phenomena. |
| Outcome: | The proposed method improves on negation and speculation datasets but there is room for improvement. |
IberoBench: A Benchmark for LLM Evaluation in Iberian Languages (2025.coling-main)
Copied to clipboard
Irene Baucells, Javier Aula-Blasco, Iria de-Dios-Flores, Silvia Paniagua Suárez, Naiara Perez, Anna Salles, Susana Sotelo Docio, Júlia Falcão, Jose Javier Saiz, Robiert Sepulveda Torres, Jeremy Barnes, Pablo Gamallo, Aitor Gonzalez-Agirre, German Rigau, Marta Villegas
| Challenge: | Existing multi-task benchmarks for Large Language Models are limited to English . a new benchmark is needed to evaluate models on a range of tasks . |
| Approach: | They propose a multilingual, multi-task benchmark for Iberian languages built on the LM Evaluation Harness framework. |
| Outcome: | The proposed benchmark covers 62 tasks divided into 179 subtasks and is available in Iberian, Basque, Catalan, Galician, European Spanish and European Portuguese. |