Papers by Simone Conia
Increasing Coverage and Precision of Textual Information in Multilingual Knowledge Graphs (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to generate knowledge graphs are unable to handle non-English textual information. |
| Approach: | They propose a task of automatic Knowledge Graph Completion to bridge the gap between English and non-English textual information. |
| Outcome: | The proposed method bridges the gap between the quantity and quality of textual information between English and non-English languages. |
SRL4E – Semantic Role Labeling for Emotions: A Unified Evaluation Framework (2022.acl-long)
Copied to clipboard
| Challenge: | Existing datasets for emotion detection are heterogeneous in size, domain, format, splits, emotion categories and role labels, hampering progress in this area. |
| Approach: | They propose a framework for annotating emotions manually using a common labeling scheme to unify several datasets tagged with emotions and semantic roles. |
| Outcome: | The proposed framework unifies datasets tagged with emotions and semantic roles by using a common labeling scheme. |
Towards Cross-Cultural Machine Translation with Retrieval-Augmented Generation from Multilingual Knowledge Graphs (2024.emnlp-main)
Copied to clipboard
| Challenge: | XC-Translate is a large-scale, manually-created benchmark for machine translation . current systems struggle to translate texts containing entity names, but KG-MT outperforms state-of-the-art approaches . |
| Approach: | They propose a method to integrate multilingual knowledge into a neural machine translation model . XC-Translate is the first large-scale, manually-created benchmark for machine translation . they propose KG-MT to integrate cultural-related references into MT models . |
| Outcome: | The proposed method outperforms state-of-the-art approaches by a large margin compared to NLLB-200 and GPT-4 . the proposed method is based on a multilingual knowledge graph and dense retrieval mechanism . |
A Tour of Explicit Multilingual Semantics: Word Sense Disambiguation, Semantic Role Labeling and Semantic Parsing (2022.aacl-tutorials)
Copied to clipboard
| Challenge: | a recent advent of pretrained language models has sparked a revolution in NLP . but, there are still questions about whether current approaches capture explicit, symbolic meaning . this tutorial will review efforts to tackle three key open problems in lexical and sentence-level semantics . |
| Approach: | This tutorial reviews recent efforts to shed light on meaning in NLP . it will focus on three key open problems in lexical and sentence-level semantics . |
| Outcome: | This tutorial reviews recent efforts to shed light on meaning in NLP . it focuses on three key open problems in lexical and sentence-level semantics . |
Nibbling at the Hard Core of Word Sense Disambiguation (2022.acl-long)
Copied to clipboard
| Challenge: | Word Sense Disambiguation (WSD) is a task that is based on a set of pre-trained language models. |
| Approach: | They propose to use Word Sense Disambiguation to test whether systems can handle ambiguous words. |
| Outcome: | The proposed benchmarks show that seven of the most representative state-of-the-art systems make trivial errors on traditional evaluation benchmarks. |
Neuralign: A Context-Aware, Cross-Lingual and Fully-Neural Sentence Alignment System for Long Texts (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing sentence alignment systems focus on auxiliary information such as document metadata and hyperparameter-sensitive techniques, and neglect the crucial role that context plays in the alignment process. |
| Approach: | They propose a context-aware, end-to-end and fully-neural architecture for sentence alignment that maps source and target sentences in long documents by contextualizing their sentence embeddings with respect to the other sentences in the document. |
| Outcome: | The proposed system maps source and target sentences in long documents by contextualizing their sentence embeddings with respect to the other sentences in the document. |
ReTraceQA: Evaluating Reasoning Traces of Small Language Models in Commonsense Question Answering (2026.acl-long)
Copied to clipboard
| Challenge: | Recent work in language modeling has led to effective SLMs with impressive performance levels across various benchmarks. |
| Approach: | They propose a benchmark that introduces process-level evaluation for commonsense reasoning tasks. |
| Outcome: | The proposed benchmarks show that large language models provide correct answers despite flawed reasoning processes in a substantial portion of cases. |
KG-TRICK: Unifying Textual and Relational Information Completion of Knowledge for Multilingual Knowledge Graphs (2025.coling-main)
Copied to clipboard
Zelin Zhou, Simone Conia, Daniel Lee, Min Li, Shenglei Huang, Umar Farooq Minhas, Saloni Potdar, Henry Xiao, Yunyao Li
| Challenge: | Existing studies have shown that combining information from KGs in different languages aids knowledge Graph Completion and Knowledge Graph Enhancement. |
| Approach: | They propose a sequence-to-sequence framework that unifies tasks of textual and relational information completion for multilingual knowledge graphs. |
| Outcome: | The proposed framework unifies tasks of KGC and KGE into a single framework. |
Unifying Cross-Lingual Semantic Role Labeling with Heterogeneous Linguistic Resources (2021.naacl-main)
Copied to clipboard
| Challenge: | Using cross-lingual techniques to perform Semantic Role Labeling (SRL) has been limited by the fact that each language adopts its own linguistic formalism . |
| Approach: | They propose a unified model to perform cross-lingual SRL over heterogeneous linguistic resources. |
| Outcome: | The proposed model is able to annotate a sentence in a single forward pass with all the inventories it was trained with, providing a tool for the analysis and comparison of linguistic theories across different languages. |
Probing for Predicate Argument Structures in Pretrained Language Models (2022.acl-long)
Copied to clipboard
| Challenge: | Recent proposed approaches have achieved impressive results in dependency- and span-based, multilingual and cross-lingual Semantic Role Labeling (SRL) |
| Approach: | They propose to probe for predicate argument structures in pretrained language models . they show that PLMs encode semantic structures directly into contextualized representations . |
| Outcome: | The proposed models have achieved impressive results in dependency- and span-based, multilingual and cross-lingual Semantic Role Labeling (SRL) |
Do Large Language Models have an English Accent? Evaluating and Improving the Naturalness of Multilingual LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Current Large Language Models (LLMs) are predominantly designed with English as the primary language, but many are still English-dominated. |
| Approach: | They propose to use automatic corpus-level metrics to assess lexical and syntactic naturalness of LLMs in a multilingual context. |
| Outcome: | The proposed method improves naturalness of LLMs in target languages without compromising performance on general-purpose benchmarks. |
VerbAtlas: a Novel Large-Scale Verbal Semantic Resource and Its Application to Semantic Role Labeling (D19-1)
Copied to clipboard
| Challenge: | VerbAtlas is a lexical-semantic resource that combines WordNet synsets into semantically-coherent frames. |
| Approach: | They propose a lexical-semantic resource that brings together WordNet synsets into semantically-coherent frames. |
| Outcome: | The proposed resource brings together all WordNet synsets into semantically-coherent frames. |
MOSAICo: a Multilingual Open-text Semantically Annotated Interlinked Corpus (2024.naacl-long)
Copied to clipboard
Simone Conia, Edoardo Barba, Abelardo Carlos Martinez Lorenzo, Pere-Lluís Huguet Cabot, Riccardo Orlando, Luigi Procopio, Roberto Navigli
| Challenge: | Existing approaches to integrate semantics into Natural Language Understanding (NLP) systems are cost-effective and environmental impact-related. |
| Approach: | They propose to provide semantically-annotated corpora for four NLU tasks across five languages and to drop the requirement of closed datasets. |
| Outcome: | The proposed model provides hundreds of millions of silver yet high-quality annotations for four NLU tasks across five languages. |
ZEBRA: Zero-Shot Example-Based Retrieval Augmentation for Commonsense Question Answering (2024.emnlp-main)
Copied to clipboard
| Challenge: | Current Large Language Models (LLMs) have shown strong reasoning capabilities in commonsense question answering benchmarks, but the process underlying their success remains largely opaque. |
| Approach: | They propose a zero-shot question answering framework that combines retrieval, case-based reasoning and introspection to improve the model's performance and interpretability. |
| Outcome: | The proposed framework outperforms existing LLMs and previous knowledge integration approaches in commonsense reasoning benchmarks and achieves an average accuracy improvement of 4.5 points. |
Entity Disambiguation with Entity Definitions (2023.eacl-main)
Copied to clipboard
| Challenge: | Entity Disambiguation (ED) is a crucial problem in Natural Language Processing (NLP). |
| Approach: | They propose to use Wikipedia titles as the textual representation of each candidate to improve the generalization capability over unseen patterns. |
| Outcome: | The proposed model improves on 2 out of 6 benchmarks and is generalized over unseen patterns. |
Exploring Non-Verbal Predicates in Semantic Role Labeling: Challenges and Opportunities (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing systems for SRL are incapable of transferring knowledge across different predicate types. |
| Approach: | They propose a new PropBank dataset which boasts wide coverage of multiple predicate types and a manually-annotated challenge set which gives equal importance to verbal, nominal, and adjectival predicates. |
| Outcome: | The proposed dataset shows that standard benchmarks do not provide an accurate picture of the current situation in SRL and that state-of-the-art systems are still incapable of transferring knowledge across different predicate types. |
UniteD-SRL: A Unified Dataset for Span- and Dependency-Based Multilingual and Cross-Lingual Semantic Role Labeling (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Multilingual and cross-lingual Semantic Role Labeling (SRL) has attracted increasing attention as multilingual text representation techniques have become more effective and widely available. |
| Approach: | They propose a benchmark for multilingual and cross-lingual, span- and dependency-based SRL that provides expert-curated parallel annotations using a common predicate-argument structure inventory. |
| Outcome: | The proposed benchmark provides expert-curated parallel annotations using a common predicate-argument structure inventory, allowing direct comparisons across languages and encouraging studies on cross-lingual transfer in SRL. |
Conception: Multilingually-Enhanced, Human-Readable Concept Vector Representations (2020.coling-main)
Copied to clipboard
| Challenge: | Word vector representations suffer from a monolingual bias due to the amount of data available across languages. |
| Approach: | They propose a technique for building language-independent vector representations of concepts which places multilinguality at its core while retaining explicit relationships between concepts. |
| Outcome: | The proposed representations outperform the state-of-the-art in multilingual and cross-lingual Semantic Word Similarity and Word Sense Disambiguation on low-resource languages. |
Multi-LMentry: Can Multilingual LLMs Solve Elementary Tasks Across Languages? (2025.emnlp-main)
Copied to clipboard
Luca Moroni, Javier Aula-Blasco, Simone Conia, Irene Baucells, Naiara Perez, Silvia Paniagua Suárez, Anna Sallés, Malte Ostendorff, Júlia Falcão, Guijin Son, Aitor Gonzalez-Agirre, Roberto Navigli, Marta Villegas
| Challenge: | a recent study focused on complex, high-level tasks, but LMentry is limited to English . a multilingual evaluation of large language models is needed to address this gap, authors say . |
| Approach: | They propose a compact benchmark that enables systematic evaluation of large language models . they propose to use tasks that are trivial for humans but remain surprisingly difficult for LLMs . |
| Outcome: | The proposed benchmark is limited to English, leaving its insights linguistically narrow. |
Universal Semantic Annotator: the First Unified API for WSD, SRL and Semantic Parsing (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing approaches to understanding textual information are still far from achieving true natural language understanding (NLU). |
| Approach: | They propose a unified API for high-quality automatic annotations of texts in 100 languages through state-of-the-art systems for Word Sense Disambiguation, Semantic Role Labeling and Semantics Parsing. |
| Outcome: | The proposed system can provide users with rich and diverse semantic information, help second-language learners, and integrate explicit semantic knowledge into downstream tasks and real-world applications. |
InVeRo-XL: Making Cross-Lingual Semantic Role Labeling Accessible with Intelligible Verbs and Roles (2021.emnlp-demo)
Copied to clipboard
| Challenge: | InVeRo-XL is an off-the-shelf system capable of annotating text with predicate sense and semantic role labels from 7 predicated-argument structure inventories in more than 40 languages. |
| Approach: | They propose to use RESTful API and Web interface to integrate sentence-level semantics into cross-lingual downstream tasks. |
| Outcome: | The proposed system can annotate text with predicate sense and semantic role labels from 7 predicated-argument structure inventories in more than 40 languages. |
Named Entity Recognition for Entity Linking: What Works and What’s Next (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Entity Linking (EL) systems have achieved impressive results on standard benchmarks thanks to the contextualized representations provided by recent pretrained language models. |
| Approach: | They propose to exploit Named Entity Recognition (NER) to narrow the gap between EL systems trained on high and low amounts of labeled data. |
| Outcome: | The proposed model can be exploited to narrow the gap between EL systems trained on high and low amounts of labeled data. |
Semantic Role Labeling Meets Definition Modeling: Using Natural Language to Describe Predicate-Argument Structures (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to Semantic Role Labeling rely on discrete labels to classify predicate senses and their arguments. |
| Approach: | They propose a generalized formulation of Semantic Role Labeling that leverages Definition Modeling to describe predicate-argument structures using natural language definitions instead of discrete labels. |
| Outcome: | The proposed model can describe predicate-argument structures using natural language definitions instead of discrete labels. |
AMuSE-WSD: An All-in-one Multilingual System for Easy Word Sense Disambiguation (2021.emnlp-demo)
Copied to clipboard
| Challenge: | Word Sense Disambiguation (WSD) is a task of associating a word in context with its most appropriate sense from a predefined sense inventory. |
| Approach: | They propose to use a state-of-the-art neural model to integrate WSD into real-world applications. |
| Outcome: | The proposed system offers high-quality sense information in 40 languages through a state-of-the-art neural model for WSD. |
Right Answer, Wrong Score: Uncovering the Inconsistencies of LLM Evaluation in Multiple-Choice Question Answering (2025.findings-acl)
Copied to clipboard
| Challenge: | Multiple-choice question answering tasks are one of the most commonly used tasks for evaluating Large Language Models (LLMs). |
| Approach: | They analyze whether existing answer extraction methods are aligned with human judgment and how they are influenced by answer constraints in the prompt across different domains. |
| Outcome: | The proposed evaluation strategies can be inconsistent with human judgment, and can lead to inaccurate and misleading comparisons. |
Echoes from Alexandria: A Large Resource for Multilingual Book Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent research in text summarization has focused on news stories, where texts are typically short and have strong layout features. |
| Approach: | They propose a resource for multilingual book summarization that uses a new extractive-then-abstractive baseline to compare the results. |
| Outcome: | The proposed resource is the largest and first to be multilingual, featuring 5 languages and 25 language pairs. |
InVeRo: Making Semantic Role Labeling Accessible with Intelligible Verbs and Roles (2020.emnlp-demos)
Copied to clipboard
| Challenge: | Semantic Role Labeling (SRL) is dependent on complex linguistic resources and sophisticated neural models, which makes the task difficult to approach for non-experts. |
| Approach: | They propose a platform for semantic role labeling that provides verb sense and semantic role information with an easy to use Web interface and RESTful APIs. |
| Outcome: | The proposed system provides human-readable verb sense and semantic role information with an easy to use Web interface and RESTful APIs. |
Bridging the Gap in Multilingual Semantic Role Labeling: a Language-Agnostic Approach (2020.coling-main)
Copied to clipboard
| Challenge: | Recent research indicates that taking advantage of complex syntactic features leads to favorable results in Semantic Role Labeling. |
| Approach: | They propose a language-agnostic model that does away with morphological and syntactic features to achieve robustness across languages. |
| Outcome: | The proposed model outperforms the state-of-the-art in all languages of the CoNLL-2009 benchmark dataset. |
Framing Word Sense Disambiguation as a Multi-Label Problem for Model-Agnostic Knowledge Integration (2021.eacl-main)
Copied to clipboard
| Challenge: | Recent studies treat Word Sense Disambiguation (WSD) as a single-label classification problem, but gold data suggests that maximizing the probability of a singular sense is not the most suitable training objective for WSD. |
| Approach: | They propose to use Word Sense Disambiguation (WSD) as a multi-label classification problem in which multiple senses can be assigned to each target word. |
| Outcome: | The proposed method bears closer resemblance to how human annotators disambiguate text and can be extended to exploit structured knowledge from semantic networks. |