Papers by Oscar Sainz
ZS4IE: A toolkit for Zero-Shot Information Extraction with simple Verbalizations (2022.naacl-demo)
Copied to clipboard
| Challenge: | Information Extraction (IE) analysts use supervised machine learning to define the schema and build a training corpus with annotated examples. |
| Approach: | They propose a workflow where the analyst verbalizes the entities/relations, which are then used by a Textual Entailment model to perform zero-shot IE. |
| Outcome: | The proposed workflow performs very well on four IE tasks with a single user interface and a video demonstration is available on vimeo. |
Textual Entailment for Event Argument Extraction: Zero- and Few-Shot with Multi-Source Learning (2022.findings-naacl)
Copied to clipboard
| Challenge: | Recent work shows that Relation Extraction tasks can be recasted as Textual Entailment tasks using verbalizations. |
| Approach: | They propose to recasted RE tasks as Textual Entailment tasks using verbalizations . they show that entailment reduces the need for manual annotation to 50% and 20% . |
| Outcome: | The proposed method reduces the need for manual annotation to 50% and 20% in event argument extraction tasks while achieving the same performance as with full training. |
Label Verbalization and Entailment for Effective Zero and Few-Shot Relation Extraction (2021.emnlp-main)
Copied to clipboard
| Challenge: | Relation extraction systems require large amounts of labeled examples which are costly to annotate. |
| Approach: | They propose to use hand-made relation extraction tasks to refine a pretrained textual entailment engine which is run as-is or further fine-tuned on labeled examples. |
| Outcome: | The proposed system achieves 63% F1 zero-shot, 69% with 16 examples per relation and 4 points short of the state-of-the-art system on the same conditions. |
NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmark (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for evaluating large language models using annotated benchmarks are in trouble . data contamination can cause wrong scientific conclusions being published . |
| Approach: | They argue that the evaluation of NLP tasks using annotated benchmarks is in trouble . they define different levels of data contamination and propose a community effort . |
| Outcome: | The proposed measures should detect when data from a benchmark was exposed to a model and flag papers with conclusions compromised by data contamination. |
Event Extraction in Basque: Typologically Motivated Cross-Lingual Transfer-Learning Analysis (2024.lrec-main)
Copied to clipboard
| Challenge: | Using a multilingual language model, Event Extraction tasks require humans to follow complicated guidelines and follow complicated rules. |
| Approach: | They propose a multilingual multilingual language model that is trained in a source language and applied to a target language. |
| Outcome: | The proposed model is based on a multilingual event extraction dataset for Basque . it shows that the shared linguistic characteristic between source and target languages does have an impact on transfer quality. |
GUIDEX: Guided Synthetic Data Generation for Zero-Shot Information Extraction (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing domain-specific IE systems require expert schema design, data annotation, and model training. |
| Approach: | They propose a method that automatically defines domain-specific schemas and infers guidelines and generates synthetically labeled instances. |
| Outcome: | The proposed method improves on seven zeroshot Named Entity Recognition benchmarks. |
Domain Adapted Distant Supervision for Pedagogically Motivated Relation Extraction (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing systems for question generation based on shallow linguistic information at text level generate questions for reading comprehension. |
| Approach: | They propose a relation extraction system that extracts pedagogically motivated relation types from a text to generate questions for reading comprehension. |
| Outcome: | The proposed system extracts pedagogically motivated relations from a text . it performs better on the subset of relations suitable for pedagical purposes. |
Instructing Large Language Models for Low-Resource Languages: A Systematic Study for Basque (2025.emnlp-main)
Copied to clipboard
Oscar Sainz, Naiara Perez, Julen Etxaniz, Joseba Fernandez de Landa, Itziar Aldabe, Iker García-Ferrero, Aimar Zabala, Ekhi Azurmendi, German Rigau, Eneko Agirre, Mikel Artetxe, Aitor Soroa
| Challenge: | Instructing language models with user intent requires large instruction datasets limited to a limited set of languages. |
| Approach: | They propose to use existing LLMs and synthetically generated instructions to train models with user intent. |
| Outcome: | The proposed model outperforms base non-instructed models on Basque without Basque instructions. |