Papers by Thomas Lavergne
Embedding Strategies for Specialized Domains: Application to Clinical Entity Recognition (P19-2)
Copied to clipboard
| Challenge: | Off-the-shelf word embeddings tend to perform poorly on texts from specialized domains such as clinical reports. |
| Approach: | They combine off-the-shelf contextual embeddings with static word2vec embedders trained on a small in-domain corpus built from task data to reach and sometimes outperform representations learned from a large corpus in the medical domain. |
| Outcome: | The proposed embedding strategies outperform representations learned from a large corpus in the medical domain. |
Decorate the Examples: A Simple Method of Prompt Design for Biomedical Relation Extraction (2022.lrec-1)
Copied to clipboard
| Challenge: | Recent research shows that prompt-based learning improves performance on relation extraction tasks. |
| Approach: | They propose a prompt-based learning method that generates comprehensive prompts for biomedical relation extraction using a ChemProt dataset. |
| Outcome: | The proposed method improves fine-tuning on a biomedical relation extraction task with a cloze-test task and fewer training examples to make reasonable predictions. |
A Diversity Diet for a Healthier Model: A Case Study of French ModernBERT (2026.findings-acl)
Copied to clipboard
| Challenge: | Diversity has been gaining interest in the NLP community in recent years. |
| Approach: | They propose to use diversity-driven sampling to pre-train models on French with a fixed compute budget. |
| Outcome: | The diversity-driven sampling reduces the pre-training dataset by 94% and the pretraining time by 73% while maintaining comparable performance. |
Re-train or Train from Scratch? Comparing Pre-training Strategies of BERT in the Medical Domain (2022.lrec-1)
Copied to clipboard
| Challenge: | Recent years have witnessed the widespread use of transfer learning techniques in Natural Language Processing (NLP) |
| Approach: | They train BERT models from scratch using many configurations involving general and medical corpora. |
| Outcome: | The initial corpus only has a weak influence when these are further pre-trained on a medical corpus. |
CharacterBERT: Reconciling ELMo and BERT for Word-Level Open-Vocabulary Representations From Characters (2020.coling-main)
Copied to clipboard
Hicham El Boukkouri, Olivier Ferret, Thomas Lavergne, Hiroshi Noji, Pierre Zweigenbaum, Jun’ichi Tsujii
| Challenge: | Pre-trained language representations from Transformers have become the most popular choice for building NLP systems. |
| Approach: | They propose a new variant of BERT that drops the wordpiece tokenization system altogether . they propose 'characterBERT' module to represent entire words by consulting their characters . |
| Outcome: | The proposed model improves performance on a variety of medical domain tasks while producing robust, word-level, and open-vocabulary representations. |
Vector Spaces for Quantifying Disparity of Multiword Expressions in Annotated Text (2024.acl-srw)
Copied to clipboard
| Challenge: | We show that multiword expressions are a good study for linguistic diversity due to theiridiosyncratic nature. |
| Approach: | They train static MWE-aware word embeddings for verbal MWEs in 14 languages . they find that the disparity measure aggregatingthem at a global scale correlates with the number of types . |
| Outcome: | The proposed method is based on a set of vector spaces for VMWEs in 14 languages. |
Handling Entity Normalization with no Annotated Corpus: Weakly Supervised Methods Based on Distributional Representation and Ontological Information (2020.lrec-1)
Copied to clipboard
Arnaud Ferré, Robert Bossy, Mouhamadou Ba, Louise Deléger, Thomas Lavergne, Pierre Zweigenbaum, Claire Nédellec
| Challenge: | Entity normalization is an important subtask of information extraction . it links entities mentions in text to categories or concepts in a reference vocabulary . |
| Approach: | They propose a method that uses corpus selection, pre-processing and weak supervision strategies to address the scarcity of training data. |
| Outcome: | The proposed method outperforms state-of-the-art methods in terms of accuracy and parametrization . it uses corpus selection, pre-processing and weak supervision strategies . |
Corpora with Part-of-Speech Annotations for Three Regional Languages of France: Alsatian, Occitan and Picard (L18-1)
Copied to clipboard
Delphine Bernhard, Anne-Laure Ligozat, Fanny Martin, Myriam Bras, Pierre Magistry, Marianne Vergez-Couret, Lucie Steiblé, Pascale Erhart, Nabil Hathout, Dominique Huck, Christophe Rey, Philippe Reynés, Sophie Rosset, Jean Sibille, Thomas Lavergne
| Challenge: | RESTAURE project aims to develop resources and tools for three regional languages of France: Alsatian, Occitan and Picard. |
| Approach: | They describe the creation of corpora with part-of-speech annotations for Alsatian, Occitan and Picard. |
| Outcome: | The authors describe the creation of annotated corpora for Alsatian, Occitan and Picard . the project is part of the RESTAURE project, which aims to develop resources and tools for these under-resourced French regional languages. |
A Dataset for Pharmacovigilance in German, French, and Japanese: Annotating Adverse Drug Reactions across Languages (2024.lrec-main)
Copied to clipboard
Lisa Raithel, Hui-Syuan Yeh, Shuntaro Yada, Cyril Grouin, Thomas Lavergne, Aurélie Névéol, Patrick Paroubek, Philippe Thomas, Tomohiro Nishiyama, Sebastian Möller, Eiji Aramaki, Yuji Matsumoto, Roland Roller, Pierre Zweigenbaum
| Challenge: | Existing clinical corpora mostly revolves around scientific articles in English . existing literature is limited to only a few scientific articles . |
| Approach: | They propose to use user-generated data sources to uncover adverse drug reactions . existing clinical corpora mostly revolves around scientific articles in english . authors provide statistics to highlight certain challenges associated with the corpus . |
| Outcome: | The proposed corpus includes 12 entity types, four attribute types, and 13 relation types . it provides strong baselines for extracting entities and relations between entities . |