Papers by Aurélie Névéol
“Women do not have heart attacks!” Gender Biases in Automatically Generated Clinical Cases in French (2025.findings-naacl)
Copied to clipboard
| Challenge: | Healthcare professionals are increasingly including Language Models (LMs) in clinical practice. |
| Approach: | They propose to use LMs to generate clinical cases in french and an automatic linguistic gender detection tool to measure gender biases. |
| Outcome: | The proposed model over-generates cases describing male patients, creating synthetic corpora that are not consistent with documented prevalence for these disorders. |
French CrowS-Pairs: Extending a challenge dataset for measuring social bias in masked language models to a language other than English (2022.acl-long)
Copied to clipboard
| Challenge: | We introduce 1,679 sentence pairs in French that cover stereotypes in ten types of bias like gender and age. |
| Approach: | They build on the US-centered CrowS-pairs dataset to create a multilingual stereotypes dataset that allows for comparability across languages and cultures. |
| Outcome: | The proposed dataset allows for comparability across languages while characterizing biases that are specific to each country and language. |
Reviewing Natural Language Processing Research (2021.eacl-tutorials)
Copied to clipboard
| Challenge: | a tutorial on reviewing is a useful tool for researchers who are new to the field of NLP. |
| Approach: | this tutorial provides an opportunity to learn the basics of reviewing . more experienced researchers might find this tutorial interesting to revise their reviewing procedure. |
| Outcome: | This tutorial teaches researchers how to revise their reviewing procedure . |
Few-shot clinical entity recognition in English, French and Spanish: masked language models outperform generative model prompting (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a critical task in information extraction that is not covered in recent benchmarks. |
| Approach: | They compare 13 auto-regressive models using prompting and 16 masked models using fine-tuning on 14 NER datasets covering English, French and Spanish. |
| Outcome: | The proposed models outperform auto-regressive models in English, French and Spanish on 14 NER datasets. |
Can Synthetic Text Help Clinical Named Entity Recognition? A Study of Electronic Health Records in French (2023.eacl-main)
Copied to clipboard
| Challenge: | In sensitive domains, the sharing of corpora is restricted due to confidentiality, copyrights or trade secrets. |
| Approach: | They use auto-regressive neural models to generate a clinical case corpus annotated with clinical entities and evaluate it for a named entity recognition task. |
| Outcome: | The proposed model can produce clinical case corpus annotated with clinical entities while maintaining confidentiality. |
Limitations of Human Identification of Automatically Generated Text (2024.lrec-main)
Copied to clipboard
Nadège Alavoine, Maximin Coavoux, Emmanuelle Esperança-Rodier, Romane Gallienne, Carlos Gonzalez Gallardo, Jérôme Goulian, Jose G. Moreno, Aurélie Névéol, Didier Schwab, Vincent Segonne, Johanna Simoens
| Challenge: | Neural text generation tools such as ChatGPT are gaining popularity . human annotations are considered gold standard labels for multiple tasks . |
| Approach: | They propose a new corpus in French and English for recognising automatically generated texts . they propose 'incontext' setup which makes explicit the interaction between two parties . |
| Outcome: | The proposed model generates fluent text, which requires much closer reading than the current model. |
CLISTER : A Corpus for Semantic Textual Similarity in French Clinical Narratives (2022.lrec-1)
Copied to clipboard
| Challenge: | Modern Natural Language Processing relies on the availability of annotated corpora for training and evaluation. |
| Approach: | They propose to annotate sentences in French using a definition of similarity guided by clinical facts and use it to evaluate the corpus. |
| Outcome: | The proposed model can capture similarity with state-of-the-art performance on the DEFT STS shared task evaluation data set. |
MEDLINE as a Parallel Corpus: a Survey to Gain Insight on French-, Spanish- and Portuguese-speaking Authors’ Abstract Writing Practice (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing corpora are used to train and evaluate machine translation systems, but little information is available about the methods used for producing the corpus, including translation direction. |
| Approach: | They used PubMed and publisher websites to obtain contact information for MEDLINE authors and asked about their abstract writing practices. |
| Outcome: | The authors of MEDLINE articles included in the English/Spanish, English/FR, and English/Portuguese (EN/PT) WMT 2019 test sets reported a response rate of over 20% . |
Three Dimensions of Reproducibility in Natural Language Processing (L18-1)
Copied to clipboard
K. Bretonnel Cohen, Jingbo Xia, Pierre Zweigenbaum, Tiffany Callahan, Orin Hargraves, Foster Goss, Nancy Ide, Aurélie Névéol, Cyril Grouin, Lawrence E. Hunter
| Challenge: | a recent editorial on reproducibility in language processing defined three dimensions of reproducibility . authors had already submitted a correction, but there is no consensus on the definitions . |
| Approach: | They propose an ontology of reproducibility in natural language processing to address these problems . they propose to analyze three dimensions of reproducible in natural languages papers . authors propose to use a 'replicability' term to describe the reproducibility of a conclusion, finding, value . |
| Outcome: | The proposed ontology aims to enhance future research and communication about the topic and retrospective meta-analyses. |
Automating Document Discovery in the Systematic Review Process: How to Use Chaff to Extract Wheat (L18-1)
Copied to clipboard
| Challenge: | Systematic reviews address research questions by comprehensively examining the entire published literature. |
| Approach: | They compare the impact of different schemes for choosing positive and negative examples from the different screening stages on the training of automated systems. |
| Outcome: | The proposed ranking system achieves an AUC of 0.803 and 0.768 when relying on gold standard decisions based on title and abstracts of articles, and an AUT of 0.625 and 0.839 when based upon gold standard decision based in full text. |
Parallel Corpora for the Biomedical Domain (L18-1)
Copied to clipboard
| Challenge: | Existing corpora of parallel corporata are being used in the biomedical domain . MT is known to support readers' access to textual documents in a language other than their native language . |
| Approach: | They propose to leverage parallel corpora to implement cross-lingual information retrieval or machine translation tools. |
| Outcome: | The proposed corpus is being used in the biomedical task at the conference on machine translation (WMT'16 and WMT'17) it can be leveraged to provide access to health information in languages other than English. |
Reviewing Natural Language Processing Research (2020.acl-tutorials)
Copied to clipboard
| Challenge: | a tutorial on reviewing research in natural language processing will cover the theory and practice of reviewing research. |
| Approach: | tutorial covers the theory and practice of reviewing research in natural language processing . authors say reviewers should be more aware of "false negatives" |
| Outcome: | tutorial covers the theory and practice of reviewing research in natural language processing . authors say their reviews leave something to be desired . |
Your Stereotypical Mileage May Vary: Practical Challenges of Evaluating Biases in Multiple Languages and Cultural Contexts (2024.lrec-main)
Copied to clipboard
Karen Fort, Laura Alonso Alemany, Luciana Benotti, Julien Bezançon, Claudia Borg, Marthese Borg, Yongjian Chen, Fanny Ducel, Yoann Dupont, Guido Ivetta, Zhijian Li, Margot Mieskes, Marco Naguib, Yuyan Qian, Matteo Radaelli, Wolfgang S. Schmeisser-Nieto, Emma Raimundo Schulz, Thiziri Saci, Sarah Saidi, Javier Torroba Marchante, Shilin Xie, Sergio E. Zanotto, Aurélie Névéol
| Challenge: | Recent studies have identified a gap in the availability of tools and resources to study bias in languages other than English and social contexts outside the north of America. |
| Approach: | They use stereotypes to build a corpus of sentence pairs that cover biases in seven cultural contexts. |
| Outcome: | The proposed resource covers a wide range of languages and cultural settings . it favors sentences that express stereotypes in most bias categories . |
The Elephant in the Room: Analyzing the Presence of Big Tech in Natural Language Processing Research (2023.acl-long)
Copied to clipboard
Mohamed Abdalla, Jan Philip Wahle, Terry Ruas, Aurélie Névéol, Fanny Ducel, Saif Mohammad, Karen Fort
| Challenge: | Recent advances in deep learning methods for natural language processing (NLP) have created new business opportunities and made NLP research critical for industry development. |
| Approach: | They examine industry presence in the field since the early 90s and characterize it using a corpus of 78,187 NLP publications and 701 resumes of NLP publication authors. |
| Outcome: | The authors find that industry presence among NLP authors has been steady before a steep increase over the past five years (180% growth from 2017 to 2022). |
A Benchmark Evaluation of Clinical Named Entity Recognition in French (2024.lrec-main)
Copied to clipboard
| Challenge: | Masked Language Models (MLMs) have shown strong performance on many NLP tasks. |
| Approach: | They evaluate masked language models for biomedical French on the task of clinical named entity recognition using gold-standard corpora. |
| Outcome: | The proposed model outperforms standard models on the task of clinical named entity recognition in biomedical French while remaining lighter than current models. |
SHADES: Towards a Multilingual Assessment of Stereotypes in Large Language Models (2025.naacl-long)
Copied to clipboard
Margaret Mitchell, Giuseppe Attanasio, Ioana Baldini, Miruna Clinciu, Jordan Clive, Pieter Delobelle, Manan Dey, Sil Hamilton, Timm Dill, Jad Doughman, Ritam Dutt, Avijit Ghosh, Jessica Zosa Forde, Carolin Holtermann, Lucie-Aimée Kaffee, Tanmay Laud, Anne Lauscher, Roberto L Lopez-Davila, Maraim Masoud, Nikita Nangia, Anaelia Ovalle, Giada Pistilli, Dragomir Radev, Beatrice Savoldi, Vipul Raheja, Jeremy Qin, Esther Ploeger, Arjun Subramonian, Kaustubh Dhole, Kaiser Sun, Amirbek Djanibekov, Jonibek Mansurov, Kayo Yin, Emilio Villa Cueva, Sagnik Mukherjee, Jerry Huang, Xudong Shen, Jay Gala, Hamdan Al-Ali, null Tair Djanibekov, Nurdaulet Mukhituly, Shangrui Nie, Shanya Sharma, Karolina Stanczak, Eliza Szczechla, Tiago Timponi Torrent, Deepak Tunuguntla, Marcelo Viridiano, Oskar Van Der Wal, Adina Yakefu, Aurélie Névéol, Mike Zhang, Sydney Zink, Zeerak Talat
| Challenge: | Large Language Models reproduce and exacerbate social biases present in training data, and resources to quantify this issue are limited. |
| Approach: | They propose a multilingual parallel dataset to examine culturally-specific stereotypes that may be learned by LLMs. |
| Outcome: | The proposed dataset includes stereotypes from 20 regions around the world and 16 languages, spanning multiple identity categories subject to discrimination worldwide. |
A Dataset for Pharmacovigilance in German, French, and Japanese: Annotating Adverse Drug Reactions across Languages (2024.lrec-main)
Copied to clipboard
Lisa Raithel, Hui-Syuan Yeh, Shuntaro Yada, Cyril Grouin, Thomas Lavergne, Aurélie Névéol, Patrick Paroubek, Philippe Thomas, Tomohiro Nishiyama, Sebastian Möller, Eiji Aramaki, Yuji Matsumoto, Roland Roller, Pierre Zweigenbaum
| Challenge: | Existing clinical corpora mostly revolves around scientific articles in English . existing literature is limited to only a few scientific articles . |
| Approach: | They propose to use user-generated data sources to uncover adverse drug reactions . existing clinical corpora mostly revolves around scientific articles in english . authors provide statistics to highlight certain challenges associated with the corpus . |
| Outcome: | The proposed corpus includes 12 entity types, four attribute types, and 13 relation types . it provides strong baselines for extracting entities and relations between entities . |