Papers with CamemBERT
A Benchmark Evaluation of Clinical Named Entity Recognition in French (2024.lrec-main)
Copied to clipboard
| Challenge: | Masked Language Models (MLMs) have shown strong performance on many NLP tasks. |
| Approach: | They evaluate masked language models for biomedical French on the task of clinical named entity recognition using gold-standard corpora. |
| Outcome: | The proposed model outperforms standard models on the task of clinical named entity recognition in biomedical French while remaining lighter than current models. |
CamemBERT-bio: Leveraging Continual Pre-training for Cost-Effective Models on French Biomedical Data (2024.lrec-main)
Copied to clipboard
| Challenge: | Clinical data in hospitals are unstructured and therefore need to be extracted from medical reports to conduct clinical studies. |
| Approach: | They propose a dedicated French biomedical model based on a public French biomedicine dataset. |
| Outcome: | The proposed model improves 2.54 points of F1-score on biomedical named entity recognition tasks. |
Data-Efficient French Language Modeling with CamemBERTa (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in NLP have significantly improved the performance of language models on a variety of tasks. |
| Approach: | They introduce a French DeBERTa model that builds upon the DeBERTAV3 architecture and training objective and evaluate its performance on a variety of French downstream tasks and datasets. |
| Outcome: | The proposed model outperforms BERT-based models on most tasks given the same amount of training tokens and trained on 30% of its input tokens. |
PAGnol: An Extra-Large French Generative Model (2022.lrec-1)
Copied to clipboard
Julien Launay, E.l. Tommasone, Baptiste Pannier, François Boniface, Amélie Chatelain, Alessandro Cappelli, Iacopo Poli, Djamé Seddah
| Challenge: | a growing number of pre-trained language models are available in many different languages. |
| Approach: | They propose a French-language GPT model with scaling laws to train it efficiently . they evaluate the models on discriminative and generative tasks in French . |
| Outcome: | The proposed model trains with the same computational budget as CamemBERT, a model 13 times smaller. |
CamemBERT: a Tasty French Language Model (2020.acl-main)
Copied to clipboard
Louis Martin, Benjamin Muller, Pedro Javier Ortiz Suárez, Yoann Dupont, Laurent Romary, Éric de la Clergerie, Djamé Seddah, Benoît Sagot
| Challenge: | Pretrained language models are now ubiquitous in Natural Language Processing, but their use in other languages is limited. |
| Approach: | They propose to train monolingual Transformer-based model for other languages using web crawled data instead of Wikipedia data and a relatively small web crawl dataset leads to better results. |
| Outcome: | The proposed model performs as well as those obtained using larger datasets. |
BARThez: a Skilled Pretrained French Sequence-to-Sequence Model (2021.emnlp-main)
Copied to clipboard
| Challenge: | Inductive transfer learning has taken the entire NLU field by storm, with models such as BERT and BART setting new state-of-the-art on countless tasks. |
| Approach: | They introduce a large-scale pretrained seq2seq model for French that is very competitive with state-of-the-art BERT-based French language models such as CamemBERT and FlauBERT. |
| Outcome: | The proposed model outperforms existing models on discriminative and generative tasks on a French summarization dataset. |
Assessing French Readability for Adults with Low Literacy: A Global and Local Perspective (2025.emnlp-main)
Copied to clipboard
| Challenge: | illiterate individuals are persons aged 15 years and above who cannot read and write with understanding a short simple statement on their everyday life. |
| Approach: | They propose a novel approach to assess french text readability for adults with low literacy skills using a global and segment-level difficulty scale. |
| Outcome: | The proposed approach addresses both global (full-text) and local (segment-level) difficulty scales. |