Papers by Benoît Crabbé
Improving Word Sense Induction through Adversarial Forgetting of Morphosyntactic Information (2024.starsem-1)
Copied to clipboard
| Challenge: | Contextualized word representations from pre-trained language models encode more information than is necessary for the identification of word senses and some of this information affect performance negatively in unsupervised settings. |
| Approach: | They propose to use a framework to erase specific information from pre-trained word models and create feature-invariant representations that are invariant to these ‘nuisance features’. |
| Outcome: | The proposed framework erases information from the representations of pre-trained language models, thereby creating feature-invariant representations. |
BERTrade: Using Contextual Embeddings to Parse Old French (2022.lrec-1)
Copied to clipboard
| Challenge: | a growing interest in digital humanities for automatic processing and annotation of historical texts is generating new models for historical languages. |
| Approach: | They use POS-tagging and dependency parsing to evaluate contextual word embedding models . Old French is one of the historical languages for which they have the largest amount of syntactically annotated data . |
| Outcome: | The proposed model can be used to improve performance in Old French, the authors show . they use POS-tagging and dependency parsing to evaluate the model's quality . |
Assessing the Capacity of Transformer to Abstract Syntactic Representations: A Contrastive Analysis Based on Long-distance Agreement (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing studies have shown that transformers are able to predict subject-verb agreement, demonstrating their ability to uncover an abstract representation of the sentence in an unsupervised way. |
| Approach: | They propose to compare how transformers handle subject-verb and object-past participle agreements in French using probing and counterfactual analysis methods. |
| Outcome: | The proposed model handles subject-verb and object-past participle agreements in a way consistent with their modeling in theoretical linguistics. |