Papers by Théo Desbordes
Can Transformers Process Recursive Nested Constructions, Like Humans? (2022.coling-1)
Copied to clipboard
| Challenge: | A recent study evaluated recursive processing in recurrent neural language models (RNN-LMs) and showed that such models perform below chance level on embedded dependencies within nested constructions. |
| Approach: | They evaluated recursive processing in recurrent neural language models and found that Transformers perform below chance level on embedded dependencies within nested constructions. |
| Outcome: | The proposed models perform below chance level on embedded dependencies within nested constructions, compared to humans. |
Assessing the influence of attractor-verb distance on grammatical agreement in humans and language models (2023.emnlp-main)
Copied to clipboard
| Challenge: | a subject-verb agreement in the presence of an attractor noun is a complex behavior . formal linguistic theories postulate the existence of an underlying structure that governs language processing . |
| Approach: | They hypothesize that the attractor-verb agreement may be a factor in grammatical decision-making . they hypothesized that classical models of attraction might suffice to explain this phenomenon . |
| Outcome: | The proposed model improves on humans and artificial neural networks while keeping the length of the sentence equal. |