Papers by Thierry Poibeau
On “Human Parity” and “Super Human Performance” in Machine Translation Evaluation (2022.lrec-1)
Copied to clipboard
| Challenge: | In this paper, we reassess claims of human parity and super human performance in machine translation. |
| Approach: | They reassess claims of human parity and super human performance in machine translation . they argue that human translation involves much more than what is embedded in automatic systems . |
| Outcome: | The proposed results show that human translation involves much more than what is embedded in automatic systems. |
Probing for the Usage of Grammatical Number (2022.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models can be used to perform a wide array of NLP tasks, but their encoding is still a mystery. |
| Approach: | They propose a usage-based probing setup to find an encoding that the model actually uses, and propose 'a use-based approach' they propose to use a behavioral task to remove the linguistic property, and to identify which encodes are used to transfer information from a noun to its head verb. |
| Outcome: | The proposed encodings are based on a behavioral task which cannot be solved without the linguistic property. |
Subject Verb Agreement Error Patterns in Meaningless Sentences: Humans vs. BERT (2022.coling-1)
Copied to clipboard
| Challenge: | Existing research shows that humans are prone to making agreement errors with specific constructions. |
| Approach: | They compare the performance of BERT-base and that of humans using crowdsourcing . they find that meaningfulness is stronger for BERT than for humans . |
| Outcome: | The proposed model performs better than humans on a crowdsourcing experiment . |
Word Order Matters When You Increase Masking (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that word order information is not always useful for pre-trained models to solve downstream tasks. |
| Approach: | They examine the effect of removing position encodings on the pre-training objective itself to see whether models can reconstruct position information from co-occurrences. |
| Outcome: | The proposed model can reconstruct position information from co-occurrences without explicit encoding, but not on some tasks. |
On the Correspondence between Compositionality and Imitation in Emergent Neural Communication (2023.findings-acl)
Copied to clipboard
| Challenge: | a study examining compositionality and imitation learning in a Lewis game demonstrates that it is difficult to imitate compositional languages. |
| Approach: | They explore the link between compositionality and imitation in a Lewis game . they show that the learning algorithm used to imitate is crucial . |
| Outcome: | The proposed model improves compositionality and imitation in a Lewis game . the study shows that compositional languages are easier to imitate . |
Multilingual Dependency Parsing for Low-Resource Languages: Case Studies on North Saami and Komi-Zyrian (L18-1)
Copied to clipboard
| Challenge: | Developing systems for low-resource languages is a crucial issue for Natural Language Processing (NLP). |
| Approach: | They propose a method for parsing low-resource languages with very small training corpora using multilingual word embeddings and annotated corporata of larger languages. |
| Outcome: | The proposed method improves dependency parsing for low-resource languages with very small training corpora compared to previous work . it also explores whether contemporary contact languages or genetically related languages would be the most fruitful starting point for multilingual parsers. |
Does BERT really agree ? Fine-grained Analysis of Lexical Dependence on a Syntactic Task (2022.findings-acl)
Copied to clipboard
| Challenge: | lexically-independent subject-verb number agreement (NA) is performed by transformer-based neural language models (NLMs) . but when as little as one attractor is present, the model fails to perform lexical generalization . |
| Approach: | They propose to disrupt lexical patterns found in naturally occurring stimuli for each targeted structure in a novel fine-grained analysis of BERT's behavior. |
| Outcome: | The proposed model generalizes well for simple templates, but fails to perform lexically-independent generalization when as little as one attractor is present. |