How Distributed are Distributed Representations? An Observation on the Locality of Syntactic Information in Verb Agreement Tasks (2022.acl-short)
Copied to clipboard
| Challenge: | Using probing, causal analysis and feature selection, we find that syntactic information is encoded locally in the transformers representations consistent with the French grammar. |
| Approach: | They address the question of the localization of syntactic information encoded in transformers representations by probing, causal analysis and feature selection methods. |
| Outcome: | The proposed representations are consistent with the object-past participle agreement in French and are consistent in both languages. |
Similar Papers
Are Transformers a Modern Version of ELIZA? Observations on French Object Verb Agreement (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have shown that unsupervised sentence representations of neural networks encode syntactic information by observing that neural language models are able to predict the agreement between a verb and its subject. |
| Approach: | They propose to take an alternative look at these results by studying whether neural networks are able to build an abstract sentence representation rather than capture surface statistical regularities. |
| Outcome: | The proposed model can achieve high accuracy on the long-range French object-verb agreement, indicating a possible flaw in the model's syntactic ability. |
Assessing the Capacity of Transformer to Abstract Syntactic Representations: A Contrastive Analysis Based on Long-distance Agreement (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing studies have shown that transformers are able to predict subject-verb agreement, demonstrating their ability to uncover an abstract representation of the sentence in an unsupervised way. |
| Approach: | They propose to compare how transformers handle subject-verb and object-past participle agreements in French using probing and counterfactual analysis methods. |
| Outcome: | The proposed model handles subject-verb and object-past participle agreements in a way consistent with their modeling in theoretical linguistics. |
The Functional Relevance of Probed Information: A Case Study (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent studies have shown that transformer models like BERT rely on number information encoded in their representations of sentences’ subjects and head verbs when performing subject-verb agreement. |
| Approach: | They propose to use probing to find out which words contain functionally relevant information encoded in the representations of subject plurality and words that agree with it in number in BERT. |
| Outcome: | The proposed model only uses the subject plurality information encoded in its representations of the subject and words that agree with it in number. |
Probing Image-Language Transformers for Verb Understanding (2021.findings-acl)
Copied to clipboard
| Challenge: | Multimodal image-language transformers have achieved impressive results on a variety of tasks that rely on fine-tuning. |
| Approach: | They collect a dataset of image-sentence pairs consisting of 421 verbs . they evaluate pretrained image-language transformers and find they fail more in situations that require verb understanding compared to other parts of speech. |
| Outcome: | The proposed model trains on a manually-annotated and smaller dataset does better on the task. |
Understanding Cross-Lingual Alignment—A Survey (2024.findings-acl)
Copied to clipboard
| Challenge: | Cross-lingual alignment is the meaningful similarity of representations across languages in multilingual language models. |
| Approach: | They propose a taxonomy of methods to improve cross-lingual alignment . they argue that an effective trade-off between language-neutral and language-specific information is key . |
| Outcome: | The proposed methods can be applied to encoder models and encoder-decoder-only models . they show that language-neutral and language-specific information is key . |
Proceedings of the Fourth Workshop on Discourse in Machine Translation (DiscoMT 2019) (D19-65)
Copied to clipboard
| Challenge: | . - (EN) |
| Approach: | . - (EN) |
| Outcome: | . - (EN) |
Transactions of the Association for Computational Linguistics, Volume 8 (2020.tacl-1)
Copied to clipboard
| Challenge: | null |
| Approach: | null |
| Outcome: | null |
Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) (2024.acl-short)
Copied to clipboard
| Challenge: | . - (EN) |
| Approach: | . - (EN) |
| Outcome: | . - (EN) |
Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop) (2024.acl-srw)
Copied to clipboard
| Challenge: | ACL Student Research Workshop (SRW) 2024 will be held in conjunction with ACL 2024 . Authors can submit both research papers and thesis proposals as non-archival . |
| Approach: | ACL Student Research Workshop (SRW) 2024 will be held in conjunction with ACL 2024 . authors can submit both research papers and thesis proposals as non-archival . |
| Outcome: | The ACL Student Research Workshop (SRW) 2024 will be held in conjunction with ACL 2024. |
Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 5: Tutorial Abstracts) (2024.acl-tutorials)
Copied to clipboard
| Challenge: | ACL tutorial session is a highlight of the conference . it aims to provide attendees with a thorough introduction to key topics in our fast-evolving research field . |
| Approach: | the ACL tutorial session is a highlight of the conference . it aims to provide attendees with a thorough introduction to key topics in the field . |
| Outcome: | the ACL tutorial session is a highlight of the conference . the review process involved multiple conferences and three reviewers . |