Papers by Michael Lepori
Picking BERT’s Brain: Probing for Linguistic Dependencies in Contextualized Embeddings Using Representational Similarity Analysis (2020.coling-main)
Copied to clipboard
| Challenge: | Contextualized word embeddings can incorporate contextual information, whereas other embeddables cannot. |
| Approach: | They propose an approach to address this question using Representational Similarity Analysis (RSA) they investigate whether verb embeddings encode verb’s subject, pronoun embedds antecedent and full-sentence representations encode sentence’s head word . |
| Outcome: | The proposed approach can adjudicate between hypotheses about which aspects of context are encoded in representations of language. |
Unequal Representations: Analyzing Intersectional Biases in Word Embeddings Using Representational Similarity Analysis (2020.coling-main)
Copied to clipboard
| Challenge: | Specifically, we probe contextualized and non-contextualized word embeddings for evidence of intersectional biases against Black women. |
| Approach: | They propose a representational similarity analysis approach to detect human-like biases in word embeddings using representational similarities analysis. |
| Outcome: | The proposed approach aligns with intersectionality theory, which states that multiple identity categories layer on top of each other to create unique modes of discrimination that are not shared by any individual category. |
Representations of Syntax [MASK] Useful: Effects of Constituency and Dependency Structure in Recursive LSTMs (2020.acl-main)
Copied to clipboard
| Challenge: | a constituency-based network generalizes more robustly than a dependency-based one . a sequential LSTM can learn the appropriate rules governing these dependencies . |
| Approach: | They evaluate whether constituency-based networks introduce biases for syntactic structure . they find that constituency networks generalize more robustly than dependency networks . |
| Outcome: | The proposed model generalizes more robustly than a dependency-based model, the study shows . it shows that data augmentation can improve the robustness of the model on small data sets. |