Papers by Fabio Zanzotto
Exploring Linguistic Properties of Monolingual BERTs with Typological Classification among Languages (2023.findings-emnlp)
Copied to clipboard
Elena Ruzzetti, Federico Ranaldi, Felicia Logozzo, Michele Mastromattei, Leonardo Ranaldi, Fabio Zanzotto
| Challenge: | a recent study examined how models for typologically similar languages encode structural information. |
| Approach: | They propose to layer-wise compare transformers for typologically similar languages to observe similarities . they use a domain adaptation on semantically equivalent texts to measure similarity . |
| Outcome: | The proposed model outperforms all other models on unseen sentences . the proposed model is based on a typologically similar language . |
Measuring bias in Instruction-Following models with P-AT (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Instruction-Following Language Models (IFLMs) are promising and versatile tools for solving many downstream, information-seeking tasks. |
| Approach: | They propose a resource to test whether IFLMs are prone to biases . they cast WEAT word tests in promptized classification tasks and associate a metric - the bias score . |
| Outcome: | The proposed resource consists of 2310 prompts and tests gender and race biases in all the analyzed models. |
HANS, are you clever? Clever Hans Effect Analysis of Neural Systems (2024.starsem-1)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have been exhibiting outstanding abilities to reason around cognitive states, intentions, and reactions of all people involved, letting humans guide and comprehend day-to-day social interactions effectively. |
| Approach: | They propose to use multiple-choice questions (MCQ) benchmarks to assess LLMs' ability to reason around cognitive states, intentions, and reactions of all people involved to investigate their resilience abilities. |
| Outcome: | The proposed models exhibit exceptional abilities to reason around cognitive states, intentions, and reactions of all people involved, letting humans guide and comprehend day-to-day social interactions effectively. |
A Trip Towards Fairness: Bias and De-Biasing in Large Language Models (2024.starsem-1)
Copied to clipboard
| Challenge: | a little or a large bias in CtB-LLMs may cause huge harm . LLaMA and OPT families have an important bias in gender, race, religion, and profession. |
| Approach: | They propose to debiase three families of Very Large-Language Models with LORA to reduce bias by 4.12 points in the normalized stereotype score. |
| Outcome: | The proposed model reduces bias up to 4.12 points in the normalized stereotype score. |