Papers by Francisco Valentini
On the Interpretability and Significance of Bias Metrics in Texts: a PMI-based Approach (2023.acl-short)
Copied to clipboard
| Challenge: | Word embeddings have been used to quantify biases in texts for years, but their statistical properties and advantages have not been studied. |
| Approach: | They propose to use PMI-based metric to quantify bias in corpora by conditional probabilities and odds ratio to approximate it. |
| Outcome: | The proposed measure can be approximated by an odds ratio, which makes statistical inferences cost-effective and meaningful. |
Investigating the Frequency Distortion of Word Embeddings and Its Impact on Bias Metrics (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent research has shown that static word embeddings can encode words’ frequencies, but little has been studied about this behavior. |
| Approach: | They propose to use static word embeddings to encode words' frequencies and to assess the impact of this relationship on embeddable bias metrics. |
| Outcome: | The proposed model shows that word embeddings can produce higher similarity between high-frequency words than other embeddables. |
MessIRve: A Large-Scale Spanish Information Retrieval Dataset (2025.emnlp-main)
Copied to clipboard
Francisco Valentini, Viviana Cotik, Damián Furman, Ivan Bercovich, Edgar Altszyler, Juan Manuel Pérez
| Challenge: | Information retrieval (IR) is the task of finding relevant documents in response to a user query. |
| Approach: | They propose a large-scale Spanish IR dataset with almost 700,000 queries from Google’s autocomplete API and relevant documents sourced from Wikipedia. |
| Outcome: | The proposed dataset covers a wide variety of topics, unlike smaller datasets. |
The Undesirable Dependence on Frequency of Gender Bias Metrics Based on Word Embeddings (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have found word embeddings can capture semantic similarity but may be affected by word frequency. |
| Approach: | They find that word embeddings can capture semantic similarity but may be affected by word frequency . they compare this effect with an alternative metric based on Pointwise Mutual Information . |
| Outcome: | The proposed method does not depend on word frequency, but it does return female bias in low frequency words. |