Papers by Nicolas Garneau
Does Mapo Tofu Contain Coffee? Probing LLMs for Food-related Cultural Knowledge (2025.naacl-long)
Copied to clipboard
Li Zhou, Taelin Karidi, Wanlong Liu, Nicolas Garneau, Yong Cao, Wenyu Chen, Haizhou Li, Daniel Hershcovich
| Challenge: | Recent studies have highlighted the presence of cultural biases in Large Language Models (LLMs), yet lack a robust methodology to dissect these phenomena comprehensively. |
| Approach: | They propose a multilingual dataset centered on food-related cultural facts and variations in food practices. |
| Outcome: | The proposed model incorporates cultural context significantly and improves its ability to access cultural knowledge. |
Defining Knowledge: Bridging Epistemology and Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing literature on large language models (LLMs) define knowledge as a fact if it correctly completes a cloze sentence . but the predictions of semantically equivalent clozing sentences are inconsistent . |
| Approach: | They review standard definitions of knowledge in epistemology and formalize interpretations applicable to LLMs. |
| Outcome: | The authors compare the preferences of philosophers and computer scientists in terms of knowledge definitions and evaluation protocols for testing knowledge in accordance with the most relevant definitions. |
A Robust Self-Learning Method for Fully Unsupervised Cross-Lingual Mappings of Word Embeddings: Making the Method Robustly Reproducible as Well (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing methods for fully unsupervised cross-lingual mapping of word embeddings are available to achieve such a mapping . |
| Approach: | They reproduce the experiments of Artetxe and Sgaard (2018) . they propose a robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings. |
| Outcome: | The proposed method is feasible with minor assumptions, and it is able to be replicated in four languages. |
MuLan: A Study of Fact Mutability in Language Models (2024.naacl-short)
Copied to clipboard
| Challenge: | Pretrained and large language models encode factual knowledge, but factual information changes over time and mutates with the passage of time. |
| Approach: | They propose to use a model to evaluate the ability of English language models to anticipate time-contingency by comparing their models to a benchmark model. |
| Outcome: | The proposed model can predict the president of a country or the winner of sa championship in time, but it is difficult to update them due to their mutability. |
LeXFiles and LegalLAMA: Facilitating English Multinational Legal Language Model Development (2023.acl-long)
Copied to clipboard
| Challenge: | In this study, we examine the performance of legal-oriented pre-trained language models. |
| Approach: | They conduct a detailed analysis on the performance of legal-oriented pre-trained language models by examining their original objective, acquired knowledge, and legal language understanding capacities. |
| Outcome: | The results show that the models' size and pre-training corpora are important for the development of domain-specific models. |