X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Language models (LMs) capture factual knowledge by filling in the blanks of cloze-style prompts. |
| Approach: | They propose a code-switching-based method to improve the ability of multilingual LMs to access knowledge and verify its effectiveness on several benchmark languages. |
| Outcome: | The proposed method improves the ability of multilingual LMs to access knowledge and verify its effectiveness on several benchmark languages. |
Similar Papers
Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual large-scale pretrained language models store factual knowledge, but large variations are observed across languages. |
| Approach: | They propose a ranking-based consistency metric to evaluate cross-lingual consistency of factual knowledge in multilingual PLMs. |
| Outcome: | The proposed metric evaluates cross-lingual consistency of factual knowledge across languages independently from accuracy. |
Tracing the Roots of Facts in Multilingual Language Models: Independent, Shared, and Transferred Knowledge (2024.eacl-long)
Copied to clipboard
| Challenge: | Using low-resource languages, multilingual language models (ML-LMs) have been developed to transfer factual knowledge across languages. |
| Approach: | They ask how ML-LMs acquire and represent factual knowledge . they use a multilingual factual information probing dataset to investigate ML . |
| Outcome: | The findings highlight the challenge of maintaining consistency factual knowledge across languages. |
Tracing Multilingual Factual Knowledge Acquisition in Pretraining (2025.findings-emnlp)
Copied to clipboard
Yihong Liu, Mingyang Wang, Amir Hossein Kargaran, Felicia Körner, Ercong Nie, Barbara Plank, François Yvon, Hinrich Schuetze
| Challenge: | Large Language Models are capable of recalling multilingual factual knowledge, but most studies evaluate only the final model, leaving the development of factual recall and crosslingual consistency unexplored. |
| Approach: | They trace how factual recall and crosslingual consistency evolve during pretraining, focusing on OLMo-7B as a case study. |
| Outcome: | The results show that fact frequency is the key to a better recall of multilingual facts, regardless of language, and some low-frequency facts in non-English languages can still be correctly recalled. |
Give Me the Facts! A Survey on Factual Knowledge Probing in Pre-trained Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models are trained on vast unlabeled data, rich in world knowledge. |
| Approach: | They propose a categorization scheme for factual probing methods based on how inputs, outputs and probed PLMs are adapted . they synthesize insights about knowledge retention and prompt optimization in PLM models and analyze obstacles to adopting them as knowledge bases . |
| Outcome: | The proposed method synthesizes insights about knowledge retention and prompt optimization in PLMs, analyzes obstacles to adopting them as knowledge bases and outline directions for future work. |
X-Fact: A New Benchmark Dataset for Multilingual Fact Checking (2021.acl-short)
Copied to clipboard
| Challenge: | Several fact-checking initiatives, such as PolitiFact, expend manual labor to investigate and determine the truthfulness of viral statements. |
| Approach: | They propose a multilingual dataset for factual verification of naturally existing claims . they use a benchmark to evaluate the multilingual models . |
| Outcome: | The proposed model achieves an F-score of around 40%, suggesting it is a challenging benchmark for multilingual fact-checking models. |
How Do Multilingual Language Models Remember Facts? (2025.findings-acl)
Copied to clipboard
| Challenge: | Prior research has focused on English monolingual models, but how these mechanisms generalize to non-English languages remains unexplored. |
| Approach: | They analyze three multilingual LLMs to find out how they can generalize recall mechanisms . they find that subject enrichment is language-independent, object extraction is language dependent . |
| Outcome: | The proposed model performs better in multilingual contexts than in English models . the model is more efficient in multi-lingual context, but it is more complex in multilinguistic models compared to English models. |
GuideQ: Framework for Guided Questioning for progressive informational collection and classification (2025.findings-naacl)
Copied to clipboard
| Challenge: | Using a new multilingual dataset, we examine how LLMs can be used to represent factual knowledge across languages. |
| Approach: | They propose a methodology to measure the extent of representation sharing across languages by repurposing knowledge editing methods. |
| Outcome: | The proposed model can answer a question consistently across languages and can store the answers in a shared representation for several languages. |
Do You Know About My Nation? Investigating Multilingual Language Models’ Cultural Literacy Through Factual Knowledge (2025.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual question-answering benchmarks do not factor in regional diversity in the information they capture and tend to be Western-centric. |
| Approach: | They propose to benchmark eight standard multilingual LLMs on XNationQA and evaluate them using two novel transference metrics. |
| Outcome: | The proposed model shows greater knowledge of cultural information in English than in the dominant language of the respective culture. |
Language Representation Projection: Can We Transfer Factual Knowledge across Languages in Multilingual Language Models? (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies show that multilingual pretrained models can recall factual knowledge without additional fine-tuning. |
| Approach: | They propose two parameter-free language representation projection modules to transfer factual knowledge between English and non-English languages. |
| Outcome: | The proposed modules improve factual knowledge retrieval accuracy and transferability across diverse non-English languages. |
Factual Consistency of Multilingual Pretrained Language Models (2022.findings-acl)
Copied to clipboard
| Challenge: | Recent work shows that monolingual English language models fill-in-the-blank differently for paraphrases describing the same fact. |
| Approach: | They propose a resource to analyze consistency of English language models . they find that mBERT is as inconsistent as English BERT in paraphrases . |
| Outcome: | The proposed model is as inconsistent as English BERT in English paraphrases, but it is more so for all the other 45 languages. |