Prix-LM: Pretraining for Multilingual Knowledge Base Construction

Challenge:	Existing methods to build and enrich multilingual knowledge bases have not been successful . knowledge expressed in different languages may be complementary and unequally distributed .
Approach:	They propose a model that integrates useful multilingual and KB-based factual knowledge into a single model.
Outcome:	The proposed model can provide richer combined knowledge than monolingual KBs.

Similar Papers

Overview

Challenge

Approach

Outcome

Enhancing Multilingual Language Model with Massive Multilingual Knowledge Triples (2022.emnlp-main)

Copied to clipboard

Linlin Liu, Xin Li, Ruidan He, Lidong Bing, Shafiq Joty, Luo Si

Semantic Text Processing

Challenge:	Existing methods for language model pretraining use limited knowledge graph data for knowledge-intensive tasks.
Approach:	They propose to make better use of multilingual annotations and language agnostic properties of KG triples for pretraining LMs.
Outcome:	The proposed models show significant performance improvements on a wide range of knowledge-intensive cross-lingual tasks.

Multilingual Knowledge Graph Completion from Pretrained Language Models with Knowledge Constraints (2023.findings-acl)

Copied to clipboard

Ran Song, Shizhu He, Shengxiang Gao, Li Cai, Kang Liu, Zhengtao Yu, Jun Zhao

Semantic Text Processing

Challenge:	Existing methods for multilingual knowledge graph completion do not align with mKGC tasks because of their English-centric bias.
Approach:	They propose to use multilingual pretrained language models to solve queries in different languages by reasoning a tail entity.
Outcome:	The proposed method outperforms the previous SOTA on Hits@1 and Hits @10 by 12.32% and 16.03% on public datasets.

X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Models (2020.emnlp-main)

Copied to clipboard

Zhengbao Jiang, Antonios Anastasopoulos, Jun Araki, Haibo Ding, Graham Neubig

Semantic Text Processing

Challenge:	Language models (LMs) capture factual knowledge by filling in the blanks of cloze-style prompts.
Approach:	They propose a code-switching-based method to improve the ability of multilingual LMs to access knowledge and verify its effectiveness on several benchmark languages.
Outcome:	The proposed method improves the ability of multilingual LMs to access knowledge and verify its effectiveness on several benchmark languages.

Language on Demand, Knowledge at Core: Composing LLMs with Encoder-Decoder Translation Models for Extensible Multilinguality (2026.acl-long)

Copied to clipboard

Mengyu Bu, Yang Feng

Multilinguality

Challenge:	Large language models exhibit strong general intelligence, yet their multilingual performance remains imbalanced.
Approach:	They propose a compositional encoder-LLM-decoder architecture that offloads multilingual understanding to external pretrained translation models while preserving the LLM as an English-centric core for general knowledge processing and reasoning.
Outcome:	The proposed architecture outperforms baseline models on four large language models across understanding, reasoning, and generation.

Pretrain-KGE: Learning Knowledge Representation from Pretrained Language Models (2020.findings-emnlp)

Copied to clipboard

Zhiyuan Zhang, Xiaoqian Liu, Yi Zhang, Qi Su, Xu Sun, Bin He

Semantic Text Processing

Challenge:	Existing knowledge graph embedding models suffer from limited knowledge representation due to sparse and noisy dataset annotations.
Approach:	They propose to use pretrained language models to enhance knowledge representation by leveraging world knowledge from pretrained models.
Outcome:	Extensive experiments show that the proposed framework can improve results over existing models.

Language Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queries (2021.eacl-main)

Copied to clipboard

Benjamin Heinzerling, Kentaro Inui

Semantic Text Processing

Challenge:	Pretrained language models have been suggested as an alternative or complement to structured knowledge bases . however, this paradigm has only been considered in a very limited setting .
Approach:	They propose a paradigm that allows LMs to store a large number of entities . they propose LM-as-KB paradigm which allows querying stored facts .
Outcome:	The proposed paradigm allows handling 21k entities whose name is found in common LM vocabularies . the proposed paradigm has only been considered in a very limited setting .

KBioXLM: A Knowledge-anchored Biomedical Multilingual Pretrained Language Model (2023.findings-emnlp)

Copied to clipboard

Lei Geng, Xu Yan, Ziqiang Cao, Juntao Li, Wenjie Li, Sujian Li, Xinjie Zhou, Yang Yang, Jun Zhang

Multilinguality

Challenge:	Existing models for multilingual biomedical training are monolingual, resulting in limited cross-lingual capability.
Approach:	They propose a model that transforms a multilingual biomedical corpus into a biomedically domain using a knowledge-anchored approach.
Outcome:	The proposed model outperforms monolingual and multilingual models in cross-lingual scenarios.

Multilingual Encoder Knows more than You Realize: Shared Weights Pretraining for Extremely Low-Resource Languages (2025.acl-long)

Copied to clipboard

Zeli Su, Ziyin Zhang, Guixian Xu, Jianing Liu, Xu Han, Ting Zhang, Yushuang Dong

Language Models

Challenge:	XLM-R and mBART have advanced multilingualism in NLP, but low-resource languages such as Tibetan, Uyghur, Kazakh, and Mongolian are underserved.
Approach:	They propose a framework for adapting multilingual encoders to text generation in extremely low-resource languages by reusing the weights between the encoder and the decoder.
Outcome:	The proposed framework performs better on various downstream tasks even when compared with much larger models.

Adapters for Enhanced Modeling of Multilingual Knowledge and Text (2022.findings-emnlp)

Copied to clipboard

Yifan Hou, Wenxiang Jiao, Meizhen Liu, Carl Allen, Zhaopeng Tu, Mrinmaya Sachan

Semantic Text Processing

Challenge:	Large language models learn facts from text corpora, but knowledge graphs contain facts in an explicit triple format, restricting their research and application.
Approach:	They propose to enhance multilingual language models with knowledge from multilingual knowledge graphs . they propose to use cross-lingual entity alignment and facts from MLKGs to improve performance .
Outcome:	The proposed model improves MLLMs with cross-lingual entity alignment and facts from multilingual knowledge graphs for many languages while maintaining performance on other general language tasks.

Multilingual Knowledge Graph Completion via Efficient Multilingual Knowledge Sharing (2025.findings-emnlp)

Copied to clipboard

Cunli Mao, Xiaofei Gao, Ran Song, Shizhu He, Shengxiang Gao, Kang Liu, Zhengtao Yu

Semantic Text Processing

Challenge:	Existing MKGC research ignores the shareability of cross-lingual knowledge.
Approach:	They propose a multilingual knowledge Graph Completion framework that leverages multilingual shared knowledge to significantly enhance performance through two components: Knowledge-level Grouped Mixture of Experts (KL-GMoE) and Iterative Entity Reranking (IER).
Outcome:	The proposed framework achieves improvements of 5.47%, 3.27%, and 1.01% in the Hits@1, Hits @3, and Hits_10 metrics, respectively, compared with existing state-of-the-art (SOTA) MKGC method.