Can LMs Learn New Entities from Descriptions? Challenges in Propagating Injected Knowledge (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for updating knowledge show little propagation of injected knowledge. |
| Approach: | They propose to inject individual facts into LMs and evaluate whether they can propagate injected facts while not changing predictions on other contexts. |
| Outcome: | The proposed model can make inferences based on injected facts and propagate them . existing methods show little propagation of injected knowledge . |
Similar Papers
Entity Cloze By Date: What LMs Know About Unseen Entities (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing literature provides benchmarks to measure LMs' knowledge about entities . |
| Approach: | They propose a framework to analyze what language models can infer about new entities that did not exist when they were pretrained. |
| Outcome: | The proposed framework shows that models more informed about the entities achieve lower perplexity on this benchmark. |
Does Pre-training Induce Systematic Inference? How Masked Language Models Acquire Commonsense Knowledge (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing evidence suggests that pre-trained Transformers encode commonsense knowledge . however, the extent to which this knowledge is acquired is unclear . |
| Approach: | They inject verbalized knowledge into pre-training minibatches and evaluate generalization . they find generalization does not improve over the course of pre- training from scratch . |
| Outcome: | The proposed model generalizes to supported inferences after pre-training on the injected knowledge. |
Do Language Models Perform Generalizable Commonsense Inference? (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent work has applied pretrained language models to populate commonsense knowledge graphs (CKGs) but there is a lack of understanding on their generalization to multiple CKGs, unseen relations, and novel entities. |
| Approach: | They analyze the ability of pretrained language models to perform generalizable commonsense inference in terms of knowledge capacity, transferability and induction. |
| Outcome: | The proposed models can adapt to different schemas defined by multiple CKGs but fail to generalize to new relations. |
How Can We Know What Language Models Know? (2020.tacl-1)
Copied to clipboard
| Challenge: | Recent work examines knowledge contained in language models by having the LM fill in the blanks of prompts such as “Obama is a __ by profession”. |
| Approach: | They propose mining-based and paraphrasing-based methods to automatically generate high-quality and diverse prompts, as well as ensemble methods to combine answers from different prompts. |
| Outcome: | The proposed methods improve accuracy from 31.1% to 39.6% on the LAMA benchmark for extracting relational knowledge from LMs. |
Give Me the Facts! A Survey on Factual Knowledge Probing in Pre-trained Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models are trained on vast unlabeled data, rich in world knowledge. |
| Approach: | They propose a categorization scheme for factual probing methods based on how inputs, outputs and probed PLMs are adapted . they synthesize insights about knowledge retention and prompt optimization in PLM models and analyze obstacles to adopting them as knowledge bases . |
| Outcome: | The proposed method synthesizes insights about knowledge retention and prompt optimization in PLMs, analyzes obstacles to adopting them as knowledge bases and outline directions for future work. |
Data Doping or True Intelligence? Evaluating the Transferability of Injected Knowledge in LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a study shows that comprehension-intensive fine-tuning tasks retain knowledge longer . however, all models exhibit significant performance drops when applying injected knowledge in broader contexts . |
| Approach: | study: comprehension-intensive fine-tuning tasks achieve higher knowledge retention rates . larger models show improved retention across all task types, study finds . |
| Outcome: | a new study shows that comprehension-intensive fine-tuning tasks retain knowledge better than mapping-oriented tasks despite exposure to identical factual content. |
X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Language models (LMs) capture factual knowledge by filling in the blanks of cloze-style prompts. |
| Approach: | They propose a code-switching-based method to improve the ability of multilingual LMs to access knowledge and verify its effectiveness on several benchmark languages. |
| Outcome: | The proposed method improves the ability of multilingual LMs to access knowledge and verify its effectiveness on several benchmark languages. |
Can Language Models Learn Embeddings of Propositional Logic Assertions? (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for automating reasoning can no longer be used for natural language tasks. |
| Approach: | They propose to use transformer-based language models to reason about knowledge expressed in natural language rather than using LMs to perform reasoning directly. |
| Outcome: | The proposed approach is feasible to some extent, but lacks robustness. |
Injecting Domain-Specific Knowledge into Large Language Models: A Comprehensive Survey (2025.findings-emnlp)
Copied to clipboard
| Challenge: | specialized LLMs are often limited in domain-specific applications that require specialized knowledge. |
| Approach: | They provide a comprehensive overview of four key methods to enhance large language models by integrating domain-specific knowledge. |
| Outcome: | The proposed methods are categorized into four key approaches: dynamic knowledge injection, static knowledge embedding, modular adapters, and prompt optimization. |
LM-CORE: Language Models with Contextually Relevant External Knowledge (2022.findings-naacl)
Copied to clipboard
| Challenge: | Large pre-trained language models can capture factual knowledge in their parameters but storing large amounts of knowledge in the model parameters is sub-optimal given the ever-growing amounts of information and resource requirements. |
| Approach: | They propose a framework that provides explicit access to contextually relevant structured knowledge to the model and train it to use that knowledge. |
| Outcome: | The proposed framework outperforms state-of-the-art knowledge-enhanced language models on knowledge probing tasks and can handle knowledge updates. |