Revealing the Parametric Knowledge of Language Models: A Unified Framework for Attribution Methods (2024.acl-long)
Copied to clipboard
| Challenge: | Language Models acquire parametric knowledge from their training process, embedding it within their weights. |
| Approach: | They propose a new evaluation framework to quantify and compare the knowledge revealed by Instance Attribution and Neuron Attributions. |
| Outcome: | The proposed evaluation framework compares the knowledge revealed by IA and NA with that of neuron attribution methods. |
Similar Papers
Neuron-Level Knowledge Attribution in Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for attribution of knowledge in large language models struggle to operate at neuron level due to computational constraints. |
| Approach: | They propose a static method for pinpointing significant neurons using three metrics . they also propose identifying "query neurons" which activate these "value neurons" |
| Outcome: | The proposed method shows superior performance across three metrics compared to seven other methods . it analyzes six types of knowledge across attention and feed-forward network layers . |
IAEval: A Comprehensive Evaluation of Instance Attribution on Natural Language Understanding (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Instance attribution (IA) aims to identify the training instances leading to the prediction of a test example. |
| Approach: | They propose a systematic and comprehensive evaluation scheme covering four significant requirements: sufficiency, completeness, stability and plausibility. |
| Outcome: | The proposed evaluation scheme covers four significant requirements: sufficiency, completeness, stability and plausibility. |
Towards Tracing Knowledge in Language Models Back to the Training Data (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work on training data attribution (TDA) may offer effective tools for identifying such examples, known as "proponents". |
| Approach: | They propose a benchmark to identify which training examples taught an LM to generate a particular factual assertion. |
| Outcome: | The proposed methods have lower proponent-retrieval precision than baselines that do not have access to the LM. |
Overestimation of Syntactic Representation in Neural Language Models (2020.acl-main)
Copied to clipboard
| Challenge: | Several testing methodologies have been developed to probe models’ syntactic representations. |
| Approach: | They propose a method to determine syntactic structure by training a model on strings generated according to a template and testing its ability to distinguish between similar ones with different syntax. |
| Outcome: | The proposed method reproduces positive results with two non-syntactic baseline language models: an n-gram model and an LSTM model trained on scrambled inputs. |
How Can We Know What Language Models Know? (2020.tacl-1)
Copied to clipboard
| Challenge: | Recent work examines knowledge contained in language models by having the LM fill in the blanks of prompts such as “Obama is a __ by profession”. |
| Approach: | They propose mining-based and paraphrasing-based methods to automatically generate high-quality and diverse prompts, as well as ensemble methods to combine answers from different prompts. |
| Outcome: | The proposed methods improve accuracy from 31.1% to 39.6% on the LAMA benchmark for extracting relational knowledge from LMs. |
How Do Language Models Acquire Character-Level Information? (2026.eacl-long)
Copied to clipboard
| Challenge: | Language models (LMs) implicitly encode character-level information, despite not being explicitly provided during training. |
| Approach: | They analyze how language models acquire character-level knowledge by comparing them to standard settings. |
| Outcome: | The results show that LMs do not treat words as opaque tokens, but instead treat them as tokens. |
Measuring the Knowledge Acquisition-Utilization Gap in Pretrained Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent research has demonstrated that pre-trained language models acquire a broad range of knowledge about linguistic structures, encyclopedic relations, levels of commonsense, and even coding and reasoning rules. |
| Approach: | They propose a systematic framework to measure parametric knowledge utilization in pre-trained language models by extracting parametric information from a PLM and constructing a downstream task around this extracted knowledge. |
| Outcome: | The proposed framework extracts parametric knowledge from a PLM and constructs a downstream task around this extracted knowledge. |
How Training Data Shapes the Use of Parametric and In-Context Knowledge in Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models leverage parametric and in-context knowledge in training . however, when these sources conflict, models arbitrate based on their internal confidence . |
| Approach: | They conduct controlled experiments using synthetic corpora to identify data properties that shape knowledge utilization. |
| Outcome: | The results show that the robust use of both knowledge sources is an emergent property . the results provide guidance for designing training data that supports the reliability of parametric and in-context knowledge in language models. |
What Has Been Enhanced in my Knowledge-Enhanced Language Model? (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing knowledge integration methods such as linear probes and prompts have key limitations in answering these questions. |
| Approach: | They propose a new probe model which integrates external knowledge from knowledge graphs into pretrained language models (LMs) ERNIE and K-Adapter are proposed as KI methods . |
| Outcome: | The proposed model interprets two well-known KELMs using graph attention on the corresponding knowledge graph for interpretation. |
Parametric Knowledge is Not All You Need: Toward Honest Large Language Models via Retrieval of Pretraining Data (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models are highly capable of answering questions, but they are often unaware of their own knowledge boundary, i.e., knowing what they know and what they don’t know. |
| Approach: | They propose a method to evaluate LLM honesty using Pythia with publicly available pretraining data. |
| Outcome: | The proposed method is based on Pythia, a truly open LLM with publicly available pretraining data. |