Towards Tracing Trustworthiness Dynamics: Revisiting Pre-training Period of Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies focus on pre-trained LLMs to better understand and improve their trustworthiness. |
| Approach: | They apply linear probing to LLMs to explore five key dimensions of trustworthiness: reliability, privacy, toxicity, fairness, and robustness. |
| Outcome: | The proposed model can distinguish concepts in each trustworthiness dimension, suggesting that it can be trained in early pre-training. |
Similar Papers
A Survey of Post-Training Scaling in Large Language Models (2025.acl-long)
Copied to clipboard
Hanyu Lai, Xiao Liu, Junjie Gao, Jiale Cheng, Zehan Qi, Yifan Xu, Shuntian Yao, Dan Zhang, Jinhua Du, Zhenyu Hou, Xin Lv, Minlie Huang, Yuxiao Dong, Jie Tang
| Challenge: | Large language models (LLMs) have demonstrated proficiency in understanding and generating human natural languages. |
| Approach: | They propose a framework for scaling large language models using supervised fine-tuning, RLxF and test-time compute methodologies. |
| Outcome: | The proposed model can be used to understand and generate human natural languages. |
Exploring Forgetting in Large Language Model Pre-Training (2025.acl-long)
Copied to clipboard
| Challenge: | Existing research on task-level forgetting in LLMs has focused on pretraining . but, there is limited attention to finer-grained forgetting during training . |
| Approach: | They investigated the existence and measurement of forgetting in pre-training . they examined low-cost, straightforward methods to mitigate forgetting during the pre- training phase . |
| Outcome: | The proposed methods could be used to mitigate forgetting during the pre-training phase and offer insights into the dynamics of forgetting. |
On the Calibration of Large Language Models and Alignment (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models are becoming more popular and are proving to be reliable . however, their reliability is often understudied due to their uncertainty and complex structure . |
| Approach: | They conduct a systematic examination of the calibration of aligned language models throughout the entire construction process including pretraining and alignment training. |
| Outcome: | The results shed light on whether popular large language models are well-calibrated and how the training process influences model calibration. |
How Trustworthy are Open-Source LLMs? An Assessment under Malicious Demonstrations Shows their Vulnerabilities (2024.naacl-long)
Copied to clipboard
| Challenge: | Rapid progress in open-source Large Language Models (LLMs) is driving AI development, but lacks sufficient trustworthiness to detect and mitigate adversarial demonstrations. |
| Approach: | They propose an extended Chain of Utterances-based (CoU) prompting strategy to attack open-source LLMs. |
| Outcome: | The proposed attack strategy is based on malicious demonstrations and toxicity tests on open-source models. |
Probing Language Models for Pre-training Data Detection (2024.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown impressive capabilities, while raising concerns about the data contamination due to privacy issues and leakage of benchmark datasets in the pre-training phase. |
| Approach: | They propose to utilize the probing technique to examine the model’s internal activations to detect pre-training data contamination by examining the model's internal activates. |
| Outcome: | The proposed method outperforms baselines and achieves state-of-the-art performance on both WikiMIA and ArxivMIA, with additional experiments confirming its efficacy. |
Formality is Favored: Unraveling the Learning Preferences of Large Language Models on Data with Conflicting Knowledge (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models have shown excellent performance on knowledge-intensive tasks, but pretraining data tends to contain misleading and conflicting information. |
| Approach: | They systematically analyze LLMs’ learning preferences for data with conflicting knowledge. |
| Outcome: | The proposed model outperforms human-level models on knowledge-intensive tasks by analyzing pretraining data. |
A Close Look into the Calibration of Pre-trained Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) may fail in giving reliable estimates of their predictive uncertainty. |
| Approach: | They conduct fine-grained control experiments to study the dynamic change in PLMs’ calibration performance in training. |
| Outcome: | The proposed methods significantly reduce PLMs’ confidence in wrong predictions. |
Give Me the Facts! A Survey on Factual Knowledge Probing in Pre-trained Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models are trained on vast unlabeled data, rich in world knowledge. |
| Approach: | They propose a categorization scheme for factual probing methods based on how inputs, outputs and probed PLMs are adapted . they synthesize insights about knowledge retention and prompt optimization in PLM models and analyze obstacles to adopting them as knowledge bases . |
| Outcome: | The proposed method synthesizes insights about knowledge retention and prompt optimization in PLMs, analyzes obstacles to adopting them as knowledge bases and outline directions for future work. |
A Survey of Confidence Estimation and Calibration in Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated impressive capabilities across a wide range of tasks in various domains, but they can be unreliable due to factual errors in their generations. |
| Approach: | They summarize recent advances in LLM confidence estimation and calibration and outline their main lessons learned. |
| Outcome: | The proposed methods can be used to assess the reliability of models and to calibrate them across tasks. |
A Comprehensive Survey on the Trustworthiness of Large Language Models in Healthcare (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a survey of large language models in healthcare raises critical concerns around trustworthiness . trustworthy of LLMs in healthcare remains underexplored, lacking a systematic review . |
| Approach: | a new survey examines the trustworthiness of large language models in healthcare . a review examines how each dimension affects reliability and ethical deployment of LLMs . |
| Outcome: | The present study examines the trustworthiness of large language models in healthcare . it identifies key gaps in existing approaches and challenges posed by evolving paradigms . |