Know the Unknown: An Uncertainty-Sensitive Method for LLM Instruction Tuning (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have impressive capabilities but face significant challenges from hallucinations, which arise from insufficient knowledge or context. |
| Approach: | They propose a novel two-stage approach for contextual question answering that enhances LLMs’ ability to recognise their knowledge boundaries while the second reinforces instruction adherence through carefully designed causal prompts. |
| Outcome: | The proposed approach significantly reduces incorrect answers in contextual QA and improves models’ faithfulness to parametric knowledge, mitigating hallucinations in general QA tasks. |
Similar Papers
R-Tuning: Instructing Large Language Models to Say ‘I Don’t Know’ (2024.naacl-long)
Copied to clipboard
Hanning Zhang, Shizhe Diao, Yong Lin, Yi Fung, Qing Lian, Xingyao Wang, Yangyi Chen, Heng Ji, Tong Zhang
| Challenge: | Existing methods for instruction tuning force the model to complete a sentence no matter whether it knows the knowledge or not. |
| Approach: | They propose a new approach to tuning large language models to refrain from answering questions beyond its parametric knowledge by identifying the disparity in parametric and parametric information. |
| Outcome: | The proposed approach improves a model’s ability to answer known questions and refrain from answering unknown questions. |
MAC-Tuning: LLM Multi-Compositional Problem Reasoning with Enhanced Knowledge Boundary Awareness (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models produce non-existing facts when faced with questions outside their parametric knowledge, which undermines their reliability. |
| Approach: | They propose a method that separates the learning of answer prediction and confidence estimation during fine-tuning on instruction data. |
| Outcome: | Experiments on multiple models and different model sizes show that the proposed method outperforms baselines by up to 25% in average precision. |
Can LLMs Learn Uncertainty on Their Own? Expressing Uncertainty Effectively in A Self-Training Manner (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) exhibit excessive, random, and uninformative uncertainty rendering them unsuitable for decision-making in human-computer interactions. |
| Approach: | They propose an uncertainty-aware instruction tuning method that aligns LLMs’ perception with the probabilistic uncertainty of the generation. |
| Outcome: | The proposed method improves LLMs' performance by 45.2%, with reasonably good out-of-domain generalization capabilities. |
From Misleading Queries to Accurate Answers: A Three-Stage Fine-Tuning Method for LLMs (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods focus on correcting the output but overlook the ability of LLMs to detect and correct misleading content in the input itself. |
| Approach: | They propose a three-stage fine-tuning method that improves LLMs' ability to detect and correct misleading information in input queries. |
| Outcome: | The proposed method improves accuracy and factuality of LLM responses while also reducing hallucinations. |
KnowTuning: Knowledge-aware Fine-tuning for Large Language Models (2024.emnlp-main)
Copied to clipboard
Yougang Lyu, Lingyong Yan, Shuaiqiang Wang, Haibo Shi, Dawei Yin, Pengjie Ren, Zhumin Chen, Maarten Rijke, Zhaochun Ren
| Challenge: | Large language models (LLMs) are a default solution for many natural language processing tasks. |
| Approach: | They propose a knowledge-aware fine-tuning method to improve LLMs' knowledge awareness . they propose augmentation and comparison stages to improve accuracy and reliability . |
| Outcome: | The proposed method generates more facts with less factual error rate under fine-grained facts evaluation. |
Exploring the Impact of Instruction-Tuning on LLM’s Susceptibility to Misinformation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies highlight that large language models are receptive to external information that contradicts their parametric knowledge, but little research has been conducted on the direct impact of instruction-tuning on this phenomenon. |
| Approach: | They examine how instruction-tuning influences LLMs' susceptibility to misinformation, particularly in knowledge conflict situations. |
| Outcome: | The proposed model is more user-oriented and more likely to accept misinformation when it is presented by the user. |
Knowledge of Knowledge: Exploring Known-Unknowns Uncertainty with Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Known-unknown questions are characterized by high uncertainty due to the absence of definitive answers. |
| Approach: | They introduce a dataset with known-unknown questions and establish a categorization framework to clarify the origins of uncertainty in such queries. |
| Outcome: | The proposed model improved in distinguishing between known and unknown queries within open-ended question-answering scenarios. |
Instruction-tuned Language Models are Better Knowledge Learners (2024.acl-long)
Copied to clipboard
Zhengbao Jiang, Zhiqing Sun, Weijia Shi, Pedro Rodriguez, Chunting Zhou, Graham Neubig, Xi Lin, Wen-tau Yih, Srini Iyer
| Challenge: | Large language models store factual knowledge in parameters, but it can become outdated as the work evolves . pre-instruction-tuning improves ability of LLMs to absorb knowledge from new documents . |
| Approach: | They propose a method that instruction-tunes on questions prior to training on documents . they propose to use QA pairs to update factual knowledge of large language models . |
| Outcome: | The proposed method outperforms instruction-tuning on documents by 17.8%. |
Beyond "I Don’t Know": Evaluating LLM Self-Awareness in Discriminating Data and Model Uncertainty (2026.acl-long)
Copied to clipboard
| Challenge: | Prior studies treat refusal as a generic "I don't know" lack of distinction limits downstream action decisions like requesting clarification or invoking external tools. |
| Approach: | They propose a benchmark to evaluate explicit uncertainty attribution in large language models. |
| Outcome: | The proposed method improves uncertainty attribution while preserving answer accuracy. |
Tuna: Instruction Tuning using Feedback from Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | LLms like LLaMA have shown to be cost-effective for generating better responses . however, the instruction-tuned model has only seen one response per instruction . |
| Approach: | They propose to fine tune an instruction-tuned LLM using probabilistic ranking and contextual ranking approaches to increase the likelihood of generating better responses. |
| Outcome: | The proposed model improves on Super Natural Instructions, LMentry and Vicuna QA. |