From Language Modeling to Instruction Following: Understanding the Behavior Shift in LLMs after Instruction Tuning (2024.naacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have achieved remarkable success in aligning with user intentions. |
| Approach: | They develop local and global explanation methods and a feed-forward-based method for input-output attribution to investigate the impact of instruction tuning on user intentions. |
| Outcome: | The proposed method compares explanations from pre-trained and instruction-tuned models . it empowers LLMs to recognize the instruction parts of user prompts, it encourages response generation . |
Similar Papers
Advancing Language Models through Instruction Tuning: Recent Progress and Challenges (2025.emnlp-tutorials)
Copied to clipboard
| Challenge: | tutorial addresses three critical questions within the field of instruction tuning: (1) What are the current focal points in instruction tuning research? (2) What are best practices in training an instruction-following model? (3) What new challenges have emerged? |
| Approach: | This tutorial presents a systematic overview of recent advances in instruction tuning. |
| Outcome: | The tutorial covers different stages in model training: supervised fine-tuning, preference optimization, and reinforcement learning. |
Layer by Layer: Uncovering Where Multi-Task Learning Happens in Instruction-Tuned Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained large language models retain task-specific knowledge, but where and to what extent they retain it remains unexplored. |
| Approach: | They investigate the task-specific information encoded in pre-trained LLMs and the effects of instruction tuning on their representations across over 60 NLP tasks. |
| Outcome: | The results show that pre-trained models retain task-specific knowledge . some tasks are already encoded in pre-train models, but others benefit from instruction tuning. |
Demystifying Instruction Mixing for Fine-tuning Large Language Models (2024.acl-srw)
Copied to clipboard
| Challenge: | Instruction tuning is effective for aligning large language models with human instructions, but the procedure to optimizing the mixing of instruction datasets is still unclear. |
| Approach: | They categorize instructions into three primary types: NLP downstream tasks, coding, and general chat. |
| Outcome: | The proposed method improves performance of large language models (LLMs) but it is difficult to combine different instruction datasets to optimize overall performance. |
Dynamics of Instruction Fine-Tuning for Chinese Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Instruction tuning is a burgeoning method to elicit the general intelligence of Large Language Models. |
| Approach: | They investigate the effects of data quantity, model size, and data construction methods on instruction tuning for Chinese LLMs. |
| Outcome: | The proposed model includes over 40,000 high-quality instruction instances covering ten underlying abilities. |
Roles of Scaling and Instruction Tuning in Language Perception: Model vs. Human Attention (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent large language models (LLMs) have shown strong abilities to understand natural language, but how these factors affect the models’ language perception is unclear. |
| Approach: | They compare the self-attention of several existing large language models in different sizes to assess the effect of scaling and instruction tuning on language perception. |
| Outcome: | The proposed models are closer to non-native speakers than native speakers in attention, suggesting a sub-optimal language perception of all models. |
Instruction-tuned Language Models are Better Knowledge Learners (2024.acl-long)
Copied to clipboard
Zhengbao Jiang, Zhiqing Sun, Weijia Shi, Pedro Rodriguez, Chunting Zhou, Graham Neubig, Xi Lin, Wen-tau Yih, Srini Iyer
| Challenge: | Large language models store factual knowledge in parameters, but it can become outdated as the work evolves . pre-instruction-tuning improves ability of LLMs to absorb knowledge from new documents . |
| Approach: | They propose a method that instruction-tunes on questions prior to training on documents . they propose to use QA pairs to update factual knowledge of large language models . |
| Outcome: | The proposed method outperforms instruction-tuning on documents by 17.8%. |
Exploring the Impact of Instruction-Tuning on LLM’s Susceptibility to Misinformation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies highlight that large language models are receptive to external information that contradicts their parametric knowledge, but little research has been conducted on the direct impact of instruction-tuning on this phenomenon. |
| Approach: | They examine how instruction-tuning influences LLMs' susceptibility to misinformation, particularly in knowledge conflict situations. |
| Outcome: | The proposed model is more user-oriented and more likely to accept misinformation when it is presented by the user. |
Multilingual Instruction Tuning With Just a Pinch of Multilinguality (2024.findings-acl)
Copied to clipboard
| Challenge: | Using multilingual instruction tuning, large language models can be used to follow instructions in multiple languages . a multilingual model can be tuned on a wide range of languages, yet most datasets are limited to English . |
| Approach: | They investigate how multilinguality during instruction tuning affects instruction-following across languages . they find that only 40 multilingual examples improve multilingual instruction- follow . |
| Outcome: | The results show that multilingual models perform better on multilingual mixtures compared to monolingual models . the results suggest that building multilingual instruction-tuned models can be done with only 2-4 languages . |
LongForm: Effective Instruction Tuning with Reverse Instructions (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work on instruction tuning relies on expensive human annotation and crowd-sourced datasets with alignment issues. |
| Approach: | They propose a method to generate instructions via LLMs from human-written corpus examples using reverse instructions. |
| Outcome: | The proposed method outperforms larger language models without instruction tuning on tasks such as story/recipe generation and long-form question answering. |
Revealing the Inherent Instructability of Pre-Trained Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained large language models perform multitask learning during their pre-training . a new technique, Response Tuning, removes the instruction and its corresponding mapping to the response from instruction tuning. |
| Approach: | They propose a method which removes the instruction and its mapping to the response from instruction tuning. |
| Outcome: | The proposed model can respond to a wide range of instructions . it can recognize and reject unsafe queries after learning from response data. |