Fanyi Yang, Jianfeng Liu, Xin Zhang, Haoyu Liu, Xixin Cao, Yuefeng Zhan, Hao Sun, Weiwei Deng, Feng Sun, Qi Zhang
| Challenge: | Instruction tuning has enabled large language models to achieve remarkable performance, yet its success heavily depends on the availability of high-quality instruction-response pairs. |
| Approach: | They propose a mutual alignment framework which enforces coherence between instructions and responses through mutual constraints. |
| Outcome: | The proposed framework generalizes well across model architectures and sizes, achieving state-of-the-art performance on LLaMA, Mistral, and Qwen models across diverse benchmarks. |
Similar Papers
Demystifying Instruction Mixing for Fine-tuning Large Language Models (2024.acl-srw)
Copied to clipboard
| Challenge: | Instruction tuning is effective for aligning large language models with human instructions, but the procedure to optimizing the mixing of instruction datasets is still unclear. |
| Approach: | They categorize instructions into three primary types: NLP downstream tasks, coding, and general chat. |
| Outcome: | The proposed method improves performance of large language models (LLMs) but it is difficult to combine different instruction datasets to optimize overall performance. |
CodecLM: Aligning Language Models with Tailored Synthetic Data (2024.findings-naacl)
Copied to clipboard
Zifeng Wang, Chun-Liang Li, Vincent Perot, Long Le, Jin Miao, Zizhao Zhang, Chen-Yu Lee, Tomas Pfister
| Challenge: | Recent work on generating diverse instructions and applying LLM to increase instruction complexity neglects downstream use cases. |
| Approach: | They propose a framework for generating high-quality synthetic data for LLM alignment with different downstream instruction distributions and LLMs. |
| Outcome: | Experiments on four open-domain instruction using the proposed framework validate the effectiveness of CodecLM over the current state-of-the-art. |
Investigating Multilingual Instruction-Tuning: Do Polyglot Models Demand for Multilingual Instructions? (2024.emnlp-main)
Copied to clipboard
Alexander Weber, Klaudia Thellmann, Jan Ebert, Nicolas Flores-Herr, Jens Lehmann, Michael Fromm, Mehdi Ali
| Challenge: | a study of multilingual pre-trained LLMs on parallel instruction-tuning benchmarks shows that instruction-following models can be used across languages by up to 9.9%. |
| Approach: | They conduct an extensive study of the performance of multilingual pre-trained LLMs instruction-tuned on parallel instruction-uning datasets. |
| Outcome: | The proposed model improves cross-lingual instruction following capabilities by 9.9% on a large and mid-sized LLM on parallel instruction-tuning datasets. |
Improving the Robustness of Large Language Models via Consistency Alignment (2024.lrec-main)
Copied to clipboard
Yukun Zhao, Lingyong Yan, Weiwei Sun, Guoliang Xing, Shuaiqiang Wang, Chong Meng, Zhicong Cheng, Zhaochun Ren, Dawei Yin
| Challenge: | Large language models have shown tremendous success in following user instructions and generating helpful responses, but their robustness is still far from optimal. |
| Approach: | They propose a two-stage training framework that helps a model generalize on following instructions via similar instruction augmentations. |
| Outcome: | The proposed training framework improves diversity and aligns the model with human expectations by differentiating subtle differences in similar responses. |
CommonIT: Commonality-Aware Instruction Tuning for Large Language Models via Data Partitions (2024.emnlp-main)
Copied to clipboard
| Challenge: | Current studies have focused on fine-tuning, but the use of instruction tuning is not as effective as fine-cuning. |
| Approach: | They propose a commonality-aware instruction tuning strategy to cluster instruction datasets into distinct groups with three proposed metrics Task, Embedding and Length. |
| Outcome: | The proposed strategy boosts an average improvement of 2.1% on the general domain and 5.2% on the special domain. |
Better Alignment with Instruction Back-and-Forth Translation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | et al., 2023) proposes a method to improve instruction-tuning data . e.g., we generate synthetic instructions using the backtranslation approach . |
| Approach: | They propose a method to improve instruction-tuning data using web-based inputs . they generate synthetic instructions using the backtranslation approach and filter the generated data . |
| Outcome: | The proposed method improves the quality of instruction-tuning data based on preprocessed texts . it yields better AlpacaEval win rates than direct distillation . |
Multilingual Pretraining and Instruction Tuning Improve Cross-Lingual Knowledge Alignment, But Only Shallowly (2024.naacl-long)
Copied to clipboard
| Challenge: | Current large language models show imbalance abilities in different languages . authors propose two approaches to improve cross-lingual knowledge alignment . |
| Approach: | They propose a framework to assess cross-lingual knowledge alignment of large language models . they propose multilingual pretraining and multilingual instruction tuning to address this problem . |
| Outcome: | The proposed framework assesses the cross-lingual knowledge alignment of LLMs in performance, consistency and conductivity levels. |
Dynamics of Instruction Fine-Tuning for Chinese Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Instruction tuning is a burgeoning method to elicit the general intelligence of Large Language Models. |
| Approach: | They investigate the effects of data quantity, model size, and data construction methods on instruction tuning for Chinese LLMs. |
| Outcome: | The proposed model includes over 40,000 high-quality instruction instances covering ten underlying abilities. |
COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuning (2025.findings-naacl)
Copied to clipboard
Yuelin Bai, Xeron Du, Yiming Liang, Leo Jin, Junting Zhou, Ziqiang Liu, Feiteng Fang, Mingshan Chang, Tianyu Zheng, Xincheng Zhang, Nuo Ma, Zekun Moore Wang, Ruibin Yuan, Haihong Wu, Hongquan Lin, Wenhao Huang, Jiajun Zhang, Chenghua Lin, Jie Fu, Min Yang, Shiwen Ni, Ge Zhang
| Challenge: | Existing datasets for Chinese instruction tuning are not well-aligned with Chinese users’ interaction patterns. |
| Approach: | They propose to use Chinese instruction tuning datasets to improve instruction fine-tuning for Chinese users. |
| Outcome: | The proposed dataset shows that Chinese models achieve competitive performance in diverse benchmarks. |
Aligning Instruction Tasks Unlocks Large Language Models as Zero-Shot Relation Extractors (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work has shown that fine-tuning large language models on large instruction-following datasets improves their performance on a wide range of NLP tasks, but they fail to outperform small LMs on relation extraction (RE), a fundamental information extraction task. |
| Approach: | They propose a framework that aligns RE with question answering (QA), a predominant task in instruction-tuning datasets. |
| Outcome: | The proposed framework outperforms small LLMs on relation extraction (RE), a fundamental information extraction task, by a large margin. |