Papers by Hongzhi Luan
BOSE: A Systematic Evaluation Method Optimized for Base Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluation methods for large language models (LLMs) are inadequate to provide solid conclusions for key experiments such as data ablation and scaling law. |
| Approach: | They propose a method specifically designed to optimize the evaluation of base models by incorporating two innovations: In-Context Light-instruction Prompt and Blank-ppl for multi-choice tasks with candidate options. |
| Outcome: | The proposed method significantly improves stability and consistency of evaluations during pre-training and consistency between base and instruct models. |