Papers by Baihe Huang
Enhancing Language Model Alignment: A Confidence-Based Approach to Label Smoothing (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have remarkable capabilities across various domains . Reinforcement Learning with Human Feedback (RLHF) phase is crucial for training . label smoothing is a technique that replaces hard labels with soft labels . |
| Approach: | They propose a method that iteratively updates the label smoothing parameter based on preference labels and model forecasts. |
| Outcome: | The proposed method improves the performance of large language models on state-of-the-art alignment tasks. |