Papers by Xiaonan He
Leveraging Web-Crawled Data for High-Quality Fine-Tuning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Currently, large language models are fine-tuned using expensive human-annotated data or GPT-4 generated data. |
| Approach: | They propose to use web-crawled data to train a language model on a smaller set of data . their results show that the model can convert web data with irregular formats into high-quality ones . |
| Outcome: | The proposed model outperforms open-source models larger than 32B and outperformed open-sourced models such as GPT-3.5. |