Papers by Seonghoon Yang
LP Data Pipeline: Lightweight, Purpose-driven Data Pipeline for Large Language Models (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Creating high-quality datasets for large language models often relies on resource-intensive, GPU-accelerated models for quality filtering, making the process time-consuming and costly. |
| Approach: | They propose a framework that operates entirely on CPUs to streamline the processes of dataset extraction, filtering, and curation. |
| Outcome: | The proposed framework reduces preparation time and costs while maintaining high data quality while enhancing the applicability of LLMs in specialized contexts. |
Rethinking KenLM: Good and Bad Model Ensembles for Efficient Text Quality Filtering in Large Web Corpora (2025.acl-short)
Copied to clipboard
| Challenge: | Existing methods to efficiently filter large web corpora require GPU resources. |
| Approach: | They propose an ensemble approach that leverages two contrasting KenLMs to filter large web corpora. |
| Outcome: | The proposed method significantly reduces noisy content while preserving high-quality content compared to the traditional KenLM training method. |