Papers by Tianyi Bai
Meta-rater: A Multi-dimensional Data Selection Method for Pre-training Language Models (2025.acl-long)
Copied to clipboard
Xinlin Zhuang, Jiahui Peng, Ren Ma, Yinfan Wang, Tianyi Bai, Xingjian Wei, Qiu Jiantao, Chi Zhang, Ying Qian, Conghui He
| Challenge: | composition of pre-training datasets for large language models remains undisclosed . current methods for evaluating data quality are limited by single-dimensional evaluation or redundancy-focused strategies. |
| Approach: | They propose a multi-dimensional data selection method that integrates dimensions with existing quality metrics through learned optimal weightings. |
| Outcome: | The proposed method doubles convergence speed for 1.3B model models and improves downstream task performance by 3.23%. |
Efficient Pretraining Data Selection for Language Models via Multi-Actor Collaboration (2025.acl-long)
Copied to clipboard
Tianyi Bai, Ling Yang, Zhen Hao Wong, Fupeng Sun, Xinlin Zhuang, Jiahui Peng, Chi Zhang, Lijun Wu, Qiu Jiantao, Wentao Zhang, Binhang Yuan, Conghui He
| Challenge: | Efficient data selection is crucial to accelerate the pretraining of language models . limited research has addressed the inherent conflicts between data selection methods . |
| Approach: | They propose a multi-actor collaborative data selection mechanism that prioritizes data based on its specific criterion and updates prioritization rules using the current state of the model. |
| Outcome: | The proposed model accelerates convergence in LM pretraining and achieves an average relative performance gain of 10.5% across multiple language model benchmarks. |
From Completion to Editing: Unlocking Context-Aware Code Infilling via Search-and-Replace Instruction Tuning (2026.acl-long)
Copied to clipboard
Jiajun Zhang, Zeyu Cui, Jiaxi Yang, Lei Zhang, Yuheng Jing, Zeyao Ma, Tianyi Bai, Zilei Wang, Qiang Liu, Liang Wang, Binyuan Hui, Junyang Lin
| Challenge: | Fill-in-the-Middle (FIM) models suffer from performance degradation and prohibitive latency. |
| Approach: | They propose a search-and-replace infilling framework that integrates agentic verification and editing into a single-pass inference process. |
| Outcome: | The proposed framework harmonizes completion tasks with the instruction-following priors of Chat LLMs, extending the paradigm from static infilling to dynamic context-aware editing. |