RedOne 2.0: Rethinking Domain-specific LLM Post-Training in Social Networking Services (2026.acl-industry)
Copied to clipboard
Fei zhao, Chonggang Lu, Haofu Qian, Fangcheng Shi, Zijie Meng, Jianzhao Huang, Zheyong Xie, Shaosheng Cao
| Challenge: | Social networking services (SNS) are critical infrastructure for global interaction . supervised fine-tuning (SFT) can improve in-domain performance, but it often induces a ”seesaw” trade-off with out-of-domain robustness . |
| Approach: | They propose an SNS-oriented LLM with a progressive, RL-prioritized post-training paradigm for fast and stable adaptation. |
| Outcome: | The proposed model improves over the previous 7B model by 2.41 on average . it also yields an 8.74 average gain over its Qwen3-4B base . |
Similar Papers
RedOne: Revealing Domain-specific LLM Post-Training in Social Networking Services (2025.emnlp-industry)
Copied to clipboard
Fei Zhao, Chonggang Lu, null Wangyue, Zheyong Xie, Ziyan Liu, Haofu Qian, Jianzhao Huang, Fangcheng Shi, Zijie Meng, Hongcheng Guo, Mingqian He, Xinze Lyu, Zheyu Ye, Weiting Liu, Boyang Wang, Shaosheng Cao
| Challenge: | Social networking services (SNS) have experienced rapid growth, which has proposed significant challenges for platform content management and interaction quality improvement. |
| Approach: | They propose a domain-specific LLM to break the performance bottleneck of single-task baselines and establish a comprehensive foundation for social networking services. |
| Outcome: | The proposed model achieves an average improvement of 14.02% across 8 major tasks and 7.56% in bilingual evaluation benchmark, compared with baseline models. |
Selective Self-to-Supervised Fine-Tuning for Generalization in Large Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) can be fine-tuned on task-specific data to improve performance on target tasks but can be overfitted resulting in a loss of generalization. |
| Approach: | They propose a method that uses the correct model responses from a training set to fine-tune the model using the correct response and the gold response for the remaining samples. |
| Outcome: | The proposed approach reduces model specialization during the fine-tuning stage while improving generalization. |
A Survey of Post-Training Scaling in Large Language Models (2025.acl-long)
Copied to clipboard
Hanyu Lai, Xiao Liu, Junjie Gao, Jiale Cheng, Zehan Qi, Yifan Xu, Shuntian Yao, Dan Zhang, Jinhua Du, Zhenyu Hou, Xin Lv, Minlie Huang, Yuxiao Dong, Jie Tang
| Challenge: | Large language models (LLMs) have demonstrated proficiency in understanding and generating human natural languages. |
| Approach: | They propose a framework for scaling large language models using supervised fine-tuning, RLxF and test-time compute methodologies. |
| Outcome: | The proposed model can be used to understand and generate human natural languages. |
LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancement (2024.findings-acl)
Copied to clipboard
Nicholas Lee, Thanakul Wattanawong, Sehoon Kim, Karttikeya Mangalam, Sheng Shen, Gopala Anumanchipalli, Michael Mahoney, Kurt Keutzer, Amir Gholami
| Challenge: | Pretrained large language models are currently state-of-the-art for solving most tasks . however, many of them are in the low-data regime, making fine-tuning challenging . a new data augmentation strategy uses a teacher LLM to augment a small seed dataset . |
| Approach: | They propose a targeted and iterative data augmentation strategy that augments a teacher LLM to fine-tune a small seed dataset by adding additional data. |
| Outcome: | The proposed approach outperforms fine-tuning and other data augmentation strategies on a small seed dataset. |
Scaling Down, Serving Fast: Compressing and Deploying Efficient LLMs for Recommendation Systems (2025.emnlp-industry)
Copied to clipboard
Kayhan Behdin, Ata Fatahibaarzi, Qingquan Song, Yun Dai, Aman Gupta, Zhipeng Wang, Hejian Sang, Shao Tang, Gregory Dexter, Sirou Zhu, Siyu Zhu, Tejas Dharamsi, Vignesh Kothapalli, Zhoutong Fu, Yihan Cao, Pin-Lun Hsu, Fedor Borisyuk, Natesh S. Pillai, Luke Simon, Rahul Mazumder
| Challenge: | Large language models (LLMs) have demonstrated remarkable performance across a wide range of industrial applications. |
| Approach: | They propose two techniques for training and deploying small language models that deliver high performance for a variety of industry use cases. |
| Outcome: | The proposed techniques retain much of the quality of larger models while reducing training/serving costs and latency. |
Demystifying Domain-adaptive Post-training for Financial LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Domain-adaptive post-training of large language models (LLMs) has emerged as promising approach for specialized domains such as medicine and finance. |
| Approach: | They propose a system to identify optimal adaptation criteria and training strategies for LLMs for the finance domain. |
| Outcome: | The proposed model achieves state-of-the-art performance across a wide range of financial tasks. |
ReTRE: Benchmarking LLM Transfer Robustness with Structure-Preserving Variants (2026.acl-long)
Copied to clipboard
| Challenge: | Learning transfer theory emphasizes that applying acquired knowledge to novel manifestations is a key signal of deep understanding |
| Approach: | They propose a benchmark that probes transfer robustness along two rewrite levels: Near Transfer and Far Transfer. |
| Outcome: | The proposed benchmark demonstrates that large language models are robust when faced with novel manifestations of the same problem. |
Middo: Model-Informed Dynamic Data Optimization for Enhanced LLM Fine-Tuning via Closed-Loop Learning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to improve data quality face limitations in static dataset curation that fail to adapt to evolving model capabilities. |
| Approach: | They propose a self-evolving framework that uses model-aware data selection and context-preserving data refinement to improve LLM performance. |
| Outcome: | The proposed framework improves the quality of seed data and boosts LLM’s performance with improving accuracy by 7.15% on average while maintaining the original dataset scale. |
BalanceSFT: Improving LLM Function Calling with Balanced Training Signals and Data Hardness (2026.findings-acl)
Copied to clipboard
Bingguang Hao, Zengzhuang Xu, Maolin Wang, Yuntao Wen, Yicheng Chen, Cunyin Peng, Long Chen, Xiangyu Zhao, Jinjie Gu, Chenyi Zhuang, Ji Zhang
| Challenge: | Currently, Supervised Fine-Tuning (SFT) is the prevailing method for equipping Large Language Models (LLMs) with function calling capabilities, but its effectiveness is often compromised by two challenges: 1) lengthy Chain-of-Thought (CoT) reasoning tokens dominate training signals over concise function calls in the learning objective; 2) scarcity of hard training examples. |
| Approach: | They propose a framework that uses a self-adjusted signal balancing loss and a hard data re-sampling strategy to selectively generate new, high-quality complex data guided by model errors. |
| Outcome: | The proposed framework surpasses state-of-the-art models like GPT-5 in function calling performance. |
Unlocking LLMs’ Self-Improvement Capacity with Autonomous Learning for Domain Adaptation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing models that use self-supervised and instruction fine-tuning can be trained using unlabeled corpora. |
| Approach: | They propose to use unlabeled target corpora to adapt large language models to new domains . they propose to employ self-supervised pre-training and instruction fine-tuning methods . |
| Outcome: | The proposed model can adapt to new domains using only a large amount of unlabeled target corpora. |