Papers by Tianhao Cheng
Adversarial Preference Optimization: Enhancing Your Alignment via RM-LLM Game (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for training large language models require additional annotations to adjust to shifted distributions. |
| Approach: | They propose an algorithm that allows LLMs and reward models to update alternatively via a min-max game to improve their alignment. |
| Outcome: | The proposed framework improves existing alignment baselines in terms of LLM helpfulness and harmlessness. |
On Diversified Preferences of Large Language Model Alignment (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) can be fine tuned with human feedback, but human preferences can be diversified due to annotators’ different tastes, which hinders the effectiveness of LLM alignment methods. |
| Approach: | They propose a calibration error metric to evaluate large language models (LLMs) and a multi-objective reward learning method to enhance the calibration performance of RMs on shared preferences. |
| Outcome: | The proposed model can be adopted as a key calibration error and MORE can achieve superior alignment performance. |
From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge (2025.emnlp-main)
Copied to clipboard
Dawei Li, Bohan Jiang, Liangjie Huang, Alimohammad Beigi, Chengshuai Zhao, Zhen Tan, Amrita Bhattacharjee, Yuxuan Jiang, Canyu Chen, Tianhao Wu, Kai Shu, Lu Cheng, Huan Liu
| Challenge: | Recent advances in Large Language Models (LLMs) inspire the "LLM-as-a-judge" paradigm . traditional methods of assessment and evaluation fail in dynamic and open-ended scenarios . |
| Approach: | They propose a paradigm where LLMs are leveraged to perform scoring, ranking, or selection for machine learning evaluation scenarios. |
| Outcome: | The proposed model-based judgment and evaluation paradigms are based on large language models and are compared to the current model-driven evaluation paradigm. |
Machine Unlearning of Pre-trained Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Using curated datasets, we establish a robust benchmark for unlearning performance, demonstrating that these methods are over 105 times more computationally efficient than retraining. |
| Approach: | They propose a framework for machine unlearning in pre-trained LLMs and integrate gradient ascent with gradient descent on in-distribution data to achieve robustness. |
| Outcome: | The proposed framework is over 105 times more efficient than retraining on in-distribution data and provides detailed guidelines for efficient hyperparameter tuning in the unlearning process. |
OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models (2025.acl-long)
Copied to clipboard
Siming Huang, Tianhao Cheng, Jason Klein Liu, Weidi Xu, Jiaran Hao, Liuyihan Song, Yang Xu, Jian Yang, Jiaheng Liu, Chenchen Zhang, Linzheng Chai, Ruifeng Yuan, Xianzhen Luo, Qiufeng Wang, YuanTao Fan, Qingfu Zhu, Zhaoxiang Zhang, Yang Gao, Jie Fu, Qian Liu, Houyi Li, Ge Zhang, Yuan Qi, Xu Yinghui, Wei Chu, Zili Wang
| Challenge: | Code LLMs lack reproducible data pipelines and training protocols for reproducible advancements in code intelligence. |
| Approach: | They propose a top-tier code LLM that releases model weights and inference code . reproducible data pipelines, rigorous experimental ablation results and training protocols are included . |
| Outcome: | The proposed model achieves comparable performance to leading models and serves as an "open cookbook" reproducible training data, rigorous experimental ablation results, and detailed training protocols are also included in the model. |