Papers by Zhengze Zhang
Reward Difference Optimization For Sample Reweighting In Offline RLHF (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are becoming more capable, but their maximum likelihood objective for the next token prediction falls short in capturing such crucial human values. |
| Approach: | They propose a reward difference prediction method that uses reward difference coefficients to reweigh sample pairs in offline RLHF and a difference model that considers rich interactions between a pair of responses. |
| Outcome: | The proposed method is effective in both automatic metrics and human evaluation. |
daDPO: Distribution-Aware DPO for Distilling Conversational Abilities (2025.findings-acl)
Copied to clipboard
| Challenge: | Knowledge distillation (KD) with Direct Preference Optimization (DPO) has emerged as a promising approach to enhance the conversational abilities of smaller models using a larger teacher model. |
| Approach: | They propose a framework that integrates the teacher's distributional information into DPO distillation while preserving theoretical guarantees. |
| Outcome: | The proposed framework outperforms existing methods in restoring performance for pruned models and enhancing smaller models within the same LLM family. |
LAVa: Layer-wise KV Cache Eviction with Dynamic Budget Allocation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for cache compression are heuristic and lack dynamic budget allocation . cnn's john mccartney and johnny mccain present a new approach for cache eviction and dynamic budgets . |
| Approach: | They propose a unified framework for cache compression that minimizes information loss in transformer residual streams. |
| Outcome: | The proposed method consistently maintains top performance across task types. |