Papers by Oleksii Kuchaiev
HelpSteer3: Human-Annotated Feedback and Edit Data to Empower Inference-Time Scaling in Open-Ended General-Domain Tasks (2025.acl-long)
Copied to clipboard
Zhilin Wang, Jiaqi Zeng, Olivier Delalleau, Daniel Egert, Ellie Evans, Hoo-Chang Shin, Felipe Soares, Yi Dong, Oleksii Kuchaiev
| Challenge: | Inference-Time Scaling is critical to the success of recent models such as OpenAI o1 and DeepSeek R1 . however, many techniques require tasks to have answers that can be verified . |
| Approach: | They use data to train dedicated Feedback and Edit Models capable of inference-time scaling for open-ended tasks. |
| Outcome: | The proposed model can reach SoTA performance on Arena Hard at 92.7 as of 5 Mar 2025. |
Tied-LoRA: Enhancing parameter efficiency of LoRA with Weight Tying (2024.naacl-long)
Copied to clipboard
| Challenge: | a new paradigm for low-rank Adaptation (LoRA) uses weight tying and selective training to improve parameter efficiency. |
| Approach: | They propose a paradigm that uses weight tying and selective training to enhance parameter efficiency of Low-rank Adaptation. |
| Outcome: | The proposed paradigm achieves comparable performance to LoRA with reduced model complexity . the proposed paradigm can be used for a variety of tasks and languages . |
Leveraging Synthetic Targets for Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Using synthetic target data, training models on synthetic targets outperforms training on actual ground-truth data. |
| Approach: | They propose a recipe for training machine translation models on synthetic target data by leveraging a large pre-trained model. |
| Outcome: | The proposed model outperforms training on real-world translation datasets. |
HelpSteer: Multi-attribute Helpfulness Dataset for SteerLM (2024.naacl-long)
Copied to clipboard
Zhilin Wang, Yi Dong, Jiaqi Zeng, Virginia Adams, Makesh Narsimhan Sreedhar, Daniel Egert, Olivier Delalleau, Jane Scowcroft, Neel Kant, Aidan Swope, Oleksii Kuchaiev
| Challenge: | Existing helpfulness preference datasets do not specify what makes some responses more helpful and others less helpful. |
| Approach: | They use a dataset that has annotated for correctness, coherence, complexity, and verbosity. |
| Outcome: | The dataset has annotations for correctness, coherence, complexity, and verbosity in addition to overall helpfulness of responses. |
Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study (2023.emnlp-main)
Copied to clipboard
Boxin Wang, Wei Ping, Peng Xu, Lawrence McAfee, Zihan Liu, Mohammad Shoeybi, Yi Dong, Oleksii Kuchaiev, Bo Li, Chaowei Xiao, Anima Anandkumar, Bryan Catanzaro
| Challenge: | a recent study shows that retrieval-augmented LMs can improve text generation quality and accuracy. |
| Approach: | They propose a model that reproduces RETRO parameters while retrieving a text corpus . they find RETRO outperforms GPT on text generation with less repetition . |
| Outcome: | The proposed model outperforms standard retrieval-augmented GPT and retrieval augmented GTP on text generation and accuracy tasks. |
GPT vs RETRO: Exploring the Intersection of Retrieval and Parameter-Efficient Fine-Tuning (2024.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained large language models can be used for specific tasks and unique information but lack the resources for extensive retraining. |
| Approach: | They propose to use PEFT methods to adapt large language models while minimizing compute requirements. |
| Outcome: | The proposed methods outperform GPT models in zero-shot settings but lag behind PEFT. |
SteerLM: Attribute Conditioned SFT as an (User-Steerable) Alternative to RLHF (2023.findings-emnlp)
Copied to clipboard
| Challenge: | supervised fine-tuning and reinforcement learning from human feedback (RLHF) are not effective in generating useful and high-quality responses. |
| Approach: | They propose a supervised fine-tuning method that empowers end-users to control responses during inference. |
| Outcome: | Experiments show that supervised fine-tuning and reinforcement learning from human feedback (RLHF) can generate helpful and high-quality responses while maintaining customizability. |