Papers by Alekh Agarwal
Conditional Language Policy: A General Framework For Steerable Multi-Objective Finetuning (2024.findings-emnlp)
Copied to clipboard
Kaiwen Wang, Rahul Kidambi, Ryan Sullivan, Alekh Agarwal, Christoph Dann, Andrea Michi, Marco Gelmi, Yunxuan Li, Raghav Gupta, Kumar Dubey, Alexandre Rame, Johan Ferret, Geoffrey Cideron, Le Hou, Hongkun Yu, Amr Ahmed, Aranyak Mehta, Leonard Hussenot, Olivier Bachem, Edouard Leurent
| Challenge: | Existing approaches for multi-objective Reinforcement Learning (RL) are difficult due to plurality of preferences and applications. |
| Approach: | They propose a framework for finetuning language models on multiple objectives using conditional language policy. |
| Outcome: | The proposed framework outperforms and Pareto-dominates existing approaches for multi-objective Reinforcement Learning (RL) it does not require training or maintaining multiple models to achieve different trade-offs between the objectives. |
Efficient End-to-End Visual Document Understanding with Rationale Distillation (2024.naacl-long)
Copied to clipboard
| Challenge: | Pre-processing tools such as optical character recognition (OCR) can map document image inputs to textual tokens, then large language models (LLMs) can reason over text. |
| Approach: | They propose a method that integrates outputs of OCR tools and larger multimodal models as intermediate "rationales" a student model is trained to predict rationales and answers based on visual documents . |
| Outcome: | The proposed model outperforms the base model on three visual document understanding benchmarks with only 1% higher computational cost. |
Optimizing Pre-Training Data Mixtures with Mixtures of Data Expert Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods to optimize language model pre-training data mixtures are difficult due to the complexity of the data mixture. |
| Approach: | They propose a method to optimize language model pre-training data mixtures by approximating cross-entropy loss via a Mixture of Data Experts (MDE). |
| Outcome: | The proposed method improves performance on a slimPajama dataset with a mixture of data experts. |