Papers by Francois Fleuret
HyperMixer: An MLP-based Low Cost Alternative to Transformers (2023.acl-long)
Copied to clipboard
Florian Mai, Arnaud Pannatier, Fabio Fehr, Haolin Chen, Francois Marelli, Francois Fleuret, James Henderson
| Challenge: | Existing MLP-based architectures that combine multiple features are expensive and require a lot of training data. |
| Approach: | They propose a simple MLP-based model which allows token mixing by dynamically applying hypernetworks to each feature independently. |
| Outcome: | The proposed model performs better than Transformers and lowers costs in terms of processing time, training data, and hyperparameter tuning. |
Language Models are Few-Shot Butlers (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained language models demonstrate strong performance in most NLP tasks when fine-tuned on small task-specific datasets. |
| Approach: | They propose a two-stage procedure to learn from a small set of demonstrations and a simple reinforcement learning algorithm to improve by interacting with an environment. |
| Outcome: | The proposed method improves with only 1.2% of the demonstrations and a simple reinforcement learning algorithm over existing methods in the ALFWorld environment. |