Papers by Romain Storaï
HARP: Hesitation-Aware Reframing in Transformer Inference Pass (2025.naacl-long)
Copied to clipboard
| Challenge: | a recent study has shown that inference steps are not equally challenging, with some being "harder" and others "easier." |
| Approach: | They propose a modified Transformer forward pass that selectively applies additional computation when the model encounters uncertainty during token generation. |
| Outcome: | The proposed method achieves performance gains while maintaining inference times twice faster than beam search. |
Intended Target Identification for Anomia Patients with Gradient-based Selective Augmentation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Identifying the intended target item from patient’s circumlocution involves the two challenges of term failure and error. |
| Approach: | They propose to robustify the model from unseen and SPE terms and enhance it with unseense terms by using gradient-based selective augmentation (GradSelect). |
| Outcome: | The proposed model outperforms existing models on the Tip of the Tongue dataset and shows that it can handle anomia patients by addressing the outlined challenges. |
Smarter, Not Harder: Training-Free Adaptive Computation for Transformers (2025.findings-acl)
Copied to clipboard
| Challenge: | Adaptive Computation in Transformers (ACT) aims to reduce computation for simpler inferences while enhancing performance by allocating more computation to complex inference steps. |
| Approach: | They propose a method that perturbs network weights rather than input embeddings to improve performance. |
| Outcome: | The proposed method outperforms beam search and hesitation-based methods but suffers from inefficiency and instability due to its reliance on randomness. |