Papers by Joaquin Vanschoren
An Empirical Study of Speculative Decoding for Small Language Models (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing studies focus on 7B-70B parameters models, leaving a knowledge gap for small language models. |
| Approach: | They propose a draft-then-verify paradigm that allows for a single forward pass through a model and transfer of all model parameters to the GPU cache. |
| Outcome: | The proposed method can be used to accelerate small language models with low computational overhead. |
Automatic Combination of Sample Selection Strategies for Few-Shot Learning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on small language models are characterised by a labelled data scarcity due to data collection/annotation costs or privacy considerations, making the training of typical deep learning models unfeasible. |
| Approach: | They propose a method for Automatic Combination of SamplE Selection Strategies to leverage the strengths and complementarity of various well-established selection objectives. |
| Outcome: | The proposed method outperforms all in-context learning strategies and performs on par or exceeds the in-constinction learning specific baselines. |