Papers by David Leslie
FIESTA: Fast IdEntification of State-of-The-Art models using adaptive bandit algorithms (P19-1)
Copied to clipboard
| Challenge: | evaluating models is a non-trivial task and requires extensive data and data splits to produce reliable comparisons. |
| Approach: | They propose a model selection approach that reduces the computational resources required to compare models based on single choices of random seeds. |
| Outcome: | The proposed model selection approach reduces the computational resources required to identify state-of-the-art models from large datasets. |
Using J-K-fold Cross Validation To Reduce Variance When Tuning NLP Models (C18-1)
Copied to clipboard
| Challenge: | a recent study shows that performance estimations are unstable and variable . this makes it difficult to use parameter tuning and model selection . |
| Approach: | They propose to use a less variable CV method to evaluate performance . they propose lower choices of K than are typically seen in the NLP literature . |
| Outcome: | The proposed method can be used for parameter tuning and performance estimation, but it is unstable and unstable. |