Papers by Lena Jurkschat
Tokenizer Choice For LLM Training: Negligible or Crucial? (2024.findings-naacl)
Copied to clipboard
Mehdi Ali, Michael Fromm, Klaudia Thellmann, Richard Rutmann, Max Lübbering, Johannes Leveling, Katrin Klug, Jan Ebert, Niclas Doll, Jasper Buschhoff, Charvi Jain, Alexander Weber, Lena Jurkschat, Hammam Abdelwahab, Chelsea John, Pedro Ortiz Suarez, Malte Ostendorff, Samuel Weinbach, Rafet Sifa, Stefan Kesselheim, Nicolas Flores-Herr
| Challenge: | Recent success of large language models has been driven by curating the training dataset composition, scaling of model architectures and advancements in pretraining objectives, leaving tokenizer influence as a blind spot. |
| Approach: | They conduct a comprehensive study on the influence of tokenizer choice on LLM downstream performance by training 24 mono- and multilingual LLMs at a 2.6B parameter scale. |
| Outcome: | The proposed model can significantly impact the model's downstream performance and training costs. |
Few-Shot Learning for Argument Aspects of the Nuclear Energy Debate (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing methods to classify aspects of arguments are expensive and require training data for further aspects and topics. |
| Approach: | They propose a supervised aspect-based argument mining task to classify arguments into semantically coherent groups referring to the same defined aspect categories. |
| Outcome: | The proposed method is able to predict share of arguments in a British newspaper corpus with 50 to 100 examples per aspect. |