Papers by Katharina Wense
Quantifying the Hyperparameter Sensitivity of Neural Networks for Character-level Sequence-to-Sequence Tasks (2024.eacl-long)
Copied to clipboard
| Challenge: | Neural networks are notoriously hard to interpret and slightly mysterious to researchers and practitioners alike. |
| Approach: | They formalize hyperparameter sensitivity using two metrics: similarity-based sensitivity and performance-based-sensitivity. |
| Outcome: | The transformer is more sensitive to hyperparameters according to both metrics, but not batch size . large models, multilinguality of NLP models and tasks make hyperparametric tuning more expensive . |
Comparing Template-based and Template-free Language Model Probing (2024.eacl-long)
Copied to clipboard
| Challenge: | Template-based and template-based approaches rank models differently except for the top domain-specific models. |
| Approach: | They evaluate 16 different cloze-task language model probing approaches on 10 probing English datasets to answer questions about model rankings and absolute scores. |
| Outcome: | The results show that the template-based and template-free approaches rank models differently except for the top domain-specific models. |
It Is Not About What You Say, It Is About How You Say It: A Surprisingly Simple Approach for Improving Reading Comprehension (2024.findings-acl)
Copied to clipboard
| Challenge: | Experimenting with 9 large language models across 3 datasets, emphasizing the context yields superior results compared to question emphasis. |
| Approach: | They ask: How does the order of inputs affect model performance? |
| Outcome: | Experiments with 9 large language models show that emphasizing the question and context improves model performance. |
Zero-Shot vs. Translation-Based Cross-Lingual Transfer: The Case of Lexical Gaps (2024.naacl-short)
Copied to clipboard
| Challenge: | lexical gaps exist in a variety of domains, such as QA, but they can only be expressed as a combination of words in another language. |
| Approach: | They compare the current performance and long-term viability of two approaches to cross-lingual transfer . they leverage lexical gaps to create a multilingual question answering dataset . |
| Outcome: | The proposed model outperforms zero-shot transfer and machine translation (MT) lexical gaps exist in a variety of domains, including linguistics, linguistic coding, and linguistic analysis. |
Desiderata For The Context Use Of Question Answering Systems (2024.eacl-long)
Copied to clipboard
| Challenge: | Prior work has uncovered a set of common problems in state-of-the-art context-based question answering systems, such as a lack of attention to the context when it conflicts with a model’s parametric knowledge and a loss of consistency with their answers. |
| Approach: | They propose to examine the desiderata for context-based question answering systems and then compare them to a set of prior work. |
| Outcome: | The proposed models are based on 15 datasets and evaluated on 5 datasets. |
TAMS: Translation-Assisted Morphological Segmentation (2024.acl-long)
Copied to clipboard
| Challenge: | Canonical morphological segmentation is a key task in endangered language documentation . training data for canonical segmentation can be difficult, making it difficult to train high quality models. |
| Approach: | They propose a model that leverages translation data to speed up canonical segmentation . they propose to use translation data as an additional signal to leverage the data . |
| Outcome: | The proposed model outperforms baseline models in a super-low resource setting but yields mixed results on training splits with more data. |
Getting The Most Out of Your Training Data: Exploring Unsupervised Tasks for Morphological Inflection (2024.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained transformers have been shown to be effective in many natural language tasks, but are under-explored for character-level sequence to sequence tasks. |
| Approach: | They propose to use pre-trained transformers for character-level morphological inflection in several languages to train models for unsupervised tasks. |
| Outcome: | The proposed model outperforms the best two shared tasks on morphological inflection and graphemeto-phoneme conversion benchmarks. |