Papers by Katharina Kann
What Would a Teacher Do? Predicting Future Talk Moves (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in natural language processing (NLP) have the ability to transform how classroom learning takes place. |
| Approach: | They propose a task that uses the academically productive talk framework to learn strategies that make for the best learning experience. |
| Outcome: | The proposed task outperforms baselines on academically productive talk (FTMP) and shows that it outperformed human performance on FTMP. |
Mind the Gap between the Application Track and the Real World (2023.acl-short)
Copied to clipboard
| Challenge: | Recent advances in NLP have led to a rise in inter-disciplinary and application-oriented research. |
| Approach: | They examine the relationship between motivations described in NLP papers and models and evaluations which comprise the proposed solution. |
| Outcome: | The proposed solution improves educational dialog understanding system when used in a realistic classroom environment. |
Transductive Auxiliary Task Self-Training for Neural Multi-Task Models (D19-61)
Copied to clipboard
| Challenge: | Multi-task learning and self-training are two common ways to improve a machine learning model’s performance in settings with limited training data. |
| Approach: | They propose a transductive auxiliary task self-training procedure that trains a model on auxiliary tasks and test instances with auxiliary labels generated by a single-task version of the model. |
| Outcome: | The proposed method improves accuracy by 9.56% over the pure multi-task model for dependency relation tagging and 13.03% for semantic taging. |
CLiMP: A Benchmark for Chinese Language Model Evaluation (2021.eacl-main)
Copied to clipboard
| Challenge: | Linguistically informed analyses of language models (LMs) contribute to understanding and improvement of such models. |
| Approach: | They introduce a corpus of Chinese linguistic minimal pairs (CLiMP) to investigate what knowledge Chinese LMs acquire. |
| Outcome: | The proposed corpus of Chinese linguistic minimal pairs (CLiMP) covers 9 major Chinese linguist phenomena. |
Subword-Level Language Identification for Intra-Word Code-Switching (N19-1)
Copied to clipboard
| Challenge: | Code-switching (CS) is a phenomenon of alternating between two or more languages in conversations . if at least one language is morphologically rich, a large number of words can be composed of morphemes from more than one language. |
| Approach: | They propose to extend the language identification task to the subword level by splitting mixed words while tagging each part with a language ID. |
| Outcome: | The proposed model outperforms the baseline on a Spanish–Wixarika and adapted German–Turkish datasets. |
AmericasNLI: Evaluating Zero-shot Natural Language Understanding of Pretrained Multilingual Models in Truly Low-resource Languages (2022.acl-long)
Copied to clipboard
Abteen Ebrahimi, Manuel Mager, Arturo Oncevay, Vishrav Chaudhary, Luis Chiruzzo, Angela Fan, John Ortega, Ricardo Ramos, Annette Rios, Ivan Vladimir Meza Ruiz, Gustavo Giménez-Lugo, Elisabeth Mager, Graham Neubig, Alexis Palmer, Rolando Coto-Solano, Thang Vu, Katharina Kann
| Challenge: | Pretrained multilingual models can perform cross-lingual transfer in a zero-shot setting, even for unseen languages. |
| Approach: | They propose to extend XNLI to 10 indigenous languages of the Americas and test multiple zero-shot and translation-based approaches. |
| Outcome: | The proposed model can perform cross-lingual transfer in a zero-shot setting even for languages unseen during pretraining. |
Intermediate-Task Transfer Learning with Pretrained Language Models: When and Why Does It Work? (2020.acl-main)
Copied to clipboard
Yada Pruksachatkun, Jason Phang, Haokun Liu, Phu Mon Htut, Xiaoyi Zhang, Richard Yuanzhe Pang, Clara Vania, Katharina Kann, Samuel R. Bowman
| Challenge: | Unsupervised pretraining has recently pushed the state of the art on many natural language understanding tasks. |
| Approach: | They perform a large-scale survey on a pretrained RoBERTa model with 110 intermediate-target task combinations and 25 probing tasks to reveal the specific skills that drive transfer. |
| Outcome: | The proposed model is trained on 110 intermediate-target task combinations and compared with 25 probing tasks to reveal the specific skills that drive transfer. |
Ethical Considerations for Machine Translation of Indigenous Languages: Giving a Voice to the Speakers (2023.acl-long)
Copied to clipboard
| Challenge: | In recent years, machine translation has become very successful for high-resource language pairs. |
| Approach: | They conduct interviews with community leaders, teachers, and language activists to shed light on ethical considerations for the automatic translation of Indigenous languages. |
| Outcome: | The results show that the inclusion of native speakers and community members is vital to performing better and more ethical research on Indigenous languages. |
Tackling the Low-resource Challenge for Canonical Segmentation (2020.emnlp-main)
Copied to clipboard
| Challenge: | morphological segmentation is a task of dividing words into their constituting morphemes . we compare two new approaches for the task when training data is limited . |
| Approach: | They propose to use an LSTM pointer-generator and a sequence-to-sequence model to perform canonical segmentation when training data is limited. |
| Outcome: | The proposed models outperform existing models on German, English, and Indonesian in low-resource scenarios by 11.4% accuracy. |
Neural Unsupervised Parsing Beyond English (D19-61)
Copied to clipboard
| Challenge: | Unsupervised parsing is a task that can be learned without substantial prior knowledge. |
| Approach: | They train an unsupervised model for Arabic, Chinese, English, and German to learn syntactic structure from unlabeled text. |
| Outcome: | The PRPN architecture outperforms trivial baselines and acquires at least some parsing ability for all languages. |
CHIA: CHoosing Instances to Annotate for Machine Translation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Neural machine translation systems perform poorly on low-resource language pairs, for which large-scale parallel data is unavailable. |
| Approach: | They propose a method for selecting instances to annotate for machine translation using existing multi-way parallel datasets. |
| Outcome: | The proposed method outperforms unsupervised methods on 20 languages and a multi-way parallel dataset on high-resource languages. |
Probing for Semantic Classes: Diagnosing the Meaning Content of Word Embeddings (P19-1)
Copied to clipboard
| Challenge: | Empirical analysis of word embeddings of ambiguous words is limited by the small size of manually annotated resources and by the fact that word senses are treated as unrelated individual concepts. |
| Approach: | They present a large dataset based on manual Wikipedia annotations and word senses, where word sense from different words are related by semantic classes. |
| Outcome: | The proposed method can predict whether a word is single-sense or multi-sensor, if the sense is frequent, and it can predict rare senses. |
Towards Realistic Practices In Low-Resource Natural Language Processing: The Development Set (D19-1)
Copied to clipboard
| Challenge: | Using development sets for low-resource training is often more effective . however, some studies show that early stopping can overestimate performance . |
| Approach: | They find that early stopping on a development set is more effective than using all available data for training. |
| Outcome: | The proposed model overestimates accuracy over languages and tasks by 1.4% compared to a more realistic set of training epochs. |
An Investigation of Noise in Morphological Inflection (2023.findings-acl)
Copied to clipboard
| Challenge: | Neural morphological inflection systems can be used for languages with very little supervised data, but are often less likely to have clean, goldstandard data. |
| Approach: | They propose an error taxonomy and annotation pipeline for inflection training data and propose a character-level masked language modeling (CMLM) pretraining objective. |
| Outcome: | The proposed pipeline is based on error taxonomy and annotation pipelines for unsupervised morphological paradigm completion. |
A Comprehensive Comparison of Neural Networks as Cognitive Models of Inflection (2022.emnlp-main)
Copied to clipboard
| Challenge: | Neural networks are at the center of a debate about human behavior in inflectional morphology. |
| Approach: | They measure correlation between human judgments and neural network probabilities for unknown word inflections. |
| Outcome: | The proposed model for morphological inflections correlates best with human wug ratings, but not with humans. |
Fortification of Neural Morphological Segmentation Models for Polysynthetic Minimal-Resource Languages (N18-1)
Copied to clipboard
| Challenge: | Morphological segmentation for polysynthetic languages is challenging because of limited training data. |
| Approach: | They propose two new multi-task training approaches that improve performance for Mexican polysynthetic languages . they also propose cross-lingual transfer as a third way to fortify their neural model . |
| Outcome: | The proposed models improve on Mexicanero, Nahuatl, Wixarika and Yorem Nokki . the proposed models reduce the amount of parameters by close to 75% . |
Morphological Processing of Low-Resource Languages: Where We Are and What’s Next (2022.findings-acl)
Copied to clipboard
Adam Wiemerslage, Miikka Silfverberg, Changbing Yang, Arya McCarthy, Garrett Nicolai, Eliana Colunga, Katharina Kann
| Challenge: | Existing models for morphological processing are not suitable for low-resource languages, but they are still lacking in the field of computational morphology. |
| Approach: | They propose to bridge two unsupervised models to understand a language’s morphology from raw text alone and propose to use them to improve their models. |
| Outcome: | The proposed models perform reasonably, but there is room for improvement. |
The World of an Octopus: How Reporting Bias Influences a Language Model’s Perception of Color (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has raised concerns about the inherent limitations of text-only pretraining. |
| Approach: | They first generate a color dataset of human-perceived color distributions for 521 common objects and then use it to analyze and compare the color distribution found in text and the distribution captured by language models. |
| Outcome: | The proposed model improves on the CoDa color distribution, while the language model improve on the ground-truth distribution. |
BPE vs. Morphological Segmentation: A Case Study on Machine Translation of Four Polysynthetic Languages (2022.findings-acl)
Copied to clipboard
| Challenge: | Morphologically rich polysynthetic languages present a challenge for NLP systems due to data sparsity. |
| Approach: | They propose to use subword segmentation to reduce data sparsity in polysynthetic languages . they compare supervised and unsupervised morphological segmentation methods to Byte-Pair Encodings . |
| Outcome: | The proposed methods outperform BPEs in MT tasks for all language pairs except for Nahuatl . the proposed methods are more efficient than supervised methods, but less sparse in fusional languages. |
Coloring the Black Box: What Synesthesia Tells Us about Character Embeddings (2021.eacl-main)
Copied to clipboard
| Challenge: | Neural network models are difficult to understand and are considered "black boxes". |
| Approach: | They use grapheme–color synesthesia to study character embeddings in English . they compare graphemes to phonemes to find the most human-like character embeds . |
| Outcome: | The results show that grapheme-to-phoneme conversion results in the most human-like character embeddings. |
English Intermediate-Task Training Improves Zero-Shot Cross-Lingual Transfer Too (2020.aacl-main)
Copied to clipboard
Jason Phang, Iacer Calixto, Phu Mon Htut, Yada Pruksachatkun, Haokun Liu, Clara Vania, Katharina Kann, Samuel R. Bowman
| Challenge: | a study of intermediate-task training in monolingual English shows that it improves model performance on non-English language understanding tasks. |
| Approach: | They evaluate whether English intermediate-task training is still helpful on non-English target tasks . BUCC and Tatoeba sentence retrieval tasks see large improvements . |
| Outcome: | The proposed model outperforms existing models on non-English language understanding tasks. |
Acrostic Poem Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Acrostic poems contain a hidden message; typically, the first letter of each line spells out a word or short phrase. |
| Approach: | They propose a task for acrostic poem generation in English with multiple constraints . they define the task as a generation task with multiple constraint constraints based on a conditional neural language model and a neural rhyming model . |
| Outcome: | The proposed task is based on a baseline model and a neural rhyming model. |
Unsupervised Morphological Paradigm Completion (2020.acl-main)
Copied to clipboard
| Challenge: | a task of generating morphological paradigms is a challenging unsupervised task for natural language processing systems . acuidados y acciones del idioma es a problem in linguistic annotators. |
| Approach: | They propose a task of unsupervised morphological paradigm completion using raw text and a lemma list. |
| Outcome: | The proposed system outperforms trivial baselines on 14 typologically diverse languages with ease and higher accuracy than minimally supervised systems. |
Making a Point: Pointer-Generator Transformers for Disjoint Vocabularies (2020.aacl-srw)
Copied to clipboard
| Challenge: | Existing neural models rely on an overlap between source and target vocabularies to perform sequence-to-sequence tasks. |
| Approach: | They propose a pointer-generator transformer model for disjoint vocabularies that does not rely on an overlap between source and target vocs. |
| Outcome: | The proposed model outperforms a standard pointer-generator transformer by an average of 5.1 WER over 15 languages. |
Match the Script, Adapt if Multilingual: Analyzing the Effect of Multilingual Pretraining on Cross-lingual Transferability (2022.acl-long)
Copied to clipboard
| Challenge: | Pretrained multilingual models enable zero-shot learning even for unseen languages . current multilingual model covers only a small subset of the world's languages - due to data sparsity, they are not likely to obtain good results for many lowresource languages. |
| Approach: | They ask: how does the number of pretraining languages influence zero-shot learning for unseen languages? do the findings change if the languages used for pretraining are all related? |
| Outcome: | The results show that pretrained models can zero-shot learn for unseen languages even for limited amounts even for low-resource languages. |
A Major Obstacle for NLP Research: Let’s Talk about Time Allocation! (2022.emnlp-main)
Copied to clipboard
| Challenge: | Subpar time allocation has been a major obstacle for natural language processing research in recent years, argues a new position paper . |
| Approach: | They propose to identify the biggest traps the NLP community falls into and suggest solutions to solve them. |
| Outcome: | The authors outline multiple concrete problems together with their negative consequences and suggest remedies to improve the status quo. |
Neural Transductive Learning and Beyond: Morphological Generation in the Minimal-Resource Setting (D18-1)
Copied to clipboard
| Challenge: | Existing lexicons have limited coverage for learning morphological inflection patterns from labeled data. |
| Approach: | They propose two new methods to solve paradigm completion, the morphological task of generating missing forms, given a partial paradigm. |
| Outcome: | The proposed methods outperform the previous state-of-the-art by 9.71% absolute accuracy on a 52-language benchmark dataset. |
Meeting the Needs of Low-Resource Languages: The Value of Automatic Alignments via Pretrained Models (2023.eacl-main)
Copied to clipboard
Abteen Ebrahimi, Arya D. McCarthy, Arturo Oncevay, John E. Ortega, Luis Chiruzzo, Gustavo Giménez-Lugo, Rolando Coto-Solano, Katharina Kann
| Challenge: | Large multilingual models have inspired a new class of word alignment methods, which work well for pretraining languages. |
| Approach: | They propose to use transformer-based word alignment methods to extract alignments from massive pretrained models. |
| Outcome: | The proposed methods outperform traditional methods for languages unseen to pretraining models, and are competitive with each other. |
Don’t Rule Out Monolingual Speakers: A Method For Crowdsourcing Machine Translation Data (2021.acl-short)
Copied to clipboard
| Challenge: | High-performing machine translation systems require large amounts of training data in the form of parallel sentences, and translators are difficult to find and expensive. |
| Approach: | They propose a data collection strategy which uses graphics interchange formats (GIFs) as a pivot to collect parallel sentences from monolingual annotators. |
| Outcome: | The proposed method collects parallel sentences from monolingual annotators in Hindi, Tamil and English. |
IGT2P: From Interlinear Glossed Texts to Paradigms (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing systems for learning morphology have limited their use to languages with publicly available structured data, such as online dictionaries like Wiktionary. |
| Approach: | They propose a task that generates entire morphological paradigms from IGT input and a language expert cleaning noisy IGT data. |
| Outcome: | The proposed task speeds up the process and generates entire morphological paradigm tables from IGT input. |
How to Adapt Your Pretrained Multilingual Model to 1600 Languages (2021.acl-long)
Copied to clipboard
| Challenge: | Pretrained multilingual models perform best for languages seen during pretraining . methods exist to improve performance for unseen languages, but have been evaluated using amounts of raw text only available for a small fraction of the world’s languages. |
| Approach: | They evaluate the performance of existing methods to adapt pretrained multilingual models to new languages using a resource available for close to 1600 languages: the New Testament. |
| Outcome: | The proposed models perform best for languages seen during pretraining . the results show that the most efficient approach is simplest and the most accurate . |
PROST: Physical Reasoning about Objects through Space and Time (2021.findings-acl)
Copied to clipboard
| Challenge: | a new dataset is available to test pretraining of physical reasoning models . state-of-the-art models are inadequate at reasoning about physical interactions, authors say . |
| Approach: | They present a dataset that contains 18,736 multiple-choice questions from 14 templates . they propose to use the dataset to probe both causal and masked language models . |
| Outcome: | The proposed dataset contains 18,736 multiple-choice questions covering 10 physical reasoning concepts. |