Papers by Andreas Grivas
Low-Rank Softmax Can Have Unargmaxable Classes in Theory but Rarely in Practice (2022.acl-long)
Copied to clipboard
| Challenge: | Probabilistic multiclass classifiers with large number of output classes are commonplace in natural language processing. |
| Approach: | They propose to use argmax to predict words from a large vocabulary in NLP models . they find that 13 out of 150 models do indeed have such unargmaxable tokens . |
| Outcome: | The proposed algorithms detect unargmaxable tokens in large language models and translation models. |
What do character-level models learn about morphology? The case of dependency parsing (D18-1)
Copied to clipboard
| Challenge: | morphologically rich languages require character-level input models to learn morphology, but some models are poor at disambiguating some words . authors of this study show that character- level models learn a lot from input input . explicit modeling of morphologies is expensive and expensive, authors say . |
| Approach: | They compare character-level models to an oracle with explicit morphological analysis . they show that explicitly modeling morphology improves their best model . |
| Outcome: | The results show that character-level models learn morphology better than word models . the authors compare character-based models to oracles on 12 languages with morphological typologies . |