Papers with bits-per-character
Form and Meaning in Intrinsic Multilingual Evaluations (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing metrics for conditional language models are not universally comparable . a number of assumptions are made about their quality in multilingual settings . |
| Approach: | They make assumptions about the quality of conditional language models and their semantic meanings that are not consistent with current metrics. |
| Outcome: | The results show that current evaluation metrics are not universally comparable . they use six metrics on two multi-parallel corpora with mono- and multilingual models . |
Improved Language Modeling by Decoding the Past (P19-1)
Copied to clipboard
| Challenge: | Existing methods to improve language modeling performance are based on regularized LSTMs with a large number of parameters and training time. |
| Approach: | They propose a method that decodes the last token in context using the predicted distribution of the next token. |
| Outcome: | The proposed method improves perplexity on the Penn Treebank dataset by 1.8 points and 2.3 points on the WikiText-2 datasets. |
Better Character Language Modeling through Morphology (P19-1)
Copied to clipboard
| Challenge: | Inflected words benefit more from explicitly modeling morphology than uninflectes . morphological supervision is also used to augment character language models in low-resource languages . |
| Approach: | They add morphological supervision to character language models via multitasking to improve BPC performance across 24 languages even when morphology data and language modeling data are disjointed. |
| Outcome: | The addition improves performance even when morphology data and language modeling data are disjointed. |