Papers with SentencePiece
SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing (D18-2)
Copied to clipboard
| Challenge: | Existing subword segmentation tools assume input is pre-tokenized into word sequences, but SentencePiece can train subword models directly from raw sentences. |
| Approach: | They propose a language-independent subword tokenizer and detokenizer for Neural-based text processing. |
| Outcome: | The proposed system achieves comparable accuracy to training from raw sentences. |
SiBert: Enhanced Chinese Pre-trained Language Model with Sentence Insertion (2020.lrec-1)
Copied to clipboard
| Challenge: | Recent studies show that pre-trained models can learn unsupervised language representations by self-supervised tasks on large-scale corpora. |
| Approach: | They propose a pre-training task called Sentence Insertion for Chinese query-passage pairs NLP tasks . they propose 'word segmentation' method to enhance Chinese Bert performance . |
| Outcome: | The proposed task improves Chinese pre-trained models significantly. |
Evaluating the Impact of Sub-word Information and Cross-lingual Word Embeddings on Mi’kmaq Language Modelling (2020.lrec-1)
Copied to clipboard
| Challenge: | Mi'kmaq is an Indigenous language spoken primarily in Eastern Canada. |
| Approach: | They consider n-gram and RNN language models for Mi'kmaq and use them to investigate their performance. |
| Outcome: | The proposed model performs better than word-level models, but does not improve over word-based models. |
How Important Is Tokenization in French Medical Masked Language Models? (2024.lrec-main)
Copied to clipboard
| Challenge: | Word tokenization into subword units has become the prevailing standard in the field of natural language processing (NLP) over recent years . the precise factors contributing to its success remain unclear . |
| Approach: | They propose a tokenization strategy that integrates morpheme-enriched word segmentation into existing tokenization methods. |
| Outcome: | The proposed tokenization strategy outperforms character and word tokenization but the precise factors contributing to its success remain unclear. |