| Challenge: | Using a multi-modal multi-granularity tokenizer, we analyze ancient Chinese scripts . a large proportion of the characters in ancient Chinese are rare or undeciphered . |
| Approach: | They propose a multi-modal multi-granularity tokenizer specifically designed for ancient Chinese scripts. |
| Outcome: | The proposed tokenizer improves on the part-of-speech tagging task on the Chu bamboo slip script. |
Similar Papers
Sub-Character Tokenization for Chinese Pretrained Language Models (2023.tacl-1)
Copied to clipboard
Chenglei Si, Zhengyan Zhang, Yingfa Chen, Fanchao Qi, Xiaozhi Wang, Zhiyuan Liu, Yasheng Wang, Qun Liu, Maosong Sun
| Challenge: | Existing tokenization methods for Chinese PLMs treat each character as an indivisible token, but ignore the unique feature of the writing system where additional linguistic information exists below the character level. |
| Approach: | They propose to encode Chinese characters into short sequences and construct Chinese vocabulary based on the encoded text. |
| Outcome: | The proposed tokenizers can tokenize inputs into much shorter sequences, improving computational efficiency. |
ACSE: An Ancient Character Semantic-Aware Embedding for Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on pre-Qin documents are insufficient to understand ancient characters . ancient characters have a low level of digitization and training corpora are extremely scarce . |
| Approach: | They propose a semantic-aware embedding for ancient Chinese characters that integrates glyphs and lexicality into modern Chinese semantic space. |
| Outcome: | The proposed model integrates glyph and lexicality of ancient characters and maps them to the modern Chinese semantic space. |
Fluid Annotation: A Granularity-aware Annotation Tool for Chinese Word Fluidity (L18-1)
Copied to clipboard
| Challenge: | Using word segmentation, we propose a wordhood annotation framework for Chinese language . word segmentations have been used for years in preprocessing NLP tasks for languages without explicit word delimiter. |
| Approach: | They propose a word-granularity-aware annotation framework for Chinese language . they argue that word segmentation is fluid in nature and that it rearranges the boundary of word segmentations and linguistic annotation. |
| Outcome: | The proposed framework rearranges the boundary between word segmentation and linguistic annotation and supports flexible annotation tasks for various linguistic and affective phenomena. |
Phonemes to the Rescue: Multilingual Tokenization Based on International Phonetic Alphabet (2026.acl-long)
Copied to clipboard
| Challenge: | Widely-used subword tokenization approaches favor high-resource languages and tokenizer-free methods yield longer sequences for scripts with a higher bytes-per-character ratio. |
| Approach: | They propose to use the International Phonetic Alphabet (IPA) as a language-agnostic input representation for multilingual tokenizers. |
| Outcome: | The proposed model improves tokenization quality and generalizes more effectively to unseen languages and scripts. |
More than Text: Multi-modal Chinese Word Segmentation (2021.acl-short)
Copied to clipboard
| Challenge: | Currently, word segmentation is performed in many languages without word delimiters. |
| Approach: | They propose to combine the multi-modality to perform Chinese word segmentation . they propose a time-dependent multi-module interactive model to integrate multi-modality information . |
| Outcome: | The proposed model integrates multi-modal information for word sequence labeling with Chinese language as target . the proposed model performs well on three training sets on Chinese and other languages without word delimiters. |
CH-SIMS: A Chinese Multimodal Sentiment Analysis Dataset with Fine-grained Annotation of Modality (2020.acl-main)
Copied to clipboard
| Challenge: | Existing studies in multimodal sentiment analysis only use unified multimodal annotations, which do not reflect the independent sentiment of single modalities. |
| Approach: | They propose a Chinese single- and multi-modal sentiment analysis dataset with multimodal and independent unimodal annotations that can be used to study the interaction between modalities. |
| Outcome: | The proposed methods achieve state-of-the-art performance and learn more distinctive unimodal representations. |
Joint Chinese Word Segmentation and Part-of-speech Tagging via Multi-channel Attention of Character N-grams (2020.coling-main)
Copied to clipboard
| Challenge: | Chinese word segmentation and part-of-speech tagging can be performed in a sequential order . existing studies have shown that jointly performing them can be effective . |
| Approach: | They propose a character-based neural model enhanced by multi-channel attention of n-grams. |
| Outcome: | The proposed model outperforms baseline models on five benchmark datasets. |
Large-Scale Corpus Construction and Retrieval-Augmented Generation for Ancient Chinese Poetry: New Method and Data Insights (2025.findings-naacl)
Copied to clipboard
| Challenge: | Ancient Chinese poetry presents unique challenges for Large Language Models due to data scarcity and limited ability of general LLMs when dealing with ACP. |
| Approach: | They propose a specialized Retrieval-Augmented Generation framework to improve LLMs' performance . they use 1.1 million ancient poems and 990K related texts to address hallucination issues . |
| Outcome: | The proposed framework improves performance of LLMs in ancient Chinese poetry domain from 49.2% to 89.0%. |
A Pragmatic Approach for Classical Chinese Word Segmentation (L18-1)
Copied to clipboard
| Challenge: | Classical Chinese word segmentation is largely neglected due to its obsoleteness . a new approach to segmentation using a marked-up corpus is needed . |
| Approach: | They propose a pragmatic approach to deal with Classical Chinese word segmentation without any marked-up corpus. |
| Outcome: | The proposed method makes the CCWS without any marked-up corpus more accurate compared with collocation-based methods. |
Trainable, Multiword-aware Linguistic Tokenization Using Modern Neural Networks (2026.eacl-srw)
Copied to clipboard
| Challenge: | Tokenization is a fundamental task in natural language processing that forms the first step of many pipelines. |
| Approach: | They propose to use a standard tokenizer trained without MWE-awareness as a baseline and a character-level SRN+CRF model to train token-level models. |
| Outcome: | The proposed tokenizers are based on a character-level and token-level sequence labeling problem and are consistent with the proposed pipelines. |