When Cantonese NLP Meets Pre-training: Progress and Challenges (2022.aacl-tutorials)
Copied to clipboard
| Challenge: | Cantonese is an influential Chinese variant with a large population of speakers worldwide. |
| Approach: | This tutorial will review Cantonese's progress in linguistics and NLP . it will introduce transformer-based pre-training methods for a wide range of downstream tasks . |
| Outcome: | This tutorial will present the main challenges for Cantonese NLP in relation to Cantonesian language idiosyncrasies of colloquialism and multilingualism. |
Similar Papers
Developing and Utilizing a Large-Scale Cantonese Dataset for Multi-Tasking in Large Language Models (2025.findings-emnlp)
Copied to clipboard
Jiyue Jiang, Alfred Kar Yin Truong, Yanyu Chen, Qinghang Bao, Sheng Wang, Pengan Chen, Jiuming Wang, Lingpeng Kong, Yu Li, Chuan Wu
| Challenge: | Cantonese is considered a low-resource language due to the dominance of Mandarin . rich colloquial vocabulary of Cantone, English loanwords, and code-switching characteristics add to the complexity of corpus collection and processing. |
| Approach: | We collect Cantonese texts from open source corpora, Hong Kong-specific forums, Wikipedia . we refine the model through supervised fine-tuning on curated Cantonesian tasks . |
| Outcome: | The model achieves state-of-the-art (SOTA) performance on four Cantonese benchmarks. |
How Well Do LLMs Handle Cantonese? Benchmarking Cantonese Capabilities of Large Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Cantonese has scant representation in NLP research, especially compared to other languages from similarly developed regions. |
| Approach: | They propose to evaluate Cantonese LLM performance in factual generation, mathematical logic, complex reasoning, and general knowledge in Cantonesian. |
| Outcome: | The proposed models will evaluate Cantonese's performance in factual generation, mathematical logic, complex reasoning, and general knowledge in Cantone. |
ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information (2021.acl-long)
Copied to clipboard
| Challenge: | ChineseBERT model incorporates glyph and pinyin information of Chinese characters into pretraining . proposed model achieves new performance boost over baseline models with fewer training steps . |
| Approach: | They propose a ChineseBERT model that incorporates glyph and pinyin information into pretraining . the glyph embedding is obtained based on different fonts of a character, and the pinyink embeddment characterizes the pronunciation of Chinese characters. |
| Outcome: | The proposed model achieves new performance boosts over baseline models with fewer training steps. |
Enriching Linguistic Representation in the Cantonese Wordnet and Building the New Cantonese Wordnet Corpus (2022.lrec-1)
Copied to clipboard
| Challenge: | Currently, our wordnet includes a little over 5,200 concepts and 16,300 senses . |
| Approach: | They propose to improve the Cantonese Wordnet by increasing the general coverage, adding functional categories, enriching verbal representations and creating the Cannese WordNet Corpus . |
| Outcome: | The new version includes a little over 5,200 concepts and 16,300 senses . |
Challenges and Strategies in Cross-Cultural NLP (2022.acl-long)
Copied to clipboard
Daniel Hershcovich, Stella Frank, Heather Lent, Miryam de Lhoneux, Mostafa Abdou, Stephanie Brandl, Emanuele Bugliarello, Laura Cabello Piqueras, Ilias Chalkidis, Ruixiang Cui, Constanza Fierro, Katerina Margatina, Phillip Rust, Anders Søgaard
| Challenge: | Various efforts have been made to accommodate linguistic diversity and serve speakers of many different languages. |
| Approach: | They propose a framework to examine cultural differences in NLP to better serve users . they argue that cultural knowledge, preferences and values can affect NLP practices . |
| Outcome: | The proposed framework examines how cultural knowledge, preferences and values can affect NLP practices. |
A Tour of Explicit Multilingual Semantics: Word Sense Disambiguation, Semantic Role Labeling and Semantic Parsing (2022.aacl-tutorials)
Copied to clipboard
| Challenge: | a recent advent of pretrained language models has sparked a revolution in NLP . but, there are still questions about whether current approaches capture explicit, symbolic meaning . this tutorial will review efforts to tackle three key open problems in lexical and sentence-level semantics . |
| Approach: | This tutorial reviews recent efforts to shed light on meaning in NLP . it will focus on three key open problems in lexical and sentence-level semantics . |
| Outcome: | This tutorial reviews recent efforts to shed light on meaning in NLP . it focuses on three key open problems in lexical and sentence-level semantics . |
Expanding Pretrained Models to Thousands More Languages via Lexicon-based Adaptation (2022.acl-long)
Copied to clipboard
| Challenge: | Recent studies have found that the performance of multilingual pretrained models is highly dependent on the availability of monolingual or parallel text in a target language. |
| Approach: | They propose to use bilingual lexicons to synthesize textual or labeled data and combine it with monolingual or parallel text when available. |
| Outcome: | The proposed methods improve performance for 19 under-represented languages with and without extra monolingual text. |
Can You Tell Me How to Get Past Sesame Street? Sentence-Level Pretraining Beyond Language Modeling (P19-1)
Copied to clipboard
Alex Wang, Jan Hula, Patrick Xia, Raghavendra Pappagari, R. Thomas McCoy, Roma Patel, Najoung Kim, Ian Tenney, Yinghui Huang, Katherin Yu, Shuning Jin, Berlin Chen, Benjamin Van Durme, Edouard Grave, Ellie Pavlick, Samuel R. Bowman
| Challenge: | State-of-the-art models in natural language processing (NLP) often incorporate sentence encoder functions which generate a sequence of vectors intended to represent the in-context meaning of each word in an input text. |
| Approach: | They conduct the first large-scale systematic study of candidate pretraining tasks, comparing 19 different tasks as alternatives and complements to language modeling. |
| Outcome: | The proposed model can be used to train sentences on language modeling tasks. |
Which *BERT? A Survey Organizing Contextualized Encoders (2020.emnlp-main)
Copied to clipboard
| Challenge: | a survey on language representation learning aims to highlight common themes . we focus on the areas of progress, compared to other fields, and discuss how each area is evaluated. |
| Approach: | They present a survey on language representation learning to highlight common themes . they compare contributions in contextualized text encoders to ideas from other fields . |
| Outcome: | The proposed survey aims to highlight common themes in the field of language representation learning. |
Mini But Mighty: Efficient Multilingual Pretraining with Linguistically-Informed Data Selection (2023.findings-eacl)
Copied to clipboard
| Challenge: | AfriBERTa shows that training transformer models from scratch on 1GB of data from many unrelated African languages outperforms massively multilingual models on downstream NLP tasks. |
| Approach: | They propose that training on smaller amounts of data but from related languages could match the performance of models trained on large, unrelated data. |
| Outcome: | The proposed model outperforms models trained on large, unrelated datasets on downstream NLP tasks. |