Diversifying language models for lesser-studied languages and language-usage contexts: A case of second language Korean (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing morpheme parsers/taggers do not work reliably and optimally for L2 data. |
| Approach: | They train a neural network model on varying L2 datasets and measure its morpheme parsing/POS tagging performance on L2 test sets. |
| Outcome: | The proposed model excels in domain-specific tokenization and POS tagging compared to the baseline model. |
Similar Papers
Beyond Distribution: Investigating Language Models’ Understanding of Sino-Korean Morphemes (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Transformer-based language models can learn compositional morphology of SK morphemes . morphological models trained on Hangul text can learn SK, but performance is based on frequency of words . |
| Approach: | They investigate whether Transformer-based language models can learn compositional morphology of Sino-Korean morphemes. |
| Outcome: | The proposed models learn the compositional morphology of SK morphemes from real and fake pairs. |
What data should I include in my POS tagging training set? (2025.findings-emnlp)
Copied to clipboard
| Challenge: | POS tagging is a crucial task for descriptive linguistics and language documentation . POS tags are not available in all languages, but are used for training sets for understudied languages . |
| Approach: | They compare POS tagging with in-context learning, active learning, and random sampling . they find that POS can deliver reasonable results for communities with limited resources . |
| Outcome: | The proposed training set for Indigenous and endangered languages performs better than random sampling. |
Morphosyntactic Tagging with a Meta-BiLSTM Model over Context Sensitive Token Encodings (P18-1)
Copied to clipboard
| Challenge: | recurrent neural networks have produced significant advances in part-of-speech tagging accuracy . a common feature of these models is the presence of rich initial word encodings . however, word or sub-word information interacts only through subsequent recursive layers . |
| Approach: | They propose to use recurrent neural networks with sentence-level context for initial character and word-based representations. |
| Outcome: | The proposed model has the highest accuracy of all participating systems in the CoNLL 2017 task. |
Distant Supervision from Disparate Sources for Low-Resource Part-of-Speech Tagging (D18-1)
Copied to clipboard
| Challenge: | Low-resource languages lack manual annotated data to learn basic models such as part-of-speech (POS) taggers. |
| Approach: | They propose a cross-lingual neural part-of-speech tagger that learns from disparate sources of distant supervision in a uniform framework. |
| Outcome: | The proposed model scales to hundreds of low-resource languages without access to gold annotated data. |
Empowering Low-Resource Regional Languages with Lexicons : A Comparative Study of NLP Tools for Morphosyntactic Analysis (2024.lrec-main)
Copied to clipboard
| Challenge: | a lack of human and financial resources makes integrating lexicon information to low-resource languages challenging. |
| Approach: | They propose to use a bilingual lexicon to integrate lexical information to low-resource language . they compare a lexiconal approach to a neural approach that uses a larger lexicone . |
| Outcome: | The proposed approach improves POS tagging while using different lexicon sizes. |
Yet Another Format of Universal Dependencies for Korean (2022.coling-1)
Copied to clipboard
Yige Chen, Eunkyul Leah Jo, Yundong Yao, KyungTae Lim, Miikka Silfverberg, Francis M. Tyers, Jungyeul Park
| Challenge: | Existing dependency parsers for Korean do not perform as well as their English counterparts due to the complexity of Korean's linguistic features. |
| Approach: | They propose a morpheme-based Korean dependency parsing format and propose to adopt it to Universal Dependencies. |
| Outcome: | The proposed format outperforms parsing results for Korean UD treebanks and detailed error analysis. |
Topicalization in Language Models: A Case Study on Japanese (2022.coling-1)
Copied to clipboard
| Challenge: | a recent study has shown that neural language models can capture discourse-level preferences in text generation . a particular aspect of discourse is the topic-comment structure . |
| Approach: | They analyze whether neural language models can capture discourse-level preferences in text generation . they use Japanese language and crowdsourced human topicalization judgment data . |
| Outcome: | The proposed model can capture human-like generalizations in discourse-level linguistic aspects. |
Code-switched Language Models Using Dual RNNs and Same-Source Pretraining (D18-1)
Copied to clipboard
| Challenge: | Using recurrent neural networks to build language models for code-switched text is an important problem with implications to downstream applications such as speech recognition and machine translation. |
| Approach: | They propose a novel recurrent neural network unit with dual components that focus on each language in the code-switched text separately and a generative model estimated using the training data. |
| Outcome: | The proposed techniques yield significant reductions in perplexity on Mandarin-English task and improve on baseline models. |
How a Bilingual LM Becomes Bilingual: Tracing Internal Representations with Sparse Autoencoders (2025.findings-emnlp)
Copied to clipboard
Tatsuro Inaba, Go Kamoda, Kentaro Inui, Masaru Isonuma, Yusuke Miyao, Yohei Oseki, Yu Takagi, Benjamin Heinzerling
| Challenge: | Using sparse autoencoders, we explore how bilingual language models develop complex internal representations. |
| Approach: | They employ sparse autoencoders to analyze bilingual language models' internal representations. |
| Outcome: | The proposed method integrates decomposed representations from a fully trained model into a mid-training model. |
Tokenization Impacts Multilingual Language Modeling: Assessing Vocabulary Allocation and Overlap Across Languages (2023.findings-acl)
Copied to clipboard
| Challenge: | Multilingual language models perform surprisingly well in a variety of NLP tasks for diverse languages. |
| Approach: | They propose to evaluate the quality of lexical representation and vocabulary overlap observed in sub-word tokenizers. |
| Outcome: | The proposed criteria show that the overlap of vocabulary across languages can be detrimental to certain downstream tasks. |