Don’t Just Scratch the Surface: Enhancing Word Representations for Korean with Hanja (D19-1)
Copied to clipboard
| Challenge: | Existing knowledge of Korean and Chinese is based on cultural and historical reasons. |
| Approach: | They propose a method for improving Korean word representations using additional linguistic annotation by leveraging the fact that Hanja is closely related to Chinese. |
| Outcome: | The proposed approach improves representations on a novel Korean news headline generation task. |
Similar Papers
Subword-level Word Vector Representations for Korean (P18-1)
Copied to clipboard
| Challenge: | Existing research on word vectors for English focuses on decomposing words into subword units and using subwords to improve performance. |
| Approach: | They propose to decompose Korean words into the jamo-level, beyond the character-level . they develop Korean test sets for word similarity and analogy and make them publicly available . |
| Outcome: | The proposed method outperforms word2vec and character-level skip-grams on similarity and analogy tasks and contributes positively toward downstream NLP tasks such as sentiment analysis. |
Unsupervised Cross-Lingual Representation Learning (P19-4)
Copied to clipboard
| Challenge: | a comprehensive survey of cutting-edge weakly-supervised and unsupervised cross-lingual word representations is presented . |
| Approach: | This tutorial provides a comprehensive survey of recent work on weakly-supervised and unsupervised cross-lingual word representations. |
| Outcome: | This tutorial provides a comprehensive survey of cutting-edge weakly-supervised and unsupervised word representations. |
Advances in Pre-Training Distributed Word Representations (L18-1)
Copied to clipboard
| Challenge: | Pre-trained word representations are a building block of many Natural Language Processing and Machine Learning applications. |
| Approach: | They propose to combine known tricks and a set of publicly available pre-trained word vector representations to train high-quality representations. |
| Outcome: | The proposed models outperform the current state of the art on a number of tasks while maintaining a high training speed to scale to massive amount of data. |
Korean Language Modeling via Syntactic Guide (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing research on pre-trained language models focuses on widely-used languages . however, not every language can benefit from such models due to computational resources . |
| Approach: | They propose to build a pre-trained language model that understands the linguistic phenomena in the target language with low resources. |
| Outcome: | The proposed model improves the performance of Korean language understanding tasks. |
Gating Mechanisms for Combining Character and Word-level Word Representations: an Empirical Study (N19-3)
Copied to clipboard
| Challenge: | Existing studies show that combining character and word-level representations improves word and sentence representations . however, word-based embeddings do not account for derivational processes resulting in syntactically-similar words with different meanings. |
| Approach: | They propose to combine character and word-level representations to improve word and sentence representations. |
| Outcome: | The proposed method performed well in several word similarity datasets. |
Improving Cross-Lingual Word Embeddings by Meeting in the Middle (D18-1)
Copied to clipboard
| Challenge: | Cross-lingual word embeddings are becoming increasingly important in multilingual NLP. |
| Approach: | They propose to apply an additional transformation after initial alignment to align two disjoint monolingual vector spaces. |
| Outcome: | The proposed approach outperforms state-of-the-art models in monolingual and cross-lingual evaluation tasks. |
How to represent a word and predict it, too: Improving tied architectures for language modelling (D18-1)
Copied to clipboard
| Challenge: | Recent state-of-the-art models use word embeddings as input and output mappings instead of tied models. |
| Approach: | They propose to decouple hidden state from word embedding prediction . they extend their proposed modification to word2vec models . |
| Outcome: | The proposed architectures achieve comparable or better results compared to previous models without tying . the proposed architecture reduces parameters, enabling more compact models and faster learning. |
Korean L2 Vocabulary Prediction: Can a Large Annotated Corpus be Used to Train Better Models for Predicting Unknown Words? (L18-1)
Copied to clipboard
| Challenge: | a recent study suggests that a classifier trained on unknown words may yield better results for L2 learners. |
| Approach: | They propose to use a supervised learning classifier to predict word complexity in Korean . they propose to train models on annotated corpus of unknown words with 71 % precision . |
| Outcome: | The proposed model recalls 80 % of unknown words with 71 % precision. |
Understanding Cross-Lingual Alignment—A Survey (2024.findings-acl)
Copied to clipboard
| Challenge: | Cross-lingual alignment is the meaningful similarity of representations across languages in multilingual language models. |
| Approach: | They propose a taxonomy of methods to improve cross-lingual alignment . they argue that an effective trade-off between language-neutral and language-specific information is key . |
| Outcome: | The proposed methods can be applied to encoder models and encoder-decoder-only models . they show that language-neutral and language-specific information is key . |
How to (Properly) Evaluate Cross-Lingual Word Embeddings: On Strong Baselines, Comparative Analyses, and Some Misconceptions (P19-1)
Copied to clipboard
| Challenge: | Cross-lingual word embeddings (CLEs) are used for downstream NLP tasks . CLEs are based on bilingual lexicon induction (BLI) evaluations vary greatly, hindering ability to interpret performance and properties of different CLE models. |
| Approach: | They evaluate CLE models for a large number of language pairs on bilingual lexicon induction and three downstream tasks. |
| Outcome: | The proposed model performance is based on supervised and unsupervised models on bilingual lexicon induction and three downstream tasks. |