Breaking the Representation Bottleneck of Chinese Characters: Neural Machine Translation with Stroke Sequence Modeling (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing research treats Chinese character as a minimum unit for representation . however, such representation suffers from two bottlenecks: 1) learning bottleneck; 2) parameter bottleneck, each individual character has to be represented by a unique vector. |
| Approach: | They propose a representation method for Chinese characters to break the representation bottlenecks . they map each stroke to a specific Latin character, thus allowing similar Chinese characters . |
| Outcome: | The proposed representation method breaks two representation bottlenecks in Chinese character representation . it maps each stroke to a specific Latin character, thus allowing similar Chinese characters to have similar representations . |
Similar Papers
Revisiting Character-Based Neural Machine Translation with Capacity and Compression (D18-1)
Copied to clipboard
| Challenge: | Translating characters instead of words or word-fragments can simplify the processing pipeline but results in longer sequences . |
| Approach: | They propose to use sequence-to-sequence architectures of sufficient depth to solve the problem . they also evaluate the performance versus computation time tradeoffs they offer . |
| Outcome: | The proposed models outperform models operating over word fragments in character-level NMT, the authors show . they also show that the proposed models do not match the performance of their deep character baseline model . |
VCWE: Visual Character-Enhanced Word Embeddings (N19-1)
Copied to clipboard
| Challenge: | Currently, word embeddings are playing a pivotal role in many natural language processing tasks. |
| Approach: | They propose a model to learn Chinese word embeddings via three-level composition . they use convolutional neural network to extract intra-character compositionality from character shape . |
| Outcome: | The proposed model performs better on word similarity, sentiment analysis, named entity recognition and part-of-speech tagging tasks. |
One Size Does Not Fit All: Comparing NMT Representations of Different Granularities (N19-1)
Copied to clipboard
| Challenge: | Recent work has shown that contextualized word representations are a viable alternative to simple word prediction tasks. |
| Approach: | They propose to use subword units and characters to model morphology, syntax, and semantics instead of word embeddings. |
| Outcome: | The proposed representations are better for modeling syntax and more robust to noisy input. |
Our Neural Machine Translation Systems for WAT 2019 (D19-52)
Copied to clipboard
| Challenge: | In the last five years, statistical machine translation is gradually fading out in favor of neural machine translation. |
| Approach: | They describe a novel Neural Machine Translation (NMT) system for the WAT 2019 translation tasks they focus on. |
| Outcome: | The proposed system improves translation accuracy while replacing absolute position representations with relative positions. |
Self-Paced Learning for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown that the training of neural machine translation (NMT) rely on the quality of artificial schedule drawn up with the handcrafted features, e.g. sentence length or word rarity. |
| Approach: | They propose to train NMT model using a self-paced learning approach that allows it to quantify the learning confidence over training examples and flexibly govern its learning via regulating the loss in each iteration step. |
| Outcome: | The proposed model outperforms baseline models and those trained with human-designed curricula on translation quality and convergence speed. |
Overcoming the Rare Word Problem for low-resource language pairs in Neural Machine Translation (D19-52)
Copied to clipboard
| Challenge: | Despite above approaches can improve the prediction of rare words, they still have challenges which have adverse effects on its effectiveness. |
| Approach: | They propose three ways to address rare-word problem in neural machine translation systems . they propose an algorithm to learn morphology of unknown words for English in supervised way to minimize adverse effect of rare- word problem. |
| Outcome: | The proposed approaches improve accuracy on two low-resource language pairs. |
On the Importance of Word Boundaries in Character-level Neural Machine Translation (D19-56)
Copied to clipboard
| Challenge: | Neural Machine Translation models typically use a fixed-size lexical vocabulary . subword segmentation methods rely on statistical heuristics that lack any linguistic notion . |
| Approach: | They propose a hierarchical decoding architecture for character-level NMT using subwords . they propose fewer parameters and a more efficient approach to perform translation at the level of words . |
| Outcome: | The proposed model can reach higher translation accuracy than the subword-level model with fewer parameters while maintaining longer-distance contextual and grammatical dependencies. |
A Discriminative Neural Model for Cross-Lingual Word Alignment (D19-1)
Copied to clipboard
| Challenge: | a novel word alignment model for machine translation has been developed for a number of languages . explicit word-to-word alignments have largely been lost in neural MT systems . |
| Approach: | They propose a discriminative word alignment model which integrates into a Transformer-based machine translation model. |
| Outcome: | The proposed model performs better on Chinese and Arabic alignments than standard models. |
ZEN: Pre-training Chinese Text Encoder Enhanced by N-gram Representations (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Experimental results show that pre-trained text encoders can perform many NLP tasks with less resource. |
| Approach: | They propose a BERT-based Chinese text encoder enhanced by n-gram representations . they show reasonable performance when ZEN is trained on a small corpus . |
| Outcome: | The proposed encoder incorporates the comprehensive information of both the character sequence and words or phrases it contains. |
Revisiting Pre-Trained Models for Chinese Natural Language Processing (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing pre-trained language models have shown tremendous improvements across various NLP tasks. |
| Approach: | They propose to revisit Chinese pre-trained language models to examine their effectiveness in a non-English language and release the Chinese pretrained model series to the community. |
| Outcome: | The proposed model improves on RoBERTa in several ways, especially the masking strategy that adopts MLM as correction (Mac). |