Structural Supervision for Word Alignment and Machine Translation (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing knowledge on syntactic structure neglects the rich structural information from target tokens and the structural similarity between the source and target sentences. |
| Approach: | They propose to incorporate syntactic structure of both source and target tokens into the encoder-decoder framework, tightly correlating the internal logic of word alignment and machine translation for multi-task learning. |
| Outcome: | The proposed method outperforms baselines on four publicly available language pairs and consistently outperformed baselines in alignment accuracy and translation quality. |
Similar Papers
Neural Machine Translation with Synchronous Latent Phrase Structure (2021.acl-srw)
Copied to clipboard
| Challenge: | grammatical information annotation requires high human resources and is not trivial due to language mismatches and out-of-vocabulary problem. |
| Approach: | They propose to incorporate grammatical information without supervising annotation by induced latent phrase structure and synchronized phrase structures in encoder and decoder to enhance explainability. |
| Outcome: | The proposed method produces better performance and explainability in translation and alignment tasks without extra resources. |
Exploiting Syntactic Structure for Better Language Modeling: A Syntactic Distance Approach (2020.acl-main)
Copied to clipboard
| Challenge: | incorporating syntactic structure into language models has been a challenge since the 1990s. |
| Approach: | They propose to use syntactic information to integrate syntastic structure into neural language models by providing ground truth parse trees as additional training signals. |
| Outcome: | The proposed model achieves lower perplexity and better quality when ground truth parse trees are provided as training signals. |
Exploiting Semantics in Neural Machine Translation with Graph Convolutional Networks (N18-2)
Copied to clipboard
| Challenge: | Semantic representations have long been argued as potentially useful for enforcing meaning preservation and improving generalization performance of machine translation methods. |
| Approach: | They propose to integrate semantic representations into neural machine translation by injecting a semantic bias into sentence encoders and achieving improvements in BLEU scores. |
| Outcome: | The proposed representations achieve better BLEU scores over the linguistic-agnostic and syntax-aware versions on the English–German language pair. |
Multilingualization of Medical Terminology: Semantic and Structural Embedding Approaches (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing methods for multilingual terminology curation are limited as they do not fit the term within existing terminology. |
| Approach: | They propose a method to encode the structural property of a term by aligning embeddings using graph convolutional networks trained from separate languages. |
| Outcome: | The proposed method can encode the structural property of a term by aligning embeddings using graph convolutional networks trained from separate languages. |
A Tree-based Decoder for Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Existing work on adding syntactic information to NMT systems is limited to linguistically-inspired tree structures. |
| Approach: | They propose an NMT model that can naturally generate the topology of an arbitrary tree structure on the target side. |
| Outcome: | The proposed model outperforms standard seq2seq models by 2.1 BLEU points and other methods for incorporating target-side syntax by 0.7 BLUE points. |
Syntactically Supervised Transformers for Faster Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Standard decoders for neural machine translation generate a single token per timestep, which slows inference . a series of controlled experiments demonstrates that SynST decodes sentences 5x faster than the baseline autoregressive Transformer. |
| Approach: | They propose a syntactically supervised Transformer that generates all target tokens in one shot . synST is a variant of the Transformer architecture that autoregressively predicts a chunked parse tree . |
| Outcome: | The proposed method decodes sentences 5x faster than the baseline method on En-De and En-Fr datasets while achieving higher BLEU scores. |
Understanding Cross-Lingual Alignment—A Survey (2024.findings-acl)
Copied to clipboard
| Challenge: | Cross-lingual alignment is the meaningful similarity of representations across languages in multilingual language models. |
| Approach: | They propose a taxonomy of methods to improve cross-lingual alignment . they argue that an effective trade-off between language-neutral and language-specific information is key . |
| Outcome: | The proposed methods can be applied to encoder models and encoder-decoder-only models . they show that language-neutral and language-specific information is key . |
Structural Supervision Improves Few-Shot Learning and Syntactic Generalization in Neural Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have not investigated the relationship between a token's frequency in the training corpus and syntactic properties models learn about it. |
| Approach: | They develop controlled experiments that probe models’ syntactic nominal number and verbal argument structure generalizations for tokens seen as few as two times during training. |
| Outcome: | The proposed models can make syntactic generalizations for tokens seen as few as two times during training and transfer them to transformed contexts. |
Cross-lingual Structure Transfer for Relation and Event Extraction (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to identify complex semantic structures are difficult to train from under-annotated sources. |
| Approach: | They exploit relation- and event-relevant language-universal features to train relation or event extractors from source annotations and apply them to target languages. |
| Outcome: | The proposed approach achieves comparable performance to state-of-the-art models trained on 3,000 manually annotated mentions. |
Top-down Tree Structured Decoding with Syntactic Connections for Neural Machine Translation and Parsing (D18-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) models are based on sequential decoding or serialisation of structured data into sequence. |
| Approach: | They propose a model that combines sequential encoder with tree-structured decoding augmented with a syntax-aware attention model. |
| Outcome: | The proposed model produces fluent translations with better reordering than previous models. |