Papers by Gongbo Tang
Encoders Help You Disambiguate Word Senses in Neural Machine Translation (D19-1)
Copied to clipboard
| Challenge: | Neural machine translation models can perform word sense disambiguation (WSD) however, it is unclear which component dominates the process of disambiguating words. |
| Approach: | They evaluate hidden states and investigate distributions of self-attention in NMT encoders and decoders to disambiguate word senses. |
| Outcome: | The proposed model outperforms encoder hidden states on large datasets . the model outpersforms decoders on large data sets . |
Understanding Pure Character-Based Neural Machine Translation: The Case of Translating Finnish into English (2020.coling-main)
Copied to clipboard
| Challenge: | Recent work shows that deeper character-based neural machine translation models outperform subword-based models. |
| Approach: | They propose to investigate the ability of character-based models to learn word senses and morphological inflections and the attention mechanism in Finnish into English translation. |
| Outcome: | The character-based models outperform subword-based model in Finnish to English translation. |
Revisiting Negation in Neural Machine Translation (2021.tacl-1)
Copied to clipboard
| Challenge: | Negation is an important linguistic phenomenon in machine translation, as errors in translating negation may change the meaning of source sentences completely. |
| Approach: | They evaluate the translation of negation in English–German (EN–DE) and English– Chinese (EN-ZH) . they find that NMT models can distinguish negation and non-negation tokens very well and encode a lot of information about negation . |
| Outcome: | The accuracy of manual evaluation in ENDE, DEEN, ENZH, and ZHEN is 95.7%, 94.8%, 93.4%, and 91.7% respectively. |
MTA4DPR: Multi-Teaching-Assistants Based Iterative Knowledge Distillation for Dense Passage Retrieval (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown the effectiveness of knowledge distillation in DPR, but there is a performance gap between the teacher and the distilled student. |
| Approach: | They propose an iterative knowledge distillation method which transfers knowledge from teacher to student with help of multiple assistants in an iterated manner. |
| Outcome: | The proposed method achieves state-of-the-art performance among models with same parameters on multiple datasets and is competitive when compared with larger models. |
Parallel Data Helps Neural Entity Coreference Resolution (2023.findings-acl)
Copied to clipboard
| Challenge: | Current neural coreference models are trained on monolingual annotated data but annotating such coreference information is expensive and challenging. |
| Approach: | They propose a simple yet effective model to exploit coreference knowledge from parallel data. |
| Outcome: | The proposed model improves on the OntoNotes 5.0 English dataset by 1.74 percentage points . it is based on an unsupervised module learning coreference from annotations . |
An Evaluation of Neural Machine Translation Models on Historical Spelling Normalization (C18-1)
Copied to clipboard
| Challenge: | In this paper, we apply different NMT models to the problem of historical spelling normalization for five languages . we find that NMT model is much better than SMT in terms of character error rate . |
| Approach: | They propose to use NMT models to solve the problem of historical spelling normalization in five languages. |
| Outcome: | The proposed method improves historical spelling normalization for five languages. |
Why Self-Attention? A Targeted Evaluation of Neural Machine Translation Architectures (D18-1)
Copied to clipboard
| Challenge: | Recent studies show that non-recurrent architectures outperform RNNs in neural machine translation. |
| Approach: | They hypothesize that CNNs and self-attentional networks could extract semantic features from source text. |
| Outcome: | The proposed architectures outperform RNNs on two tasks: subject-verb agreement and word sense disambiguation. |