Papers by Wenming Xiao
E2E-VLP: End-to-End Vision-Language Pre-training Enhanced by Visual Learning (2021.acl-long)
Copied to clipboard
| Challenge: | Existing vision-language pre-training methods use a two-step training procedure to learn visual features from image-text pairs. |
| Approach: | They propose a vision-language pre-trained model for V+L understanding and generation using a unified Transformer framework. |
| Outcome: | The proposed model can learn visual representation and semantic alignments between image and text on visual-text pairs and on visual processing tasks. |
SentiX: A Sentiment-Aware Pre-Trained Model for Cross-Domain Sentiment Analysis (2020.coling-main)
Copied to clipboard
| Challenge: | Pre-trained language models have been widely applied to cross-domain NLP tasks like sentiment analysis, but fine-tuning them on the source domain tends to overfit, leading to inferior results on the target domain. |
| Approach: | They propose to pre-train a sentiment-aware language model (SentiX) via domain-invariant sentiment knowledge from large-scale review datasets and utilize it for cross-domain sentiment analysis tasks without fine-tuning. |
| Outcome: | The proposed model achieves state-of-the-art in all the cross-domain sentiment analysis tasks and can be trained with only 1% samples and better than BERT with 90% samples. |
Lexicon Enhanced Chinese Sequence Labeling Using BERT Adapter (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for Chinese sequence labelling only fuse lexicon features via a shallow and random initialized sequence layer and do not integrate them into the bottom layers of BERT. |
| Approach: | They propose a Lexicon Enhanced BERT model which integrates external lexicon knowledge into BERT layers directly by a lexiccon Adapter layer. |
| Outcome: | The proposed model integrates external lexicon knowledge into BERT layers directly by a Lexicon Adapter layer. |