Papers by Yichao Lu
Pretrain-Finetune Based Training of Task-Oriented Dialogue Systems in a Real-World Setting (2021.naacl-industry)
Copied to clipboard
| Challenge: | a challenge in building task-oriented dialogue systems is the limited amount of supervised training data available. |
| Approach: | They propose a method for training retrieval-based dialogue systems using annotated data and a larger, unlabeled dataset. |
| Outcome: | The proposed method improves model performance offline and online compared with no pretraining . the model is deployed in an agent-support application and evaluated on live customer service contacts . |
Adversarial Learning with Contextual Embeddings for Zero-resource Cross-lingual Classification and NER (D19-1)
Copied to clipboard
| Challenge: | Contextual word embeddings have demonstrated state-of-the-art performance on various NLP tasks. |
| Approach: | They propose to use adversarial learning to improve upon multilingual BERT's zero-resource cross-lingual performance by aligning embeddings of English documents and their translations. |
| Outcome: | The multilingual version of BERT performs surprisingly well in cross-lingual settings, even when only labeled English data is used to finetune the model. |
The Multilingual Amazon Reviews Corpus (2020.emnlp-main)
Copied to clipboard
| Challenge: | The corpus contains reviews in English, Japanese, German, French, Spanish, and Chinese, which were collected between 2015 and 2019 . |
| Approach: | They propose to use mean absolute error (MAE) instead of classification accuracy for this task since MAE accounts for ordinal nature of the ratings. |
| Outcome: | The proposed model uses mean absolute error (MAE) instead of classification accuracy since MAE accounts for ordinal nature of the ratings. |
Don’t Use English Dev: On the Zero-Shot Cross-Lingual Evaluation of Contextual Embeddings (2020.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual contextual embeddings have demonstrated state-of-the-art performance in zero-shot cross-lingual transfer learning. |
| Approach: | They show that English dev accuracy makes it difficult to obtain reproducible results . they recommend providing oracle scores alongside zero-shot results if possible . |
| Outcome: | mBERT and XLM have shown strong performance on cross-lingual recognition, text classification, dependency parsing, and other tasks. |
Unsupervised Bitext Mining and Translation via Self-Trained Contextual Embeddings (2020.tacl-1)
Copied to clipboard
| Challenge: | Existing methods to extract parallel sentences from unaligned text yield surprisingly good results. |
| Approach: | They propose an unsupervised method to create pseudo-parallel corpora for machine translation (MT) from unaligned text using multilingual BERT to create source and target sentence embeddings for nearest-neighbor search and adapt the model via self-training. |
| Outcome: | The proposed method outperforms existing methods and outperformed previous state-of-the-art methods by boosting translation performance by up to 3.5 BLEU on the WMT’14 French-English and WMT'16 German-English tasks. |
Goal-Oriented End-to-End Conversational Models with Profile Features in a Real-World Setting (N19-2)
Copied to clipboard
| Challenge: | a recent study has focused on how algorithmic improvements help model performance on fabricated datasets. |
| Approach: | They propose two approaches to train conversational neural models for goal-oriented conversational systems . they train models on historical chat transcripts and test on live contacts . |
| Outcome: | The proposed model is able to generate top-four responses on live contacts . the model is also able for customer profile features to assess their impact on performance . |