Papers by Changwei Hu
TNT: Text Normalization based Pre-training of Transformers for Content Moderation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Language model pre-training (self-supervised or unsupervised learning) has been widely used in a multitude of language processing tasks such as named entity recognition, sentiment analysis, question answering and content moderation. |
| Approach: | They propose a new language pre-training model TNT for content moderation that uses a combination of masking strategy and text normalization to learn from text. |
| Outcome: | The proposed model outperforms baselines on hate speech classification task and is a potential approach to misspelling correction. |
Repulsive Attention: Rethinking Multi-head Attention as Bayesian Inference (2020.emnlp-main)
Copied to clipboard
Bang An, Jie Lyu, Zhenyi Wang, Chunyuan Li, Changwei Hu, Fei Tan, Ruiyi Zhang, Yifan Hu, Changyou Chen
| Challenge: | Existing studies show that multi-head attention is an effective module in deep neural networks, but there are no explicit mechanisms guaranteeing this property. |
| Approach: | They propose a non-parametric approach that explicitly improves the repulsiveness in multi-head attention and consequently strengthens model’s expressiveness. |
| Outcome: | The proposed approach improves the repulsiveness in multi-head attention and strengthens model’s expressiveness. |
BERT-Beta: A Proactive Probabilistic Approach to Text Moderation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to text moderation are reactive and do not account for user generated content. |
| Approach: | They propose a text toxicity propensity model to characterize extent to which a user generated text attracts toxic comments and introduce a beta regression model to do the probabilistic modeling. |
| Outcome: | The proposed model performs well in comprehensive experiments and is scalable. |
HABERTOR: An Efficient and Effective Deep Hatespeech Detector (2020.emnlp-main)
Copied to clipboard
| Challenge: | HABERTOR model is a highly efficient and effective alternative to BERT for the hatespeech classification task. |
| Approach: | They propose to modify BERT's HABERTOR model to generate its own vocabularies and pre-trained it using the largest scale hatespeech dataset. |
| Outcome: | The proposed model is faster, more efficient and more robust than existing methods for hatespeech classification. |