Papers by Yiren Chen
Syntax-BERT: Improving Pre-trained Transformers with Syntax Trees (2021.eacl-main)
Copied to clipboard
| Challenge: | Pre-trained language models like BERT achieve superior performances in various NLP tasks without explicit consideration of syntactic information. |
| Approach: | They propose a plug-and-play framework that incorporates syntax trees into pre-trained Transformers. |
| Outcome: | The proposed framework improves on pre-trained models on natural language understanding datasets and shows that it can be used to train pre-structured neural networks. |
TencentPretrain: A Scalable and Flexible Toolkit for Pre-training Models of Different Modalities (2023.acl-demo)
Copied to clipboard
Zhe Zhao, Yudong Li, Cheng Hou, Jing Zhao, Rong Tian, Weijie Liu, Yiren Chen, Ningyuan Sun, Haoyan Liu, Weiquan Mao, Han Guo, Weigang Gou, Taiqiang Wu, Tao Zhu, Wenhang Shi, Chen Chen, Shan Huang, Sihong Chen, Liqun Liu, Feifei Li, Xiaoshuai Chen, Xingwu Sun, Zhanhui Kang, Xiaoyong Du, Linlin Shen, Kimmo Yan
| Challenge: | Several pre-training models of different modalities are showing a rising trend of homogeneity in their model structures. |
| Approach: | They propose a toolkit that supports pre-training models of different modalities. |
| Outcome: | The proposed toolkit can match the performance of the original implementations on text, vision, and audio benchmarks. |
A Simple and Effective Method to Improve Zero-Shot Cross-Lingual Transfer Learning (2022.coling-1)
Copied to clipboard
Kunbo Ding, Weijie Liu, Yuejian Fang, Weiquan Mao, Zhe Zhao, Tao Zhu, Haoyan Liu, Rong Tian, Yiren Chen
| Challenge: | Existing zero-shot cross-lingual transfer methods rely on parallel corpora or bilingual dictionaries . however, its effect is limited by the gap between embedding clusters of different languages . |
| Approach: | They propose Embedding-Push, Attention-Pull, and Robust targets to transfer English embeddings to virtual multilingual embedders without semantic loss. |
| Outcome: | Experimental results show that the proposed method outperforms existing methods on cross-lingual tasks and can achieve a better multilingual alignment. |
InfiMM: Advancing Multimodal Understanding with an Open-Sourced Visual Language Model (2024.findings-acl)
Copied to clipboard
Haogeng Liu, Quanzeng You, Yiqi Wang, Xiaotian Han, Bohan Zhai, Yongfei Liu, Wentao Chen, Yiren Jian, Yunzhe Tao, Jianbo Yuan, Ran He, Hongxia Yang
| Challenge: | InfiMM is a multimodal large language model that adapts to complex vision-language tasks. |
| Approach: | They present a Multimodal Large Language Model that adapts to intricate vision-language tasks using large-scale training data and comprehensive training strategies. |
| Outcome: | Empirical evaluations across a variety of benchmarks underscore InfiMM’s remarkable capability in multimodal understanding. |
Parameter-efficient Continual Learning Framework in Industrial Real-time Text Classification System (2022.naacl-industry)
Copied to clipboard
Tao Zhu, Zhe Zhao, Weijie Liu, Jiachi Liu, Yiren Chen, Weiquan Mao, Haoyan Liu, Kunbo Ding, Yudong Li, Xuefeng Yang
| Challenge: | Existing continual learning methods use data replay, parameter isolation and regularization to mitigate catastrophic forgetting. |
| Approach: | They propose a parameter-efficient continual learning framework that updates parameters offline and then trains using an online regularization method. |
| Outcome: | The proposed framework reduces catastrophic forgetting and saves the model with the changed parameters instead of all parameters. |