Papers by Changliang Li
LightNER: A Lightweight Tuning Paradigm for Low-resource NER via Pluggable Prompting (2022.coling-1)
Copied to clipboard
Xiang Chen, Lei Li, Shumin Deng, Chuanqi Tan, Changliang Xu, Fei Huang, Luo Si, Huajun Chen, Ningyu Zhang
| Challenge: | Existing approaches for Named Entity Recognition (NER) use extensive labeled data for model training, which struggles in low-resource scenarios. |
| Approach: | They propose a lightweight tuning paradigm for low-resource NER via pluggable prompting . they construct a learnable verbalizer of entity categories without any label-specific classifiers . |
| Outcome: | The proposed model outperforms baselines and class transfer models in low-resource scenarios. |
A Self-Attentive Model with Gate Mechanism for Spoken Language Understanding (D18-1)
Copied to clipboard
| Challenge: | Spoken language understanding (SLU) involves intent determination and slot filling . existing joint learning methods only consider joint learning by sharing parameters on surface level rather than semantic level. |
| Approach: | They propose a self-attentive model to fully utilize the semantic correlation between slot and intent. |
| Outcome: | The proposed model outperforms existing methods in both intent detection and slot filling tasks on ATIS benchmarks. |
Does Multi-Encoder Help? A Case Study on Context-Aware Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | In encoder-decoder neural models, multiple encoders are used to represent contextual information in addition to the individual sentence. |
| Approach: | They propose to use multiple context encoders to encode the individual sentences in document-level neural machine translation (NMT) They propose a noisy dropout setup and a single-encoder approach to encode context sentences. |
| Outcome: | The proposed approach encodes the context and the current sentence without contexts. |
RankNAS: Efficient Neural Architecture Search by Pairwise Ranking (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods require training millions of architectures to estimate the accuracy of the search results. |
| Approach: | They propose a performance ranking method (RankNAS) that uses pairwise ranking and search space pruning to enlarge the search space. |
| Outcome: | The proposed method significantly accelerates NAS through pairwise ranking and search space pruning. |
Learning Architectures from an Extended Search Space for Language Modeling (2020.acl-main)
Copied to clipboard
Yinqiao Li, Chi Hu, Yuhao Zhang, Nuo Xu, Yufan Jiang, Tong Xiao, Jingbo Zhu, Tongran Liu, Changliang Li
| Challenge: | Neural architecture search (NAS) has advanced in recent years but most NAS systems restrict search to learning architectures of a recurrent or convolutional cell. |
| Approach: | They propose a general approach to learn both intra-cell and inter-cell architectures . they implement their approach in a differentiable architecture search system . |
| Outcome: | The proposed approach outperforms the baseline on PTB and WikiText data and shows good transferability to other systems. |
Layer-Wise Multi-View Learning for Neural Machine Translation (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to neural machine translation are limited to the topmost encoder layer’s context representation and cannot perceive the lower encoder layers. |
| Approach: | They propose a layer-wise multi-view learning approach to solve this problem by incorporating an auxiliary view into the model. |
| Outcome: | The proposed model can achieve stable results over multiple strong baselines and is agnostic to network architectures. |
Learning Deep Transformer Models for Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Neural machine translation models have advanced the previous state-of-the-art by learning mappings between sequences via neural networks and attention mechanisms. |
| Approach: | They propose to use layer normalization to pass the combination of previous layers to the next layer to improve the model. |
| Outcome: | The proposed model outperforms the shallow Transformer-Big/Base baseline model on English-German and Chinese-English tasks by 0.4-2.4 BLEU points. |