Papers by Dehong Ma
Self-Renewal Prompt Optimizing with Implicit Reasoning (2024.findings-emnlp)
Copied to clipboard
Zihan Liang, Ben Chen, Zhuoran Ran, Zihan Wang, Huangyu Dai, Yufei Ma, Dehong Gao, Xiaoyan Cai, Libin Yang
| Challenge: | Recent advances in NLP have been driven by the development of Large Language Models (LLMs). |
| Approach: | They propose a self-renewal approach to optimize LLM outputs to better align with human preferences without supervised fine-tuning. |
| Outcome: | The proposed approach improves outputs to better align with human preferences across LLMs and tasks without supervised fine-tuning. |
Joint Learning for Targeted Sentiment Analysis (D18-1)
Copied to clipboard
| Challenge: | Recent studies have attempted to perform two tasks separately, e.g., target extraction and sentiment classification. |
| Approach: | They propose a hierarchical stack bidirectional gated recurrent units (HSBi-GRU) model which allows the target label to influence their sentiment label. |
| Outcome: | The proposed model outperforms baseline models on two datasets and shows that it can learn abstract features. |
Exploring Sequence-to-Sequence Learning in Aspect Term Extraction (P19-1)
Copied to clipboard
| Challenge: | Aspect term extraction (ATE) aims at identifying all aspect terms in a sentence . sequence labeling based methods cannot make full use of overall meaning of sentence if they have dependencies between labels. |
| Approach: | They propose to formalize ATE as a sequence-to-sequence (Seq2Seque) learning task . they propose gated unit networks and position-aware attention mechanism to make it suit to ATE . |
| Outcome: | The proposed learning task is effective when labels correspond to words one by one . the proposed learning system is gated unit networks and position-aware attention mechanism . |
MoDULA: Mixture of Domain-Specific and Universal LoRA for Multi-Task Learning (2024.emnlp-main)
Copied to clipboard
Yufei Ma, Zihan Liang, Huangyu Dai, Ben Chen, Dehong Gao, Zhuoran Ran, Wang Zihan, Linbo Jin, Wen Jiang, Guannan Zhang, Xiaoyan Cai, Libin Yang
| Challenge: | Recent advances in open-source Large Language Models (LLMs) have achieved notable successes in natural language processing. |
| Approach: | They propose a Parameter Efficient Fine-Tuning paradigm for improved fine-tuning and parameter efficiency in multi-task learning. |
| Outcome: | The proposed model outperforms existing methods on multi-task learning while reducing training costs by over 80% without losing general capability. |
Do It Once: An Embarrassingly Simple Joint Matching Approach to Response Selection (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing matching models for response selection perform the independent matching (IM) approach. Existing models for matching only perform one match regardless of the number of options. |
| Approach: | They propose a joint matching approach which performs matching only once regardless of the number of options. |
| Outcome: | The proposed approach outperforms existing models and reduces training time by over half. |
PILE: Pairwise Iterative Logits Ensemble for Multi-Teacher Labeled Distillation (2022.emnlp-industry)
Copied to clipboard
Lianshang Cai, Linhao Zhang, Dehong Ma, Jun Fan, Daiting Shi, Yi Wu, Zhicong Cheng, Simiu Gu, Dawei Yin
| Challenge: | Pre-trained language models have been a key part of ranking systems . knowledge distillation is widely used to maintain high performance while keeping efficient computations. |
| Approach: | They propose an algorithm to combine knowledge from multi-teachers and label information to achieve competitive performance in offline and online experiments. |
| Outcome: | The proposed method has been deployed in a real-world commercial search system. |
Text Level Graph Neural Network for Text Classification (D19-1)
Copied to clipboard
| Challenge: | Recent researches have explored graph neural network (GNN) techniques on text classification, but they are faced with the problems of fixed corpus level graph structure which don’t support online testing and high memory consumption. |
| Approach: | They propose a graph neural network model that builds graphs for each input text with global parameters sharing instead of a single graph for the whole corpus. |
| Outcome: | The proposed model outperforms existing models on several text classification datasets even with consuming less memory. |
UEGP: Unified Expert-Guided Pre-training for Knowledge Rekindle (2024.findings-naacl)
Copied to clipboard
Yutao Mou, Kexiang Wang, Jianhe Lin, Dehong Ma, Jun Fan, Daiting Shi, Zhicong Cheng, Gu Simiu, Dawei Yin, Weiran Xu
| Challenge: | Existing paradigms for pre-training and fine-tuning have limitations . knowledge rekindle aims to break through performance upper bounds of experts without introducing additional annotated data. |
| Approach: | They propose a new paradigm for pre-training and fine-tuning that aims to re-incorporate the fine- tuned expert model into the training cycle and break through performance upper bounds of experts. |
| Outcome: | The proposed model breaks through performance upper bounds of experts without additional annotated data. |