Papers by Jianhua Lu
MergeME: Model Merging Techniques for Homogeneous and Heterogeneous MoEs (2025.naacl-long)
Copied to clipboard
Yuhang Zhou, Giannis Karamanolakis, Victor Soto, Anna Rumshisky, Mayank Kulkarni, Furong Huang, Wei Ai, Jianhua Lu
| Challenge: | State-of-the-art methods for merging expert models with different architectures do not address parameter interference and require extensive fine-tuning to restore performance. |
| Approach: | They propose a method for merging experts with different architectures into a unified Mixture-of-Experts model with a goal of enhancing performance in each domain while retaining effectiveness on general tasks. |
| Outcome: | Experiments across multiple domains show that the proposed methods reduce fine-tuning costs and improve performance over state-of-the-art methods. |
SeqVAT: Virtual Adversarial Training for Semi-Supervised Sequence Labeling (2020.acl-main)
Copied to clipboard
| Challenge: | Empirical studies show that virtual adversarial training (VAT) significantly improves the sequence labeling performance over baselines under supervised and semi-supervised settings. |
| Approach: | They propose a method which naturally applies VAT to sequence labeling models with conditional random field (CRF) Empirical studies show that SeqVAT significantly improves the sequence labelling performance over baselines under supervised settings, and outperforms state-of-the-art approaches under semi-supervised settings. |
| Outcome: | Empirical results show that the proposed method outperforms state-of-the-art approaches under semi-supervised settings. |
Enhance Robustness of Sequence Labelling with Masked Adversarial Training (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Adversarial training (AT) has shown strong regularization effects on deep learning algorithms by introducing small input perturbations to improve model robustness. |
| Approach: | They propose to use adversarial training to improve robustness from contextual information in sequence labelling tasks by masking or replacing some words in the sentence. |
| Outcome: | The proposed method shows significant improvements on accuracy and robustness of sequence labelling on CoNLL 2000 and 2003 benchmarks. |
SSR: Utilizing Simplified Stance Reasoning Process for Robust Stance Detection (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods for stance detection are task-agnostic, which fail to utilize task knowledge to better discriminate between genuine and bias features. |
| Approach: | They propose to incorporate stance reasoning process as task knowledge to aid in learning genuine features without using targets. |
| Outcome: | The proposed model achieves better performance than previous task-agnostic debiasing methods on new test sets. |
Industry Scale Semi-Supervised Learning for Natural Language Understanding (2021.naacl-industry)
Copied to clipboard
| Challenge: | Obtaining human annotation is expensive and time-consuming process. |
| Approach: | They propose a semi-supervised learning pipeline which leverages millions of unlabeled examples to improve natural language understanding tasks. |
| Outcome: | The proposed pipeline can be used to improve natural language understanding tasks. |
An Empirical Analysis of Leveraging Knowledge for Low-Resource Task-Oriented Semantic Parsing (2023.findings-acl)
Copied to clipboard
Mayank Kulkarni, Aoxiao Zhong, Nicolas Guenon des mesnards, Sahar Movaghati, Mukund Sridhar, He Xie, Jianhua Lu
| Challenge: | Task-oriented semantic parsing is a new approach to represent the meaning of user requests with arbitrarily nested semantics. |
| Approach: | They propose to use knowledge-enhanced encoders to parse user requests with arbitrarily nested semantics. |
| Outcome: | The proposed model improves performance in low-resource and low-compute settings. |