Papers by Ruidan He
On the Effectiveness of Adapter-based Tuning for Pretrained Language Model Adaptation (2021.acl-long)
Copied to clipboard
Ruidan He, Linlin Liu, Hai Ye, Qingyu Tan, Bosheng Ding, Liying Cheng, Jiawei Low, Lidong Bing, Luo Si
| Challenge: | Existing studies have shown that adapter-based tuning is more parameter-efficient than fine-tuning. |
| Approach: | They propose to add adapter modules to a pretrained language model and update the parameters of adapter module when learning on a downstream task. |
| Outcome: | The proposed method outperforms fine-tuning on low-resource and cross-lingual tasks and settings. |
Domain Generalization for Text Classification with Memory-Based Supervised Contrastive Learning (2022.coling-1)
Copied to clipboard
| Challenge: | Existing approaches to cross-domain text classification focus on one-to-one domain adaptation. |
| Approach: | They propose a framework for domain generalization that uses contrastive learning with a memory-saving queue. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on Amazon review sentiment datasets and rumour detection datasets. |
Document-Level Relation Extraction with Adaptive Focal Loss and Knowledge Distillation (2022.findings-acl)
Copied to clipboard
| Challenge: | Document-level relation extraction (DocRE) is a more challenging task than sentence-level one. |
| Approach: | They propose a semi-supervised framework for document-level relation extraction with three components . they use an axial attention module for learning the interdependency among entity-pairs . |
| Outcome: | The proposed model outperforms baseline models on two DocRE datasets and outperformed previous models on human annotated data and distantly supervised data. |
MELM: Data Augmentation with Masked Entity Language Modeling for Low-Resource NER (2022.acl-long)
Copied to clipboard
| Challenge: | Named entity recognition (NER) tasks have limited amount of labeled data . data augmentation methods suffer from token-label misalignment, which leads to unsatsifactory performance. |
| Approach: | They propose a data augmentation framework that explicitly injects NER labels into sentence context and generates high-quality augmented data with novel entities. |
| Outcome: | The proposed framework outperforms baseline methods on low-resource tasks. |
Bootstrapped Unsupervised Sentence Representation Learning (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to learn sentence representations rely on quality labeled data. |
| Approach: | They propose a Siamese Network which maximizes similarity between two augmented views of each sentence. |
| Outcome: | The proposed method outperforms state-of-the-art methods on STS and classification tasks. |
IAM: A Comprehensive and Large-Scale Dataset for Integrated Argument Mining Tasks (2022.acl-long)
Copied to clipboard
| Challenge: | Argument mining (AM) is a computational process that is used to analyze information in a debating system. |
| Approach: | They propose to use a large dataset to automate the manual process of debating . they propose to integrate claim extraction, stance classification and evidence extraction tasks . |
| Outcome: | The proposed tasks can extract claims, stances, evidence and more from a large dataset . the proposed tasks are highly efficient and can be applied to argument mining tasks . |
An Unsupervised Sentence Embedding Method by Mutual Information Maximization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Sentence BERT is inefficient for sentence-pair tasks as it needs to evaluate combinatorially many sentence pairs which is very time-consuming. |
| Approach: | They propose a lightweight extension on top of BERT and a self-supervised learning objective to derive meaningful sentence embeddings in an unsupervised manner. |
| Outcome: | The proposed method outperforms baselines on common semantic textual similarity tasks and downstream supervised tasks and achieves performance competitive with supervised methods on various tasks. |
Cross-lingual Aspect-based Sentiment Analysis with Aspect Term Code-Switching (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on Aspect-based sentiment analysis (ABSA) focus on English texts, but handling it in resource-poor languages remains a challenge. |
| Approach: | They propose an unsupervised cross-lingual transfer method for the Aspect-based sentiment analysis task . they propose an aspect code-switching mechanism to augment training data with code-linked bilingual sentences . |
| Outcome: | The proposed method preserves task-specific knowledge in the target language. |
Adaptive Semi-supervised Learning for Cross-domain Sentiment Classification (D18-1)
Copied to clipboard
| Challenge: | Existing methods for cross-domain sentiment classification are difficult and costly . domain adaptation is difficult because data in source and target domains are drawn from different distributions. |
| Approach: | They propose a semi-supervised learning approach that minimizes the distance between source and target instances in embedded feature space. |
| Outcome: | The proposed approach can improve on baseline methods in various settings. |
Exploiting Document Knowledge for Aspect-level Sentiment Classification (P18-2)
Copied to clipboard
| Challenge: | Existing public aspect-level datasets for aspect-based sentiment classification are small . existing methods for aspect level sentiment classification require annotation of all opinion targets . |
| Approach: | They propose two approaches that transfer knowledge from document-level data to improve aspect-level sentiment classification. |
| Outcome: | The proposed methods improve aspect-level sentiment classification on 4 public datasets. |
Feature Adaptation of Pre-Trained Language Models across Languages and Domains with Robust Self-Training (2020.emnlp-main)
Copied to clipboard
| Challenge: | Adapting pre-trained language models (PrLMs) to new domains has gained much attention . Adaptation of PrLMs to newdomains is important, but requires fine-tuning . |
| Approach: | They propose to use PrLMs to adapt to new domains without fine-tuning . they use class-aware feature self-distillation to learn discriminative features . |
| Outcome: | The proposed model can learn discriminative features from pre-trained language models without fine-tuning. |
Enhancing Multilingual Language Model with Massive Multilingual Knowledge Triples (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for language model pretraining use limited knowledge graph data for knowledge-intensive tasks. |
| Approach: | They propose to make better use of multilingual annotations and language agnostic properties of KG triples for pretraining LMs. |
| Outcome: | The proposed models show significant performance improvements on a wide range of knowledge-intensive cross-lingual tasks. |
Effective Attention Modeling for Aspect-Level Sentiment Classification (C18-1)
Copied to clipboard
| Challenge: | Aspect-level sentiment classification aims to determine sentiment polarity of review sentence towards opinion target . main challenge is to separate different opinion contexts for different targets . |
| Approach: | They propose a method that captures the semantic meaning of the opinion target and a model that incorporates syntactic information into the attention mechanism. |
| Outcome: | The proposed method captures the semantic meaning of the opinion target and incorporates syntactic information into the attention mechanism. |
An Interactive Multi-Task Learning Network for End-to-End Aspect-Based Sentiment Analysis (P19-1)
Copied to clipboard
| Challenge: | Aspect-based sentiment analysis produces a list of aspect terms and their corresponding sentiments for a sentence. |
| Approach: | They propose an interactive multi-task learning network which can learn multiple tasks simultaneously . they use a shared set of latent variables to iteratively pass information between tasks . |
| Outcome: | The proposed method outperforms existing methods on three benchmark datasets. |