Papers by Shuangzhi Wu
Robust Machine Reading Comprehension by Learning Soft labels (2020.coling-main)
Copied to clipboard
| Challenge: | Neural models have achieved great success on the task of machine reading comprehension, which are typically trained on hard labels. |
| Approach: | They propose a robust training method for machine reading comprehension models to address label sparseness problem by using three strategies to train models on soft labels. |
| Outcome: | The proposed method improves the baseline model performance and achieves state-of-the-art performance on NewsQA and QUOREF. |
Contrastive Learning with Prompt-derived Virtual Semantic Prototypes for Unsupervised Sentence Embedding (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies focus on instance-wise contrastive learning, attempting to construct positive pairs with textual data augmentation. |
| Approach: | They propose a novel Contrastive learning method with Prompt-derived Virtual semantic prototypes that constructs virtual semantic prototype to each instance and derives negative prototypes by using the negative form of the prompts. |
| Outcome: | The proposed method performs on semantic textual similarity, transfer, and clustering tasks compared to baselines. |
Enhancing Dialogue Summarization with Topic-Aware Global- and Local- Level Centrality (2023.eacl-main)
Copied to clipboard
| Challenge: | Experimental results show that our model outperforms strong baselines on three public dialogue summarization datasets: CSDS, MC, and SAMSUM. |
| Approach: | They propose a topic-aware global-local centrality model to help select the salient context from all sub-topics. |
| Outcome: | The proposed model outperforms baselines on three public dialogue summarization datasets: CSDS, MC, and SAMSUM. |
Emotion Classification by Jointly Learning to Lexiconize and Classify (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to identify emotions in short text are limited and lack coverage and inaccuracies when applied to informal short text. |
| Approach: | They propose a novel emotional network to jointly learn sentence emotions and construct emotion lexicons which are dynamically adapted to a given context. |
| Outcome: | The proposed model outperforms several approaches proposed in previous studies and achieves new state-of-the-art on the benchmark Twitter dataset. |
Recurrent Attention for Neural Machine Translation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent research questions the importance of dot-product self-attention in Transformer models and shows that most attention heads learn simple positional patterns. |
| Approach: | They propose a novel mechanism to replace dot-product self-attention with a recurrent atteNtion mechanism that directly learns attention weights without token-to-token interaction. |
| Outcome: | The proposed model outperforms the Transformer model on translation tasks with fewer parameters and inference time. |
Towards Modeling Role-Aware Centrality for Dialogue Summarization (2022.aacl-short)
Copied to clipboard
| Challenge: | Existing methods for dialogue summarization consider roles separately where interactions among different roles are not fully explored. |
| Approach: | They propose a novel role-aware centrality model to capture role interactions by involving role prompts to control what kind of summary to generate. |
| Outcome: | The proposed model achieves state-of-the-art on two public benchmark datasets, CSDS and MC. |
Iterative Nearest Neighbour Machine Translation for Unsupervised Domain Adaptation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for supervised domain adaptation of machine translation focus on fine-tuning, which is non-extensible. |
| Approach: | They propose to perform unsupervised domain adaptation in a non-parametric manner by using in-domain monolingual data and performing nearest neighbour inference on both forward and backward directions. |
| Outcome: | The proposed method significantly improves the in-domain translation performance and achieves state-of-the-art results among non-parametric methods. |
Improving Unsupervised Extractive Summarization with Facet-Aware Modeling (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing extractive summarization methods tend to select sentences within the same facet, which leads to facet bias. |
| Approach: | They propose a facet-aware centrality-based ranking model that gives a weight to the sentence centrality score. |
| Outcome: | The proposed method outperforms baseline models on a wide range of summarization tasks and performs comparably to other models. |
Learning Confidence for Transformer-based Neural Machine Translation (2022.acl-long)
Copied to clipboard
| Challenge: | A well-calibrated confidence estimate is not sufficient for neural machine translation (NMT) where probabilities from softmax distribution fail to describe when the model is probably mistaken. |
| Approach: | They propose an unsupervised confidence estimate learning jointly with the training of a neural machine translation model to quantify confidence. |
| Outcome: | The proposed model outperforms standard label smoothing and can predict failures in two real-world scenarios. |
Modeling Multi-Granularity Hierarchical Features for Relation Extraction (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing work on relation extraction focuses on constructing explicit structured features using knowledge graph and dependency tree. |
| Approach: | They propose a method to extract multi-granularity features based solely on the original input sentences. |
| Outcome: | The proposed method outperforms state-of-the-art models that even use external knowledge on three public benchmarks: SemEval 2010 Task 8, Tacred, and Tacred Revisited. |
DEPN: Detecting and Editing Privacy Neurons in Pretrained Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have demonstrated that pretrained language models memorize and regurgitate a significant portion of training data, including atypical data points that appear only once in the training data. |
| Approach: | They propose a method to locate and erase risky neurons in order to eliminate the impact of privacy data in the model in batches. |
| Outcome: | The proposed method eliminates the impact of privacy data in the model in batches without affecting the model's performance. |
Improving Translation Quality Estimation with Bias Mitigation (2023.acl-long)
Copied to clipboard
| Challenge: | State-of-the-art translation Quality Estimation models are biased, relying on monolingual features while ignoring the bilingual semantic alignment. |
| Approach: | They propose a method to mitigate the bias of translation quality estimation models by contrastive learning between clean and noisy sentence pairs. |
| Outcome: | The proposed method improves the estimation performance while mitigating the bias. |
Unsupervised Keyphrase Extraction by Jointly Modeling Local and Global Context (2021.emnlp-main)
Copied to clipboard
| Challenge: | Embedding based methods are widely used for unsupervised keyphrase extraction tasks. |
| Approach: | They propose a method where local and global contexts are jointly modeled. |
| Outcome: | The proposed method outperforms most models while generalizing better on input documents with different domains and length. |
Task-guided Disentangled Tuning for Pretrained Language Models (2022.findings-acl)
Copied to clipboard
| Challenge: | Pretrained language models are fine-tuned on task-specific datasets, but fail to capture task- specific patterns. |
| Approach: | They propose a method which disentangles task-relevant signals from entangled representations. |
| Outcome: | The proposed method improves generalization of representations by disentangling task-relevant signals from the entangled representations. |
Attention Calibration for Transformer in Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Attention mechanisms have been ubiquitous in neural machine translation (NMT) however, many studies doubt whether highlyattended inputs have a large impact on the model outputs. |
| Approach: | They propose to introduce a mask perturbation model that automatically evaluates each input’s contribution to the model outputs. |
| Outcome: | The proposed model is more uniform at lower layers while more concentrated on the specific inputs at higher layers. |
An Efficient Coarse-to-Fine Facet-Aware Unsupervised Summarization Framework Based on Semantic Blocks (2022.coling-1)
Copied to clipboard
| Challenge: | Existing unsupervised summarization methods fail to consider efficiency and effectiveness when the input document is extremely long. |
| Approach: | They propose an efficient Coarse-to-Fine Facet-Aware Ranking framework for unsupervised long document summarization based on the semantic block. |
| Outcome: | The proposed framework can achieve new state-of-the-art unsupervised summarization results on Gov-Report, billSum, arXiv, and PubMed. |