Papers by Deming Ye
Coreferential Reasoning Learning for Language Representation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing language representation models cannot explicitly handle coreference, which is essential to the coherent understanding of the whole discourse. |
| Approach: | They propose a language representation model that captures coreferential relations in context. |
| Outcome: | The proposed model can achieve significant improvements on downstream NLP tasks while maintaining comparable performance to baseline models on other common NLP task. |
Packed Levitated Marker for Entity and Relation Extraction (2022.acl-long)
Copied to clipboard
| Challenge: | Existing work on entity and relation extraction ignores the interrelation between spans . a novel approach to extract better span representations from pre-trained languages is needed . |
| Approach: | They propose a span representation approach that packs Levitated Markers to consider interrelation between spans. |
| Outcome: | The proposed model improves on baselines on six NER benchmarks and achieves a 4.1%-4.3% strict relation F1 improvement with higher speed over previous state-of-the-art models. |
TR-BERT: Dynamic Token Reduction for Accelerating BERT Inference (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing pre-trained language models (PLMs) are expensive in inference, making them impractical in resource-limited real-world applications. |
| Approach: | They propose a dynamic token reduction approach to accelerate PLMs' inference by adapting the layer number of each token to avoid redundant calculation. |
| Outcome: | The proposed approach speeds up BERT by 2-5 times and improves performance in long-text tasks with less computation. |
OpenNRE: An Open and Extensible Toolkit for Neural Relation Extraction (D19-3)
Copied to clipboard
| Challenge: | OpenNRE provides a framework to implement neural relation extraction (RE) . the toolkit provides various functional modules based on TensorFlow and PyTorch . |
| Approach: | OpenNRE is an open-source framework to implement neural relation extraction models. they also release an online system to meet real-time extraction without any training and deployment. |
| Outcome: | OpenNRE provides a framework to implement neural models for relation extraction (RE) the toolkit also includes an online system to meet real-time extraction without training and deployment . |
Plug-and-Play Knowledge Injection for Pre-trained Language Models (2023.acl-long)
Copied to clipboard
Zhengyan Zhang, Zhiyuan Zeng, Yankai Lin, Huadong Wang, Deming Ye, Chaojun Xiao, Xu Han, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou
| Challenge: | Existing knowledge injection methods are not suitable for enhancing pre-trained language models with external knowledge bases. |
| Approach: | They propose a plug-and-play knowledge injection method where knowledge bases are injected into frozen existing downstream models by a knowledge plugin. |
| Outcome: | The proposed method improves the performance of knowledge injection on knowledge-driven tasks while keeping model parameters frozen. |
A Simple but Effective Pluggable Entity Lookup Table for Pre-trained Language Models (2022.acl-short)
Copied to clipboard
| Challenge: | Existing pre-trained language models cannot recall factual knowledge of entities exhibited in large-scale corpora, especially those rare entities. |
| Approach: | They propose to build a pluggable Entity Lookup Table (PELT) on demand by aggregating the entity’s output representations of multiple occurrences in the corpora. |
| Outcome: | The proposed model can transfer entity knowledge from out-of-domain corpora into PLMs with different architectures. |
DocRED: A Large-Scale Document-Level Relation Extraction Dataset (P19-1)
Copied to clipboard
Yuan Yao, Deming Ye, Peng Li, Xu Han, Yankai Lin, Zhenghao Liu, Zhiyuan Liu, Lixin Huang, Jie Zhou, Maosong Sun
| Challenge: | Existing relation extraction methods focus on extracting intra-sentence relations for single entities. |
| Approach: | They propose a relation extraction dataset from Wikipedia and Wikidata with three features . document-level relation extraction is a task to identify relational facts between entities . |
| Outcome: | The proposed dataset is the largest human-annotated dataset for document-level RE from plain text. |