| Challenge: | Concept prerequisite learning (CPL) is a task of building a concept graph by structuring open knowledge in prerequisite relations. |
| Approach: | They propose to use both content-based and graph-based models to build a concept graph by structuring open knowledge in prerequisite relations. |
| Outcome: | The proposed approach improves F1 scores by 10% on three public benchmarks. |
Similar Papers
Heterogeneous Graph Neural Networks for Concept Prerequisite Relation Learning in Educational Data (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to learn prerequisite relations among concepts are lacking . concepts are crucial for learning, organizing, applying and generating knowledge . |
| Approach: | They propose a concept prerequisite relation learning approach which combines concept representation and concept pairwise features to make it more practical. |
| Outcome: | The proposed method achieves state-of-the-art results on four datasets. |
Unsupervised Cross-Domain Prerequisite Chain Learning using Variational Graph Autoencoders (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods to learn prerequisite relations between concepts require annotated concept pairs during training. |
| Approach: | They propose to use an optimized variational graph autoencoder to learn prerequisite chains in unsupervised manner using an information-rich domain and an information poor domain. |
| Outcome: | The proposed model learns to transfer concept prerequisite relations from an information-rich domain (source domain) to an information poor domain (target domain) the annotated data and resources as well as the code will be made publicly available. |
R-VGAE: Relational-variational Graph Autoencoder for Unsupervised Prerequisite Chain Learning (2020.coling-main)
Copied to clipboard
| Challenge: | Concept prerequisite chain learning is an unsupervised task with no access to labeled concept pairs during training. |
| Approach: | They propose a model that uses deep learning representations to predict concept relations . they frame concept prerequisite chain learning as an unsupervised task with no labeled concept pairs . |
| Outcome: | The proposed model outperforms semi-supervised methods in terms of accuracy and F1 score. |
Distilling Relation Embeddings from Pretrained Language Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models capture a surprisingly rich amount of lexical knowledge, but it is unclear to what extent relation embeddings can be used to encode relational knowledge. |
| Approach: | They found that word vector differences capture lexical relations . relationship embeddings can be used to encode relational knowledge . |
| Outcome: | The results are highly competitive on analogy (unsupervised) and relation classification (supervised) benchmarks, even without any task-specific fine-tuning. |
Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs (P19-1)
Copied to clipboard
| Challenge: | Existing knowledge graphs (KGs) are incomplete or partial information, in the form of missing relations between entities, which gives rise to the task of knowledge base completion (also known as relation prediction). |
| Approach: | They propose to capture both entity and relation features in any given neighborhood and encapsulate relation clusters and multi-hop relations in their attention-based model. |
| Outcome: | The proposed model captures both entity and relation features in any given neighborhood and also encapsulates relation clusters and multi-hop relations. |
Orthogonal Relation Transforms with Graph Context Modeling for Knowledge Graph Embedding (2020.acl-main)
Copied to clipboard
| Challenge: | Existing knowledge graph embeddings have improved the knowledge graph link prediction task, but complex relations such as N-to-1, 1-to-N and N- to-N remain challenging to predict. |
| Approach: | They propose to extend RotatE from 2D complex domain to high dimensional space with orthogonal transforms to model relations. |
| Outcome: | The proposed method improves on N-to-1, 1-to-N and N- to-N cases while maintaining the capability for modeling symmetric/anti-symmetric, inverse and compositional relations. |
Improved Knowledge Distillation for Pre-trained Language Models via Knowledge Selection (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on knowledge distillation have shown that not all knowledge is necessary for learning a good student model. |
| Approach: | They propose an actor-critic approach to selecting appropriate knowledge to transfer during the process of knowledge distillation. |
| Outcome: | The proposed method outperforms several strong knowledge distillation baselines significantly on the GLUE datasets. |
Pretrain-KGE: Learning Knowledge Representation from Pretrained Language Models (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing knowledge graph embedding models suffer from limited knowledge representation due to sparse and noisy dataset annotations. |
| Approach: | They propose to use pretrained language models to enhance knowledge representation by leveraging world knowledge from pretrained models. |
| Outcome: | Extensive experiments show that the proposed framework can improve results over existing models. |
Understanding In-Context Learning via Supportive Pretraining Data (2023.acl-long)
Copied to clipboard
| Challenge: | In-context learning (ICL) is a form of learning that provides a handful of examples at inference time, but it is not well understood why it emerges as the model has never been specifically trained on such demonstrations. |
| Approach: | They adapt an iterative, gradient-based approach to find a small subset of pretraining data that supports ICL and compare it with random subsets of pretrain data. |
| Outcome: | The proposed method improves the model's ICL ability by 18% if it is continued on a small subset of pretraining data. |
Jointly Predicting Predicates and Arguments in Neural Semantic Role Labeling (P18-2)
Copied to clipboard
| Challenge: | Recent models that use gold predicates only use a single predicate at a time. |
| Approach: | They propose an end-to-end approach for jointly predicting all predicates, arguments spans, and the relations between them. |
| Outcome: | The proposed model can model overlapping spans across different predicates in the same output structure without gold predicate predications. |