| Challenge: | Existing work on lexical relations based on distributed representations has differed widely. |
| Approach: | They propose a model that generates taxonomy paths for hypernym prediction using WordNet sequences. |
| Outcome: | The hypo2path model outperforms the best model by 4.11 points in hit-at-one (H@1) The proposed model outpersforms previous models by a factor of 0.9. |
Similar Papers
Generating Knowledge Graph Paths from Textual Definitions using Sequence-to-Sequence Models (N19-1)
Copied to clipboard
| Challenge: | a novel method for mapping unrestricted text to knowledge graph entities is proposed . a proof-of-concept experiment has encouraging results comparable to those of state-of the-art systems. |
| Approach: | They propose a method for mapping unrestricted text to knowledge graph entities by framing the task as a sequence-to-sequence problem. |
| Outcome: | The proposed method produces highly interpretable predictions comparable to state-of-the-art systems. |
Better Language Model with Hypernym Class Prediction (2022.acl-long)
Copied to clipboard
| Challenge: | Class-based language models (LMs) have been devised to address context sparsity in n-gram LMs for decades. |
| Approach: | They propose to use class-based prediction to improve generalization for rare words by annealing from predicting the class to token prediction during training. |
| Outcome: | The proposed model improves perplexity without sacrificing performance on rare words. |
An Empirical Exploration of Local Ordering Pre-training for Structured Prediction (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have shown that pre-training contextualized encoders with language model objectives is effective for structured prediction. |
| Approach: | They propose a semi-supervised method for pre-training contextualized encoders with language model objectives. |
| Outcome: | The proposed method is effective on three typical structured prediction tasks in four languages. |
Hypernym Discovery via a Recurrent Mapping Model (2021.findings-acl)
Copied to clipboard
| Challenge: | Empirical studies on SemEval-2018 Task 9 confirm the effectiveness of the presented model. |
| Approach: | They propose a parallel style model that maps query words to their hypernyms . they use a lexical-semantic relation to name a specific instance or subtype hyponym . |
| Outcome: | Empirical results on SemEval-2018 Task 9 confirm the effectiveness of the proposed model. |
HyperBox: A Supervised Approach for Hypernym Discovery using Box Embeddings (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing methods for hypernym detection rely on word distribution. |
| Approach: | They propose a model to learn box embeddings for hypernym discovery by using a dataset . they compare the performance of their model on medical and music domains . |
| Outcome: | The proposed model outperforms existing methods on most evaluation metrics on medical and music domains. |
TEMP: Taxonomy Expansion with Dynamic Margin Loss through Taxonomy-Paths (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing taxonomies are unable to maintain coverage due to the rising of new concepts . TEMP uses pre-trained contextual encoders to predict the position of new ideas . |
| Approach: | They propose a self-supervised taxonomy expansion method that ranks taxonomies by ranking them . they use pre-trained contextual encoders to train the model with dynamic margin loss . |
| Outcome: | The proposed method outperforms state-of-the-art taxonomy expansion methods by 14.3% and 15.8% on public benchmarks. |
Lexical Entailment with Hierarchy Representations by Deep Metric Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing lexical entailment studies cannot be applied to words that are not included in the training dataset. |
| Approach: | They propose a method that learns a mapping from word embeddings to hierarchical embedds to predict hypernymy relations among words. |
| Outcome: | The proposed method achieves state-of-the-art performance and robustness for unknown words. |
Can You Tell Me How to Get Past Sesame Street? Sentence-Level Pretraining Beyond Language Modeling (P19-1)
Copied to clipboard
Alex Wang, Jan Hula, Patrick Xia, Raghavendra Pappagari, R. Thomas McCoy, Roma Patel, Najoung Kim, Ian Tenney, Yinghui Huang, Katherin Yu, Shuning Jin, Berlin Chen, Benjamin Van Durme, Edouard Grave, Ellie Pavlick, Samuel R. Bowman
| Challenge: | State-of-the-art models in natural language processing (NLP) often incorporate sentence encoder functions which generate a sequence of vectors intended to represent the in-context meaning of each word in an input text. |
| Approach: | They conduct the first large-scale systematic study of candidate pretraining tasks, comparing 19 different tasks as alternatives and complements to language modeling. |
| Outcome: | The proposed model can be used to train sentences on language modeling tasks. |
How to represent a word and predict it, too: Improving tied architectures for language modelling (D18-1)
Copied to clipboard
| Challenge: | Recent state-of-the-art models use word embeddings as input and output mappings instead of tied models. |
| Approach: | They propose to decouple hidden state from word embedding prediction . they extend their proposed modification to word2vec models . |
| Outcome: | The proposed architectures achieve comparable or better results compared to previous models without tying . the proposed architecture reduces parameters, enabling more compact models and faster learning. |
WordNet Is All You Need: A Surprisingly Effective Unsupervised Method for Graded Lexical Entailment (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a simple unsupervised method for predicting graded lexical entailment in English relies on WordNet . despite its simplicity, our method outperforms all previous methods using WordNet as weak supervision. |
| Approach: | They propose an unsupervised method which relies exclusively on WordNet for predicting graded lexical entailment in English. |
| Outcome: | The proposed method outperforms existing methods on the largest GLE dataset using WordNet. |