KGLM: Integrating Knowledge Graph Structure in Language Models for Link Prediction (2023.starsem-1)
Copied to clipboard
| Challenge: | Knowledge graphs are incomplete in the information they represent, necessitating knowledge graph completion tasks. |
| Approach: | They propose a new entity/relation embedding layer that learns to differentiate distinctive entity and relation types, thus allowing the model to learn the structure of the knowledge graph. |
| Outcome: | The proposed language model learns to differentiate distinct entity and relation types, thus learning the structure of the knowledge graph. |
Similar Papers
PipeNet: Question Answering with Semantic Pruning over Knowledge Graphs (2024.starsem-1)
Copied to clipboard
| Challenge: | Existing approaches to utilizing explicit knowledge graphs (KGs) are limited by the number of nodes in the subgraph. |
| Approach: | They propose a grounding-pruning-reasoning pipeline to prune noisy nodes in subgraphs to improve the efficiency of graph reasoning with KG. |
| Outcome: | The proposed method reduces computation cost and memory usage while obtaining decent representation of pruned subgraphs. |
Exploring Semantics in Pretrained Language Model Attention (2024.starsem-1)
Copied to clipboard
| Challenge: | Abstract Meaning Representations (AMRs) encode the semantics of sentences in the form of graphs. |
| Approach: | They propose to use attention heads of two LMs to detect semantic relations encoded in AMRs. |
| Outcome: | The proposed models detect semantic relations without fine tuning, using both unsupervised and supervised learning techniques. |
Comparison and Combination of Sentence Embeddings Derived from Different Supervision Signals (2022.starsem-1)
Copied to clipboard
| Challenge: | Existing methods to derive sentence embeddings have not been well understood what properties are captured in the resulting sentences depending on the supervision signals. |
| Approach: | They propose to combine two types of sentence embedding methods with similar architectures and tasks to investigate their properties. |
| Outcome: | The proposed methods perform better on unsupervised and downstream tasks than the proposed methods on untrained STS tasks and probing tasks. |
Teach the Rules, Provide the Facts: Targeted Relational-knowledge Enhancement for Textual Inference (2021.starsem-1)
Copied to clipboard
| Challenge: | InferBERT is a method to enhance transformer-based inference models with relevant relational knowledge. |
| Approach: | They propose to enhance transformer-based inference models with relevant relational knowledge by injecting relevant facts at test time into the model. |
| Outcome: | The proposed method outperforms existing models on the challenge datasets while outperforming existing models. |
JSEEGraph: Joint Structured Event Extraction as Graph Parsing (2023.starsem-1)
Copied to clipboard
| Challenge: | Existing approaches model event extraction using simplified datasets or sequence-labeling-based encodings. |
| Approach: | They propose a graph-based event extraction framework that explicitly encodes entities and events in a single semantic graph. |
| Outcome: | The proposed framework can handle nested event structures and solve different IE tasks jointly. |
What’s wrong with your model? A Quantitative Analysis of Relation Classification (2024.starsem-1)
Copied to clipboard
| Challenge: | A major trend in NLP research aims at designing more sophisticated setups to improve the state-of-the-art (SOTA) on a target task. |
| Approach: | They propose an in-depth analysis suite for Relation Classification to be used for prediction tasks. |
| Outcome: | The proposed model improves over the baseline by >3 Micro-F1 . the proposed model is based on a case study and a preliminary error-guided analysis . |
Empirical Sufficiency Lower Bounds for Language Modeling with Locally-Bootstrapped Semantic Structures (2023.starsem-1)
Copied to clipboard
| Challenge: | a recent attempt at language modeling with predicted semantic structure failed to establish empirical lower bounds on what could have made the attempt successful. |
| Approach: | They propose a concise binary vector representation of semantic structure at the lexical level and evaluate how good an incremental tagger needs to be to achieve better-than-baseline performance. |
| Outcome: | The proposed model can achieve better-than-baseline performance without losing its main advantages and lower bounds on prediction quality can't be established via a single score alone. |
Speedy Gonzales: A Collection of Fast Task-Specific Models for Spanish (2024.starsem-1)
Copied to clipboard
| Challenge: | Large language models (LLMs) are a common and successful approach to language and retrieval tasks. |
| Approach: | They evaluate the available large language models in Spanish and then use knowledge distillation to refine and distill them. |
| Outcome: | The proposed models are fine-tuned and distilled on knowledge distillation and are available on huggingface.co/dccuchile. |
NeuralLog: Natural Language Inference with Joint Neural and Logical Reasoning (2021.starsem-1)
Copied to clipboard
| Challenge: | Currently, symbolic and deep learning approaches to NLI are receiving less attention. |
| Approach: | They propose a symbolic-based inference framework that integrates symbolic reasoning and semantic formalism to solve NLI tasks. |
| Outcome: | The proposed framework improves accuracy on the NLI task and on the SICK and MED datasets. |
Joint Training for Learning Cross-lingual Embeddings with Sub-word Information without Parallel Corpora (2020.starsem-1)
Copied to clipboard
| Challenge: | Existing methods for learning cross-lingual word embeddings incorporate sub-word information during training. |
| Approach: | They propose a method that incorporates sub-word information during training to learn cross-lingual word embeddings from monolingual data and a bilingual lexicon. |
| Outcome: | The proposed method improves on bilingual lexicon induction, monolingual word similarity, and document classification using low-resource languages. |