Introducing Graph Context into Language Models through Parameter-Efficient Fine-Tuning for Lexical Relation Mining (2025.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models can effectively mine lexical relations between word pairs . however, graph features and semantic knowledge of pre-tried models are lacking in the task. |
| Approach: | They propose a parameter-efficient fine-tuning method which integrates graph features and semantic representations for lexical relation classification and lexic entailment tasks. |
| Outcome: | The proposed method integrates graph features and semantic representations for lexical relation mining tasks. |
Similar Papers
No clues good clues: out of context Lexical Relation Classification (2023.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models (PTLMs) are used to predict lexical relations between words. |
| Approach: | They propose to use pre-trained language models to fine-tune and exploit verbalized text for linguistically motivated tasks. |
| Outcome: | The proposed model outperforms graded Lexical Entailment and lexical relation classification with very simple prompts. |
Linguistic Cues for LLM-based Implicit Discourse Relation Classification (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have been successful in many NLP tasks, but they struggle to capture subtle lexical relations between arguments. |
| Approach: | They propose a strategy that enriches arguments with explicit lexical-level semantic cues before fine-tuning. |
| Outcome: | The proposed approach improves F1 scores in cross-domain scenarios by more than 10 points compared to baselines. |
Enhancing Lexical Relation Mining with Structured Sememe Knowledge (2026.acl-long)
Copied to clipboard
| Challenge: | Existing top-performing methods for Lexical Relation Mining rely on pre-trained language models yet fail to distinguish nuanced lexical relations. |
| Approach: | They propose a framework to leverage structured sememe knowledge to enhance LRC and LE. |
| Outcome: | The proposed method outperforms existing methods on benchmarks and outperformed the LLMs. |
KG-Adapter: Enabling Knowledge Graph Integration in Large Language Models through Parameter-Efficient Fine-Tuning (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are criticized for lack of expertise and knowledge conflict . KG-Adapter is a parameter-level KG integration method for decoder-only LLMs . |
| Approach: | They propose a parameter-level KG integration method based on parameter-efficient fine-tuning . they use KG-Adapter to integrate knowledge graphs with LLMs and perform joint reasoning . |
| Outcome: | The proposed method outperforms the current state-of-the-art method on four datasets for two different tasks. |
Probing Pretrained Language Models for Lexical Semantics (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have focused on morphosyntactic, semantic, and world knowledge, but it remains unclear to what extent LMs derive lexical type-level knowledge from words in context. |
| Approach: | They propose to use multilingual and monolingual LMs to extract lexical type-level knowledge from words in context. |
| Outcome: | The proposed models perform well across six typologically diverse languages and five lexical tasks. |
LexFit: Lexical Fine-Tuning of Pretrained Language Models (2021.acl-long)
Copied to clipboard
| Challenge: | Transformer-based language models implicitly store a wealth of lexical semantic knowledge, but it is non-trivial to extract that knowledge effectively from their parameters. |
| Approach: | They propose to expose and enrich lexical knowledge from transformer-based language models to serve as effective decontextualized word encoders even when fed input words "in isolation" |
| Outcome: | The proposed model outperforms standard static WEs and vanilla LMs in lexical tasks over four established tasks in 8 languages. |
BertNet: Harvesting Knowledge Graphs with Arbitrary Relations from Pretrained Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to construct knowledge graphs are limited to a small set of relations due to manual cost or restrictions in text corpus. |
| Approach: | They propose to automatically construct knowledge graphs (KGs) of diverse new relations from pretrained language models that accept knowledge queries with prompts. |
| Outcome: | The proposed framework extracts knowledge of over 400 new relations from pretrained language models, including RoBERTaNet, with minimal input of a relation definition and a few shot of example entity pairs. |
Language Models for Lexical Inference in Context (2021.eacl-main)
Copied to clipboard
| Challenge: | Lexical inference in context (LIiC) is a variant of the natural language inference task focused on lexical semantics. |
| Approach: | They propose three approaches based on pretrained language models for LIiC . they propose a few-shot NLI classifier and a relation induction approach based upon handcrafted patterns expressing the semantics of lexical inference. |
| Outcome: | The proposed approaches outperform the previous state of the art and show their potential for LIiC. |
Language Models and Semantic Relations: A Dual Relationship (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies on language models for the extraction of semantic relations have focused on injecting semantic knowledge into these models to enhance them. |
| Approach: | They propose to extract lexical semantic relations from a BERT model and inject them into it using unsupervised methods based on semantic similarity at word and sentence levels. |
| Outcome: | The proposed method allows to enrich a BERT model without using any external semantic resource. |
PALT: Parameter-Lite Transfer of Language Models for Knowledge Graph Completion (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained language models (LMs) are a powerful transfer learning approach for knowledge graph (KG) completion. |
| Approach: | They propose a parameter-lite transfer learning approach for pretrained language models for knowledge graph (KG) completion. |
| Outcome: | The proposed model outperforms the state-of-the-art models on a knowledge graph completion benchmark by tuning 1% of the parameters. |