StructSP: Efficient Fine-tuning of Task-Oriented Dialog System by Using Structure-aware Boosting and Grammar Constraints (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing models that learn hierarchical structure information representations do not perform well on task-oriented dialog systems. |
| Approach: | They propose a hierarchical structure information representation model that reinforces the semantic awareness of a pre-trained language model by a two-step fine-tuning mechanism. |
| Outcome: | The proposed model is better than existing models at learning the contextual representations of utterances embedded within its hierarchical semantic structure and improves system performance. |
Similar Papers
Structure-aware Fine-tuning for Code Pre-trained Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing CodePTMs are mainly structure-free and structurebased, but how to fine-tune them remains a challenge. |
| Approach: | They propose a plug-and-play fine-tuning method that incorporates structural knowledge into pre-trained code models. |
| Outcome: | The proposed method can benefit CodePTMs more with limited training data. |
Span Fine-tuning for Pre-trained Language Models (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to fine-tune pre-trained language models are time-consuming and lack flexibility. |
| Approach: | They propose a span fine-tuning method which allows for a more efficient and efficient way of incorporating span-level information into pre-training. |
| Outcome: | Experiments on GLUE benchmark show that the proposed method significantly enhances the PrLM and offers more flexibility in an efficient way. |
Pass-Tuning: Towards Structure-Aware Parameter-Efficient Tuning for Code Representation Learning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Code pre-trained models have been proposed and widely applied in the domain of code intelligence. |
| Approach: | They propose a method that uses a plug-and-play graph neural network module as a tunable prefix to exploit structural information of source code. |
| Outcome: | The proposed method exploits structural information of source code and could replace full fine-tuning. |
Retrofitting Structure-aware Transformer Language Model for End Tasks (2020.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that structure-aware Transformer language model achieves improved perplexity, meanwhile inducing accurate syntactic phrases. |
| Approach: | They propose to exploit syntactic distance to encode phrasal constituency and dependency connection into Transformer language model and leverage it for structure integration. |
| Outcome: | The proposed model achieves significant improvements for both semantic- and syntactic-dependent tasks. |
Enhancing Large Language Model for Knowledge Graph Completion via Structure-Aware Alignment-Tuning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge graph completion methods ignore inconsistent representation spaces between natural language and graph structures, leading to duplicate works and time-consuming processes. |
| Approach: | They propose a framework that enhances LLMs for KGC via structure-aware alignment-tuning to align graph embeddings with the natural language space through multi-task contrastive learning. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on two KGC tasks across four benchmark datasets. |
Span-based Hierarchical Semantic Parsing for Task-Oriented Dialog (D19-1)
Copied to clipboard
| Challenge: | Existing semantic parsers score intents and slots as labels of nesting nodes, but decode a valid tree globally. |
| Approach: | They propose a span-based semantic parser for parsing compositional utterances into Task Oriented Parse (TOP) the parsers score labels of the tree nodes covering each token span independently, but decode a valid tree globally. |
| Outcome: | The proposed parser outperforms previous methods on the TOP dataset in accuracy and training speed. |
Coarse-to-Fine Decoding for Neural Semantic Parsing (P18-1)
Copied to clipboard
| Challenge: | Experimental results show that semantic parsing is more efficient than using simple decoders. |
| Approach: | They propose a structure-aware neural architecture which decomposes the semantic parsing process into two stages. |
| Outcome: | The proposed architecture consistently improves performance on four datasets characteristic of different domains and meaning representations. |
The Power of Prompt Tuning for Low-Resource Semantic Parsing (2022.acl-short)
Copied to clipboard
| Challenge: | Prompt tuning is an effective method for adapting pre-trained language models to downstream tasks. |
| Approach: | They propose to use prompt tuning for semantic parsing to map natural language utterances onto formal meaning representations. |
| Outcome: | The proposed method outperforms the fine-tuned model on low-resource splits of Overnight and TOPv2 on language representations with increasing model scale and target representations. |
HTML: Hierarchical Topology Multi-task Learning for Semantic Parsing in Knowledge Base Question Answering (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches struggle with mapping questions to precise logical forms . Existing frameworks struggle with complex mapping of questions to logical form . |
| Approach: | They propose a framework that leverages a hierarchical multi-task learning paradigm to enhance the performance of logical form generation. |
| Outcome: | The proposed framework outperforms supervised fine-tuning methods and training-free ones on large language models. |
Structure-aware Domain Knowledge Injection for Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Structure-aware Continual Pre-Training (SCPT) and Structure-Aware Supervised Fine-Tuning (SSFT) are two-stage strategies for knowledge injection and alignment that reduces the training corpus needs to 5% while achieving 100% of traditional knowledge injection performance. |
| Approach: | They propose a method to efficiently transform foundation Large Language Models into domain specialists by using two-stage strategies: Structure-aware Continual Pre-Training and Structure-Aware Supervised Fine-Tuning. |
| Outcome: | The proposed method significantly reduces the training corpus needs to a mere 5% while achieving 100% of traditional knowledge injection performance. |