Improving Demonstration Diversity by Human-Free Fusing for Text-to-SQL (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have explored selecting relevant demonstrations from a human-labeled demonstration pool, but these methods lack diversity and incur high labeling costs. |
| Approach: | They propose a method that iteratively fuses demonstrations to create a diverse demonstration pool based on human labeling or even from scratch with LLMs, reducing labeling costs. |
| Outcome: | The proposed method achieves an average improvement of 2.1% based on existing labeling and 5.5% from scratch on mainstream datasets. |
Similar Papers
Increasing Diversity While Maintaining Accuracy: Text Data Generation with Large Language Models and Human Interventions (2023.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) can be used to generate text data for training and evaluating other models. |
| Approach: | They propose to use logit suppression and temperature sampling to diversify text generation but at the cost of data accuracy. |
| Outcome: | The proposed approach can increase diversity but at the cost of data accuracy. |
Enhancing Text-to-SQL Capabilities of Large Language Models: A Study on Prompt Design Strategies (2023.findings-emnlp)
Copied to clipboard
Linyong Nan, Yilun Zhao, Weijin Zou, Narutatsu Ri, Jaesung Tae, Ellen Zhang, Arman Cohan, Dragomir Radev
| Challenge: | In-context learning (ICL) is a new approach to natural language processing tasks that rely on large language models to make predictions based on context . recent studies have shown that neural symbolic design is the preferred choice for question answering systems because of its limited working memory and unreliable long-term memory. |
| Approach: | They propose to extend in-context learning to question answering tasks that utilize structured knowledge sources and to explore various prompt design strategies for employing LLMs. |
| Outcome: | The proposed approach outperforms the state-of-the-art system by 2.5 points and the best fine-tuned system by 5.1 points on the Spider dataset. |
Selective Demonstrations for Cross-domain Text-to-SQL (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models with in-context learning have demonstrated impressive generalization capabilities in the cross-domain text-to-SQL task without the use of in-domain annotations. |
| Approach: | They propose a demonstration selection framework that utilizes both out-of-domain examples and synthetically generated in-domain demonstration examples to construct demonstrations. |
| Outcome: | The proposed framework outperforms baseline methods on two cross-domain text-to-SQL datasets with improvements of 1.1 and 11.8 points in execution accuracy. |
PTD-SQL: Partitioning and Targeted Drilling with LLMs in Text-to-SQL (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are powerful tools for Text-to-SQL tasks . SQL solutions have a relatively fixed pattern, allowing for categorical thinking . |
| Approach: | They propose that query group partitioning allows LLMs to focus on learning the thought processes specific to a single problem type, thus enhancing their reasoning abilities across diverse difficulty levels and problem categories. |
| Outcome: | The proposed model outperforms state-of-the-art models on the Spider and BIRD datasets. |
Strategic Demonstration Selection for Improved Fairness in LLM In-Context Learning (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies highlight the effectiveness of using in-context learning (ICL) to steer large language models in processing tabular data. |
| Approach: | They propose a method that uses clustering and evolutionary strategies to curate a representative sample set from training data. |
| Outcome: | The proposed method significantly improves fairness across various metrics, showing its efficacy in real-world scenarios. |
Cool-Fusion: Fuse Large Language Models without Training (2025.acl-long)
Copied to clipboard
| Challenge: | Cool-Fusion is a simple yet effective approach to combine two or more heterogeneous large language models . |
| Approach: | They propose a method that fuses the knowledge of two or more heterogeneous large language models to leverage complementary strengths. |
| Outcome: | The proposed method increases accuracy from three strong source LLMs on GSM8K by 17.4%. |
Improving Text Embeddings with Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for obtaining text embeddings require complex training pipelines . authors leverage proprietary LLMs to generate diverse synthetic data for text embeds based on 93 languages . |
| Approach: | They propose a method for obtaining high-quality text embeddings using only synthetic data and less than 1k training steps. |
| Outcome: | The proposed method achieves strong performance on competitive text embedding benchmarks without using any labeled data. |
DCG-SQL: Enhancing In-Context Learning for Text-to-SQL with Deep Contextual Schema Link Graph (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for Text-to-SQL show little improvement compared to random selections . Existing approaches rely on intrinsic capabilities of hyper-scaled LLMs, not useful demonstrations. |
| Approach: | They propose a novel approach to effectively retrieving demonstrations and generating SQL queries by linking a question and its database schema items. |
| Outcome: | The proposed method shows consistent improvements in performance and efficiency across hyper-scaled LLMs and small LLM. |
Solid-SQL: Enhanced Schema-linking based In-context Learning for Robust Text-to-SQL (2025.coling-main)
Copied to clipboard
| Challenge: | Existing text-to-SQL approaches have overlooked the critical aspect of system robustness. |
| Approach: | They propose a robust text-to-SQL solution that integrates with LLMs . their method achieves SOTA SQL execution accuracy levels of 82.1% and 58.9% . |
| Outcome: | The proposed solution achieves SOTA SQL execution accuracy levels of 82.1% and 58.9% on the general Spider and Bird benchmarks. |
Synthesizing Text-to-SQL Data from Weak and Strong LLMs (2024.acl-long)
Copied to clipboard
| Challenge: | a capability gap exists between open-source and closed-source large language models (LLMs) . the adoption of closed-sourced LLMs introduces concerns pertaining to openness, privacy, and substantial costs. |
| Approach: | They propose a synthetic data approach that combines strong and weak models for error information . they demonstrate the effectiveness of SENSE, a specialized text-to-SQL model . |
| Outcome: | The proposed method enhances the domain generalization of text-to-SQL models and explores the potential of error data supervision through preference learning. |