BookSQL: A Large Scale Text-to-SQL Dataset for Accounting Domain (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing models for accounting databases that can be queried using natural language are lacking in some domains. |
| Approach: | They propose a large-scale text-to-SQL dataset for accounting and financial domains . they propose 'bookSQl' to be used to query accounting databases using natural language . |
| Outcome: | The proposed model performs poorly on the existing model, pointing towards a more focused model for this domain. |
Similar Papers
A Review of Cross-Domain Text-to-SQL Models (2020.aacl-srw)
Copied to clipboard
| Challenge: | WikiSQL and Spider are cross-domain text-to-SQl datasets that have attracted much attention from the research community. |
| Approach: | They propose to divide top models into two paradigms and evaluate their models for schema linking, pretrained word embeddings, reasoning assistance modules. |
| Outcome: | The proposed models have over 90% execution accuracy, the authors show . the proposed models are more complex and more complex than the proposed ones . |
Recent Advances in Text-to-SQL: A Survey of What We Have and What We Expect (2022.coling-1)
Copied to clipboard
| Challenge: | text-to-SQL is a language processing and database-based language processing (NLP) task is to convert natural utterances into SQL queries and its practical application is to build natural language interfaces to database systems. |
| Approach: | They propose to conduct a systematic survey of text-to-SQL to examine the challenges and potential future directions. |
| Outcome: | The proposed system converts natural utterances into SQL queries and is a representative task in semantic parsing. |
DuSQL: A Large-Scale and Pragmatic Chinese Text-to-SQL Dataset (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing text-to-SQL parsing methods mainly focus on English, but there is no labeled data available for the language . a larges-scale and pragmatic Chinese dataset is used for cross-domain text- to-Sql task . |
| Approach: | They propose a larges-scale Chinese dataset for a cross-domain text-to-SQL task . they analyze questions from several representative applications and use an effective data construction framework . |
| Outcome: | The proposed dataset contains 200 databases, 813 tables, and 23,797 question/SQL pairs. |
GraphQL Query Generation: A Large Training and Benchmarking Dataset (2024.emnlp-industry)
Copied to clipboard
Manish Kesarwani, Sambit Ghosh, Nitin Gupta, Shramona Chakraborty, Renuka Sindhgatta, Sameep Mehta, Carlos Eberhardt, Dan Debrunner
| Challenge: | GraphQL is a powerful query language for APIs, but crafting complex GraphqL queries can be challenging. |
| Approach: | a team of researchers has created a large-scale, cross-domain text-to-GraphQL query operation dataset . the dataset includes 10,940 training triples spanning 185 cross-source data stores and 957 test triples over 14 data stores. |
| Outcome: | The proposed dataset includes 10,940 training triples and 957 test triples over 14 data stores. |
Are ChatGPT and GPT-4 General-Purpose Solvers for Financial Text Analytics? A Study on Several Typical Tasks (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Recent large language models such as ChatGPT and GPT-4 have shown exceptional capabilities of generalist models . however, their applicability and effectiveness in specific domains like finance needs a better understanding . |
| Approach: | They conduct empirical studies to compare the performance of ChatGPT and GPT-4 on financial text analytical problems using eight benchmark datasets from five categories of tasks. |
| Outcome: | The proposed models outperform the state-of-the-art models on a wide range of financial text analytical tasks. |
BOOKSUM: A Collection of Datasets for Long-form Narrative Summarization (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing text summarization datasets include short-form source documents that lack long-range causal and temporal dependencies and contain strong layout and stylistic biases. |
| Approach: | They propose a dataset for long-form narrative summarization that uses human written summaries on three levels of difficulty. |
| Outcome: | The proposed dataset covers documents from the literature domain, such as novels, plays and stories, and includes highly abstractive, human written summaries on three levels of difficulty. |
Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task (D18-1)
Copied to clipboard
Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang, Dragomir Radev
| Challenge: | Existing datasets for semantic parsing are too small in terms of number of programs for training modern data-intensive models. |
| Approach: | They propose a large-scale complex and cross-domain semantic parsing task for a database . they use a dataset with 10,181 questions and 5,693 unique complex SQL queries . |
| Outcome: | The proposed task is different from previous tasks because it uses the same database and program . the best model achieves only 9.7% exact matching accuracy on a database split setting. |
Large Language Models as Financial Data Annotators: A Study on Effectiveness and Efficiency (2024.lrec-main)
Copied to clipboard
Toyin D. Aguda, Suchetha Siddagangappa, Elena Kochkina, Simerjot Kaur, Dongsheng Wang, Charese Smiley
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable performance in data annotation tasks on general domain datasets, but their effectiveness on domain specific datasets remains under-explored. |
| Approach: | They compare the annotations produced by three LLMs against expert annotators and crowdworkers. |
| Outcome: | The proposed models outperform expert crowdworkers and crowd-sourced annotators on domain specific datasets. |
Investigating Table-to-Text Generation Capabilities of Large Language Models in Real-World Information Seeking Scenarios (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Existing table-to-text generation techniques that transform complex tabular data into comprehensible narratives are lacking in real-world applications. |
| Approach: | They investigate the table-to-text capabilities of different LLMs using four datasets within two real-world information seeking scenarios. |
| Outcome: | The proposed models can generate table-to-text data in two real-world information seeking scenarios and perform better than existing models. |
NarrativeXL: a Large-scale Dataset for Long-Term Memory Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | 990,595 questions are needed to solve ultra-long-context reading comprehension problems. |
| Approach: | They propose a large-scale reading comprehension dataset using 1,500 hand-curated fiction books and a set of reading comprehension questions based on these summaries. |
| Outcome: | The proposed reading comprehension dataset is larger than the closest alternatives and has more questions than the existing models. |