Haowei Zhang, Shengyun Si, Yilun Zhao, Lujing Xie, Zhijian Xu, Lyuhao Chen, Linyong Nan, Pengcheng Wang, Xiangru Tang, Arman Cohan
| Challenge: | Existing methods for table-to-text generation are limited and benchmarked on a limited number of datasets. |
| Approach: | They propose to use open-source tools to reproduce existing large language models for performance comparison and expedite the development of new models. |
| Outcome: | The proposed toolkit compares existing large language models on 9 table-to-text generation datasets and maintains a leaderboard to provide insights for future work. |
Similar Papers
Investigating Table-to-Text Generation Capabilities of Large Language Models in Real-World Information Seeking Scenarios (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Existing table-to-text generation techniques that transform complex tabular data into comprehensible narratives are lacking in real-world applications. |
| Approach: | They investigate the table-to-text capabilities of different LLMs using four datasets within two real-world information seeking scenarios. |
| Outcome: | The proposed models can generate table-to-text data in two real-world information seeking scenarios and perform better than existing models. |
TabGenie: A Toolkit for Table-to-Text Generation (2023.acl-demo)
Copied to clipboard
| Challenge: | TabGenie enables researchers to explore, preprocess, and analyze data-to-text generation datasets. |
| Approach: | They present TabGenie, a toolkit which enables researchers to explore, preprocess, and analyze a variety of data-to-text generation datasets. |
| Outcome: | The toolkit provides an interactive mode for debugging table-to-text generation, side-by-side comparison of generated system outputs, and easy exports for manual analysis. |
Beyond Traditional Benchmarks: Analyzing Behaviors of Open LLMs on Data-to-Text Generation (2024.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for data-to-text generation are saturated, and there is no way to test them. |
| Approach: | They propose a tool for collecting structured data from public APIs to analyze the behavior of open large language models on the task of data-to-text generation. |
| Outcome: | The proposed model can generate fluent and coherent texts in zero-shot settings from data in common formats collected with Quintd. |
Text-to-Table: A New Way of Information Extraction (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for information extraction are not well understood . text-to-table is a problem that aims to extract information from text data . |
| Approach: | They propose a new problem setting of information extraction, called text-to-table . they formalize text- to-table as a sequence-tosequence problem . |
| Outcome: | The proposed method outperforms existing methods on text-to-table tasks. |
TaKG: A New Dataset for Paragraph-level Table-to-Text Generation Enhanced with Knowledge Graphs (2022.findings-aacl)
Copied to clipboard
| Challenge: | Existing table-to-text generation benchmarks have some limitations, such as E2E and ToTTo focusing on singlesentence generation tasks. |
| Approach: | They propose a new table-to-text generation dataset called TaKG that uses a set of knowledge graphs to enhance table input. |
| Outcome: | The proposed model outperforms existing models for short-text generation tasks and shows reliable performance on long-text generated across a variety of metrics. |
ToTTo: A Controlled Table-To-Text Generation Dataset (2020.emnlp-main)
Copied to clipboard
Ankur Parikh, Xuezhi Wang, Sebastian Gehrmann, Manaal Faruqui, Bhuwan Dhingra, Diyi Yang, Dipanjan Das
| Challenge: | Existing methods for data-to-text generation often hallucinate phrases not supported by the Wikipedia table. |
| Approach: | They propose a controlled task where annotators directly revise existing Wikipedia sentences to generate a one-sentence description. |
| Outcome: | The proposed task produces a one-sentence description from a Wikipedia table and highlighted cells. |
OpenRT: An Open-source Framework for Reasoning Over Tabular Data (2023.acl-demo)
Copied to clipboard
| Challenge: | Existing table pre-training methods are benchmarked on a limited number of datasets with varying configurations, resulting in a lack of unified, standardized, fair, and comprehensive comparison between methods. |
| Approach: | They propose to use OpenRT to reproduce existing table pre-training models and develop new models quickly. |
| Outcome: | The proposed framework reproduces existing table pre-training models and compares them against four question answering, one fact checking, and one faithful text generation datasets. |
Recent Advances in Text-to-SQL: A Survey of What We Have and What We Expect (2022.coling-1)
Copied to clipboard
| Challenge: | text-to-SQL is a language processing and database-based language processing (NLP) task is to convert natural utterances into SQL queries and its practical application is to build natural language interfaces to database systems. |
| Approach: | They propose to conduct a systematic survey of text-to-SQL to examine the challenges and potential future directions. |
| Outcome: | The proposed system converts natural utterances into SQL queries and is a representative task in semantic parsing. |
Text-Tuple-Table: Towards Information Integration in Text-to-Table Generation via Global Tuple Extraction (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to condensing textual information into concise and structured tables are limited in their applicability in broader contexts. |
| Approach: | They propose a benchmark dataset for generating summary tables of competitions based on real-time commentary texts that incorporates large-scale textual information into concise and structured tables. |
| Outcome: | The proposed method exhibits strong generalization abilities, surpassing previous approaches on several other text-to-table datasets. |
WikiTableT: A Large-Scale Data-to-Text Dataset for Generating Wikipedia Article Sections (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing datasets for data-to-text generation focus on single-sentence generation or long-form generation. |
| Approach: | They create a dataset that pairs Wikipedia sections with tabular data and various metadata. |
| Outcome: | The proposed dataset can generate fluent and high quality texts but struggle with coherence and factuality. |