DETQUS: Decomposition-Enhanced Transformers for QUery-focused Summarization

Challenge:	Query-focused tabular summarization is an emerging task in table-to-text generation . traditional transformer-based approaches face challenges due to token limitations and the complexity of reasoning over large tables.
Approach:	They propose a system that leverages tabular decomposition alongside a fine-tuned encoder-decoder model to improve summarization accuracy.
Outcome:	a new system outperforms the state-of-the-art REFACTOR model in a Query-focused tabular summarization task . the proposed system achieves a ROUGE-L score of 0.4437, outperforming the previous state- of-the art model .

Similar Papers

Overview

Challenge

Approach

Outcome

QTSumm: Query-Focused Summarization over Tabular Data (2023.emnlp-main)

Copied to clipboard

Yilun Zhao, Zhenting Qi, Linyong Nan, Boyu Mi, Yixin Liu, Weijin Zou, Simeng Han, Ruizhe Chen, Xiangru Tang, Yumo Xu, Dragomir Radev, Arman Cohan

Text Generation

Challenge:	Existing text generation systems that can provide accurate table summaries can facilitate more efficient access to relevant data insights.
Approach:	They propose a query-focused task where text generation models have to perform human-like reasoning and analysis over the given table to generate a tailored table summary.
Outcome:	The proposed method improves existing baselines on table-to-text generation and large language models by concatenating generated facts to the model input.

LMGQS: A Large-scale Dataset for Query-focused Summarization (2023.findings-emnlp)

Copied to clipboard

Ruochen Xu, Song Wang, Yang Liu, Shuohang Wang, Yichong Xu, Dan Iter, Pengcheng He, Chenguang Zhu, Michael Zeng

Summarization

Challenge:	Lack of large-scale datasets for query-focused summarization hinders model development . lack of data limits the ability of QFS models to train robust neural models .
Approach:	They propose to generate a query for each summary sentence in a generic summarization annotation using a pretrained language model.
Outcome:	The proposed model achieves state-of-the-art zero-shot and supervised performance on multiple existing QFS benchmarks.

Stepwise Extractive Summarization and Planning with Structured Transformers (2020.emnlp-main)

Copied to clipboard

Shashi Narayan, Joshua Maynez, Jakub Adamek, Daniele Pighin, Blaz Bratanic, Ryan McDonald

Language Models

Challenge:	Existing approaches to extractive summarization use transformers to learn the structure of long inputs.
Approach:	They propose encoder-centric stepwise models for extractive summarization using structured transformers – HiBERT and Extended Transformers .
Outcome:	The proposed models outperform previous models on CNN/DailyMail extractive summarization and Rotowire table-to-text generation.

TabSQLify: Enhancing Reasoning Capabilities of LLMs Through Table Decomposition (2024.naacl-long)

Copied to clipboard

Md Nahid, Davood Rafiei

Multimodality

Challenge:	Large language models struggle with large tables due to their limited input length . a novel method that decomposes tables into smaller and relevant sub-tables reduces the computational load on LLMs .
Approach:	They propose a method that leverages text-to-SQL generation to decompose tables into smaller and relevant sub-tables . the method can reduce the input context length significantly, making it more scalable and efficient .
Outcome:	The proposed method performs remarkably well on the WikiTQ benchmark and on the TabFact benchmark.

HETFORMER: Heterogeneous Transformer with Sparse Attention for Long-Text Extractive Summarization (2021.emnlp-main)

Copied to clipboard

Ye Liu, Jianguo Zhang, Yao Wan, Congying Xia, Lifang He, Philip Yu

Information Extraction & Text Mining

Challenge:	Existing methods for summarizing semantic graph structure from raw text are cumbersome and inefficient for long-text documents.
Approach:	They propose a Transformer-based pre-trained model with multi-granularity sparse attentions for long-text extractive summarization.
Outcome:	The proposed model performs state-of-the-art on single- and multi-document summarization tasks while using less memory and fewer parameters.

CLTR: An End-to-End, Transformer-Based System for Cell-Level Table Retrieval and Table Question Answering (2021.acl-demo)

Copied to clipboard

Feifei Pan, Mustafa Canim, Michael Glass, Alfio Gliozzo, Peter Fox

Natural Language Interfaces

Challenge:	Existing systems that retrieve tables based on keyword queries and table contents often result in poor quality . a growing demand for natural language questions over tables to be used for QA .
Approach:	They propose an end-to-end transformer-based table question answering system that takes natural language questions and massive table corpora as inputs to retrieve the most relevant tables.
Outcome:	The proposed system can retrieve relevant tables and locate the correct cells to answer questions.

TableFormer: Robust Transformer Modeling for Table-Text Encoding (2022.acl-long)

Copied to clipboard

Jingfeng Yang, Aditya Gupta, Shyam Upadhyay, Luheng He, Rahul Goel, Shachi Paul

Language Models

Challenge:	Existing tables models require linearization of the table structure, where row or column order is encoded as an unwanted bias.
Approach:	They propose a robust and structurally aware table-text encoding architecture TableFormer where tabular structural biases are incorporated completely through learnable attention biase.
Outcome:	The proposed architecture outperforms strong baselines on SQA, WTQ and TabFact table reasoning datasets and achieves state-of-the-art performance on SQ.

Transformer Grammars: Augmenting Transformer Language Models with Syntactic Inductive Biases at Scale (2022.tacl-1)

Copied to clipboard

Laurent Sartran, Samuel Barrett, Adhiguna Kuncoro, Miloš Stanojević, Phil Blunsom, Chris Dyer

Language Models

Challenge:	a novel class of Transformer language models that combine expressive power, scalability, and strong performance of Transformers and recursive syntactic compositions.
Approach:	They introduce Transformer Grammars, a class of Transformer language models that combine expressive power and recursive syntactic compositions.
Outcome:	The proposed model outperforms strong baselines on sentence-level language modeling perplexity and syntax-sensitive language evaluation metrics.

Controllable Text Summarization: Unraveling Challenges, Approaches, and Prospects - A Survey (2024.findings-acl)

Copied to clipboard

Ashok Urlana, Pruthwik Mishra, Tathagato Roy, Rahul Mishra

Text Generation

Challenge:	scholarly attention has turned to the development of text summarization methods that are more closely tailored and controlled to align with specific objectives and user needs.
Approach:	They formalize a controllable text summarization task and categorize controllability attributes according to their shared characteristics and objectives.
Outcome:	The proposed method is tailored to meet the specific intent and needs of users.

Learning to Prioritize: Precision-Driven Sentence Filtering for Long Text Summarization (2022.lrec-1)

Copied to clipboard

Alex Mei, Anisha Kabir, Rukmini Bapat, John Judge, Tony Sun, William Yang Wang

Summarization

Challenge:	Neural text summarization models are limited by their maximum input length, posing a challenge to summarizing longer texts comprehensively.
Approach:	They propose a pre-processing layer that removes low-quality sentences in articles to improve existing summarization models.
Outcome:	The proposed approach improves state-of-the-art summarization models on WikiHow and Reddit TIFU datasets by 3.84 and 8.57 points on the full test set and the long article subset.