| Challenge: | Existing methods based on pre-trained language models have shown superior performance over tabular tasks despite showing inherent problems such as not using the right evidence and inconsistent predictions across inputs. |
| Approach: | They utilize Pattern-Exploiting Training (PET) on pre-trained language models to strengthen tabular reasoning models’ pre-existing knowledge and reasoning abilities. |
| Outcome: | The proposed model exhibits superior understanding of knowledge facts and tabular reasoning compared to baseline models. |
Similar Papers
Generative Table Pre-training Empowers Models for Tabular Prediction (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to use table pre-training to boost tabular prediction performance remain open . a bachelor's degree earns less than 50K, and a generative LM can be used to unify tasks via one LM. |
| Approach: | They propose a method that leverages table pre-training to empower tabular prediction models. |
| Outcome: | The proposed method outperforms baseline models on 12 datasets and can be easily combined with various backbone models. |
An Efficient Retrieval-Based Method for Tabular Prediction with LLM (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for tabular prediction rely on extensive pre-training or fine-tuning of LLMs . a retrieval-based approach eliminates the need for training any modules or performing data augmentation . |
| Approach: | They propose a retrieval-based approach that utilizes the powerful capabilities of large language models in representation, comprehension, and inference. |
| Outcome: | The proposed method exhibits strong predictive performance on tabular prediction task, affirming its practicality and effectiveness. |
Is My Model Using the Right Evidence? Systematic Probes for Examining Evidence-Based Tabular Reasoning (2022.tacl-1)
Copied to clipboard
| Challenge: | Existing models that claim to reason about evidence should avoid spurious patterns . tabular inputs are well-suited for the study—they admit systematic probes . |
| Approach: | They propose to use tabular data to test whether models can reason about evidence . they show that a RoBERTa-based model fails to reason on the following counts . |
| Outcome: | The proposed model fails to reason on tabular data on the following counts . the model is over-sensitive to annotation artifacts and ignores relevant parts of the evidence . |
Improving and Simplifying Pattern Exploiting Training (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have shown that pre-trained language models can learn well when primed with only a few labeled examples. |
| Approach: | They propose a method that uses task-specific unlabeled data to provide denser supervision during fine-tuning. |
| Outcome: | The proposed approach outperforms GPT-3 on SuperGLUE without any unlabeled data. |
Right for the Right Reason: Evidence Extraction for Trustworthy Tabular Reasoning (2022.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that tabular reasoning models use spurious correlations and focus on false evidence or ignore it altogether. |
| Approach: | They propose a task where models need to extract evidence and then inference labels . they crowdsource evidence row labels and develop unsupervised evidence extraction strategies . |
| Outcome: | The proposed approach outperforms baseline models on the inference task using only the automatically extracted evidence as the premise. |
TABBIE: Pretrained Representations of Tabular Data (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing work on tabular representation-learning jointly models tables and associated text using self-supervised objective functions derived from pretrained language models such as BERT. |
| Approach: | They propose a tabular representation-learning model that integrates tabular data with a pretraining objective function that detects corrupted cells. |
| Outcome: | The proposed model understands complex table semantics and numerical trends. |
Exploiting Cloze-Questions for Few-Shot Text Classification and Natural Language Inference (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing approaches to learning from examples are limited due to the vast number of languages, domains and tasks. |
| Approach: | They propose a semi-supervised training procedure that reformulates input examples as cloze-style phrases to help language models understand a given task. |
| Outcome: | The proposed approach outperforms supervised training and strong semi-supervised approaches in low-resource settings by a large margin. |
Leveraging Data Recasting to Enhance Tabular Reasoning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to create tabular inference data are limited by human annotation and synthetic generation. |
| Approach: | They propose a framework for semi-automatically recasting tabular data to make use of both approaches. |
| Outcome: | The proposed framework can be used to build tabular NLI instances from five datasets. |
Towards Interpretable Tabular Reasoning: Enhancing LLM Reasoning on Tabular Data with Pre-Constructed Logic Graph (2026.acl-long)
Copied to clipboard
Lirong Gao, Zewei Yu, Zhongrui Yin, Qi Zhang, Yuke Zhu, Bo Zheng, Haobo Wang, Junbo Zhao, Gang Chen, Sheng Guo
| Challenge: | Tabular data is used in fields such as finance and healthcare due to its heterogeneity and complexity. |
| Approach: | They propose a Logic-Graph-Enhanced LLM Reasoning framework that integrates the strengths of tree-based models and LLMs to improve their interpretability. |
| Outcome: | The proposed framework outperforms tree-based models and state-of-the-art LLMs on tabular prediction tasks, achieving superior accuracy and interpretability. |
To Pretrain or Not to Pretrain: Examining the Benefits of Pretrainng on Resource Rich Tasks (2020.acl-main)
Copied to clipboard
| Challenge: | Existing studies on pretraining NLP models with variants of Masked Language Model (MLM) objectives have shown that the number of training samples used in the downstream task is limited. |
| Approach: | They propose to use MLM objectives to pretrain NLP models with variants of Masked Language Model (MLM) objectives to improve accuracy on downstream tasks. |
| Outcome: | The proposed model can reach a diminishing return point as the supervised data size increases significantly. |