TabXEval: Why this is a Bad Table? An eXhaustive Rubric for Table Evaluation (2025.findings-acl)
Copied to clipboard
| Challenge: | a rubric-based evaluation framework for tables often overlooks subtle structural and content-level discrepancies. |
| Approach: | They propose a rubric-based evaluation framework that integrates structural descriptors with contextual signals to enable more precise and consistent table comparison. |
| Outcome: | The proposed framework is based on a multi-domain benchmark with human annotations and perturbations. |
Similar Papers
Is This a Bad Table? A Closer Look at the Evaluation of Table Generation from Text (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing measures for table quality evaluation fail to capture the overall semantics of tables . failure to accurately assess table quality can result in including subpar content or overlooking valuable tables in documents. |
| Approach: | They propose a method that captures table semantics by breaking down a table into atomic statements and comparing them with ground truth statements. |
| Outcome: | The proposed method shows stronger correlation with human judgments of table quality across four datasets. |
TableEval: A Real-World Benchmark for Complex, Multilingual, and Multi-Structured Table Question Answering (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing TableQA benchmarks focus on simple flat tables and suffer from data leakage . current benchmarks are monolingual and fail to capture cross-lingual variability . |
| Approach: | They propose a table-based TableQA benchmark to evaluate LLMs on real-world tasks. |
| Outcome: | The proposed benchmarks show that they achieve high agreement with human judgment . the proposed framework improves on the alignment between model responses and reference answers . |
TabReX: Tabular Referenceless eXplainable Evaluation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing metrics for evaluating the quality of tables generated by large language models flatten tables into text, ignoring structure or relying on fixed references that limit generalization. |
| Approach: | They propose a reference-less framework for evaluating tabular generation via graph-based reasoning . tabReX converts source text and generated tables into canonical knowledge graphs . |
| Outcome: | The proposed framework provides a high correlation with expert rankings and stable under harder perturbations. |
LongTableBench: Benchmarking Long-Context Table Reasoning across Real-World Formats and Domains (2025.findings-emnlp)
Copied to clipboard
Liyao Li, Jiaming Tian, Hao Chen, Wentao Ye, Chao Ye, Haobo Wang, Ningtao Wang, Xing Fu, Gang Chen, Junbo Zhao
| Challenge: | Evaluating 52 LLMs reveals that only the strongest models maintain robust performance under increasing context lengths and format diversity. |
| Approach: | They propose a benchmark for evaluating long-context reasoning over semi-structured tables across diverse formats, tasks, and domains. |
| Outcome: | The proposed model outperforms compression-based approaches on tasks requiring semantic integration. |
MT-RAIG: Novel Benchmark and Evaluation Framework for Retrieval-Augmented Insight Generation over Multiple Tables (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies on table-based reasoning focus on a single gold table, not multiple tables . a persistent demand for robust table understanding systems is resulting from the complexity of table data . |
| Approach: | They propose a MT-RAIG Bench to evaluate systems on Retrieval-Augmented Insight Generation over Mulit-Tables. |
| Outcome: | The proposed framework improves human quality judgments on the generated insights. |
TabDSR: Decompose, Sanitize, and Reason for Complex Numerical Reasoning in Tabular Data (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models often underperform due to complex queries, noisy data, and limited numerical capabilities. |
| Approach: | They propose a framework that integrates seamlessly with mainstream LLMs to improve tabular reasoning. |
| Outcome: | The proposed framework outperforms existing methods in state-of-the-art analysis. |
RubricBench: Aligning Model-Generated Rubrics with Human Standards (2026.acl-long)
Copied to clipboard
Junyi Zhou, Qiyuan Zhang, Yufei Wang, Fuyuan Lyu, Yidong Ming, Can Xu, Qingfeng Sun, Kai Zheng, Peng Kang, Xue Liu, Chen Ma
| Challenge: | Existing benchmarks lack discriminative complexity and ground-truth rubric annotations required for rigorous evaluation. |
| Approach: | They propose a curated benchmark with 1,147 pairwise comparisons to assess the reliability of rubric-based evaluation. |
| Outcome: | The proposed benchmarks show that they support diverse domains, exhibit discriminative ability, provide high-quality annotations, and include human-authored rubrics. |
RealHiTBench: A Comprehensive Realistic Hierarchical Table Benchmark for Evaluating LLM-Based Table Analysis (2025.findings-acl)
Copied to clipboard
Pengzuo Wu, Yuhang Yang, Guangcheng Zhu, Chao Ye, Hong Gu, Xu Lu, Ruixuan Xiao, Bowen Bao, Yijing He, Liangyu Zha, Wentao Ye, Junbo Zhao, Haobo Wang
| Challenge: | Existing benchmarks for large language models focus on simple, flat table structures. |
| Approach: | They propose a benchmark to evaluate the performance of both Large Language Models and Multimodal LLMs across a variety of input formats for complex tabular data, including LaTeX, HTML, and PNG. |
| Outcome: | The proposed benchmark evaluates the performance of LLMs and Multimodal LLM models across a variety of input formats for complex tabular data, including LaTeX, HTML, and PNG. |
TART: An Open-Source Tool-Augmented Framework for Explainable Table-based Reasoning (2025.findings-naacl)
Copied to clipboard
| Challenge: | Current Large Language Models lack ability to understand table structures and apply precise numerical reasoning. |
| Approach: | They propose a tool-augmented reasoning framework for table-based tasks that integrates LLMs with specialized tools. |
| Outcome: | The proposed framework improves on the TOOLTAB dataset, a benchmark for LLMs in table–tool integration. |
TableFormer: Robust Transformer Modeling for Table-Text Encoding (2022.acl-long)
Copied to clipboard
| Challenge: | Existing tables models require linearization of the table structure, where row or column order is encoded as an unwanted bias. |
| Approach: | They propose a robust and structurally aware table-text encoding architecture TableFormer where tabular structural biases are incorporated completely through learnable attention biase. |
| Outcome: | The proposed architecture outperforms strong baselines on SQA, WTQ and TabFact table reasoning datasets and achieves state-of-the-art performance on SQ. |