What Really Matters for Table LLMs? A Meta-Evaluation of Model and Data Effects (2026.findings-eacl)
Copied to clipboard
Naihao Deng, Sheng Zhang, Henghui Zhu, Shuaichen Chang, Jiani Zhang, Alexander Hanbo Li, Chung-Wei Hang, Hideo Kobayashi, Yiqun Hu, Patrick Ng
| Challenge: | a series of paradigm shifts have come with distinct characteristics and challenges associated with table modeling. |
| Approach: | They propose to replicate four table LLMs by instruction-tuning three foundation models on four existing datasets. |
| Outcome: | The results show that base model choice plays a more dominant role than training data itself. |
Similar Papers
Rethinking Table Instruction Tuning (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have overlooked the impact of hyperparameters on table understanding abilities . authors show that smaller learning rates and fewer training instances can enhance table understanding while preserving general capabilities. |
| Approach: | They propose a hyperparameter-based instruction-tuned model for table-related tasks that improves out-of-domain table understanding ability and general capabilities. |
| Outcome: | The proposed model outperforms existing models on table-related tasks while maintaining strong out-of-domain generalization and general capabilities. |
Dissecting Human and LLM Preferences (2024.acl-long)
Copied to clipboard
| Challenge: | a recent study shows that human and Large Language Model preferences are important for model fine-tuning and evaluation. |
| Approach: | They dissect the preferences of human and 32 different Large Language Models to understand their quantitative composition. |
| Outcome: | The proposed model is compared with 32 different large language models using real-world user-model conversations. |
LLM See, LLM Do: Leveraging Active Inheritance to Target Non-Differentiable Objectives (2024.emnlp-main)
Copied to clipboard
| Challenge: | Historically, high-quality labeled data has been costly to curate due to scarcity of available data and financial cost. |
| Approach: | They characterize the impact of passive inheritance of model properties by studying how the source of synthetic data shapes models’ internal biases, calibration and preferences, and their generations’ textual attributes. |
| Outcome: | The proposed model inheritance can increase lexical diversity or reduce toxicity. |
Demystifying Instruction Mixing for Fine-tuning Large Language Models (2024.acl-srw)
Copied to clipboard
| Challenge: | Instruction tuning is effective for aligning large language models with human instructions, but the procedure to optimizing the mixing of instruction datasets is still unclear. |
| Approach: | They categorize instructions into three primary types: NLP downstream tasks, coding, and general chat. |
| Outcome: | The proposed method improves performance of large language models (LLMs) but it is difficult to combine different instruction datasets to optimize overall performance. |
Take the essence and discard the dross: A Rethinking on Data Selection for Fine-Tuning Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing studies focus on data selection but lack a clear, unified framework . variability in experimental settings complicates systematic comparisons . |
| Approach: | They propose a three-stage scheme to standardize data selection for fine-tuning large language models . they propose unified comparison approach that incorporates ratio-based efficiency and ranking-based feasibility metrics to address inconsistencies across experiments. |
| Outcome: | The proposed scheme outperforms existing methods in a dozen key studies and identifies key challenges. |
Serial Position Effects of Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Serial position effects (SPE) are well-documented cognitive biases in human behavior. |
| Approach: | They propose to use binary choices instead of multiple choices where feasible . they also suggest limiting prompt length and placing crucial information at the beginning of prompts . |
| Outcome: | The proposed framework shows that the effects are widespread across LLMs and the proposed mitigation methods are effective. |
LLMs are Biased Teachers: Evaluating LLM Bias in Personalized Education (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies have shown that relying on LLMs as information providers may hurt student learning. |
| Approach: | They introduce and apply two bias score metrics to evaluate LLMs for bias in the personalized educational setting, specifically on the models’ roles as “teachers.” |
| Outcome: | The proposed models harm student learning by perpetuating harmful stereotypes and reversing them. |
From Language Modeling to Instruction Following: Understanding the Behavior Shift in LLMs after Instruction Tuning (2024.naacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have achieved remarkable success in aligning with user intentions. |
| Approach: | They develop local and global explanation methods and a feed-forward-based method for input-output attribution to investigate the impact of instruction tuning on user intentions. |
| Outcome: | The proposed method compares explanations from pre-trained and instruction-tuned models . it empowers LLMs to recognize the instruction parts of user prompts, it encourages response generation . |
Tokenizer Choice For LLM Training: Negligible or Crucial? (2024.findings-naacl)
Copied to clipboard
Mehdi Ali, Michael Fromm, Klaudia Thellmann, Richard Rutmann, Max Lübbering, Johannes Leveling, Katrin Klug, Jan Ebert, Niclas Doll, Jasper Buschhoff, Charvi Jain, Alexander Weber, Lena Jurkschat, Hammam Abdelwahab, Chelsea John, Pedro Ortiz Suarez, Malte Ostendorff, Samuel Weinbach, Rafet Sifa, Stefan Kesselheim, Nicolas Flores-Herr
| Challenge: | Recent success of large language models has been driven by curating the training dataset composition, scaling of model architectures and advancements in pretraining objectives, leaving tokenizer influence as a blind spot. |
| Approach: | They conduct a comprehensive study on the influence of tokenizer choice on LLM downstream performance by training 24 mono- and multilingual LLMs at a 2.6B parameter scale. |
| Outcome: | The proposed model can significantly impact the model's downstream performance and training costs. |
From Parameters to Performance: A Data-Driven Study on LLM Structure and Development (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models have revolutionized a wide range of domains, driving significant advancements in both technology and real-world applications. |
| Approach: | They present a large-scale dataset encompassing diverse open-source LLM structures and their performance across multiple benchmarks. |
| Outcome: | The proposed model validates the relationship between structural configurations and performance across multiple benchmarks and further corroborates the findings using mechanistic interpretability techniques. |