Generalization or Memorization? Multi-Agent vs. Baseline LLMs and AutoML Models for Tabular Classification (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly used for structured tabular data. |
| Approach: | They evaluate a representative modular Multi-Agent LLM framework against state-of-the-art AutoML systems and established baselines. |
| Outcome: | The proposed model outperforms AutoML on pre-cutoff and post-cut off datasets. |
Similar Papers
Confronting LLMs with Traditional ML: Rethinking the Fairness of Large Language Models in Tabular Classifications (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent studies suggest using large language models to make tabular classifications . however, LLMs have been shown to exhibit harmful social biases based on stereotypes and inequalities present in society. |
| Approach: | They propose to use large language models to make tabular classifications . they show that LLMs inherit biases from their training data . |
| Outcome: | The proposed models exhibit harmful biases that reflect stereotypes and inequalities in society. |
ALPACA AGAINST VICUNA: Using LLMs to Uncover Memorization of LLMs (2025.naacl-long)
Copied to clipboard
Aly M. Kassem, Omar Mahmoud, Niloofar Mireshghallah, Hyunwoo Kim, Yulia Tsvetkov, Yejin Choi, Sherif Saad, Santu Rana
| Challenge: | Existing studies have shown that pre-trained LLMs emit training data up to 150 more often than in regular operation. |
| Approach: | They propose a black-box prompt optimization method where an attacker LLM agent uncovers higher levels of memorization in a victim agent . |
| Outcome: | The proposed method shows 23.7% more overlap with training data compared to state-of-the-art baselines. |
Rethinking Tabular Data Understanding with Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are capable of various tasks, yet their capability in interpreting and reasoning over tabular data remains an underexplored area. |
| Approach: | They propose a method for table structure normalization to improve model performance . they propose aggregation of multiple reasoning pathways to improve performance based on textual and symbolic reasoning. |
| Outcome: | The proposed method improves performance on symbolic reasoning tasks with textual reasoning slightly outperforming symbolic reasoning on tables. |
Memory-Augmented LLM-based Multi-Agent System for Automated Feature Generation on Tabular Data (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for automated feature generation rely on predefined operator libraries and do not incorporate feature semantics, limiting their ability to produce high-quality features. |
| Approach: | They propose a Memory-Augmented LLM-based Multi-Agent System (MALMAS) that decomposes the generation process into agents with distinct responsibilities. |
| Outcome: | The proposed method extracts informative features from raw tabular data without manual intervention and is crucial for accurate, generalizable machine learning. |
Multi-level Diagnosis and Evaluation for Robust Tabular Feature Engineering with Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in large language models have shown promise in feature engineering for tabular data, but reliability concerns persist due to variability in generated outputs. |
| Approach: | They propose a multi-level diagnosis and evaluation framework to assess the robustness of large language models in feature engineering across diverse domains. |
| Outcome: | The proposed framework assesses the robustness of large language models across domains. |
Beyond Instruction Optimization: Multi-Agent Error-Driven Class Description Refinement for LLM-Based Classification (2026.acl-industry)
Copied to clipboard
| Challenge: | Large Language Models have demonstrated considerable efficacy in classification tasks . however, their performance depends on two critical prompt components: Task Instructions (HOW to classify) and Class Descriptions (WHAT defines each class). |
| Approach: | They propose a multi-agent framework for iteratively refining class descriptions based on classification errors. |
| Outcome: | Empirical evaluation shows up to 20.71% accuracy improvements over static class descriptions. |
AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments (2026.acl-long)
Copied to clipboard
Zhiheng Xi, Dingwen Yang, Jiaqi Liu, Jixuan Huang, Honglin Guo, Baodai Huang, Tinggang Chen, Qi Zhang, Zhonghang Lu, Chenyu Liu, Jiajun Sun, Jiazheng Zhang, Dingwei Zhu, Xin Guo, Junzhe Wang, Zhihao Zhang, Yuming Yang, Junjie Ye, Minghe Gao, Dongrui Liu, Jiaming Ji, Guohao Li, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Existing benchmarks evaluate agents in simplified, idealized settings, relying on pre-packaged tool interfaces, overlooking critical steps, and assume inputs are clean and fully specified. |
| Approach: | They propose a framework that evaluates language agents in simplified, idealized settings . they show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 . |
| Outcome: | Experiments on 15 proprietary and open-source models show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 . |
An Efficient Retrieval-Based Method for Tabular Prediction with LLM (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for tabular prediction rely on extensive pre-training or fine-tuning of LLMs . a retrieval-based approach eliminates the need for training any modules or performing data augmentation . |
| Approach: | They propose a retrieval-based approach that utilizes the powerful capabilities of large language models in representation, comprehension, and inference. |
| Outcome: | The proposed method exhibits strong predictive performance on tabular prediction task, affirming its practicality and effectiveness. |
AgentGym: Evaluating and Training Large Language Model-based Agents across Diverse Environments (2025.acl-long)
Copied to clipboard
Zhiheng Xi, Yiwen Ding, Wenxiang Chen, Boyang Hong, Honglin Guo, Junzhe Wang, Xin Guo, Dingwen Yang, Chenyang Liao, Wei He, Songyang Gao, Lu Chen, Rui Zheng, Yicheng Zou, Tao Gui, Qi Zhang, Xipeng Qiu, Xuanjing Huang, Zuxuan Wu, Yu-Gang Jiang
| Challenge: | Large language models (LLMs) are promising foundations to build generally-capable agents . however, the community lacks a unified interactive framework that covers diverse environments for comprehensive evaluation of agents. |
| Approach: | They propose a framework that features 7 real-world scenarios, 14 environments, and 89 tasks for unified, real-time, and concurrent agent interaction. |
| Outcome: | The proposed framework features 7 real-world scenarios, 14 environments, and 89 tasks for unified, real-time, and concurrent agent interaction. |
An Improved, Strong Baseline for Pre-Trained Large Language Models as Task-Oriented Dialogue Systems (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have shown that Large Language Models perform insufficiently as TOD systems. |
| Approach: | They propose a self-checking mechanism to improve LLM performance as TOD systems. |
| Outcome: | The proposed model outperforms existing models and improves their performance. |