Papers by Tushar Kataria
TRANSIENTTABLES: Evaluating LLMs’ Reasoning on Temporally Evolving Semi-structured Tables (2025.naacl-long)
Copied to clipboard
| Challenge: | a recent study shows that large language models are limited in their ability to reason over time due to static datasets. |
| Approach: | They present a dataset that includes 3,971 questions derived from over 14,000 tables . they introduce a template-based question-generation pipeline that harnesses LLMs to refine questions . |
| Outcome: | The proposed model improves on the TRANSIENTTABLES dataset . it demonstrates that the model can reason over time, even when it is not static . |
Evaluating Concurrent Robustness of Language Models Across Diverse Challenge Sets (2024.emnlp-main)
Copied to clipboard
| Challenge: | Language models display sensitivity to input perturbations, causing concerns about trust among users. |
| Approach: | They propose a methodology to examine how input perturbations affect language models across various scales, including pre-trained models and large language models. |
| Outcome: | The proposed methods enhance the model’s robustness to input perturbations and if exposure to one perturbation enhances or diminishes its performance with respect to other perturbations. |
NTSEBENCH: Cognitive Reasoning Benchmark for Vision Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent advances in large language models have demonstrated their strong performance on IQ test questions, achieving high scores across many languages. |
| Approach: | They propose a dataset to evaluate cognitive multimodal reasoning and problem-solving skills of large models. |
| Outcome: | The proposed dataset contains 2,728 multiple-choice questions and 4,642 images spanning 26 categories. |
InfoSync: Information Synchronization across Multilingual Semi-structured Tables (2023.findings-acl)
Copied to clipboard
| Challenge: | Information Synchronization of semi-structured data across languages is challenging . culture differences, topic preferences, and editing inconsistency lead to information mismatches . |
| Approach: | They propose a method for tabular synchronization that uses information from Wikipedia tables in one language with tables in another language. |
| Outcome: | The proposed method achieves an acceptance rate of 77.28% on Wikipedia . english articles across the web are more timely updated than other languages . |
Leveraging LLM For Synchronizing Information Across Multilingual Tables (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent research has sought to improve cross-language synchronization of Wikipedia tables using rule-based methods, but they struggle with complexity and generalization. |
| Approach: | They propose to use a dataset to simulate the process of updating outdated Wikipedia tables and introduce a task decomposition strategy that enhances coherence and accuracy. |
| Outcome: | The proposed model outperforms baselines in Information Updation (1.79%) and Information Addition (20.58%), highlighting its strength in dynamically updating and enriching data across architectures. |