Papers by Ayan Sar
Multi-Constraint State Tracking with Negation: A Diagnostic Benchmark for LLM World Modeling (2026.acl-srw)
Copied to clipboard
| Challenge: | Existing evaluations of Large Language Models (LLMs) lack the ability to maintain and update internal representations of an evolving world despite their ubiquity in real-world language. |
| Approach: | They propose a diagnostic benchmark for multi-constraint state tracking that evaluates an LLM's ability to maintain consistent world models across sequences of actions involving inventory changes, spatial movement, temporal ordering, and systematic negation. |
| Outcome: | The proposed model is based on 100,847 questions spanning 12 real-world domains and includes culturally diverse entity names to enable analysis of cross-cultural robustness. |