Papers by Xinyu Pi
Towards Robustness of Text-to-SQL Models Against Natural and Realistic Adversarial Table Perturbation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing Text-to-SQL parsers are vulnerable to perturbations in NL questions . we propose the Adversarial Table Perturbation (ATP) as a new attacking paradigm . |
| Approach: | They propose to use the Adversarial Table Perturbation to measure robustness of Text-to-SQL parsers against adversarial perturbations. |
| Outcome: | The proposed approach outperforms baseline methods in robustness evaluations on ADVETA and can be used in future projects. |
Reasoning Like Program Executors (2022.emnlp-main)
Copied to clipboard
Xinyu Pi, Qian Liu, Bei Chen, Morteza Ziyadi, Zeqi Lin, Qiang Fu, Yan Gao, Jian-Guang Lou, Weizhu Chen
| Challenge: | Existing language models are inadequate in reasoning, according to studies . a new reasoning pre-training paradigm is based on pretraining language models with programs . |
| Approach: | They propose a reasoning pre-training paradigm that empowers language models to harvest reasoning knowledge possessed by program executors. |
| Outcome: | The proposed reasoning pre-training paradigm can boost models' reasoning skills . it can be instantiated by different kinds of program executors and run on a single database . |
Temporal Leakage in Search-Engine Date-Filtered Web Retrieval: A Retrospective Forecasting Case Study (2026.acl-short)
Copied to clipboard
| Challenge: | Search-engine date filters are widely used to enforce pre-cutoff retrieval in retrospective evaluations of search-augmented forecasters. |
| Approach: | They propose stronger retrieval safeguards or evaluation on frozen, time-stamped web snapshots to prevent post-cutoff leakage. |
| Outcome: | The proposed approach is unreliable across two major search engines, and the results are inflated. |
Do Vision-Language Models Have Internal World Models? Towards an Atomic Evaluation (2025.findings-acl)
Copied to clipboard
Qiyue Gao, Xinyu Pi, Kevin Liu, Junrong Chen, Ruolan Yang, Xinqi Huang, Xinyu Fang, Lu Sun, Gautham Kishore, Bo Ai, Stone Tao, Mengyang Liu, Jiaxi Yang, Chao-Jung Lai, Chuanyang Jin, Jiannan Xiang, Benhao Huang, Zeming Chen, David Danks, Hao Su, Tianmin Shu, Ziqiao Ma, Lianhui Qin, Zhiting Hu
| Challenge: | Recent studies have evaluated and shown limitations in specific capabilities such as visual understanding, but a systematic evaluation of VLMs’ fundamental WM abilities remains absent. |
| Approach: | They propose a framework that assesses perception and prediction to provide an atomic evaluation of VLMs as WMs. |
| Outcome: | The proposed framework assesses perception and prediction abilities on 15 latest VLMs and compares them to human-level models. |
UOUO: Uncontextualized Uncommon Objects for Measuring Knowledge Horizons of Vision Language Models (2024.emnlp-main)
Copied to clipboard
Xinyu Pi, Mingyuan Wu, Jize Jiang, Haozhen Zheng, Beitong Tian, ChengXiang Zhai, Klara Nahrstedt, Zhiting Hu
| Challenge: | Vision-Language Models (VLMs) perform on par with larger models in general domain visual grounding and question-answering benchmarks. |
| Approach: | They propose a "Uncontextualized Uncommon Objects" benchmark to evaluate their performance on common datasets. |
| Outcome: | The proposed benchmark focuses on systematically testing VLMs with both large and small parameter counts on rare and specialized objects. |