Papers by Ziqiao Ma
DANLI: Deliberative Agent for Following Natural Language Instructions (2022.emnlp-main)
Copied to clipboard
Yichi Zhang, Jianing Yang, Jiayi Pan, Shane Storks, Nikhil Devraj, Ziqiao Ma, Keunwoo Yu, Yuwei Bao, Joyce Chai
| Challenge: | Recent work on embodied AI agents that can perform tasks by following human language instructions is limited by reactive methods, which are insufficient for long-horizon complex tasks. |
| Approach: | They propose a neuro-symbolic deliberative agent that, while following language instructions, proactively applies reasoning and planning based on its neural and symbolic representations acquired from past experience. |
| Outcome: | The proposed agent achieves greater than 70% improvement over reactive baselines on the challenging TEACh benchmark. |
Training Turn-by-Turn Verifiers for Dialogue Tutoring Agents: The Curious Case of LLMs as Your Coding Tutors (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have focused on coding tutoring, but their capabilities in guiding users to solve complex tasks remain underexplored. |
| Approach: | They propose a novel agent workflow, Trace-and-Verify, which combines knowledge tracing to estimate a student’s knowledge state and turn-by-turn verification to ensure effective guidance toward task completion. |
| Outcome: | The proposed agent workflow achieves significantly higher success rates than existing tutoring agents. |
Towards A Holistic Landscape of Situated Theory of Mind in Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent inquiries reveal a lack of robust ToM in large language models . current models focus on different aspects of ToM and are prone to shortcuts and data leakage. |
| Approach: | They propose to taxonomize machine ToM into 7 mental state categories and delineate existing benchmarks to identify under-explored aspects of ToM. |
| Outcome: | The proposed model breaks ToM into individual components and treats LLMs as agents physically and socially situated in interactions with humans. |
Vision-Language Models Mistake Head Orientation for Gaze Direction: Nonverbal Conversation Cues (2026.findings-acl)
Copied to clipboard
Zory Zhang, Pinyuan Feng, Bingyang Wang, Tianwei Zhao, Suyang Yu, Qingying Gao, Hokin Deng, Ziqiao Ma, Yijiang Li, Dezhi Luo
| Challenge: | Where someone looks is a nonverbal communication cue that children and adults readily use. |
| Approach: | They used 1,360 real-world photos to construct evaluation stimuli for Vision-Language Models (VLMs) they found a substantial performance gap between VLMs and humans . |
| Outcome: | The proposed model outperforms existing models in predicting gaze direction using head orientation rather than eye appearance. |
World-to-Words: Grounded Open Vocabulary Acquisition through Fast Mapping in Vision-Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | GOVA examines grounding and bootstrapping in open-world language learning. |
| Approach: | They propose a visually-grounded language model that uses grounding as an objective . they propose GOVA to investigate grounding and bootstrapping in open-world language learning . |
| Outcome: | The proposed model is faster and faster grounded than previous models, the authors show . they show that grounding helps the model to learn unseen words more rapidly and robustly . |
From Behavioral Performance to Internal Competence: Interpreting Vision-Language Models with VLM-Lens (2025.emnlp-demos)
Copied to clipboard
Hala Sheta, Eric Haoran Huang, Shuyu Wu, Ilia Alenabi, Jiajun Hong, Ryker Lin, Ruoxi Ning, Daniel Wei, Jialin Yang, Jiawei Zhou, Ziqiao Ma, Freda Shi
| Challenge: | Existing vision-language models are based on exactmatch based accuracy and its derivations to evaluate performance. |
| Approach: | They propose a toolkit that supports systematic benchmarking, analysis, and interpretation of vision-language models by extracting intermediate outputs from any layer during the forward pass of open-source VLMs. |
| Outcome: | The proposed toolkit supports 16 state-of-the-art base VLMs and their over 30 variants and is extensible to accommodate new models without changing the core logic. |
Do Vision-Language Models Have Internal World Models? Towards an Atomic Evaluation (2025.findings-acl)
Copied to clipboard
Qiyue Gao, Xinyu Pi, Kevin Liu, Junrong Chen, Ruolan Yang, Xinqi Huang, Xinyu Fang, Lu Sun, Gautham Kishore, Bo Ai, Stone Tao, Mengyang Liu, Jiaxi Yang, Chao-Jung Lai, Chuanyang Jin, Jiannan Xiang, Benhao Huang, Zeming Chen, David Danks, Hao Su, Tianmin Shu, Ziqiao Ma, Lianhui Qin, Zhiting Hu
| Challenge: | Recent studies have evaluated and shown limitations in specific capabilities such as visual understanding, but a systematic evaluation of VLMs’ fundamental WM abilities remains absent. |
| Approach: | They propose a framework that assesses perception and prediction to provide an atomic evaluation of VLMs as WMs. |
| Outcome: | The proposed framework assesses perception and prediction abilities on 15 latest VLMs and compares them to human-level models. |
NLP Reproducibility For All: Understanding Experiences of Beginners (2023.acl-long)
Copied to clipboard
| Challenge: | a study with 93 students in an introductory NLP class shows that beginners' programming skill and comprehension of research papers have a limited impact on their effort spent completing the exercise. |
| Approach: | a study conducted with 93 students in an introductory NLP course questioned them on their programming background and programming background. |
| Outcome: | The results show that beginners' programming skill and comprehension of research papers have a limited impact on their effort spent on the exercise. |
DOROTHIE: Spoken Dialogue for Handling Unexpected Situations in Interactive Autonomous Driving Agents (2022.findings-emnlp)
Copied to clipboard
Ziqiao Ma, Benjamin VanDerPloeg, Cristian-Paul Bara, Yidong Huang, Eui-In Kim, Felix Gervits, Matthew Marge, Joyce Chai
| Challenge: | Empowering autonomous driving agents with the ability to navigate in a continuous and dynamic environment is critical. |
| Approach: | They propose a novel interactive simulation platform that enables the creation of unexpected situations on the fly to support empirical studies on situated communication with autonomous driving agents. |
| Outcome: | The proposed platform enables the creation of unexpected situations on the fly to support empirical studies on situated communication with autonomous driving agents. |
Learning Language through Grounding (2025.naacl-tutorial)
Copied to clipboard
| Challenge: | This tutorial provides a historical overview of grounding and discusses its use in computational linguistics and in computational language processing. |
| Approach: | They introduce the concept of grounding and discuss future directions and open challenges . they will delve into recent progress in learning lexical semantics, syntax, and complex meanings through various forms of ground. |
| Outcome: | This course will provide an overview of the field of grounding and discuss future directions and challenges related to large language models and scaling. |
Babysit A Language Model From Scratch: Interactive Language Learning by Trials and Demonstrations (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent advances in large language models have adopted a non-interactive training paradigm, and refined pre-trained models through feedback afterward. |
| Approach: | They propose a trial-and-demonstration learning framework that incorporates student trials, teacher demonstrations, and a reward conditioned on language competence at various developmental stages. |
| Outcome: | The proposed framework accelerates word acquisition for student models of equal and smaller numbers of parameters and a strong correlation between the frequency of words in trials and learning curves. |