Papers by Ruolan Yang
Incomplete Utterance Rewriting by A Two-Phase Locate-and-Fill Regime (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing models with incomplete utterances have too large search space, resulting in poor quality of rewriting results. |
| Approach: | They propose a 2-phase rewriting framework which predicts empty slots in the utterance that need to be completed and generates the part to be filled into each position. |
| Outcome: | The proposed framework achieves state-of-the-art results on several public rewriting datasets. |
Do Vision-Language Models Have Internal World Models? Towards an Atomic Evaluation (2025.findings-acl)
Copied to clipboard
Qiyue Gao, Xinyu Pi, Kevin Liu, Junrong Chen, Ruolan Yang, Xinqi Huang, Xinyu Fang, Lu Sun, Gautham Kishore, Bo Ai, Stone Tao, Mengyang Liu, Jiaxi Yang, Chao-Jung Lai, Chuanyang Jin, Jiannan Xiang, Benhao Huang, Zeming Chen, David Danks, Hao Su, Tianmin Shu, Ziqiao Ma, Lianhui Qin, Zhiting Hu
| Challenge: | Recent studies have evaluated and shown limitations in specific capabilities such as visual understanding, but a systematic evaluation of VLMs’ fundamental WM abilities remains absent. |
| Approach: | They propose a framework that assesses perception and prediction to provide an atomic evaluation of VLMs as WMs. |
| Outcome: | The proposed framework assesses perception and prediction abilities on 15 latest VLMs and compares them to human-level models. |
ChatMatch: Evaluating Chatbots by Autonomous Chat Tournaments (2022.acl-long)
Copied to clipboard
| Challenge: | Existing automated evaluation systems of chatbots rely on static chat scripts as ground truth, which is hard to obtain. |
| Approach: | They propose an interactive chatbot evaluation framework that allows chatbots to compete with each other like in a sports tournament. |
| Outcome: | The proposed framework can rank chatbots independently from their model architectures and domains . existing evaluation systems rely on static chat scripts as ground truth . |