Papers by Ben Kao
OpenGraph: Towards Open Graph Foundation Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Graph Neural Networks (GNNs) have emerged as promising techniques for encoding structural information and improving performance in tasks like link prediction and node classification. |
| Approach: | They propose a graph foundation model that generalizes to unseen graph data with different properties. |
| Outcome: | The proposed model achieves remarkable zero-shot graph learning performance across various settings. |
OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis (2025.acl-long)
Copied to clipboard
Qiushi Sun, Kanzhi Cheng, Zichen Ding, Chuanyang Jin, Yian Wang, Fangzhi Xu, Zhenyu Wu, Chengyou Jia, Liheng Chen, Zhoumianze Liu, Ben Kao, Guohao Li, Junxian He, Yu Qiao, Zhiyong Wu
| Challenge: | Graphical User Interface (GUI) agents powered by Vision-Language Models (VLMs) have demonstrated human-like computer control capability. |
| Approach: | They propose a GUI data synthesis pipeline that reverse engineers GUI trajectory construction process by executing pre-defined tasks. |
| Outcome: | The proposed GUI data synthesis pipeline overcomes the bottlenecks of previous methods that rely on pre-defined tasks and limited data diversity. |
Good for Misconceived Reasons: An Empirical Revisiting on the Need for Visual Context in Multimodal Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies report improvements when equipping models with multimodal information, but it remains unclear whether such improvements actually come from the multimodal part. |
| Approach: | They propose to extend conventional text-only translation models with multimodal information by extending them with visual input. |
| Outcome: | The proposed models replicate similar gains as recently developed multimodal-integrated systems achieved, but learn to ignore multimodal information. |
Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT (2020.acl-main)
Copied to clipboard
| Challenge: | Recent pre-trained language models achieve state-of-the-art performance for downstream NLP tasks. |
| Approach: | They propose a parameter-free probing technique for analyzing pre-trained language models . their method does not require direct supervision from probing tasks . |
| Outcome: | The proposed method improves on linguistically-uninformed baselines on pre-trained language models. |
OS-Sentinel: Towards Safety-Enhanced Mobile GUI Agents via Hybrid Validation in Realistic Workflows (2026.acl-long)
Copied to clipboard
Qiushi Sun, Mukai Li, Zhoumianze Liu, Zhihui Xie, Fangzhi Xu, Zhangyue Yin, Kanzhi Cheng, Zehao Li, Zichen Ding, Qi Liu, Zhiyong Wu, Zhuosheng Zhang, Ben Kao, Lingpeng Kong
| Challenge: | Existing methods for detecting unsafe mobile GUI agents are underexplored. |
| Approach: | They propose a mobile agent safety detection framework that integrates a formal verifier and a VLM-based contextual judge to detect system-level violations. |
| Outcome: | The proposed framework achieves 10%–30% improvements over existing approaches across multiple metrics. |
Lexical Knowledge Internalization for Neural Dialog Generation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing knowledge-grounded dialog models ignore the knowledge that resides in people's minds during a conversation. |
| Approach: | They propose to integrate lexical knowledge internally into the model's parameters instead of further conditioning them on external knowledge . they adopt contrastive learning approach and use a dictionary-based token-level lexicon retriever that requires only weak supervision. |
| Outcome: | The proposed model can relate J.K Rowling to Khalsa Aid with the knowledge retrieved from Wikipedia. |