Papers by Ziyun Zhang
InfiniteWeb: Scalable Web Environment Synthesis for GUI Agent Training (2026.acl-long)
Copied to clipboard
| Challenge: | Existing GUI agent benchmarks are manually constructed and lack scale and diversity as training environments. |
| Approach: | They propose a GUI agent training system that automatically generates web environments at scale. |
| Outcome: | The proposed system outperforms commercial GUI agents at realistic website construction and improves on OSWorld and Online-Mind2Web. |
From Off-Policy to On-Policy: Enhancing GUI Agents via Bi-level Expert-to-Policy Assimilation (2026.acl-long)
Copied to clipboard
| Challenge: | Vision-language models are increasingly deployed as computer-use agents that operate desktops and browsers. |
| Approach: | They propose a method that turns static expert traces into policy-aligned guidance . they propose RLVR with a per-task, dynamically updated cache to decompose planning and execution . |
| Outcome: | The proposed model improves UITARS1.5-7B success from 22.87% to 32.13% on OSWorld-Verified and raises a held-out split from 5.74% to 10.30% on MMBench-GUI and Online-Mind2Web. |
Optimizing Entity Resolution in Voice Interfaces: An ASR-Aware Entity Reference Expansion Approach (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Automatic Speech Recognition (ASR) errors in voice-based dialog systems pose significant impediments to downstream tasks. |
| Approach: | They propose an automatic speech recognition (ASR) error-aware loss function to inject failed mentions and resolved entity names into the knowledge graph to enhance its awareness of unresolved mentions. |
| Outcome: | The proposed system enhances the knowledge graph's awareness of unresolved mentions by injecting pairs of failed mentions and resolved entities into the knowledge map. |
LaMPE: Length-aware Multi-grained Positional Encoding for Adaptive Long-context Scaling Without Training (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) experience significant performance degradation when the input exceeds the pretraining context window due to the out-of-distribution (OOD) behavior of Rotary Position Embedding (RoPE). |
| Approach: | They propose a training-free method that remaps out-of-distribution (OOD) positions into the in-distance range with fixed mapping strategies, ignoring the dynamic relationship between input length and effective context window. |
| Outcome: | Experiments on three representative LLMs across five mainstream long-context benchmarks show that the proposed method achieves significant performance improvements compared to existing methods. |
UI-E2I-Synth: Advancing GUI Grounding with Large-Scale Instruction Synthesis (2025.findings-acl)
Copied to clipboard
| Challenge: | Graphical User Interface (GUI) agents that utilize human-like vision perception capabilities are gaining a wider applicability compared to GUI metadata-based approaches. |
| Approach: | They propose a large-scale data synthesis pipeline for generating varying complex instruction datasets using GPT-4o instead of human annotators. |
| Outcome: | The proposed model achieves superior performance in GUI instruction grounding, demonstrating the advancements of proposed data synthesis pipeline. |
EthicMind: A Risk-Aware Framework for Ethical-Emotional Alignment in Multi-Turn Dialogue (2026.acl-long)
Copied to clipboard
| Challenge: | Existing dialogue models address empathy and ethical safety in isolation . Existing models fail to adapt their behavior as ethical risk and user emotion evolve . |
| Approach: | They propose a risk-aware framework that integrates ethical-emotional alignment in dialogue as an explicit turn-level decision problem. |
| Outcome: | The proposed framework achieves more consistent ethical guidance and emotional engagement than baselines in ethically complex interactions. |