Papers by Jingjie Zheng
GRADA: Graph-based Reranking against Adversarial Documents Attack (2025.emnlp-main)
Copied to clipboard
Jingjie Zheng, Aryo Pradipta Gema, Giwon Hong, Xuanli He, Pasquale Minervini, Youcheng Sun, Qiongkai Xu
| Challenge: | Retrieval Augmented Generation (RAG) frameworks are susceptible to adversarial attacks that manipulate the retrieval process by introducing documents that are adversarially similar to the query. |
| Approach: | They propose a framework that integrates external retrieval modules into RAG frameworks to improve the factual accuracy of large language models. |
| Outcome: | The proposed framework reduces adversarial attacks by 80% while maintaining minimal loss in accuracy. |
Widget Captioning: Generating Natural Language Description for Mobile User Interface Elements (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing tools for examining and fixing missing captions are lacking in mobile UIs. |
| Approach: | They propose a task for automatically generating language descriptions for UI elements from multimodal input including both the image and structural representations of user interfaces. |
| Outcome: | The proposed task can generate captions from image and structural representations of UI elements. |
MUG: Interactive Multimodal Grounding on User Interfaces (2024.findings-eacl)
Copied to clipboard
| Challenge: | Prior studies modeled multimodal UI grounding in one round, but such an interaction is inherently iterative. |
| Approach: | They propose a task where a user and an agent collaborate on an interface screen . they use a dataset of 77,820 sequences of human user-agent interaction on mobile interfaces . |
| Outcome: | The proposed task improves the absolute task completion by 18% over the entire test set and 31% over the challenging split. |