Papers by Jinyuan Li
Self-Prompting Large Language Models for Zero-Shot Open-Domain QA (2024.naacl-long)
Copied to clipboard
| Challenge: | Open-Domain Question Answering (ODQA) aims to answer questions without explicitly providing specific background documents. |
| Approach: | They propose a framework to explicitly utilize the massive knowledge encoded in LLM parameters and their strong instruction understanding abilities. |
| Outcome: | The proposed framework surpasses state-of-the-art methods on three widely-used ODQA datasets and achieves comparable performance with customized fine-tuned models on full training data. |
D-Artemis: A Deliberative Cognitive Framework for Mobile GUI Multi-Agents (2026.findings-acl)
Copied to clipboard
Hongze Mi, Yibo Feng, WenJie Lu, Yuqi Wang, Jinyuan Li, Song Cao, He Cui, Tengfei Tian, Xuelin Zhang, Haotian Luo, Di Sun, Jun Fang, Hua Chai, Naiqiang Tan, Gang Pan
| Challenge: | Graphical User Interface (GUI) agents aim to automate a wide spectrum of human tasks by emulating user interaction. |
| Approach: | They propose a deliberative framework that leverages a fine-grained tip retrieval mechanism to inform its decision-making process. |
| Outcome: | The proposed framework achieves SOTA among open-source general models on AndroidWorld and ScreenSpot-V2 . it leverages a fine-grained, app-specific tip retrieval mechanism to inform its decision-making process . |
Prompting ChatGPT in MNER: Enhanced Multimodal Named Entity Recognition with Auxiliary Refined Knowledge (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to enhance textual entity prediction neglect the need for external knowledge or encounter high redundancy in the retrieved knowledge. |
| Approach: | They propose a framework that leverages ChatGPT as an implicit knowledge base and heuristically generates auxiliary knowledge for more efficient entity prediction. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on two classic datasets and exhibits a stronger robustness and generalization capability. |
Attention Consistency for LLMs Explanation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing interpretability methods face limitations such as low resolution and high computational cost. |
| Approach: | They propose a multi-layer attention consistency score to estimate the importance of input tokens in large language models. |
| Outcome: | The proposed heuristic achieves a favorable trade-off between interpretability quality and computational efficiency . |
VP-MEL: Visual Prompts Guided Multimodal Entity Linking (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for multimodal entity linking rely on mention words as retrieval cues, which limits their ability to effectively utilize information from both images and text. |
| Approach: | They propose a visual prompt-guided multimodal entity linking task for a text-image pair . they propose VPWiki to facilitate this task and a framework to capture latent information. |
| Outcome: | The proposed framework outperforms baseline methods on a VPWiki dataset. |
LLMs as Bridges: Reformulating Grounded Multimodal Named Entity Recognition (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for Grounded Multimodal Named Entity Recognition (GMNER) lack a strong correlation between image-text pairs and is ungroundable. |
| Approach: | They propose a framework that reformulates GMNER into a joint MNER-VE-VG task by leveraging large language models as a connecting bridge. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on the existing GMNER dataset and achieves absolute leads of 10.65%, 6.21%, and 8.83% in all three subtasks. |
Self-prompted Chain-of-Thought on Large Language Models for Open-domain Multi-hop Reasoning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing open-domain question-answering methods lack quality assurance . existing methods lack scalability and poor diversity, hindering LLMs' capabilities . |
| Approach: | They propose an open-domain multi-hop reasoning framework to answer multi-choice questions . they propose an adaptive sampler for in-context selection and self-prompted inference . |
| Outcome: | The proposed framework surpasses the existing SOTA methods on large-scale datasets and doubles the zero-shot performance of small-scale LLMs. |