Papers by Qingyun Li
Few-Shot Multimodal Named Entity Recognition Based on Mutlimodal Causal Intervention Graph (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for multimodal named entity recognition are limited due to limited resources. |
| Approach: | They propose a Few-shot Multimodal Named Entity Recognition task to address these relation types by constructing a multimodal graph and a new multimodal causal intervention strategy. |
| Outcome: | The proposed model improves on two multimodal named entity recognition datasets. |
Schema-Guided Culture-Aware Complex Event Simulation with Multi-Agent Role-Play (2024.emnlp-demo)
Copied to clipboard
Sha Li, Revanth Gangi Reddy, Khanh Nguyen, Qingyun Wang, Yi Fung, Chi Han, Jiawei Han, Kartik Natarajan, Clare Voss, Heng Ji
| Challenge: | Complex news events require swift responses from government and society, authors say . relying on historical events to project the future is insufficient, they say - a simulator for complex news events is needed . |
| Approach: | They propose a controllable complex news event simulator guided by event schema and user-provided assumptions . they incorporate a geo-diverse commonsense and cultural norm-aware knowledge enhancement component . |
| Outcome: | The proposed simulator achieves higher coherence and appropriateness than existing models. |
COVID-19 Literature Knowledge Graph Construction and Drug Repurposing Report Generation (2021.naacl-demos)
Copied to clipboard
Qingyun Wang, Manling Li, Xuan Wang, Nikolaus Parulian, Guangxing Han, Jiawei Ma, Jingxuan Tu, Ying Lin, Ranran Haoran Zhang, Weili Liu, Aabhas Chauhan, Yingjun Guan, Bangzheng Li, Ruisong Li, Xiangchen Song, Yi Fung, Heng Ji, Jiawei Han, Shih-Fu Chang, James Pustejovsky, Jasmine Rah, David Liem, Ahmed ELsayed, Martha Palmer, Clare Voss, Cynthia Schneider, Boyan Onyshkevych
| Challenge: | a new framework to digest relevant biomedical knowledge is needed to combat COVID-19 . quantity of research results is a bottleneck, and false information promoted in publications . |
| Approach: | a team of researchers has developed a framework to extract multimedia knowledge elements from scientific literature to combat COVID-19. |
| Outcome: | a new framework extracts fine-grained multimedia knowledge elements from scientific literature . it provides detailed contextual sentences, subfigures, and knowledge subgraphs as evidence . the framework is based on a case study of drug repurposing . |
Towards Objective Fine-tuning: How LLMs’ Prior Knowledge Causes Potential Poor Calibration? (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have enabled powerful domain-specific applications through supervised fine-tuning. |
| Approach: | They propose a cognition-aware framework that applies targeted learning strategies according to the model’s prior knowledge to improve calibration. |
| Outcome: | The proposed framework significantly improves calibration while maintaining performance, achieving an average 57% reduction in ECE compared to standard fine-tuning in Llama3-8B. |
OS-Symphony: A Holistic Framework for Robust and Generalist Computer-Using Agents (2026.acl-long)
Copied to clipboard
Bowen Yang, Kaiming Jin, Zhenyu Wu, Zhaoyang Liu, Qiushi Sun, Zehao Li, JingJing Xie, Zhoumianze Liu, Fangzhi Xu, Kanzhi Cheng, Yian Wang, Qingyun Li, Yu Qiao, Zun Wang, Zichen Ding
| Challenge: | Vision-Language Models (VLMs) lack visual-aware tutorial retrieval and historical visual context curation and pruning. |
| Approach: | They propose a framework that integrates an orchestrator and a Reflection-Memory Agent for robust automation. |
| Outcome: | Experimental results show that OS-Symphony delivers substantial performance gains across model scales. |
Chem-FINESE: Validating Fine-Grained Few-shot Entity Extraction through Text Reconstruction (2024.findings-eacl)
Copied to clipboard
| Challenge: | Existing frameworks for fine-grained few-shot entity extraction are difficult to implement in the chemical domain due to the information overload of scientific papers. |
| Approach: | They propose a sequence-to-sequence based few-shot entity extraction approach . it uses a seq2seq entity extractor and a self-validation module to reconstruct original input sentence . |
| Outcome: | The proposed framework achieves 8.26% and 6.84% performance gains on two datasets. |
Multimedia Generative Script Learning for Task Planning (2023.findings-acl)
Copied to clipboard
| Challenge: | Goal-oriented generative script learning aims to generate subsequent steps to reach a specific goal . ability to capture historical states in visual modalities provides detailed information not covered by text . |
| Approach: | They propose a goal-oriented generative script learning task to generate subsequent steps by tracking historical states in both text and vision modalities. |
| Outcome: | The proposed task outperforms baselines in three aspects of the current task. |