Papers by Jingwei Zhang
Copied to clipboard
| Challenge: | Existing studies attribute object hallucinations to linguistic priors and data biases . MFCD method removes hallucinian distribution in the original output distribution . |
| Approach: | They propose a method that removes the hallucination distribution in the original output distribution . they propose MFCD to mitigate hallucinism in large visual-language models . |
| Outcome: | The proposed method reduces hallucination distributions without training or external tools . the proposed method can be applied to various LVLMs without modifying model architecture or training . |
Copied to clipboard
| Challenge: | Recent studies show that pre-trained vision-language models perform well in cross-modal tasks, including referring expression comprehension. |
| Approach: | They propose a method that enables VL models to reason with implicit text . they propose to use a dataset to align the text with objects in the images . |
| Outcome: | The proposed method improves performance 37.94% on referring expression comprehension task. |
Copied to clipboard
| Challenge: | Recent advances in Relation Extraction (RE) emphasize Zero-Shot methodologies, aiming to recognize unseen relations between entities with no annotated data. |
| Approach: | They propose a plug-in retrieval adjuster that allows rapid fine-tuning without accessing LLMs’ parameters. |
| Outcome: | The proposed model demonstrates comparable performance on multiple benchmarks. |
Copied to clipboard
| Challenge: | Existing methods to predict relationships with given entity pairs are lacking in supervised methods. |
| Approach: | They propose a framework for zero-shot Relation Extraction that includes two modules: Custom Embedding and Dynamic Aggregation. |
| Outcome: | The proposed framework shows competitive performance on two ZSRE datasets. |
Copied to clipboard
| Challenge: | Existing methods to mitigate hallucinations generate erroneous or fabricated information. |
| Approach: | They propose a rank-response-based model that annotates pair-reponses and trains alignment algorithms to improve the correspondence between images and text. |
| Outcome: | The proposed model outperforms the DPO method and outperfies existing methods on two MLLMs of different sizes and four widely used benchmarks. |
Copied to clipboard
| Challenge: | Existing research on multimodal metaphors does not address categorizing the source and target domains in metaphors beyond the English language. |
| Approach: | They propose a Cascading Domain Knowledge Integration benchmark to detect metaphors by introducing domain-specific lexical features. |
| Outcome: | The proposed dataset includes 13,820 text-image pairs of advertisements with manual annotations of the occurrence of metaphors, domain categories, and sentiments metaphors convey. |
Copied to clipboard
| Challenge: | Existing methods to generate financial market analysis text require extensive financial knowledge and skill of financial analysts. |
| Approach: | They propose a task to generate financial market analysis reports using financial market data using a financial knowledge graph. |
| Outcome: | The proposed framework outperforms large-scale language models and retrieval-augmented baselines in the financial market analysis generation task. |
Copied to clipboard
| Challenge: | Current methods for multi-modal entity alignment ignore relative interactions between modalities and the accuracy of weights. |
| Approach: | They propose a relative interaction and calibration framework for multi-modal entity alignment that uses attention mechanisms to perceive the uncertainty of the weight for each modality. |
| Outcome: | The proposed framework outperforms baselines across 5 datasets and 23 settings. |
Copied to clipboard
| Challenge: | Existing methods focus on interactions between neighboring entities in the structural modality while neglecting interactions between entities in visual and attribute modalities. |
| Approach: | They propose a structure-guided multimodal entity alignment method which prioritizes structural information from knowledge graphs to enhance the visual and attribute modalities. |
| Outcome: | The proposed method achieves state-of-the-art performance across multiple datasets, validating its effectiveness and superiority in practical applications. |
Copied to clipboard
| Challenge: | Current approaches to temporal knowledge representation face limited generalization to unseen facts and insufficient interpretability of reasoning processes. |
| Approach: | They propose a framework that uses a denoising diffusion process to complete reasoning tasks . they propose introducing a noise source and historical conditionguiding mechanism to improve interpretability . |
| Outcome: | The proposed framework outperforms state-of-the-art methods on three benchmark datasets. |
Copied to clipboard
| Challenge: | Entity Alignment (EA) is a critical task in Knowledge Graph (KG) integration. |
| Approach: | They propose a novel approach that leverages the data characteristics of synthetic benchmarks to improve performance in real-world datasets. |
| Outcome: | The proposed approach outperforms state-of-the-art models on real-world datasets and achieves a 29.94% improvement in Hits@1 on DOREMUS and 5.64% improvement on AGROLD. |
Copied to clipboard
| Challenge: | Multi-modal entity alignment (MMEA) aims to identify equivalent entities between two multimodal knowledge graphs. |
| Approach: | They propose a novel LLMguided MMEA framework that prioritizes noise reduction before fusion. |
| Outcome: | The proposed framework prioritizes noise reduction before fusion and improves semantics on the noisy FB YG dataset. |
Copied to clipboard
| Challenge: | Existing document-level relation extraction models focus on individual entity pairs, limiting their ability to handle complex reasoning tasks. |
| Approach: | They propose a document-level relation extraction framework based on global relations and entity pair reasoning that captures fine-grained interactions between entity pairs. |
| Outcome: | The proposed framework outperforms existing models on widely-used datasets. |
Copied to clipboard
| Challenge: | Existing MKGC methods train with all modalities available, implicitly assuming consistent complementarity . however, this often induces modality dependence and modality competition under heterogeneous noise, which can hinder robust multi-modal fusion and limit overall performance. |
| Approach: | They propose a framework to infer missing links in multimodal knowledge graphs by leveraging structured triples together with auxiliary modalities such as text and images. |
| Outcome: | The proposed framework outperforms baselines and achieves new state-of-the-art results. |
Copied to clipboard
| Challenge: | Document-level relation extraction (DocRE) aims to determine which relations hold between a given entity pair in a document. |
| Approach: | They propose a document-level relation extraction paradigm that decouples existing losses into independent positive and negative losses, which interact solely with a shared threshold. |
| Outcome: | The proposed model outperforms existing models on four datasets and achieves state-of-the-art results. |
Copied to clipboard
| Challenge: | Existing methods for document-level relation extraction ignore bidirectional mention interaction when generating relational features for entity pairs. |
| Approach: | They propose a document-level relation extraction model that incorporates bidirectional mention fusion and a simple yet effective evidence extraction module for relation prediction. |
| Outcome: | The proposed model achieves SOTA performance and the proposed method is effective and general when integrated into existing models. |
Copied to clipboard
| Challenge: | Existing approaches to merge multi-modal knowledge only use one fusion strategy . however, the impact of the fusion on individual entities could be ignored . |
| Approach: | They propose an adaptive multi-modal feature fusion strategy for entity alignment that selects the optimal entity-level feature blending strategy. |
| Outcome: | The proposed model achieves state-of-the-art (SOTA) performance compared to models using the same modality on a dataset with multiple inconsistent images and styles. |
Copied to clipboard
| Challenge: | Existing embedding-based EA methods encode entities as embeddables and learn to align embeddibles. |
| Approach: | They propose to capture three types of logical inference paths with Non-Axiomatic Logic to iteratively align entities and relations by integrating the conclusions of the inference path. |
| Outcome: | The proposed method outperforms state-of-the-art methods in terms of Hits@1 on all three datasets of DBP15K with both supervised and unsupervised settings. |
Copied to clipboard
| Challenge: | Optical character recognition (OCR) is a relatively new form of tablature recognition, but its accuracy is limited due to its unbounded composition and manuscript-level variability. |
| Approach: | They propose a method that predicts component sequences under a zero-shot split and synthesize manuscript-like training images via component-wise style recomposition and manuscript-domain noise modeling. |
| Outcome: | The proposed method achieves 63.02% sequence accuracy on real-world Jianzi benchmark, surpassing Gemini-3-Pro by 35.11%. |
Copied to clipboard
| Challenge: | Temporal knowledge graph reasoning (TKGR) is a crucial task that involves reasoning at known timestamps to complete the future facts. |
| Approach: | They propose a temporal knowledge graph reasoning model with logicality and densification strategy that captures temporal evolving pattern and structural information in TKGs. |
| Outcome: | The proposed model outperforms the state-of-the-art models and is based on a structure-aware language model with logicality and densification strategy. |
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated powerful capabilities in natural language processing, yet their vast number of parameters poses challenges for deployment and inference efficiency. |
| Approach: | They propose a structured pruning algorithm that derives the importance of different components based on intermediate data dependencies and removes coupled components across different layers simultaneously. |
| Outcome: | The proposed algorithm reduces model size and accelerates inference without specialized operators and libraries, while maintaining its utility as versatile problem solvers. |
Copied to clipboard
| Challenge: | Existing methods for generating static slides or text summaries are limited to producing narrated presentations. |
| Approach: | They propose a multimodal agent that transforms long-form documents into narrated presentations. |
| Outcome: | The present agent produces fully synchronized visual and spoken content that closely mimics human-style presentations. |
Copied to clipboard
| Challenge: | CPsyExam prioritizes psychological knowledge and case analysis separately, recognizing the significance of applying psychological knowledge to real-world scenarios. |
| Approach: | They propose a psychological benchmark, CPsyExam, constructed from questions from Chinese examination systems. |
| Outcome: | The proposed benchmark prioritizes psychological knowledge and case analysis separately, recognizing the significance of applying psychological knowledge to real-world scenarios. |
Copied to clipboard
| Challenge: | Document-level relation extraction (DocRE) aims to identify relations for a given entity pair within a document. |
| Approach: | They propose to partition the label space into different sub-label spaces and learn an adaptive threshold for each sub-labeled space. |
| Outcome: | The proposed model outperforms single-loss methods on the concurrent application of multiple losses across four datasets. |
Copied to clipboard
| Challenge: | Existing prototype-based methods for ZSRE ignore abundant side information and suffer from a significant encoding gap between prototypes and sentences. |
| Approach: | They propose a framework to encode schema alignment to enhance prototype-based ZSRE methods. |
| Outcome: | The proposed method outperforms existing methods on FewRel and Wiki-ZSL datasets and exhibits substantially faster performance and reduces the need for extensive manual labor in prototype construction. |
Copied to clipboard
| Challenge: | Entity alignment (EA) aims to identify entities in different knowledge graphs (KGs) that represent the same real-world object. |
| Approach: | They propose an end-to-end EA framework based on large language models that requires no training to implement. |
| Outcome: | The proposed framework significantly reduces the reliance on seed entity pairs while achieving state-of-the-art (SOTA) performance on diverse datasets. |
Copied to clipboard
| Challenge: | Existing methods focus on sentence-level or singledocument settings, resulting in one-sided relation transfer contextual bias and incomplete reasoning chains. |
| Approach: | They propose a framework to explicitly decouple and preserve bidirectional bridge evidence and a dynamic loss optimization objective to separate head and tail contexts. |
| Outcome: | The proposed framework decouples and preserves bidirectional bridge evidence while capturing global dependencies through iterative message passing. |
Copied to clipboard
| Challenge: | Existing datasets lack consulting knowledge, resulting in LLMs lacking professional consulting competence. |
| Approach: | They propose a report-based multi-turn dialogue reconstruction framework for Chinese psychological counseling that uses large language models to assist counseling. |
| Outcome: | The proposed framework is open-source and can be used in future research. |
Copied to clipboard
| Challenge: | Existing offline alternatives to Reinforcement Learning from Human Feedback (RLHF) are available at https://github.com/AIR-hl/MWPO. |
| Approach: | They propose an offline method to optimize preference pairs based on implicit reward margins and response length margins by reweighting them using a geometric mixture. |
| Outcome: | The proposed method outperforms state-of-the-art methods on four different scales and reduces generation length by 9.4%. |
Copied to clipboard
| Challenge: | Document-level relation extraction (DocRE) aims to extract relations between entities in a document. |
| Approach: | They propose an entity pair-guided relation summarization and retrieval model for DocRE . the model uses entity pairs to guide relation summaries and retrievals . |
| Outcome: | The proposed model achieves state-of-the-art (SOTA) performance on three datasets. |
Copied to clipboard
| Challenge: | Commosense knowledge graphs (CKGC) are powerful representations of real-world commonsense knowledge. |
| Approach: | They propose a framework that uses automatically generated prompt templates combined with pre-trained language models to improve CKGC performance. |
| Outcome: | The proposed framework mitigates the long-tail problem and improves CKGC performance on a large dataset. |
Copied to clipboard
| Challenge: | Existing methods for temporal knowledge graph extrapolation neglect the complex semantic relationships between relations when modeling their dynamic evolution. |
| Approach: | They propose a method for extracting semantic relationships to achieve TKG extrapolation . they use large language models to analyze the types of relations in TKGs . |
| Outcome: | The proposed method improves on five TKG datasets and shows performance gains. |
Copied to clipboard
| Challenge: | Existing methods for zero-shot relationship extraction do not distinguish between unseen, semantically similar relations. |
| Approach: | They propose a framework to enable global reasoning across a set of predictions. |
| Outcome: | The proposed framework outperforms existing methods and establishes new state-of-the-art results on widely used datasets. |
Copied to clipboard
| Challenge: | Document-level relation extraction (DocRE) provides a broad context for extracting relations for entities. |
| Approach: | They propose a method that utilizes LLMs as a refiner and task distribution and probability fusion to refine LLM-based relation extraction methods. |
| Outcome: | The proposed method outperforms existing LLM-based methods without fine-tuning by 25.2% F1. |
Copied to clipboard
| Challenge: | Existing verification approaches, such as Process Reward Models, are computationally expensive and limited to specific domains. |
| Approach: | They propose a transformer-based probe that uses internal states of frozen LLMs to estimate credibility of reasoning steps during generation. |
| Outcome: | The proposed probes match or exceed PRMs that are up to 810 larger. |
Copied to clipboard
| Challenge: | Existing methods for predicting future facts from time-evolving graphs rely on statistical co-occurrences and extensive path enumeration. |
| Approach: | They propose a Critic-Guided Rule Induction method which treats temporal rules as rule hypotheses to be examined and adopts a decoupled Generation-Discrimination pipeline to induce rules that are high-coverage and high-precision. |
| Outcome: | The proposed method outperforms strong baselines on three benchmarks and achieves state-of-the-art performance. |
Copied to clipboard
| Challenge: | Existing methods to extract triplets for unseen relations rely on costly fine-tuning and lack structured semantic guidance. |
| Approach: | They propose a framework that adopts a "frame first, then extract" paradigm to extract triplets from unstructured text. |
| Outcome: | The proposed framework achieves competitive zero-shot performance on multiple benchmarks and can be used to enhance existing extraction methods. |
Copied to clipboard
| Challenge: | a new framework automates deployment and debugging of AI projects . complexity of environment configurations, dependency conflicts, and debuggering issues hinder scalability and adoption. |
| Approach: | They propose an end-to-end framework that automates AI project deployment . they conducted experiments on 30 AI deployment cases to evaluate its effectiveness . |
| Outcome: | The proposed framework reduces deployment time and improves success rates by reducing human intervention. |
Copied to clipboard
| Challenge: | Existing methods for multimodal entity alignment overlook the quality of input modality embeddings during modality interaction, amplifying noise propagation while suppressing discriminative feature representations. |
| Approach: | They propose a model for capturing latent modal association for multimodal entity alignment using a self-attention mechanism to enhance salient information while attenuating noise within individual modality embeddings. |
| Outcome: | The proposed model achieves an absolute 3.1% higher Hits@1 score than the sota method. |
Copied to clipboard
| Challenge: | Existing approaches to integrate large language models into cross-lingual entity alignment tasks pose challenges in handling large-scale data, generating suitable data samples, and adapting prompts for the EA task. |
| Approach: | They propose a framework that integrates distance feature extraction, sample **Seg**mentation, and zero-shot prompts to integrate LLMs into cross-lingual entity alignment tasks. |
| Outcome: | The proposed framework is able to extract features from large-scale data and adapt prompts to the task. |
Copied to clipboard
| Challenge: | Existing methods to extract triplets from context often decompose into named entity recognition and relation classification, which may introduce error propagation. |
| Approach: | They propose a Relation-centric joint ZSRTE method which leverages unseen relation labels to extract triplets in one go. |
| Outcome: | The proposed method achieves state-of-the-art performance with fewer parameters and does not rely on synthetic data or manual labor. |
Copied to clipboard
| Challenge: | Existing work does not fully distinguish the contribution of different mentions to entity representation and the importance of mentions in evidence sentences. |
| Approach: | They propose a document-level relation extraction task that uses entity mentions to identify relations between entities in a text. |
| Outcome: | The proposed model achieves state-of-the-art on widely-adopted datasets. |