Papers by Han Jin
Copied to clipboard
| Challenge: | Existing fine-tuning algorithms for vision-language models are restricted by patient privacy concerns and can contain imperceptible noise. |
| Approach: | They propose a framework to mitigate adversarial noise and mitigate upstream noise during fine-tuning. |
| Outcome: | The proposed framework improves model robustness and transferability while decreasing noise levels negatively impact downstream performance. |
Copied to clipboard
| Challenge: | Large language models (LLMs) are widely used in commercial applications . low latency is crucial due to system latency, query concurrency, and computational resources constraints. |
| Approach: | They propose a system that can be resource-efficiently served by addressing bottlenecks beyond LLM inference . they propose 4.3 speed up over vLLM and 1.5 higher throughput . |
| Outcome: | The proposed system outperforms state-of-the-arts with 1.5 higher throughput . it achieves 4.3 speed up with 64 concurrent requests on Mixtral 8x7B . |
Copied to clipboard
| Challenge: | Existing models for text-rich networks do not take inter-document structure into account. |
| Approach: | They propose a pretraining framework for a text-rich network using a masked language model and a masking node prediction framework. |
| Outcome: | The proposed model outperforms baselines on four tasks in academic and e-commerce domains. |
Copied to clipboard
| Challenge: | Existing Relation extraction models require extensive annotated training data, which is costly and labor-intensive to collect. |
| Approach: | They propose a new zero-shot RE task where only relation definitions are provided instead of seen-unseen relation instances. |
| Outcome: | The proposed task significantly improves cost-effective zero-shot performance by large margins. |
Copied to clipboard
| Challenge: | Existing models for implicit hate speech detection do not have significant advantage over cross-entropy loss-based learning. |
| Approach: | They propose a label-aware hard negative sampling strategy that encourages the model to learn detailed features from hard negative samples instead of random batch. |
| Outcome: | The proposed models outperform existing models for implicit hate speech detection both in- and cross-datasets. |
Copied to clipboard
| Challenge: | a recent study examined the effects of media framing on public perception and understanding of news articles. |
| Approach: | They propose to extract framing devices employed by media to assess their role in framating the narrative. |
| Outcome: | The proposed method surpasses baseline models and offers a more detailed and explainable analysis of media framing effects. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable performance across a diverse set of domain-specific tasks. |
| Approach: | They propose a non-monolithic LLM querying system that seamlessly integrates various LLM experts into a single query interface and dynamically routes incoming queries to the most high-performant expert based on query’s requirements. |
| Outcome: | The proposed model improves query efficiency by 40% and costs by 30% while maintaining or enhancing model performance by 10%. |
Copied to clipboard
| Challenge: | Existing surveys on scientific LLMs focus on one or two fields or a single modality. |
| Approach: | They survey 260 scientific LLMs and examine their architectures and pre-training techniques . they also discuss commonalities and differences between LLM architectures . |
| Outcome: | The proposed model architectures and evaluation techniques are used to improve scientific discovery. |
Copied to clipboard
| Challenge: | Language models (LMs) have demonstrated impressive reasoning capabilities across domains . but their ability to handle PSPACE-complete problems remains underexplored . a new benchmark for regex minimization is proposed to evaluate LMs' reasoning capabilities . |
| Approach: | They propose a benchmark for regex minimization to evaluate LMs' reasoning power . they use a million regexes paired with their minimal equivalents to evaluate their performance . |
| Outcome: | The proposed model can solve NP-complete problems, but their ability to handle PSPACE-complete ones remains underexplored. |
Copied to clipboard
| Challenge: | Existing efforts to improve reasoning efficiency of large language models focus on modifying the reinforcement learning reward, such as adding length penalties. |
| Approach: | They propose a training framework that elicits efficient reasoning through reasoning vectors and a framework that allows the model to generate high-quality responses during reinforcement learning. |
| Outcome: | The proposed framework reduces reasoning length by 30% while maintaining stability, while retaining high accuracy. |
Copied to clipboard
| Challenge: | Existing models for fine-grained speaking styles are limited in terms of accuracy, coverage, and naturalness. |
| Approach: | They propose a model that pre-trains with coarse captions and annotates with a pipeline that grounds captions in audio. |
| Outcome: | The proposed model outperforms existing models with fine-grained style annotations . it integrates global and fine-granular supervision, enabling unified representations based on the proposed model . |
Copied to clipboard
| Challenge: | Existing methods to augment knowledge graph completion require factual triples or manual prompts to extract knowledge from a pre-trained language model. |
| Approach: | They propose a tool that generates quality query prompts and retrieves support information from large text corpora to probe knowledge from a pre-trained language model. |
| Outcome: | The proposed method outperforms embedding-based, graph-based and PLM-based methods on two benchmark datasets. |
Copied to clipboard
| Challenge: | Reinforcement learning (RL)-based post-training often improves the reasoning performance of large language models beyond the training domain, while supervised fine-tuning (SFT) frequently leads to general capabilities forgetting. |
| Approach: | They propose a feature-level mechanistic analysis methodology to probe RL generalization using a controlled experimental setup. |
| Outcome: | The proposed method identifies a compact, task-agnostic set of features that directly mediate generalization across diverse tasks. |
Copied to clipboard
| Challenge: | Recent studies show that pre-trained masked language models can be factual knowledge bases. |
| Approach: | They conduct a rigorous study to explore the underlying predicting mechanisms of MLMs . they find that previous decent performance mainly owes to the biased prompts which overfit dataset artifacts a . |
| Outcome: | The proposed model improves on illustrative cases and external contexts . the results question the previous findings that MLMs can be reliable factual knowledge bases . |
Copied to clipboard
| Challenge: | OpenAI introduces deliberative alignment (DA) to enhance safety of its o-series models, but effectiveness of this approach in open-source LLMs is understudied. |
| Approach: | They propose a case-augmented deliberative alignment method for large language models . they propose to use reinforcement learning on self-generated safety reasoning chains . |
| Outcome: | The proposed method avoids narrowly enumerated rules and allows broader adaptability. |
Copied to clipboard
| Challenge: | LongLeader aims to assess different LLMs' long-context comprehension abilities . long-constext comprehension is a key bottleneck for many use cases . |
| Approach: | They propose a leaderboard to assess different LLMs' long-context comprehension abilities . they offer open-source access to the benchmarks and maintain a dedicated website . |
| Outcome: | The proposed model assesses different LLMs on selected benchmarks and provides open-source access to the benchmarks. |
Copied to clipboard
| Challenge: | Existing methods focus on designing efficient multimodal fusion frameworks to bridge the semantic gap between images and texts. |
| Approach: | They propose a covariance matrix-driven image channel allocation method that expands the number of original channel maps and assigns importance scores to the expanded channel maps. |
| Outcome: | The proposed method achieves state-of-the-art on three public multimodal fake news detection benchmark datasets. |
Copied to clipboard
| Challenge: | Existing gradient-based attribution methods are inapplicable to adversarial attacks . et al.: Targeted neuron tuning improves model robustness against jailbreak attacks despite the model's vulnerability to jailbreak. |
| Approach: | They propose a gradient-based method to identify key neurons sensitive to adversarial behaviors in open-ended generation tasks. |
| Outcome: | The proposed method detects key neurons sensitive to adversarial behaviors in open-ended tasks. |
Copied to clipboard
| Challenge: | Concept editing aims to control specific concepts in large language models (LLMs) however, there is a lack of rigorous theoretical analysis and a unified perspective to systematically understand and compare these methods. |
| Approach: | They propose a paradigm where conceptual injection is aligned at the neuron level. |
| Outcome: | The proposed paradigm offers a clear framework and valuable insights for advancing interpretability and controlled generation in large language models. |
Copied to clipboard
| Challenge: | Existing methods for event prediction are incomplete and noisy. |
| Approach: | They propose to use news-related event schemas to extract newsworthy events . they build a demo website and include a video demonstrating the framework . |
| Outcome: | The proposed framework can be applied to a wide variety of newsworthy scenarios. |
Copied to clipboard
| Challenge: | Existing evaluation methods are inadequate to evaluate large language models (LLMs). |
| Approach: | They propose a fine-grained generative LLM evaluator with instance-level customazable evaluation criteria that can be used to evaluate large language models. |
| Outcome: | The proposed model outperforms existing LLM evaluators and instruction-tuned LLMs on multiple benchmarks and sets new SOTA results. |
Copied to clipboard
| Challenge: | Large language models have achieved remarkable success across a wide range of tasks, yet their performance remains heavily biased toward high-resource languages. |
| Approach: | They propose a pipeline for advancing Tibetan language modeling through multilingual continual pre-training with Tibetan, Chinese, and English. |
| Outcome: | The proposed model outperforms open-source and Tibetan-focused models on diverse tasks. |
Copied to clipboard
| Challenge: | Existing methods to generalize hate speech detection models have been limited by the labeling criteria between datasets. |
| Approach: | They propose a framework that uses the concept of multi-agent for hate speech detection that uses a set of labeling criteria to create multiple agents based on the induced labeling of given datasets. |
| Outcome: | The proposed framework achieves superior cross-evaluation performance compared to methods that focus on specific labeling criteria or majority voting methods. |
Copied to clipboard
| Challenge: | Reinforcement learning with verifiable rewards (RLVR) has emerged as a paradigm for enhancing the reasoning capabilities of large language models. |
| Approach: | They propose a positive-advantage reweighting approach that regulates model entropy by adjusting the loss weights assigned to tokens with positive advantages during RLVR training. |
| Outcome: | The proposed approach regulates model entropy by adjusting loss weights assigned to tokens with positive advantages during RLVR training while maintaining competitive performance. |
Copied to clipboard
| Challenge: | a growing number of LLMs have been used to provide reasoning, writing, text-editing capabilities. |
| Approach: | They propose a method to inject imperceptible phantom tokens into LLMs to deceive users . the technique generates outputs that appear plausible to users but are in fact incorrect . |
| Outcome: | a new method injects imperceptible tokens into documents to deceive users . the proposed framework is compared to baselines to show its effectiveness . |
Copied to clipboard
| Challenge: | Factuality evaluation aims to detect factual errors produced by language models and guide the development of more factual models. |
| Approach: | They propose a framework that leverages FenCE to improve the factuality of LM generators by constructing training data. |
| Outcome: | The proposed framework improves the factuality of LM generators by enhancing their training data. |
Copied to clipboard
| Challenge: | Existing methods to extract event records from text decompose complex structure prediction task into multiple subtasks. |
| Approach: | They propose a sequence-to-structure generation paradigm that can extract events from text . they propose unified event extraction, constrained decoding algorithm and curriculum learning algorithm . |
| Outcome: | The proposed method can achieve competitive performance using record-level annotations in both supervised learning and transfer learning settings. |
Copied to clipboard
| Challenge: | Existing methods for multimodal content detection fail to capture cross-modal semantic inconsistencies and ignore inherent noise in multimodal features. |
| Approach: | They propose a multimodal rumor detection method based on a frequency domain spectral selection method and entropy-guided uncertainty fusion method to capture cross-modal semantic inconsistencies. |
| Outcome: | The proposed method outperforms state-of-the-art methods in multimodal rumor detection . it shows stronger detection capability and robustness on multiple datasets . |
Copied to clipboard
| Challenge: | Long-form question answering requires two procedures: information retrieval and information synthesis. |
| Approach: | They propose a Chinese long-form question answering dataset called WebCPM . the dataset is based on a web search interface that engages with a search engine in real time . |
| Outcome: | The proposed dataset generates answers that are no worse than human-written ones . the dataset is the first Chinese LFQA dataset . |
Copied to clipboard
| Challenge: | Current reinforcement learning methods suffer from coarse-grained, trajectory-level rewards that provide insufficient learning signals for complex multi-turn interactions, leading to training stagnation. |
| Approach: | They propose a novel RL algorithm for training large language models for multi-turn tool-integrated reasoning (TIR) that incorporates three innovations: turn-level reward assignment that provides fine-grained feedback for individual turns, return-based advantage estimation where normalized discounted returns are calculated as advantages, and self-supervised reward shaping that exploits self-supervision signals from generated code to densify sparse binary outcome-based rewards. |
| Outcome: | The proposed algorithm outperforms GRPO by 3.0% across diverse math reasoning benchmarks and improves grepo by 3.9% on commonsense reasoning and program synthesis tasks. |
Copied to clipboard
| Challenge: | Using the ABT structure, academic abstracts are structured to provide clear and concise prose, but a lack of clarity and logical coherence is a challenge for authors struggling with English proficiency or academic writing conventions. |
| Approach: | They propose a framework that identifies the key components of an abstract and reorients itself to properly reflect the ABT logical progression. |
| Outcome: | The proposed framework improves comprehensibility of academic writing, particularly for non-native English speakers, and is based on a human evaluation and automated metrics. |
Copied to clipboard
| Challenge: | Recent advances in multimodal recommenders lack explicit reasoning and self-awareness of uncertainty. |
| Approach: | They propose a reasoning-augmented multimodal agent structured around a three-stage explicit reasoning pipeline. |
| Outcome: | The proposed agent improves ranking metrics and performance on four standard recommendation tasks across five real-world datasets. |
Copied to clipboard
| Challenge: | Existing benchmarks for long-context capability are too synthetic and do not represent the real world usage of LLMs. |
| Approach: | They propose a length-controllable, real-life reflective benchmark that disentangles baseline knowledge from long-context capabilities. |
| Outcome: | Experiments show that the proposed benchmarks disentangle baseline knowledge from long-context capabilities. |
Copied to clipboard
| Challenge: | Existing studies suggest augmenting LLMs with external text corpora to alleviate hallucination problems. |
| Approach: | They propose to augment large language models with text units retrieved from external knowledge corpora to alleviate the issue. |
| Outcome: | The proposed framework outperforms baselines on GRBench with three LLMs and shows that iterative reasoning outperformed the baselines. |
Copied to clipboard
| Challenge: | Large language models (LLMs) have achieved remarkable progress, yet their internal mechanisms remain largely opaque. |
| Approach: | They propose an agent-based framework that recasts feature interpretation from a passive, single-pass generation task into an explanation-driven process. |
| Outcome: | The proposed framework produces explanations with significantly higher generative and predictive accuracy compared to state-of-the-art baselines. |
Copied to clipboard
| Challenge: | Existing studies focus on specific aspects or applications, but this study provides a comprehensive overview of Protein-specific large language models. |
| Approach: | This paper proposes a structured taxonomy of state-of-the-art ProteinLLMs . they analyze how they leverage large-scale protein sequence data for improved accuracy . |
| Outcome: | The proposed model covers their architectures, training datasets, evaluation metrics, and diverse applications. |
Copied to clipboard
| Challenge: | Recent works have proposed novel tree Transformers to capture the syntactic structure in source code. |
| Approach: | They propose a novel tree Transformer encoding node positions based on a description method for tree structures to incorporate inductive bias into Transformer. |
| Outcome: | The proposed model outperforms baselines on code summarization and completion tasks across two languages, and it is able to perform better on both local and global paradigms. |
Copied to clipboard
| Challenge: | Graph Neural Networks (GNNs) with CLIP pipeline are difficult because of the scarcity of labeled data and text supervision, different levels of downstream tasks, and conceptual gaps between domains. |
| Approach: | They propose a multi-modal prompt learning paradigm to adapt pre-trained GNNs to downstream tasks with weak text supervision. |
| Outcome: | The proposed model can generalize graphs to unseen classes with weak text supervision. |
Copied to clipboard
| Challenge: | Current development practices face a dichotomy between automation and performance. |
| Approach: | They propose a framework to empower LLMs with the capability of automated explicit vectorization. |
| Outcome: | The proposed framework achieves state-of-the-art performance on the SSE and AVX subsets of SimdBench. |
Copied to clipboard
| Challenge: | Structured information extraction from long, multilingual scanned financial documents is a core requirement in industrial KYC and compliance workflows. |
| Approach: | They propose a framework for structured information extraction from long, multilingual scanned financial documents . they combine image preprocessing, multilinguistic OCR, hybrid page-level retrieval and VLMs . |
| Outcome: | The proposed pipeline outperforms direct PDF-to-VLM baselines on 120 production KYC documents. |
Copied to clipboard
| Challenge: | Existing event-centric knowledge graphs rely on explicit connectives to extract relations between events. |
| Approach: | They propose a knowledge projection paradigm for event relation extraction using commonalities between events. |
| Outcome: | The proposed method achieves state-of-the-art performance and extrinsic results verify the extracted event relations. |
Copied to clipboard
| Challenge: | Recent advances in pretraining methods have achieved promising results on NLP tasks . however, it is unclear which pretraining objective is the most effective for each downstream task . |
| Approach: | They evaluate the effectiveness of domain-adaptive pretraining objectives on downstream tasks . they use open-domain data to pretrain language models like BERT and SpanBERT . |
| Outcome: | The proposed model improves on two dialogue understanding tasks with domain-adaptive pretraining objectives. |
Copied to clipboard
| Challenge: | Multimodal Large Language Models (MLLMs) excel in tasks ranging from image captioning to complex reasoning. |
| Approach: | They propose a contrastive decoding framework that dynamically calibrates each token generation by mining the model’s internal perceptual discrepancies. |
| Outcome: | The proposed framework mitigates hallucination while enhancing general reasoning capabilities. |
Copied to clipboard
| Challenge: | Existing methods for personality detection are limited due to the latent and relative nature of personality and lack of annotated datasets. |
| Approach: | They propose a method that exploits the inherent knowledge of Large Language Models to capture the relative nature of personality traits by comparing contrasting perspectives. |
| Outcome: | The proposed approach exploits the inherent knowledge of Large Language Models to capture the relative nature of personality traits. |
Copied to clipboard
| Challenge: | Existing studies on event-centric opinion mining focus on entity-centric opinions . entity-centered opinions focus on sentimental polarity of events, while event-centered ones focus on content . |
| Approach: | They propose to perform event-centric opinion mining on event-argument structure and expression categorizing theory and benchmark it against a pioneer corpus. |
| Outcome: | The proposed task is feasible and challenging, and the results are beneficial for future studies. |