Papers by Xu Chu
Copied to clipboard
| Challenge: | Existing approaches to multimodal representation learning focus on directional alignment and embedding magnitudes (L2-norm) however, these methods often fail to account for the intrinsic role of L2-norm in the contrastive process. |
| Approach: | They propose a plug-and-play framework that optimizes L2-norm alignment and Directional consistency jointly. |
| Outcome: | The proposed framework achieves consistent and significant performance gains over established baselines across 95 tasks using UniIR and VLM2Vec-V2 frameworks. |
Copied to clipboard
| Challenge: | Topic modeling (TM) is a classic unsupervised learning task in the field of natural language processing. |
| Approach: | They propose a new taxonomy that emphasizes the role of LLMs and the design of end-to-end workflows. |
| Outcome: | The proposed taxonomy emphasizes the role of LLMs and the design of end-to-end workflows. |
Copied to clipboard
| Challenge: | Existing approaches to RAG neglect system state variables, resulting in poor performance and erroneous knowledge accumulation. |
| Approach: | They propose a framework that incorporates a Turing Complete System to manage state variables and manage retrieval halting. |
| Outcome: | The proposed framework improves on seven real-world healthcare datasets and shows that it is more accurate than existing methods. |
Copied to clipboard
| Challenge: | Recent studies show that supervised fine-tuning (SFT) is a common approach for reasoning in large language models. |
| Approach: | They propose to use supervised fine-tuning (SFT) on chain-of-thought trajectories demonstrations . they find that incorporating negative traxories yields substantial OOD generalization gains . |
| Outcome: | The proposed scheme yields 5.51% OOD gain over positive-only training. |
Copied to clipboard
| Challenge: | Discourse structure analysis is an important research topic in natural language processing. |
| Approach: | They propose to construct a macro discourse structure framework and annotate 147 Newswire articles. |
| Outcome: | The proposed framework can lay the foundation for further analysis of macro discourse structure. |
Copied to clipboard
| Challenge: | Continual learning (CL) for large language models (LLMs) aims to enable sequential knowledge acquisition without catastrophic forgetting. |
| Approach: | They propose a framework that aligns replay schedules with a model-centric notion of time. |
| Outcome: | Experiments on three benchmarks show that FOREVER consistently mitigates catastrophic forgetting. |
Copied to clipboard
| Challenge: | Existing models struggle to detect elaborately disguised malicious URLs, despite their ability to process malicious URL's. |
| Approach: | They propose a benchmark to evaluate LLMs’ vulnerabilities to malicious URLs and a lightweight defense module to mitigate the vulnerability. |
| Outcome: | The proposed framework analyzes 61,845 attack instances spanning 10 real-world scenarios and 7 categories of real malicious websites. |
Copied to clipboard
| Challenge: | Large language models (LLMs) have attracted significant interest from the research community due to their broad applicability in many language-oriented tasks. |
| Approach: | They propose a framework which uses pre-training datasets to rewrite instructions and generate negative responses to preserve the performance of the original LLM. |
| Outcome: | The proposed framework can erase the pre-training data while maintaining the performance of the original model. |
Copied to clipboard
| Challenge: | Recent studies have demonstrated the potential of large language models (LLMs) for automatic error detection in math word problems (MWPs). |
| Approach: | They propose a framework that generates adaptive reference solutions using LLMs to enhance error detection by reducing conformity bias in MWPs. |
| Outcome: | The proposed framework mitigates the performance gap between conventional and alternative solutions in MWPs, especially when combined with reasoning-enhancing techniques like chain-of-thought prompting. |
Copied to clipboard
| Challenge: | Effective domain adaptation typically involves supervised fine-tuning on carefully selected instruction-tuned data. |
| Approach: | They propose a model-centric data selection framework that aligns data selection with the model’s knowledge distribution to improve model performance. |
| Outcome: | The proposed framework outperforms existing methods by up to 2.97% accuracy in the healthcare domain. |
Copied to clipboard
| Challenge: | Current mitigation strategies fail to preserve contextual reasoning capabilities in risky scenarios, leading to systemic risks for legal compliance. |
| Approach: | They propose to use reinforcement learning with a rule-based reward to incentivize contextual reasoning capabilities while enhancing compliance with safety and privacy norms. |
| Outcome: | The proposed model outperforms Qwen2.5-7B-Instruct model in safety and privacy benchmarks and achieves +8.58% accuracy improvement. |
Copied to clipboard
| Challenge: | Document images are characterized by higher resolutions, denser content, and more complex structural layouts. |
| Approach: | They propose a 1.2B-parameter document parsing vision-language model that decouples layout analysis from local content recognition. |
| Outcome: | The proposed model surpasses general-purpose and domain-specific models on multiple benchmarks while maintaining significantly lower computational overhead. |
Copied to clipboard
| Challenge: | Existing GUI agents interact with the environment through extracted structured data, which can be notably lengthy (e.g., HTML) and occasionally inaccessible (e-book). |
| Approach: | They propose to enhance SeeClick with GUI grounding pre-training and devise a method to automate curation of GUI ground data. |
| Outcome: | The proposed agent improves ScreenSpot, the first realistic GUI grounding benchmark that encompasses mobile, desktop, and web environments. |
Copied to clipboard
| Challenge: | Experiments show that LAS cuts token usage by 43% and reduces end-to-end latency by more than 36%, while causing at most a 1.4 percentage-point drop in accuracy compared with a strong fixed workflow. |
| Approach: | They propose a system that dynamically chooses the right workflow for each query. |
| Outcome: | Experiments show that LAS cuts token usage by 43% and reduces end-to-end latency by more than 36% while causing at most a 1.4 percentage-point drop in accuracy compared with a strong fixed workflow. |
Copied to clipboard
| Challenge: | Graph Neural Networks (GNNs) and graph transformers are inadequate for tasks with limited generalization. |
| Approach: | They propose a multi-stage graph reasoning framework based on vision-language models that incrementally samples and visualizes task-relevant subgraphs. |
| Outcome: | The proposed framework outperforms existing benchmarks in Graph-related tasks. |
Copied to clipboard
| Challenge: | Recent studies attempt to obtain optimal or suboptimal arrangements based on statistical results or using dataset-based search, but these methods increase inference overhead while leaving the model’s inherent order bias unresolved. |
| Approach: | They propose Dual Group Advantage Optimization (DGAO) which aims to improve model accuracy and order stability simultaneously. |
| Outcome: | The proposed method improves model accuracy and order stability while penalizing order-sensitive or incorrect responses. |
Copied to clipboard
| Challenge: | Recent approaches suffer from insufficient and repetitive knowledge retrieval, tedious and time-consuming query parsing, and monotonous knowledge utilization. |
| Approach: | They propose a retrieval-augmented generation framework which leverages LLMs’ powerful reasoning capacity to compensate for the incompleteness of user queries. |
| Outcome: | The proposed framework improves the accuracy and reliability of Large Language Models (LLMs) by combining the rich knowledge of LLMs with Hypothesis Outputs. |
Copied to clipboard
| Challenge: | Current mathematical benchmarks focus on evaluating MLLMs’ problem-solving ability, yet there is a crucial gap in addressing more complex scenarios such as error detection. |
| Approach: | They propose to evaluate multimodal error detection by evaluating two sub-tasks error step identification and error categorization. |
| Outcome: | The proposed task evaluates MLLMs' ability to handle multimodal questions compared to text-only models. |
Copied to clipboard
| Challenge: | Existing methods for Named Entity Recognition (NER) ignore the internal state of the target model. |
| Approach: | They propose a framework to repair model-specific errors by using a model-based approach . they employ cross-validation to identify model- specific Hard Data and a memory tree to induce macro-level error patterns from micro-level failures. |
| Outcome: | The proposed framework yields significant performance gains on Twitter and other platforms. |
Copied to clipboard
| Challenge: | Existing models for knowledge editing focus on knowledge-level or static visual domains, overlooking dynamic semantics. |
| Approach: | They propose a benchmark for modeling large language models using six representative models . they analyze the strengths and limitations of existing models and identify new directions . |
| Outcome: | The proposed benchmark extends existing models from static modalities to dynamic video scenarios. |
Copied to clipboard
| Challenge: | Current reinforcement learning paradigms rely on outcome-based rewards, overlooking latent logical fallacies in intermediate steps. |
| Approach: | They propose a specialized audit model augmented with external tools to identify local logical ruptures and calibrate reward signals. |
| Outcome: | The proposed framework improves accuracy and logical rigor in high-stakes domains. |
Copied to clipboard
| Challenge: | Discourse analysis is becoming increasingly important in the field of natural language processing. |
| Approach: | They propose to annotate macro discourse information and additional discourse information to make annotation more objective and accurate. |
| Outcome: | The results show that the annotations are more objective and accurate than the previous ones. |
Copied to clipboard
| Challenge: | PromptSculptor automates the iterative prompt optimization process for Text-to-Image models . previous work focused on generating detailed, high-quality prompts based on user feedback . |
| Approach: | They propose a framework that decomposes a task into four specialized agents . they use Chain-of-Thought reasoning to transform a short, vague user prompt into a comprehensive, refined prompt. |
| Outcome: | The proposed framework significantly improves output quality and reduces iterations needed for user satisfaction. |
Copied to clipboard
| Challenge: | Continual learning (CL) is crucial for large language models without costly retraining. |
| Approach: | They propose a framework for recurrent knowledge identification and fusion that enables dynamic estimation of parameter importance distributions to enhance knowledge transfer. |
| Outcome: | The proposed framework mitigates catastrophic forgetting and enhances knowledge transfer. |
Copied to clipboard
| Challenge: | Existing topic seed words are difficult to incorporate into topic models due to the semantic diversity of natural language. |
| Approach: | They propose a neural topic model enhanced with supervisions from seed words on word and document levels. |
| Outcome: | The proposed model outperforms the state-of-the-art seeded topic models in terms of topic quality and classification accuracy. |
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) have demonstrated significant strides in generating high-quality speech . discretizing speech by neural audio codecs often results in sequences that differ from text sequences . |
| Approach: | They quantitatively analyze the Discrete Representation Inconsistency phenomenon within popular audio tokenizers such as EnCodec. |
| Outcome: | The proposed method mitigates the DRI phenomenon within popular audio tokenizers such as EnCodec. |
Copied to clipboard
| Challenge: | Existing studies employ a one-time generation approach to synthesize multi-turn dialogue samples, resulting in low therapy fidelity and failing to capture decision-making rationale behind each response. |
| Approach: | They propose a data synthesis framework that synthesizes multi-turn dialogue samples and incrementally generates stage-aligned counseling dialogues. |
| Outcome: | The proposed framework significantly improves therapy fidelity and logical coherence in AI counseling. |
Copied to clipboard
| Challenge: | Existing methods for integrating internal and external knowledge lack effective control mechanisms for generating hallucinations and dealing with outdated knowledge. |
| Approach: | They propose a framework that decouples, identifies, and purposefully optimizes parameter subspaces related to adherence and robustness. |
| Outcome: | The proposed framework decouples, identifies, and purposefully optimizes parameter subspaces related to adherence and robustness. |
Copied to clipboard
| Challenge: | Knowledge Editing (KE) has gained increasing attention, yet current evaluation frameworks do not integrate KE into real-world application scenarios. |
| Approach: | They propose a script-based benchmark which encompasses both counterfactual and temporal edits and integrates token-level and text-level evaluation methods. |
| Outcome: | The proposed method combines token-level and text-level evaluation methods with a new fact-based evaluation framework. |
Copied to clipboard
| Challenge: | Recent studies have incorporated reward models to guide response selection or decoding, aiming to obtain higher-quality data. |
| Approach: | They propose a Hierarchical Sampling framework for self-taught reasoners that allocates a fixed sampling budget to problem boundary-level problems and then reallocates the remaining budget toward high-utility problems during a re-sampling phase. |
| Outcome: | The proposed framework outperforms baseline models without additional sampling budgets across multiple reasoning benchmarks and backbone LLMs. |
Copied to clipboard
| Challenge: | Existing Large Language Models (LLMs) generate brief answers without reasoning processes and explanations. |
| Approach: | They propose supervised fine-tuning and tree search to enhance LLMs’ reasoning capabilities on domain tasks. |
| Outcome: | The proposed model improves on stock investment recommendation and legal reasoning QA tasks. |
Copied to clipboard
| Challenge: | Existing benchmarks for audio-centric interaction have impeded advancements in this field . AIR-Bench evaluates LALMs' ability to understand audio signals and interact with humans . |
| Approach: | They propose a benchmark to evaluate the ability of large audio-language models to understand audio signals . they use 19 tasks with approximately 19k single-choice questions to examine single-task ability . |
| Outcome: | The proposed framework evaluates the ability of large audio-language models to understand audio signals and interact with humans in the textual format. |
Copied to clipboard
| Challenge: | Numerical reasoning requires both natural language understanding and arithmetic computation. |
| Approach: | They propose a graph representation for the context of the passage and question needed for numerical reasoning. |
| Outcome: | The proposed model achieves remarkable results in benchmark datasets such as DROP. |
Copied to clipboard
| Challenge: | Deductive and inductive reasoning are fundamental components of human cognition . authors present a benchmark to assess their performance in procedural planning . |
| Approach: | They propose a benchmark to assess the deductive and inductive reasoning abilities of LLMs . they propose IMSE to enable LLM to generate multiple similar procedural plans . |
| Outcome: | The proposed method improves inductive reasoning abilities of LLMs, the authors show . they show that LLM models show excellent deductive reasoning capabilities but suboptimal inductive performance. |
Copied to clipboard
| Challenge: | Existing tools that integrate chain-of-thought reasoning and code execution lack metacognitive awareness to integrate tools. |
| Approach: | They propose a framework that synergizes structured exploration with off-policy RL optimization to create a cycle between metacognitive tool-use decisions and evolving capabilities. |
| Outcome: | The proposed framework improves over 11% on MATH500 and 9.4% on AIME without o1-like CoT. |
Copied to clipboard
| Challenge: | Weight quantization has emerged as a popular solution to reduce memory and computational demands. |
| Approach: | They propose a framework that synergizes Quantization-Aware Training (QAT) with Knowledge Distillation (KD) to boost the performance of LLMs at sub-4-bit. |
| Outcome: | The proposed framework outperforms existing QAT methods on language understanding and complex reasoning benchmarks on sub-4-bit models. |
Copied to clipboard
| Challenge: | Model Context Protocol (MCP) introduces an easy-to-use ecosystem for users and developers, but it also brings underexplored safety risks. |
| Approach: | They propose a framework that addresses the missing safety mechanisms in MCP and a taxonomy that captures diverse range of unsafe behaviors observed in MMP scenarios. |
| Outcome: | The proposed framework improves safety performance on state-of-the-art LLMs by capturing unsafe behaviors and analyzing the results. |
Copied to clipboard
| Challenge: | Existing methods to extract multiple events with triggers and arguments are invalid as there may be multiple events. |
| Approach: | They propose a framework for event extraction which models the relations between arguments by an event matrix. |
| Outcome: | The proposed framework beats all the advanced competitors on 3 widely-used datasets. |
Copied to clipboard
| Challenge: | Existing topic models suffer from poor performance when applied to short text contents due to the limited length of a single topic. |
| Approach: | They propose a neural short text topic model that augments reconstruction labels with k-nearest documents to complement relevant but unobserved words. |
| Outcome: | The proposed model outperforms the state-of-the-art models on multiple public short-text datasets and can derive high-quality topics and document representations. |
Copied to clipboard
| Challenge: | Existing word embeddings can be used to learn sentence embedds on the sentence level. |
| Approach: | They propose a sentence embedding method that uses the inner product to compute semantic similarity between sentences. |
| Outcome: | The proposed method encodes sentences better in the sense of semantic structures. |
Copied to clipboard
| Challenge: | Code LLMs lack reproducible data pipelines and training protocols for reproducible advancements in code intelligence. |
| Approach: | They propose a top-tier code LLM that releases model weights and inference code . reproducible data pipelines, rigorous experimental ablation results and training protocols are included . |
| Outcome: | The proposed model achieves comparable performance to leading models and serves as an "open cookbook" reproducible training data, rigorous experimental ablation results, and detailed training protocols are also included in the model. |
Copied to clipboard
| Challenge: | Current methods for Continual Dialogue State Tracking (DST) struggle with catastrophic forgetting and knowledge transfer between tasks. |
| Approach: | They propose a framework for task skill localization and consolidation that enables effective knowledge transfer without relying on memory replay. |
| Outcome: | The proposed framework shows a 7.6% increase in Avg. JGA and 11% rise in BWT metrics over existing state-of-the-art methods. |
Copied to clipboard
| Challenge: | Existing methods to detect and correct spelling errors in Chinese take external input or just heuristic rules. |
| Approach: | They propose to incorporate phonological and visual similarity knowledge into Chinese language models by using a specialized graph convolutional network. |
| Outcome: | The proposed method outperforms existing models on three human-annotated datasets. |
Copied to clipboard
| Challenge: | Existing methods for ambiguous queries struggle to retrieve high-quality documents . DFAMS outperforms advanced FR methods by 14.37% in knowledge classification accuracy . |
| Approach: | They propose a framework that leverages dynamic information flow to identify latent query intents and construct semantically aligned knowledge partitions for accurate retrieval across heterogeneous sources. |
| Outcome: | The proposed framework outperforms existing methods in classification accuracy and retrieval recall tests. |
Copied to clipboard
| Challenge: | Recent advances in model distillation show that data from advanced reasoning models can effectively train smaller student models. |
| Approach: | They propose a method to use both positive and negative distilled reasoning traces to maximize LLM reasoning performance in offline settings. |
| Outcome: | The proposed model outperforms existing methods in the distillation context. |
Copied to clipboard
| Challenge: | Recent advances in speech large language models exhibit suboptimal performance in adhering to speech instructions. |
| Approach: | They propose a method to pre-train large-scale unsupervised speech-text sequences . they use text-to-speech conversion to generate textual continuations corresponding to provided speech segments . |
| Outcome: | The proposed model achieves superior or competitive results across diverse speech processing tasks. |
Copied to clipboard
| Challenge: | Existing training-based model editing methods struggle to incorporate new knowledge while preserving unrelated general knowledge. |
| Approach: | They propose a framework that uses geometric relationships to differentiate between neurons associated with new knowledge updates and those related to general knowledge perturbations. |
| Outcome: | The proposed framework avoids updating neurons with directions approximately orthogonal to existing knowledge, thus preserving the model’s generalization ability. |
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs)-based Role-Playing Language Agents (RPLAs) have attracted broad attention in various applications. |
| Approach: | They propose a benchmark for evaluating character thought generation using literature . they propose 'MIRROR' which generates character thoughts by retrieving memories, predicting character reactions, and synthesizing motivations. |
| Outcome: | The proposed benchmark outperforms existing methods in evaluating character thought generation. |
Copied to clipboard
| Challenge: | Recent model merging-based methods struggle to effectively manage the trade-off between learning new knowledge and preventing catastrophic forgetting. |
| Approach: | They propose a model merging framework that utilizes learning and forgetting signals from the training trajectory to dynamically monitor the model’s training status. |
| Outcome: | The proposed framework achieves significant performance improvements over existing state-of-the-art methods on three CL benchmarks with various model sizes (from 770M to 13B). |
Copied to clipboard
| Challenge: | Methods for controlling large language models (LLMs) are often studied in isolation, obscuring connections and making comparison difficult. |
| Approach: | They propose a preference-utility analysis that separates control effects into preference and utility, and measures both on a shared log-odds scale using polarity-paired contrastive examples. |
| Outcome: | The proposed approach improves preference while preserving utility. |
Copied to clipboard
| Challenge: | Existing studies on explanation stability under real user noise are limited . decoder LLMs produce significantly more stable explanations than encoder baselines . |
| Approach: | They propose a black-box robustness evaluation framework for token-level explanations based on leave-one-out occlusion . they propose to operationalize explanation robustness with top-token flip rate under realistic perturbations at multiple severity levels . |
| Outcome: | The proposed framework is compared with baseline models and encoder and decoder families. |