Papers by Hu Cao
Copied to clipboard
| Challenge: | generative models struggle with logic-intensive instruction following, exposing a persistent reasoning–execution gap. |
| Approach: | They propose a task-agnostic reasoning architecture for general image generation . they propose pixel-level feedback to ground the Thinker's policy in pixel feedback . |
| Outcome: | The proposed system significantly improves image reasoning and generation quality. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly used for code editing, yet the full-code generation paradigm suffers from severe efficiency bottlenecks. |
| Approach: | They propose to use a structure-aware diff format to train LLMs to choose the most token-efficient format between a given diff format and full code. |
| Outcome: | The proposed approach matches the most token-efficient format with full-code generation while reducing latency and cost by over 30% on long-code editing tasks. |
Copied to clipboard
| Challenge: | Existing knowledge editing methods focus on single editing, failing to meet the requirements for lifelong editing. |
| Approach: | They propose an approach that selects editing layer based on the pattern matching degree of editing knowledge across different layers in language models. |
| Outcome: | The proposed method improves on GPT2-XL and GPT-J in lifelong editing compared to state-of-the-art methods . |
Copied to clipboard
| Challenge: | Recent studies have shown that using external knowledge such as pre-trained word embeddings or pre-train language models only achieved limited performance improvements but with huge computational overhead. |
| Approach: | They propose to incorporate external knowledge into neural topic modeling by pre-trained word embeddings (PWEs) or pre-train language models (PLMs) they propose to fine-tune the neural topic model on the target dataset and reduce the huge size of training data. |
| Outcome: | The proposed approach outperforms current state-of-the-art neural topic models and some topic modeling approaches enhanced with PWEs or PLMs on three datasets and greatly reduces the huge size of training data. |
Copied to clipboard
| Challenge: | Existing evaluation benchmarks focus on static evaluation of large multimodal models . existing evaluation paradigms neglect a critical aspect of clinical practice: longitudinal analysis . |
| Approach: | They propose a temporal perception and reasoning benchmark to assess models' temporal grounding and consistency. |
| Outcome: | ELTLM features a hierarchical task taxonomy comprising Temporal Perception QA and Temporal Reasoning QA. |
Copied to clipboard
| Challenge: | Multi-agent LLMs are rapidly moving from prototype to real-world use . network topology is a first-order security parameter in multi-aggent systems . |
| Approach: | They propose a framework for comparing topology-conditioned memory leakage in multi-agent LLM systems. |
| Outcome: | The proposed framework evaluates topology-conditioned memory leakage in multi-agent LLM systems. |
Copied to clipboard
| Challenge: | Recent advances in Large Language Models have fostered a new class of generative linguistic steganography, claim “provably secure” by theoretically aligning the stego distribution with the language model’s natural distribution. |
| Approach: | They propose a framework that transforms the detection task from semantic classification to a statistical audit of the sampling mechanism. |
| Outcome: | The proposed framework breaks the security of AC and Meteor with high detection accuracy, whereas state-of-the-art semantic steganalyzers degrade to random guessing. |
Copied to clipboard
| Challenge: | Large language models (LLMs) can call tools effectively, but they remain brittle in multi-turn execution. |
| Approach: | They propose a framework that converts execution errors into on-policy corrective supervision within the RL training loop. |
| Outcome: | The proposed framework improves the error recovery rate of Qwen3-8B by 5.7% absolute and overall accuracy by 4.0% on BFCL v4 Multi-Turn. |
Copied to clipboard
| Challenge: | Existing approaches to interactive facial image editing treat multi-turn editing as a sequence of successive single-turn edits, leading to attribute forgetting and error accumulation. |
| Approach: | They propose a framework for interactive facial image editing through dialogues based on the CelebA-HQ dataset and a benchmark dataset to evaluate this. |
| Outcome: | The proposed framework outperforms existing methods and improves existing ones. |
Copied to clipboard
| Challenge: | Existing methods for calibration of large reasoning models (LRMs) focus on clean inputs, leaving noise unexplored. |
| Approach: | They propose a confidence calibration framework for character-level noisy inputs that extracts uncertainty signals from both the empirical answer distribution and the model’s predictive distribution and integrates them via a learned calibrator. |
| Outcome: | Experiments on multiple mathematical reasoning benchmarks show that DisCal outperforms existing calibration methods under noisy inputs, reducing expected calibration error (ECE) by up to 39.21% and improving Area Under the Receiver Operating Characteristic Curve (AUROC) by 31.44%. |
Copied to clipboard
| Challenge: | Existing solutions to reasoning tasks require extensive human annotations or fail in scenarios with inconsistent responses. |
| Approach: | They propose a new method that enables LLMs to self-rank their responses without additional resources. |
| Outcome: | The proposed method improves reasoning performance of ChatGPT and GPT-4 with 13% improvement over existing methods. |
Copied to clipboard
| Challenge: | Existing methods of content moderation are infeasible due to over-editing and compromise the advertiser’s original semantic intent. |
| Approach: | They propose a framework to harmonize compliance with original intent preservation that integrates a data-driven framework and a curriculum to enforce compliance while maximizing semantic consistency. |
| Outcome: | The proposed framework outperforms state-of-the-art baselines on industrial datasets and on online A/B testing on industrial video. |
Copied to clipboard
| Challenge: | Program induction (PI) is a promising paradigm for using knowledge bases (KBs) to help large language models answer complex knowledge-intensive questions. |
| Approach: | They propose a plug-and-play framework that enables large language models to induce programs over any low-resourced KB. |
| Outcome: | Experiments show that KB-Plugin outperforms SoTA low-resourced PI methods with 25x smaller backbone LLM on large-scale and domain-specific KBs and even approaches the performance of supervised methods. |
Copied to clipboard
| Challenge: | Existing benchmarks for multimodal satirical cognition hinder evaluation of multimodal Sarcasm Understanding . lack of a unified benchmark for holistic satire cognition hampers evaluation of MSU . |
| Approach: | They propose a framework to decouple experts into orthogonal shared perception and private execution streams to physically block gradient interference between tasks. |
| Outcome: | The proposed framework achieves superior performance on DocMSU-PLUS. |
Copied to clipboard
| Challenge: | Existing methods for hallucination detection focus on implicit neural uncertainty or explicit symbolic reasoning, ignoring factual hallucinosities. |
| Approach: | They propose a framework that bridges neural features and symbolic judgments for hallucination detection by leveraging a "meta-judgment" process to map symbolic labels back into the feature space. |
| Outcome: | Extensive experiments on 4 public datasets, across 4 LLMs, against 8 baselines demonstrate the superiority of LaaB. |
Copied to clipboard
| Challenge: | Existing studies focus on the recognition step, while paying less attention to sign language translation. |
| Approach: | They propose a task-aware instruction network, namely TIN-SLT, for sign language translation, by introducing the isntruction module and the learning-based feature fuse strategy into a Transformer network. |
| Outcome: | The proposed system outperforms existing solutions on two benchmark datasets, PHOENIX-2014-T and ASLG-PC12, and outperformed previous best solutions by 1.65 and 1.42 in terms of BLEU-4. |
Copied to clipboard
| Challenge: | Existing methods for training large language models require additional annotations to adjust to shifted distributions. |
| Approach: | They propose an algorithm that allows LLMs and reward models to update alternatively via a min-max game to improve their alignment. |
| Outcome: | The proposed framework improves existing alignment baselines in terms of LLM helpfulness and harmlessness. |
Copied to clipboard
| Challenge: | Existing datasets suffer from outdated and insufficient challenging content, neglecting human-like reasoning, and limited reliability due to single-LLM generation. |
| Approach: | They propose a human-in-the-loop, multi-agent data generation framework that integrates reasoning-dense filters, multiagent collaboration, and human mathematicians’ evaluations to ensure the reliability and quality of the dataset. |
| Outcome: | The proposed framework improves accuracy and quality of the 2,000-synthesized datasets by integrating reasoning-dense filters, multi-agent collaboration, and human mathematicians’ evaluations. |
Copied to clipboard
| Challenge: | Semi-structured tables remain a major obstacle for automated data processing and analytics. |
| Approach: | They propose a technique called Loop Reference Decoding which identifies expandable groups and replicates each group using a concise loop over its repetitive region. |
| Outcome: | The proposed technique reduces output length from O(N M) to approximately O(K) Extensive experiments on HiTab and MultiHiertt show that it boosts Llama-2 and Mistral models by more than 20%, and GPT-4o by over 4%. |
Copied to clipboard
| Challenge: | Existing joint models for intent detection and slot filling show insufficient robustness . however, some small changes of inputs can fool the models to produce wrong predictions . |
| Approach: | They propose a joint adversarial training model that generates adversarials to attack the joint model and trains the model to defend against the adversarial examples. |
| Outcome: | The proposed model achieves significantly higher scores and improves robustness on two datasets. |
Copied to clipboard
| Challenge: | Existing language evaluation benchmarks for English are limited to English . lack of such benchmarks makes it difficult to replicate success in other languages . |
| Approach: | They introduce a large-scale Chinese language understanding evaluation benchmark . the benchmark uses a set of current state-of-the-art pre-trained Chinese models . |
| Outcome: | The first large-scale Chinese Language Understanding Evaluation (CLUE) benchmark is released . the benchmark evaluates models across a wide range of tasks on original Chinese text . existing language evaluation benchmarks are mostly limited to English . |
Copied to clipboard
| Challenge: | Existing models that use Large Language Models (LLMs) show superior performance in various tasks, but lack of controllability leads to unfocused conversations or task failure. |
| Approach: | They propose a standard operating procedure (SOP) framework to regulate dialogue flow by integrating Chain of Thought reasoning and supervised fine-tuning for SOP prediction. |
| Outcome: | The proposed method achieves a 27.95% improvement in action accuracy compared to baseline models based on GPT-3.5 and also shows notable gains for open-source models. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have driven the rise of agentic workflows . yet, how can we attribute performance gains to individual upgrades and their interactions? |
| Approach: | They propose a game-theoretic framework that models component upgrades as players and evaluates component coalitions to compute Shapley values. |
| Outcome: | The proposed framework provides interaction-aware attribution and recommendation for model allocation under a fixed workflow structure. |
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly used in daily applications, from content generation to code writing. |
| Approach: | They construct a longitudinal dataset of 412 human authors and 6,086 documents spanning 2012–2024 and compare them to trajectories generated by three representative LLMs. |
| Outcome: | The results show that LLMs produce greater lexical diversity but exhibit substantially reduced semantic and cognitive–emotional drift relative to humans. |
Copied to clipboard
| Challenge: | Existing methods for name tagging in low-resource languages or domains require extensive human efforts for training annotations. |
| Approach: | They propose a neural model for name tagging based on weakly labeled (WL) data. |
| Outcome: | The proposed model outperforms existing models in five low-resource languages and fine-grained food domains and shows that it is more efficient and efficient than existing models. |
Copied to clipboard
| Challenge: | Adaptive Retrieval-Augmented Generation (RAG) is an effective strategy to alleviate hallucination of large language models (LLMs). |
| Approach: | They propose a novel adaptive RAG model that extracts self-aware uncertainty of large language models from their internal states and invokes retrieval accordingly. |
| Outcome: | The proposed model outperforms existing adaptive RAG methods on complex and simple Question Answering datasets. |
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated remarkable proficiency in handling a wide range of tasks within the software engineering domain, but their ability to perform code migration—adapting code to different environments—remains underexplored. |
| Approach: | They propose a benchmark to evaluate large language models’ performance in handling code migration tasks. |
| Outcome: | The proposed benchmark comprises 922 data points across 19 Python and Java packages and offers three tasks to systematically evaluate code migration: identifying version-incompatible functions, determining function changes, and adapting code to target environments. |
Copied to clipboard
| Challenge: | Structured table extraction from unstructured text is critical for automating data processing tasks across industries where accuracy and reliability are paramount. |
| Approach: | They propose a natural language-based method for extracting structured tables from text . they use Python classes or SQL statements to explicitly construct table structures . |
| Outcome: | The proposed method improves F1 scores and mitigates hallucinations . it integrates with standard SQL databases and Python workflows, ensuring seamless deployment . |
Copied to clipboard
| Challenge: | Existing methods for evaluating large language models face challenges in managing semantic intricacies and optimizing the efficiency of the search process. |
| Approach: | They propose a framework that reconceptualizes test case generation as a strategic planning problem, leveraging Monte Carlo Tree Search. |
| Outcome: | Experiments on a range of LLM architectures show that the proposed framework achieves state-of-the-art attack success rates without sacrificing computational efficiency. |
Copied to clipboard
| Challenge: | Text-to-image models generate harmful content when unsafe prompts are submitted . authors propose a method to jailbreak text-to image models with safety guardrails . |
| Approach: | They propose a method to jailbreak text-to-image models with safety guardrails . they use a fine-tuned large language model to generate adversarial prompts based on unsafe prompts. |
| Outcome: | The proposed method bypasses safety guardrails and outperforms existing no-box attacks . the proposed method generates adversarial prompts efficiently after fine-tuning the model . |
Copied to clipboard
| Challenge: | Existing work on fake news detection does not consider the temporal shift issue caused by the rapidly-evolving nature of news data. |
| Approach: | They propose a framework to forecast temporal patterns of news data and guide detector to fast adapt to future distributions. |
| Outcome: | The proposed framework forecasts temporal distribution patterns and guides detector to fast adapt to future distribution. |
Copied to clipboard
| Challenge: | Relation extraction (RE) aims to identify the semantic relations between named entities in text. |
| Approach: | They propose a novel relation extraction model that encodes document information in terms of entity global and local representations and context relation representations. |
| Outcome: | The proposed model achieves superior performance on two public datasets for document-level RE. |
Copied to clipboard
| Challenge: | Existing methods for fact-checking lack coherence and context, whereas abstractive methods lack cohesion and context. |
| Approach: | They propose a framework that generates Chinese user-specific debunking passages . they propose to use a generative AI framework to generate context-sensitive responses . |
| Outcome: | The proposed framework generates Chinese user-specific debunking passages by iteratively refining outputs based on simulated user feedback. |
Copied to clipboard
| Challenge: | Existing Vision-language models are prone to hallucinating nonexistent entities or events and missing subtle but critical visual cues. |
| Approach: | They propose a Traffic VideoQA Benchmark that enforces a single structured decision pattern over each video question quadruple and provides actionable diagnostics that decompose failures into positive omission, positive swap, negative hallucination, mutual-exclusivity violation. |
| Outcome: | The proposed model detects true hazards when an accident occurs, and rejects plausible-but-false hypotheses under near-identical counterfactual scenes. |
Copied to clipboard
| Challenge: | Existing dense retrieval methods have achieved notable progress, but their effectiveness in legal case retrieval remains limited. |
| Approach: | They propose a self-evolving framework for rule-driven query rewriting that enhances BM25 without any parameter training. |
| Outcome: | The proposed framework outperforms non-evolutionary baselines, including human-designed rules and greedy rule selection, especially when powered by a high-capacity core LLM. |
Copied to clipboard
| Challenge: | Autoregressive models assign low probabilities to tokens that need corrections . grammatical error correction (GEC) is widely applied to natural language processing tasks . |
| Approach: | They propose to use a non-autoregressive model as an auxiliary model to train GEC models to correct grammatical errors in sentences. |
| Outcome: | The proposed method outperforms baselines on English and Chinese GEC tasks significantly. |
Copied to clipboard
| Challenge: | Knowledge Graph (KG) and attention mechanism have been demonstrated effective in introducing and selecting useful information for weakly supervised methods. |
| Approach: | They propose a paradigm to quantitatively evaluate the effect of attention and KG on bag-level relation extraction (RE) they propose to incorporate entity prior to KG-enhanced attention to improve RE performance . |
| Outcome: | The proposed model achieves significant improvements on two real-world datasets compared with three state-of-the-art baselines. |
Copied to clipboard
| Challenge: | Recent studies have validated that large language models (LLMs) are capable of solving some KBQA problems, but there has been little discussion on the differences in LLMs’ proficiency in formal languages used in semantic parsing. |
| Approach: | They propose to evaluate the understanding and generation ability of large language models (LLMs) to deal with differently structured logical forms by examining the inter-conversion of natural and formal language through in-context learning of LLMs. |
| Outcome: | The proposed model can understand formal languages as well as humans, but generating correct logical forms remains a challenge. |
Copied to clipboard
| Challenge: | Instruction-fine-tuned large language models (LLMs) under 14B parameters underperform on NLU tasks . we explore a framework to improve the NLU capabilities of LLMs . |
| Approach: | They propose to use Proximal Policy Optimization to improve NLU capabilities . they frame NLU as a reinforcement learning environment and optimize for reward signals . |
| Outcome: | The proposed framework outperforms supervised fine-tuning on GLUE and superGLUE tasks. |
Copied to clipboard
| Challenge: | FlowSearch is a multi-agent framework that actively constructs and evolves a dynamic structured knowledge flow to drive subtask execution and reasoning. |
| Approach: | They propose a multi-agent framework that actively constructs and evolves a dynamic structured knowledge flow to drive subtask execution and reasoning. |
| Outcome: | The proposed framework achieves competitive performance on GAIA, HLE, GPQA and TRQA benchmarks and is available to download. |
Copied to clipboard
| Challenge: | Event extraction (EE) is an essential task of information extraction, which aims to extract structured event information from unstructured text. |
| Approach: | They propose a tagging scheme and a model to form EE as word-word relation recognition using parallel grid tapping. |
| Outcome: | The proposed model achieves state-of-the-art on 3 overlapped and nested EE benchmarks and faster than baselines. |
Copied to clipboard
| Challenge: | Neural Processing Units (NPUs) are critical for AI infrastructure, but their development remains a bottleneck due to vendor-specific Domain-Specific Languages (DSLs). |
| Approach: | They propose a framework for NPU kernel development that bridges the gap in hardware-specific coding . compiler success on complex Level-2 kernels improves from 0% to 95.5%, they say . |
| Outcome: | The proposed framework bridges the gap in hardware-specific coding, showing a near-zero success rate on complex kernels. |
Copied to clipboard
| Challenge: | Visual Document Retrieval (VDR) is of importance in multimodal retrieval applications. |
| Approach: | They propose a two-stage pruning and merging frameworks that combine pruning and merge techniques to achieve higher compression rates. |
| Outcome: | The proposed framework outperforms existing methods on 29 visual document retrieval datasets. |
Copied to clipboard
| Challenge: | Existing multi-LLM collaboration systems often encounter scalability challenges when integrating new LLMs and tasks. |
| Approach: | They propose a Scalable Multi-LLM Collaboration System to coordinate multiple open-source LLMs. |
| Outcome: | The proposed system outperforms prevailing closed-source LLMs on eight mainstream benchmarks on multiple tasks. |
Copied to clipboard
| Challenge: | Current psychological LLMs are constrained by passive response mechanisms, limiting their capacity to deploy proactive strategies for psychological counseling. |
| Approach: | They propose a dataset that provides a multi-turn conversation dataset with interpretive labels including strategy decision logic and reaction attribution. |
| Outcome: | The proposed model significantly improves proactive questioning capacity, conversation depth, and response quality. |
Copied to clipboard
| Challenge: | Current event prediction methods lack rigorous uncertainty quantification, which limits their reliability for decision-making. |
| Approach: | They propose a conformal prediction framework that applies conformal predictions to event prediction to address this challenge. |
| Outcome: | The proposed framework guarantees coverage while improving efficiency on three public datasets. |
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) generate content that can be untruthful or harmful. |
| Approach: | They propose a method that leverages model feedback for alignment . they use a base language model to generate initial responses, critiqued and refined . |
| Outcome: | The proposed method outperforms strong baselines across diverse tasks and model sizes. |
Copied to clipboard
| Challenge: | Existing approaches to role-playing emotional companion products lack sustained personalization and contextual adaptability, limiting their effectiveness in real-world settings. |
| Approach: | They propose a virtual pet agent that can enhance user engagement through rich, dynamic pet behaviors and interactions tailored to individual preferences. |
| Outcome: | The proposed system has been deployed in a real-world, non-commercial product for 200 days and has demonstrated its effectiveness in practical applications. |
Copied to clipboard
| Challenge: | Existing methods to fix non-compliant images suffer from over-editing, destroying original intent and perceptual similarity. |
| Approach: | They propose a framework for the minimalist rectification of non-compliant image ads. |
| Outcome: | The proposed framework outperforms state-of-the-art baselines in both compliance and preservation of visual and commercial consistency. |
Copied to clipboard
| Challenge: | Open Domain Multi-Hop Question Answering (ODMHQA) is one of the most challenging tasks in Natural Language Processing (NLP) |
| Approach: | They propose a mechanism that leverages the intrinsic capabilities of Large Language Models to judge whether the generated answers are off-topic. |
| Outcome: | The proposed method reduces the occurrence of off-topic answers by nearly 13%, improving the performance in Exact Match (EM) by nearly 3% compared to the baseline method without the Dr3 mechanism. |
Copied to clipboard
| Challenge: | Existing RLHF frameworks face inference bottlenecks and complexity barriers restricting their accessibility for newcomers. |
| Approach: | They propose an open-source RLHF framework that can be used to train large language models. |
| Outcome: | The proposed framework achieves superior training efficiency with speedups ranging from 1.22 to 1.68 across different model sizes compared to state-of-the-art frameworks, while requiring significantly fewer lines of code for implementation. |
Copied to clipboard
| Challenge: | Document Information Extraction (DIE) has attracted increasing attention due to its various advanced applications in the real world. |
| Approach: | They propose a multi-modal generation method without predefined label categories for real-world scenarios using a spatial encoder and modal-aware mask module. |
| Outcome: | The proposed method can deal with complex documents that are hard to serialize into sequential order. |
Copied to clipboard
| Challenge: | Existing methods to train dense passage retrieval have a large data gap between upstream and downstream relevance. |
| Approach: | They propose a method to pre-train the dense retriever with the text relevance induced by hyperlinks within Web documents. |
| Outcome: | The proposed method outperforms existing methods under different scenarios and in the open-domain question answering domain. |