Papers by Xia Hu
Copied to clipboard
| Challenge: | Object categories are typically organized into a multi-granularity taxonomic hierarchy . traditional uni-modal approaches focus primarily on image features, revealing limitations in complex scenarios. |
| Approach: | They propose a framework that combines vision-language models with a deeper exploitation of the hierarchy. |
| Outcome: | The proposed framework shows significant improvements on 11 diverse visual recognition benchmarks. |
Copied to clipboard
| Challenge: | NLCO evaluates large language models for combinatorial optimization (CO) . existing evaluations emphasize relatively simple reasoning competencies . |
| Approach: | They propose a combinatorial optimization benchmark that evaluates large language models on CO reasoning. |
| Outcome: | The proposed model can handle combinatorial optimization without writing code or calling external solvers. |
Copied to clipboard
| Challenge: | Large language models have revolutionized the field of NLP by achieving state-of-the-art performance on various tasks. |
| Approach: | They investigate the membership inference attack by using model's API to determine if a sample was part of the training data. |
| Outcome: | The proposed model is able to identify if a sample was part of the training data and exploits its similarity and resistance to document modifications as potential MI signals on widely used datasets. |
Copied to clipboard
| Challenge: | Recent studies have focused on compressing pre-trained language models (PLMs) however, few studies have examined the impact of compression on generalizability and robustness of compressed models for out-of-distribution data. |
| Approach: | They propose to use knowledge distillation and pruning to reduce model generalization and generalization on out-of-distribution data. |
| Outcome: | The proposed compression techniques overfit on shortcut samples and generalize poorly on hard ones. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have remarkable out-of-domain generalizability to novel optimization tasks. |
| Approach: | They propose a series of instruction-tuned LLMs for molecule optimization that outperform state-of-the-art instruction-based LLM models. |
| Outcome: | mathttMuMOInstruct outperforms state-of-the-art LLMs on 5 in-domain and 5 out-of domain tasks. |
Copied to clipboard
| Challenge: | Existing mechanisms compromise ownership rights or raise data privacy concerns . existing mechanisms compromise security of released large language models . |
| Approach: | They propose a TaylorMLP to preserve the ownership of large language models by transforming the weights of LLMs into Taylor-series parameters instead of releasing original weights . |
| Outcome: | The proposed model preserves ownership of large language models and prevents their abuse by adjusting the generation speed and causing low-speed token generation. |
Copied to clipboard
| Challenge: | End-to-end aspect-based sentiment analysis (EASA) is a natural language processing task that requires a deep understanding of the running text. |
| Approach: | They propose a method to improve EASA with CCG supertags that carry syntactic and semantic information of the associated words. |
| Outcome: | The proposed approach outperforms baselines and achieves state-of-the-art results on all datasets. |
Copied to clipboard
| Challenge: | Large Language Models exhibit a confidence distortion problem on multichoice question-answering . Self-Ensemble solves this problem by splitting the choices into several groups . |
| Approach: | They propose a method that splits LLM choices into several groups and ensembles them to reach a final decision. |
| Outcome: | The proposed method outperforms standard inference and baseline methods on MCQA. |
Copied to clipboard
| Challenge: | Recent studies indicate that NLU models are prone to rely on shortcut features for prediction, without achieving true language understanding. |
| Approach: | They propose a shortcut mitigation framework to suppress NLU models from making overconfident predictions for samples with large shortcut degree. |
| Outcome: | The proposed framework suppresses the model from making overconfident predictions for samples with large shortcut degree. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have been used for selection and training of data for active learning. |
| Approach: | They propose an intuitive taxonomy that categorizes LLM-based active learning techniques and discuss the transformative roles they can play in the active learning loop. |
| Outcome: | The proposed model can generate entirely new data instances and provide more cost-effective annotations with fewer labeled data instances. |
Copied to clipboard
| Challenge: | Existing methods for instruction data selection have limitations such as relying on fragile external APIs, being affected by biases in GPT models, or reducing the diversity of the selected instruction dataset. |
| Approach: | They propose an industrial-friendly, expert-aligned and diversity-preserved instruction data selection method: Clustering and Ranking (CaR). |
| Outcome: | The proposed method outperforms Alpaca's existing methods by 32.1% in GPT-4 evaluations. |
Copied to clipboard
| Challenge: | Existing methods require training millions of architectures to estimate the accuracy of the search results. |
| Approach: | They propose a performance ranking method (RankNAS) that uses pairwise ranking and search space pruning to enlarge the search space. |
| Outcome: | The proposed method significantly accelerates NAS through pairwise ranking and search space pruning. |
Copied to clipboard
| Challenge: | Large language models (LLMs) produce natural language explanations, but they lack faithfulness and do not reflect the evidence the model uses to decide. |
| Approach: | They propose a model-agnostic framework that evaluates and improves the faithfulness of LLM explanations without token masking or task-specific heuristics. |
| Outcome: | The proposed framework improves faithfulness of large language models without masking or heuristics. |
Copied to clipboard
| Challenge: | Hierarchical text classification is a challenging task in natural language processing. |
| Approach: | They propose a method which integrates the results of diverse prompting strategies to promote LLMs’ reliability. |
| Outcome: | The proposed method boosts the performance of single prompting strategies and achieves SOTA results on three benchmark datasets. |
Copied to clipboard
| Challenge: | Anomaly detection (AD) is an important machine learning task with many real-world uses, including fraud detection, medical diagnosis, and industrial monitoring. |
| Approach: | They propose a benchmark that evaluates how large language models (LLMs) can help with NLP anomaly detection. |
| Outcome: | The proposed model can perform zero-shot detection without tasks-specific training, data augmentation and model selection, and it can suggest unsupervised AD models. |
Copied to clipboard
| Challenge: | Long context capability is a crucial competency for large language models as it mitigates the human struggle to digest long-form texts. |
| Approach: | They propose to evaluate 10+ state-of-the-art approaches for long context-capable LLMs. |
| Outcome: | The proposed methods are compared against 10+ state-of-the-art approaches across seven categories of long context tasks. |
Copied to clipboard
| Challenge: | Data augmentation (DA) is a key technique for enhancing model performance by diversifying training examples without the need for additional data collection. |
| Approach: | They examine various strategies that utilize LLMs for data augmentation, including a novel exploration of learning paradigms where LLM-generated data is used for diverse forms of further training. |
| Outcome: | The proposed approach addresses the primary open challenges faced by LLMs in the field of large language models and aims to serve as a comprehensive guide for researchers and practitioners. |
Copied to clipboard
| Challenge: | Existing work rely on compressing long contexts into soft prompts, but soft prompt compression encounters limitations in transferability . natural language (NL) prompts are incompatible with back-propagation, and NL prompts lack flexibility in imposing length constraints. |
| Approach: | They propose a framework that compresses long prompts into NL formatted Capsule Prompts. |
| Outcome: | The proposed framework reduces 81.4% of the original length, decreases inference latency up to 4.5x, and saves 80.1% of budget overheads while providing transferability across diverse LLMs and different datasets. |
Copied to clipboard
| Challenge: | Existing tuning methods for medical AI models are monologue-based . existing benchmarks are based on licensing exams or research articles . |
| Approach: | They propose a benchmark to expose limitations of monologue-based tuning for medical AI models . they use a large dialogue dataset to capture stepwise diagnostic reasoning . |
| Outcome: | The proposed model outperforms monologue-tuned models on a medical question answering task and improves accuracy on standard medical QA benchmarks. |
Copied to clipboard
| Challenge: | Existing methods to train student models on the generated outputs of teacher models are not efficient for ICL. |
| Approach: | They propose to align the output of smaller (student) models with that of larger (teacher) models by incorporating a ranking loss and aligning the token-level output distribution. |
| Outcome: | The proposed model outperforms baseline models on a variety of tasks involving language understanding, reasoning, and coding. |
Copied to clipboard
| Challenge: | Existing models for text style transfer struggle with complex styles . existing models perform well on simple styles like sentiment and formality . |
| Approach: | They propose a multi-agent self-check framework that includes a large language model as a planner for disentangling subtasks and expert agents for executing the subtask. |
| Outcome: | The proposed framework significantly improves style strength and content preservation on simple and complex style datasets. |
Copied to clipboard
| Challenge: | Existing methods for QA in industrial environments are inherently relational and often updated. |
| Approach: | They propose a framework that optimizes retrieval and generation through two components: Graph-aware Retrieval and evidence-constrained reinforcement learning. |
| Outcome: | Experiments on an internal advertising QA dataset show consistent gains across expert-judged dimensions including accuracy, completeness, safety, and URL validity. |
Copied to clipboard
| Challenge: | Existing strategies for spatial localization are limited due to their limited capacity to perceive positional data. |
| Approach: | They propose a location-based approach that leverages locational data to optimize interaction preferences. |
| Outcome: | The proposed approach achieves SOTA results across offline benchmarks and real-world evaluations. |
Copied to clipboard
| Challenge: | Speculative decoding (SD) has proven to be effective for autoregressive generation in large language models (LLMs), however its application to vision-language models (VLMs) remains relatively unexplored. |
| Approach: | They propose a Speculative Decoding framework for vision-language models that integrates a neural architecture search framework and target-aware supernet training to identify optimal interaction strategies. |
| Outcome: | DREAM-S achieves 3.85 speedup compared to baselines on well-established vision-language models. |
Copied to clipboard
| Challenge: | Lexically-constrained sequence decoding allows for explicit positive or negative phrase-based constraints to be placed on target output strings in machine translation or monolingual text rewriting tasks. |
| Approach: | They propose a vectorized dynamic beam allocation algorithm which extends work in lexically-constrained decoding to work with batching. |
| Outcome: | The proposed method improves on natural language inference, question answering and machine translation tasks by fivefold . |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly serving as evaluators in Natural Language Generation (NLG) tasks. |
| Approach: | They propose a framework that measures the discernment of Large Language Models (LLMs) across diverse NLG tasks. |
| Outcome: | The proposed framework provides quantitative discernment scores for LLMs across four NLG tasks. |
Copied to clipboard
| Challenge: | distributing LLMs without a proven track record like ‘meta-llama‘ or ‘qwen‘ rarely gains community traction. |
| Approach: | They propose a simple, efficient, yet specific recipe for a backdoor LoRA to be injected into task-enhancing LoRAs and examine the mechanisms of such infections. |
| Outcome: | The proposed model allows attackers to scale the distribution of compromised LoRAs with minimal effort by leveraging the rich pool of shared LoRA assets. |
Copied to clipboard
| Challenge: | et al., 2024) show large language models exhibit internal chain-of-thought, meaning they decompose and execute composite tasks layer-by-layer. |
| Approach: | They propose a method to decode hidden states using LogitLens . they also propose 'chain of thought' to decompose and execute composite tasks . |
| Outcome: | The proposed method decodes hidden states and shows consistent execution pattern . it opens avenues for fine-grained, instruction-level activation steering. |
Copied to clipboard
| Challenge: | Existing methods neglect stylistic modeling and rely on static thresholds, which greatly limits the detection performance. |
| Approach: | They propose a framework that enables stylistics-aware uncertainty quantification through conditional threshold estimation. |
| Outcome: | The proposed framework achieves an average improvement 11.34% in detection performance compared to baselines. |
Copied to clipboard
| Challenge: | Neural machine translation (NMT) is sensitive to domain shift, resulting in failure for sentences with large numbers of unknown words and lack of supervision for domain-specific words. |
| Approach: | They propose an unsupervised method which fine-tunes a pre-trained out-of-domain NMT model using a pseudo-in-domain corpus. |
| Outcome: | The proposed method improves in five domains without using in-domain parallel sentences and up to 2 BLEU over strong back-translation baselines. |
Copied to clipboard
| Challenge: | Concept-based models lack explainability and need predefined concepts and manual labeling in molecular science. |
| Approach: | They propose a framework that leverages Large Language Models to generate and label predictive molecular concepts without human input. |
| Outcome: | The proposed framework outperforms existing models on several benchmarks while maintaining explainability and allowing easy intervention. |
Copied to clipboard
| Challenge: | Retrieval-Augmented Generation (RAG) enhances large language models by incorporating external knowledge, yet traditional singleround retrieval struggles with complex multistep reasoning. |
| Approach: | They propose a framework that introduces path-centric reward shaping for agentic RAG training. |
| Outcome: | The proposed framework improves on existing methods with an average accuracy gain of 7.7 points. |
Copied to clipboard
| Challenge: | Existing LLMs rely on surface-level similarity or manual heuristics to evaluate personalization . Existing evaluation protocols for personalization are lacking sufficient data-driven validation. |
| Approach: | They propose a benchmark to assess personalization by mining CIPDs to quantify individual preferences. |
| Outcome: | The proposed benchmark provides a more comprehensive and discriminative standard than generic metrics. |
Copied to clipboard
| Challenge: | Analogical reasoning is a unique ability of humans to address unfamiliar challenges by transferring strategies from relevant past experiences. |
| Approach: | They propose to use self-generated random examples to improve performance on a variety of reasoning tasks by incorporating relevant examples from relevant past experiences. |
| Outcome: | The proposed methods achieve comparable or even better performance on GSM8K with random biological examples. |
Copied to clipboard
| Challenge: | Large Foundation Models (LFMs) have transformed the landscape of AI research and day-to-day life. |
| Approach: | They propose a framework that delineates GUI agents' perception, reasoning, planning, and acting capabilities. |
| Outcome: | The proposed framework delineates their perception, reasoning, planning, and acting capabilities. |
Copied to clipboard
| Challenge: | Recent years have seen an unprecedented surge in the development and application of large language models (LLMs) however, the development of LLMs is a complex endeavor, requiring substantial investments in terms of financial and computational resources. |
| Approach: | They propose a mechanism wherein a unique key prompt is embedded within the LLM to protect it from unauthorized access and potential theft. |
| Outcome: | The proposed protection can protect the model without significantly impacting its original function. |
Copied to clipboard
| Challenge: | a library to facilitate the development, use, and evaluation of large language models (LLMs) is presented. |
| Approach: | They propose a unified library to facilitate the development, use and evaluation of large language models (LLMs). |
| Outcome: | The proposed library is based on extensive experiments in a variety of evaluation settings. |
Copied to clipboard
| Challenge: | Existing approaches to distilling large language models (LLMs) are inefficient and generate excessively long chain-of-thought reasoning even for inputs that admit concise solutions. |
| Approach: | They propose a distillation framework that empowers non-reasoning LLMs to think only when necessary. |
| Outcome: | The proposed framework reduces reasoning length up to 71% with minimal accuracy loss while preserving accuracy. |
Copied to clipboard
| Challenge: | Empirical results demonstrate that our method improves dialogue summarization, achieving a 1.5% increase in ROUGE scores and a 0.3% improvement in BERT scores in few-shot settings. |
| Approach: | They propose Mutual Reinforcing Data Synthesis (MRDS) within large language models to enhance few-shot dialogue summarization task. |
| Outcome: | Empirical results show that the proposed method improves dialogue summarization, achieving a 1.5% increase in ROUGE scores and a 0.3% improvement in BERT scores in few-shot settings. |
Copied to clipboard
| Challenge: | Existing detection methods rely on white-box assumptions or require prohibitively high computational and interaction costs, rendering them ineffective under practical black-box scenarios. |
| Approach: | They propose a framework that evades black-box detection methods based on style transfer by using style-injection supervised fine-tuning and direct preference optimization to shape distributions of AI-generated texts to resemble those of human-written texts. |
| Outcome: | The proposed framework achieves an average Attack Success Rate (ASR) of 92%, surpassing the strongest baselines by an average of 24% while maintaining superior linguistic quality. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly adopted across real-world applications . traditional evaluations rely on expensive, domain-specific ground-truth labels . obtaining labeled data is expensive, time-consuming, and often requires domain expertise . |
| Approach: | They propose a ground-truth-free evaluation framework focused on reasoning consistency and instruction following. |
| Outcome: | The proposed framework outperforms existing label-free methods, including majority voting, triplet ranking, and peer-review approaches. |
Copied to clipboard
| Challenge: | Existing methods for molecule optimization fail to capture property-specific objectives . a series of instruction-tuned LLMs can perform targeted property-specific optimization . |
| Approach: | They propose a set of instruction-tuned LLMs that can perform targeted property-specific optimization. |
| Outcome: | a new instruction-tuned LLM can perform targeted property-specific optimization. |