Papers by Zixuan Wang
Copied to clipboard
| Challenge: | Existing adaptive methods focus on a single axis, overlooking evidence need and reasoning depth are only partially correlated. |
| Approach: | They propose a dual-axis routing framework that separates retrieval necessity from reasoning necessity under a user-defined cost–quality trade-off. |
| Outcome: | The proposed framework reduces token usage and latency while improving answer quality over strong baselines. |
Copied to clipboard
| Challenge: | Temporal Knowledge Graphs (TKGs) are used in many different areas of research. |
| Approach: | They propose to use a beam search policy to induce multiple clues from historical facts . they propose to adopt a graph convolution network based sequence method to deduce answers from clues . |
| Outcome: | The proposed model can predict future facts in two stages, Clue Searching and Temporal Reasoning. |
Copied to clipboard
| Challenge: | Existing methods for retrieval-based in-context learning ignore model biases and fail to retrieve the most appropriate demonstrations for different LLMs. |
| Approach: | They propose a model-specific demonstration retrieval method that considers the biases of different LLMs at inference time. |
| Outcome: | The proposed method improves performance on seen and unseen tasks with multi-scale inference LLMs by up to 41.2%. |
Copied to clipboard
| Challenge: | Current fine-grained error analyses do not ground the errors to the reasons why the annotated text spans are erroneous. |
| Approach: | They use a bi-directional grounding scheme to ground erroneous text in two directions . if the error spans of both directions are consistent, the explanation is valid . |
| Outcome: | The proposed grounding process improves translation error detection significantly. |
Copied to clipboard
| Challenge: | Modern large-scale Pre-trained Language Models focus on text reconstruction, but have not sought to learn latent-level interpretable representations of sentences. |
| Approach: | They propose a new pre-training objective that enables the model to learn latent types . the objective allows the model a self-supervised way to extract sentence-level keywords . |
| Outcome: | The proposed model learns interpretable latent type categories without external knowledge and improves downstream tasks. |
Copied to clipboard
| Challenge: | Existing methods for achieving this alignment involve employing reinforcement learning from human feedback (RLHF) Existing approaches involve using RLHF to fine-tune LLMs based on human labels . however, RLRF is susceptible to instability during fine- tuning and presents challenges in implementation. |
| Approach: | They propose to use reinforcement learning from human feedback to fine-tune large language models with human preferences to achieve precise control of model behavior. |
| Outcome: | Experiments show that RAHF can be used to capture and manipulate representations to align with a broad spectrum of human preferences or values rather than being confined to a single concept or function. |
Copied to clipboard
| Challenge: | Recent industrial credit scoring models rely heavily on manually tuned statistical learning methods due to the complexity of heterogeneous financial data and the challenge of modeling evolving creditworthiness. |
| Approach: | They propose a framework that reformulates credit scoring as a multi-scale sequential learning problem. |
| Outcome: | FinLangNet improves KS and bad debt rate by 6.3 pp in real world deployments. |
Copied to clipboard
| Challenge: | Instruction-tuned language models (LMs) are increasingly deployed as interactive services across various applications. |
| Approach: | They propose a benchmark to evaluate models' ability to follow the instruction hierarchy by comparing their models to a set of benchmarks. |
| Outcome: | The proposed benchmark covers 3,538 examples across nine tasks covering cases where instructions in different priorities either align or conflict. |
Copied to clipboard
| Challenge: | Existing methods to modify LMs suffer from sub-optimal locality, where irrelevant neighborhood examples can be adversely influenced. |
| Approach: | They propose to use a model editing method to modify specific examples in LMs to improve locality and reasoning capability by directing the hidden state of edit example towards spaces where semantics are sparse. |
| Outcome: | The proposed method improves locality and reasoning capability on two datasets. |
Copied to clipboard
| Challenge: | Existing methods for inference-time steering fail to be effective, utility-preserving and training-efficient due to rigid, one-size-fits-all designs and limited adaptability. |
| Approach: | They propose a steering framework that decomposes inference-time steering into two stages . they propose 'conditional steering' mechanism that preserves model utility by avoiding unnecessary steering . a 'mixture-of-Steering-Experts' mechanism captures multimodal nature of desired steering behaviors . |
| Outcome: | The proposed framework outperforms the state-of-the-art methods on safety and truthfulness benchmarks. |
Copied to clipboard
| Challenge: | Existing approaches to identifying inappropriate content require extensive human-labeled data and lack cross-issue generalization. |
| Approach: | They propose a reasoning-enhanced multimodal large language model (MLLM) pretraining paradigm for unified inappropriate content detection. |
| Outcome: | The proposed model improves the MLLM's performance in both zero-shot and supervised fine-tuning settings and shows strong generalization capabilities to emergent, previously unseen issues. |
Copied to clipboard
| Challenge: | Existing studies on parameter-efficient fine-tuning (PEFT) have produced many state-of-the-art results by adapting LLMs to new tasks, but it requires substantial training data and time to enhance model performance. |
| Approach: | They propose a parameter-efficient fine-tuning framework which efficiently transfers knowledge from a small expert model to a target large model via embedding layers. |
| Outcome: | The proposed framework accelerates domain-specific fine-tuning, improves model performance and remains robust across diverse model families and PEFT methods. |
Copied to clipboard
| Challenge: | Label-free reinforcement learning enables large language models to improve reasoning capabilities . but as training maximizes self-consistency, output diversity collapses, authors say . authors propose a framework where a single model alternates between generator and verifier roles . |
| Approach: | They propose a framework where a model alternates between generator and verifier roles, bootstrapping each other. |
| Outcome: | Experiments show that CoVerRL outperforms label-free baselines on reasoning benchmarks . the framework can be used to improve reasoning abilities without ground-truth supervision . |
Copied to clipboard
| Challenge: | Existing methods for event prediction are incomplete and noisy. |
| Approach: | They propose to use news-related event schemas to extract newsworthy events . they build a demo website and include a video demonstrating the framework . |
| Outcome: | The proposed framework can be applied to a wide variety of newsworthy scenarios. |
Copied to clipboard
| Challenge: | Existing benchmarks focus on narrow tasks and leave a fundamental question unanswered . Existing models only focus on specific tasks, requiring rigorous reasoning and knowledge . |
| Approach: | They propose a benchmark to connect theoretical foundations with practical business knowledge and applications. |
| Outcome: | The benchmark systematically evaluates both open-source and commercial LLMs . it reveals how theoretical knowledge translates into practical performance in business . |
Copied to clipboard
| Challenge: | Existing knowledge graph question answering methods rely on LLM-induced type systems with inconsistent granularity or perform multi-hop reasoning without explicit target-type constraints. |
| Approach: | They propose a type-constrained knowledge graph question answering framework that reasons over a relation-centric ontology graph. |
| Outcome: | The proposed framework achieves state-of-the-art and produces ontology-grounded reasoning chains with substantial Hit@1 gains. |
Copied to clipboard
| Challenge: | Empirical evidence indicates that Large Language Models exhibit spontaneous cross-lingual alignment in Information Extraction (IE) however, a significant imbalance across languages persists, highlighting an underlying deficiency. |
| Approach: | They propose a code LLM with advanced cross-lingual and multilingual capabilities for universal IE that standardizes the representation of multilingual schemas using Python classes and conducts IE alignment instruction tuning on translated instance prediction task. |
| Outcome: | The proposed model surpasses ChatGPT and SoTA by 30.17% without training in 29 unseen languages and significantly improves cross-lingual IE transferability. |
Copied to clipboard
| Challenge: | Current methods for evaluating LLMs’ veracity are limited by the need for extensive human labor, test data contamination, or limited scope, hindering efficient and effective exposure of errors. |
| Approach: | They propose a framework that extracts fact triplets to generate diverse question types using rule-based natural language processing techniques. |
| Outcome: | The proposed framework can trigger factual errors in up to 55% of questions in large LLMs while maintaining coverage of questions. |
Copied to clipboard
| Challenge: | Existing approaches to longterm memory rely on rigid retrieval granularity, accumulation-heavy maintenance strategies, and coarse-grained update mechanisms. |
| Approach: | They propose a framework that leverages coordinated agents to manage memory across multiple granularities. |
| Outcome: | The proposed framework outperforms state-of-the-art benchmarks while reducing token consumption by approximately 80%. |
Copied to clipboard
| Challenge: | Logical reasoning is a central capability in LLMs, but understanding their abilities remains poorly understood. |
| Approach: | They propose to refine the knowledge representations of each reasoning type in LLMs to encourage complementarity . they propose to use complementary loss and subspace constraint loss to enhance complementarities . |
| Outcome: | The proposed framework encourages complementarity between the different types of reasoning in LLMs. |
Copied to clipboard
| Challenge: | Despite the development of many subdirections, Cross-Document Cross-Lingual NLI remains largely unexplored. |
| Approach: | They propose a novel paradigm that extends traditional NLI capabilities to multi-document, multilingual scenarios by integrating RST-enhanced graph fusion with interpretability-aware prediction. |
| Outcome: | The proposed method improves on existing models and document-level NLI to multi-document, multilingual scenarios. |
Copied to clipboard
| Challenge: | Existing context condensing methods cannot accurately understand the full context, as there is a considerable amount of information loss in the condensed process. |
| Approach: | They propose a framework to extend the fixed context length of any decoder-only LLM by distilling crucial information from long sequences. |
| Outcome: | The proposed framework extends the fixed context length of any decoder-only LLM, allowing it to focus on relevant information from very long sequences. |
Copied to clipboard
| Challenge: | Despite their impressive capabilities, LLMs struggle with complex computations and delivering accurate, timely information. |
| Approach: | They propose a framework that prompts LLM agents to ask questions when they encounter obstacles due to unclear instructions and an automated evaluation tool called ToolEvaluator. |
| Outcome: | The proposed framework outperforms existing frameworks for tool learning in the Noisy ToolBench. |
Copied to clipboard
| Challenge: | a new system extracts supporting and refuting claims from COVID-19 related news . the system is publicly available at GitHub and DockerHub, with complete documentation. |
| Approach: | They propose a COVID-19 Claim Radar system that extracts supporting and refuting claims . the system leverages Wikidata as the hub to consolidate coreferential knowledge elements . |
| Outcome: | The system extracts supporting and refuting claims from COVID-19 pandemic information . it leverages Wikidata as the hub to merge coreferential knowledge elements . |
Copied to clipboard
| Challenge: | Experimental results show that events can greatly improve the quality of KG embeddings on multiple downstream tasks. |
| Approach: | They propose an event-enhanced KG embedding model that incorporates events into KGs . they first incorporate event nodes by building a heterogeneous network with event argument links . |
| Outcome: | The proposed model incorporates event nodes into the original knowledge graphs . it can be used to fuse event information into the KG embeddings on multiple tasks . |
Copied to clipboard
| Challenge: | Effective content moderation is essential for video platforms to safeguard user experience and uphold community standards. |
| Approach: | They propose a method to transform a generative MLLM into a multimodal classifier using minimal discriminative training data. |
| Outcome: | The proposed method improves F1 score by 66.50% over traditional classifiers while requiring only 2% of the fine-tuning data. |
Copied to clipboard
| Challenge: | Existing frameworks for fine-grained few-shot entity extraction are difficult to implement in the chemical domain due to the information overload of scientific papers. |
| Approach: | They propose a sequence-to-sequence based few-shot entity extraction approach . it uses a seq2seq entity extractor and a self-validation module to reconstruct original input sentence . |
| Outcome: | The proposed framework achieves 8.26% and 6.84% performance gains on two datasets. |
Copied to clipboard
| Challenge: | Prompts are essential for guiding model output and influencing content generation. |
| Approach: | They propose to attack models with prompt leakage and prompt jailbreak attacks . they summarize the experimental setups of these methods and explore the relationship between prompt threats and prompt injection attacks. |
| Outcome: | The proposed methods summarize the experimental setups and examine the relationship between prompt threats and prompt injection attacks. |
Copied to clipboard
| Challenge: | Existing approaches that integrate LLMs and KGs either underutilize the reasoning abilities of LLM or suffer from prohibitive computational costs due to tight coupling. |
| Approach: | They propose a framework that can strike a balance between performance and efficiency via an iterative paradigm. |
| Outcome: | The proposed framework can strike a balance between performance and efficiency via an iterative paradigm. |
Copied to clipboard
| Challenge: | Entity Alignment (EA) is a crucial step in unifying data from heterogeneous sources and plays a critical role in data-driven AI applications. |
| Approach: | They propose a framework that incorporates large language models to improve EA. |
| Outcome: | The proposed framework incorporates large language models (LLMs) to improve EA accuracy while preserving efficiency. |
Copied to clipboard
| Challenge: | Existing datasets focus on relation extraction between two entities in one sentence, and some focus on cross-sentence relationships. |
| Approach: | They propose to use a Chinese multi-party dialogue dataset for automatic extraction of dialogue-based character relationships. |
| Outcome: | The proposed dataset extracts relationships between 140 entities on the CRECIL corpus and another existing relation extraction corpus. |
Copied to clipboard
| Challenge: | Summarizing biomedical discovery from genomics data is done manually but is slowing down the progress of scientific discovery. |
| Approach: | They propose a novel task of generating sentences to summarize a genomics data matrix using neural text generation. |
| Outcome: | The proposed model improves on the previous models and can be applied to other biomedical and natural language processing applications. |
Copied to clipboard
| Challenge: | Parameter Efficient Fine-Tuning (PEFT) has gained significant attention for its ability to achieve competitive results while updating only a small subset of trainable parameters. |
| Approach: | They propose a new approach to fine-tuning neural models that scales and biases the representation produced at each layer. |
| Outcome: | The proposed approach reduces the number of trainable parameters by a factor of 25,700 compared to full parameter fine-tuning and by . 32 compared with LoRA. |
Copied to clipboard
| Challenge: | Existing methods implicitly model time constraints by learning time-aware embeddings of questions and candidate answers, which is far from understanding the question comprehensively. |
| Approach: | They propose a temporal-based temporal programming method that leverages the in-context learning ability of Large Language Models to understand combinatory time constraints in questions. |
| Outcome: | The proposed method outperforms existing methods on multiTQ and CronQuestions datasets and is highly efficient on multi-level questions. |
Copied to clipboard
| Challenge: | Attention-based neural machine translation models selectively focus on specific source positions to produce a translation. |
| Approach: | They propose to replace the attention component with a neural hidden Markov model that selectively focuss on specific source positions to produce a translation. |
| Outcome: | The proposed model performs better than the state-of-the-art attention-based models on the GermanEnglish and ChineseEnglish translation tasks. |
Copied to clipboard
| Challenge: | Existing methods for finetuning and retrieval-augmented generation suffer from hallucination risk and semantic drift. |
| Approach: | They propose a framework for a dual-retriever based on the legal syllogism and the nature of different legal data. |
| Outcome: | The proposed framework mitigates hallucinations while improving explainability of legal reasoning. |
Copied to clipboard
| Challenge: | Current evaluation methods for large language models rely on static benchmarks . limited knowledge coverage and fixed difficulties hinder the targeted optimizations resulting in superficial evaluations of LLMs - a problem that has been addressed by JudgeAgent . |
| Approach: | They propose a knowledge-driven and dynamic evaluation framework for large language models . judgeAgent leverages LLM agents equipped with context graphs to traverse knowledge structures . |
| Outcome: | The proposed framework can achieve comprehensive evaluations and facilitate effective model iterations. |
Copied to clipboard
| Challenge: | Existing solutions to address inefficiency in large-scale integrity enforcement on short-form video platforms require multiple specialized vertical modules . |
| Approach: | They propose a lightweight risk-aware routing framework that selectively releases low-risk content while dispatching high-risk instances to appropriate vertical modules. |
| Outcome: | The proposed framework selectively releases low-risk content while dispatching high-risk instances to appropriate vertical modules. |
Copied to clipboard
| Challenge: | Current Large Language Models focus on syntax and ignore the vital semantic signals in code comments. |
| Approach: | They propose a Multi-Agent framework for COmment-guided code Refactoring that populates original code with precise comments to provide necessary semantic guidance for subsequent refactoring. |
| Outcome: | The proposed framework significantly improves code quality and achieves higher developer acceptance compared to baselines. |
Copied to clipboard
| Challenge: | Existing federated learning frameworks require substantial data and computational resources to develop large language models. |
| Approach: | They propose a method that distributes a quantized version of the model’s parameters during training and combine it with a popular fine-tuning method to significantly reduce communication costs. |
| Outcome: | The proposed method enables accurate estimations for parameter updates while preventing clients from accessing a model whose performance is comparable to the centrally hosted one. |