Papers by Qiang Liu
Copied to clipboard
| Challenge: | Hallucination is a significant barrier to the effective application of Large Language Models (LLMs). |
| Approach: | They propose an Attention-Guided SElf-Reflection approach for hallucination detection in Large Language Models. |
| Outcome: | The proposed method significantly outperforms existing methods in zero-shot hallucination detection on four widely-used LLMs across three different halluciation benchmarks. |
Copied to clipboard
| Challenge: | Existing methods focus on replicating dialogues in textual form, neglecting the role’s voice traits as a crucial effect in interaction, which tends to be more immersive experiences in realistic scenarios. |
| Approach: | They propose a first seamless speech-language personality interaction model to achieve immersive RPAs with low latency. |
| Outcome: | The proposed model exhibits role-specific personality traits and vocal traits throughout the interaction, enabling a mixture of speech and language responses. |
Copied to clipboard
| Challenge: | Object hallucination has been an Achilles’ heel which hinders the broader applications of large vision-language models (LVLMs). |
| Approach: | They propose a logical closed loop-based framework for Object Hallucination Detection and Mitigation that uses logical consistency probing to raise questions with logical correlations to determine hallucinations. |
| Outcome: | The proposed method can be applied to all existing LVLMs and is effective and general. |
Copied to clipboard
| Challenge: | Existing approaches to learn item embeddings for categorical features are limited by the frequency of items in real-world. |
| Approach: | They propose a method that transfers knowledge from frequent items to rare items by introducing an auxiliary transfer loss. |
| Outcome: | The proposed framework significantly boosts the performance on a variety of NLP and recommendation system tasks. |
Copied to clipboard
| Challenge: | Existing methods for large language models suffer from two major issues: in-domain data are scarce compared with general domain-agnostic data. |
| Approach: | They propose a task-oriented in-domain data augmentation framework that uses in- domain data selection and task-orientated synthetic passage generation to adapt LLMs to two domains: advertisement and math. |
| Outcome: | The proposed framework improves LLM performance by 8% in the advertisement domain and 7.5% in the math domain. |
Copied to clipboard
| Challenge: | Existing legal language models struggle with dynamic courtroom interactions, resulting in overfitting to standardized legal tasks. |
| Approach: | They propose a new adversarial evolutionary approach for agents that performs dynamic knowledge learning and evolution through structured adversarials in a simulated courtroom program. |
| Outcome: | The proposed approach outperforms existing LLM-based models in three critical dimensions: cognitive agility, professional knowledge, and logical rigor. |
Copied to clipboard
| Challenge: | Vision-Language Models (VLMs) have demonstrated impressive capabilities in code generation across various domains, but their ability to replicate complex, multi-panel visualizations remains largely unassessed. |
| Approach: | They propose a large-scale benchmark to evaluate chart generation from large- scale raw data and assess iterative code refinement in a multi-turn conversational setting. |
| Outcome: | The new benchmark evaluates 14 leading VLMs on real-world data and shows they struggle with complex plot structures and authentic data. |
Copied to clipboard
| Challenge: | Existing benchmarks for lexical substitution (LS) are limited and limited in coverage . despite extensive research on Lexical Substitution in various languages, there is limited evidence for LS in Chinese. |
| Approach: | They propose to use human and machine collaboration to construct a Chinese LS dataset . they combine four unsupervised LS methods to generate candidate substitutes . |
| Outcome: | The proposed method outperforms existing benchmarks on the Chinese lexical substitution task. |
Copied to clipboard
| Challenge: | Existing frameworks prioritize structural architectures and role assignments but neglect granular mechanics of agent collaboration. |
| Approach: | They propose to use centralized governance, instructor-led participation, ordered interaction patterns to optimize task accuracy and computational efficiency. |
| Outcome: | The proposed model improves task accuracy and computational efficiency under two context-dependent scenarios. |
Copied to clipboard
| Challenge: | Existing methods define important nodes as important and target them for attacks if the model treats nodes’ predictive influence more uniformly . Existing approaches target high predictive influence nodes but are vulnerable to malicious message injection attacks. |
| Approach: | They propose a defense mechanism that encourages the model to learn graph representations where nodes with varying importance have a more uniform influence on predictions. |
| Outcome: | Extensive experiments on the Twitter and Weibo datasets show that similarizing the predictive Influence of nodes with Contrastive Learning significantly enhances resistance against LLM-driven message injection attacks. |
Copied to clipboard
| Challenge: | Existing approaches to personalized text generation rely on retrieval-augmented generation and parameter-efficient fine-tuning. |
| Approach: | They propose a training-free framework that disentangles and represents personalized writing style as a vector in LLM’s activation-space. |
| Outcome: | The proposed framework achieves 8% relative improvement in personalized generation while reducing storage requirements by 1700 over PEFT method. |
Copied to clipboard
| Challenge: | Large Vision-Language Models (LVLMs) have impressive capabilities across visual tasks, yet they remain hindered by the persistent challenge of hallucinations. |
| Approach: | They propose a novel approach that dynamically adapts decoding strategies by evaluating the correctness of the model’s attention on image tokens to distinguish the correct attention. |
| Outcome: | Extensive experiments show that the proposed approach outperforms existing decoding methods across multiple mainstream benchmarks, effectively mitigating hallucinations in LVLMs. |
Copied to clipboard
| Challenge: | Existing knowledge editing techniques show limitations when applied to multi-hop reasoning . residual single-hop knowledge causes edited models to revert to original answers . |
| Approach: | They propose a knowledge editing method that incorporates a Knowledge Erasure mechanism for Large language model Editing (KELE) they propose an erasure function for residual knowledge and an injection function for new knowledge . |
| Outcome: | The proposed method significantly improves multi-hop reasoning capability of edited models. |
Copied to clipboard
| Challenge: | Existing language models are inadequate in reasoning, according to studies . a new reasoning pre-training paradigm is based on pretraining language models with programs . |
| Approach: | They propose a reasoning pre-training paradigm that empowers language models to harvest reasoning knowledge possessed by program executors. |
| Outcome: | The proposed reasoning pre-training paradigm can boost models' reasoning skills . it can be instantiated by different kinds of program executors and run on a single database . |
Copied to clipboard
| Challenge: | Existing studies have not identified a link between video caption evaluation and T2V generation. |
| Approach: | They propose a video caption evaluation scheme specifically designed for T2V generation that integrates video annotation with caption evaluation. |
| Outcome: | The proposed system is agnostic to any particular caption format and can be used for training. |
Copied to clipboard
| Challenge: | Existing methods for text-to-image synthesis lack systematic error analysis and refinement strategies, resulting in limited reliability and effectiveness. |
| Approach: | They propose a plug-and-play multi-agent system called GenPilot that integrates error analysis, clustering-based adaptive exploration, fine-grained verification and a memory module for iterative optimization. |
| Outcome: | The proposed method improves text consistency and structural coherence on images with a plug-and-play system. |
Copied to clipboard
| Challenge: | Existing studies on fact verification lack a high-quality dataset for explainability . existing systems lack evidence retrieval and veracity prediction, limiting the ability to verify a claim . |
| Approach: | They propose a dataset for multi-hop explainable fact verification that summarises and modifies Wikipedia documents. |
| Outcome: | The proposed dataset aims to improve the accuracy of multi-hop explainable fact verification systems. |
Copied to clipboard
| Challenge: | Existing methods for generating adversarial code examples face challenges such as limted availability of substitute variables and the creation of adversarials with noticeable perturbations. |
| Approach: | They propose a search seed based on historical attacks to find adversarial substitutes . they employ a pre-trained variable name encoder to map the search seed to a continuous vector space . |
| Outcome: | The proposed approach outperforms baseline methods in terms of ASR and QT. |
Copied to clipboard
| Challenge: | Existing approaches to constraint-aware planning fail to enhance the model’s intrinsic focus on constraints. |
| Approach: | They propose a constraint-aware reinforcement learning framework that encourages constraint focus and penalizes neglect of LLMs. |
| Outcome: | The proposed framework outperforms existing frameworks and state-of-the-art reasoning models in a number of real-world applications. |
Copied to clipboard
| Challenge: | Existing models have a performance gap of 20% between classifying fake news and real news, making them less suitable for practical deployment. |
| Approach: | They propose to adopt an LLM to generate fake news in three different styles, which are later incorporated into the training set to augment the representation of fake news. |
| Outcome: | The proposed model achieves state-of-the-art performance on two benchmark datasets and improves detection accuracy by 24.02% and 11.06% respectively. |
Copied to clipboard
| Challenge: | Existing graph-based models excel at capturing structural information within TKGs but lack semantic comprehension abilities. |
| Approach: | They propose a plug-and-play module to enhance the performance of graph-based TKG models by exploring high-order histories step-by-step. |
| Outcome: | Experiments on three datasets and backbones show that CoH is effective in capturing high-order historical information for LLMs. |
Copied to clipboard
| Challenge: | Open domain question answering systems often rely on information retrieved from large collections of text to answer questions. |
| Approach: | They evaluate and benchmark three powerful Large Language Models with a dataset . they find that 25% of unambiguous open domain questions can lead to conflicting contexts . |
| Outcome: | The proposed model can't be used to answer questions with conflicting contexts . it can be fine tuned to provide richer information into the model's training . |
Copied to clipboard
| Challenge: | Recent Large Reasoning Models (LRMs) have demonstrated the ability to generate long chains of thought (LongCoT) LongCoT still faces challenges such as redundancy and logical incoherence. |
| Approach: | They propose a neural-symbolic reasoning framework that generates chains of thought . they propose Logic-Thinker, which transforms symbolic solvers into chains of thoughts . |
| Outcome: | The proposed framework outperforms models fine-tuned with ThinkerCoT on logic reasoning tasks. |
Copied to clipboard
| Challenge: | Named entity recognition (NER) is the recognition of entities with specific meanings in the text, mainly including person, organization, location, etc. |
| Approach: | They propose an edge-aware node joint update module and introduce a node-awful edge update module to explore hidden in structured information and solve the wrong dependency label information to some extent. |
| Outcome: | The proposed model can exploit the structured information on the dependency tree to improve the recognition of long entities. |
Copied to clipboard
| Challenge: | Existing approaches to tool invocation are often unnecessarily long and require lengthy reasoning paths. |
| Approach: | They propose a framework for stepwise code generation that improves LLM tool invocation . they incorporate two distinct process rewards: the On-the-spot and the Latent Reward . |
| Outcome: | The proposed framework improves LLM tool invocation by leveraging the concise nature of code. |
Copied to clipboard
| Challenge: | State-of-the-art NLP models can be fooled by human-unaware transformations such as synonymous word substitution. |
| Approach: | They propose a method that constructs a stochastic ensemble by applying random word substitutions on the input sentences and leverages the statistical properties to provably certify the robustness. |
| Outcome: | The proposed method outperforms state-of-the-art methods on IMDB and Amazon text classification tasks with practically meaningful certified accuracy. |
Copied to clipboard
| Challenge: | Lexical substitution (LS) is an extremely powerful technology that can be used as a backbone of various NLP applications such as writing assistance. |
| Approach: | They propose two simple decoding strategies that focus on the variations of the target word during decoding to generate substitutes from a paraphraser. |
| Outcome: | The proposed methods outperform state-of-the-art LS methods based on pre-trained language models on three benchmarks. |
Copied to clipboard
| Challenge: | Existing benchmarks designed to evaluate the reasoning capabilities of large models are limited in scope and lack flexibility to adapt difficulty according to evolving reasoning capacities of models. |
| Approach: | They propose a benchmark that incorporates multidisciplinary questions to evaluate the reasoning capabilities of large models and can adjust and update question difficulty based on the reasoning abilities of advanced models. |
| Outcome: | The proposed benchmark incorporates multidisciplinary questions to evaluate the reasoning capabilities of large models and can adjust and update question difficulty based on the reasoning abilities of advanced models. |
Copied to clipboard
| Challenge: | UnrealLLM is a novel framework that connects natural language descriptions with the professional PCG system (Unreal Engine 5) to automate scene generation. |
| Approach: | They propose a novel multi-agent framework that connects natural language descriptions with the professional PCG system (Unreal Engine 5) to automate scene generation. |
| Outcome: | The proposed framework achieves competitive performance in technical metrics and aesthetic quality, offering unique advantages in generation scale and interactivity. |
Copied to clipboard
| Challenge: | Despite advances in improving large language model (LLM) to refuse to answer malicious instructions, LLMs remain vulnerable to jailbreak attacks where attackers generate instructions with distributions differing from safety alignment corpora. |
| Approach: | They propose a framework that leverages embedding space distribution analysis to generate jailbreak-like instructions. |
| Outcome: | The proposed framework shows significant decreases in attack success rate on Qwen2.5, Llama3.1, and Llma3.2 without compromising their utility. |
Copied to clipboard
| Challenge: | Integration of large language models into electronic design automation has been a key driver in eDA. |
| Approach: | They propose a family of large language models that unifies generation- and embedding-based tasks related to RTL. |
| Outcome: | The proposed model achieves state-of-the-art performance across all evaluated tasks. |
Copied to clipboard
| Challenge: | Visually Rich Document Understanding (VRDU) frameworks are a key area of research . early approaches to VRDU relied on manually crafted rules and domain-specific heuristics . conventional deep learning approaches do not integrate the diverse modalities in documents . |
| Approach: | They review recent advances in MLLM-based Visually Rich Document Understanding (VRDU) their findings highlight emerging trends and promising research directions . |
| Outcome: | The proposed frameworks are scalable, reliable, and adaptable, the authors argue . their findings highlight emerging trends and promising research directions . |
Copied to clipboard
| Challenge: | Documents that are image-based are difficult to extract because of document variability. |
| Approach: | They propose a human-in-the-spiral assistive document annotation platform to extract structured data from document collections. |
| Outcome: | The proposed framework reduces annotation time by at least 41% while showing consistent performance gains over three iterations. |
Copied to clipboard
| Challenge: | Long context large language models (LLMs) pose significant challenges for efficient serving due to the large memory footprint and high access overhead of KV cache. |
| Approach: | They propose a retrieval-based method to reduce the memory footprint of LLMs . they propose Windowed Rotary Position Embedding and query-aware vector quantization . |
| Outcome: | The proposed method can achieve lower performance degradation with lower overhead compared to existing methods . it can reduce the memory footprint and access overhead of long context large language models . |
Copied to clipboard
| Challenge: | Extensive experiments demonstrate the effectiveness of SGTC across various tasks. |
| Approach: | They propose a generative tool invocation framework that introduces structure-aware semantic tokenization to encode tools as discrete code sequences. |
| Outcome: | The proposed framework reduces the size of the representation space and underutilizes collaborative signals among tools in downstream tasks. |
Copied to clipboard
| Challenge: | Existing methods for editing large language models struggle to track and incorporate changes in knowledge associated with edits, which limits the generalization ability of post-edit LLMs in processing edited knowledge. |
| Approach: | They propose a model editing method that leverages knowledge graphs to enhance LLM editing by capturing changes in associated knowledge by constructing an external graph. |
| Outcome: | The proposed method improves the generalization ability of LLMs in processing edited knowledge. |
Copied to clipboard
| Challenge: | Existing models for generating mathematical word problems are lacking in educational assessment. |
| Approach: | They propose an end-to-end neural model to generate diverse mathematical word problems from commonsense knowledge graph and equations. |
| Outcome: | The proposed model outperforms the SOTA models in terms of evaluation metrics and topic relevance. |
Copied to clipboard
| Challenge: | Existing methods for debiasing factchecking models learn such biases instead of understanding the semantic relationship between the claim and evidence. |
| Approach: | They propose a counterfactual framework CLEVER which is augmentation-free and mitigates biases on the inference stage. |
| Outcome: | The proposed method is augmentation-free and mitigates biases on the inference stage. |
Copied to clipboard
| Challenge: | Abstractive summarization is a task that generates short and concise summaries of user generated reviews. |
| Approach: | They propose an interactive attention mechanism to learn the representations of context and aspect words within reviews, acted as an encoder. |
| Outcome: | The proposed model achieves impressive results compared to other strong competitors on a real-life dataset. |
Copied to clipboard
| Challenge: | Existing methods for Temporal Knowledge Graph reasoning capture intra- and inter-time latent relations between entities that appear at different times. |
| Approach: | They propose a Latent relations Learning method for TKG reasoning that captures latent relations between entities at different times. |
| Outcome: | The proposed method exploits the intra- and inter-time latent relations of entities at different times. |
Copied to clipboard
| Challenge: | Existing models rely on historical information to learn embeddings for entities, but ignore the evolution of facts. |
| Approach: | They propose a Temporal Meta-learning framework to learn evolutionary meta-knowledge from TKGs. |
| Outcome: | The proposed method improves on four widely-used datasets and three backbones on a wide range of scenarios on tKGs. |
Copied to clipboard
| Challenge: | Multiple Choice Question Answering (MCQA) is a fundamental format for various tasks in NLP, such as commonsense reasoning. |
| Approach: | They propose a method to increase the number of correct options in a dataset. |
| Outcome: | The proposed method improves the performance of multiple choice question answering (MCQA) and improves its accuracy. |
Copied to clipboard
| Challenge: | DUBLIN is a pixel-based visual document understanding model that does not rely on OCR. |
| Approach: | They propose a pixel-based visual document understanding model that does not rely on OCR. |
| Outcome: | The proposed model performs on extractive tasks such as DocVQA, InfoVQA and AI2D, and strong performance on abstraction datasets such as VisualMRC and text captioning. |
Copied to clipboard
| Challenge: | Standard test sets for supervised learning evaluate in-distribution generalization but are misleading when a dataset has systematic gaps. |
| Approach: | They propose a more rigorous annotation paradigm for NLP that helps to close systematic gaps in the test data. |
| Outcome: | The proposed model performs significantly lower on contrast sets than on the original test sets—up to 25% in some cases. |
Copied to clipboard
| Challenge: | OpenOmni is an open-source, end-to-end pipeline benchmarking tool for multimodal conversational agents. |
| Approach: | They developed an open-source, end-to-end pipeline benchmarking tool to help solve these issues. |
| Outcome: | OpenOmni integrates speech-to-text, emotion detection, and large language models with the ability to integrate customized models. |
Copied to clipboard
| Challenge: | Large Vision-Language Models (LVLMs) generate responses that are plausible but incorrect or unsupported—commonly referred to as hallucinations. |
| Approach: | They propose a representation-level intervention framework that modulates hallucination-related features during inference by probing their encoded features. |
| Outcome: | The proposed framework reduces hallucinations while maintaining the performance and generalization capabilities of Large Vision-Language Models (LVLMs). |
Copied to clipboard
| Challenge: | Existing approaches to reinforcement learning (RL) rely on static, in-epoch metrics that overlook training dynamics, often introducing low-utility or outdated data. |
| Approach: | They propose a plug-and-play module that prioritizes cross-epoch ambiguous samples to neutralize the noise from stale experiences. |
| Outcome: | Extensive experiments on nine LLMs show that Adaptive Ambiguity Replay outperforms state-of-the-art baselines on real-world code editing tasks. |
Copied to clipboard
| Challenge: | Live morph resolution task is used to detect e-commerce live streaming violations . morphs are used to evade scrutiny and engage in false advertising . |
| Approach: | They propose a task to detect morph violations in live streaming scenarios . they use large language models to generate additional training data . |
| Outcome: | The proposed method improves performance and improves live streaming regulation. |
Copied to clipboard
| Challenge: | Using reinforcement learning from human feedback, large language models perform poorly when applied to colloquial subtitle translation tasks. |
| Approach: | They propose an adversarial training framework that iteratively updates the offline reward model and the online LLM to improve training outcomes. |
| Outcome: | The proposed training framework significantly improves upon translation baselines. |
Copied to clipboard
| Challenge: | Existing zero-cost (ZC) proxies rely on expert knowledge and incur significant trial-and-error costs. |
| Approach: | They propose a framework that automatically designs zero-cost (ZC) proxies for various tasks and incorporates genetic programming to find the optimal symbolic composition. |
| Outcome: | The proposed framework achieves higher ranking consistency than human-designed proxies on NLP tasks. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have remarkable reasoning capabilities in complex tasks such as mathematics and coding. |
| Approach: | They propose an entropy-modulation method that adaptively reweighs tokens based on theoretically-estimated entropic variations. |
| Outcome: | The proposed method outperforms state-of-the-art methods in six mathematical reasoning and three coding benchmarks. |
Copied to clipboard
| Challenge: | Existing methods for evaluating expressive speech focus on word accuracy, naturalness, signal quality, or emotional intensity at the utterance level. |
| Approach: | They propose a framework for Evaluating Expressive Appropriateness in speech that assesses whether a speech sample aligns with the underlying communicative intent implied by its discourse-level narrative context. |
| Outcome: | The proposed framework outperforms existing speech evaluation and analysis systems on a human-annotated test set. |
Copied to clipboard
| Challenge: | Recent advances in recommender systems have been overlooked due to their emphasis on textual content. |
| Approach: | They propose to introduce large language models into recommendation models to exploit the semantic understanding and strong transferability of LLMs. |
| Outcome: | The proposed approach significantly boosts an item’s exposure by altering its textual content during the testing phase, without requiring direct interference with the model’s training process. |
Copied to clipboard
| Challenge: | Large language model editing methods suffer from overfitting, where factual updates can propagate beyond their intended scope, overemphasizing the edited target even when it’s contextually inappropriate. |
| Approach: | They propose a framework for precise and controllable knowledge editing that utilizes two-phase representations and a linear transformation to compute a directional "belief shift" vector. |
| Outcome: | The proposed framework significantly reduces overfitting across nearly all evaluation metrics and on COUNTERFACT and MQuAKE. |
Copied to clipboard
| Challenge: | Fill-in-the-Middle (FIM) models suffer from performance degradation and prohibitive latency. |
| Approach: | They propose a search-and-replace infilling framework that integrates agentic verification and editing into a single-pass inference process. |
| Outcome: | The proposed framework harmonizes completion tasks with the instruction-following priors of Chat LLMs, extending the paradigm from static infilling to dynamic context-aware editing. |
Copied to clipboard
| Challenge: | Code Large Language Models have limited ability to reason about runtime behavior and understand functionality . authors present a generic framework to support integrating semantic information to code task-relevant prompts . |
| Approach: | a study examines the role of trace-based semantic information in boosting supervised fine-tuning and post-phase inference of Code LLMs. |
| Outcome: | a new framework integrates semantic information to code task-relevant prompts . the proposed framework shows that trace-based semantic information boosts reasoning ability . |
Copied to clipboard
| Challenge: | Prior work has explored step-level supervision using Shannon-entropy-based uncertainty signals, which conflate inherent state complexity with agent confidence. |
| Approach: | They propose a hierarchical group-based RL framework that leverages normalized entropy to locate outlier steps associated with trajectory neglect and optimizes them via a mechanism of trajectory-aware reward and trajectory-independent penalty. |
| Outcome: | Experiments on ALFWorld, WebShop, and Search-Augmented QA show that STAPO achieves state-of-the-art performance while substantially alleviating trajectory neglect. |
Copied to clipboard
| Challenge: | Large language models (LLMs) have achieved remarkable performance on various NLP tasks, yet their potential in more challenging task like finance, has not been fully explored. |
| Approach: | They propose a benchmark to assess the financial knowledge of large language models (LLMs) in China. |
| Outcome: | The proposed benchmark is the most comprehensive evaluation benchmark to date for LLMs in finance. |
Copied to clipboard
| Challenge: | Relevance modeling between queries and items is a key component of commercial search engines. |
| Approach: | They propose a framework for continual pre-training of LLMs to enhance domain knowledge . they employ queries and multi-field item to jointly pre-train for enhancing domain knowledge. |
| Outcome: | The proposed model achieves convincing performance compared to strong baselines. |
Copied to clipboard
| Challenge: | Large language models (LLMs) augmented with retrieval systems have significantly advanced natural language processing tasks by integrating external knowledge sources. |
| Approach: | They propose a method that conditions large language models to generate answers even in the absence of reliable knowledge. |
| Outcome: | The proposed approach balances accuracy with appropriate abstention, enhancing the reliability and trustworthiness of retrieval-augmented systems. |
Copied to clipboard
| Challenge: | Distantly supervised relation extraction (DSRE) methods are not capable of extracting relation labels for individual sentences. |
| Approach: | They propose a semi-supervised learning relation extraction framework for sentence-level DSRE . they discard only the labels of the noisy samples and utilize them as unlabeled samples . |
| Outcome: | The proposed framework achieves significant performance enhancements on two real-world datasets. |
Copied to clipboard
| Challenge: | Existing approaches to detect jailbreak prompts rely on static model components or fixed decision thresholds. |
| Approach: | They propose a dynamic jailbreak detection framework that employs reinforcement learning for adaptive threshold selection. |
| Outcome: | Experimental results show that the framework outperforms baselines in detection performance while maintaining high computational efficiency. |