Papers by Qin Lu
Copied to clipboard
| Challenge: | a new lyric-to-melody generation system bridges the gap between lyrics and melodies . previous generation systems lack paired data and lack of control on generated melodie. |
| Approach: | They develop a lyric-to-melody generation system with music template to bridge the gap between lyrics and melodies. |
| Outcome: | The proposed system bridges the gap between lyrics and melodies by using music template. |
Copied to clipboard
| Challenge: | Existing lifelong topic models focus on indomain text streams in which each chunk only contains documents from a single domain. |
| Approach: | They develop a lifelong collaborative model that uses non-negative matrix factorization to learn topics and domain-specific word embeddings. |
| Outcome: | The proposed model can learn topics and domain-specific word embeddings from a lifelong collaborative model. |
Copied to clipboard
| Challenge: | a comparative analysis of paper (meta-)reviews by large language models (LLMs) aims to identify and distinguish LLMs from human activities . |
| Approach: | They present a comparative analysis to identify and distinguish LLM activities from human activities. |
| Outcome: | The proposed analysis aims to improve recognition of instances when someone implicitly uses LLMs for reviewing activities. |
Copied to clipboard
| Challenge: | Existing methods for enhancing dialogue performance rely on summarizing behavior . e-commerce chatbots need to align their dialogue strategies with human behavior to achieve coherent, human-like conversations with customers. |
| Approach: | They propose a method to extract core patterns from dialogue data and integrate them into models by mining service thought processes using a multi-agent aPproach. |
| Outcome: | The proposed method outperforms manual methods and outperfies baselines on Taobao in China. |
Copied to clipboard
| Challenge: | Extreme-scale language models have shown exceptional performance on a variety of language tasks, but the degree of control offered by these models through pure prompting is limited. |
| Approach: | They propose an inference-time policy adapter which tailors a large base model without fine-tuning it. |
| Outcome: | The proposed model outperforms baseline methods on five challenging text generation tasks and even over GPT-4. |
Copied to clipboard
| Challenge: | Xia et al., 2018) demonstrate that a large language model can generate and maintain high-quality code documentation. |
| Approach: | They propose a large language model powered open-source framework for generating, maintaining, and updating code documentation. |
| Outcome: | The proposed framework generates high-quality documentation for the entire project. |
Copied to clipboard
| Challenge: | Recent approaches to annotate data focus on labeling, but lack holistic process control . a novel system that integrates task assignment, data annotation, and quality/cost management is needed . |
| Approach: | They propose a multi-agent system that integrates task assignment, data annotation, and quality/cost management. |
| Outcome: | The proposed system automates human management by using a collaborative multi-agent system. |
Copied to clipboard
| Challenge: | Existing work on retrieval-based chatbots has low-quality affect response . Existing frameworks for obtaining affective response are based on Retrieve-and-Rerank . |
| Approach: | They propose a retrieval-based framework which provides affective response for retrieval chatbots by using a new discriminate-and-rewrite mechanism. |
| Outcome: | The proposed framework outperforms existing baselines and can guarantee the quality of the response and satisfy the affect label. |
Copied to clipboard
| Challenge: | Automatic Chinese irony detection often lacks labeled benchmark datasets . despite its pervasive nature, irony is a trope whose actual meaning differs from what is literally enunciated. |
| Approach: | They propose to use a Chinese benchmark dataset for automatic Chinese irony detection to provide a benchmark for machine learning models. |
| Outcome: | The proposed dataset includes more than 8.7K posts, collected from Weibo, a micro blogging platform. |
Copied to clipboard
| Challenge: | Normative studies on modality for English words are relatively common . however, they are limited to a relatively small number of languages and require costly ratings. |
| Approach: | They aim to learn a mapping between word embeddings and modality norms by training on a high-resource language and testing on . monolingual and crosslingual word embeds are used to predict modality association scores . |
| Outcome: | The proposed model predicts modality associations even when trained on an English resource and tested on a completely unseen language. |
Copied to clipboard
| Challenge: | Few-shot Event Detection (FSED) requires limited labeled data and expensive manual labeling. |
| Approach: | They propose a prototype-based prompt-instance Interaction with causal Intervention model to utilize both prompts and verbalizers and effectively eliminate all biases. |
| Outcome: | The proposed model utilizes both prompts and verbalizers and eliminates all biases on RAMS and ACE datasets. |
Copied to clipboard
| Challenge: | Emotion recognition in conversations (ERC) is a task that aims to recognize the emotion of each utterance in conversations. |
| Approach: | They propose an iterative emotion interaction network which uses iterativly predicted emotion labels instead of gold emotion labels to explicitly model the emotion interaction. |
| Outcome: | The proposed method retains state-of-the-art performance on two datasets and achieves high accuracy. |
Copied to clipboard
| Challenge: | Existing methods for stance detection are task-agnostic, which fail to utilize task knowledge to better discriminate between genuine and bias features. |
| Approach: | They propose to incorporate stance reasoning process as task knowledge to aid in learning genuine features without using targets. |
| Outcome: | The proposed model achieves better performance than previous task-agnostic debiasing methods on new test sets. |
Copied to clipboard
| Challenge: | Existing approaches to enhance multilingual reasoning capabilities rely on costly multilingual training or employ prompting with external translation tools. |
| Approach: | They propose a training-free inference-time method to enhance multilingual reasoning capabilities via Representation Engineering without additional training data or tools. |
| Outcome: | The proposed method outperforms existing methods on four reasoning benchmarks in English and Thai and Swahili. |
Copied to clipboard
| Challenge: | Pre-trained encoder-only and sequence-to-sequence models are computationally expensive. |
| Approach: | They propose a recipe to initialize one model from the other to improve pre-training efficiency. |
| Outcome: | The proposed method matches the performance of a from-scratch model with a multilingual encoder while reducing the total compute cost by 27%. |
Copied to clipboard
| Challenge: | LongLeader aims to assess different LLMs' long-context comprehension abilities . long-constext comprehension is a key bottleneck for many use cases . |
| Approach: | They propose a leaderboard to assess different LLMs' long-context comprehension abilities . they offer open-source access to the benchmarks and maintain a dedicated website . |
| Outcome: | The proposed model assesses different LLMs on selected benchmarks and provides open-source access to the benchmarks. |
Copied to clipboard
| Challenge: | Large Vision-Language Models (LVLMs) have impressive multimodal abilities but remain prone to multilingual object hallucination. |
| Approach: | They propose a cross-lingual attention intervention method to mitigate multilingual object hallucination in LVLMs by aligning attention patterns. |
| Outcome: | The proposed method improves 13.56% (up to 30%) on the POPE and 21.75% on the hallucination subsets across languages. |
Copied to clipboard
| Challenge: | Parameter-efficient fine-tuning (PEFT) is a low-cost alternative to full fine-timing due to the massive overhead. |
| Approach: | They propose a Mixture-of-Experts approach that enhances specialization while maintaining low resource overhead. |
| Outcome: | The proposed approach outperforms or matches state-of-the-art methods on GLUE, GSM8K, MBPP, and a text rewriting task from SmolTalk. |
Copied to clipboard
| Challenge: | Existing large language models focus on causal coherence, neglecting the complex story arcs and orchestration inherent in human narratives. |
| Approach: | They propose a high-dimensional framework for narrative orchestration that unifies human and model perspectives while jointly characterizing narrative function and structure in a common space. |
| Outcome: | The proposed framework unifies human and model perspectives while jointly characterizing narrative function and structure in a common space. |
Copied to clipboard
| Challenge: | Existing attempts to generate empathy with other-awareness ignore to include self-a awareness to consider the own views of the self in their responses. |
| Approach: | They propose to include self-awareness to consider the own views of the self in empathetic response generation by integrating three stages of self-other awareness into the process. |
| Outcome: | The proposed method is superior to existing methods on the benchmark dataset. |
Copied to clipboard
| Challenge: | Empirical results suggest that scale is not the only way to build commonsense capabilities. |
| Approach: | They propose a commonsense distillation framework that can achieve a competitive level of commonsensing without relying on the benefits of scale. |
| Outcome: | The proposed framework breaks the dependence on the extreme-scale teacher model with two innovations: (1) the novel adaptation of NeuroLogic Decoding to enhance the generation quality of the weak, off-the-shelf language models, and (2) self-imitation learning to iteratively learn from the model’s own enhanced commonsense acquisition capabilities. |
Copied to clipboard
| Challenge: | Existing work on geometry problem solving treats calculation and proving as two specific tasks hindering a deep model to unify reasoning ability on multiple math tasks. |
| Approach: | They propose a large-scale Unified Geometry problem benchmark to unify geometry on multiple math tasks. |
| Outcome: | The proposed framework outperforms the existing model with 5.6% and 3.2% accuracies on calculation and proving problems. |
Copied to clipboard
| Challenge: | Automatic speech recognition (ASR) for children remains challenging due to developmental variability and the scarcity of high-quality corpora. |
| Approach: | They propose a large-scale Chinese child speech corpus that contains 112.5 hours of speech from 498 children and 500 caregivers. |
| Outcome: | The proposed model improves in-domain and cross-domain performance on children's speech. |
Copied to clipboard
| Challenge: | Existing methods for retrieving documents and ads use one-to-few mappings and time-consuming content extraction. |
| Approach: | They propose a framework that leverages LLM-generated commercial intents as an intermediate semantic representation to directly retrieve ads for queries in real-time. |
| Outcome: | The proposed framework has been implemented in a real-world online system, handling daily search volumes in billions. |
Copied to clipboard
| Challenge: | Text in domains like social media has its own salient characteristics. |
| Approach: | They propose a method to obtain domain knowledge and integrate it with general knowledge to improve emotion classification. |
| Outcome: | The proposed method improves performance of emotion classification on Twitter data. |
Copied to clipboard
| Challenge: | Large language models achieve effective safety alignment at the time of release, but fine-tuning often compromises safety mechanisms. |
| Approach: | They propose a method that performs safety realignment for large language models . they identify unsafe delta parameters from the fine-tuned models and recalibrate the retained parameters . |
| Outcome: | The proposed method improves safety performance on safety benchmarks and jailbreak attacks while maintaining their performance on downstream tasks. |
Copied to clipboard
| Challenge: | Existing RLVR algorithms rely on rigid, uniform, and symmetric trust region mechanisms . current algorithms lack robustness, asymmetric signal reliability and inefficient gradient utilization . |
| Approach: | They propose a framework to harmonize three dimensions of RLVR algorithms, a paper argues . a binary cutoff is used to discard valuable reinforcement signals, they argue . |
| Outcome: | The proposed framework outperforms baselines in evaluating a robust RLVR solution. |
Copied to clipboard
| Challenge: | Chain-of-Thought (CoT) is a key technique for enhancing the performance of Large Language Models. |
| Approach: | They propose a framework that optimizes outputs by utilizing wrong information and multi-perspective verification. |
| Outcome: | The proposed framework surpasses all baselines on 8 datasets and 5 LLMs. |
Copied to clipboard
| Challenge: | Standard algorithms for Large Language Models (LLMs) enforce stability via "hard clipping" but relying on log-probability gradient yields divergent weights as probabilities vanish, destabilizing LLM training. |
| Approach: | They propose a decoupled gradient policy optimization that uses a decay mechanism to decouple the probability of a boundary token. |
| Outcome: | The proposed algorithm outperforms baselines on various mathematical benchmarks. |
Copied to clipboard
| Challenge: | Existing deep neural network models lack mechanisms to highlight important sentiment terms. |
| Approach: | They propose a method to incorporate affective knowledge into deep neural network models by mapping affective influence vectors to an affective impact value and integrating them into long-term memory models to highlight affective terms. |
| Outcome: | The proposed approach improves on three large datasets by 1.0% to 1.5% on the benchmark datasets. |
Copied to clipboard
| Challenge: | Existing work on reinforcement learning has focused on single-turn tasks such as solving math problems. |
| Approach: | They propose a framework that learns directly from online interactions by asynchronously generating diverse trajectories, guided by binary rewards depending on task success. |
| Outcome: | Experiments on the WebArena-Lite benchmark show that the framework outperforms state-of-the-art methods and strong proprietary models. |
Copied to clipboard
| Challenge: | Recent advances in Large Language Models have transformed ML/AI development . a reevaluation of AutoML principles for Retrieval-Augmented Generation (RAG) systems is needed. |
| Approach: | They propose a framework for hyper-parameter tuning and a hierarchical MAB method for efficient exploration of large search spaces. |
| Outcome: | The proposed framework outperforms baseline methods in more challenging optimization scenarios. |
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) have greatly advanced problem solving in diverse domains such as mathematical reasoning and knowledge reasoning. |
| Approach: | They propose a thought prompting approach called 'Everything of Thoughts' it leverages pretrained reinforcement learning and Monte Carlo Tree Search to incorporate external domain knowledge and planning capability into thoughts. |
| Outcome: | The proposed approach outperforms existing approaches on game of 24, 8-Puzzle, and Pocket Cube. |
Copied to clipboard
| Challenge: | Recent advances in NoSQL database support focus on English . however, the intricacy and heterogeneity of NoSqL query languages present a formidable challenge . |
| Approach: | They propose a multilingual benchmark for natural language to NoSQL query generation that covers six languages. |
| Outcome: | The proposed framework improves performance in English and non-English settings, while ignoring lexical and syntactic differences. |
Copied to clipboard
| Challenge: | Existing studies on large language models focus on literal-level translation quality, such as adequacy and fluency. |
| Approach: | They propose a Culture-Aware Novel-Driven Parallel Dataset for Machine Translation and a multi-dimensional evaluation framework for assessing cultural translation quality. |
| Outcome: | The proposed model improves evaluation reliability in LLM-as-a-judge scenarios under culture-aware constraints. |
Copied to clipboard
| Challenge: | Existing large language models (LLMs) ignore this diversity by reasoning in a single dominant language. |
| Approach: | They propose a family of reasoning models that can adaptively reason in an advantageous language on a per-instance basis. |
| Outcome: | The proposed model can reason in a single dominant language on a per-instance basis. |
Copied to clipboard
| Challenge: | Existing methods to evict KV cache during inference phase are impractical for industrial-grade applications. |
| Approach: | They propose a method that combines token-wise KV cache eviction with PagedAttention and propose 'zipage' it achieves 95% of the performance of Full KV inference engines while delivering over 2.1 speedup . |
| Outcome: | The proposed method achieves 95% of the performance of Full KV inference engines while delivering over 2.1 speedup on large-scale mathematical reasoning tasks. |
Copied to clipboard
| Challenge: | Existing studies on hallucinations in large language models are limited to a single scenario, either cross-lingual or cross-modal. |
| Approach: | They propose a joint Cross-lingual and Cross-modal hallucinations benchmark to fill this gap . they incorporate cross-lingual, cross-modal scenarios to assess hallucinic capabilities . |
| Outcome: | The proposed benchmark incorporates both cross-lingual and cross-modal hallucination scenarios to assess the cross-linguistic and crossmodal capabilities of LLMs. |
Copied to clipboard
| Challenge: | Existing methods for reinforcement learning with verifiable rewards are limited by the complexity of the problem and the complexity. |
| Approach: | They propose a theoretically-grounded dual-pronged optimization framework for reinforcement learning with verifiable rewards that compensates for gradient attenuation of high-confidence correct actions while utilizing entropy changes as computable indicators to stabilize excessive update magnitudes. |
| Outcome: | The proposed framework compensates for gradient attenuation of high-confidence correct actions while utilizing entropy changes as computable indicators to stabilize excessive update magnitudes. |
Copied to clipboard
| Challenge: | Recent studies have evaluated and shown limitations in specific capabilities such as visual understanding, but a systematic evaluation of VLMs’ fundamental WM abilities remains absent. |
| Approach: | They propose a framework that assesses perception and prediction to provide an atomic evaluation of VLMs as WMs. |
| Outcome: | The proposed framework assesses perception and prediction abilities on 15 latest VLMs and compares them to human-level models. |
Copied to clipboard
| Challenge: | Open-source large models are rapidly catching up with the closed-source models . however, many current inference tools are not as simple and convenient to use. |
| Approach: | They develop an open-source library to simplify the deployment and management of large models. |
| Outcome: | The proposed library outperforms open-source models and offers high throughput and low latency. |
Copied to clipboard
| Challenge: | Existing approaches for Sequence to Sequence learning have been developed . convolutional neural networks and self-attention networks are the most popular . |
| Approach: | They propose to integrate convolutional and self-attention layers into a double path network for sequence to sequence learning. |
| Outcome: | The proposed method significantly improves performance over state-of-the-art systems. |
Copied to clipboard
| Challenge: | Existing methods focus on local optimal while ignoring sole-mention disambiguation boosted by richer context from other mentions’ disambiguating processes. |
| Approach: | They propose an approach to extracting medical entity disambiguation using memory mechanism and memorized entity information (M3E) they use a memory mechanism module that performs memory caching, retrieval, fusion and cross-network residual to aid the disambiguations of remaining mentions. |
| Outcome: | The proposed method outperforms state-of-the-art methods on two benchmark datasets. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have limited fault localization capabilities due to limited context length. |
| Approach: | They propose a hierarchical localization reward model to evaluate and select the most accurate fault localization candidates from the outputs of LLMs. |
| Outcome: | The proposed model improves the final line-level localization recall by 12% on the SWE-Bench-Lite dataset. |
Copied to clipboard
| Challenge: | Existing role-playing structures lack cognitive consistency in complex scenarios . Existing models excel in math and coding tasks but lack coherent reasoning . |
| Approach: | They propose a metacognition-driven framework that enhances role-playing performance . experimental results show performance improvements across varying scenario complexities . |
| Outcome: | The proposed framework outperforms existing models in social intelligence tasks and shows strength in long-context comprehension and group-level social interactions. |
Copied to clipboard
| Challenge: | Existing fine-tuning approaches that focus on English-centric training corpora often introduce implicit cross-lingual alignment, overlooking the potential for more profound, latent-level cross-linguistic interactions. |
| Approach: | They propose a multilingual fine-tuning paradigm that explicitly establishes a cross-lingual connection mechanism at the latent level. |
| Outcome: | The proposed model outperforms vanilla SFT and offers a strong latent-level alternative to data-level augmentation methods. |
Copied to clipboard
| Challenge: | Recent studies in formal mathematical reasoning have shown an unstoppable growth trend. |
| Approach: | They constructed 18k high-quality instruction-response pairs across five mainstream formal specification languages and evaluated them against ten open-sourced LLMs. |
| Outcome: | The proposed model compared instruction-response pairs across five formal specification languages and found that the LLMs were good at writing proof segments when given either the code, or the detailed description of proof steps. |
Copied to clipboard
| Challenge: | QA-LIGN decomposes monolithic rewards into interpretable principle-specific evaluations . scalar rewards obscure which objectives drive the training signal . |
| Approach: | a new method decomposes monolithic rewards into interpretable principle-specific evaluations . QA-LIGN reduces attack success rates by up to 68.7% while maintaining a 0.67% false refusal rate . |
| Outcome: | QA-LIGN reduces attack success rates by up to 68.7% while maintaining a 0.67% false refusal rate . the results outperform DPO and GRPO with state-of-the-art reward models given equivalent training . |
Copied to clipboard
| Challenge: | Existing methods for enhancing response accuracy and accuracy struggle with open-domain QA tasks because they perform independent retrieval operations without maintaining a summarizing memory or using adaptive retrieval strategies. |
| Approach: | They propose a method that integrates non-parametric knowledge from external knowledge bases into models to enhance response accuracy while mitigating factual errors and hallucinations. |
| Outcome: | The proposed method improves on open-domain QA datasets and reduces noise and hallucinations due to redundant information and insufficient information integration. |
Copied to clipboard
| Challenge: | Mandarin Alphabetical Words (MAWs) are a key component of Modern Chinese . they are characterized by unique code-mixing idiosyncrasies influenced by language exchanges . |
| Approach: | They propose to construct a large collection of Mandarin Alphabetic Words from Sina Weibo . they propose to use a web-based technique to identify and validate MAWs . |
| Outcome: | The proposed method identifies 16,207 Mandarin Alphabetic Words (MAWs) using a web-based technique . the results show that the proposed method is useful for linguistic research and inquiries . |
Copied to clipboard
| Challenge: | State-of-the-art neural models typically encode document-query pairs using cross-attention for re-ranking. |
| Approach: | They propose to fine tune a pretrained encoder-decoder model using document to query generation. |
| Outcome: | The proposed model achieves comparable results to more expensive approaches while being 6.8X faster. |
Copied to clipboard
| Challenge: | Existing approaches lack flexibility to address diverse and ever-evolving user queries in open domains. |
| Approach: | They propose to evaluate LLMs on open-domain knowledge that requires tools to solve diverse and ever-evolving user queries. |
| Outcome: | The proposed system outperforms baselines in the open domain task-solving benchmark. |
Copied to clipboard
| Challenge: | Existing methods for evaluating the perceptual quality of synthetic speech are limited due to the complexity of perceptual quality factors and the diversity of speech generation tasks. |
| Approach: | They propose a new paradigm for enabling large language models to conduct structured speech quality evaluation using a large-scale dataset. |
| Outcome: | The proposed model performs well across tasks and languages. |
Copied to clipboard
| Challenge: | Existing GUI benchmarks lack fine-grained diagnostics to identify which capabilities lead to task failures. |
| Approach: | They propose a multilingual P R GUI Benchmark to assess LVLMs' language capabilities . they propose XLI to align non-English hidden states with English ones during inference . |
| Outcome: | The proposed benchmark reveals consistent gaps between English and non-English settings . it reduces the cross-lingual gaps with an average gain of 6.5% in non- English settings compared to static benchmarks . |
Copied to clipboard
| Challenge: | Existing paradigms for text generation are left-to-right decoding from autoregressive language models. |
| Approach: | They propose a decoding algorithm that incorporates heuristic estimates of future cost that are efficient for large-scale language models. |
| Outcome: | The proposed method outperforms baselines on five generation tasks and achieves new state-of-the-art performance on table-to-text generation, constrained machine translation, and keyword-constrained generation. |