Papers by Wei Gao
Copied to clipboard
| Challenge: | Existing evaluation metrics for open-domain dialogue systems are limited by the diversity of possible outcomings. |
| Approach: | They propose a method to augment a reference set to improve reliability . they propose BLEU to measure similarity between a predicted response and a small set of references . |
| Outcome: | The proposed model improves the reliability of reference-based metrics with augmented reference sets. |
Copied to clipboard
| Challenge: | Existing data augmentation methods for event extraction are costly and time-consuming. |
| Approach: | They propose a data augmentation framework that randomly masks out an adjunct sentence fragment and infills a variable-length text span with a fine-tuned infilling model. |
| Outcome: | The proposed framework can generate more diverse data while keeping the original structure unchanged . it can replace a fragment of arbitrary length in the text with another fragment of variable length . |
Copied to clipboard
| Challenge: | Social media has become a fertile ground for nurturing rumors and misinformation due to its lack of systematic moderation. |
| Approach: | They propose a framework to enhance the joint predictive capabilities of LLMs for stance detection and rumor verification tasks. |
| Outcome: | The proposed framework outperforms state-of-the-art methods and generalizes to non-LLMs accommodated as task models. |
Copied to clipboard
| Challenge: | Inductive reasoning is a core component of human intelligence. |
| Approach: | They propose a task to induce natural language rules from natural language facts using natural language as representation for knowledge instead of formal language. |
| Outcome: | The proposed task surpasses baselines in both automatic and human evaluations. |
Copied to clipboard
| Challenge: | Recent large language models (LLMs) have shown superior performance in a variety of tasks due to the sub-linearly increasing computational costs. |
| Approach: | They propose a new MoE architecture with designed layers where experts have different sizes to mitigate this defect. |
| Outcome: | The proposed architecture surpasses existing MoEs by adaptively assigning the parameter budget to experts while maintaining the same total parameter size and number of experts. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly integrated into our daily lives, raising ethical concerns, especially about perpetuating stereotypes. |
| Approach: | They propose a method that incorporates a neutral word semantics-based loss function to alleviate the deterioration of the LMS during debiasing. |
| Outcome: | The proposed method alleviates the deterioration of the Language Modeling Score (LMS) by incorporating a neutral word semantics-based loss function. |
Copied to clipboard
| Challenge: | a comparative analysis of paper (meta-)reviews by large language models (LLMs) aims to identify and distinguish LLMs from human activities . |
| Approach: | They present a comparative analysis to identify and distinguish LLM activities from human activities. |
| Outcome: | The proposed analysis aims to improve recognition of instances when someone implicitly uses LLMs for reviewing activities. |
Copied to clipboard
| Challenge: | Existing studies on cognitive distortion have limited generalizability and performance of models in large-scale and cross-linguistic contexts. |
| Approach: | They propose a multi-task learning model based on teacher student architecture solution which improves generalization performance. |
| Outcome: | The proposed model improves generalizability and interpretability of the proposed model. |
Copied to clipboard
| Challenge: | Parameter-Efficient Fine-Tuning (PEFT) methods have gained popularity for adapting pre-trained Large Language Models (LLMs) to downstream tasks. |
| Approach: | They propose a method to optimize the importance of full layers with layer-wise importance scoring by leveraging the estimated importance scores. |
| Outcome: | The proposed method is compatible with PEFT methods that operate on a per-layer basis and achieves better performance. |
Copied to clipboard
| Challenge: | acquiring domain-specific knowledge often requires professional expert manpower. |
| Approach: | They propose a generic framework for generating evaluation datasets for domain-specific LLMs. |
| Outcome: | The proposed framework reduces the reliance on expert manpower while ensuring that the collected data is uniformly distributed. |
Copied to clipboard
| Challenge: | Experimental results show that, as the instruction data increases, LoRAMoE can significantly improve the ability to process downstream tasks, while maintaining the world knowledge stored in the LLM. |
| Approach: | They propose a framework that introduces several low-rank adapters and integrates them by using a router network to freeze the backbone model and force a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks. |
| Outcome: | The proposed framework freezes the backbone model and forces a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks, to alleviate world knowledge forgetting. |
Copied to clipboard
| Challenge: | Existing personalized microblog sentiment classification methods suffer from the insufficiency of discriminative tweets for personalization learning. |
| Approach: | They propose to use user-attention-based Convolutional Neural Networks to capture individuality and opinion bias in microblog posts and a novel adversarial cross-lingual learning framework to enrich the user post representation. |
| Outcome: | The proposed method outperforms state-of-the-art baseline algorithms with large margins on English and Chinese microblog datasets. |
Copied to clipboard
| Challenge: | Existing methods for extracting medical decision trees rely on manual annotation . PI-LoRA is a low-rank adaptation method for extract medical decision tree from clinical guidelines and textbooks . |
| Approach: | They propose a low-rank adaptation method for automatically extracting medical decision trees from clinical guidelines and textbooks. |
| Outcome: | The proposed method outperforms existing methods for the Text2MDT task while maintaining a lightweight architecture. |
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) represent significant strides toward artificial general intelligence (AGI). |
| Approach: | They introduce OpenHuEval, the first benchmark for LLMs focusing on the Hungarian language and specifics. |
| Outcome: | The framework reveals intrinsic patterns and mechanisms of LLMs in non-English languages, with Hungarian serving as an example. |
Copied to clipboard
| Challenge: | Existing privacy-preserving inference methods sacrifice utility or efficiency, authors say . current approaches suffer a trilemma between privacy, utility, and efficiency, they say . |
| Approach: | They propose a model-agnostic framework for privacy-preserving LLM inference that reformulates privacy protection at the batch level rather than the individual-prompt level. |
| Outcome: | The proposed model-agnostic framework achieves 20% higher utility than previous models . it reduces query cost by up to 5 compared to non-batched inference . |
Copied to clipboard
| Challenge: | Existing EE datasets define fixed event types and design specific schemas for each of them, failing to cover diverse events emerging from the online text. |
| Approach: | They propose to use a sentence-level dataset to benchmark Open Event Extraction without restricting event types. |
| Outcome: | The proposed dataset contains more than 42,000 news titles in 34 topics collected from Chinese web pages. |
Copied to clipboard
| Challenge: | Large language models (LLMs) generate outputs that stray from user input or contravene established knowledge. |
| Approach: | They propose a new phenomenon, Authority Bias, where LLMs favor one knowledge source over the other . they propose atomic information that generates conflicts and a Conflict Detection Enhanced Query framework . |
| Outcome: | The proposed framework reduces Authority bias in large language models . it detects conflicts, performs credibility assessment on conflicting paragraphs, and detects perturbed text . |
Copied to clipboard
| Challenge: | Large language models are reshaping internet services, and serving them is costly. |
| Approach: | They propose an efficient distributed LLM serving system that splits prefill and decode requests into smaller chunks . |
| Outcome: | The proposed system reduces TTFT, TPOT, and latency compared to the state-of-the-art system. |
Copied to clipboard
| Challenge: | Task-oriented dialog systems typically manage structured knowledge to guide goal-oriented conversations. |
| Approach: | They propose a TOD system with hybrid knowledge management, HyKnow, which extends the belief state to manage both structured and unstructured knowledge. |
| Outcome: | The proposed model outperforms existing TOD systems in the evaluation of a multiWOZ dataset on unstructured knowledge with strong end-to-end performance. |
Copied to clipboard
| Challenge: | Existing models generate explanations that appear coherent while containing unfaithful intermediate steps. |
| Approach: | They propose a causality-inspired framework for evaluating CoT quality using controlled perturbations as an instrumental signal to separate genuine step-to-step dependence from bias-driven artifacts. |
| Outcome: | Experiments on GSM8K, MATH, and CommonsenseQA show that FACT-E improves reasoning-trajectory selection and yields stronger in-context learning exemplars. |
Copied to clipboard
| Challenge: | Existing knowledge injection methods fail to understand the semantics of tweets . |
| Approach: | They propose a method to flexibly inject knowledge into a pre-trained language model and adaptively expand tweets context. |
| Outcome: | The proposed method is based on two training stages to flexibly inject knowledge into the pre-trained language model and adaptively expand tweets context. |
Copied to clipboard
| Challenge: | Existing approaches to improve contextual faithfulness treat the LLM as a black box, generating responses that are inconsistent with the provided context. |
| Approach: | They propose a framework for faithful RAG that operates in three stages: (i) fine-grained knowledge pruning to filter irrelevant context, (ii) latent conflict probing to identify hard conflicts in the model’s latent space, and (iv) conflict-aware attention to modulate attention heads toward faithful context integration. |
| Outcome: | Experiments show that ProbeRAG significantly improves both accuracy and contextual faithfulness. |
Copied to clipboard
| Challenge: | Existing pre-training methods focus on single-modal tasks or multi-modal ones . large-scale pre- training has drawn much attention in both the community of Compute Vision (CV) and Natural Language Processing (NLP). |
| Approach: | They propose a UNIfied-MOdal pre-training architecture which can adapt to both single-modal and multi-modal understanding and generation tasks. |
| Outcome: | The proposed model can learn more generalizable representations with rich non-paired single-modal data. |
Copied to clipboard
| Challenge: | et al., 2022) argue that the current models for drug discovery lack the ability to integrate molecules, proteins, and natural language. |
| Approach: | They propose a framework that integrates biological knowledge with chemical knowledge and natural language associations. |
| Outcome: | The proposed framework shows superior performance across a wide range of tasks. |
Copied to clipboard
| Challenge: | Large language models (LLMs) are promising foundations to build generally-capable agents . however, the community lacks a unified interactive framework that covers diverse environments for comprehensive evaluation of agents. |
| Approach: | They propose a framework that features 7 real-world scenarios, 14 environments, and 89 tasks for unified, real-time, and concurrent agent interaction. |
| Outcome: | The proposed framework features 7 real-world scenarios, 14 environments, and 89 tasks for unified, real-time, and concurrent agent interaction. |
Copied to clipboard
| Challenge: | Existing GUI agents focus on enhancing the accuracy of individual actions and lack effective mechanisms for detecting and recovering from errors. |
| Approach: | They propose a framework that incorporates a backtracking mechanism to enhance the task completion capabilities of GUI agents by verifier, judger, and reflector components. |
| Outcome: | The proposed framework improves task success rate and step accuracy on Mobile3M and Auto-UI benchmarks. |
Copied to clipboard
| Challenge: | Existing studies on medication recommendation mainly rely on EHRs, but some details of interactions between doctors and patients may be ignored or omitted in EHR. |
| Approach: | They propose to use medical dialogues to recommend medications with medical dialogue data . they propose to model dialogue structure and disease knowledge aware network . |
| Outcome: | The proposed method is a promising solution to recommend medications with medical dialogues. |
Copied to clipboard
| Challenge: | Recent sparse decoding methods improve efficiency but suffer from KV cache misalignment, resulting in performance degradation. |
| Approach: | They propose a method that combines block-sparse attention with periodic dense rectification to bound error accumulation and preserve alignment with the pretraining distribution. |
| Outcome: | Experiments on math reasoning, language modeling, and retrieval tasks show that ReSA achieves near-lossless generation quality with significantly improved efficiency. |
Copied to clipboard
| Challenge: | Large language models (LLMs) exhibit remarkable capabilities across many tasks, but face critical challenges in the CSC scenario: (1) poor generalization to rare entities in open-domain searches; and (2) failure to adapt to temporal entity variations due to static parameters, resulting in serious over-correction issues. |
| Approach: | They propose a Chinese Spelling Check system with RAG and multi-task learning that integrates dynamic knowledge retrieval and entity-centric RAG to address rare entities. |
| Outcome: | The proposed system outperforms existing baselines in the CSC task and achieves a maximum improvement of +9.92% on the search scenario benchmark and +3.2% on the general-domain dataset. |
Copied to clipboard
| Challenge: | Large Language Models have demonstrated remarkable capabilities in open-domain dialogues, but their performance in service dialogues remains suboptimal. |
| Approach: | They propose a framework that enables agents to learn effective strategies without large-scale human annotations. |
| Outcome: | The proposed framework decouples user modeling into two components that provide adaptive training scenarios rather than acting as an unfair adversary. |
Copied to clipboard
| Challenge: | Existing approaches to represent knowledge in the low-dimensional space are to leverage large-scale unsupervised text corpus to train fixed or contextual representations. |
| Approach: | They propose to leverage large-scale unsupervised text corpus to train fixed or contextual language representations and to express knowledge into a knowledge graph (KG) they incorporate distributional representations of a KG onto the representations from pre-trained language models, via simply concatenation or multi-head attention. |
| Outcome: | The proposed models outperform the other models on the COIN: COmmonsense INference in Natural Language Processing (COIN) Workshop datasets. |
Copied to clipboard
| Challenge: | Existing models lack cultural alignment across modalities and languages . a new framework to assess cultural awareness across linguistics and languages is needed . |
| Approach: | They propose a framework that integrates tri-modally aligned cultural benchmarks and a five-dimensional evaluation protocol to assess cross-country awareness disparities. |
| Outcome: | The proposed framework assesses cultural awareness disparities across modalities and languages . it is the first dataset aligned at the input level across text, image, and speech . |
Copied to clipboard
| Challenge: | Recent studies have shown that large language models (LLMs) have impressive capabilities in dealing with new tasks with the help of in-context learning (ICL). |
| Approach: | They propose to concate the image and text embeddings to enhance the retrieval performance of a visual-language task and to calculate a list-wise ranking loss for training the embeddable model. |
| Outcome: | The proposed framework fine-tunes the CLIP embedding model to better meet the needs of the large vision-language models. |
Copied to clipboard
| Challenge: | Existing knowledge graph question answering methods only search for the answer in a large knowledge graph. |
| Approach: | They propose to partition retrieved knowledge subgraphs into smaller sub-KSGs and then use a graph-augmented learning to rank method to select the top-ranked sub-kSGs. |
| Outcome: | The proposed method can capture global interactions in question and subgraphs and local interactions on the full KSG and top-ranked sub-KSGs respectively. |
Copied to clipboard
| Challenge: | Existing methods for debunking fake news rely on blending of authentic and fabricated content by creators. |
| Approach: | They propose a model that detects misinformation at sentence-level using social media conversations . they use a bag-level annotation system to train the model . |
| Outcome: | The proposed model outperforms existing state-of-the-art models on three real-world benchmarks and outperformed existing state of the art models in debunking fake news at sentence and article levels. |
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown continuously improving multilingual capabilities. |
| Approach: | They evaluate the ability of open LLMs to handle multilingual machine translation tasks using a parallel-first monolingual-second data mixing strategy. |
| Outcome: | The proposed model outperforms state-of-the-art models and achieves competitive performance with Google Translate and GPT-4-turbo. |
Copied to clipboard
| Challenge: | Recent approaches demonstrate that MLLMs can be adapted into competitive embedding models via large-scale contrastive learning. |
| Approach: | They propose a compressed pre-training phase which serves as a warm-up stage for contrastive learning. |
| Outcome: | The proposed model achieves state-of-the-art among MLLMs of comparable size on the MMEB, realizing optimization in both efficiency and effectiveness. |
Copied to clipboard
| Challenge: | Existing efficient reasoning methods rely on explicit length penalties for excessive verbosity on simple queries. |
| Approach: | They propose a training-time intervention that selectively suppresses redundant tokens . they find length shift occurs when models generate unnecessary reasoning on trivial inputs - a phenomenon that is often unexplored . |
| Outcome: | The proposed method reduces inference token usage by 78% while increasing accuracy compared to the initial policy and surpasses state-of-the-art efficient reasoning methods. |
Copied to clipboard
| Challenge: | Existing approaches to multilingual sequence-to-sequence pre-training rely on monolingual corpora and sometimes synthetic document-level bilingual corporata. |
| Approach: | They propose to leverage document-level trilingual parallel corpora to improve sequence-to-sequence multilingual pre-training by using a novel method called Grafting. |
| Outcome: | The proposed method achieves strong state-of-the-art (SOTA) scores on three multilingual document-level machine translation benchmarks and one cross-lingual abstractive summarization benchmark. |
Copied to clipboard
| Challenge: | Sentence function is an important linguistic feature indicating the communicative purpose of a sentence in a conversation. |
| Approach: | They propose a structured meta-learning approach for dialogue generation on infrequent sentence functions. |
| Outcome: | The proposed approach improves informativeness and relevance of dialogue generation on infrequent sentence functions while preserving knowledge generalization for similar sentence functions. |
Copied to clipboard
| Challenge: | Existing methods for offsite-tuning of large language models require high computational costs and lack theoretical analysis. |
| Approach: | They propose an offsite-tuning approach that selectively applies compression techniques such as rank compression and channel pruning to preserve the gradients of fine-tuned adapters while ensuring privacy. |
| Outcome: | The proposed method surpasses existing OT methods in privacy protection and model performance. |
Copied to clipboard
| Challenge: | Graph-based Retrieval-Augmented Generation (GraphRAG) is a new approach to document retrieval, but it is not suitable for legal reasoning. |
| Approach: | They propose a framework for reliable legal reasoning that structures knowledge as relational graphs and uses a multi-agent system to verify validity. |
| Outcome: | The proposed framework outperforms existing GraphRAG models in accurate and trustworthy legal analysis. |
Copied to clipboard
| Challenge: | Existing methods for few-shot and zero-shot fact verification require a large set of training data. |
| Approach: | They propose a method to prompt pre-trained language models to be consistent to improve the factuality assessment capability of PLMs. |
| Outcome: | The proposed method outperforms state-of-the-art few-shot fact verification models with a small number of unlabeled instances on zero-shot verification. |
Copied to clipboard
| Challenge: | Claim verification is cumbersome and inefficient for human fact-checkers to find consistent pieces of evidence. |
| Approach: | They propose an end-to-end hierarchical attention network that learns to represent coherent evidence and their semantic relatedness with the claim. |
| Outcome: | The proposed model outperforms state-of-the-art models on three datasets . it is based on a coherence-based attention layer and entailment-based one . |
Copied to clipboard
| Challenge: | Existing fake news detection models are opaque and lack deductive transparency . a framework for dialectical structured reasoning is proposed to address this limitation . |
| Approach: | They propose a framework that model fake news detection as an explicit dialectical process over multimodal social context. |
| Outcome: | The proposed framework achieves state-of-the-art while producing transparent explanations that mirror human reasoning process. |
Copied to clipboard
| Challenge: | Existing approaches to tool learning rely on hand-crafted prompts and natural language reasoning, making multi-step planning difficult and lacking precise error diagnosis and reflection mechanisms. |
| Approach: | They propose a framework that reformulates tool learning as a code generation task. |
| Outcome: | The proposed framework achieves superior performance in task completion accuracy and execution reliability compared to existing approaches. |
Copied to clipboard
| Challenge: | Experimental results show that Sequence-to-sequence models tend to generate generic/dull responses . |
| Approach: | They propose a statistical re-weighting method that assigns different weights for multiple responses of the same query. |
| Outcome: | The proposed method improves acceptance rate of generated responses and significantly reduces generated generic responses. |
Copied to clipboard
| Challenge: | Existing methods for automated fact-checking often overlook deceptive misinformation styles in generated explanations. |
| Approach: | They propose a framework that explicitly controls reasoning style by anchoring explanations to the predicted verdict. |
| Outcome: | The proposed framework achieves state-of-the-art under LLaMA-series models with 465 samples. |
Copied to clipboard
| Challenge: | Existing approaches to empathetic response generation ignore the emotion cause . existing dialogue systems lack emotion understanding and empathy . |
| Approach: | They propose a framework that integrates emotion cause information into empathetic response generation by predicting context emotion labels and sequence of emotion cause-oriented labels. |
| Outcome: | The proposed framework improves empathetic response generation by incorporating emotion cause information into the model. |
Copied to clipboard
| Challenge: | Existing studies fail to provide comprehensive service satisfaction analysis . Existing models fail to include satisfaction polarity classification and sentimental utterance identification . |
| Approach: | They propose a model that predicts customer sentiments and aggregates them into service satisfaction polarity. |
| Outcome: | The proposed model predicts customer sentiments and aggregates them into service satisfaction polarity and reasoning clues. |
Copied to clipboard
| Challenge: | Emotion cause analysis (ECA) is an emerging topic in natural language processing, which aims to identify the reasons behind a given emotion. |
| Approach: | They propose to detect the precise boundaries of text spans conveying accurate emotion causes from the given context by a sequence labeling and position identification problem. |
| Outcome: | The proposed methods outperform existing models on two benchmark datasets on the emotion cause analysis task. |
Copied to clipboard
| Challenge: | Existing studies focus on learning global or local correspondence, but lack fine-grained local-global alignment. |
| Approach: | They propose a High Order Semantic Alignment (HOSA) model that can provide complementary and comprehensive semantic clues to infer correlation scores. |
| Outcome: | The proposed model outperforms state-of-the-art models in retrieving the most relevant results. |
Copied to clipboard
| Challenge: | Current state-of-the-art LLMs exhibit clear limitations on multiple tasks, while performing well on tasks that involve contextual semantic understanding. |
| Approach: | They propose a mouse-based benchmark to evaluate LLMs' performance on NLP tasks involving Chouxiang Language. |
| Outcome: | The proposed benchmark evaluates the performance of LLMs on six NLP tasks involving Chouxiang Language. |
Copied to clipboard
| Challenge: | Existing retrieval-based dialogue systems suffer from slow inference or huge number of parameters. |
| Approach: | They propose a lightweight fully convolutional architecture for response selection using convolution. |
| Outcome: | The proposed architecture extracts matching features of context and response from 3D views. |
Copied to clipboard
| Challenge: | Reinforcement learning with verifiable rewards (RLVR) has emerged as a paradigm for enhancing the reasoning capabilities of large language models. |
| Approach: | They propose a positive-advantage reweighting approach that regulates model entropy by adjusting the loss weights assigned to tokens with positive advantages during RLVR training. |
| Outcome: | The proposed approach regulates model entropy by adjusting loss weights assigned to tokens with positive advantages during RLVR training while maintaining competitive performance. |
Copied to clipboard
| Challenge: | Existing systems for conversational recommender systems (CRS) have strong results in movies, but games present distinct challenges . MATCHA framework provides specialized agents for intent parsing, tool-augmented retrieval, multi-LLM ranking, and stronger safety. |
| Approach: | They propose a framework for conversational recommender systems that assigns specialized agents for intent parsing, tool-augmented retrieval, multi-LLM ranking and risk control. |
| Outcome: | MATCHA outperforms baselines on real user request dataset, improves Hit@5 by 20%, reduces popularity bias by 24%, and achieves 97.9% adversarial defense. |
Copied to clipboard
| Challenge: | Existing large language models have exacerbated fairness issues in tabular data generation . inherent historical biases in tabulated data cause LLMs to exacerbate fairness problems . |
| Approach: | They propose a universal debiasing framework that minimizes group-level dependencies . it leverages the autoregressive structure and analytic sampling distributions of LLM-based tabular data generators . |
| Outcome: | The proposed framework minimizes group-level dependencies while reducing mutual information between advantaged and protected attributes. |
Copied to clipboard
| Challenge: | Existing methods for detecting rumors are difficult to implement and require a lot of effort. |
| Approach: | They propose two recursive neural models that follow tweets' propagation layouts to learn discriminative features from tweets and generate more powerful representations for rumors detection. |
| Outcome: | The proposed models perform better than state-of-the-art approaches on two public Twitter datasets and show superior performance on detecting rumors at very early stage. |
Copied to clipboard
| Challenge: | Existing evaluation methods for document summarization require human annotations and annotations. |
| Approach: | They propose a method which measures the quality of a summary by measuring its semantic similarity with a pseudo reference summary, using contextualized embeddings and soft token alignment techniques. |
| Outcome: | The proposed method correlates better with human ratings by 18- 39% compared to the state-of-the-art evaluation metrics. |
Copied to clipboard
| Challenge: | Existing word embeddings only consider contextual information, which is suboptimal when used in various tasks due to a lack of task-specific features. |
| Approach: | They propose a task-oriented word embedding method that regularizes the distribution of words to enable a clear classification boundary. |
| Outcome: | The proposed method outperforms the state-of-the-art methods on a text classification task. |
Copied to clipboard
| Challenge: | Existing supervised relation extraction methods can still misclassify unknown relations into known relations due to the lack of supervision signals. |
| Approach: | They propose a method that regularizes the model by dynamically synthesizing negative instances that can provide the missing supervision signals. |
| Outcome: | The proposed method achieves SOTA unknown relation detection without compromising the classification of known relations. |
Copied to clipboard
| Challenge: | Recent advances in graph neural networks have made it difficult to capture user preferences. |
| Approach: | They propose a graph contrastive learning recommendation model based on noise augmentation that integrates truncated singular value decomposition in the feature engineering stage. |
| Outcome: | The proposed model reduces dimensionality and denoises the original data. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) unintentionally memorize sensitive data, posing privacy and security risks. |
| Approach: | They propose a framework that reconciles unlearning efficacy and utility preservation by using a latent-space gating mechanism to simulate internal recovery attempts. |
| Outcome: | The proposed framework achieves superior trade-off between unlearning efficacy and model utility. |
Copied to clipboard
| Challenge: | Existing knowledge editing techniques rely on memorizing updated knowledge, impeding LLMs from effectively combining the new knowledge with their inherent knowledge when answering questions. |
| Approach: | They propose a Learning to Edit framework that equips LLMs with the ability to apply updated knowledge to input questions through a two-phase process . |
| Outcome: | The proposed framework outperforms existing methods in knowledge editing tasks and compares it with four benchmarks and two LLM architectures. |
Copied to clipboard
| Challenge: | Existing generation-based models generate generic and safe responses such as "So am I" or "I don't know" |
| Approach: | They propose to predict the mediators to preserve relevant information and auto-regressively incorporate the mediator into generating process. |
| Outcome: | The proposed model generates relevant and informative responses and outperforms the state-of-the-art in terms of automatic metrics and human evaluations. |
Copied to clipboard
| Challenge: | Generated infographics may appear correct at first glance but contain easily overlooked issues, such as distorted data encoding or incorrect textual content. |
| Approach: | They propose to evaluate reliability of text-to-infographic generation using IGenBench . they employ multimodal large language models to verify each question . |
| Outcome: | The proposed framework decomposes reliability verification into atomic yes/no questions based on a taxonomy of 10 question types. |
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly used in interactive applications, and human evaluation remains the gold standard for assessing their performance in multi-turn conversations. |
| Approach: | They propose to use large language models to simulate users for automatic assistant evaluation. |
| Outcome: | The proposed model outperforms human evaluations on two interactive tasks and achieves Spearman’s of 0.7 on both tasks. |
Copied to clipboard
| Challenge: | Existing GRPO-based methods allocate sampling uniformly across tasks regardless of difficulty, propagate misleading learning signals and incur high sample-collection costs. |
| Approach: | They propose a framework that allocates sampling based on per-task success rates and performs fine-grained step-level optimization. |
| Outcome: | The proposed method improves sample efficiency and training stability over existing GRPO variants and three ablation variants on OSWorld and AndroidWorld. |
Copied to clipboard
| Challenge: | a new benchmark evaluates video-based optical character recognition (Video OCR) performance of multi-modal models in videos . the benchmark aims to improve video LLMs' ability to extract text from video content . previous benchmarks have focused on video QA, but not video-related QA. |
| Approach: | They propose to evaluate the video OCR performance of multi-modal models in videos . they use a semi-automated approach that integrates the OCR ability of image LLMs with manual refinement . |
| Outcome: | The proposed benchmark includes 1,028 videos and 2,961 question-answer pairs . it integrates the OCR ability of image LLMs with manual refinement . |
Copied to clipboard
| Challenge: | Small language models (SLMs) are promising for real-world deployment but struggle with high-stakes legal reasoning tasks. |
| Approach: | They propose a diagnostic-driven synthesis framework that extracts and refines reasoning trajectories from a capable teacher via fine-grained prompting and a self-reflective verification is employed to adaptively select the most effective data for the SLM student. |
| Outcome: | The proposed framework extracts and refines reasoning trajectories from a capable teacher via fine-grained prompting, then a self-reflective verification is employed to adaptively select the most effective data for the student. |
Copied to clipboard
| Challenge: | Long-context modeling is crucial for next-generation language models, but high computational cost of standard attention mechanisms poses significant computational challenges. |
| Approach: | They propose a natively trained Sparse Attention mechanism that integrates algorithms with hardware-aligned optimizations to achieve efficient long-context modeling. |
| Outcome: | The proposed model maintains or exceeds Full Attention models across general benchmarks, long-context tasks, and instruction-based reasoning. |
Copied to clipboard
| Challenge: | Personalized MGT detection remains largely underexplored due to personalization challenges . large language models (LLMs) can imitate personal writing styles, but they can generate fake news and misinformation. |
| Approach: | They propose a benchmark to evaluate detector robustness under personalization . they attribute this limitation to a feature-inversion trap that flips the effect in personalized contexts . |
| Outcome: | The proposed framework predicts detector robustness under personalization with an 85% correlation to actual results. |
Copied to clipboard
| Challenge: | Existing methods for OOD detection are based on labeled in-domain data . detecting out-of-domain (OOD) or unknown intents is challenging . |
| Approach: | They propose a novel reassigned contrastive learning method to discriminate IND intents for over-confident OOD and an adaptive class-dependent local threshold mechanism to separate similar IND and OOD intents. |
| Outcome: | The proposed method is effective for both aspects of overconfidence issues. |
Copied to clipboard
| Challenge: | a standard evaluation setup for supervised machine learning tasks does not hold for natural language generation tasks. |
| Approach: | They propose to use reference-free machine translation evaluation to compare source texts to system translations to find key limitations. |
| Outcome: | The proposed metrics perform poorly as semantic encoders for reference-free machine translation evaluation. |
Copied to clipboard
| Challenge: | Current sign language recognition methods use spatial graphs and temporal modules to capture spatial and temporal features, but their spatial graph modules are typically built on fixed graph structures. |
| Approach: | They propose a new spatial architecture that captures input-sensitive joint relationships and a temporal module to model multi-scale temporal information to capture complex human dynamics. |
| Outcome: | The proposed method achieves state-of-the-art accuracy on four large-scale SLR benchmarks. |
Copied to clipboard
| Challenge: | Existing Large Language Model (LLM) enabled agents lack flexibility to respond to users’ varying needs and preferences. |
| Approach: | They propose a test-time user-preference alignment strategy that optimizes the persona prompt, ensuring real-time preference alignment through textual loss feedback between simulated and ground-truth responses. |
| Outcome: | The proposed framework outperforms baseline methods in real-time and in real applications. |
Copied to clipboard
| Challenge: | Large language models have demonstrated exceptional performance across a wide range of tasks . however, selecting the optimal LLM to respond to a user query often necessitates a delicate balance between performance and cost. |
| Approach: | They propose a multi-LLM routing framework that efficiently routes user queries to the most suitable LLM. |
| Outcome: | The proposed framework outperforms baseline methods in terms of effectiveness and interpretability. |
Copied to clipboard
| Challenge: | Existing methods for vision-language pre-training can only learn from aligned image-caption data and rely heavily on expensive regional features. |
| Approach: | They propose an end-to-end unified-modal pre-training framework for joint learning . they propose to conduct grounded learning on both images and texts via a sharing grounded space . |
| Outcome: | The proposed model improves visual and visual semantic alignment on images and texts. |
Copied to clipboard
| Challenge: | generative models struggle to distinguish subtle differences among retrieved knowledge records, resulting in suboptimal quality of generated responses. |
| Approach: | They propose to use maximum marginal likelihood to train a perceptive retriever by utilizing signals from response generation for supervision. |
| Outcome: | The proposed approach improves on three task-oriented dialogue datasets using T5 and ChatGPT as the backbone models. |
Copied to clipboard
| Challenge: | Existing methods for mental health risk assessment rely on subjective textual records . however, these uncertainties can cause inconsistent and unreliable predictions . |
| Approach: | They propose a method that integrates objective behavior data alongside subjective mental records for robust mental health risk assessment. |
| Outcome: | The proposed approach achieves significant improvements over general LLMs. |
Copied to clipboard
| Challenge: | Existing work has been using decoding-free candidate selection methods to obtain candidate probability from initial output logits over vocabulary. |
| Approach: | They propose to evaluate a set of tasks using decoding-free candidate selection methods on a comprehensive set of questions. |
| Outcome: | The proposed methods are evaluated on a set of tasks including five multiple-choice QA tasks with a small candidate pool and four clinical decision tasks with 10k+ options. |
Copied to clipboard
| Challenge: | Existing studies fine-tune discriminative models on specific defined intent classes, preventing them from being directly adopted to new intent domains. |
| Approach: | They propose to use a pre-trained generative intent model to detect new intents from different domains with no parameter updates. |
| Outcome: | The proposed model outperforms baselines that need further fine-tuning or domain-specific samples. |
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have demonstrated impressive capabilities in generating coherent, informative, and fluent verbal reasoning. |
| Approach: | They propose to use fine-grained feedback with reinforcement retrieval to enhance fact-checking on news claims by using black-box LLM. |
| Outcome: | The proposed model improves on two public datasets for fact-checking on news claims using fine-grained feedback with reinforcement retrieval (FFRR). |
Copied to clipboard
| Challenge: | Recent advances in Video Large Language Models (Video-LLMs) have achieved exceptional performance on tasks like video question answering and captioning. |
| Approach: | They propose a decoding strategy that leverages sparse top-K attention and dense full attention to accelerate Video-LLMs without loss. |
| Outcome: | The proposed approach achieves a 1.94 walltime speedup in video processing. |
Copied to clipboard
| Challenge: | Empirical natural language processing (NLP) systems involve interoperation among multiple components . a wealth of NLP toolkits exist ( 4), such as spaCy, DKPro, CoreNLP. |
| Approach: | They propose a unified open-source framework that supports fast development of NLP workflows . framework includes processors for NLP tasks, visualization, and annotation . |
| Outcome: | The framework offers processors for NLP tasks, visualization, and annotation, and is extensible . it is delivered through two modularized yet integratable open-source projects, Forte and Stave . |
Copied to clipboard
| Challenge: | Code LLMs lack reproducible data pipelines and training protocols for reproducible advancements in code intelligence. |
| Approach: | They propose a top-tier code LLM that releases model weights and inference code . reproducible data pipelines, rigorous experimental ablation results and training protocols are included . |
| Outcome: | The proposed model achieves comparable performance to leading models and serves as an "open cookbook" reproducible training data, rigorous experimental ablation results, and detailed training protocols are also included in the model. |
Copied to clipboard
| Challenge: | Existing methods for rumor detection follow tree edges or treat all posts fully-connected during feature learning. |
| Approach: | They propose a new rumor detection model based on tree transformer to better utilize user interactions in the dialogue . they propose to use post-level self-attention to aggregate the intra-/inter-subtree stances . |
| Outcome: | The proposed model improves rumor detection performance on social media conversations . it is based on a conversation tree that encodes important information indicative of credibility . |
Copied to clipboard
| Challenge: | Typoglycemia is a phenomenon where people can read words even when the middle letters of the words are scrambled. |
| Approach: | They propose a reliable metric to quantify the degree of semantic reconstruction and validate its effectiveness. |
| Outcome: | The proposed metric quantifies the degree of semantic reconstruction and validates its effectiveness. |
Copied to clipboard
| Challenge: | AutoSDT-5K is the only automatically collected and the largest open dataset for data-driven scientific discovery. |
| Approach: | They propose an automatic pipeline that collects high-quality coding tasks in real-world data-driven discovery workflows. |
| Outcome: | The proposed pipeline synthesizes accurate tasks and tasks from a dataset of 5,404 tasks covering four scientific disciplines and 756 Python packages. |
Copied to clipboard
| Challenge: | Large language models (LLMs) have revolutionized natural language processing, but their practical implementation as autonomous multi-agent systems remains fraught with unresolved challenges. |
| Approach: | They propose a dynamic graph selector that redefines LLM-based MAS by exploiting the intrinsic properties of individual inputs to intelligently direct query trajectories. |
| Outcome: | The proposed framework exceeds state-of-the-art approaches in question answering, mathematical deduction, and code generation benchmarks. |
Copied to clipboard
| Challenge: | Neural conversation models are easy to generate bland and generic responses . however, their improvement of generating high-quality responses is still unsatisfactory . |
| Approach: | They propose to use a discrete latent variable with an explicit semantic meaning to improve the conditional variational autoencoder on short-text conversation. |
| Outcome: | The proposed model outperforms various kinds of generation models under automatic and human evaluations and generates more diverse and informative responses. |
Copied to clipboard
| Challenge: | Existing mitigation strategies for Text-to-Speech systems require excessive training resources or inference latency. |
| Approach: | They propose a GFlOwNet-guided distribution AlignmenT framework that mitigates hallucinations without relying on massive resources or inference latency. |
| Outcome: | The proposed framework reduces over 50% character error rates and lowers uncertainty by up to 58% on challenging test cases. |
Copied to clipboard
| Challenge: | Recent studies have demonstrated that large language models possess robust pattern recognition and semantic understanding capabilities over time series data. |
| Approach: | They propose a time series model that converts time series input into token embeddings and aligns time sequence embeddables with text prompts. |
| Outcome: | The proposed framework achieves the state-of-the-art (SOTA) performance and has potentials for wide industrial usages. |
Copied to clipboard
| Challenge: | Existing research has analyzed various factors indicating the conversational purpose such as emotions, topics, word orders, syntactic patterns and other aspects. |
| Approach: | They propose to annotate a short-text conversation dataset with annotated sentences and train conversation models conditioned on the sentence functions. |
| Outcome: | The proposed model can predict the quality of the returned responses. |
Copied to clipboard
| Challenge: | Recent studies on fine-grained intent detection have focused on collecting large-scale and high-quality samples via crowdsourcing resulting in data scarcity. |
| Approach: | They propose an iterative differential generation framework with contrastive feedback to generate high-quality pseudo samples and accurately capture the crucial nuances in target class distribution. |
| Outcome: | The proposed framework generates high-quality pseudo samples and captures crucial nuances in target class distribution. |
Copied to clipboard
| Challenge: | Neither outline-based code generation nor common code translation techniques can adequately address this challenge, despite their prevalence in existing systems. |
| Approach: | They have developed an algorithm that employs a multi-agent pipeline to handle embedded code migration under the TSL paradigm. |
| Outcome: | The proposed algorithm outperforms the baseline by 50.5% for pass rate and 13.0% for completeness across all tasks in RIOT and Zephyr. |
Copied to clipboard
| Challenge: | Large Language Model (LLM)-based optimization has shown promise for autonomous problem solving, but most approaches cast LLMs as passive constraint checkers rather than proactive strategy designers. |
| Approach: | They propose an end-to-end Automated Constraint Optimization method that tightly couples operations-research principles of constraint relaxation with LLM reasoning. |
| Outcome: | Extensive experiments on three challenging COP benchmarks validate AutoCO’s consistent effectiveness and superior performance, especially in hard regimes where current methods degrade. |
Copied to clipboard
| Challenge: | Existing evaluation metrics are not capable of evaluating text quality. |
| Approach: | They propose a metric that compares system output against reference texts based on semantics rather than surface forms. |
| Outcome: | The proposed metric shows a high correlation with human judgment of text quality on a number of text generation tasks. |
Copied to clipboard
| Challenge: | Existing methods to reduce hallucinations in large language models are inaccurate and inaccuracies in the generated feedback. |
| Approach: | They propose a method that helps LLMs determine whether to utilize multiple generated feedback responses and how to identify the most useful ones. |
| Outcome: | Extensive experiments show that the proposed method outperforms baselines on encyclopedic and commonsense knowledge QA tasks. |
Copied to clipboard
| Challenge: | Dependency parsers are not designed for capturing interaction between opinion words and aspect words. |
| Approach: | They propose to learn an aspect-centric tree structure to shorten distance between aspects and opinion words. |
| Outcome: | The proposed model outperforms baselines on five aspect-based sentiment datasets. |
Copied to clipboard
| Challenge: | Existing models fail to capture and model customer intention effectively because of insufficient information exploitation and only apparent information like descriptions and titles are used. |
| Approach: | They propose to exploit existing session data to capture and model intention in E-commerce product purchase sessions using a multimodal benchmark. |
| Outcome: | The proposed framework can bridge the gap between intention understanding in simplified research cases like co-buy intention and more complex yet practical scenarios like session history. |
Copied to clipboard
| Challenge: | Existing work exploits easily accessible co-occurrence information of events to learn event representations. |
| Approach: | They propose a weakly supervised contrastive learning method and a prototype-based clustering method for event representation learning. |
| Outcome: | The proposed framework outperforms baselines on Hard Similarity and Transitive Sentence Similarity tasks. |
Copied to clipboard
| Challenge: | Gene Ontology (GO) terms are used to describe gene function in biology and bio-medicine. |
| Approach: | They propose a task to generate term names for GO and build a large-scale benchmark dataset. |
| Outcome: | The proposed model outperforms baselines by incorporating the relations between genes, words and terms for term name generation. |
Copied to clipboard
| Challenge: | Prompt-based in-context learning and parameter fine-tuning are dominant paradigms for incorporating external information into large language models, but they incur high inference costs or require expensive retraining. |
| Approach: | They propose to convert prompts into temporary adapter weights to bridge this gap by converting prompts to temporary adapters. |
| Outcome: | The proposed model outperforms baselines on long-context language modeling and downstream NLU and summarization benchmarks while significantly reducing memory footprint and latency. |
Copied to clipboard
| Challenge: | Existing methods to detect contaminated texts focus on quantifying contamination status instead of accurately gauging model performance. |
| Approach: | They propose a Knowledge-grounded Interactive Evaluation framework which incorporates an LLM-powered “interactor” role for the first time to accomplish a dynamic contamination-resilient evaluation. |
| Outcome: | The proposed framework is based on a question in a standard LLM benchmark and can be used to evaluate models in real-world conversations. |
Copied to clipboard
| Challenge: | Existing adversarial training methods require multi-step gradient ascents or word substitutions to obtain adversarials, which impairs the effectiveness of adversariarial training. |
| Approach: | They propose a procedure for instead adversarial training with only clean data that estimates the adversarials loss by perturbing the input data’s probability distribution rather than their embeddings. |
| Outcome: | The proposed procedure reduces time consumption by up to 70% compared to current best-performing adversarial training methods. |
Copied to clipboard
| Challenge: | Large language models (LLMs) have revolutionized natural language processing with impressive performance across various tasks. |
| Approach: | They propose a framework for automated evaluations of large language models . they open-source their code at https://github.com/WisdomShell/FreeEval . |
| Outcome: | The framework is open-source and can be used to develop and validate new evaluation methods. |
Copied to clipboard
| Challenge: | Obtaining large-scale, high-quality real-world fact-checking datasets is costly . generalizability of detectors trained on synthetic data to real-life scenarios remains unclear . |
| Approach: | They propose to use synthetic data to learn from real-world data to detect multimodal misinformation . they propose to combine model-agnostic data selection methods with real-life data distributions . |
| Outcome: | The proposed method improves the performance of a small MLLM on real-world fact-checking datasets, surpassing GPT-4V. |
Copied to clipboard
| Challenge: | rumor detection models have been designed with oversimplifcation and evaluated inappropriately on a few datasets where the actual early-stage information is largely missing. |
| Approach: | They propose a new Benchmark dataset for EArly Rumor Detection based on claims from fact-checking websites and a novel model based upon neural Hawkes process for EARD. |
| Outcome: | The proposed model can guide a generic rumor detection model to make timely, accurate and stable predictions. |
Copied to clipboard
| Challenge: | Creating spreadsheet formulas remains a tedious and error-prone task for many end-users . a novel task is proposed to generate spreadsheet formulae from a user's NL query . |
| Approach: | They propose a task to generate formulas that are grounded on a spreadsheet table given a Natural Language query as input. |
| Outcome: | The proposed task generates formulas that are grounded on a spreadsheet table, given a natural language query as input. |
Copied to clipboard
| Challenge: | Recent advances in mobile Graphical User Interface (GUI) agents highlight the growing need for comprehensive evaluation benchmarks. |
| Approach: | They propose an online benchmark with 1080 tasks from 80 Chinese apps that measures task execution, complex reasoning, noise robustness and auto-eval framework with a reset mechanism. |
| Outcome: | The proposed benchmark measures task execution, complex reasoning, and noise robustness of agents by including 5 subsets, which set multiple evaluation dimensions. |