Papers by Daling Wang
Copied to clipboard
| Challenge: | Existing research on multimodal dialogues focuses on textual response generation and visual response selection based on the dialogue context. |
| Approach: | They propose a generative model framework for multimodal dialogue response generation that ground the conversation on an image. |
| Outcome: | The proposed system provides users with an enhanced conversational experience. |
Copied to clipboard
| Challenge: | Prior studies on stickers focused on sentiment analysis and recommendation systems, overlooking their vast potential in empathetic response generation. |
| Approach: | They propose a multimodal empathetic dialogue dataset, STICKERCONV, which simulates human behavior with stickers, and propose evaluative metrics based on LLM. |
| Outcome: | The proposed framework generates contextually relevant and emotionally resonant multimodal empathetic responses, contributing to the advancement of more nuanced and engaging e-dialog systems. |
Copied to clipboard
| Challenge: | Current approaches for Multimodal Sentiment Analysis (MSA) rely on parameter-heavy LLMs for classification, overlooking multimodal sentiment reasoning generation in resource-limited environments. |
| Approach: | They propose a multimodal sentiment reasoning distillation model that employs a teacher-assistant-student paradigm to address deployment constraints in resource-limited environments. |
| Outcome: | The proposed model performs well on a resource-limited JMSRC task with only 3B parameters and shows generalization and interpretability. |
Copied to clipboard
| Challenge: | Existing personalized microblog sentiment classification methods suffer from the insufficiency of discriminative tweets for personalization learning. |
| Approach: | They propose to use user-attention-based Convolutional Neural Networks to capture individuality and opinion bias in microblog posts and a novel adversarial cross-lingual learning framework to enrich the user post representation. |
| Outcome: | The proposed method outperforms state-of-the-art baseline algorithms with large margins on English and Chinese microblog datasets. |
Copied to clipboard
| Challenge: | Existing zero-shot dialogue generation systems rely on large-scale pre-trained language models. |
| Approach: | They propose a multilingual learning framework for zero-shot dialogue generation that can transfer knowledge from an English corpus to a non-English corpus with zero samples. |
| Outcome: | The proposed framework can transfer knowledge from an English corpus to a non-English corpus with zero samples. |
Copied to clipboard
| Challenge: | Existing approaches to learning KG triplets ignore ternary propagation patterns and ignore zero-shot, few-shot and synonymity problems. |
| Approach: | They propose a framework for contrastive learning based on ternary propagation patterns among head, relation and tail. |
| Outcome: | Experiments on benchmarks show that TernaryCL is superior to state-of-the-art models. |
Copied to clipboard
| Challenge: | Existing methods for inductive knowledge Graphs are limited by sparsity and implicit transfer. |
| Approach: | They propose a Contrastive Learning framework with graph guided Variational autoencoder on Meta-KGs to capture and transfer entities. |
| Outcome: | The proposed framework outperforms state-of-the-art methods with extensive experiments. |
Copied to clipboard
| Challenge: | Existing models of seeker simulations are limited by the cost and ethical concerns of involving real seekers in mental health research. |
| Approach: | They propose an emotional and cognitive dynamic agent system equipped with tertiary memory to enable dynamic control of the simulator's configurations. |
| Outcome: | The proposed system achieves more realistic seeker simulation compared to baselines. |
Copied to clipboard
| Challenge: | Medical reasoning models are constrained by parametric knowledge and can induce hallucinations and spurious attributions. |
| Approach: | They propose a model that uses a multi-hop med-search QA synthesis method to apply the DR paradigm in medical contexts. |
| Outcome: | The proposed model outperforms larger medical reasoning models on medical benchmarks. |
Copied to clipboard
| Challenge: | Existing studies lack the perception of fine-grained dialogue emotion propagation, and have limitations in reasoning about the intentions of users on cognition, which affect the quality of empathetic response. |
| Approach: | They propose to use commonsense reasoning and reinforcement learning to generate empathetic response based on in-context commonsensing and contextual reasoning to broaden cognitive boundaries. |
| Outcome: | The proposed model outperforms state-of-the-art models in automatic and human evaluation. |
Copied to clipboard
| Challenge: | Existing methods for model merging struggle to maintain performance gains as the number of merged models increases. |
| Approach: | They propose a Reparameterized Heavy-Tailed method to extend the merged model’s coverage and enhance performance. |
| Outcome: | The proposed method extends the merged model’s coverage and enhances performance on 19 benchmarks, including knowledge-intensive and general-purpose tasks. |
Copied to clipboard
| Challenge: | Large language models (LLMs) inherit contamination from training corpora, directional bias under social-desirability framing, and limited responsiveness to context beyond the item text. |
| Approach: | They propose a paradigm that reformulates TAT, Rorschach, and SCT with newly generated stimuli and organises assessment as a three-stage pipeline. |
| Outcome: | The proposed paradigm reformulates TAT, Rorschach, and SCT with newly generated stimuli and organises assessment as a three-stage pipeline. |
Copied to clipboard
| Challenge: | Empathetic conversation is a crucial characteristic in daily conversations between individuals. |
| Approach: | They propose an Emotional Knowledge Tool Calling framework which encapsulates commonsense knowledge bases as empathetic tools, enabling LLMs to integrate external knowledge flexibly. |
| Outcome: | The proposed framework can generate empathetic responses effectively on the TOOL-ED dataset. |
Copied to clipboard
| Challenge: | Existing approaches to multimodal sentiment analysis treat entire modality as an independent unit for feature enhancement or denoising, which often suppresses redundant noise at the cost of weakening critical information. |
| Approach: | They propose a ModaLity-aware noise dynAmic editiNg framework that performs modality-awful block partitioning by dividing features of each modality into multiple blocks. |
| Outcome: | Experiments on five models and four datasets show that MoLAN+ achieves the state-of-the-art performance. |
Copied to clipboard
| Challenge: | Existing approaches to fine-tuning language models use zeroth-order optimizers to conserve GPU memory. |
| Approach: | They propose a full-parameter fine-tuning strategy which updates a subset of parameters at each training step. |
| Outcome: | The proposed approach reduces the amount of gradients and optimizer state parameters residing in GPU memory at the same time, thereby reducing GPU memory usage. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have made safety issues of LLMs more prominent and critical. |
| Approach: | They propose a framework which attacks LLMs through semantic camouflage and replaces unsafe content with semantic features to conceal malicious intent . |
| Outcome: | The proposed framework outperforms existing models in over 80% of cases and is highly effective against various defenses. |
Copied to clipboard
| Challenge: | Existing research focuses solely on text, leaving a gap with practical applications. |
| Approach: | They propose to synthesize a multimodal conversational recommendation dataset using multimodal large language models to automatically synthesized data from 7,000 conversations in the Clothing domain. |
| Outcome: | The proposed dataset contains 83,148 utterances from 7,000 conversations centered around the Clothing domain. |
Copied to clipboard
| Challenge: | Argumentation is a key part of human reasoning and decision-making . existing argumentative corpora focus on single-turn settings, but multi-turn dialogues are often realized as multi-turned dialogues . |
| Approach: | They present a dataset for strategic multi-turn argumentation dialogues . they annotate each utterance with five strategy types, allowing multiple strategies per utterrance . |
| Outcome: | The proposed dataset shows that explicit prompting improves fluency, stylistic coherence and persuasiveness. |
Copied to clipboard
| Challenge: | Existing methods ignore the contexts around the emotion word which can provide an emotion cause clue. |
| Approach: | They propose a co-attention neural network model for emotion cause analysis with emotional context awareness. |
| Outcome: | The proposed model outperforms the state-of-the-art methods. |
Copied to clipboard
| Challenge: | Emotion cause analysis (ECA) is an emerging topic in natural language processing, which aims to identify the reasons behind a given emotion. |
| Approach: | They propose to detect the precise boundaries of text spans conveying accurate emotion causes from the given context by a sequence labeling and position identification problem. |
| Outcome: | The proposed methods outperform existing models on two benchmark datasets on the emotion cause analysis task. |
Copied to clipboard
| Challenge: | Existing visual perception systems focus on region-level segmentation in single-turn dialogues . existing systems cannot reason at the pixel level and comprehend dynamic user intent . |
| Approach: | They propose a task that tracks evolving user intent via multi-turn interactions for fine-grained segmentation. |
| Outcome: | The proposed method outperforms existing baselines in segmentation and reasoning metrics. |
Copied to clipboard
| Challenge: | Existing retrieval-based dialogue systems suffer from slow inference or huge number of parameters. |
| Approach: | They propose a lightweight fully convolutional architecture for response selection using convolution. |
| Outcome: | The proposed architecture extracts matching features of context and response from 3D views. |
Copied to clipboard
| Challenge: | Current methods for harmful meme detection lack the knowledge required to identify such hate . current methods lack the ability to identify cultural stereotypes and visual metaphors . |
| Approach: | They propose a framework that decomposes meme analysis into a human-inspired reasoning process . they propose DR-HM to transfer knowledge from closed-source models while mitigating biases . |
| Outcome: | The proposed framework outperforms existing methods on three benchmark datasets. |
Copied to clipboard
| Challenge: | Existing benchmarks for video understanding often focus on specific aspects, overlooking the holistic nature of video content. |
| Approach: | They propose a temporal-oriented benchmark for fine-grained understanding on dense dynamic videos with two complementary tasks: captioning and QA. |
| Outcome: | The proposed model performs well on diverse video scenarios and dynamic videos, with interpretable and robust evaluation criteria. |
Copied to clipboard
| Challenge: | Existing methods suffer from incomprehensive persona tags that have unique and obscure meanings to describe human’s personality. |
| Approach: | They propose a graph convolution network model with addressee selecting mechanism that integrates personas, dialogue utterances, and external text knowledge in a unified graph. |
| Outcome: | The proposed model outperforms baselines by large margins and improves persona consistency in the generated responses. |
Copied to clipboard
| Challenge: | Existing research results on explicit sentiment analysis are limited . implicit sentiment analysis is a process of analyzing text based on whether it contains explicit sentiment words. |
| Approach: | They propose a model that integrates external knowledge and contextual features . they use a knowledge graph to supplement implicit sentiment expression . |
| Outcome: | The proposed model can achieve better results on the SMP2019 implicit sentiment analysis dataset. |
Copied to clipboard
| Challenge: | Existing multimodal emotion and intent recognition tasks focus on classification, not rationale and intrinsic connections between these states. |
| Approach: | They propose a task that requires models to jointly predict emotion and intent while generating natural language explanations for why they co-occur. |
| Outcome: | The proposed model outperforms baseline models in prediction and explanation generation. |
Copied to clipboard
| Challenge: | Existing studies only considered the representation of a single image-text post . Fig. 1 shows that multimodal sentiment expressions have global characteristics . |
| Approach: | They propose a multi-channel Graph Neural Networks with Sentiment-awareness approach for image-text sentiment detection. |
| Outcome: | The proposed approach is effective for image-text sentiment detection on three publicly available datasets. |
Copied to clipboard
| Challenge: | Existing methods for encoding dialogues do not capture interaction information between roles, thus ignore interaction-related key information. |
| Approach: | They propose a contrastive learning based interaction-aware model for the role-oriented dialogue summarization namely CIAM and use it to train the decoder to learn role-level interaction. |
| Outcome: | The proposed model captures interaction information between different roles and produces informative summaries on two public datasets. |
Copied to clipboard
| Challenge: | Parameter-Efficient Fine-Tuning (PEFT) is an alternative to Full-Parameter Fine-tuning, but its effectiveness on complex tasks such as reasoning and instruction-following remains unclear. |
| Approach: | They propose to use PEFT to reduce the number of trainable parameters while freezing the weights of LLMs. |
| Outcome: | The proposed methods perform well on standard tasks, but weaknesses on complex and adversarial settings call for new directions beyond current paradigms. |
Copied to clipboard
| Challenge: | Multimodal instruction fine-tuning degrades textual reasoning capability, undermining multimodal performance. |
| Approach: | They propose a plateau-guided model merging method that selectively injects base language model parameters into MLLMs to mitigate this degradation. |
| Outcome: | The proposed framework reduces multimodal instruction fine-tuning degradation by incorporating a plateau-guided model merging method into MLLMs. |
Copied to clipboard
| Challenge: | Existing methods for question generation suffer from dullness and deviation problem, which can lead to deviated or dull questions. |
| Approach: | They propose two methods to enhance semantic coherence between question and answer by using a coherent score and adversarial training to explicitly control question generation. |
| Outcome: | The proposed methods outperform state-of-the-art baseline algorithms with large margins in raising semantic coherent questions. |
Copied to clipboard
| Challenge: | Existing methods for iterative retrieval-augmented generation (iRAG) suffer from greedy single-path expansion and granularity–demand mismatch . |
| Approach: | They propose a model that constructs candidate triples and history-conditionally integrates them to distill core triples to generate the next-hop query. |
| Outcome: | The proposed model mitigates the greedy single-path expansion and granularity–demand mismatch by preserving multiple plausible evidence chains. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable capabilities, yet their further evolution is often hampered by the scarcity of high-quality training data and the heavy reliance of traditional methods on expert-labeled data. |
| Approach: | They propose a paradigm that enables LLMs to train themselves by generating, cleaning, reviewing and annotating data with preference information. |
| Outcome: | The proposed model can generate, clean, review, and annotate data with preference information significantly reducing time and cost of post-training data construction. |
Copied to clipboard
| Challenge: | Existing approaches to reduce overthinking require additional rollout computation or externally labeled datasets. |
| Approach: | They propose a Neuron-based Early reAsoning exiT framework that monitors neuron-level activation dynamics to enable training-free early exits. |
| Outcome: | The proposed framework reduces the amount of reasoning steps generated by LRMs while maintaining accuracy. |
Copied to clipboard
| Challenge: | Existing generative models for dialogue use the last hidden state to summarize the history of the dialogue. |
| Approach: | They propose a Pseudo-Variational Gated Recurrent Unit (PVGRU) that summarises the accumulated distribution variations of subsequences and builds a model based on it. |
| Outcome: | The proposed model can improve diversity and relevance of responses on two benchmark datasets. |
Copied to clipboard
| Challenge: | Multi-turn, long-horizon tasks require dozens of sequential model calls per episode. |
| Approach: | They propose a cost-aware multi-turn LLM routing tool which encodes interaction history and candidate models into joint history–model embeddings and learns an outcome estimator from logged trajectories to predict turn-level model utility. |
| Outcome: | The proposed model reduces cost and performance by 58.7% on ScienceWorld and on Humanity’s Last Exam (HLE) and even reduces costs for held-out tasks. |
Copied to clipboard
| Challenge: | Existing methods for fraud detection rely on transcribed text, lacking acoustic cues . a proposed framework for audio-based slow-thinking fraud detection eliminates transcription errors . |
| Approach: | They propose a framework for audio-based slow-thinking fraud detection that eliminates transcription errors and rewards slow-thought reasoning by capturing fine-grained audio details. |
| Outcome: | The proposed method improves accuracy, inference efficiency, and real-time processing capabilities. |
Copied to clipboard
| Challenge: | Existing query-based alignment modules enforce uniform cross-attention across all layers, leading to computational redundancy. |
| Approach: | They propose a framework that allows for asynchronous query-based alignment with large-scale visual features. |
| Outcome: | The proposed framework matches or surpasses baseline performance while reducing alignment FLOPs by approximately 37% during training and inference. |
Copied to clipboard
| Challenge: | Existing question reformulation models are based on supervised question labels without considering feedback information from answers. |
| Approach: | They propose a question reformulation model that integrates conversational history information with reinforcement learning. |
| Outcome: | The proposed model is more effective in conversational machine comprehension with reinforcement learning. |
Copied to clipboard
| Challenge: | Prior studies have focused on strengthening multimodal reasoning by improving representation alignment or increasing computation, but these methods do not characterize the differences in visual demands across tasks. |
| Approach: | They propose an entropy-driven task-adaptive visual attention allocation framework that uses visual attention entropic as a control signal to dynamically allocate attention according to task demands. |
| Outcome: | The proposed framework achieves consistent performance gains across diverse reasoning tasks, datasets, and models, providing a clear direction toward more reliable multimodal reasoning. |
Copied to clipboard
| Challenge: | Recent performance boosting for dialogue response selection task achieved by Cross-Encoder based models is limited and the learned models have poor generalization capability in realistic scenarios. |
| Approach: | They propose a model that combines the representation-based Bi-Encoder and interaction-based Cross-Encoding to achieve better semantic representation. |
| Outcome: | The proposed model can achieve state-of-the-art performance on three benchmark datasets for multi-turn response selection. |
Copied to clipboard
| Challenge: | Existing studies require massive labeled data to train models for multimodal data analysis. |
| Approach: | They propose a novel multimodal prompt model that captures specific aspect terms in a few-shot scenario. |
| Outcome: | The proposed model outperforms baselines on two MABSA-related tasks on a few-shot dataset. |
Copied to clipboard
| Challenge: | Existing speech-to-speech large language models rely on ASR transcription or use encoders to extract latent representations, weakening affective information and contextual coherence in multi-turn dialogues. |
| Approach: | They propose a framework for speech-based empathetic response generation that captures turn-level affective states and dialogue-level emotional dynamics. |
| Outcome: | The proposed framework outperforms baselines in automatic and human evaluations and remains robust across different Large Language Model (LLM) backbones. |