Papers by Yue Zhao
Copied to clipboard
| Challenge: | CLIP revolutes vision-language pretraining by using contrastive learning on paired web data. |
| Approach: | They propose to combine a "adapter ensemble" with traditional machine learning techniques to augment large-scale pretrained vision-language models. |
| Outcome: | The proposed model outperforms baselines and derives improvement when the number of ensemble parameters increases. |
Copied to clipboard
| Challenge: | Existing evaluation methods for Figure-to-Text tasks are limited due to the inherent ambiguity and semantic compression of figures, the generated texts suffer from factual inaccuracies, incomplete coverage, and weak logical reasoning. |
| Approach: | They propose a five-dimensional reference-free evaluation method aligned with expert criteria to support fine-grained evaluation. |
| Outcome: | The proposed method outperforms Gemini-2.0 and Claude-3.5 with only 0.9B parameters. |
Copied to clipboard
| Challenge: | Existing approaches to RAG neglect system state variables, resulting in poor performance and erroneous knowledge accumulation. |
| Approach: | They propose a framework that incorporates a Turing Complete System to manage state variables and manage retrieval halting. |
| Outcome: | The proposed framework improves on seven real-world healthcare datasets and shows that it is more accurate than existing methods. |
Copied to clipboard
| Challenge: | Existing datasets often rely on synthetic data or figure-caption pairs, failing to capture the depth and complexity of geoscientific reasoning. |
| Approach: | They propose a multimodal scientific dataset and benchmark curated from open-access publications. |
| Outcome: | MSEarth features over 289K figures with captions enriched by contextual discussions and reasoning from original papers. |
Copied to clipboard
| Challenge: | generative models struggle with logic-intensive instruction following, exposing a persistent reasoning–execution gap. |
| Approach: | They propose a task-agnostic reasoning architecture for general image generation . they propose pixel-level feedback to ground the Thinker's policy in pixel feedback . |
| Outcome: | The proposed system significantly improves image reasoning and generation quality. |
Copied to clipboard
| Challenge: | Existing methods for stance detection for pure texts have limited results to multi-modal content. |
| Approach: | They propose a multi-modal stance detection framework that leverages target information to learn multi-modal stance features from textual and visual modalities. |
| Outcome: | The proposed framework achieves state-of-the-art in multi-modal stance detection on five datasets based on Twitter . |
Copied to clipboard
| Challenge: | Existing studies on hallucination focus on text or vision, while few audio-oriented studies are limited in scale, modality coverage, and diagnostic depth. |
| Approach: | They propose a large-scale benchmark for evaluating hallucinations across speech, sound, and music. |
| Outcome: | The proposed model improves hallucination rate, yes/no bias, error-type analysis, and refusal rate. |
Copied to clipboard
| Challenge: | Modern language models excel at factual reasoning but struggle with value diversity, authors say . task-sensitive tasks such as hate speech expose this limitation . human disagreement captures the diversity of plausible human perspectives, authors argue . |
| Approach: | They evaluate four large language models with human disagreements on five datasets . they find multi-perspective in-context learning outperforms standard prompting . |
| Outcome: | The proposed approach outperforms standard prompting on English labels while disaggregated soft predictions better align with human judgments in Arabic and Italian datasets. |
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have enabled high-fidelity generation of synthetic user conversation. |
| Approach: | They propose a taxonomy covering user granularity and simulation objectives . they analyze core techniques and evaluation methodologies to help them understand the latest developments . |
| Outcome: | The proposed model enables high-fidelity generation of synthetic user conversation. |
Copied to clipboard
| Challenge: | Large language models (LLMs) have impressive performance but face high computational costs and latency, limiting their deployment in resource-constrained settings. |
| Approach: | They propose a robust RAG framework for large language models via Margin-aware Preference Optimization to enhance the accuracy and reliability of SLMs. |
| Outcome: | The proposed framework surpasses state-of-the-art benchmarks on three open-domain question answering tasks. |
Copied to clipboard
| Challenge: | Effective domain adaptation typically involves supervised fine-tuning on carefully selected instruction-tuned data. |
| Approach: | They propose a model-centric data selection framework that aligns data selection with the model’s knowledge distribution to improve model performance. |
| Outcome: | The proposed framework outperforms existing methods by up to 2.97% accuracy in the healthcare domain. |
Copied to clipboard
| Challenge: | Embodied Question Answering (EQA) tasks are primarily focused on indoor environments, leaving the complexities of urban settings unexplored. |
| Approach: | They propose a task where an embodied agent answers open-vocabulary questions in dynamic city spaces. |
| Outcome: | The proposed agent achieves 60.7% of human-level answering accuracy compared to baselines . the proposed agent outperforms existing agents in open-ended city spaces . |
Copied to clipboard
| Challenge: | Multi-agent LLMs are rapidly moving from prototype to real-world use . network topology is a first-order security parameter in multi-aggent systems . |
| Approach: | They propose a framework for comparing topology-conditioned memory leakage in multi-agent LLM systems. |
| Outcome: | The proposed framework evaluates topology-conditioned memory leakage in multi-agent LLM systems. |
Copied to clipboard
| Challenge: | Multipanel images are a common form of visual representations, and humans can achieve approximately 99% accuracy on these questions. |
| Approach: | They propose a benchmark that tests multipanel visual reasoning models with 6,600 triplets of questions, answers, and multipanel images. |
| Outcome: | The proposed benchmark features 6,600 triplets of questions, answers, and multipanel images that challenge state-of-the-art Multimodal Large Language Models (MLLMs) human users can attain approximately 99% accuracy on these questions, compared with previous benchmarks. |
Copied to clipboard
| Challenge: | Existing benchmarks for Large Language Models (LLMs) are limited to false belief tasks, highlighting bottlenecks in specific dimensions. |
| Approach: | They propose a benchmark to evaluate Large Language Models' Theory of Mind capabilities . they evaluate 8000 bilingual instances across 46 paradigms and validated by 49 human annotators . |
| Outcome: | The proposed benchmark reveals performance heterogeneities and bottlenecks in 22 representative models. |
Copied to clipboard
| Challenge: | Existing methods for fingerprinting model ownership traces are vulnerable to illegal plagiarism and are not reliable. |
| Approach: | They propose a rule-driven fingerprinting framework that encodes contextual correlations across multiple dialogue turns. |
| Outcome: | The proposed framework achieves stronger stealth and robustness than previous work. |
Copied to clipboard
| Challenge: | Recent approaches suffer from insufficient and repetitive knowledge retrieval, tedious and time-consuming query parsing, and monotonous knowledge utilization. |
| Approach: | They propose a retrieval-augmented generation framework which leverages LLMs’ powerful reasoning capacity to compensate for the incompleteness of user queries. |
| Outcome: | The proposed framework improves the accuracy and reliability of Large Language Models (LLMs) by combining the rich knowledge of LLMs with Hypothesis Outputs. |
Copied to clipboard
| Challenge: | a lack of benchmarks capture real-world, cross-platform heterogeneity in GUI training . traditional methods to train GUI agents rely on centralized data collection and manual labeling . |
| Approach: | They propose a benchmark for developing and evaluating federated GUI agents across mobile, web and desktop platforms. |
| Outcome: | The proposed benchmarks show that cross-platform collaboration improves performance and identify platform and OS as the most influential factors. |
Copied to clipboard
| Challenge: | Existing approaches for optimizing domain-level sampling strategies struggle with maintaining intra-domain consistency and accurately measuring domain impact. |
| Approach: | They propose to use a Fisher-Information Matrix-guided metric to measure domain impact to ensure intra-domain consistency and accuracy. |
| Outcome: | The proposed model achieves 3.4% higher average performance while maintaining comparable training efficiency. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have been used for selection and training of data for active learning. |
| Approach: | They propose an intuitive taxonomy that categorizes LLM-based active learning techniques and discuss the transformative roles they can play in the active learning loop. |
| Outcome: | The proposed model can generate entirely new data instances and provide more cost-effective annotations with fewer labeled data instances. |
Copied to clipboard
| Challenge: | Existing approaches to prompt optimization trade off signal quality against computational cost. |
| Approach: | They propose a framework that uses a first-order gradient approximation to score segment importance in a continuous masking direction. |
| Outcome: | The proposed framework improves efficiency and robustness by using a first-order gradient approximation to score segment importance in a continuous masking direction. |
Copied to clipboard
| Challenge: | Current approaches for detoxification or preventing jailbreaking involve fine-tuning billions of parameters through gradient descent with substantial computational cost. |
| Approach: | They propose to use supervised fine-tuning and Reinforcement Learning from human feedback to modify LLMs' behavior by directly editing a small subset of parameters. |
| Outcome: | Experiments show that editing a small subset of parameters can modulate specific behaviors of LLMs, such as detoxification and resistance to jailbreak, with only inference-level computational resources. |
Copied to clipboard
| Challenge: | Existing routing methods rely on direct mapping from queries to models based on surface-level features, leading to poor generalizability on out-of-distribution data. |
| Approach: | They propose a new routing framework that recasts the routing task as a matching process of sifting similar queries from historical logs. |
| Outcome: | The proposed framework improves matching accuracy while lowering inference costs . it decouples linguistic surface forms from task-intrinsic requirements . |
Copied to clipboard
| Challenge: | Existing methods for logical reasoning with large language models suffer from insufficient rule semantic grounding and weak rule application mechanisms. |
| Approach: | They propose a theory-of-mind driven neuro-symbolic reasoning framework that integrates natural language and symbolic representations throughout the reasoning process. |
| Outcome: | The proposed model surpasses state-of-the-art models in reasoning accuracy and flexibility. |
Copied to clipboard
| Challenge: | Existing methods for large language model reasoning suffer from exploration collapse due to the semantic homogeneity of random rollouts. |
| Approach: | They propose to use latent policy optimization via iterative information bottleneck to optimize reasoning trajectories by diversifying reasoning . |
| Outcome: | Empirical results show that the proposed method achieves state-of-the-art performance with margins of up to 5.3% in accuracy and 7.4% in diversity metrics. |
Copied to clipboard
| Challenge: | Existing language evaluation benchmarks for English are limited to English . lack of such benchmarks makes it difficult to replicate success in other languages . |
| Approach: | They introduce a large-scale Chinese language understanding evaluation benchmark . the benchmark uses a set of current state-of-the-art pre-trained Chinese models . |
| Outcome: | The first large-scale Chinese Language Understanding Evaluation (CLUE) benchmark is released . the benchmark evaluates models across a wide range of tasks on original Chinese text . existing language evaluation benchmarks are mostly limited to English . |
Copied to clipboard
| Challenge: | Existing approaches to enhance speech translation focus on enhancing knowledge transfer . factors in speech that are not relevant to translation content, such as timbre and rhythm, often limit the efficiency of knowledge transfer. |
| Approach: | They propose a framework that excludes content-agnostic perturbations from speech representations to mitigate their negative impact on ST. |
| Outcome: | The proposed framework significantly improves translation performance across all translation directions in three settings and achieves preeminent performance under a *transcript-free* setting. |
Copied to clipboard
| Challenge: | Prior studies have examined the impact of structured output on LLMs’ generation quality, often presenting one-way findings. |
| Approach: | They propose to derive five potential causal structures characterizing the influence of structured output on LLMs’ generation using one assumed and two guaranteed constraints. |
| Outcome: | The proposed pipeline can be extended to other modules and is not limited to structured output but can be used in industrial applications. |
Copied to clipboard
| Challenge: | Emotional support conversation (ESC) aims to alleviate the emotional distress of individuals through effective conversations. |
| Approach: | They propose a framework that bootstraps the planning during ESC and determines the optimal strategy based on long-term returns. |
| Outcome: | The proposed framework outperforms baseline models on ESC datasets and can be used to guide the LLM to response. |
Copied to clipboard
| Challenge: | Until recently, zero-shot stance detection was limited to in-domain tasks. |
| Approach: | They propose a method for stance detection that trains a model that can generalize well to unseen targets across multiple domains. |
| Outcome: | The proposed method generalizes well to unseen targets across multiple domains over baselines on most benchmarks. |
Copied to clipboard
| Challenge: | Existing methods for document-level Event Causality Identification rely on local semantic similarity for independent event-pair discrimination . Existing approaches ignore the influence of the overall narrative backbone in the propagation of causal dependencies and the role differentiation of events within multi-cause/multi-effect structures. |
| Approach: | They propose a suggest-verify-revise approach for document-level Event Causality Identification with narrative consistency (SVRECI) they integrate heuristic causal suggestions generated by an LLM with structural suggestions derived from hypergraph modeling . |
| Outcome: | The proposed approach outperforms existing methods on event-storylines and Causal-TimeBank datasets. |
Copied to clipboard
| Challenge: | Existing target-oriented dialogs take a local and greedy strategy for response generation, where global planning is absent. |
| Approach: | They propose a global planning method for target-oriented dialog on a commonsense knowledge graph to adjust local response generation towards the global target. |
| Outcome: | The proposed method can reach the target with a higher success rate, fewer turns, and more coherent responses. |
Copied to clipboard
| Challenge: | Existing approaches to program repair provide only post-hoc, non-verifiable explanations that are not executable or verifiably. |
| Approach: | They propose a framework that couples repair generation with machine-checkable executable explanations for quantum programs where correctness hinges on subtle semantic properties such as circuit equivalence and fidelity preservation. |
| Outcome: | Experiments on QASMBench with mutation-generated quantum program bugs show that the proposed framework improves both semantic precision and explanation faithfulness over baselines that rely on unconstrained or purely natural-language explanations. |
Copied to clipboard
| Challenge: | Anomaly detection (AD) is an important machine learning task with many real-world uses, including fraud detection, medical diagnosis, and industrial monitoring. |
| Approach: | They propose a benchmark that evaluates how large language models (LLMs) can help with NLP anomaly detection. |
| Outcome: | The proposed model can perform zero-shot detection without tasks-specific training, data augmentation and model selection, and it can suggest unsupervised AD models. |
Copied to clipboard
| Challenge: | Existing neural end-to-end dialogue models have limitations on exactly interpreting the linguistic structures in dialogue history context. |
| Approach: | They propose to directly measure the capability of neural end-to-end dialogue models on understanding the entity-oriented structures via question answering. |
| Outcome: | The proposed model can understand large-scale English and Chinese human human dialogues using a large-format dataset. |
Copied to clipboard
| Challenge: | a framework for constructing dialogue world models for natural language tasks is currently lacking. |
| Approach: | They propose a framework that can be used to train a dialogue world model. |
| Outcome: | The proposed framework can predict future utterances and user beliefs . it can achieve state-of-the-art performance on emotion classification and sentiment identification . |
Copied to clipboard
| Challenge: | Grammatical Error Correction (GEC) systems perform well in academic benchmarks, but in practical applications they may not correct errors when users perform irrelevant modifications. |
| Approach: | They propose a benchmark to evaluate the context robustness of Grammatical Error Correction systems. |
| Outcome: | The proposed method improves the accuracy of errors corrected by human annotations. |
Copied to clipboard
| Challenge: | Existing LLMs are opaque and difficult to interpret, resulting in limited interpretability. |
| Approach: | They propose an interaction-aware profile generator that jointly produces user and item profiles conditioned on both user history and item evidence. |
| Outcome: | The proposed model outperforms baselines on three real-world datasets. |
Copied to clipboard
| Challenge: | Existing approaches focus on global planning, which plans toward the target before the conversation. |
| Approach: | They propose to generate a global path as a natural language sentence instead of a sequence of nodes. |
| Outcome: | The proposed method has fewer turns, more coherent semantics, and higher success rate than baselines. |
Copied to clipboard
| Challenge: | Existing QA evaluation methods struggle with open-ended and unstructured responses. |
| Approach: | They propose a hybrid framework that combines rule-based reliability with LLM-based adaptability to overcome these challenges. |
| Outcome: | The proposed framework outperforms existing models like GPT-4o and Claude-3 in accuracy and cost. |
Copied to clipboard
| Challenge: | Existing methods for concept-level grounding and instruction-level reasoning use coarse representations and iterative mask filtering. |
| Approach: | They propose an instruction-following extension of the Segment Anything Model 3 family that unifies concept-level grounding and instruction-level reasoning within a single segmentation framework. |
| Outcome: | Experiments show that SAM3-I achieves appealing performance across referring and reasoning-based segmentation while maintaining its strong concept recall ability. |
Copied to clipboard
| Challenge: | Recent work of GUI action grounding fine-tunes data from pre-trained MLLMs, but data is limited to specific GUI environments. |
| Approach: | They propose to use a GUI-based agent to collect environment-specific data and fine-tune GUI grounding models with the collected data. |
| Outcome: | The proposed model can be extended to other GUI environments to improve performance. |
Copied to clipboard
| Challenge: | Graphical User Interfaces (GUIs) are a pivotal medium for human-computer interaction. |
| Approach: | They propose a series of datasets for training visual-based GUI agents using general VLMs. |
| Outcome: | The proposed GUICourse datasets show that even a small-sized GUI agent performs better on GUI tasks. |
Copied to clipboard
| Challenge: | Existing methods for event detection use sentence-level contextual information. |
| Approach: | They propose a document embedding enhanced bi-RNN model to detect events in sentences . they use hierarchical and supervised attention based RNN to learn document embeds . |
| Outcome: | The proposed model compares with state-of-the-art models on a ACE-2005 dataset. |
Copied to clipboard
| Challenge: | Fine-tuned pre-trained language models (LMs) have enormous success in many natural language processing tasks, but they still require excessive labeled data in the fine-tuning stage. |
| Approach: | They propose a framework to enable fine-tuning pre-trained language models with weak supervision without any labeled data. |
| Outcome: | The proposed framework outperforms the strongest baseline and achieves competitive performance with fully-supervised fine-tuning methods. |
Copied to clipboard
| Challenge: | Existing code translation models only learn the contextual semantics of code during pre-training, neglecting executability information closely related to the execution state of the code. |
| Approach: | They propose an LLM specifically designed for code translation called ExeCoder . it uses executability representations such as functional semantics and syntax structures to enhance LLMs' capabilities. |
| Outcome: | The proposed model outperforms existing open-source code translation models on two metrics. |
Copied to clipboard
| Challenge: | Existing models for GUI understanding ignore a key GUI-referring task: screen reading based on user-indicated points. |
| Approach: | They propose a Tree-of-Lens agent that constructs a Hierarchical Layout Tree based on user input points and a GUI screenshot. |
| Outcome: | The proposed agent can interpret the Screen Point-and-Read task on mobile, web, and operating systems. |
Copied to clipboard
| Challenge: | PTLMs have shown remarkable success in multiple information extraction tasks . however, their performance in real-world scenarios falls short of expectations . |
| Approach: | They propose to use an entity-centric dataset to evaluate PTLMs' performance . they find that inadequate annotations in benchmark datasets lead to spurious correlations . |
| Outcome: | The proposed dataset disentangles the falsely-coupled segment and entity annotations that arises from the block-level annotation of FUNSD. |
Copied to clipboard
| Challenge: | Existing models of layout reading order do not convey the complete reading order information in the layout. |
| Approach: | They propose to model layout reading order as ordering relations over layout elements . they propose a reading-order-relation-enhancing pipeline to improve model performance . |
| Outcome: | The proposed model outperforms existing models on a visual-rich document dataset and on eight cross-domain VrD-IE/QA tasks without targeted optimization. |
Copied to clipboard
| Challenge: | Multiple-choice question answering (MCQA) uses text-to-text framework . but, there is an under-utilization of the decoder and knowledge that can be decoded . |
| Approach: | They propose a generative multiple-choice question answering model which generates a clue from the question and leverages it to enhance a reader for MCQA. |
| Outcome: | The proposed model outperforms text-to-text models on multiple MCQA datasets. |
Copied to clipboard
| Challenge: | Large multimodal models exhibit remarkable intelligence, yet their embodied cognitive abilities during motion in open-ended urban aerial spaces remain to be explored. |
| Approach: | They propose a benchmark to evaluate whether large multimodal models can process continuous first-person visual observations like humans. |
| Outcome: | The proposed model can process first-person visual observations like humans, enabling recall, perception, reasoning, and navigation. |
Copied to clipboard
| Challenge: | Commercial news provides rich semantics and timely information for automated financial risk detection. |
| Approach: | They propose a semi-supervised Semantic-Topological Iteration Network, STINMatch, along with a news-enterprise knowledge graph to endorse the risk detection enhancement. |
| Outcome: | The proposed model outperforms existing models in terms of generalization and semantics and annotation. |
Copied to clipboard
| Challenge: | Large language models have achieved remarkable success in Natural Language Processing, yet their cross-lingual consistency remains a significant challenge. |
| Approach: | They propose a method to identify cross-lingual weaknesses in Large Language Models . they construct bilingual question pairs that expose performance discrepancies between English and target languages . |
| Outcome: | The proposed method uncovers over 50% accuracy drops in target languages across models. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) produce outputs that deviate from factual reality, especially in sensitive applications such as medical consultation and legal advice. |
| Approach: | They propose a Siamese network-based model that leverages LLMs’ inner states for factual detection. |
| Outcome: | The proposed model achieves over 96% accuracy on a custom-collected factual detection dataset. |
Copied to clipboard
| Challenge: | Anomaly detection (AD) is an important machine learning task, but its effectiveness in detecting harmful content, phishing attempts, and spam reviews is limited. |
| Approach: | They introduce NLP-ADBench, the most comprehensive NLP anomaly detection benchmark to date . it includes eight curated datasets and 19 state-of-the-art algorithms . |
| Outcome: | The NLP-ADBench benchmark includes 19 state-of-the-art methods and 8 curated datasets . no single model dominates across all datasets, indicating need for automated model selection . |
Copied to clipboard
| Challenge: | Low-resource languages, especially those written in rare scripts, remain unsupported by large language models due to lack of training data. |
| Approach: | They evaluate 20 under-represented languages across three state-of-the-art multilingual LLMs and compare their methods to parameter-efficient fine-tuning. |
| Outcome: | The proposed methods compare with parameter-efficient fine-tuning (PEFT) on low-resource languages. |
Copied to clipboard
| Challenge: | In-context learning (ICL) performance is highly sensitive to prompt design, yet the impact of class label options (e.g. lexicon or order) in zero-shot classification remains underexplored. |
| Approach: | They propose a post-hoc method for selecting optimal label sets in zero-shot ICL with large language models. |
| Outcome: | The proposed method consistently achieves performance gains of 0.54 to 0.76 compared to the conventional method. |
Copied to clipboard
| Challenge: | Existing Multimodal Large Language Models (MLLMs) are predominantly trained on consistent visual-textual inputs, leaving open the question of whether they can handle semantic mismatches in layout-rich content. |
| Approach: | They propose to use multimodal inconsistency reasoning to assess MLLMs' ability to reason about semantic mismatches in webpages, presentation slides, and posters. |
| Outcome: | The proposed model outperforms open-source models in detecting inconsistencies in webpages, presentation slides, and posters while remaining vulnerable to inconsistent errors. |
Copied to clipboard
| Challenge: | Existing methods for self-training are interpreted as teacher-student frameworks, where the teacher generates pseudo-labels and the student makes predictions. |
| Approach: | They propose a differentiable self-training method that treats teacher-student as a Stackelberg game where a leader is always in a more advantageous position than a follower. |
| Outcome: | The proposed model outperforms existing methods on semi- and weakly-supervised learning tasks on semi and weak supervised tasks. |
Copied to clipboard
| Challenge: | Existing studies link hallucination to data or representation biases, but their causal origins remain unclear. |
| Approach: | They propose a causal framework to analyze and mitigate hallucination in vision-language models by using counterfactual analysis to estimate the Natural Direct Effect (NDE) of each modality and their interaction. |
| Outcome: | The proposed framework significantly reduces hallucination while preserving task performance while retaining reliability. |
Copied to clipboard
| Challenge: | Legal question answering (LQA) aims to bridge the gap between limited availability of legal professionals and the extensive volume of legal issues. |
| Approach: | They propose a legal knowledge retriever and a hierarchical legal knowledge integration framework to address multiple user-specific circumstances. |
| Outcome: | The proposed framework outperforms baselines on the legal community question-answering dataset. |
Copied to clipboard
| Challenge: | Existing methods to learn from preference-based feedback are expensive and scarce. |
| Approach: | They propose a framework that synergistically combines self-rewarding and active learning through human-AI collaboration. |
| Outcome: | The proposed framework outperforms existing methods on three reasoning benchmarks and achieves average improvements of +13.25% on GSM8K, +8.19% on MATH, and +13.16% on WebInstruct. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable linguistic comprehension and generation capability, but when applied to specialized industries, they face challenges such as hallucination, insufficient domain knowledge, and failing to incorporate the latest domain knowledge. |
| Approach: | They propose a paradigm for fine-tuning LLMs that effectively injects domain-specific knowledge while safeguarding inference-time data privacy. |
| Outcome: | The proposed model protects private data while enhancing the model's knowledge. |
Copied to clipboard
| Challenge: | Existing methods for ambiguous queries struggle to retrieve high-quality documents . DFAMS outperforms advanced FR methods by 14.37% in knowledge classification accuracy . |
| Approach: | They propose a framework that leverages dynamic information flow to identify latent query intents and construct semantically aligned knowledge partitions for accurate retrieval across heterogeneous sources. |
| Outcome: | The proposed framework outperforms existing methods in classification accuracy and retrieval recall tests. |
Copied to clipboard
| Challenge: | Existing evaluation methods rely on rule-based matching with shallow semantic understanding or adopt LLM-as-a-Judge approaches that incur high cost and latency while offering limited error interpretability. |
| Approach: | They propose a curriculum learning based hierarchical framework for QA task evaluation that supports quick scoring and fine-grained error analysis. |
| Outcome: | The proposed framework outperforms baseline methods on quick scoring and error analysis tasks while being 25 faster. |
Copied to clipboard
| Challenge: | Existing methods for short TST are difficult to implement and can cause content degradation. |
| Approach: | They propose a method to vary the style polarity of text while preserving semantic content. |
| Outcome: | The proposed method improves over baselines and is highly efficient. |
Copied to clipboard
| Challenge: | Contextualized embeddings are expensive and resource-demanding, hence environmentally unfriendly. |
| Approach: | They propose a method to convert contextualized embeddings from pre-trained models into static embeddables using synonym knowledge and weighted vector distribution. |
| Outcome: | The proposed method outperforms baseline embeddings by a large margin through extrinsic and intrinsic tasks. |
Copied to clipboard
| Challenge: | Existing knowledge graphs are incomplete in tracking complex semantic relations of the target-oriented dialogue. |
| Approach: | They combine methods of knowledge retrieval and relationship prediction to construct a context-related dynamic KG and a metric to evaluate the tracked path automatically. |
| Outcome: | The proposed method can control the agent more logically and smoothly toward the complex target. |
Copied to clipboard
| Challenge: | Existing video moderation systems rely on fragmented black-box classification models that are difficult to maintain and lack transparency. |
| Approach: | They propose a Unified Vision-Language model for Video Moderation that generates policy-aware captions that serve as an interpretable intermediate representation. |
| Outcome: | The proposed model reduces violation leakage and overkill rate by 42.7% while reducing maintenance costs. |
Copied to clipboard
| Challenge: | Existing systems lack a self-emotion determination mechanism to drive the streaming text-to-speech (TTS) synthesis. |
| Approach: | They propose an emotion-planning framework that determines the emotion prior to the textual generation, grounding the downstream emotional TTS in a streaming manner. |
| Outcome: | The proposed framework outperforms baselines on DailyDialog, EmoryNLP, IMEOCAP, and MELD on emotional alignment, contextual coherence, and expressive fluency. |
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly deployed in security-sensitive applications . recent defenses rely on supervised fine-tuning with benign and malicious labels . position bias arises when benign content placed later in a prompt is rejected at much higher rates . |
| Approach: | They analyze three recurring shortcut behaviors induced by supervised fine-tuning . position bias arises when benign content placed later in a prompt is rejected . token trigger bias occurs when strings common in attack data raise rejection probability . |
| Outcome: | The proposed model overrides intended logic when adversarial instructions appear . the proposed model has low rejection rates but narrow correlations in defense data . |
Copied to clipboard
| Challenge: | Existing medical Large Language Models (LLMs) follow a reactive paradigm, risking diagnostic errors by answering before seeking sufficient details. |
| Approach: | They propose a reinforcement learning framework that transitions LLMs toward a proactive paradigm, enabling them to ask clinically valuable questions before decision-making. |
| Outcome: | Experiments on partial-information medical benchmarks show that ProMed outperforms state-of-the-art methods by 6.29% on average and delivers a 54.45% gain over the reactive paradigm. |
Copied to clipboard
| Challenge: | Existing approaches to textual robustness evaluation focus on slightly modifying the input data, which maintains the original meaning and results in a different prediction. |
| Approach: | They propose a multilingual robustness evaluation toolkit for NLP that integrates universal text transformations, task-specific transformations and adversarial attack. |
| Outcome: | The toolkit includes universal text transformation, task-specific transformation, adversarial attack, subpopulation, and their combinations to provide comprehensive robustness analyses. |
Copied to clipboard
| Challenge: | Existing approaches to training agents for visual-language models trap them in local optima, hindering exploration and error correction with the environment. |
| Approach: | They propose a hierarchical training recipe that bridges atomic action execution and strategic task completion. |
| Outcome: | The proposed training recipe bridges atomic action execution and strategic task completion. |
Copied to clipboard
| Challenge: | Multimodal machine translation (MMT) models focus on intermodal interactions, but focus on simple interactions between nouns and entities in image, overlooking global semantic alignment. |
| Approach: | They propose a Text-Image In-depth Questioning method to deepen interactions and optimize translations by utilizing visual data to capture global semantic alignment. |
| Outcome: | The proposed method achieves state-of-the-art results on five translation directions of Multi30K and AmbigCaps, with +2.35 BLEU on the challenging MSCOCO benchmark. |