Papers by Yi Zhao
Copied to clipboard
| Challenge: | Existing pruning methods require inefficient retraining for billion-scale LLMs or rely on heuristicically designed metrics to determine pruning masks, leading to performance degradation. |
| Approach: | They propose a convex optimization model that induces sparsity in large language models by leveraging FISTA. |
| Outcome: | The proposed method can remove 50% of model parameters while retaining 98.6% and 95.6% of the zero-shot performance. |
Copied to clipboard
| Challenge: | Lowrank adaptation and its variants introduce significant latency in multi-tenant settings, hindering their applications in the industry. |
| Approach: | They propose a framework to fine-tune LoRA modules on a large-scale instruction tuning dataset. |
| Outcome: | The proposed framework outperforms existing PEFT methods and significantly reduces inference latency. |
Copied to clipboard
| Challenge: | Existing methods for analyzing and utilizing toxic samples are limited . current methods fail to fully harness their potential . |
| Approach: | They propose a diverse detoxification framework that leverages toxic samples' diversity . they propose MPSG strategy and SC-DPO approach to elicit personalized toxic responses . |
| Outcome: | The proposed framework could be used to optimize large language models for user safety . it incorporates two components: MPSG strategy and SC-DPO approach . |
Copied to clipboard
| Challenge: | Existing approaches to mathematical reasoning rely on static heuristics or pre-determined reasoning strategies. |
| Approach: | They propose an adaptive framework that integrates fuzzy theory into LLM-based mathematical reasoning. |
| Outcome: | The proposed framework outperforms state-of-the-art models while offering effective and interpretable diagnostics of intermediate problem-solving states. |
Copied to clipboard
| Challenge: | Existing models for multiparty dialogue question answering (QA) do not consider logical inference relations in multiparty dialogs, leading to suboptimal performance. |
| Approach: | They propose a memory network with logical inference for extractive QA in multiparty dialogues. |
| Outcome: | The proposed model achieves state-of-the-art on Molweni and FriendsQA benchmarks. |
Copied to clipboard
| Challenge: | Few-shot crosslingual transfer outperforms zero-shot with pretrained encoders like multilingual BERT. |
| Approach: | They conduct an experimental study on 40 sets of sampled few shots for six diverse NLP tasks across up to 40 languages. |
| Outcome: | The proposed model outperforms state-of-the-art approaches on lexical features and a full model finetuning approach outperformed several state- of-the art approaches. |
Copied to clipboard
| Challenge: | Sparse Mixture-of-Experts (SMoE) architectures require loading all expert parameters . previous work focused on expert pruning and merging but focused on neuron-level structure . |
| Approach: | They propose a task-agnostic framework for expert pruning and reconstruction . it prunes redundant experts using router statistics, then decomposes them into neuron-level expert segments . |
| Outcome: | The proposed framework reduces the number of experts and memory usage, making it easier to deploy. |
Copied to clipboard
| Challenge: | SimulSpeech is an end-to-end simultaneous speech to text translation system . conventional approaches to simultaneous speech translation divide the translation process into two stages . |
| Approach: | They develop an end-to-end simultaneous speech to text translation system which translates speech in source language to text in target language concurrently. |
| Outcome: | The proposed system achieves reasonable BLEU scores and lower delay compared to full-sentence translation model. |
Copied to clipboard
| Challenge: | Non-autoregressive text to speech models ignore correlation in time and frequency domains, causing blurry results. |
| Approach: | They revisit the problem of over-smoothness in non-autoregressive text to speech models . they use methods that reduce complexity of data distributions and improve modeling methods . |
| Outcome: | The proposed models achieve better voice quality and faster inference speed than autoregressive models. |
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have demonstrated their remarkable capabilities in natural language understanding and generation, but they struggle with formal logical reasoning. |
| Approach: | They propose to incorporate visual logic diagrams into LLMs’ reasoning workflows to enhance their performance on formal logic tasks. |
| Outcome: | The proposed model improves on syllogistic and conditional reasoning with programmatically generated Venn, Euler, and Linear diagrams. |
Copied to clipboard
| Challenge: | Existing methods for data augmentation do not fully exploit the potential of DA in NLP. |
| Approach: | They propose an easy and plug-in framework for data augmentation to support effective text classification. |
| Outcome: | The proposed framework outperforms existing methods in most cases, but not using agent networks or pre-trained generation networks. |
Copied to clipboard
| Challenge: | Existing RAG methods focus on external retrieval, while ignoring the rich content of the model. |
| Approach: | They propose a framework that enhances explicit synergy over parametric and retrieved knowledge by integrating external retrieval components into the input context of the LLMs. |
| Outcome: | The proposed framework enhances explicit synergy over parametric and retrieved knowledge. |
Copied to clipboard
| Challenge: | Pre-trained language models (PrLMs) have shown impressive improvements for various downstream tasks including various dialogue related ones. |
| Approach: | They propose to use pre-trained language models to simulate dialogue features on general plain text with common language model training objectives to improve performance. |
| Outcome: | The proposed method is fine-tuned on three public multi-turn dialogue datasets and achieves significant and consistent improvement over the plain PrLMs. |
Copied to clipboard
| Challenge: | Large language models (LLMs) face memory challenges due to the high cost of backpropagation. |
| Approach: | They propose a zeroth-order (ZO) optimization that matches memory usage to inference . they propose scalable and memory-efficient zeroth order (ZE) optimizer that integrates annealed A-GNB gradients with diagonal Hessian estimation and layer-wise clipping as a second-order pre-conditioner. |
| Outcome: | The proposed algorithm outperforms state-of-the-art methods with an average speedup of 20 over MeZO on RoBERTa-large and OPT-1.3B. |
Copied to clipboard
| Challenge: | Existing approaches to VideoQA focus on utilizing frame- or object-level visual representations, but they neglect visual-language interactions. |
| Approach: | They propose to break down video into trajectories and first leverage trajectory feature in VideoQA to enhance alignment between two modalities. |
| Outcome: | The proposed method outperforms all the state-of-the-art models on the NExT-QA benchmark. |
Copied to clipboard
| Challenge: | Existing benchmarks for Large Language Models (LLMs) are limited to false belief tasks, highlighting bottlenecks in specific dimensions. |
| Approach: | They propose a benchmark to evaluate Large Language Models' Theory of Mind capabilities . they evaluate 8000 bilingual instances across 46 paradigms and validated by 49 human annotators . |
| Outcome: | The proposed benchmark reveals performance heterogeneities and bottlenecks in 22 representative models. |
Copied to clipboard
| Challenge: | Low-rank adaptation and its mixture-of-experts (MOE) methods are highly effective but introduce significant latency in multi-tenant settings due to the LoRA modules and MOE routers added to multiple linear modules. |
| Approach: | They propose a low-rank adaptation variant that considers each LoRA module as an expert and employs a prompt-aware routing mechanism. |
| Outcome: | Extensive analysis on commonsense reasoning tasks and math reasoning tasks show that MiLoRA outperforms strong PEFT baselines with comparable tunable parameter budgets. |
Copied to clipboard
| Challenge: | Existing QG systems perform substantially worse in answering multi-hop questions than single-hop ones. |
| Approach: | They propose a framework that progressively increases question difficulty through step-by-step rewriting under the guidance of an extracted reasoning chain. |
| Outcome: | The proposed framework increases question difficulty through step-by-step rewriting under the guidance of an extracted reasoning chain. |
Copied to clipboard
| Challenge: | Existing models for matching dialogue responses rely on semantic and functional dependencies . a recent study only uses the last utterance in context for matching a reply . |
| Approach: | They propose a model that matches a response with its multi-turn context using attention. |
| Outcome: | The proposed model outperforms the state-of-the-art models on two large-scale multi-turn response selection tasks. |
Copied to clipboard
| Challenge: | Large language models (LLMs) have evolved into interactive agents capable of planning, tool use, and task execution across various tasks. |
| Approach: | They propose a platform that leverages large language models to generate agent-tuning data for fine-tuneing smaller, specialized models. |
| Outcome: | MIMIR enables large models to simulate various roles and create interaction data, which can then be used to fine-tune open-source models like LLaMA2. |
Copied to clipboard
| Challenge: | Existing methods for evaluating novelty have been proposed, but there is no systematic evaluation of their ability to generate novelty evaluations. |
| Approach: | They propose a benchmark to evaluate large language models’ ability to generate novelty evaluations in support of human peer review. |
| Outcome: | The proposed framework evaluates the quality of LLM-generated novelty evaluations under different prompting strategies. |
Copied to clipboard
| Challenge: | Existing methods to improve inference efficiency target to reduce per-layer latency, but ignore cumulative latency due to number of layers. |
| Approach: | They propose to identify quasi-independent layers that can be concurrently computed to significantly decrease inference latency. |
| Outcome: | Empirical results show that the proposed method reduces latency by 48.3% on LLaMA-33B while maintaining close level of performance. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are efficient assistants to humans in software development tasks, but they can cause errors during the development process. |
| Approach: | They propose an intention aligned multi-agent framework that ensures that all agents work based on a consensus. |
| Outcome: | The proposed framework reduces errors and improves the quality of generated software code. |
Copied to clipboard
| Challenge: | Non-autoregressive (NAR) models generate all tokens in parallel, resulting in faster generation speed compared to autoregressive models. |
| Approach: | They propose to use knowledge distillation and source-target alignment to bridge the gap between NAR and autoregressive models in various tasks. |
| Outcome: | The proposed techniques can speed up NAR models in some tasks but not all . the proposed techniques reduce target token dependency while allowing for faster inference . |
Copied to clipboard
| Challenge: | Experimental results show that Generative adversarial networks sacrifice sample diversity for quality and speed, while diffusion models exhibit outperformed sample quality and diversity at a high computational cost. |
| Approach: | They propose to combine GANs and diffusion probabilistic models to achieve better sample quality and diversity. |
| Outcome: | The proposed models outperform GANs and diffusion models in speech synthesis . the proposed models enjoy an efficient 4-step sampling process and exhibit better sample diversity . |
Copied to clipboard
| Challenge: | Video Content Discovery (VCD) is to identify specific videos defined by a pre-specified text policy. |
| Approach: | They propose a Vision-Language Large Model-driven video content discovery system called VENUS to solve these problems. |
| Outcome: | The proposed system generates high-quality, VCD-specific data for model training and extends it to support it better. |
Copied to clipboard
| Challenge: | PlotGen-Bench evaluates vision-language models' ability to generate executable visualization code from plots under realistic and complex visualization requirements. |
| Approach: | They propose a benchmark to evaluate plot-to-code generation in vision-language models . they use Matplot, Matplos, Mat3D, Mat4D, and Mat4E to evaluate their performance . |
| Outcome: | The proposed benchmark covers 9 major categories, 30 subcategories, and 3 core tasks . it covers 2D, 3D and animated plots across 5 widely used visualization libraries. |
Copied to clipboard
| Challenge: | Existing slot filling models memorize inherent patterns of entities and contexts from training data. |
| Approach: | They propose a perturbed semantic structure awareness transferring method for slot filling models . they use two MLM-based training strategies to learn contextual semantic structure and word distribution . |
| Outcome: | The proposed method outperforms existing methods and gains strong generalization while preventing model from memorizing inherent patterns of entities and contexts. |
Copied to clipboard
| Challenge: | Existing benchmarks lack comprehensive evaluations, particularly in multi-level reasoning, making it difficult to identify model limitations. |
| Approach: | They propose to use Agri-CM3 to assess multi-level reasoning in agricultural management by integrating multiple data modalities. |
| Outcome: | The Agri-CM3 benchmark includes 3,939 images and 15,901 multi-level multiple-choice questions with detailed explanations. |
Copied to clipboard
| Challenge: | Existing models for PRVR use unimodal features, but powerful pretrained vision-language models like CLIP are underexplored. |
| Approach: | ProPy is a model with systematic architectural adaptation of CLIP specifically designed for PRVR. |
| Outcome: | ProPy outperforms existing models on three public datasets in terms of performance on the datasets. |
Copied to clipboard
| Challenge: | Existing benchmarks for large language models (LLMs) are coarse, single-dimensional metrics and do not explicitly assess fine-grained legal reasoning. |
| Approach: | They propose a Practical Law Benchmark to evaluate large language models in real-world legal practice scenarios. |
| Outcome: | The proposed model is based on 850 questions and 13 scenarios with expert-designed evaluation rubrics. |
Copied to clipboard
| Challenge: | Existing memory systems rely on static, hand-crafted update rules for personalization, but sparse outcome rewards provide weak supervision, resulting in unstable long-horizon optimization. |
| Approach: | They propose a memory guideline optimization framework that learns how memory should be organized and what information to update. |
| Outcome: | The proposed framework learns how memory should be organized and what information to update. |
Copied to clipboard
| Challenge: | Large language models struggle to process lengthy inputs due to limited length generalization and attention’s quadratic computational demands. |
| Approach: | They propose a training-free framework that allows each head to attend to important context chunks instead of allowing each head a full sentence . |
| Outcome: | The proposed framework unlocks multi-head attention's untapped potential by allowing each head to attend to important context chunks instead of the full sentence. |
Copied to clipboard
| Challenge: | Large language models (LLMs) have been successful in understanding language and processing text, but their cost prohibits their practical applications. |
| Approach: | They propose a multi-agent collaboration method that breaks down lengthy documents into smaller, more manageable chunks and organizes the member agents to read their assigned chunks. |
| Outcome: | The proposed method achieves 16.42% and 1.63% accuracy gains over existing models on single-hop and multi-hop QA settings. |
Copied to clipboard
| Challenge: | Existing safety benchmarks focus on explicitly harmful content, but ignore context-dependent expressions such as dogwhistles. |
| Approach: | They propose a benchmark for evaluating LLM safety under dogwhistle-driven prompts . their findings expose a blind spot in current safety evaluation practices . |
| Outcome: | The proposed benchmark compared safety performance with toxic terms using dogwhistle-driven prompts. |
Copied to clipboard
| Challenge: | Existing research on building ES conversation systems only considered single-turn interactions with users, which is over-simplified and has limited support for multi-turn systems. |
| Approach: | They propose a multi-turn ES conversation system that uses lookahead heuristics to estimate future user feedback after using particular strategies. |
| Outcome: | The proposed system significantly outperforms baselines in both dialogue generation and strategy planning. |
Copied to clipboard
| Challenge: | Large language models (LLMs) are a challenge due to their internal reasoning processes. |
| Approach: | They propose an algorithm that can optimize attention matrices by performing attention mapping between small and large LLMs. |
| Outcome: | The proposed framework can reduce KV cache usage by 22.1% and accelerate prefill by 15% without sacrificing performance. |
Copied to clipboard
| Challenge: | Anomaly detection (AD) is an important machine learning task with many real-world uses, including fraud detection, medical diagnosis, and industrial monitoring. |
| Approach: | They propose a benchmark that evaluates how large language models (LLMs) can help with NLP anomaly detection. |
| Outcome: | The proposed model can perform zero-shot detection without tasks-specific training, data augmentation and model selection, and it can suggest unsupervised AD models. |
Copied to clipboard
| Challenge: | Existing approaches to retrieve information from large language models (LLMs) but they fail to address the preference gap between retrievers and LLMs. |
| Approach: | They propose a retrieval module that dynamically injects retrieved information into the input context of large language models (LLMs) This approach aligns the retriever’s and LLM’s preferences by defining a new metric, “gain”, which measure how well an input passage contributes to correct outputs. |
| Outcome: | The proposed approach has shown significant success in various NLP tasks, but there is a preference gap between retrievers and LLMs. |
Copied to clipboard
| Challenge: | Entity recognition is a widely benchmarked task in natural language processing . a neural architecture called BiLSTM-CRF is used to model the language sequences . |
| Approach: | They propose a neural architecture called BiLSTM-CRF to model the language sequences. |
| Outcome: | The proposed system achieves state-of-the-art on English entity recognition task and also in other languages. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated exceptional capabilities across diverse natural language processing tasks. |
| Approach: | They propose a training-free method for unifying different specialized LLMs into a single model using model-wise and layer-wise pruning and scaling. |
| Outcome: | The proposed method outperforms existing merging techniques and surpasses models fine-tuned on combined datasets in most scenarios. |
Copied to clipboard
| Challenge: | Existing jailbreak methods struggle to balance effectiveness with robustness against adaptive safety mechanisms. |
| Approach: | They propose a novel approach that targets Large Reasoning Models through an adaptive encryption pipeline designed to overwhelm their reasoning capabilities. |
| Outcome: | The proposed approach achieves an attack success rate of 85.6% on OpenAI GPT-o4-mini, outperforming state-of-the-art baselines by a significant margin of 17.2%. |
Copied to clipboard
| Challenge: | Vision Language Models struggle with cultural-specific knowledge, especially in languages other than English and in underrepresented cultural contexts. |
| Approach: | They propose a visual question answering (VQA) dataset with text-image pairs across 30 languages and dialects and a training dataset. |
| Outcome: | The proposed model performs better with correct location context, but struggles with adversarial contexts and predicting specific regional cuisines and languages. |
Copied to clipboard
| Challenge: | Existing methods to verify factuality of claims do not provide sufficient evidence for explainable fact-checking systems. |
| Approach: | They propose a method to automatically retrieve and summarize evidence from the Web and a novel multilingual explainable fact-checking dataset on the Russia-Ukraine conflict in 2022. |
| Outcome: | The proposed method can retrieve and summarize evidence from the Web and generate explanations in 16 languages. |
Copied to clipboard
| Challenge: | Existing visual relationship detection models only use numeric ids of relation labels for training, but ignore semantic correlation between labels. |
| Approach: | They propose a visual Relationship prediction framework that transfers natural language knowledge from Contrastive Language-Image Pre-training models to enhance the relationship prediction. |
| Outcome: | The proposed framework improves visual relationship prediction by matching semantic correlations with relation triplets. |
Copied to clipboard
| Challenge: | Existing methods rely on information entropy as the metric to compress lexical units, but ignore attention-critical tokens and information . recent advent of In-Context Learning (ICL), Chain-of-Thought (CoT), and Retrieval Augmented Generation (RAG) technologies has significantly invigorated the landscape of applications based on Large Language Models (LLMs). |
| Approach: | They propose a dynamic attention-aware approach to task-agnostic prompt compression . they integrate entropy and attention information to achieve fine-grained prompt compression. |
| Outcome: | Experiments show that the proposed approach improves across tasks and LLMs. |
Copied to clipboard
| Challenge: | Existing evaluations of LLMs in finance are text-only, monolingual, and largely saturated by current models. |
| Approach: | They propose a multilingual and multimodal benchmark for evaluating LLMs in real financial contexts. |
| Outcome: | The first expert-annotated multilingual and multimodal benchmark is released . it evaluates 21 leading LLMs and shows they perform better in multilingual settings . |
Copied to clipboard
| Challenge: | Knowledge distillation is a technique of transferring knowledge from large, complex models to smaller ones. |
| Approach: | They propose a method utilizing chain-of-thought distillation to transfer knowledge from large, complex models to smaller ones by maximizing mutual information of the representation features of the two tasks. |
| Outcome: | The proposed method outperforms the state-of-the-art knowledge distillation method on four datasets. |
Copied to clipboard
| Challenge: | Emotion cause extraction (ECE) aims to extract the causes behind certain emotion in text. |
| Approach: | They propose a bidirectional hierarchical attention network corresponding to the specified candidate cause clause to capture document-level context in a structured and dynamic manner. |
| Outcome: | The proposed method achieves competitive performances on two public datasets in Chinese and English. |
Copied to clipboard
| Challenge: | Experiments show that reinforcement learning (RL) can refine the reasoning abilities of large language models (LLMs) but requires a key prerequisite: the model must already be able to generate high-utility reasoning paths with non-negligible probability. |
| Approach: | They propose a framework that uses answer-conditioned reasoning as a variational surrogate for question-only reasoning. |
| Outcome: | Experiments on 11 benchmarks and 3 models show that RAVR reduces hesitation, strengthens conclusion consolidation, and promotes problem-specific strategies in reasoning. |
Copied to clipboard
| Challenge: | Existing models for GUI understanding ignore a key GUI-referring task: screen reading based on user-indicated points. |
| Approach: | They propose a Tree-of-Lens agent that constructs a Hierarchical Layout Tree based on user input points and a GUI screenshot. |
| Outcome: | The proposed agent can interpret the Screen Point-and-Read task on mobile, web, and operating systems. |
Copied to clipboard
| Challenge: | Experimental results show that dual encoders outperform sparse and dense retrievers on the BEIR dataset significantly. |
| Approach: | They challenge belief that bottleneck layer is too limited for out-of-domain generalization . they scale up the model while keeping bottleneck as a single dot-product with a fixed size . |
| Outcome: | The proposed model outperforms sparse and dense retrievers on the BEIR dataset significantly. |
Copied to clipboard
| Challenge: | PTLMs have shown remarkable success in multiple information extraction tasks . however, their performance in real-world scenarios falls short of expectations . |
| Approach: | They propose to use an entity-centric dataset to evaluate PTLMs' performance . they find that inadequate annotations in benchmark datasets lead to spurious correlations . |
| Outcome: | The proposed dataset disentangles the falsely-coupled segment and entity annotations that arises from the block-level annotation of FUNSD. |
Copied to clipboard
| Challenge: | Existing models of layout reading order do not convey the complete reading order information in the layout. |
| Approach: | They propose to model layout reading order as ordering relations over layout elements . they propose a reading-order-relation-enhancing pipeline to improve model performance . |
| Outcome: | The proposed model outperforms existing models on a visual-rich document dataset and on eight cross-domain VrD-IE/QA tasks without targeted optimization. |
Copied to clipboard
| Challenge: | Existing studies study sentiment and emotion separately and do not fully exploit the complementary knowledge behind the two. |
| Approach: | They propose a multimodal sentiment knowledge-sharing framework that unifies MSA and ERC tasks from features, labels, and models. |
| Outcome: | The proposed framework achieves consistent improvements on four public benchmark datasets on MOSI, MOSEI, MELD, and IEMOCAP. |
Copied to clipboard
| Challenge: | Large Reasoning Models suffer from high inference latency due to autoregressive reasoning . SpecReason adopts a polling-based design that repeatedly invokes the LRM for verification at every step . |
| Approach: | They propose a trigger-based collaborative reasoning framework that delegates most reasoning to the SRM and activates LRM intervention only when necessary. |
| Outcome: | The proposed framework reduces latency and API cost by 73.3% under edge–cloud conditions. |
Copied to clipboard
| Challenge: | Existing methods for speech editing still suffer from over-smoothing problem and lack of robustness due to stutter. |
| Approach: | They propose a stutter-oriented automatic speech editing model that incorporates sutter information into the hidden sequence. |
| Outcome: | The proposed model achieves state-of-the-art performance on a speech recording dataset . it can improve fluency of stuttering speech in terms of objective and subjective metrics. |
Copied to clipboard
| Challenge: | Existing Retrieval-Augmented Generation systems treat structure as a physical navigational skeleton rather than intrinsic semantic knowledge. |
| Approach: | They propose a framework that redefining hierarchy as intrinsic semantics and uses snippets to enrich hierarchical lineage. |
| Outcome: | The proposed framework outperforms state-of-the-art hierarchical and graph-based benchmarks on FinTierQA Gold. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown remarkable capabilities in automating code generation, but they suffer from insufficient exploration of the vast solution space. |
| Approach: | They propose a large-scale LLM-driven code generation framework that efficiently finds high-quality solutions in only a few iterations. |
| Outcome: | The proposed framework outperforms baselines while maintaining reasonable time and computational costs. |
Copied to clipboard
| Challenge: | Unified Multimodal Models have achieved remarkable success in cross-modal comprehension, but a gap persists in their ability to translate internal knowledge into faithful and controllable synthesis. |
| Approach: | They propose a self-improvement framework that partitions a single UMM into three collaborative roles: Proposer, Solver, and Judge. |
| Outcome: | The proposed framework improves on TIIF, DPG, CompBench and UniCycle benchmarks. |
Copied to clipboard
| Challenge: | Anomaly detection (AD) is an important machine learning task, but its effectiveness in detecting harmful content, phishing attempts, and spam reviews is limited. |
| Approach: | They introduce NLP-ADBench, the most comprehensive NLP anomaly detection benchmark to date . it includes eight curated datasets and 19 state-of-the-art algorithms . |
| Outcome: | The NLP-ADBench benchmark includes 19 state-of-the-art methods and 8 curated datasets . no single model dominates across all datasets, indicating need for automated model selection . |
Copied to clipboard
| Challenge: | Inertial sensors can measure the acceleration and angular velocity of moving objects and are widely used in electronic devices such as smartphones, smartwatches, and fitness bands. |
| Approach: | They propose to use Chinese glyph encoding, forced optimal transport, and semantic relevance alignment to acquire unlimited training samples for Chinese inertial writing recognition. |
| Outcome: | The proposed system improves the performance of six widely used classifiers from 6.7% to 98.4%. |
Copied to clipboard
| Challenge: | Existing methods for parameter-efficient fine-tuning excel in the context of single-backbone multi-tenant applications. |
| Approach: | They propose to integrate a lightweight vector generator within each Transformer layer to improve prompt-aware representation adjustment. |
| Outcome: | The proposed method surpasses current benchmarks in terms of performance despite having a similar number of adjustable parameters. |
Copied to clipboard
| Challenge: | Tabular data is an essential resource for many fields, but current methods do not fully utilize the rich information available in tables. |
| Approach: | They propose a method that utilizes metadata information to generate tabular data . they propose long-token partitioning algorithms that enable AIGT to model tables of any scale . |
| Outcome: | The proposed approach achieves state-of-the-art on 14 out of 20 public datasets and two real industry datasets within the Alipay risk control system. |
Copied to clipboard
| Challenge: | Existing VLMs are insensitive to information differences induced by slight perspective changes. |
| Approach: | They propose a visual perspective-taking task that requires robots to interpret human-centric instructions and identify corresponding objects from robot perspectives. |
| Outcome: | The proposed method improves performance by up to 18% and generalizes effectively to robotic and dynamic scenarios. |
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have significantly enhanced automated program synthesis. |
| Approach: | They propose a model-adaptive and verification–enhanced framework for competition-level code generation that leverages adaptive assessment aligned with the model’s capabilities to select planning strategies while providing timely feedback and correction via multi-perspective verification. |
| Outcome: | The proposed framework outperforms existing state-of-the-art approaches on livecodebench, humanEval+, MBPP+, and codecontests, and achieves pass@1 results exceeding 3%–40%. |
Copied to clipboard
| Challenge: | Existing methods that rely on limited demos and out-of-demonstration (OOD) queries fail when faced with out- of-demotion queries. |
| Approach: | They propose a query-aware prompting method that elicits the inherent generalizability of large language models by query-based demo generation. |
| Outcome: | The proposed method outperforms state-of-the-art methods in the OOD setting and two public math benchmarks. |
Copied to clipboard
| Challenge: | Existing methods for expressive text-to-speech only implicitly learn prosody with masked token reconstruction tasks. |
| Approach: | They propose a cross-modal contrastive pre-training framework that learns from prosody variance of the same text token under different contexts. |
| Outcome: | The proposed framework can learn from prosody variance of a text token under different contexts. |
Copied to clipboard
| Challenge: | Existing document understanding models focus on single-modal inputs such as images or texts. |
| Approach: | They propose to use a spatial-aware adapter to adapt transformer-based language models to document domain to exploit multi-modal information. |
| Outcome: | The proposed model significantly improves the OOD detection performance compared to using a standard language model and to competitive baselines. |
Copied to clipboard
| Challenge: | Existing techniques for pitch correction are limited to intonation but ignore the overall aesthetic quality. |
| Approach: | They propose a novel time-warping approach for pitch correction to synchronize the amateur recording with the template pitch curve. |
| Outcome: | The proposed model improves intonation and vocal tone while keeping content and vocal timbre. |
Copied to clipboard
| Challenge: | a new generation of (M)LLMs is enabling the creation of superintelligent AI assistants . OS Agents can complete tasks autonomously and have the potential to significantly enhance the lives of billions of users worldwide. |
| Approach: | They propose to build OS Agents that operate within operating systems' GUIs and GUIs . they examine evaluation metrics and benchmarks to identify promising directions . |
| Outcome: | The proposed agents are based on operating systems (OS) and operating systems frameworks. |
Copied to clipboard
| Challenge: | Existing models for speech-to-speech translation suffer from distinct degradation in noisy environments and fail to translate visual speech. |
| Approach: | They propose a text-based audio-visual speech-to-speech translation model that integrates visual information with audio-only data to improve system robustness. |
| Outcome: | The proposed model outperforms models trained on audio-only corpus in two languages . it also improves with low-resource audio-visual data, compared with baselines . |
Copied to clipboard
| Challenge: | Interpretative audiobooks are becoming more popular, but their manual creation process remains time-consuming and resource-intensive. |
| Approach: | They propose a multi-agent collaboration system that leverages large language models and speech synthesis technology to generate podcast-like audiobook interpretations. |
| Outcome: | The proposed system is open source and open to the public. |
Copied to clipboard
| Challenge: | Existing methods for enhancing large language models (LLMs) lack explicit mechanisms for guiding diverse exploration and instead prioritize efficiency and performance over diversity. |
| Approach: | They propose a reinforcement learning-based framework that decomposes the generation process into explicitly planned intermediate steps and introduces divergence at the planning phase based on diversity variation. |
| Outcome: | The proposed method significantly outperforms existing baselines on creative writing benchmarks on a semi-structured long chain-of-thought (CoT) it introduces divergence at the planning phase based on diversity variation, alongside a group-aware diversity reward to encourage distinct trajectories. |
Copied to clipboard
| Challenge: | Existing approaches to multi-hop question answering lack effective control over reasoning paths, leading to astray results. |
| Approach: | They propose a framework for multi-hop question answering that trains an end-to-end reasoning path navigator to provide a powerful sub-question decomposer by fine-tuning the Llama3.1-8B model. |
| Outcome: | The proposed framework trains an end-to-end reasoning path navigator . it is able to provide a powerful sub-question decomposer by fine-tuning the Llama3.1-8B model . |
Copied to clipboard
| Challenge: | Expressive text-to-speech aims to generate high-quality samples with rich prosody . prosodic attributes in highly dynamic voices are difficult to capture and model without intonation . |
| Approach: | They propose a pipeline that enhances prosody modeling and sampling by introducing a self-supervised masked autoencoder and a diffusion model to sample diverse prosodic patterns within the latent space. |
| Outcome: | The proposed pipeline achieves new state-of-the-art in text-to-speech with natural and expressive synthesis. |