Papers by Yun Chen
Copied to clipboard
| Challenge: | In this paper, we propose to extend the recently introduced model-agnostic meta-learning algorithm for low-resource neural machine translation (NMT). |
| Approach: | They propose to extend the recently introduced meta-learning algorithm for low-resource neural machine translation (NMT) they frame low-Resource translation as a meta- learning problem where we learn to adapt to low-REsource languages based on multilingual high-resourced language tasks. |
| Outcome: | The proposed meta-learning algorithm outperforms the multilingual, transfer learning based approach and can train a competitive NMT system with only a fraction of training examples. |
Copied to clipboard
| Challenge: | Existing processes that reward for each step are one-directional and lack a mechanism to model the distance to the final target. |
| Approach: | They propose a process supervision model that evaluates the correctness of previous steps and the probability of future success. |
| Outcome: | The proposed model outperforms existing supervision models like ORM and PRM on reasoning tasks and improves solution re-design. |
Copied to clipboard
| Challenge: | Existing adapter-based transfer methods treat instruction-tuned models as passive targets . direct fine-tuning can disrupt this delicate balance and lead to instability or performance degradation. |
| Approach: | They propose a framework that incorporates instruction-level guidance into task adaptation. |
| Outcome: | The proposed framework outperforms direct fine-tuning and representative transfer-based baselines while maintaining robust generalization and favorable test-time scaling behavior. |
Copied to clipboard
| Challenge: | Existing studies have shown that personality-guided code generation improves software development outcomes when individuals are assigned tasks that match their personality types. |
| Approach: | They evaluate how emulating personality traits appropriate to the coding tasks affects LLM performance by using seven widely adopted LLMs. |
| Outcome: | The proposed approach improves pass rates in 23 out of 28 LLM-dataset combinations, while emulating personality traits can be easily integrated with other prompting strategies to further boost performance. |
Copied to clipboard
| Challenge: | Existing work on improving cross-lingual transferability of NMT model is under-explored. |
| Approach: | They propose a model that leverages a multilingual pretrained encoder to improve cross-lingual transferability. |
| Outcome: | The proposed model outperforms mBART and m2m-100 on a zero-shot cross-lingual transfer task. |
Copied to clipboard
| Challenge: | Efficient finetuning of large language models (LLMs) aims to adapt the LLMs with reduced computational and memory costs. |
| Approach: | They propose a simple yet effective method that initializes low-rank matrices with Gaussian distribution and zero values while keeping the original weight matrics frozen. |
| Outcome: | The proposed approach only updates the minor components of the weight matrix while keeping the principal singular components frozen. |
Copied to clipboard
| Challenge: | High-quality instruction data is crucial for developing large language models (LLMs), yet existing approaches struggle to effectively control instruction complexity. |
| Approach: | They propose a framework that compresses instructions into a compact tag space and enhances complexity through RL-guided tag expansion. |
| Outcome: | The proposed framework outperforms existing methods in the evaluation of instruction complexity augmentation and semantic compression of text into a compact tag space. |
Copied to clipboard
| Challenge: | Large vision-language models (LVLMs) are gaining traction in clinical tasks such as diagnostic support, report generation, and medical question answering. |
| Approach: | They present a systematic evaluation of nine DPO variants applied to two leading medical LVLMs. |
| Outcome: | The proposed model improves alignment and reduces severe hallucinations, but yields inconsistent gains over supervised fine-tuning. |
Copied to clipboard
| Challenge: | Existing multilingual vision-language pretrained models are biased towards English due to the lack of sufficient non-English image-text pairs. |
| Approach: | They propose to train a retrieval-efficient dual-stream multilingual VLP model by aligning CLIP model and a multilingual text encoder through a novel Triangle Cross-modal Knowledge Distillation method. |
| Outcome: | Empirical results show that mCLIP achieves new state-of-the-art performance for both zero-shot and finetuned multilingual image-text retrieval tasks. |
Copied to clipboard
| Challenge: | Commercial large language models (LLMs) have made great progress in various NLP tasks. |
| Approach: | They propose a black-box jailbreak framework for automated red teaming of Large language models using an iterative optimization algorithm to conceal malicious content and memory reframing. |
| Outcome: | The proposed framework outperforms existing jailbreak defense methods and highlights the need to develop more effective and practical defense strategies. |
Copied to clipboard
| Challenge: | Existing preference learning methods rely heavily on curated data from humans or advanced LLMs, which is costly and difficult to scale. |
| Approach: | They propose a framework that leverages implicit preferences in unlabeled user-generated content to generate preference data. |
| Outcome: | The proposed framework transforms user-generated content into user queries and generates responses from the policy model. |
Copied to clipboard
| Challenge: | Existing approaches to enhance output diversity but compromise quality of outputs. |
| Approach: | They propose a training-free plug-and-play method that enhances output diversity while preserving generation quality. |
| Outcome: | The proposed method enhances output diversity while maintaining an optimal balance between diversity and quality. |
Copied to clipboard
| Challenge: | LoRA-Flow uses lightweight modules to customize large language models for downstream tasks . previous work on LoRA combination relied on task-level weights for each involved LoRA . |
| Approach: | They propose a LoRA-Flow approach that uses dynamic weights to adjust the impact of different LoRAs. |
| Outcome: | The proposed method outperforms baselines with task-level weights on six generative tasks. |
Copied to clipboard
| Challenge: | Existing approaches to improve large language models' ability to understand and reason are limited by external feedback. |
| Approach: | They propose a feedback-free reflection mechanism that requires only a single inference pass without external feedback. |
| Outcome: | The proposed method is based on an industrial e-commerce benchmark and public datasets. |
Copied to clipboard
| Challenge: | Recent advances in large language models have achieved promising performances across various applications, but the challenge of integrating long-tail knowledge continues to impede the seamless adoption of LLMs in specialized domains. |
| Approach: | They propose a dynamic co-augmentation framework for the refinement of large language models and knowledge graphs in the context of Alzheimer's Disease. |
| Outcome: | The proposed framework can be used to study Alzheimer's Disease (AD) using LLMs and KGs. |
Copied to clipboard
| Challenge: | Recent approaches to reduce resource requirements for task-specific large language models have been developed. |
| Approach: | They propose a delta compression approach that optimizes for importance of a model . they use SVD to dynamically adjust the sparsity ratios of different vectors based on their importance . |
| Outcome: | The proposed approach achieves state-of-the-art in retaining task-specific knowledge even at high sparsity ratios. |
Copied to clipboard
| Challenge: | Proximal Policy Optimization (PPO) is central to aligning Large Language Models with verifiable rewards. |
| Approach: | They propose a scalable algorithm that harmonizes sample efficiency with stability of outcome-based updates. |
| Outcome: | The proposed algorithm outperforms standard PPO and matches the performance of computation-heavy group-based methods. |
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated remarkable capabilities across a wide range of tasks, including instruction following, mathematical problem solving, and coding generation. |
| Approach: | They propose a method that truncates both preferred and dispreferred responses to match the shorter one’s length. |
| Outcome: | The proposed approach improves over standard implementations and achieves 11.8 points in AlpacaEval 2 and overall improvements across downstream tasks. |
Copied to clipboard
| Challenge: | Recent advances in prompt engineering have created impediments for end users to adopt . however, prompt engineering remains an impedance due to rapid advances in models, tasks, and associated best practices. |
| Approach: | They propose to define APO as a 5-part unifying framework and categorize all relevant works based on their salient features. |
| Outcome: | The proposed framework aims to improve the performance of large language models on various tasks. |
Copied to clipboard
| Challenge: | Full-state latent communication in LLMs suffers from memory overhead scaling linearly with collaboration rounds. |
| Approach: | They propose a lightweight module that uses learnable semantic probes to compress KV caches into fixed-size representations. |
| Outcome: | The proposed module reduces KV cache memory by over 99% and inference latency by approximately 20% on seven benchmarks spanning six models . it outperforms text-based methods by 1.7 percentage points on average across all configurations while outperforming existing methods by 1.7%. |
Copied to clipboard
| Challenge: | Existing approaches to correct factually inaccurate outputs are lacking the semantic richness needed to properly understand its internal states of trustworthiness and honesty. |
| Approach: | They propose a framework for factuality alignment that integrates natural-language uncertainty signals with external knowledge and computes confidence scores and semantic entropy from LLM outputs. |
| Outcome: | Extensive experiments on four knowledge-intensive benchmarks show that FAITH improves the factual accuracy and truthfulness of Large Language Models (LLMs). |
Copied to clipboard
| Challenge: | Existing knowledge distillation models are not optimized for dealing with pairs (or tuples) of texts. |
| Approach: | They propose a framework for distilling fast and accurate models on text pair tasks using a scalable end-to-end training strategy. |
| Outcome: | Empirical studies on academic and real-world e-commerce benchmarks show the proposed framework can achieve speedups of over 350x and minimal quality drop relative to the cross-attention teacher BERT model. |
Copied to clipboard
| Challenge: | Existing methods for generating large language models rely on student-generated outputs, which introduce generation errors and misguide the distillation process. |
| Approach: | They propose a multi-granularity semantic revision method for LLM distillation that corrects errors using teacher-generated tokens and re-generates the sequence to minimize errors. |
| Outcome: | The proposed method reduces errors and misguides distillation on student models and improves consistency between teacher and student outputs. |
Copied to clipboard
| Challenge: | Existing studies focus on English-centric aspects of sentiment analysis, limiting scope for multilingual evaluation and research. |
| Approach: | They propose to use a multilingual dataset to analyze aspects with associated sentiment elements in text. |
| Outcome: | The proposed dataset is the most extensive multilingual parallel dataset for ABSA to date. |
Copied to clipboard
| Challenge: | Existing methods to augment pre-trained language models with disease knowledge are lacking. |
| Approach: | They propose a method to augment BERT-like pre-trained language models with disease knowledge. |
| Outcome: | The proposed method improves on a suite of BERT models over three tasks. |
Copied to clipboard
| Challenge: | Existing methods for reinforcement learning (RL) are limited by poor data efficiency and weak generalization. |
| Approach: | They propose a novel architecture that integrates large language models into episodic RL. |
| Outcome: | The proposed architecture achieves 2–6 higher data efficiency than baselines and is the only method to solve complex tasks like UnlockLocal with over 90% success. |
Copied to clipboard
| Challenge: | Existing graph RAGs decouple retrieval and reasoning processes, preventing adaptability . existing graph Raggings depend heavily on ground-truth entities, which are often unavailable in open-domain settings. |
| Approach: | They propose a graph retriever that is trained end-to-end with large-scale graphs . structure and semantic features are encoded via soft tokens and the verbalized graph . |
| Outcome: | The proposed approach improves the performance of large-scale graph retrieval models by grounding it with external knowledge. |
Copied to clipboard
| Challenge: | Existing screen datasets focus on low-level structural and component understanding or on a much higher-level composite task such as navigation and task completion for autonomous agents. |
| Approach: | They propose to annotate 86k question-answer pairs over the RICO dataset to benchmark screen content understanding. |
| Outcome: | The proposed dataset covers full answers, short answer phrases, and corresponding UI contents with bounding boxes, enabling four subtasks to address various application scenarios. |
Copied to clipboard
| Challenge: | Existing systems require users to manually select models or employ rigid routing rules that fail to capture the continuous spectrum of query complexity. |
| Approach: | They propose a quality-constrained intelligent prompt routing framework that automatically selects optimal models based on predicted response quality and user-specified tolerance levels. |
| Outcome: | The proposed framework achieves 43.9% cost reduction while maintaining quality parity with strongest model in the Claude family and processes requests with sub-150ms latency. |
Copied to clipboard
| Challenge: | Prior work suggests that Transformer captures poor word alignments through its attention mechanism. |
| Approach: | They propose two new word alignment induction methods that use attention weights to capture accurate word alignments. |
| Outcome: | The proposed methods outperform baselines on three publicly available datasets and are significantly better than GIZA++. |
Copied to clipboard
| Challenge: | Multimodal large language models have advanced rapidly, yet most remain English-centric . scaling multilingual multimodal instruction tuning is limited by the scarcity and high cost of non-English image–text supervision. |
| Approach: | They propose a framework that decouples multilingual language enhancement from visual alignment by composing complementary task vectors over a shared LLM backbone. |
| Outcome: | The proposed framework achieves competitive performance with a fully multimodally trained model using less than 2% of the text data. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated impressive capabilities for text rewriting, however creating a smaller yet potent language model presents two formidable challenges: costly data collection and absence of emergent capabilities. |
| Approach: | They propose a new instruction tuning method to develop a mo-bile text rewriting model that leverages LLM-generated data and heuristic reinforcement learning, eliminating the need for human data collection. |
| Outcome: | The proposed model surpasses the current state-of-the-art LLMs in text rewriting while maintaining a significantly reduced model size using public benchmark EditEval and our new benchmark. |
Copied to clipboard
| Challenge: | Large pretrained language models (LMs) have been criticized for lack of grounding, i.e., connecting words to their meanings in the physical world. |
| Approach: | They compare vision-and-language (VL) models trained jointly on text and image or video data to find out how they compare to text-only counterparts. |
| Outcome: | The proposed model outperforms the text-only variants on a commonsense question answering task. |
Copied to clipboard
| Challenge: | Existing benchmarks for legal intelligence are limited to static evaluation paradigms or simplified scenarios. |
| Approach: | They introduce J1-ENVS, the first interactive and dynamic legal environment tailored for LLM-based agents. |
| Outcome: | The proposed framework assesses task performance and procedural compliance across legal proficiency levels. |
Copied to clipboard
| Challenge: | Experimental evaluations show that UnitY outperforms a single-pass speech-to-unit translation model by 2.5-4.2 ASR-BLEU with 2.83x decoding speed-up. |
| Approach: | They propose a two-pass direct S2ST architecture which generates textual representations and predicts discrete acoustic units . they show that UnitY outperforms a single-pass speech-to-unit translation model by 2.5-4.2 ASR-BLEU with 2.83x decoding speed-up. |
| Outcome: | The proposed architecture outperforms a single-pass speech-to-unit translation model by 2.5-4.2 ASR-BLEU with 2.83x decoding speed-up on large datasets. |
Copied to clipboard
| Challenge: | Existing direct speech-to-speech translation models rely on text generation as an intermediate step. |
| Approach: | They propose a direct speech-to-speech translation model that translates speech from one language to another without relying on intermediate text generation. |
| Outcome: | The proposed model produces 6.7 BLEUs in the Fisher Spanish-English dataset when trained without any text transcripts and with text supervision. |
Copied to clipboard
| Challenge: | Existing large language models (LLMs) are prone to misuse and misinformation, posing serious compliance risks. |
| Approach: | They propose a bilingual red-teaming benchmark to test an LLM’s refusal of requests that violate financial compliance. |
| Outcome: | The proposed benchmark is based on real-world financial crime cases and ethical violations and includes 14 subcategories covering financial crimes and ethical breaches. |
Copied to clipboard
| Challenge: | Existing retrieval-augmented generation paradigms rely heavily on public knowledge . Existing RAGs reliant on public information and often falter when faced with domain-specific queries. |
| Approach: | They propose a framework that combines a data-construction modeling approach with a scalable synthetic data-generation pipeline to optimize domain-specific retrieval performance. |
| Outcome: | The proposed framework optimizes domain-specific retrieval performance and bolsters retriever robustness. |
Copied to clipboard
| Challenge: | Existing robustness evaluations rely on hand-crafted templates or a limited set of perturbation rules, resulting in model failure. |
| Approach: | They propose a framework inspired by software stress testing that generates adversarial variants via a multi-round rewrite-verify loop, ensuring semantic consistency while successfully inducing model failure. |
| Outcome: | The proposed framework generates adversarial variants dynamically for each LLM, minimizing the risk of data contamination. |
Copied to clipboard
| Challenge: | Foundation models for single-cell RNA sequencing ignore biological prior knowledge encoded in gene regulatory relationships and fail to leverage multi-omics signals. |
| Approach: | They propose a framework that integrates multi-scale gene regulatory networks into RNA foundation model training. |
| Outcome: | The proposed framework improves on state-of-the-art models on three downstream tasks . it integrates multi-scale gene regulatory networks (GRNs) from multi-omics data into training . |
Copied to clipboard
| Challenge: | Multilingual pretrained language models (mPLMs) have shown their effectiveness in multilingual word alignment induction, but these methods usually start from mBERT or XLM-R. |
| Approach: | They propose to fine tune multilingual sentence Transformer LaBSE for alignment induction using parallel corpus and a parallel corpora model. |
| Outcome: | The proposed model outperforms existing models on seven language pairs and achieves new state-of-the-art on zero-shot language pairs. |
Copied to clipboard
| Challenge: | Sequence modeling is a simple yet versatile task that can be applied to more complex decision-making domains. |
| Approach: | They build a sequence modeling Transformer which takes a language instruction, actions, and environmental observations as inputs and then trains a model to reconstruct environmental layouts. |
| Outcome: | The proposed model can reconstruct environmental layouts from the inputs of the model and language instructions play a role in the reconstruction accuracy. |
Copied to clipboard
| Challenge: | Neural based end-to-end frameworks have achieved remarkable success in speech-totext tasks, such as automatic speech recognition (ASR) and speech- totext translation (ST). |
| Approach: | They propose to combine Transducer and Attention based Encoder-Decoder (TAED) for speech-to-text tasks and leverage AED's strength in non-monotonic sequence to sequence learning while retaining Transducers streaming property. |
| Outcome: | The proposed model outperforms Transducer and Attention based Encoder-Decoder (TAED) on the MuST-C dataset and shows that it is not bound by any specific language model. |
Copied to clipboard
| Challenge: | TrickCatcher generates test cases that pass existing tests yet contain bugs . a recent study found that tricky bugs are not detected by test suites . |
| Approach: | They propose an LLM-powered approach to generating test cases for uncovering bugs in plausible programs . they use a PUT and specification to generate program variants, an input generator and an Llm to construct test inputs . |
| Outcome: | The proposed approach achieves recall, precision, and F1 scores that are 1.80, 2.65, and 1.66 . trickCatcher generates program variants based on the program under test and its specification . |
Copied to clipboard
| Challenge: | Existing methods for sarcasm detection focus on fusing text and image information to establish cross-modal correlations, overlooking the significance of original unimodal incongruity information. |
| Approach: | They propose a multi-modal incongruity learning module to capture inconcluity information simultaneously at the text-level, image-level and cross-modal-level. |
| Outcome: | The proposed model outperforms state-of-the-art methods on a publicly available dataset. |
Copied to clipboard
| Challenge: | Existing methods for multimodal sentiment detection use the same fixed framework to classify the sentiment polarity of image-text pairs. |
| Approach: | They propose a multimodal dynamic interaction model that uses a fixed framework to classify the sentiment polarity of a given imagetext pair. |
| Outcome: | The proposed model outperforms state-of-the-art models on three publicly available datasets. |
Copied to clipboard
| Challenge: | Existing methods to unlearning large language models often memorize sensitive or harmful information, but they struggle with the forget-retain trade-off due to the polysemantic nature of LLMs parameters. |
| Approach: | They propose a representation-guided low-rank unlearning approach that leverages the geometric properties of representation spaces to achieve robust and precise unlearning. |
| Outcome: | The proposed approach outperforms state-of-the-art models on TOFU and WMDP benchmarks while maintaining higher model utility. |
Copied to clipboard
| Challenge: | Existing studies have focused on the potential misuse of large language models (LLMs) however, the ability to align LLMs with human values is still vulnerable to malicious attacks. |
| Approach: | They propose a red-teaming strategy to enhance LLM safety by using a framework to design jailbreak prompts automatically. |
| Outcome: | The proposed framework achieves attack success rates of 88% and 60% in cold-start scenarios. |
Copied to clipboard
| Challenge: | Existing evaluations of LLMs in finance are text-only, monolingual, and largely saturated by current models. |
| Approach: | They propose a multilingual and multimodal benchmark for evaluating LLMs in real financial contexts. |
| Outcome: | The first expert-annotated multilingual and multimodal benchmark is released . it evaluates 21 leading LLMs and shows they perform better in multilingual settings . |
Copied to clipboard
| Challenge: | Recent studies have employed large language models (LLMs) as reference-free metrics for NLG evaluation, enhancing adaptability to new tasks tasks. |
| Approach: | They propose a method that leverages large language models to integrate insights from various assistant evaluators. |
| Outcome: | The proposed approach achieves a 0.962 system-level Kendall-Tau correlation with humans on SummEval and a 0.7444 turn-level Spearman correlation on TopicalChat, which is significantly higher than baseline methods. |
Copied to clipboard
| Challenge: | Existing methods for decoding text using beam search are expensive and require reinforcement learning. |
| Approach: | They propose a method that allows us to reap the full benefits of beam search with no additional computational cost. |
| Outcome: | The proposed method outperforms greedy decoding and beam search on machine translation tasks with minimal computational cost. |
Copied to clipboard
| Challenge: | Existing data selection methods suffer from severe domain specificity . existing methods for general instruction-following fail on reasoning tasks . |
| Approach: | They propose a framework that operationalizes contrastive entropy as a domain-adaptive selection criterion through warmup calibration, bi-directional NLL filtering, and entropic-based ranking. |
| Outcome: | Experiments show that InstructDiff outperforms baseline training on reasoning tasks while using only 10% of the data. |
Copied to clipboard
| Challenge: | Recent pre-trained language models achieve state-of-the-art performance for downstream NLP tasks. |
| Approach: | They propose a parameter-free probing technique for analyzing pre-trained language models . their method does not require direct supervision from probing tasks . |
| Outcome: | The proposed method improves on linguistically-uninformed baselines on pre-trained language models. |
Copied to clipboard
| Challenge: | Existing unsupervised neural machine translation systems can degrade when labeled data is limited. |
| Approach: | They propose a multilingual pretraining and multilingual fine-tuning for facilitating cross-lingual transfer in zero-shot translation using a parallel dataset. |
| Outcome: | The proposed model outperforms state-of-the-art models on many-to-English translation by over 7.2 and 5.0 BLEU. |
Copied to clipboard
| Challenge: | Recent advances in large reasoning models have broadened the capabilities of medical artificial intelligence. |
| Approach: | They propose a reasoning framework for complex medical inference that reformulates medical reasoning as a parallelizable directed acyclic graph process based on Petri Net theory. |
| Outcome: | The proposed reasoning framework improves strong general-purpose LLMs by up to 8.9%. |
Copied to clipboard
| Challenge: | Recent studies explore approaches to synthesize instruction data with open-sourced LLMs but require high-quality human-crafted seed data. |
| Approach: | They propose an end-to-end framework to synthesize high-quality instruction data with open-sourced LLMs and sampled unlabeled documents, eliminating the need for seed data. |
| Outcome: | The proposed framework synthesizes high-quality instruction data with open-sourced LLMs and sampled unlabeled documents, eliminating the need for seed data. |
Copied to clipboard
| Challenge: | In-context instruction tuning is a method that fine tunes with supervised instruction data to enhance the instruction following ability of large language models. |
| Approach: | They propose an in-context instruction tuning method that incorporates positive and negative examples into the prompt for better performance. |
| Outcome: | The proposed method outperforms baseline instruction tuning methods on in-domain and out-domain tasks up to 9.16 and 3.14 average ROUGE-L scores. |
Copied to clipboard
| Challenge: | Existing definition generation tasks require a dictionary with complex definitions and a corpus containing arbitrary simple texts to generate them. |
| Approach: | They propose a multitasking framework SimpDefiner that only requires a standard dictionary with complex definitions and a corpus containing arbitrary simple texts. |
| Outcome: | The proposed framework outperforms the baseline model by a 1.77 SARI score on the English dataset, and raises the proportion of the low level (HSK level 1-3) words in Chinese definitions by 3.87%. |
Copied to clipboard
| Challenge: | Existing large language model evaluation benchmarks focus on English, while current multilingual tasks lack parallel questions that specifically assess cross-lingual reasoning abilities. |
| Approach: | They propose a comprehensive benchmark covering 29 languages, built on an English benchmark. |
| Outcome: | The MMLU-ProX is a comprehensive benchmark covering 29 languages, built on an English benchmark. |
Copied to clipboard
| Challenge: | Multi-agent LLMs are prone to adversarial attacks because of constraints such as limited token bandwidth and latency between message delivery. |
| Approach: | They propose a permutation-invariant adversarial attack that optimizes prompt distribution across latency and bandwidth constraints to bypass distributed safety mechanisms within the system. |
| Outcome: | The proposed method outperforms conventional attacks by up to 7 on multiple models. |
Copied to clipboard
| Challenge: | a cloud-based smart compose system is designed to improve human-to-human conversation efficiency. |
| Approach: | They propose a cloud-based smart compose system to improve conversation efficiency . they propose heuristics to achieve the best trade-off between quality and latency . |
| Outcome: | The proposed system reduces latency without losing composing quality further. |
Copied to clipboard
| Challenge: | Existing models for long sequences are not efficient due to the quadratic space and time complexity of the self-attention modules. |
| Approach: | They propose to reduce the quadratic complexity to linear (modulo logarithmic factors) by low-dimensional projection and row selection. |
| Outcome: | The proposed methods outperform transformer-based models with smaller time/space footprint on the Long Range Arena benchmark. |
Copied to clipboard
| Challenge: | Large language models (LLMs) are inherently dual-use and can be leveraged for both beneficial and harmful purposes. |
| Approach: | They propose a retention-prioritized gradient synthesis framework that decouples task-specific gradient extraction from conflict-aware combination. |
| Outcome: | The proposed method achieves tighter alignment on WMDP Bio and RWKU benchmarks. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have been successful in Text-to-SQL tasks, but their deployment in real-world environments is hindered by latent reliability issues. |
| Approach: | They propose a framework to autonomously uncover latent failure patterns in LLM-based Text-to-SQL generation. |
| Outcome: | The proposed framework uncovers a substantial number of failure cases on state-of-the-art open-source LLMs. |
Copied to clipboard
| Challenge: | Existing studies on unsupervised headline generation focus on a standard dataset and mono-style corpora. |
| Approach: | They propose an unsupervised approach for stylistic headline generation using a pretrained BART model decorated with adapters responsible for different styles. |
| Outcome: | The proposed method separates the task of style learning and headline generation, allowing for the generation of diverse headlines with diverse styles. |
Copied to clipboard
| Challenge: | SportQA is a benchmark specifically designed for evaluating Large Language Models (LLMs) sports knowledge is characterized by its fast pace, variety of types, abundance of strategies, and rich player narratives . |
| Approach: | They propose a benchmark specifically designed for evaluating Large Language Models in the context of sports understanding. |
| Outcome: | The proposed benchmark aims to bridge the gap between existing and specialized benchmarks in sports understanding. |
Copied to clipboard
| Challenge: | Recent advances in machine learning (ML) are attributed to large language models (LLMs), but their escalating memory requirements require developers to partition a large model to distribute it across multiple GPUs or TPUs. |
| Approach: | They propose a lightweight and user-friendly tool to automate distributed training and inference for LLMs and to simplify ML pipeline development. |
| Outcome: | The proposed tool automates distributed training and inference for LLMs, and simplifies ML pipeline development. |
Copied to clipboard
| Challenge: | Despite the success of neural machine translation, simultaneous neural machine translators are challenging due to syntactic structure difference and simultaneity requirements. |
| Approach: | They propose a framework for adapting neural machine translation to translate simultaneously . they propose 'prefix translation' that utilizes a consecutive NMT model to translate source prefixes . |
| Outcome: | The proposed framework balancing quality and latency on three translation corpora and two language pairs shows that it performs well. |
Copied to clipboard
| Challenge: | Large language models have shown promise in clinical decision making, but current approaches struggle to localize and correct reasoning errors at specific steps of the reasoning process. |
| Approach: | They propose a process reward modeling framework that leverages retrieval-augmented generation to verify each reasoning step against established medical knowledge bases. |
| Outcome: | The proposed model improves on five medical QA benchmarks and two open-ended diagnostic tasks by 13.50% on MedQA. |
Copied to clipboard
| Challenge: | Existing methods for contrastive pre-training ignore the relevance between codes in large code corpus. |
| Approach: | They propose a Soft-labeled contrastive pre-training framework with positive sample construction methods to learn functional-level code representation. |
| Outcome: | The proposed framework can obtain fine-grained soft-labels through an iterative adversarial manner and use them to learn better code representation. |
Copied to clipboard
| Challenge: | Existing methods to train speech models without labeled data are limited for most languages. |
| Approach: | They propose a pipeline approach to build speech translation systems without labeled data by leveraging recent advances in unsupervised speech recognition, machine translation and speech synthesis. |
| Outcome: | The proposed approach outperforms the state-of-the-art in unsupervised speech recognition by 3.2 BLEU on the Libri-Trans benchmark and the best supervised end-to-end models from only two years ago by an average of 5.0 BLUE over five X-En directions. |
Copied to clipboard
| Challenge: | Recent studies have introduced legal theories into LLM workflows to improve their understanding of legal texts and reasoning accuracy. |
| Approach: | They evaluate an expert-annotated four-element knowledge base covering 155 criminal charges. |
| Outcome: | The proposed model can be used to analyze criminal charges and retrieve them in legal cases. |
Copied to clipboard
| Challenge: | Large language models (LLMs) are pretrained on multilingual corpora but exhibit suboptimal performance on low-resource languages. |
| Approach: | They propose a framework that integrates representations from all encoder layers and an adaptive fusion-enhanced attention mechanism to enable layer-wise interaction between the LLM and the multilingual encoder. |
| Outcome: | Experiments on multilingual reasoning tasks show that the proposed framework outperforms baselines. |
Copied to clipboard
| Challenge: | Existing pre-training language models have been successful in natural language understanding and autoregressive generation tasks, but non-autoregressive models have not been sufficiently successful. |
| Approach: | They propose a pre-trained masked language model (MLM) and a non-autoregressive generation model with a lightweight decorator. |
| Outcome: | The proposed model outperforms the previous mask-predict model on translation datasets by 19.9x. |
Copied to clipboard
| Challenge: | Large language models have demonstrated outstanding performance in various natural language processing tasks, but their security capabilities in the financial domain have not been explored. |
| Approach: | They propose to use a benchmark to evaluate large language models' financial domain knowledge and practical abilities. |
| Outcome: | The proposed benchmark evaluates large language models' financial domain knowledge and practical abilities. |
Copied to clipboard
| Challenge: | Existing methods for unsupervised text style transfer struggle to achieve high style conversion rate and low content loss. |
| Approach: | They propose a collaborative learning framework for unsupervised text style transfer using a pair of bidirectional decoders. |
| Outcome: | The proposed framework achieves strong empirical results on style compatibility and content preservation. |
Copied to clipboard
| Challenge: | Existing single-cell LLMs struggle to integrate spatial information into natural language, limiting their ability to capture biological relationships. |
| Approach: | They propose a framework that integrates both single-cell expression and spatial information into natural language using a multi-sentence approach. |
| Outcome: | The proposed framework outperforms existing single-cell LLMs on preprocessed IMC datasets for diabetes and brain tumors while improving interpretability. |
Copied to clipboard
| Challenge: | Existing prompt-based methods craft meticulous text guidelines and examples to facilitate SQL generation, but their accuracy is hindered by the large semantic gap between the texts and the low-resource SQL programs. |
| Approach: | They propose to use Python as a pivot to bridge between natural language query and SQL program. |
| Outcome: | The proposed method improves the execution accuracy of the best-performing baseline by up to 3.20. |
Copied to clipboard
| Challenge: | Existing MLLMs are strong at understanding single plots, but struggle with multi-step reasoning . Existing approaches to manage context in chart reasoning include text-based chain-of-thought prompting . |
| Approach: | They propose a hierarchical visual agent framework that iteratively constructs a working context in an image–text space. |
| Outcome: | The proposed framework improves on strong multimodal baselines. |