Papers by Jin Xu
Copied to clipboard
| Challenge: | Existing fine-tuning algorithms for vision-language models are restricted by patient privacy concerns and can contain imperceptible noise. |
| Approach: | They propose a framework to mitigate adversarial noise and mitigate upstream noise during fine-tuning. |
| Outcome: | The proposed framework improves model robustness and transferability while decreasing noise levels negatively impact downstream performance. |
Copied to clipboard
| Challenge: | Large language models (LLMs) are widely used in commercial applications . low latency is crucial due to system latency, query concurrency, and computational resources constraints. |
| Approach: | They propose a system that can be resource-efficiently served by addressing bottlenecks beyond LLM inference . they propose 4.3 speed up over vLLM and 1.5 higher throughput . |
| Outcome: | The proposed system outperforms state-of-the-arts with 1.5 higher throughput . it achieves 4.3 speed up with 64 concurrent requests on Mixtral 8x7B . |
Copied to clipboard
| Challenge: | Large language models exhibit significant performance discrepancies between high- and low-resource languages. |
| Approach: | They present an open-source multilingual LLM with 8 billion parameters and a multilingual instruction dataset. |
| Outcome: | The proposed model achieves consistent multilingual representations across languages. |
Copied to clipboard
| Challenge: | Existing top-k attention methods struggle to strike a balance between efficiency and accuracy. |
| Approach: | They propose a top-k attention approach that integrates low-overhead techniques into the Top-k Attention process to achieve 7.2 speedup compared to vanilla full attention. |
| Outcome: | The proposed approach achieves 7.2 speedup compared to current top-k attention methods while maintaining model accuracy. |
Copied to clipboard
| Challenge: | Existing models for review representations of unseen or anonymous users are limited by their in-domain nature. |
| Approach: | They propose to use in-domain user and product information to generalize reviews . they use switch knowledge distillation to learn review representations for unseen users . |
| Outcome: | The proposed model performs well for existing or anonymous unseen users. |
Copied to clipboard
| Challenge: | Existing chart understanding benchmarks are overwhelmingly English-centric, limiting their accessibility and relevance to global audiences. |
| Approach: | They propose a multilingual chart question answering benchmark that enables efficient multilingual generation via data translation and code reuse. |
| Outcome: | The proposed benchmark systematically evaluates multilingual chart understanding on state-of-the-art LVLMs and shows a significant performance gap between English and other languages. |
Copied to clipboard
| Challenge: | Existing methods for handwriting generation capture global dependencies and can generate high-quality handwritten samples. |
| Approach: | They propose a Transformer-based model for ink generation, TrInk, which captures global dependencies. |
| Outcome: | The proposed model reduces character error rate and word error rate by 35.56% on the IAM-OnDB dataset compared to previous models. |
Copied to clipboard
| Challenge: | Existing RAG frameworks rely on Automatic Speech Recognition to process speech input, which discards crucial audio information and increases computational overhead. |
| Approach: | They propose a retrieval augmented generation framework with native, end-to-end audio support that integrates audio and text into a unified knowledge representation. |
| Outcome: | The proposed framework can perform 10x faster than current pipelines while delivering 10x acceleration. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable performance across a diverse set of domain-specific tasks. |
| Approach: | They propose a non-monolithic LLM querying system that seamlessly integrates various LLM experts into a single query interface and dynamically routes incoming queries to the most high-performant expert based on query’s requirements. |
| Outcome: | The proposed model improves query efficiency by 40% and costs by 30% while maintaining or enhancing model performance by 10%. |
Copied to clipboard
| Challenge: | Existing knowledge conflicts in RALMs can ensnare them in a tug-of-war between knowledge and evidence, limiting their practical applicability. |
| Approach: | They propose a method called Conflict-Disentangle Contrastive Decoding (CD2) to better calibrate the model’s confidence. |
| Outcome: | The proposed method can resolve knowledge conflicts in large language models with the help of conflict-disentangle contrast decoding (CD2) . |
Copied to clipboard
| Challenge: | Long prompts contain redundant information and are sensitive to the position of key information in long context scenarios. |
| Approach: | They propose a training-free prompt compression framework that retains key information at token level while removing distracting tokens. |
| Outcome: | The proposed framework outperforms existing methods on long context benchmarks. |
Copied to clipboard
| Challenge: | Pre-trained language models have been proposed and applied to many NLP tasks, yielding state-of-the-art performance, but high storage and computational costs obstruct them to be effectively deployed on resource-constrained devices and real-time applications. |
| Approach: | They propose a BERT distillation method which allows each intermediate student layer to learn from any intermediate teacher layers. |
| Outcome: | The proposed method can learn from different teacher layers adaptively for different NLP tasks. |
Copied to clipboard
| Challenge: | Existing studies have only considered language models as knowledge bases in a static setting . memorizing conflicting information is still challenging for LMs and hinders memorization of other unrelated one-to-one relationships. |
| Approach: | They propose two requirements for treating language models as temporal knowledge bases . they propose a dataset which is aimed at probing temporally-scoped knowledge . |
| Outcome: | The proposed model can store conflicting information and use stored knowledge for temporal knowledge queries. |
Copied to clipboard
| Challenge: | Existing methods to expand internal memory boundaries of language models by providing external context can often conflict, leading to knowledge conflicts. |
| Approach: | They propose a method that prunes conflicting attention heads without updating model parameters. |
| Outcome: | The proposed method can flexibly control eight LMs to use internal memory or external context without updating model parameters. |
Copied to clipboard
| Challenge: | Recent advances have showcased the potential of Large Language Models (LLMs) in executing reasoning tasks, particularly facilitated by Chain-of-Thought (CoT) prompting. |
| Approach: | They propose to use Large Language Models to perform tasks with subjectivity and personalized preferences as inputs to RecSys. |
| Outcome: | The proposed framework aligns with real human judgment on the coherence and faithfulness of LLM reasoning responses. |
Copied to clipboard
| Challenge: | Existing stance detection methods treat the task as a classification problem, where models output a stance label without providing interpretable reasoning paths. |
| Approach: | They propose a framework that generates, evaluates, and integrates multiple reasoning paths to improve accuracy, robustness, and transparency in stance detection. |
| Outcome: | The proposed framework outperforms existing models on the SEM16, VAST, and PStance datasets and is highly interpretable and reliable. |
Copied to clipboard
| Challenge: | Existing data augmentation paradigms isolate data synthesis from label validation, thereby reducing their utility for complex reasoning tasks. |
| Approach: | They propose a framework for enhancing reasoning-focused data augmentation in few-shot learning scenarios that integrates four agents through two synergistic phases: diverse data generation and label verification. |
| Outcome: | The proposed framework achieves the highest average improvement in task accuracy in both fine-tuning and in-context learning paradigms. |
Copied to clipboard
| Challenge: | Continual learning models adapt well to the latest data but lose ability to remember past data due to changes in the data source. |
| Approach: | They propose a hierarchical replay framework that allows models to keep a small memory of previous learned data that uses replay. |
| Outcome: | The proposed model outperforms previous continual learning methods in mitigating catastrophic forgetting. |
Copied to clipboard
| Challenge: | Existing methods for stance detection focus on background information and not on the accompanying input texts. |
| Approach: | They propose to prompt Large Language Models to explicitly extract the relationship between paired text and unseen target as contextual knowledge and inject it into a generation model BART to exploit the rich contexts and semantics. |
| Outcome: | The proposed model is able to detect stance labels in zero-shot and cross-target scenarios. |
Copied to clipboard
| Challenge: | Document images are characterized by higher resolutions, denser content, and more complex structural layouts. |
| Approach: | They propose a 1.2B-parameter document parsing vision-language model that decouples layout analysis from local content recognition. |
| Outcome: | The proposed model surpasses general-purpose and domain-specific models on multiple benchmarks while maintaining significantly lower computational overhead. |
Copied to clipboard
| Challenge: | FinSight is the first multi-agent framework for automating end-to-end professional, multimodal financial reports. |
| Approach: | They propose a code agent with variable memory architecture that unifies data, tools, and agents into a programmable variable space. |
| Outcome: | The proposed framework outperforms leading deep research systems in factual accuracy, analytical depth, and presentation quality. |
Copied to clipboard
| Challenge: | Existing benchmarks for MLM agents in interactive environments are limited by their focus on a single environment, lack of detailed and generalized evaluation methods, and the complexity of constructing tasks and evaluators. |
| Approach: | They propose a cross-environment agent benchmark framework that integrates graph-based evaluation and task generation methods. |
| Outcome: | The proposed framework supports multiple devices and can be easily extended to any environment with a Python interface. |
Copied to clipboard
| Challenge: | Existing methods for argument quality assessment do not consider multi-perspective evaluation due to subjective nature of arguments. |
| Approach: | They propose a multi-persona framework for argument quality assessment that simulates diverse evaluator perspectives through large language models. |
| Outcome: | The proposed framework outperforms baselines while providing comprehensive multi-perspective rationales on IBM-Rank-30k and IBM-ArgQ-5.3kArgs datasets. |
Copied to clipboard
| Challenge: | Large language models implicitly fabricate information when inputs are incomplete, causing confidence but unreliable conclusions. |
| Approach: | They propose a framework for grounded reasoning under incomplete information that decomposes reasoning into two stages . they propose stage-specific rewards to penalize hallucinations, enabling models to detect gaps, stop proactively, and resume reasoning after clarification. |
| Outcome: | The proposed framework improves premise detection and task success by 30% . it also reduces average response length by over 20% . |
Copied to clipboard
| Challenge: | Experimental protocols in organic synthesis specify not only the intended transformation, but also an executable sequence of operations and conditions. |
| Approach: | They propose a human-validated benchmark for verifiable experimental procedure reasoning . they instantiate 7306 benchmark tasks across six complementary formats . |
| Outcome: | The proposed benchmarks show that the evaluations are less diagnostic of procedure-level decision making. |
Copied to clipboard
| Challenge: | Existing studies have focused on specialized BERT-variants and recent LLMs to reason inconsistencies. |
| Approach: | They propose to incorporate task-specific taxonomy into inferences to facilitate both zero-shot and supervised paradigms. |
| Outcome: | The proposed model outperforms specialized non-LLM and recent LLM models in a number of domains. |
Copied to clipboard
| Challenge: | Existing models for fine-grained speaking styles are limited in terms of accuracy, coverage, and naturalness. |
| Approach: | They propose a model that pre-trains with coarse captions and annotates with a pipeline that grounds captions in audio. |
| Outcome: | The proposed model outperforms existing models with fine-grained style annotations . it integrates global and fine-granular supervision, enabling unified representations based on the proposed model . |
Copied to clipboard
| Challenge: | Existing benchmarks fail to adequately evaluate the proficiency of Large Language Models (LLMs) Existing standards do not cover the skills needed to evaluate LLMs in scientific literature analysis. |
| Approach: | They propose a benchmark to evaluate the proficiency of large language models in scientific literature analysis. |
| Outcome: | SciAssess evaluates 11 LLMs on multiple tasks across scientific fields. |
Copied to clipboard
| Challenge: | Existing Multimodal Large Language Models lack general structure understanding abilities for text-rich document images. |
| Approach: | They propose to use unified structure learning to boost the performance of MLLMs by encoding structure information into text-rich images. |
| Outcome: | The proposed model achieves state-of-the-art on 10 visual document understanding benchmarks. |
Copied to clipboard
| Challenge: | Multimodal Large Language Models (MLLMs) have improved document understanding performance but generate thousands of visual tokens for a single document image, leading to excessive GPU memory and slower inference times. |
| Approach: | They propose a high-resolution document compression module to generate 324 tokens for a single document image. |
| Outcome: | The proposed module reduces first token latency by more than 50% and improves document comprehension performance. |
Copied to clipboard
| Challenge: | Existing mechanisms compromise ownership rights or raise data privacy concerns . existing mechanisms compromise security of released large language models . |
| Approach: | They propose a TaylorMLP to preserve the ownership of large language models by transforming the weights of LLMs into Taylor-series parameters instead of releasing original weights . |
| Outcome: | The proposed model preserves ownership of large language models and prevents their abuse by adjusting the generation speed and causing low-speed token generation. |
Copied to clipboard
| Challenge: | Existing benchmarks for insurance claims adjudication are limited to information retrieval or simple multiple-choice setups. |
| Approach: | They propose a benchmark that provides complete reasoning traces linking factual inputs, relevant policy clauses, and final verdicts. |
| Outcome: | The proposed benchmark shows that models often produce correct decisions but fail to provide precise justifications, highlighting a critical discrepancy between decision accuracy and logical reasoning capabilities. |
Copied to clipboard
| Challenge: | Large language models behave consistently with human goals, values and intentions, but are computationally expensive. |
| Approach: | They propose a framework that enables weak-to-strong alignment transfer via concept transplantation. |
| Outcome: | The proposed framework surpasses instruction-tuned models in terms of truthfulness. |
Copied to clipboard
| Challenge: | Argument Mining (AM) is hindered by the scarcity of structure-annotated datasets, which are expensive to create manually. |
| Approach: | They propose to use quality-oriented synthesis and diversity-oriented approach to generate argumentative texts with diverse topics and argument structures. |
| Outcome: | The proposed approach significantly improves existing models in full-data and low-resource settings. |
Copied to clipboard
| Challenge: | Knowledge distillation (KD) compresses large language models into lightweight versions called student models. |
| Approach: | They propose to align the entire feature dynamics between teacher and student models by using two additional loss terms to achieve this. |
| Outcome: | The proposed method matches the entire feature dynamics between teacher and student models rather than just the final states. |
Copied to clipboard
| Challenge: | a new study presents scaling with gradient grouping (SGG) the adaptive learning rate scaling approach is based on per-parameter statistics, which incurs memory overhead. |
| Approach: | They propose an optimizer wrapper that improves adaptive learning rate estimation by dynamic grouping and group-specific scaling. |
| Outcome: | The proposed algorithm improves learning rate estimation on diverse models with different model sizes and batch sizes. |
Copied to clipboard
| Challenge: | MCoT requires models to leverage knowledge from both textual and visual modalities for step-by-step reasoning. |
| Approach: | They propose a benchmark to address the challenges of MCoT, and evaluate it using vision large language models. |
| Outcome: | The proposed benchmark addresses the above challenges and shows that current models still struggle to reason in M3CoT. |
Copied to clipboard
| Challenge: | Existing methods for evaluating concepts from different perspectives lack a unified formalization. |
| Approach: | They propose a formal definition of concepts generalizing to diverse concept-based explanations’ settings and apply it to other types of explanations or tasks. |
| Outcome: | Extensive experimental analysis was carried out to determine the evaluation measures for explanation evaluation measures. |
Copied to clipboard
| Challenge: | Existing approaches restrict students to following a single golden rationale and treat different reasoning paths independently, causing suboptimal performance. |
| Approach: | They propose a capability-adaptive framework that transitions distillation from passive mimicry to active cognitive construction and employ a feedback-driven inertia calibration mechanism to align supervision with the student’s current adaptability. |
| Outcome: | Experiments show that the proposed framework achieves state-of-the-art performance on both in-distribution and out-of distribution benchmarks. |
Copied to clipboard
| Challenge: | Persuasive dialogue models rely on utterance semantic matching and a key aspect has been ignored . compared with utterrance semantics, conversation strategies are high-level concepts, which can be informative and provide complementary information to achieve effective persuation. |
| Approach: | They propose to model conversation semantics and strategies to match them using a BERT-like module and an auto-regressive predictor. |
| Outcome: | The proposed model improves state-of-the-art by 5% on a small and 37% on 'large' datasets. |
Copied to clipboard
| Challenge: | Existing reasoning methods excel in structured domains like math and code, but they are not all effective in knowledge-intensive tasks. |
| Approach: | They introduce a framework that enhances large language model reasoning by integrating external tool-using agents. |
| Outcome: | The proposed framework achieves state-of-the-art among public models and delivers comparable performance to OpenAI Deep Research. |
Copied to clipboard
| Challenge: | Recent studies show that pre-trained masked language models can be factual knowledge bases. |
| Approach: | They conduct a rigorous study to explore the underlying predicting mechanisms of MLMs . they find that previous decent performance mainly owes to the biased prompts which overfit dataset artifacts a . |
| Outcome: | The proposed model improves on illustrative cases and external contexts . the results question the previous findings that MLMs can be reliable factual knowledge bases . |
Copied to clipboard
| Challenge: | Existing methods for Multimodal Stance Detection struggle with generalizing to unseen targets and handling modality inconsistencies. |
| Approach: | They propose a multimodal stability detection model which captures target-specific relationships and balances modality contributions by iterative reasoning. |
| Outcome: | Experiments on the MMSD and MultiClimate datasets show that the proposed model outperforms state-of-the-art models with optimal results achieved using RoBERTa, ViT, and an iterative depth of 5. |
Copied to clipboard
| Challenge: | Existing approaches to generate research ideas rely on retrieval or prompt engineering to generate ideas. |
| Approach: | They propose a method that uses iterative planning and search to boost creative potential of LLMs by integrating external knowledge with broader and deeper insights. |
| Outcome: | The proposed method outperforms the current state-of-the-art in generating 2.5 times more top-rated ideas based on 170 seed papers in a Swiss Tournament evaluation. |
Copied to clipboard
| Challenge: | Large language models (LLMs) often exhibit poor performance on knowledge-intensive tasks, such as commonsense reasoning. |
| Approach: | They propose a method to elicit, filter and integrate knowledge in large language models (LINKED) they propose 'reward model' to filter out noisy knowledge and 'take marginal consistent reasoning module' |
| Outcome: | The proposed method outperforms SOTA baselines on two commonsense reasoning tasks. |
Copied to clipboard
| Challenge: | Current evaluation frameworks are static and vulnerable to benchmark data contamination . current models are ineffective at assessing reasoning under temporal uncertainty . |
| Approach: | They propose a live-based benchmark that simulates the real-world "fog of war" they propose evaluating models on their ability to reason with evolving, incomplete information . |
| Outcome: | The proposed model outperforms proprietary state-of-the-art models in classification and evidence mode . it also provides a component to monitor BDC explicitly . |
Copied to clipboard
| Challenge: | Gradient-based data influence approximation is not feasible in practice. |
| Approach: | They propose a gradient-based data selection framework with clustering and a modified Upper Confidence Bound algorithm to solve this problem. |
| Outcome: | The proposed framework can achieve comparable results to the original gradient-based data selection methods while reducing computational consumption. |
Copied to clipboard
| Challenge: | GraphRAG framework is designed to enhance LLMs in generating evidence-based medical responses. |
| Approach: | They propose a graph-based Retrieval-augmented generation framework to enhance LLMs in generating evidence-based medical responses. |
| Outcome: | The proposed framework outperforms state-of-the-art models on 9 medical Q&A benchmarks, 2 health fact-checking datasets, and a long-form generation test set. |
Copied to clipboard
| Challenge: | Error correction is widely used in automatic speech recognition (ASR) to post-process the generated sentence. |
| Approach: | They propose a fast correction model that takes multiple ASR candidates as input for better correction accuracy. |
| Outcome: | The proposed model can reduce the word error rate (WER) with multiple candidates by 3.2% and 2.6%. |
Copied to clipboard
| Challenge: | Existing studies for visually-situated language understanding have shown shallow zero-shot visual text recognition ability when fed a low-resolution image with salient text information. |
| Approach: | They propose a model for universal OCR-free visually-situated language understanding based on the Multimodal Large Language Model (MLLM) their model is jointly finetuned on a wide range of visually situated language understanding tasks via a unified instruction format. |
| Outcome: | The proposed model achieves state-of-the-art ocr-free performance in 8 out of 10 visually-situated language understanding tasks across 5 domains: documents, tables, charts, natural images, and webpage screenshots. |
Copied to clipboard
| Challenge: | Existing methods for multimodal summarization ignore the structure and semantics of the whole video and article. |
| Approach: | They propose a semantic-consistent cross-domain summarization model that extracts features from video and article and uses fusion methods to select representative one. |
| Outcome: | The proposed model produces high-quality multimodal summaries on three MSMO datasets. |
Copied to clipboard
| Challenge: | Large language models struggle with complex reasoning tasks, such as mathematical problem-solving. |
| Approach: | They constructed a symbolic multi-step reasoning task to investigate the information propagation mechanisms in Transformer models when solving the task through direct answering and Chain-of-Thought (CoT) reasoning. |
| Outcome: | The proposed algorithm improves on 7 multi-step reasoning datasets, while introducing only 132 trainable parameters. |
Copied to clipboard
| Challenge: | Long context capability is a crucial competency for large language models as it mitigates the human struggle to digest long-form texts. |
| Approach: | They propose to evaluate 10+ state-of-the-art approaches for long context-capable LLMs. |
| Outcome: | The proposed methods are compared against 10+ state-of-the-art approaches across seven categories of long context tasks. |
Copied to clipboard
| Challenge: | Existing methods for decoding target language are degenerate, hallucinating or empty. |
| Approach: | They propose a method that tunes down the Softmax temperature to reduce autoregressive over-smoothness by label smoothing the output distributions. |
| Outcome: | The proposed method improves MBR in various settings. |
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) have demonstrated significant strides in generating high-quality speech . discretizing speech by neural audio codecs often results in sequences that differ from text sequences . |
| Approach: | They quantitatively analyze the Discrete Representation Inconsistency phenomenon within popular audio tokenizers such as EnCodec. |
| Outcome: | The proposed method mitigates the DRI phenomenon within popular audio tokenizers such as EnCodec. |
Copied to clipboard
| Challenge: | Existing methods for forcing alignment are language-specific and prone to temporal shifts. |
| Approach: | They propose a slot-filling paradigm that uses time indices to predict slot positions. |
| Outcome: | The proposed method reduces accumulated temporal shifts by 69% compared with prior methods. |
Copied to clipboard
| Challenge: | Existing multimodal summarization methods are limited to monolingual videos . a proposed task aims to generate cross-lingual summaries from multimodal inputs . |
| Approach: | They propose a task to generate cross-lingual summaries from multimodal inputs of videos . they propose fusion network that integrates multimodal and cross-linguistic information . |
| Outcome: | The proposed task outperforms existing methods on a reorganized How2 dataset on the reorganized How2 data set. |
Copied to clipboard
| Challenge: | Existing approaches to RLVR use multiple-choice questions as verifiable rewards . however, not all tasks provide reliable verification . |
| Approach: | They propose a framework that actively constructs high-quality distractors to block elimination shortcuts and promote deep reasoning. |
| Outcome: | The proposed method significantly improves reasoning capabilities of Large Language Models. |
Copied to clipboard
| Challenge: | Recent studies have shown that multimodal large language models can be useful for chart understanding, but their size limits their use in resource-constrained environments. |
| Approach: | They propose an efficient multimodal large language model with only 3B parameters for chart understanding. |
| Outcome: | The proposed model outperforms several chart-understanding MLLMs with up to 13B parameters on ChartQA, Chart-to-Text, Chart to Table, OpenCQA, and ChartX. |
Copied to clipboard
| Challenge: | Unlike existing MoE approaches that rely on fixed TopK Routing, our dynamic expert selection framework dynamically allocates experts based on the confidence level in expert selection for each input. |
| Approach: | They propose a dynamic expert selection framework that dynamically allocates experts based on the confidence level in expert selection for each input. |
| Outcome: | The proposed method achieves an average improvement of 0.7% with less than 90% activated parameters and outperforms dense models in QA and machine translation tasks. |
Copied to clipboard
| Challenge: | Existing ABSA research relies on coarse-grained categorical labels, which limits its ability to capture nuanced affective states. |
| Approach: | They propose a dimensional approach that represents sentiment with continuous valence–arousal (VA) scores, enabling fine-grained analysis at both the aspect and sentiment levels. |
| Outcome: | The proposed approach represents sentiment with continuous valence–arousal (VA) scores, enabling fine-grained analysis at both the aspect and sentiment levels. |
Copied to clipboard
| Challenge: | Chain-of-Thought (CoT) prompting relies on the initial decisions, causing errors in early steps to accumulate and impact the final answers. |
| Approach: | They propose a divide-and-conquer style algorithm that leverages large language models to raise and answer sub-questions until collecting enough information to tackle the original one. |
| Outcome: | The proposed algorithm is more robust to errors and errors than CoT prompting and Tree-of-Thought prompting methods. |
Copied to clipboard
| Challenge: | Lossless compression has made significant advancements in Genomics Data storage, sharing and management. |
| Approach: | They propose a novel agent-based GD Compressor with 3 layers with a multi-agent named Leader and Worker. |
| Outcome: | The proposed method improves on existing methods with low-level modeling and limited adaptability and user-unfriendly interface. |
Copied to clipboard
| Challenge: | Several noise-robust losses have been proposed and evaluated on tasks in computer vision, but they use a single dataset-wise hyperparamter to control the strength of noise resistance. |
| Approach: | They propose to change single dataset-wise hyperparameters of noise resistance to be instance-wise. |
| Outcome: | The proposed frameworks increase noise-robustness on noisy and corrupted NLP datasets. |
Copied to clipboard
| Challenge: | Existing methods for resolving repository-level debugging are limited by their interdependencies. |
| Approach: | They propose a RelationGraph-based approach that integrates large language models with structural search and synchronization techniques for coordinated program repair across codebases. |
| Outcome: | SynFix resolves 52.33% of issues in SWE-bench-lite, 55.8% in Swe-bech-verified and 29.86% in S WE-beach-full. |
Copied to clipboard
| Challenge: | Existing methods operate by learning to fuse modalities, leading to frequent misjudgments. |
| Approach: | They propose a paradigm shift from *learning to fuse* to *learning the reason's process' inspired by the dual-process theory of human cognition, MIND operationalizes a self-improving loop. |
| Outcome: | The proposed model significantly outperforms baseline models and exhibits strong generalization. |
Copied to clipboard
| Challenge: | Existing methods to extract event records from text decompose complex structure prediction task into multiple subtasks. |
| Approach: | They propose a sequence-to-structure generation paradigm that can extract events from text . they propose unified event extraction, constrained decoding algorithm and curriculum learning algorithm . |
| Outcome: | The proposed method can achieve competitive performance using record-level annotations in both supervised learning and transfer learning settings. |
Copied to clipboard
| Challenge: | Existing LLMs do not possess consistent values, but many have been developed to align them at the behavioral level, including supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF). |
| Approach: | They propose a Controlled Value Vector Activation method that directly aligns the internal values of Large Language Models by interpreting how a value is encoded in their latent representations. |
| Outcome: | The proposed method achieves highest success rate across 10 basic values without hurting model performance and fluency, and ensures target values even with opposite and potentially malicious input prompts. |
Copied to clipboard
| Challenge: | Vision-Language Models (VLMs) lack visual-aware tutorial retrieval and historical visual context curation and pruning. |
| Approach: | They propose a framework that integrates an orchestrator and a Reflection-Memory Agent for robust automation. |
| Outcome: | Experimental results show that OS-Symphony delivers substantial performance gains across model scales. |
Copied to clipboard
| Challenge: | LLM-based agents are powerful tools for automating complex scientific workflows, especially in chemistry, but their single-task performance is limited by tool constraints. |
| Approach: | They propose a framework that optimizes the collective capabilities of specialized tools by dynamic coordination within individual tasks. |
| Outcome: | The proposed framework outperforms chemistry-specialized models, generalist LLMs, and agent systems with tool orchestration. |
Copied to clipboard
| Challenge: | Evaluations in machine learning rarely use the latest metrics, datasets, or human evaluation in favor of remaining compatible with prior work. |
| Approach: | They propose to use the Generation, Evaluation, and Metrics Benchmark to integrate new evaluation methods into existing evaluations. |
| Outcome: | The proposed evaluation infrastructure bridges the gap between the advantages of leaderboards and in-depth and evolving evaluations by allowing model developers to benefit from each other's work. |
Copied to clipboard
| Challenge: | Experimental results show that UI-Copilot-7B achieves state-of-the-art performance on challenging MemGUI-Bench, outperforming strong 7B-scale GUI agents such as GUI-Owl-7B and UITARS-1.5-7B. |
| Approach: | They propose a collaborative framework where the GUI agent focuses on task execution while a lightweight copilot provides on-demand assistance for memory retrieval and numerical computation. |
| Outcome: | The proposed framework outperforms GUI-Owl-7B and UI-TARS-1.5-7B on MemGUI-Bench and delivers 17.1% improvement on AndroidWorld over the base Qwen model. |
Copied to clipboard
| Challenge: | Existing approaches to augment Large Language Models (LLMs) with computational capabilities have focused on short Chain-of-thought (CoT) integrating tool-use into long CoT remains underexplored due to the scarcity of training data and the challenge of integrating it without compromising the model’s intrinsic long-chain reasoning. |
| Approach: | They propose a framework that enables spontaneous tool-use during long CoT reasoning without additional human annotation. |
| Outcome: | Experiments on AIME and GPQA-Diamond show that DART significantly outperforms existing methods, successfully harmonizing tool execution with long CoT reasoning. |
Copied to clipboard
| Challenge: | Event Extraction (EE) is a long-standing target, but lacks an efficient and effective annotation framework to construct the corresponding datasets. |
| Approach: | They propose an LLM-based collaborative annotation framework that refines annotations of triggers from distant supervision and carries out argument annotation. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on the largest EE dataset to date . it achieves the F1 scores of 90% and 85.3% on the human-annotated test set . |
Copied to clipboard
| Challenge: | Existing MVQA models ignore multi-level progressive capabilities due to unspecific data and plain architecture. |
| Approach: | They propose a multi-level visual language model for medical visual question answering (MVQA) which covers multi- level questions and answers as well as reasoning processes from visual clues to semantic cognition. |
| Outcome: | The proposed model outperforms existing medical multimodal large language models on a multi-level instruction dataset and a feature alignment module. |
Copied to clipboard
| Challenge: | Large language models exhibit limitations when handling complex mathematical reasoning and logical inference tasks. |
| Approach: | They propose a sparsification strategy to reduce token costs within Multi-agent Debate (MAD) this strategy minimizes ineffective exchanges of information and unproductive discussions among agents . |
| Outcome: | The proposed approach reduces token costs by up to 94.5% while maintaining performance degradation below 2.0%. |
Copied to clipboard
| Challenge: | Existing Large Language Models (LLMs) and multimodal models are unable to illustrate figurative language based on literal objects, ignoring the underlying groundings and associations across disparate metaphorical domains. |
| Approach: | They propose a grounding-based method for metaphor illustration that integrates metaphorical knowledge into systematic instructions for existing large language models. |
| Outcome: | The proposed method is superior to existing LLMs, diffusion models, or their direct collaboration. |
Copied to clipboard
| Challenge: | Graphical User Interface (GUI) agents powered by Vision-Language Models (VLMs) have demonstrated human-like computer control capability. |
| Approach: | They propose a GUI data synthesis pipeline that reverse engineers GUI trajectory construction process by executing pre-defined tasks. |
| Outcome: | The proposed GUI data synthesis pipeline overcomes the bottlenecks of previous methods that rely on pre-defined tasks and limited data diversity. |
Copied to clipboard
| Challenge: | Existing benchmarks for audio-centric interaction have impeded advancements in this field . AIR-Bench evaluates LALMs' ability to understand audio signals and interact with humans . |
| Approach: | They propose a benchmark to evaluate the ability of large audio-language models to understand audio signals . they use 19 tasks with approximately 19k single-choice questions to examine single-task ability . |
| Outcome: | The proposed framework evaluates the ability of large audio-language models to understand audio signals and interact with humans in the textual format. |
Copied to clipboard
| Challenge: | Existing studies focus on limited control signals such as topic, stance, length, style, strategy, audience, and key aspects, failing to capture this complexity. |
| Approach: | They propose a benchmark that integrates multi-dimensional control into a single instruction to evaluate LLMs' ability to produce persuasive arguments. |
| Outcome: | The proposed benchmarks show that existing models fail to capture multifaceted argumentative control signals. |
Copied to clipboard
| Challenge: | Long-form question answering requires two procedures: information retrieval and information synthesis. |
| Approach: | They propose a Chinese long-form question answering dataset called WebCPM . the dataset is based on a web search interface that engages with a search engine in real time . |
| Outcome: | The proposed dataset generates answers that are no worse than human-written ones . the dataset is the first Chinese LFQA dataset . |
Copied to clipboard
| Challenge: | aims to find more accurate syntactic grammars for accompanying text using video data. |
| Approach: | They build a video-aided grammar induction model that can learn video-span correlation without manual features. |
| Outcome: | The proposed model can learn video-span correlation without manual features adopted by previous systems. |
Copied to clipboard
| Challenge: | Existing storytelling systems suffer from insufficient understanding of event correlations and inadequate awareness of event temporal order. |
| Approach: | They propose a narrative order aware framework to generate coherent stories with flashbacks . they propose 'bidirectional pretraining model with Optimal Transport Reward' to improve quality . |
| Outcome: | The proposed framework generates coherent stories with flashbacks with a novel optimal transport reward. |
Copied to clipboard
| Challenge: | Large language models (LLMs) have achieved remarkable progress, yet their internal mechanisms remain largely opaque. |
| Approach: | They propose an agent-based framework that recasts feature interpretation from a passive, single-pass generation task into an explanation-driven process. |
| Outcome: | The proposed framework produces explanations with significantly higher generative and predictive accuracy compared to state-of-the-art baselines. |
Copied to clipboard
| Challenge: | Existing LLM-based agent models exhibit significant deviations from real-world fund companies. |
| Approach: | They propose a multi-agent financial system that incorporates simulated trading . they propose simulated trades are evaluated without assuming actual risks . |
| Outcome: | The proposed system evaluates various investment strategies without assuming actual risks without involving real-world investors. |
Copied to clipboard
| Challenge: | Existing methods for large language models (LLMs) are limited by their aggressive sample permutation and lack a detailed understanding of the underlying reasons for the reversal curse. |
| Approach: | They propose a method which enhances bidirectional entity correlation modeling and pairwise relationship reasoning to overcome the reversal curse. |
| Outcome: | The proposed method overcomes the reversal curse by augmenting the samples with entity order-reversals and semantically preserved question-answer pairs. |
Copied to clipboard
| Challenge: | Prior studies have focused on designing customized MAS for specific tasks . a critical research question remains: do LLM agent groups exhibit a form of "general intelligence" |
| Approach: | They find a Collective Intelligence factor in human groups that captures their general capability. |
| Outcome: | The proposed model predicts the ACI factor based on the features of LLM agent groups and can improve generalization abilities. |
Copied to clipboard
| Challenge: | Existing summarization methods compress content for gist browsing, but they break prerequisite logic in instructional videos. |
| Approach: | They propose a framework that decouples epistemic planning from content generation. |
| Outcome: | The proposed framework outperforms strong end-to-end baselines on Knowledge Progression Consistency and Learning Objective Coverage. |
Copied to clipboard
| Challenge: | Multi-tenant Model-as-a-Service (MaaS) workloads exhibit non-stationarity across multiple time scales . existing request schedulers often rely on a fixed policy that remains unchanged at runtime . |
| Approach: | They propose a hierarchical multi-agent scheduler that operates in a layered closed loop . they propose to maintain 1.2–3.0 higher Goodput than SGLang and vLLM . |
| Outcome: | Experiments show that H-MAS achieves 1.2–3.0 higher Goodput than SGLang and vLLM . it maintains more stable QoS under diverse request lengths and heterogeneous SLO targets . |
Copied to clipboard
| Challenge: | Clinical natural language processing (NLP) is a subfield that requires the extraction, analysis, and interpretation of unstructured clinical text. |
| Approach: | They propose a model which infuses knowledge into clinical text generation with LLMs for clinical NLP tasks. |
| Outcome: | The proposed model improves performance across 8 clinical NLP tasks and 18 datasets by 7.7%-8.7% on average. |
Copied to clipboard
| Challenge: | Existing deep learning models for EHRs rely on knowledge from a single source and do not capture the semantic information for medical codes. |
| Approach: | They propose a Retrieval AugMentation pipeline to augment clinical prediction on EHRs . they use multiple knowledge sources to convert them into text and use consistency regularization to capture complementary information from patient visits and summarized knowledge. |
| Outcome: | Experiments on two EHR datasets show that RAM-EHR improves clinical prediction tasks. |
Copied to clipboard
| Challenge: | despite advances in large language models, challenges persist due to hallucination-models generating inaccurate content. |
| Approach: | They propose a framework that integrates multi-perspective verification with Retrieval-Augmented Generation to address these challenges. |
| Outcome: | The proposed method outperforms existing models on the SemEval-2016 and VAST datasets. |
Copied to clipboard
| Challenge: | Existing methods implicitly model time constraints by learning time-aware embeddings of questions and candidate answers, which is far from understanding the question comprehensively. |
| Approach: | They propose a temporal-based temporal programming method that leverages the in-context learning ability of Large Language Models to understand combinatory time constraints in questions. |
| Outcome: | The proposed method outperforms existing methods on multiTQ and CronQuestions datasets and is highly efficient on multi-level questions. |
Copied to clipboard
| Challenge: | Recent vision-language models (VLMs) have shown impressive capabilities as general visual assistants, but there are two challenges to their performance: (1) lacking task diversity in pretraining and visual instruction tuning; (2) annotation error and bias in GPT-4 synthesized instruction tuning data. |
| Approach: | They propose a two-stage instruction tuning framework that fine tunes VLMs firstly and further tuned on GPT-4 synthesized data. |
| Outcome: | The proposed framework outperforms the traditional single-stage visual instruction tuning framework and achieves state-of-the-art performance across a wide range of multi-modal evaluation benchmarks. |
Copied to clipboard
| Challenge: | Existing datasets focus on a single type of spoken style, such as disfluencies. |
| Approach: | They propose a Chinese Spoken-to-Written style conversion dataset with 7,237 spoken sentences extracted from transcribed conversational texts. |
| Outcome: | The proposed dataset covers four major conversion problems corresponding to the majority of spoken styles. |
Copied to clipboard
| Challenge: | Large language models (LLMs) memorize evaluation data during training, inflating performance metrics and undermining genuine generalization assessment. |
| Approach: | They propose a framework to detect and quantify benchmark data contamination (BDC) by synthesizing contamination scores via a fuzzy inference system. |
| Outcome: | The proposed framework detects and quantifies BDC risk across semantic, informational, data, and label levels. |
Copied to clipboard
| Challenge: | Existing retrieval augmented language models often overlook effective alignment with human preferences. |
| Approach: | They propose a benchmark to evaluate RMs in retrieval augmented language models . they incorporate 18 RAG subsets, six retrievers, and 24 RALMs to increase diversity . |
| Outcome: | The proposed benchmark combines 18 RAG subsets, six retrievers, and 24 RALMs to increase diversity of data sources. |
Copied to clipboard
| Challenge: | Current development practices face a dichotomy between automation and performance. |
| Approach: | They propose a framework to empower LLMs with the capability of automated explicit vectorization. |
| Outcome: | The proposed framework achieves state-of-the-art performance on the SSE and AVX subsets of SimdBench. |
Copied to clipboard
| Challenge: | Existing event-centric knowledge graphs rely on explicit connectives to extract relations between events. |
| Approach: | They propose a knowledge projection paradigm for event relation extraction using commonalities between events. |
| Outcome: | The proposed method achieves state-of-the-art performance and extrinsic results verify the extracted event relations. |
Copied to clipboard
| Challenge: | Recent advances in pretraining methods have achieved promising results on NLP tasks . however, it is unclear which pretraining objective is the most effective for each downstream task . |
| Approach: | They evaluate the effectiveness of domain-adaptive pretraining objectives on downstream tasks . they use open-domain data to pretrain language models like BERT and SpanBERT . |
| Outcome: | The proposed model improves on two dialogue understanding tasks with domain-adaptive pretraining objectives. |
Copied to clipboard
| Challenge: | Temporal knowledge graph reasoning is a crucial task for answering time-dependent questions within a knowledge graph (KG). |
| Approach: | They propose a temporal KG reasoning benchmark with over 200k entities and 960k questions that facilitate complex, multi-relational and multi-hop reasoning. |
| Outcome: | The proposed model is able to conduct pattern-aware and time-sensitive reasoning across temporal KGs and is scalable to a wide range of data conditions. |
Copied to clipboard
| Challenge: | Existing knowledge base question answering systems that parse natural language questions into knowledge oriented program language (KoPL) . |
| Approach: | They propose a knowledge base question answering system that integrates human into the loop to edit and debug queries. |
| Outcome: | The proposed system can debug and edit knowledge base questions on a million-entity-level . it provides auto-completion for its knowledge base schema and user interaction can fix a large portion of wrong KoPL programs to acquire the correct answer. |
Copied to clipboard
| Challenge: | Recent advances in speech large language models exhibit suboptimal performance in adhering to speech instructions. |
| Approach: | They propose a method to pre-train large-scale unsupervised speech-text sequences . they use text-to-speech conversion to generate textual continuations corresponding to provided speech segments . |
| Outcome: | The proposed model achieves superior or competitive results across diverse speech processing tasks. |
Copied to clipboard
| Challenge: | Large language models (LLMs) have been used in clinical decision support, medical education and patient communication. |
| Approach: | They propose a benchmark to evaluate large language models in the domain of medical ethics and assess their grasp of medical ethical principles and their application across diverse scenarios. |
| Outcome: | The proposed framework assesses the models’ grasp of medical ethics principles and their ability to apply them across diverse scenarios. |
Copied to clipboard
| Challenge: | Evaluation and Management (E/M) coding is performed by physicians and trained human coders who review clinical encounter notes and electronic health record data to assign appropriate codes. |
| Approach: | They propose a framework that automates evaluation and management coding tasks using the Current Procedural Terminology (CPT) taxonomy. |
| Outcome: | The proposed framework achieves an increase in coding accuracy of more than 36% over a commercial CPT E/M coding system and almost 5% over our strongest single-prompt baseline. |
Copied to clipboard
| Challenge: | Existing methods for extracting chemical procedures from literature are insufficient and low-quality due to the inherent ambiguity of chemical language and the high cost of human annotation. |
| Approach: | They propose a fully fine-tuned large language model (LLM) as a chemical executor to convert between unstructured experimental procedures and structured action sequences. |
| Outcome: | The proposed model outperforms the baseline model on R2D and D2A tasks by 10%. |
Copied to clipboard
| Challenge: | Document-level relation extraction (DocRE) provides a broad context for extracting relations for entities. |
| Approach: | They propose a method that utilizes LLMs as a refiner and task distribution and probability fusion to refine LLM-based relation extraction methods. |
| Outcome: | The proposed method outperforms existing LLM-based methods without fine-tuning by 25.2% F1. |
Copied to clipboard
| Challenge: | Experimental results show that n-gram models can achieve satisfactory performance on a large proportion of testing cases. |
| Approach: | They propose to learn a neural LM that fits the residual between an n-gram LM and the real-data distribution. |
| Outcome: | The proposed model achieves additional performance gains over popular standalone models on three typical language tasks. |
Copied to clipboard
| Challenge: | Existing methods of multi-modal grammar induction focus on grammar inducing from text-image pairs, but videos provide even richer information, such as static objects and actions. |
| Approach: | They propose a video-aided grammar induction model which learns a constituency parser from unlabeled text and its corresponding video. |
| Outcome: | The proposed model outperforms existing systems on three benchmarks. |
Copied to clipboard
| Challenge: | Existing training methods for code generation do not improve code correctness and efficiency. |
| Approach: | They propose a framework that integrates preference learning into code generation to improve code correctness and efficiency. |
| Outcome: | The proposed framework improves code correctness and efficiency by integrating preference learning into code generation. |
Copied to clipboard
| Challenge: | Recent advances in deep learning have significantly impacted the legal domain. |
| Approach: | They propose a multi-agent framework for judicial decision-making that simulates the court trial process . they propose 420 Chinese judgment documents to support their framework and build a large-scale legal knowledge base . |
| Outcome: | The proposed framework outperforms existing methods in various aspects, especially in generating legal articles. |
Copied to clipboard
| Challenge: | Increasing interest in sports has led to the rapid advancement of large models, particularly multimodal large language models (MLLMs) . linguistic intelligence is a key component of large-model-driven sports intelligence . |
| Approach: | They propose to establish a foundation for advancing research and practical development of large-model-driven sports intelligence. |
| Outcome: | The proposed model-driven sports intelligence will be able to process and generate sports-related language effectively and process multiple data modalities. |
Copied to clipboard
| Challenge: | Existing studies on Large Language Models (LLMs) have failed to evaluate their performance in event reasoning with a single event relational type or reasoning format. |
| Approach: | They propose a benchmark to evaluate LLMs' event reasoning capability using a single event relational type or reasoning format. |
| Outcome: | The proposed model improves on 10K diverse instruction-tuning demonstrations to alleviate event reasoning-oriented data scarcity. |
Copied to clipboard
| Challenge: | Existing knowledge graph-to-text generation methods focus on sequence-to sequence generation, but the linearized order of KG is obtained through a heuristic search without data-driven optimization. |
| Approach: | They propose to generate easy-to-understand sentences from the knowledge graph . they incorporate part-of-speech syntactic tags to constrain the positions to copy words from the KG and employ a semantic context scoring function to evaluate the semantic fitness for each word in its local context. |
| Outcome: | The proposed method achieves state-of-the-art on two datasets, WebNLG and DART, and achieves high consistency. |
Copied to clipboard
| Challenge: | Existing studies on personalized sentiment classification consider document reviews as overall text unit and incorporate backgrounds (i.e., user and product information) Existing methods for personalized sentiment modeling have quadratic costs that increase with text length and heterogeneous mixes of background information and textual information. |
| Approach: | They propose a knowledge-enhanced and parameter-efficient layer normalization model that leverages pretrained checkpoints and background information into transformer structures. |
| Outcome: | The proposed model can be used to improve pretrained language models in document reviews and incorporate background information with parameter-efficient fine-tuning and knowledge injecting. |
Copied to clipboard
| Challenge: | Large language models have shown a powerful ability for text generation, but undesired behaviors such as toxicity and hallucinations can manifest. |
| Approach: | They propose to formalize text generation as a future-constrained generation problem to minimize undesirable behaviors and enforce faithfulness to instructions. |
| Outcome: | The proposed approach is effective across three tasks, including keyword-constrained generation, toxicity reduction, and factual correctness in question-answering. |
Copied to clipboard
| Challenge: | Existing studies on event-centric opinion mining focus on entity-centric opinions . entity-centered opinions focus on sentimental polarity of events, while event-centered ones focus on content . |
| Approach: | They propose to perform event-centric opinion mining on event-argument structure and expression categorizing theory and benchmark it against a pioneer corpus. |
| Outcome: | The proposed task is feasible and challenging, and the results are beneficial for future studies. |