Papers by Xu Jia
Copied to clipboard
| Challenge: | Existing methods for identifying student misconceptions overlook students' reasoning processes, authors report . |
| Approach: | They propose a knowledge distillation framework that mines high-value samples from existing data. |
| Outcome: | The proposed framework outperforms sota LLM and standard fine-tuned 72B models on cross-topic tests. |
Copied to clipboard
| Challenge: | Large Language Models struggle with complex, multi-step operational tasks because they remain static during inference and cannot learn from past experience. |
| Approach: | They propose a framework that organizes cross-domain insights to facilitate orchestration of long-horizon workflows. |
| Outcome: | The proposed framework outperforms existing methods on the TAC productivity benchmark and shows strong cross-task transferability. |
Copied to clipboard
| Challenge: | Existing text watermarking methods disrupt visual-textual alignment, leaving semantic-critical concepts vulnerable. |
| Approach: | They propose a vision-aligned framework that embeds detectable watermarks into outputs . they combine localized patch affinity, global semantic coherence, contextual attention patterns . |
| Outcome: | The proposed framework shows lower PPL and higher BLEU than conventional methods with near-perfect detection (98.8% AUC). |
Copied to clipboard
| Challenge: | Existing approaches to multi-turn dialogues lack contextual consistency and dependencies, and models struggle to maintain factual faithfulness as interaction turns increase. |
| Approach: | They propose an adaptive context refactoring framework that monitors and reshapes the interaction history to mitigate contextual inertia and state drift. |
| Outcome: | The proposed model outperforms baselines while reducing token consumption. |
Copied to clipboard
| Challenge: | Existing discrepancies between token-level objective and overall sequence-level quality of a model are causing exposure bias and other issues in NMT. |
| Approach: | They propose a contrastive preference model that integrates an indicator function to fine-tune a pre-trained model in Neural Machine Translation. |
| Outcome: | The proposed model outperforms the traditional Plackett-Luce model on three language pairs and also outperFORMs token-level and sequence-level baseline models. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have performed impressively in various NLP tasks, but their inherent hallucination phenomena severely challenge their credibility in complex reasoning. |
| Approach: | They propose to integrate explainable Knowledge Graphs (KGs) with LLMs to alleviate hallucinations . they construct subgraphs to enhance the retrieval capabilities of KGs via CoT reasoning. |
| Outcome: | Extensive experiments on two KGQA datasets show that the proposed model achieves convincing performance compared to strong baselines. |
Copied to clipboard
| Challenge: | Quantization has shown promise for Large Language Models, but current methods require lengthy training to alleviate quantization loss. |
| Approach: | They propose to decouple weights and incorporate Low-Rank adapters to reduce weight sharing . they validate the approach on LLaMA2 families and Mistral on downstream evaluation . |
| Outcome: | The proposed approach shows high performance while reducing deployment time faced with multiple scenarios. |
Copied to clipboard
| Challenge: | Existing approaches to relevance modeling have lacked generalization and accuracy . recent studies have focused on capturing the semantic relationships between queries and items . |
| Approach: | They propose a framework that integrates world knowledge stored in LLMs with specialized domain knowledge represented by user behavior data for promising performance. |
| Outcome: | The proposed framework can handle full-scale search traffics of Alipay with acceptable cost and latency. |
Copied to clipboard
| Challenge: | Existing approaches to rerank and align documents based on reasoning capabilities of large language models (LLMs) . prior work shows that LLMs have exceptional reasoning and text generation capabilities . |
| Approach: | They propose a rationale extraction method that leverages reasoning capabilities of large language models to extract the rationales necessary for answering a query. |
| Outcome: | The proposed method is compared with baseline methods on two tasks across three datasets. |
Copied to clipboard
| Challenge: | Learning transfer theory emphasizes that applying acquired knowledge to novel manifestations is a key signal of deep understanding |
| Approach: | They propose a benchmark that probes transfer robustness along two rewrite levels: Near Transfer and Far Transfer. |
| Outcome: | The proposed benchmark demonstrates that large language models are robust when faced with novel manifestations of the same problem. |
Copied to clipboard
| Challenge: | Existing methods to generate human-like questions rely on paraphrases to generate good questions. |
| Approach: | They propose to integrate paraphrase knowledge into question generation to generate human-like questions by combining paraphrases with a back-translation method. |
| Outcome: | The proposed model achieves obvious performance gain over several strong baselines and human evaluation validates that it can ask questions of high quality by leveraging paraphrase knowledge. |
Copied to clipboard
| Challenge: | Current Information Seeking (InfoSeeking) agents struggle to maintain focus and coherence during long-horizon exploration, as tracking search states within one plain-text context is inherently fragile. |
| Approach: | They propose a structured planning framework that reformulates the InfoSeeking task as a Table Completion task. |
| Outcome: | The proposed framework outperforms state-of-the-art frameworks across three kinds of benchmarks, including multi-agent framework and commercial systems. |
Copied to clipboard
| Challenge: | a diversity advanced actor-critical reinforcement learning framework is used to improve NLP generalization and accuracy. |
| Approach: | They introduce Diversity Advanced Actor-Critic reinforcement learning framework to improve NLP generalization and accuracy. |
| Outcome: | The proposed framework outperforms domain adaptation and generalization baselines without using any target domain knowledge. |
Copied to clipboard
| Challenge: | Existing approaches for optimizing domain-level sampling strategies struggle with maintaining intra-domain consistency and accurately measuring domain impact. |
| Approach: | They propose to use a Fisher-Information Matrix-guided metric to measure domain impact to ensure intra-domain consistency and accuracy. |
| Outcome: | The proposed model achieves 3.4% higher average performance while maintaining comparable training efficiency. |
Copied to clipboard
| Challenge: | Existing benchmarks focus on isolated function/class-level generation, neglecting complete microservice repository generation. |
| Approach: | They propose a multilingual benchmark for repository-level end-to-end web microservice generation that reflects real-world development workflows. |
| Outcome: | The benchmark compared 106 repositories across 18 domains and 11 frameworks and 1,258 API endpoints and 2,335 test cases. |
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) models focus on word-level information, while segment-based models focus only on word level information. |
| Approach: | They propose a Modularized Interaction Network (MIN) model which utilizes both word-level information and segment-level dependencies. |
| Outcome: | The proposed model outperforms the current state-of-the-art models on three NER benchmark datasets. |
Copied to clipboard
| Challenge: | Existing methods to evaluate NLP models are limited to news domains and cannot be generalized to other domains. |
| Approach: | They propose a measure of NLP quality based on robustness . they measure consistency of cross-domain accuracy and introduce coefficient of variation and gamma-Robustness based upon human evaluation . |
| Outcome: | The proposed approach shows higher agreement with human evaluation than accuracy scores on ranking machine translation systems. |
Copied to clipboard
| Challenge: | Existing methods for event argument extraction cannot adequately model the correlation between event arguments and their roles. |
| Approach: | They propose a Bayesian model to jointly extract event arguments using Gibbs sampling . they train two neural networks to model prior distribution and conditional distribution over event arguments . |
| Outcome: | The proposed model can achieve comparable results to existing methods on two widely-used datasets. |
Copied to clipboard
| Challenge: | Few-shot text matching is a more practical technique to determine whether two texts are semantically identical. |
| Approach: | They propose a pluggable prompt learning method for few-shot text matching . they use the semantics of instances to regulate the effects of the gate on the prompt tokens . |
| Outcome: | The proposed method outperforms baselines on MRPC and QQP. |
Copied to clipboard
| Challenge: | Existing methods for temporal relational forecasting are limited and require limited training data. |
| Approach: | They propose a retrieval-augmented generation framework that uses temporal logical rule-based retrieval and parameter-efficient instruction tuning to solve temporal knowledge forecasting challenges. |
| Outcome: | The proposed framework outperforms conventional methods in the temporal knowledge graph domain with low computation resources. |
Copied to clipboard
| Challenge: | Existing knowledge editing methods struggle to reason about related conceptual knowledge effectively, despite a lack of model-level relational reasoning. |
| Approach: | They propose a benchmark to assess concept-level and instance-level relational reasoning abilities of edited models. |
| Outcome: | The proposed model obtains the best scores on the memory-based in-context editing baseline, MICE, suggesting a promising direction for model editing. |
Copied to clipboard
| Challenge: | Existing methods for event argument extraction (EAE) lack cross-event information and require longer role sequences . et al. (2017): outperforms state-of-the-art methods for EE. |
| Approach: | They propose a separation-and-fusion paradigm to separate the acquisition of cross-event information and fuse it into the argument extraction of a target event. |
| Outcome: | The proposed model outperforms the state-of-the-art models on four widely used datasets. |
Copied to clipboard
| Challenge: | Existing methods for developing compact and efficient large language models lack token-level dependencies and linguistic diversity. |
| Approach: | They propose a logits-based fine-tuning framework that integrates supervised learning and knowledge distillation to build enriched training targets using teacher logits and ground truth labels. |
| Outcome: | The proposed method outperforms existing methods on a large-scale logits dataset and a series of science-focused models. |
Copied to clipboard
| Challenge: | Existing memory systems rely on static, hand-crafted update rules for personalization, but sparse outcome rewards provide weak supervision, resulting in unstable long-horizon optimization. |
| Approach: | They propose a memory guideline optimization framework that learns how memory should be organized and what information to update. |
| Outcome: | The proposed framework learns how memory should be organized and what information to update. |
Copied to clipboard
| Challenge: | Using a relevant QE model is also very important in QE. |
| Approach: | They evaluate a wide range of advanced sentence-level Quality Estimation models including Support Vector Regression, Ride Regression and Bayesian Neural Networks. |
| Outcome: | The proposed models behave differently in evaluation settings depending on whether test data come from the same domain as the training data or not. |
Copied to clipboard
| Challenge: | Large Language Models have shown impressive ability to perform in-context learning from only a few examples, but their accuracy varies widely from task to task. |
| Approach: | They propose a method that trains a meta-model using LLM confidence scores as features to perform ICL accuracy estimation. |
| Outcome: | The proposed method improves over baselines across 7 out of 12 settings and achieves the same accuracy as evaluating on 40 sampled examples per task. |
Copied to clipboard
| Challenge: | EMBYTE is a byte-level tokenization model that reduces embedding parameters by up to 94% . it is also resilient to privacy threats such as gradient inversion attacks . |
| Approach: | EMBYTE is a byte-level tokenization model that decomposes subwords into fine-grained byte embeddings and then compresses them via neural projection. |
| Outcome: | EMBYTE achieves substantial embedding compression while preserving accuracy and enhancing privacy. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) struggle with capturing long-distance dependencies within sequences to deeply understand semantics. |
| Approach: | They propose a system that captures relevant information within a fixed window size and provides precise answers to queries. |
| Outcome: | The proposed system can read Harry Potter within 30s and accurately answer the questions. |
Copied to clipboard
| Challenge: | Recent years, pre-trained language models (PLMs) have achieved promising results on various NLP tasks. |
| Approach: | They propose an open-source toolkit for big model inference and tuning which can support big model tuning at extremely low computation cost. |
| Outcome: | The proposed toolkit can support big model inference and tuning at extremely low computation cost. |
Copied to clipboard
| Challenge: | Existing models confuse implicit and explicit sentiment, making it difficult to extract quadruples effectively. |
| Approach: | They propose a framework that leverages distinct labeled features from diverse reviews and incorporates pseudo-token prompts to harness the semantic knowledge of pre-trained models. |
| Outcome: | The proposed framework improves over four public datasets, averaging 1.99% F1 improvement, particularly in instances involving implicit sentiment. |
Copied to clipboard
| Challenge: | prevailing pre-training approaches for large language models involve several complexities. |
| Approach: | They propose a low-cost training recipe and a robust optimization approach to mitigate training instability . they also propose synthesis, curriculum, and data selection pipelines to integrate data . |
| Outcome: | The proposed model achieves top-tier performance among models with similar parameter scale . it is comparable to industry-leading models that require significantly more data . |
Copied to clipboard
| Challenge: | Multi-agent systems (MAS) powered by large language models struggle to adapt to evolving task dependencies and to handle uncertainties. |
| Approach: | They propose a Dynamic Environment-Aware Manager-Player Agents Coordination framework that enhances multi-agent coordination through long-term strategic planning. |
| Outcome: | The proposed framework outperforms traditional reinforcement learning and human-agent collaboration in the Overcooked simulation. |
Copied to clipboard
| Challenge: | Existing LLMs do not possess consistent values, but many have been developed to align them at the behavioral level, including supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF). |
| Approach: | They propose a Controlled Value Vector Activation method that directly aligns the internal values of Large Language Models by interpreting how a value is encoded in their latent representations. |
| Outcome: | The proposed method achieves highest success rate across 10 basic values without hurting model performance and fluency, and ensures target values even with opposite and potentially malicious input prompts. |
Copied to clipboard
| Challenge: | Modern machine learning works with massive amounts of data on a range of tasks like language modeling, object detection, and data mining. |
| Approach: | They propose a probabilistic robustness rewarded data optimization approach to enhance the model's generalization power by selecting training data that optimizes probabilistic metrics. |
| Outcome: | The proposed approach achieves +17.2% increase of accuracy and -28.05 decrease of perplexity on unknown-domain test sets. |
Copied to clipboard
| Challenge: | Emotion recognition in conversation is important for an empathetic dialogue system to understand the user’s emotion and then generate appropriate emotional responses. |
| Approach: | They propose to use multimodal directed acyclic graphs to integrate multimodal information and contextual information into a DAG architecture to exploit multimodal contexts. |
| Outcome: | Comparative studies on IEMOCAP and MELD show that the proposed model outperforms state-of-the-art models. |
Copied to clipboard
| Challenge: | Graphical User Interface (GUI) agents powered by Vision-Language Models (VLMs) have demonstrated human-like computer control capability. |
| Approach: | They propose a GUI data synthesis pipeline that reverse engineers GUI trajectory construction process by executing pre-defined tasks. |
| Outcome: | The proposed GUI data synthesis pipeline overcomes the bottlenecks of previous methods that rely on pre-defined tasks and limited data diversity. |
Copied to clipboard
| Challenge: | Existing benchmarks for realistic financial analysis fail to capture realistic financial situations involving cross-document retrieval, multi-page evidence integration, and diverse analytical tasks. |
| Approach: | They propose a multi-modal financial RAG benchmark that evaluates large language models in realistic financial analysis settings. |
| Outcome: | The proposed framework achieves the strongest overall performance across all models. |
Copied to clipboard
| Challenge: | Existing research mainly focuses on performance upper bounds in static environments, overlooking stochastic real-world deployment. |
| Approach: | They propose a dynamic evaluation environment that simulates a "trainee" agent continuously exploring a novel setting. |
| Outcome: | The proposed model evaluates agents in a dynamic evaluation environment that simulates a "trainee" agent continuously exploring a novel setting. |
Copied to clipboard
| Challenge: | Despite advances in large language models, they face substantial challenges in terms of safety. |
| Approach: | They develop a safety-aware decoding strategy for large language models to defend against jailbreak attacks. |
| Outcome: | The proposed strategy outperforms six defense methods against jailbreak attacks on five LLMs. |
Copied to clipboard
| Challenge: | Existing systems treat roles as static prompts and rely on one-shot safety filters . a self-evolving LLM agent is proposed that learns from role-based social experience . |
| Approach: | They propose a self-evolving LLM agent that learns from role-based social experience and explicitly models communicator-level individual traits informed by prior communication questionnaires and clinical literature. |
| Outcome: | The proposed agent learns from role-based social experience and models communicator-level individual traits informed by prior communication questionnaires and clinical literature. |
Copied to clipboard
| Challenge: | Experimental results show superior performance on perplexity and topic coherence measures compared to state-of-the-art topic models. |
| Approach: | They propose to incorporate topic coherence measures as reward signals to guide the learning of a VAE-based topic model. |
| Outcome: | The proposed model is able to separating background words dynamically from topic words eliminating the pre-processing step of filtering infrequent and/or top frequent words, typically required for learning traditional topic models. |
Copied to clipboard
| Challenge: | High-quality data is the cornerstone of advancing large language models, but the supply of premium data is nearing depletion, while vast stale corpora remain underutilized. |
| Approach: | They propose a framework to restore stale data affinity by quantifying the latent value of samples and employing a dynamic renovation strategy selection mechanism to determine the optimal component-level strategy. |
| Outcome: | The proposed framework achieves performance improvements using less than 10% of the data volume, underscoring that the latent potential of stale corpora remains largely untapped. |
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated impressive capabilities in multi-step and long-chain reasoning, but extending their reasoning capabilities to encompass deep interactions with search remains a non-trivial challenge. |
| Approach: | They propose a framework for Reasoning–Search integration that integrates multi-reward signals to optimize the reasoning–search interaction trajectories. |
| Outcome: | Experiments on seven datasets show that R-Search significantly outperforms mainstream RAG baselines. |
Copied to clipboard
| Challenge: | Existing summarization methods compress content for gist browsing, but they break prerequisite logic in instructional videos. |
| Approach: | They propose a framework that decouples epistemic planning from content generation. |
| Outcome: | The proposed framework outperforms strong end-to-end baselines on Knowledge Progression Consistency and Learning Objective Coverage. |
Copied to clipboard
| Challenge: | Existing legal judgment prediction methods struggle with logical errors when conducting complex legal reasoning. |
| Approach: | They propose a method which enhances LJP reliability through step-wise verification and correction of the reasoning process. |
| Outcome: | The proposed model significantly improves concordance with court decisions from 72.37 to 80.27 on LLAMA-3.1-70B. |
Copied to clipboard
| Challenge: | a new approach to natural language processing uses arbitrary symbols to represent meaning . Soundex, MetaPhone, NYSIIS, logogram are used as inputs for NLP . |
| Approach: | They propose to use arbitrary symbols to represent linguistic meaning of a word . they propose to integrate codewords with text to provide more reliable inputs . |
| Outcome: | The proposed approach outperforms state-of-the-art models on machine translation, language modeling, and part-of speech tagging. |
Copied to clipboard
| Challenge: | Existing approaches focus on diagnostic reasoning based on internal model knowledge or static knowledge bases. |
| Approach: | They propose a two-stage diagnostic reasoning framework that integrates multi-perspective evidence to generate a diagnostic prediction. |
| Outcome: | The proposed method generates suspected diagnoses and reasoning traces from web search, SOAP-formatted case, and clinical case database. |
Copied to clipboard
| Challenge: | Multi-tenant Model-as-a-Service (MaaS) workloads exhibit non-stationarity across multiple time scales . existing request schedulers often rely on a fixed policy that remains unchanged at runtime . |
| Approach: | They propose a hierarchical multi-agent scheduler that operates in a layered closed loop . they propose to maintain 1.2–3.0 higher Goodput than SGLang and vLLM . |
| Outcome: | Experiments show that H-MAS achieves 1.2–3.0 higher Goodput than SGLang and vLLM . it maintains more stable QoS under diverse request lengths and heterogeneous SLO targets . |
Copied to clipboard
| Challenge: | Various visionlanguage pre-training (VLP) models learn cross-modal alignment from large-scale well-aligned image-text datasets without leveraging external knowledge. |
| Approach: | They propose a knowledge-guided fashion-domain language-image pre-training framework that learns fine-grained representations in e-commerce domain and utilizes external knowledge to improve the pre-train efficiency. |
| Outcome: | The proposed framework outperforms state-of-the-art models on Amazon and Fashion-Gen datasets by large margins. |
Copied to clipboard
| Challenge: | Existing methods for data augmentation produce low readability or semantic consistency. |
| Approach: | They propose a framework which augments data through reinforcement learning guided conditional generation. |
| Outcome: | The proposed framework improves F1 performance on three different classification tasks by 8.7% on average when given only 10% of the whole data for training. |
Copied to clipboard
| Challenge: | Existing data filtering methods rely on coarse-grained scores that lack granularity to identify nuanced semantic flaws. |
| Approach: | They propose a "Decomposition-then-Evaluation" paradigm that breaks model responses into constituent cognitive components. |
| Outcome: | The proposed model outperforms models trained on larger datasets in three key areas . the authors show that Logical Coherence is the most critical factor in data quality evaluation . |
Copied to clipboard
| Challenge: | despite the adoption of Large Language Models (LLMs), contract revision remains impeded because generic models treat strict legal constraints as mere suggestions. |
| Approach: | They propose a risk-constrained bilevel Stackelberg framework that models high-stakes revision as a strategic interaction rather than an open-ended conversation. |
| Outcome: | The proposed framework achieves state-of-the-art performance with an average RRR of 84.21% and enhanced token efficiency. |
Copied to clipboard
| Challenge: | Supervised fine-tuning (SFT) is a widely used method for adapting Large Language Models to specific tasks. |
| Approach: | They propose a method that uses supervised fine-tuning to train a reward model for reinforcement learning. |
| Outcome: | The proposed method outperforms existing methods on in-domain benchmarks but surpasses them 50 times on out-of-domain and cross-task evaluations. |
Copied to clipboard
| Challenge: | Existing methods for temporal knowledge graph completion (TKGC) focus on extracting information from timestamps and insufficiently utilizing implied information in relations. |
| Approach: | They propose a temporal knowledge graph completion model with prompts that converts quadruples into pre-trained language inputs and prompts to make coherent sentences with implicit semantic information. |
| Outcome: | The proposed model can make coherent sentences with implicit semantic information. |
Copied to clipboard
| Challenge: | Existing methods for generating large language models face limitations in key aspects such as retrieval triggers and contextual scrutiny of retrieval content. |
| Approach: | They propose a dynamic RAG method that uses cognitive detection and contextual retrieval optimization to determine when retrieval is needed and what to retrieve for LLMs. |
| Outcome: | The proposed method achieves superior performance on all tasks, demonstrating the effectiveness of the proposed method. |
Copied to clipboard
| Challenge: | a new framework to analyze how latent concepts are encoded in representations learned in pre-trained lan-guage models is proposed . conceptX uses clustering to discover the encoded concepts and align them with a large set of human-defined concepts. |
| Approach: | They propose a framework to analyze how latent concepts are encoded in representations learned within pre-trained lan-guage models. |
| Outcome: | The proposed framework explains encoded concepts by aligning with human-defined concepts. |
Copied to clipboard
| Challenge: | Existing methods for continual prompt tuning are limited by the ever-growing parameter scale of modern language models (e.g., GPT-4 that may have 1.76 trillion parameters). |
| Approach: | They propose a method for continual prompt tuning that enables the lifelong learning of a pre-trained language model by adding a task-specific prompt to a queue of older tasks. |
| Outcome: | The proposed method outperforms the state-of-the-art methods substantially on continual prompt tuning benchmarks. |
Copied to clipboard
| Challenge: | Existing approaches to retrieval-augmented generation still face problems with low context utilization and frequent hallucinations. |
| Approach: | They propose a framework that reformulates retrieval and generation as constrained optimization and path planning. |
| Outcome: | The proposed framework significantly improves reasoning accuracy on complex queries while reducing hallucinations. |
Copied to clipboard
| Challenge: | a new approach to grouping input words based on their semantic diversity is proposed . high-dimensional inputs and learning complexity hinders deep learning generalization, authors say . |
| Approach: | They propose a way to group input words based on their semantic diversity to simplify input language representation with low ambiguity. |
| Outcome: | The proposed methods generalize NLP models and demonstrate improvements on medium-scale machine translation tasks. |
Copied to clipboard
| Challenge: | Existing models with quadratic time and memory complexity restrict input length . however, analyzing extensive sequential contexts is challenging . |
| Approach: | They propose a neural network architecture that captures contextual dependencies in linear time and a nonlinear readout to model short-term dependencies within sentences. |
| Outcome: | The proposed model outperforms baseline models on EmoryNLP datasets and on IEMOCAP and MultiWOZ datasets. |
Copied to clipboard
| Challenge: | Existing work has not studied how byte encoding can benefit endangered languages . multilingual neural machine translation (MNMT) models suffer from out-of-vocabulary issues and representation bottleneck . |
| Approach: | They propose a multilingual multilingual neural machine translation system to alleviate the representation bottleneck and improve translation performance in endangered languages. |
| Outcome: | The proposed system outperforms subword-based models on twelve languages up to +18.5 BLEU points, an 840% relative improvement over baseline models. |
Copied to clipboard
| Challenge: | Large language models (LLMs) face factual hallucination and knowledge obsolescence when tackling knowledge-intensive tasks. |
| Approach: | They propose a layer-knowledge guided attention method which harnesses the layer-wise knowledge of large language models to optimize per-layer attention on useful passages. |
| Outcome: | The proposed method outperforms existing methods on RALM benchmarks. |
Copied to clipboard
| Challenge: | Existing agents lack generalization and specialization capabilities for open-ended tasks . specialized generalists are often underdeveloped in real-world environments . |
| Approach: | They propose a platform to dynamically integrate heterogeneous agents for automating computer tasks . they propose specialized generalist agent MetaAgent with the AgentToken strategy . |
| Outcome: | The proposed platform expands capabilities of existing agents in generalization and specialization . it can be used to automate open-ended tasks in real-world environments . |
Copied to clipboard
| Challenge: | Existing Text-to-SQL research focuses on specific database systems, limiting adaptability to different dialects. |
| Approach: | They propose a framework that employs Object Relational Mapping (ORM) code as an intermediate language to bridge this gap. |
| Outcome: | The proposed framework outperforms existing methods that generate SQL queries directly. |
Copied to clipboard
| Challenge: | generative models have been used for various NLP tasks but their application in the field of input methods remains under-explored. |
| Approach: | They propose a novel Generative Input paradigm that uses prompts to handle all input scenarios and other intelligent auxiliary input functions, optimizing the model with user feedback. |
| Outcome: | The proposed paradigm achieves state-of-the-art in the Full-mode Key-sequence to Characters task and surpasses GPT-4 in the other input methods. |
Copied to clipboard
| Challenge: | Existing methods for classification are overly confident on unseen examples . despite recent advances in NLP, some categories of distribution shift still pose serious challenges. |
| Approach: | They propose a method that generates OOD examples representative of novel classes and trains to decrease confidence on them. |
| Outcome: | The proposed method improves classifiers' ability to detect and abstain on novel class examples over previous methods by 2.3% and 5.5% over previous approaches. |
Copied to clipboard
| Challenge: | Existing financial benchmarks suffer from limited language and task coverage, low-quality datasets, and inadequate adaptability for LLM evaluation. |
| Approach: | They propose a bilingual benchmark for financial LLMs that assesses models’ language understanding and generation capabilities. |
| Outcome: | The proposed bilingual benchmark assesses models’ language understanding and generation capabilities. |
Copied to clipboard
| Challenge: | Relevance modeling between queries and items is a key component of commercial search engines. |
| Approach: | They propose a framework for continual pre-training of LLMs to enhance domain knowledge . they employ queries and multi-field item to jointly pre-train for enhancing domain knowledge. |
| Outcome: | The proposed model achieves convincing performance compared to strong baselines. |
Copied to clipboard
| Challenge: | Chain-of-Thought (CoT) prompts elicit multi-step reasoning, yet how reasoning related structure is expressed during training remains poorly understood. |
| Approach: | They propose a framework that tracks span-level gradients during fine-tuning on reasoning benchmarks to understand how models develop structured, step-by-step reasoning capabilities. |
| Outcome: | The proposed framework tracks span-level gradients during fine-tuning on reasoning benchmarks to understand how models develop structured, step-by-step reasoning capabilities. |
Copied to clipboard
| Challenge: | Existing studies on Large Language Models (LLMs) have failed to evaluate their performance in event reasoning with a single event relational type or reasoning format. |
| Approach: | They propose a benchmark to evaluate LLMs' event reasoning capability using a single event relational type or reasoning format. |
| Outcome: | The proposed model improves on 10K diverse instruction-tuning demonstrations to alleviate event reasoning-oriented data scarcity. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) struggle to automate real-world vulnerability detection due to the heterogeneity of vulnerability patterns and manual prompt engineering for massive weakness categories is unscalable. |
| Approach: | They propose a retrieval-augmented multi-agent framework for precise and broad-coverage vulnerability detection using a coarse-to-fine strategy. |
| Outcome: | The proposed framework outperforms the baseline model on 130 CWE types and achieves 34.79% Macro-F1 performance. |
Copied to clipboard
| Challenge: | Existing benchmarks focus on evaluating MLLMs’ pre-existing knowledge or perceptual understanding, often neglecting the critical capability of reasoning. |
| Approach: | They propose a benchmark designed for visual clue-driven reasoning in daily scenarios that combines rigorous grounding in authentic daily activities and challenging query design that necessitates more than surface-level perception. |
| Outcome: | The proposed benchmark identifies visual clues and their ability to provide robust reasoning in daily scenarios. |
Copied to clipboard
| Challenge: | Recent studies have shifted paradigms and leveraged Large Language Models (LLMs) to tackle the challenging task of Text-to-SQL. |
| Approach: | They propose a framework that leverages large language models to generate SQL queries . they exploit prior knowledge from the LLM to enhance embedding-based retriever . |
| Outcome: | The proposed method improves embedding-based retriever and reduces cost. |