Papers by Xu Bai
Copied to clipboard
| Challenge: | Existing datasets often rely on synthetic data or figure-caption pairs, failing to capture the depth and complexity of geoscientific reasoning. |
| Approach: | They propose a multimodal scientific dataset and benchmark curated from open-access publications. |
| Outcome: | MSEarth features over 289K figures with captions enriched by contextual discussions and reasoning from original papers. |
Copied to clipboard
| Challenge: | Existing top-k attention methods struggle to strike a balance between efficiency and accuracy. |
| Approach: | They propose a top-k attention approach that integrates low-overhead techniques into the Top-k Attention process to achieve 7.2 speedup compared to vanilla full attention. |
| Outcome: | The proposed approach achieves 7.2 speedup compared to current top-k attention methods while maintaining model accuracy. |
Copied to clipboard
| Challenge: | Large language models (LLMs) exhibit substantial capabilities yet face challenges such as hallucination, outdated knowledge, and untraceable reasoning processes. |
| Approach: | They propose a retrieval-augmented generation approach that leverages adaptive adversarial training to dynamically adjust the model’s training process in response to retrieval noises. |
| Outcome: | The proposed approach improves the performance of the LLaMA-2 7B model under diverse noise conditions. |
Copied to clipboard
| Challenge: | Existing methods to generate responses using beam search focus on current optimal results. |
| Approach: | They propose a beam search method that uses a Prospective-Performance Network to predict the future reward of a partially-generated response. |
| Outcome: | The proposed method can increase the quality and diversity of generated responses with high inference efficiency. |
Copied to clipboard
| Challenge: | Existing models struggle to balance predictive accuracy with human-understandable rationales. |
| Approach: | They propose to enhance LLMs by leveraging rationale distillation and domain knowledge injection for trustworthy multimodal rationale generation. |
| Outcome: | Experiments on real-world medical datasets show that ClinRaGen achieves state-of-the-art performance in disease diagnosis and rationale generation. |
Copied to clipboard
| Challenge: | Existing word embeddings are limited in their ability to represent fixed vectors . instead, they incorporate relational dependencies of different words into their embeddables - a limitation that is addressed by a multiplex model . |
| Approach: | They propose a word embedding model which incorporates relational dependencies of different words into their embeddables. |
| Outcome: | The proposed model can be easily extended according to various relations among words. |
Copied to clipboard
| Challenge: | Conceptualization is a fundamental element of human cognition and plays a pivotal role in generalizable reasoning. |
| Approach: | They propose to categorize different types of conceptualizations into four levels based on the types of instances being conceptualized. |
| Outcome: | The proposed categorization of different types of conceptualizations into four levels based on the types of instances being conceptualized . |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated significant advancements in various fields, notably in Role-Playing Conversational Agents (RPCAs). |
| Approach: | They propose an Anchoring-Guidance Fine-Tuning Framework to integrate relevant expert knowledge into RPCAs' training process to mitigate this issue. |
| Outcome: | The proposed framework significantly improves the RPCAs’ performance in handling role-specific professional queries while preserving their robust role-playing abilities. |
Copied to clipboard
| Challenge: | Document images are characterized by higher resolutions, denser content, and more complex structural layouts. |
| Approach: | They propose a 1.2B-parameter document parsing vision-language model that decouples layout analysis from local content recognition. |
| Outcome: | The proposed model surpasses general-purpose and domain-specific models on multiple benchmarks while maintaining significantly lower computational overhead. |
Copied to clipboard
| Challenge: | a number of studies have shown that transformer-based language models detect when a word is anomalous in context, but likelihood scores do not tell the cause of the anomaly. |
| Approach: | They propose to use Gaussian models for density estimation at intermediate layers of three language models to evaluate grammaticality. |
| Outcome: | The proposed method on BLiMP shows that language models employ different mechanisms to detect different types of linguistic anomalies. |
Copied to clipboard
| Challenge: | Existing medical reasoning datasets are limited in scale and typically rely on incomplete data. |
| Approach: | They propose to use ReasonMed to train medical reasoning models using a multi-agent generation, verification, and refinement pipeline. |
| Outcome: | The largest medical reasoning dataset to date surpasses the prior best sub-10B models by 4.17% and even exceeds LLaMA3.1-70B on PubMedQA by 4.60%. |
Copied to clipboard
| Challenge: | Existing methods for multimodal aspect-based sentiment classification rely on superficial correlations and spurious cues. |
| Approach: | They propose a Dual-Path Counterfactual Integration framework that explicitly models counterfactual reasoning in multimodal contexts. |
| Outcome: | The proposed framework improves model robustness by explicitly modeling counterfactual reasoning in multimodal contexts. |
Copied to clipboard
| Challenge: | Character linking is the task of linking mentioned people in conversations to the real world . human use of pronouns or normal entities makes it difficult to link mentioned people to real people . a critical step towards understanding conversations is grounding mentioned people - a goal of the natural language processing community . |
| Approach: | They propose to integrate richer context from the coreference relations among different mentions to help the linking task. |
| Outcome: | The proposed model outperforms all previous models on both tasks. |
Copied to clipboard
| Challenge: | Existing benchmarks on longcontext large language models fail to reflect their deep understanding capabilities across diverse tasks. |
| Approach: | They propose a benchmark to assess the ability of long-context large language models to handle long-text problems. |
| Outcome: | The proposed model achieves 50.1% accuracy when directly answering the questions . human experts achieve only 53.7% accuracy under a 15-minute time constraint . |
Copied to clipboard
| Challenge: | Existing models for text-to-image generation have been underperforming in image-totext generation tasks. |
| Approach: | They propose a framework that uses a split BERT to create a dedicated latent space for captions and integrates a regularization module to manage varying text lengths. |
| Outcome: | The proposed framework achieves state-of-the-art performance on the MS COCO dataset with 38.2 BLEU@4 and 126.2 CIDEr . |
Copied to clipboard
| Challenge: | Existing quantization methods are compromising performance of large language models (LLMs) despite their high computational intensity, LLMs are still demanding intensive computation. |
| Approach: | They propose to generate the KV cache of pivot tokens losslessly from the full-precision model. |
| Outcome: | The proposed method generates the KV cache of pivot tokens losslessly from the full-precision model with no extra inference overhead. |
Copied to clipboard
| Challenge: | lexicalist linguistic theories assume argument structure is predictable from meaning of verbs . construction grammarians propose argument structure constructions distinct from verbs. |
| Approach: | They adapt psycholinguistic studies to probe for the existence of argument structure constructions in Transformer-based language models. |
| Outcome: | The proposed method could be used to probe argument structure constructions in LMs . the study shows that LM learners prefer grouping by construction over verb grouping . |
Copied to clipboard
| Challenge: | Existing memory systems for LLMs store isolated records and retrieve fragments . Existing systems store isolated data and fragments, limiting their ability to consolidate evolving experience and resolve conflicts. |
| Approach: | They propose an engram-inspired memory operating system that implements an 'engram'-inspired lifecycle for computational memory. |
| Outcome: | Experiments on LoCoMo, LongMemEval, and PersonaMeM-v2 show that EverMemeOS outperforms state-of-the-art methods on memory-augmented reasoning tasks. |
Copied to clipboard
| Challenge: | Existing work attempts to address these challenges using Pretrained Language Models (PLMs) but the diversity of surface form expressions can hinder the model’s ability to capture accurate correlations, especially when the context is lengthy and complex. |
| Approach: | They propose a method known as Graph-as-Token (GST) to incorporate AMRs into PLMs to assist the model in understanding complex semantic information. |
| Outcome: | The proposed method outperforms existing methods and significantly improves performance on both Natural Questions and TriviaQA. |
Copied to clipboard
| Challenge: | Existing benchmark-based evaluations cannot accurately reflect the performance of real-world applications. |
| Approach: | They propose a reliable strategy for domains to choose more robust LLMs for real-world applications. |
| Outcome: | The proposed strategy addresses the challenges faced by domains to choose more robust LLMs for real-world applications. |
Copied to clipboard
| Challenge: | Recent work shows that pre-training in-domain language models can boost performance when adapting to a new domain. |
| Approach: | They propose to combine annotation and pre-training to maximize performance under budget constraints. |
| Outcome: | The proposed approach is based on the annotation cost of three procedural text datasets and pre-training cost of 3 in-domain language models. |
Copied to clipboard
| Challenge: | Existing efforts to learn meaningful representations at the instance level are limited. |
| Approach: | They propose a deep embedded clustering model with cluster-level representation learning to jointly learn cluster and instance level representations. |
| Outcome: | The proposed model produces meaningful clusters on real-world short text datasets. |
Copied to clipboard
| Challenge: | Large language models excel in various language tasks, while large multimodal models effectively handle visual-language problems. |
| Approach: | They propose to use a multimodal multimodal model evaluation benchmark to evaluate model performance in Chinese K12 classrooms. |
| Outcome: | The proposed model evaluation tool is integrated with the CMMaTH dataset. |
Copied to clipboard
| Challenge: | Transferability estimation has been a topic of great interest in computer vision fields . a lack of a comprehensive comparison between these estimation methods is a problem . |
| Approach: | They conduct a thorough survey of existing methods to find the most suitable model . they also outline difficulties of consideration of training details and applicability to text generation . |
| Outcome: | The proposed methods perform well with superiorities in effectiveness and efficiency. |
Copied to clipboard
| Challenge: | Existing benchmarks focus on specific application scenarios, emphasizing task completion but failing to dissect the underlying skills that drive these outcomes. |
| Approach: | They propose a Massive Multitask Agent Understanding benchmark that evaluates LLMs across five domains and offline tasks. |
| Outcome: | The Massive Multitask Agent Understanding (MMAU) benchmark evaluates models across five domains including Tool-use, Directed Acyclic Graph (DAG) QA, Data Science and Machine Learning coding, Contest-level programming and Mathematics. |
Copied to clipboard
| Challenge: | Experimental results demonstrate that a Pruned interpretable knowledge Graph Learning framework for explainable stance detection is state-of-the-art for social media stance prediction. |
| Approach: | They propose a Pruned interpretable knowledge Graph Learning framework for explainable stance detection that incorporates commonsense knowledge and prunes redundant information to ensure precision and minimize noise. |
| Outcome: | The proposed framework achieves state-of-the-art on three public datasets. |
Copied to clipboard
| Challenge: | Existing benchmarks that focus on knowledge-intensive tasks do not reflect diverse educational scenarios. |
| Approach: | They propose a benchmark that incorporates 9 major scenarios and 4,000 educational contexts. |
| Outcome: | The proposed model performs comparable to state-of-the-art large models on the test set. |
Copied to clipboard
| Challenge: | Existing approaches to integrate thoughts with actions can cause irreversible error propagation . Xi et al., 2023; Zhang eet coll., 2023) have focused on enhancing large language model (LLM) agents capable of helping humans tackle real-world challenges. |
| Approach: | They propose a framework called Generator-Assistant Stepwise Rollback to induce better decision-making for LLM agents by integrating a generator and an assistant to examine each action produced by the generator. |
| Outcome: | The proposed framework improves on three widely used benchmarks and can integrate seamlessly with other methods. |
Copied to clipboard
| Challenge: | Existing NL2SQL systems rely on in-context learning with only correct examples . current test-time scaling methods often decompose questions arbitrarily, resulting in poor performance . |
| Approach: | They propose a structured decomposition and experience-aware self-correction framework for NL2SQL . they build a dynamic memory of successful queries and historical error–fix pairs . |
| Outcome: | The proposed framework achieves 68.5% execution accuracy on BIRD, setting new state of the art among open, zero-fine-tuning methods. |
Copied to clipboard
| Challenge: | Event Extraction (EE) is a long-standing target, but lacks an efficient and effective annotation framework to construct the corresponding datasets. |
| Approach: | They propose an LLM-based collaborative annotation framework that refines annotations of triggers from distant supervision and carries out argument annotation. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on the largest EE dataset to date . it achieves the F1 scores of 90% and 85.3% on the human-annotated test set . |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) and Large Multimodal Models have exceeded general human capabilities in various tasks. |
| Approach: | They present an Olympiad-level bilingual multimodal scientific benchmark featuring 8,476 problems from Olympiad level mathematics and physics competitions. |
| Outcome: | The best performing model, GPT-4V, attains an average score of 17.97% on OlympiadBench, with a mere 10.74% in physics, highlighting the benchmark rigor and the intricacy of physical reasoning. |
Copied to clipboard
| Challenge: | Existing studies on word class flexibility have been fraught with difficulties in quantifying it accurately and at scale. |
| Approach: | They propose a method to quantify word class flexibility in 37 languages using contextualized word embeddings. |
| Outcome: | The proposed method builds on recent work in contextualized word embeddings to quantify semantic shift between word classes and uncovers shared tendencies in class flexibility across languages. |
Copied to clipboard
| Challenge: | Existing benchmarks focus on evaluating single-round interactions, neglecting other critical aspects. |
| Approach: | They propose a benchmark to evaluate full-duplex speech language models in multi-round settings . they segment continuous full-dual dialogues into discrete turns for evaluation . |
| Outcome: | The proposed benchmark compared full-duplex speech language models with full-dual speech models . the results show that the models perform better in multi-round settings than standard models compared to benchmarks . |
Copied to clipboard
| Challenge: | Currently, knowledge graphs are decoupled from their downstream application, resulting in suboptimal graph structures. |
| Approach: | They propose a framework to directly optimize KG construction for task performance using Reinforcement Learning (RL). |
| Outcome: | The proposed framework improves performance across multiple QA benchmarks and consistently achieves significant performance gains over task-agnostic baseline graphs. |
Copied to clipboard
| Challenge: | Distantly supervised relation extraction (RE) has attracted much attention in the past few years . previous methods to evaluate models manually or directly on autolabeled data have produced inaccurate evaluations . |
| Approach: | They propose to use distant supervision to generate large-scale autolabeled data . they build manually-annotated test sets for two DS-RE datasets and evaluate models . |
| Outcome: | The proposed method produces 53% wrong labels at the entity pair level in the popular NYT10 dataset. |
Copied to clipboard
| Challenge: | Goal-oriented script planning is used by humans to plan for typical activities . however, this capability remains underexplored due to several challenges . |
| Approach: | They propose a framework that enables product-enriched scripts by associating products with each step based on the semantic similarity between the actions and their purchase intentions. |
| Outcome: | The proposed framework can generate product-enriched scripts from 2.4 million scripts . human annotations are conducted to provide gold labels for a sampled subset . |
Copied to clipboard
| Challenge: | Existing research on reinforcement learning for LLMs under data scarcity has not been unified. |
| Approach: | They propose a top-up hierarchical framework built around three complementary perspectives: data-centric, training-centric and framework-centric. |
| Outcome: | The proposed framework provides a clear conceptual foundation for understanding the design space of data-efficient RL for large language models and to guide researchers working in this emerging area. |
Copied to clipboard
| Challenge: | Existing methods for acquiring large-scale intentions generate product-centric intentions without product images and incur high costs for scalability. |
| Approach: | They propose a multimodal framework that allows Large Vision-Language Models to infer purchase intentions from multimodal product metadata and prioritize human-centric ones. |
| Outcome: | The proposed framework shows that it is robust to different prompts and superior to previous methods. |
Copied to clipboard
| Challenge: | Existing approaches to classify aspects with aspect sentiment bias are hard to find . |
| Approach: | They propose a no-aspect differential sentiment framework for the ABSA task that eliminates aspect sentiment bias and uses differential sentiment loss instead of cross-entropy loss to better classify the sentiments. |
| Outcome: | The proposed framework can be combined with almost all traditional ABSA methods. |
Copied to clipboard
| Challenge: | Notable examples include OpenAI’s o1/o3/o4 series and DeepSeek-R1 . |
| Approach: | They develop a framework to identify suboptimal subtrajectories based on human-established criteria . they also use a sampling algorithm to select data whose reasoning process is free from suboptimally subtravertories to the highest degree . |
| Outcome: | The proposed method reduces the number of suboptimal subtrajectories by 25.9% during the inference process. |
Copied to clipboard
| Challenge: | Large language models excel at processing unstructured data, but integrating time series data with text remains a challenge. |
| Approach: | They propose a self-supervised multimodal framework that uses prompt-guided learning to unify heterogeneous data types. |
| Outcome: | The proposed framework outperforms state-of-the-art approaches on disease diagnosis tasks using real-world datasets. |
Copied to clipboard
| Challenge: | Existing approaches to generalize commonsense reasoning lack instantiated knowledge and require pre-built concept taxonomies and annotations. |
| Approach: | They propose a framework that iteratively performs contextualized conceptualization and instantiation over commonsense knowledge bases by instructing large language models to generate both types of knowledge with critic filtering. |
| Outcome: | Empirical results show that distilling CANDLE on student models provides benefits across three downstream tasks. |
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated impressive performance in various natural language processing tasks. |
| Approach: | They propose a benchmark for the evaluation of large language models in the IP domain . they also propose supervised multilingual large language model called MoZi . |
| Outcome: | The proposed model outperforms four well-known LLMs on the MoZIP benchmark . the most powerful ChatGPT does not reach the passing level . |
Copied to clipboard
| Challenge: | Existing methods for generating responses following a desired style are lacking of parallel data for training. |
| Approach: | They propose a KL loss and a style classifier to fine-tune response generation . they show that their model can significantly outperform state-of-the-art methods . |
| Outcome: | The proposed model outperforms state-of-the-art models in style consistency and contextual coherence with two public datasets. |
Copied to clipboard
| Challenge: | Existing evaluations of multimodal abductive reasoning are limited to static, single-agent tasks. |
| Approach: | They propose a multiagent evaluation suite that deconstructs the current evaluations of multimodal abductive reasoning in vision–language models. |
| Outcome: | The evaluation suite is based on two core components: DixitArena and DixitsBench. |
Copied to clipboard
| Challenge: | Oracle bone script (OBS) documents are the oldest continuously-used writing system in the world and are important for linguistic and historical research. |
| Approach: | They construct an information system for OBS to symbolize, serialize, and store OBS data at the character-level using efficient databases and retrieval modules. |
| Outcome: | The proposed system symbolizes, serializes, and stores OBS data at the character-level, based on efficient databases and retrieval modules. |
Copied to clipboard
| Challenge: | Existing models with no effective open category data during training are limited by the lack of effective open categories data during the training stage. |
| Approach: | They propose an approach to generate effective open category samples in the training stage and without requiring prior knowledge or external datasets. |
| Outcome: | The proposed approach generates effective synthetic open category samples in the training stage and without requiring any prior knowledge or external datasets. |
Copied to clipboard
| Challenge: | Syntax-controlled paraphrase generation aims to produce paraphrase conform to given syntactic patterns. |
| Approach: | They propose a model that captures parent-child and sibling relations and a syntax encoder to capture alignment relations. |
| Outcome: | The proposed model achieves state-of-the-art in terms of semantic and syntactic quality on two popular benchmark datasets. |