Papers by Xi Ye
Copied to clipboard
| Challenge: | Existing models are often used as black boxes to adapt to new domains, but there is no single recipe for making them work. |
| Approach: | They propose to use black box models to improve their performance on new domains by leveraging explanations of their behavior. |
| Outcome: | The proposed method improves model generalization performance on two tasks using explanations. |
Copied to clipboard
| Challenge: | Existing efforts to capture event argument interactions are limited by the argument role type information of contextual entities. |
| Approach: | They propose to capture event argument interactions as a Seq2Seq-like learning problem where a sentence with a specific event trigger is mapped to a sequence of event argument roles. |
| Outcome: | The proposed neural architecture generates argument roles by incorporating contextual entities’ argument role predictions, like a word-by-word text generation process, thereby distinguishing implicit argument distribution patterns within an event more accurately. |
Copied to clipboard
| Challenge: | Existing benchmarks for logical reasoning in large language models lack language naturalness or limited complexity. |
| Approach: | They propose to use first-order logic annotations to evaluate logical reasoning capabilities of large language models. |
| Outcome: | The proposed dataset evaluates the FOL reasoning ability of supervised fine-tuning on medium-sized language models. |
Copied to clipboard
| Challenge: | Existing benchmarks evaluate agents in simplified, idealized settings, relying on pre-packaged tool interfaces, overlooking critical steps, and assume inputs are clean and fully specified. |
| Approach: | They propose a framework that evaluates language agents in simplified, idealized settings . they show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 . |
| Outcome: | Experiments on 15 proprietary and open-source models show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 . |
Copied to clipboard
| Challenge: | MCTS methods retain only the single highest-reward trajectory, discarding comparative signals present in the many explored paths. |
| Approach: | They propose a framework that transforms supervision extraction into a synthesis procedure. |
| Outcome: | The proposed framework matches or exceeds baselines on 60K CRPS-synthesized examples on out-of-domain benchmarks. |
Copied to clipboard
| Challenge: | Existing multi-answer question answering systems struggle to retrieve and synthesize a large number of evidence passages. |
| Approach: | They propose a multi-answer question answering framework that generates a large set of passages and then processes each passage individually to generate an initial high-recall but noisy answer set. |
| Outcome: | The proposed framework outperforms baselines on the QAMPARI and RoMQA datasets, achieving an average F1 score improvement of 11.17%. |
Copied to clipboard
| Challenge: | Current methods rely on ranking losses to teach reward model to assess preferences, but they are susceptible to noise and ambiguous data, often failing to deeply understand human intentions. |
| Approach: | They propose a method that incorporates contrastive learning into the reward modeling process to enhance generalization and stabilize the reinforcement learning training process. |
| Outcome: | The proposed method enhances generalization of the reward model, stabilizes the reinforcement learning training process, and improves the final alignment with human preferences. |
Copied to clipboard
| Challenge: | Large language models (LLMs) have remarkable capabilities in learning from expla- nations in prompts, but there has been limited understanding of exactly how these explana- tions function or why they are effective. |
| Approach: | They propose a maximal marginal relevance-based exemplar selection approach to construct exemplar sets that are both relevant and comple- mentary. |
| Outcome: | The proposed model improves in- context learning performance across three tasks on multiple LLMs. |
Copied to clipboard
| Challenge: | Large language models generate "hallucinated" answers that are not factual . despite their widespread adoption, they can generate plausiblesounding but nonfactual information. |
| Approach: | They propose a framework that tunes large language models to self-ground claims and provide citations to retrieved documents. |
| Outcome: | The proposed framework generates superior grounded responses with more accurate citations compared to prompting-based approaches and post-hoc citing-based methods. |
Copied to clipboard
| Challenge: | Recent research in interpretability of neural models has yielded numerous token attribution techniques, but it is hard to evaluate whether these explanations are faithful. |
| Approach: | They propose to use pairwise attributions to connect outputs to high-level model behavior to examine how well different attribution techniques align with this assumption on realistic counterfactuals in the case of reading comprehension (RC). |
| Outcome: | The proposed methods are better suited to RC than token-level attributions across different RC settings, and the best performance comes from a modification that was proposed to an existing pairwise attribution method. |
Copied to clipboard
| Challenge: | Existing methods for sentence-level event detection depend on manual annotations or domain expertise to design sophisticated templates and rules. |
| Approach: | They propose a dialogue-based explanation paradigm to enhance sentence semantics for event detection. |
| Outcome: | The proposed method can be applied to two event detection datasets. |
Copied to clipboard
| Challenge: | a multimodal protein language model (LLM) integrates sequence, structure, and function into functional annotation. |
| Approach: | They propose a multimodal protein language model that synergistically aligns bimodal representations with the textual modality to advance protein functional annotation. |
| Outcome: | The proposed model synergizes bimodal representations with the textual modality to advance protein functional annotation. |
Copied to clipboard
| Challenge: | Existing methods for large-scale retrieval are trained with 0-1 hard labels that indicate whether a query is relevant to a document, ignoring rich information of the relevance degree. |
| Approach: | They propose to introduce label enhancement for the first time to characterize query-document relevance degree by embedding label distribution into contextual embeddables. |
| Outcome: | The proposed method significantly outperforms existing retrieval models and its counterparts equipped with two alternative methods on English and Chinese large-scale retrieval tasks. |
Copied to clipboard
| Challenge: | In-context examples can improve the performance of knowledge-rich tasks such as question answering by triggering a language model to surface information stored in its parametric knowledge. |
| Approach: | They propose to construct in-context example sets based on model's parametric knowledge by prompting models with 'unknown' examples. |
| Outcome: | The proposed model can perform better on in-context examples in three multi-answer question answering datasets, and prompting with ‘unknown’ examples decreases the performance. |
Copied to clipboard
| Challenge: | Recent work has identified retrieval heads as a subset of attention heads responsible for retrieving salient information in long-context language models. |
| Approach: | They introduce a retrieval head that uses attention scores to enhance retrieval from long context . they use QRRetriever to select the most relevant parts with the highest retrieval scores . |
| Outcome: | The proposed retrieval heads outperform other retrieval-based retrieval retrievers on BEIR benchmarks. |
Copied to clipboard
| Challenge: | Existing KBQA approaches struggle with generalization of unseen KB schema items . Rank-and-generate approach solves coverage issue with strong generalization . |
| Approach: | They propose a Rank-and-Generate approach for KBQA that uses a generation model to generalize to unseen KB schema items. |
| Outcome: | The proposed approach outperforms the prior state-of-the-art on GrailQA and WebQSP datasets. |
Copied to clipboard
| Challenge: | Explanation has long been a part of communication, where humans use language to elucidate each other and transmit information about mechanisms of events. |
| Approach: | They review the opportunities and challenges of explanations in the era of large language models and examine how they can be used to generate explanations. |
| Outcome: | The proposed methods are based on the models of large language models (LLMs) and their opaque nature. |
Copied to clipboard
| Challenge: | Document-level relation extraction requires inter-sentence reasoning capabilities to capture local and global contextual information for multiple relation facts. |
| Approach: | They propose to characterize the interaction between sentences and potential relation instances via a Graph Enhanced Dual Attention network (GEDA) . they also propose a simple yet effective regularizer based on the natural duality of the S2R and R2S attentions, whose weights are also supervised by the supporting evidence of relation instances during training. |
| Outcome: | The proposed model achieves competitive performance on an existing large-scale dataset while the predictions can be interpretable and easily observed. |
Copied to clipboard
| Challenge: | Pretrained language models exhibit impressive generalization capabilities, but behave unpredictably under certain domain shifts. |
| Approach: | They propose to incorporate attributions into a few-shot model predicting out-of-domain (OOD) performance task to find out if models agree with pathological heuristics that may indicate worse generalization capabilities. |
| Outcome: | The proposed model-based model-learning model can perform better on a few-shot example set, and incorporate feature attributions to improve it. |
Copied to clipboard
| Challenge: | Existing dialogue datasets contain lots of noise in their state annotations. |
| Approach: | They propose a framework to train robust dialogue state tracking models by combining pseudo and vanilla labels by a common weighting parameter. |
| Outcome: | The proposed framework achieves state-of-the-art accuracy of 80.10% on multiWOZ 2.4. |
Copied to clipboard
| Challenge: | Existing studies show that advanced LLMs produce text indistinguishable from human writing. |
| Approach: | They propose a benchmark to assess persona simulation across diverse contexts by decomposing the evaluation into six fundamental capabilities including opinion consistency, memory recall, logical reasoning, persona tone, and syntactic style. |
| Outcome: | The proposed model achieves moderate accuracy but falls short of the basic capabilities needed to simulate personas in real-world contexts. |
Copied to clipboard
| Challenge: | Existing methods for multimodal program synthesis combine noisy signals from the user with hard constraints on the program’s behavior. |
| Approach: | They propose an optimal neural synthesis approach where the goal is to find a program that satisfies user-provided constraints while also maximizing the program’s score with respect to a neural model. |
| Outcome: | The proposed approach outperforms prior state-of-the-art methods in terms of accuracy and efficiency and finds model-optimal programs more frequently. |
Copied to clipboard
| Challenge: | Existing datasets for regex generation from natural language are limited in complexity . Existing regex synthesis datasets are simple and the language used to describe them is not diverse . |
| Approach: | They propose a dataset for regex generation from natural language that generates regexes using a probabilistic grammar and pre-defined macros. |
| Outcome: | The proposed dataset is compared to existing datasets for regex generation from natural language . it generates the regexes using a probabilistic grammar with pre-defined macros observed from real-world StackOverflow posts. |
Copied to clipboard
| Challenge: | Recent work shows how to prompt large language models with explanations to obtain strong performance on textual reasoning tasks. |
| Approach: | They propose to optimize explanation-infused prompts in a blackbox fashion by using leave-one-out schemes and a two-stage framework. |
| Outcome: | The proposed method improves prompts over crowdworker annotations and naive search strategies. |
Copied to clipboard
| Challenge: | a new approach to training large language models (LLMs) overlooks task-specific characteristics in tool use, leading to performance bottlenecks. |
| Approach: | They propose a task-feature-based framework that mitigates the effects of suboptimal training data . they use a dataset to train large-scale LLMs and a reward mechanism tailored to error categories . |
| Outcome: | The proposed framework matches or surpasses open- and closed-source LLMs in tool-use performance using only 1,217 training data points. |
Copied to clipboard
| Challenge: | Existing text mining models are trained with 0-1 hard label that indicates whether an instance belongs to a class, ignoring rich information of the relevance degree. |
| Approach: | They propose a keyword-based method to automatically generate soft labels from hard labels . they exploit relevance between labels and instances to incorporate them into models . |
| Outcome: | The proposed method improves models under balanced and unbalanced conditions. |
Copied to clipboard
| Challenge: | Existing Knowledge Graph Construction (KGC) tasks rely on static information extraction with a closed set of pre-defined schemas. |
| Approach: | They propose a static knowledge Graph Construction task that extracts entity, relation, and event based on dynamically changing schema graph without retraining. |
| Outcome: | The proposed system outperforms existing methods but still has room for improvement . it can extract entity, relation, and event based on dynamically changing schema graph without re-training . |
Copied to clipboard
| Challenge: | Recent efforts to classify unstructured texts into specific types have been limited in practical scenarios. |
| Approach: | They propose to use Chinese text conversations and phone conversations to expand event detection to the scenarios involving informal and heterogeneous texts. |
| Outcome: | The proposed dataset is based on user reviews, text conversations, and phone conversations in a leading e-commerce platform for food service. |
Copied to clipboard
| Challenge: | Recent systems for converting natural language descriptions into regexes have achieved some success, but typically deal with short, formulaic text and can only produce simple regexe. |
| Approach: | They propose a framework for regex synthesis in a context where both natural language and examples are available. |
| Outcome: | The proposed framework achieves state-of-the-art on two prior datasets and a real-world dataset, which existing neural systems completely fail on. |
Copied to clipboard
| Challenge: | Existing studies on human-like behaviors in foundation models do not verify their faithfulness . a simple application of psychological tools cannot faithfully characterize all human-type behaviors . |
| Approach: | They propose a framework to characterize humanoid behaviors in foundation models . they argue that a simple application of psychological tools cannot faithfully characterize all human-like behaviors . |
| Outcome: | The proposed framework assesses the faithfulness of results based on reproducibility, internal consistency, and generalizability. |
Copied to clipboard
| Challenge: | Reinforcement Learning (RL) in real-world environments often suffers from ambiguous or incomplete supervision. |
| Approach: | They propose a framework that enhances value modeling for robust RL in LLM post-training by integrating auxiliary losses guided by entropy and perplexity from a frozen language model and variational information bottleneck. |
| Outcome: | The proposed framework outperforms baselines on multi-turn dialogue, math reasoning, and science QA with rule-based and model-based rewards. |
Copied to clipboard
| Challenge: | Currently, there are no efficient reinforcement learning (RL) frameworks specifically designed for tool use. |
| Approach: | They propose an automated environment construction pipeline that incorporates scenario decomposition, document generation, function integration, complexity scaling, and localized deployment to enable high-quality training environments without external tools. |
| Outcome: | The proposed framework significantly improves the models’ tool-use performance without degrading their general capabilities. |
Copied to clipboard
| Challenge: | Existing methods for decoding conditional text are slow to apply to large numbers of hypotheses. |
| Approach: | They propose a method that can efficiently encode lattices of generated outputs using Transformers. |
| Outcome: | The proposed method can extract high-quality hypotheses from lattices with minimal degradation error compared to naive reranking methods. |
Copied to clipboard
| Challenge: | Effective evaluation of multi-hop tool use is critical for analyzing the understanding, reasoning, and function-calling capabilities of large language models. |
| Approach: | They propose a dataset that provides rigorous evaluation of multi-hop tool use. |
| Outcome: | The proposed model achieves 49.04% accuracy across five model families. |
Copied to clipboard
| Challenge: | Existing studies on biases within specific domains, such as finance, remain limited. |
| Approach: | They propose a framework to detect, detect, analyze and mitigate financial biases in large language models. |
| Outcome: | The proposed framework reduces bias by 68% for the most biased model, according to key metrics. |
Copied to clipboard
| Challenge: | Existing methods for pretraining data mixing for large language models neglect significant inter-domain overlaps and commonalities, failing to control the global diversity of the constructed training dataset. |
| Approach: | They propose a sample-wise data mixture approach that performs global cross-domain sampling by systematically evaluating the quality and diversity of each sample. |
| Outcome: | The proposed method exceeds existing domain-based methods in multiple downstream tasks and perplexity assessments. |
Copied to clipboard
| Challenge: | Existing benchmarks lack social metadata and evaluation framework to meet this urgent evaluation needs. |
| Approach: | They propose a benchmark capable of evaluating HPA and three fact-checking tasks. |
| Outcome: | The proposed framework improves HPA and computational efficiency for RLM-driven systems. |
Copied to clipboard
| Challenge: | Recent studies have identified a vulnerability in large language models (LLMs) during customization. |
| Approach: | They propose an adaptive data curation approach that allows any text to be curated to enhance its effectiveness in counteracting harmful samples during customization. |
| Outcome: | The proposed approach reduces compromising effects and generates 100% safe responses. |
Copied to clipboard
| Challenge: | Existing benchmarks for Deep Research Agents (DRAs) treat report generation as a single-shot writing task. |
| Approach: | They propose an evaluation suite that establishes multi-turn report revision as a new axis. |
| Outcome: | The evaluation suite establishes multi-turn report revision as a new axis. |
Copied to clipboard
| Challenge: | Existing methods to predict performance of large language models are lacking . authors propose a size-dependent mutual information predictor for closed-book question answering accuracy . |
| Approach: | They propose a size-dependent mutual information predictor that integrates knowledge frequency, knowledge specificity, and model size to forecast closed-book question answering accuracy. |
| Outcome: | The proposed method outperforms baseline models and achieves R2 > 0.7 in predicting QA accuracy without additional training. |
Copied to clipboard
| Challenge: | Existing methods for large-scale query-document retrieval are expensive and require sparse handcrafted features. |
| Approach: | They propose a quadrupletBERT model for effective and efficient retrieval using pre-trained language models like BERT. |
| Outcome: | The proposed model improves retrieval phase and leverages distances between simple negative and hard negative instances to obtain better embeddings. |