Papers by Wei Yin
Copied to clipboard
| Challenge: | Existing evaluation benchmarks for ORMs are largely text-centric or limited to bimodal tasks . a new study examines the effectiveness of Omni-RewardBench for ORms across modalities . |
| Approach: | They propose a hybrid automatic-annotation and human-verification pipeline to construct high-quality evaluation data. |
| Outcome: | The proposed model is the first benchmark for comprehensive evaluation of ORMs across modalities. |
Copied to clipboard
| Challenge: | a comparative analysis of paper (meta-)reviews by large language models (LLMs) aims to identify and distinguish LLMs from human activities . |
| Approach: | They present a comparative analysis to identify and distinguish LLM activities from human activities. |
| Outcome: | The proposed analysis aims to improve recognition of instances when someone implicitly uses LLMs for reviewing activities. |
Copied to clipboard
| Challenge: | Lowrank adaptation and its variants introduce significant latency in multi-tenant settings, hindering their applications in the industry. |
| Approach: | They propose a framework to fine-tune LoRA modules on a large-scale instruction tuning dataset. |
| Outcome: | The proposed framework outperforms existing PEFT methods and significantly reduces inference latency. |
Copied to clipboard
| Challenge: | Existing approaches to reinforcement learning from human feedback (RLHF) require expensive human-annotated datasets and proprietary models like GPT-4 to annotate preference pairs. |
| Approach: | They propose a self-synthetic framework for LLM alignment where all training data, including prompts (i.e., user queries), responses, and preferences, are generated by the model itself. |
| Outcome: | The proposed framework enhances the model’s chat capabilities on standard benchmarks like AlpacaEval 2.0 while maintaining strong performance on downstream objective tasks. |
Copied to clipboard
| Challenge: | Named entity recognition (NER) suffers from the scarcity of annotated training data, especially for low-resource languages without labeled data. |
| Approach: | They propose a cross-lingual entity projection framework to enable zero-shot cross-linguistic NER with the help of a multilingual labeled sequence translation model. |
| Outcome: | The proposed method outperforms the baseline method on two benchmarks by a large margin of +3 7 F1 scores and achieves state-of-the-art performance. |
Copied to clipboard
| Challenge: | Existing domain-adaptive pre-training (DAPT) models tend to forget the general knowledge acquired by general PLMs, leading to catastrophic forgetting and sub-optimal performance. |
| Approach: | They propose a framework which augments the domain-specific PLM by a memory built from the frozen general PLM without losing the general knowledge. |
| Outcome: | The proposed framework augments the domain-specific PLM by a memory built from the frozen general PLM without losing the general knowledge. |
Copied to clipboard
| Challenge: | Evidence retrieval is used to enhance Large Language Models (LLMs) but in real-world applications, it often returns lengthy documents with redundant or irrelevant content, confusing downstream readers. |
| Approach: | They propose a framework that reformulates evidence retrieval as a dynamic tree expansion process. |
| Outcome: | The proposed framework outperforms existing methods on five datasets. |
Copied to clipboard
| Challenge: | Empirical results show that iterAlign improves truthfulness, helpfulness, harmlessness and honesty, improving the LLM alignment by up to 13.5% in harmlessness. |
| Approach: | They propose a data-driven constitution discovery and self-alignment framework called IterAlign to overcome these drawbacks by leveraging red teaming to uncover weaknesses of an LLM. |
| Outcome: | Empirical results show that iterAlign improves truthfulness, helpfulness, harmlessness and honesty by up to 13.5%. |
Copied to clipboard
| Challenge: | achieving data-efficient post-training of Large Language Models is a key research question. |
| Approach: | They propose a taxonomy of data-efficient LLM post-training methods from a data-centric perspective. |
| Outcome: | The proposed methods cover data selection, data quality enhancement, synthetic data generation, data distillation and compression, and self-evolving data ecosystems. |
Copied to clipboard
| Challenge: | Existing models that only use auxiliary languages to encourage multilingual agreement ignore the relationships between different language pairs. |
| Approach: | They propose a multilingual agreement-based method which explicitly models the agreement between different translation directions by randomly substituting some fragments of the source language with their counterpart translations of auxiliary languages. |
| Outcome: | The proposed method improves on the multilingual translation task of 10 language pairs. |
Copied to clipboard
| Challenge: | Existing methods for learning from noisy labels are difficult to improve . existing methods identify noisy labels and use active learning to query experts . |
| Approach: | They propose a collaborative learning framework to combine LLMs and small models for learning from noisy labels. |
| Outcome: | The proposed framework outperforms state-of-the-art baselines on synthetic and real-world noise datasets. |
Copied to clipboard
| Challenge: | Existing platforms lack a mechanism for user actions to dynamically reshape the environment. |
| Approach: | They propose a novel agent-based simulation platform for recommender systems with a robust interaction mechanism. |
| Outcome: | The proposed platform improves the credibility of the simulation and replicates the Matthew Effect and Brand Loyalty. |
Copied to clipboard
| Challenge: | Existing methods to predict missing facts in knowledge graphs are limited in language alignment . SS-AGA uses seed alignment as an edge type to fuses all KGs as a whole graph . |
| Approach: | They propose a self-supervised adaptive graph alignment method that fuses all KGs as a whole graph by regarding alignment as 'a new edge type' they propose SS-AGA method that uses relation-aware attention weights to capture potential alignment pairs in a new paradigm. |
| Outcome: | The proposed method can predict missing facts in a knowledge graph (KG) but language alignment is scarce and new alignment identification is noisy. |
Copied to clipboard
| Challenge: | Existing work on the Mutual Reinforcement Effect in information extraction has not been empirically validated . 76 percent of the 21 sub-datasets exhibit the Mutual Reforcement effect across languages . |
| Approach: | They propose a multilingual MRE mix dataset that integrates 21 sub-datasets covering English, Japanese, and Chinese. |
| Outcome: | The proposed framework reduces manual annotation effort while preserving structural requirements of MRE tasks. |
Copied to clipboard
| Challenge: | Existing methods to generate valid and fluent questions from text are limited and insufficient for training. |
| Approach: | They propose to generate multi-hop reasoning questions from the raw text in a low resource circumstance by deducing over multiple relations on several sentences in the text. |
| Outcome: | The proposed model can be applied to the task of machine reading comprehension and achieve significant performance improvements. |
Copied to clipboard
| Challenge: | Existing pre-training methods underutilize the benefits of language understanding for generation. |
| Approach: | They propose a GAN-style model for encoder-decoder pre-training with an auxiliary discriminator. |
| Outcome: | The proposed model outperforms existing pre-trained models and achieves state-of-the-art performance. |
Copied to clipboard
| Challenge: | Large language models (LLMs) exhibit remarkable capabilities across many tasks, but face critical challenges in the CSC scenario: (1) poor generalization to rare entities in open-domain searches; and (2) failure to adapt to temporal entity variations due to static parameters, resulting in serious over-correction issues. |
| Approach: | They propose a Chinese Spelling Check system with RAG and multi-task learning that integrates dynamic knowledge retrieval and entity-centric RAG to address rare entities. |
| Outcome: | The proposed system outperforms existing baselines in the CSC task and achieves a maximum improvement of +9.92% on the search scenario benchmark and +3.2% on the general-domain dataset. |
Copied to clipboard
| Challenge: | Embodied planning requires agents to make coherent multi-step decisions based on dynamic visual observations and verbal goals. |
| Approach: | They propose an On-policy Reinforcement fine-tuning framework with offline rewards for Embodied Task Planning that preserves generalization benefits of RFT while addressing costly interaction and sparse rewards. |
| Outcome: | The proposed framework outperforms closed-source and online-RL methods on EmbodiedBench, a recent benchmark for interactive embodied tasks. |
Copied to clipboard
| Challenge: | Existing approaches to managing non-linear dialogue flow are misaligned with the intrinsically hierarchical and branching structure of natural discourse. |
| Approach: | They propose a framework that models multi-turn dialogue history as a dynamic tree structure. |
| Outcome: | The proposed framework enhances task completion rates and improves token efficiency across various LLMs. |
Copied to clipboard
| Challenge: | Existing studies focus on pre-trained LLMs to better understand and improve their trustworthiness. |
| Approach: | They apply linear probing to LLMs to explore five key dimensions of trustworthiness: reliability, privacy, toxicity, fairness, and robustness. |
| Outcome: | The proposed model can distinguish concepts in each trustworthiness dimension, suggesting that it can be trained in early pre-training. |
Copied to clipboard
| Challenge: | Recent advances in large language models have been remarkable . users face a choice between using cloud-based LLMs for generation quality or local-based ones for lower computational cost . |
| Approach: | They propose a new LLM utilization paradigm that facilitates collaborative operation . they evaluate AdaSwitch across 7 benchmarks and compare it to other LLMs . |
| Outcome: | The proposed model improves performance of local and cloud agents across 7 benchmarks . it achieves competitive results compared to the cloud agent while utilizing less computational overhead. |
Copied to clipboard
| Challenge: | Existing fake news detection models are opaque and lack deductive transparency . a framework for dialectical structured reasoning is proposed to address this limitation . |
| Approach: | They propose a framework that model fake news detection as an explicit dialectical process over multimodal social context. |
| Outcome: | The proposed framework achieves state-of-the-art while producing transparent explanations that mirror human reasoning process. |
Copied to clipboard
| Challenge: | Existing methods for enhancing Large Language Models (LLMs) struggle with novelty and Reinforcement Learning from human feedback (RLHF) is costly. |
| Approach: | They propose to use a Reward Model (RM) and a principle-guided LLM-as-a-Judge to enhance creative output over baselines. |
| Outcome: | The proposed approach significantly enhances creative output over baselines, but the principle-guided LLM-as-a-Judge yields superior generation quality. |
Copied to clipboard
| Challenge: | Recent advances in large language models have significantly improved automated code generation . however, the translation of complex mobile UI designs into high-fidelity front-end code remains a challenge . |
| Approach: | They propose a collaborative multi-agent system to reconstruct static single-page apps from mockups. |
| Outcome: | The proposed system outperforms existing methods in reconstructing complex app pages . the code and data will be released upon paper acceptance . |
Copied to clipboard
| Challenge: | Existing frameworks rely on static or rule-based topologies that fail to adapt to task requirements. |
| Approach: | They propose a generative framework that generates highly task-adaptive topologies . they validated the framework on multiple benchmarks and validated it on multiple platforms . |
| Outcome: | The proposed framework outperforms existing frameworks in task-adaptive communication topologies. |
Copied to clipboard
| Challenge: | Existing methods for commonsense question generation produce shallow questions that can be answered by simple word matching. |
| Approach: | They propose a task of commonsense question generation that aims to yield deep-level questions from the text. |
| Outcome: | The proposed model can yield deep-level and to-the-point questions from the text. |
Copied to clipboard
| Challenge: | Existing approaches to planning involve implicit planning or introduce explicit planners without systematically optimizing the planning stage. |
| Approach: | They propose an end-to-end RL framework that enhances the planning capabilities of deep research agents. |
| Outcome: | Experiments show that DeepPlanner improves planning quality and achieves state-of-the-art results under a lower training budget. |
Copied to clipboard
| Challenge: | Large language models (LLMs) often produce factually incorrect information, also known as hallucination. |
| Approach: | They propose a framework for verifiable text generation with evolving memory and self-reflection that incorporates long-term memory to retain documents and recent documents. |
| Outcome: | The proposed framework outperforms baselines on five datasets across three knowledge-intensive tasks. |
Copied to clipboard
| Challenge: | Existing work on reinforcement learning has focused on single-turn tasks such as solving math problems. |
| Approach: | They propose a framework that learns directly from online interactions by asynchronously generating diverse trajectories, guided by binary rewards depending on task success. |
| Outcome: | Experiments on the WebArena-Lite benchmark show that the framework outperforms state-of-the-art methods and strong proprietary models. |
Copied to clipboard
| Challenge: | Prior work limits search depth to reduce cost, but this often leads to underexploration of complex questions. |
| Approach: | They propose a reinforcement learning framework that evaluates each search step via self-generated intermediate answers. |
| Outcome: | Extensive experiments on multiple benchmarks show that AutoSearch achieves a superior accuracy-efficiency trade-off, alleviating over-searching while preserving search quality. |
Copied to clipboard
| Challenge: | Multiple-choice question datasets like Massive Multitask Language Understanding (MMLU) have inevitably led to benchmark contamination, resulting in unreliable evaluation. |
| Approach: | They propose a contamination-free MCQ benchmark called MMLU-CF which reassesses LLMs’ understanding of world knowledge by averting both unintentional and malicious data contamination. |
| Outcome: | The proposed MMLU-CF reassesses LLMs’ understanding of world knowledge by averting both unintentional and malicious data contamination. |
Copied to clipboard
| Challenge: | Sequence labeling aims to predict fine-grained sequences of labels for text, but lack of token-level annotated data hinders the effectiveness of supervised methods. |
| Approach: | They propose a Meta Teacher-Student (MetaTS) Network to alleviate data scarcity by leveraging large multilingual unlabeled data. |
| Outcome: | The proposed meta learning method alleviates data scarcity by leveraging large multilingual unlabeled data. |
Copied to clipboard
| Challenge: | Existing definitions of streaming LLMs are fragmented and lack a systematic taxonomy . large language models are pre-trained on static and full-context corpora . |
| Approach: | They propose a systematic taxonomy of current streaming Large Language Models and propose underlying methodologies for streaming LLMs. |
| Outcome: | The proposed model is based on data flow and dynamic interaction to clarify existing ambiguities. |
Copied to clipboard
| Challenge: | Existing research on information extraction tasks focuses on one specific task, but in real-world scenarios, new data of different IE tasks and domains come in a stream over time. |
| Approach: | They propose a parameter- and deployment-efficient prompt tuning method to evaluate the UIE system under a “lifelong learning” setting. |
| Outcome: | The proposed method is able to learn new tasks without forgetting old ones and expand knowledge and functionalities without retraining the whole system. |
Copied to clipboard
| Challenge: | Existing methods focus on point-wise memory, losing durative information that captures persistent states and evolving patterns. |
| Approach: | They propose a memory framework that models semantic time for point-wise memory and supports the construction and utilization of durative memory. |
| Outcome: | Experiments on LongMemEval and LoCoMo show that the proposed method outperforms existing methods and achieves up to 12.2% improvement in accuracy. |
Copied to clipboard
| Challenge: | Transformer-based pre-training models like BERT are computationally expensive and limited to resource-constrained devices. |
| Approach: | They propose a method which ternarizes the weights in a fine-tuned BERT model. |
| Outcome: | The proposed method outperforms the other methods on the GLUE and SQUAD benchmarks while being 14.9x smaller. |
Copied to clipboard
| Challenge: | Recent legislation of the "right to be forgotten" has led to the interest in machine unlearning . MU can be used to forget specific training instances as if they have never existed . |
| Approach: | They propose a general unlearning framework called KGA to induce forgetfulness . they propose several unlearning evaluation metrics with pertinence . |
| Outcome: | The proposed framework improves on large-scale datasets and provides insight into unlearning for NLP tasks. |
Copied to clipboard
| Challenge: | Existing work on document-level ASR error correction ignores contextual information . however, there are limited studies on incorporating contextual information into AEC . |
| Approach: | They propose a context-aware method that retrieves contextual information from a datastore . they use two English and two Chinese datasets to model document-level AEC . |
| Outcome: | The proposed model can utilize contextual information to improve document-level AEC . the data store containing contextual information provides even better results . |
Copied to clipboard
| Challenge: | Existing text generation models follow the sequence-to-sequence paradigm . generative grammar suggests humans generate language by learning language grammar . |
| Approach: | They propose a syntax-guided generation schema that searches the syntax tree in a top-down direction. |
| Outcome: | The proposed method outperforms autoregressive baselines on paraphrase generation and machine translation. |
Copied to clipboard
| Challenge: | Existing ensemble methods for ensembling large language models rely on fixed weighting strategies that fail to adapt to dynamic, context-dependent characteristics of LLMs. |
| Approach: | They propose a framework that reformulates LLM ensemble through a Markov Decision Process. |
| Outcome: | The proposed framework outperforms existing methods by 3.3% on a diverse set of tasks while achieving lower time latency. |
Copied to clipboard
| Challenge: | Existing methods for prompt tuning require many soft tokens to guarantee performance . large language models still require a large amount of GPU memory and computations to fine-tune . |
| Approach: | They propose to use a parameter-efficient soft prompt generator to generate idiosyncratic soft prompts for each input instruction. |
| Outcome: | The proposed method outperforms the baselines with comparable tunable parameters and is more efficient than LoRA under the single-backbone multi-tenant setting. |
Copied to clipboard
| Challenge: | Existing methods for event extraction neglect grammatical incorrectness, structure misalignment, and semantic drifting . et al., 2004; Ahn, 2006) show that the proposed method generates more diverse text representations for event extracting compared with the state-of-the-art. |
| Approach: | They propose a framework for event extraction that generates additional training data and iteratively selects the effective subset from the generated training data. |
| Outcome: | The proposed method generates more diverse representations of training data and achieves comparable results with the state-of-the-art. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) and Multimodal LLMs (MLLMs) show strong performance in complex reasoning tasks, but their ability to extract symbolic laws from time series data remains underexplored. |
| Approach: | They propose a benchmark to assess symbolic reasoning over real-world time series across three tasks: multivariate symbolic regression, Boolean network inference, and causal discovery. |
| Outcome: | The proposed framework integrates LLMs with genetic programming to form a closed-loop symbolic reasoning system. |
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated extraordinary capabilities in natural language understanding, generation, and reasoning. |
| Approach: | They propose a plug-and-play LLM model that embeds a user-specific embedding for each individual by modeling her historical contexts through a lightweight plug-in user embedder module. |
| Outcome: | Experiments on various tasks in the language model personalization (LaMP) benchmark show that the proposed model significantly outperforms existing personalized LLM approaches. |
Copied to clipboard
| Challenge: | a new generation of (M)LLMs is enabling the creation of superintelligent AI assistants . OS Agents can complete tasks autonomously and have the potential to significantly enhance the lives of billions of users worldwide. |
| Approach: | They propose to build OS Agents that operate within operating systems' GUIs and GUIs . they examine evaluation metrics and benchmarks to identify promising directions . |
| Outcome: | The proposed agents are based on operating systems (OS) and operating systems frameworks. |
Copied to clipboard
| Challenge: | Large Language Model (LLM)-based optimization has shown promise for autonomous problem solving, but most approaches cast LLMs as passive constraint checkers rather than proactive strategy designers. |
| Approach: | They propose an end-to-end Automated Constraint Optimization method that tightly couples operations-research principles of constraint relaxation with LLM reasoning. |
| Outcome: | Extensive experiments on three challenging COP benchmarks validate AutoCO’s consistent effectiveness and superior performance, especially in hard regimes where current methods degrade. |
Copied to clipboard
| Challenge: | Existing models fail to capture and model customer intention effectively because of insufficient information exploitation and only apparent information like descriptions and titles are used. |
| Approach: | They propose to exploit existing session data to capture and model intention in E-commerce product purchase sessions using a multimodal benchmark. |
| Outcome: | The proposed framework can bridge the gap between intention understanding in simplified research cases like co-buy intention and more complex yet practical scenarios like session history. |
Copied to clipboard
| Challenge: | Existing vision-language models lack fine-grained classification, single-view imagery, and inaccurate metadata. |
| Approach: | They propose a hierarchical, multi-view benchmark to evaluate VLMs across three levels of cognitive complexity. |
| Outcome: | The proposed benchmark evaluates vision-language models across three levels of complexity . it systematically identifies five primary failure modes . the proposed benchmarks are available on https://github.com/meituan/DiningBench. |
Copied to clipboard
| Challenge: | Existing mPLM-based methods focus on designing costly model pre-training while ignoring equally crucial downstream adaptation. |
| Approach: | They propose a meta graph learning method that extracts meta-knowledge from historical CLT experiences to learn to cross-lingual transfer. |
| Outcome: | The proposed method can learn to cross-lingual transfer by extracting meta-knowledge from historical CLT experiences (tasks) it can also capture intrinsic language relationships to explicitly guide cross-linguistic transfer. |
Copied to clipboard
| Challenge: | Recent research has highlighted a significant inefficiency associated with the slow thinking paradigm . models often overthink simple tasks while underthinking complex challenges . |
| Approach: | They propose a framework for adaptive reasoning preference control that dynamically adjusts reflection preferences based on group-level distributional statistics of reasoning length and reflection intensity. |
| Outcome: | The proposed framework reduces average response length by 17.1%-21.4% while improving performance by 6.12-6.59 points under 32K token budgets. |