Papers by Ye Feng
Copied to clipboard
| Challenge: | Existing top-k attention methods struggle to strike a balance between efficiency and accuracy. |
| Approach: | They propose a top-k attention approach that integrates low-overhead techniques into the Top-k Attention process to achieve 7.2 speedup compared to vanilla full attention. |
| Outcome: | The proposed approach achieves 7.2 speedup compared to current top-k attention methods while maintaining model accuracy. |
Copied to clipboard
| Challenge: | a number of tools are used to perform complex tasks, but the tool utilization process can cause errors. |
| Approach: | They propose a critique evaluation benchmark for tool learning that analyzes function-calling errors on tool evaluation benchmarks. |
| Outcome: | The proposed critique evaluation benchmark holds diverse tool-use errors with varying complexities, which better reflects real-world scenarios. |
Copied to clipboard
| Challenge: | Existing approaches to model the relations between domains and slots fail to address these issues and can be generalized to unseen domains. |
| Approach: | They propose a Dynamic Schema Graph Fusion Network which generates a dynamic schema graph to explicitly fuse prior slot-domain membership relations and dialogue-aware dynamic slot relations. |
| Outcome: | The proposed model outperforms existing methods on benchmark datasets showing that it can extract users' goals or intentions as dialogue states and keep them updated over the whole dialogue. |
Copied to clipboard
| Challenge: | Current studies focus on single-language or single-document tasks for news summarization . lack of a benchmark inhibits researchers from adequately studying this invaluable problem. |
| Approach: | They propose a novel task that unifies Multi-lingual, Cross-lingual and Multi-document Summarization into one task. |
| Outcome: | The proposed task encapsulates the real-world requirements all-in-one and is validated by extensive analysis. |
Copied to clipboard
| Challenge: | Existing methods to learn from unlabeled data are difficult for zero-shot text classification tasks. |
| Approach: | They propose a self-training based method to efficiently leverage unlabeled data. |
| Outcome: | The proposed method significantly outperforms existing methods in zero-shot text classification tasks on benchmarks and a real-world e-commerce dataset. |
Copied to clipboard
| Challenge: | Text-Centric Visual Question Answering (TEC-VQA) is a text-centric visual task understanding tool. |
| Approach: | They introduce a benchmark that features human expert annotations across 9 languages . they prioritize the text in question-answer pairs while disregarding visual text in images . |
| Outcome: | The proposed benchmarks prioritize the text in question-answer pairs while disregarding visual text in images. |
Copied to clipboard
| Challenge: | Existing approaches to enhance multilingual reasoning capabilities rely on costly multilingual training or employ prompting with external translation tools. |
| Approach: | They propose a training-free inference-time method to enhance multilingual reasoning capabilities via Representation Engineering without additional training data or tools. |
| Outcome: | The proposed method outperforms existing methods on four reasoning benchmarks in English and Thai and Swahili. |
Copied to clipboard
| Challenge: | Recent years, Chinese Spelling Check (CSC) has been greatly improved by designing task-specific pre-training methods or introducing auxiliary tasks. |
| Approach: | They propose to decompose Chinese Spelling Check into detection, reasoning, and searching subtasks and to train a module that is compatible with existing CSC models. |
| Outcome: | The proposed module can be trained for one model and benefit other models. |
Copied to clipboard
| Challenge: | Document-level relation extraction (RE) is more challenging than sentence RE as it often requires reasoning over multiple sentences. |
| Approach: | They propose a method to heuristically select evidence sentences for document-level relation extraction. |
| Outcome: | The proposed method can be easily combined with BiLSTM to achieve good performance on benchmark datasets even better than fancy graph neural network based methods. |
Copied to clipboard
| Challenge: | Large language models (LLMs) suffer from severe hallucination issues due to the knowledge misalignment between the pre-training stage and the supervised fine-tuning stage. |
| Approach: | They propose a training objective with an abstention mechanism that selectively rejects tokens that misalign with the desired knowledge distribution via a special [REJ] token. |
| Outcome: | The proposed model selectively rejects tokens that misalign with the desired knowledge distribution via a special [REJ] token. |
Copied to clipboard
| Challenge: | Large Vision-Language Models (LVLMs) have impressive multimodal abilities but remain prone to multilingual object hallucination. |
| Approach: | They propose a cross-lingual attention intervention method to mitigate multilingual object hallucination in LVLMs by aligning attention patterns. |
| Outcome: | The proposed method improves 13.56% (up to 30%) on the POPE and 21.75% on the hallucination subsets across languages. |
Copied to clipboard
| Challenge: | Existing approaches to decode large language models adopt a homogeneous architecture . autoregressive decoding is a bottleneck because tokens must be generated sequentially . |
| Approach: | They propose a framework that organizes heterogeneous position-specialized draft modules into a horizontal cascade. |
| Outcome: | The proposed framework outperforms the current state-of-the-art (EAGLE3) and achieves 3.72x acceleration over vanilla decoding. |
Copied to clipboard
| Challenge: | Existing methods for integrating spatial layouts with text have limitations . existing methods produce overly long text sequences or lack autoregressive traits of LLMs . |
| Approach: | They introduce Interleaving Layout and Text in a Large Language Model (LayTextLLM) they use OCR-derived text and spatial layouts to integrate with LLMs for document understanding . |
| Outcome: | The proposed model shows an increase in performance in KIE and VQA tasks. |
Copied to clipboard
| Challenge: | Existing versions of MultiWOZ 2.0 have been published, but there are still lots of noisy labels in the training set. |
| Approach: | They propose a framework to train dialogue state tracking models from noisy labels instead of improving annotation quality further by using auxiliary models. |
| Outcome: | The proposed framework improves the goal accuracy of DST models by 28.16% on MultiWOZ 2.0 and 8.41% on MultiWoz 2.4, compared to using only the vanilla noisy labels. |
Copied to clipboard
| Challenge: | Existing LLMs' abilities to detect evidence in long contexts are far inferior to humans. |
| Approach: | They propose a benchmark to assess LLMs' abilities in evidence and multi-step commonsense reasoning within a long context. |
| Outcome: | The proposed method improves the performance of LLMs in evidence detection and commonsense reasoning. |
Copied to clipboard
| Challenge: | Existing benchmark-based evaluations cannot accurately reflect the performance of real-world applications. |
| Approach: | They propose a reliable strategy for domains to choose more robust LLMs for real-world applications. |
| Outcome: | The proposed strategy addresses the challenges faced by domains to choose more robust LLMs for real-world applications. |
Copied to clipboard
| Challenge: | a new framework for population-based evolution of large language models is emerging . a population-driven evolution of LLMs is a key component of evolution, authors say . |
| Approach: | They propose a framework that allows for population-based evolution of large language models . they start with a population of parent LLMs and allow this population to evolve . |
| Outcome: | The proposed framework outperforms existing methods on 12 datasets. |
Copied to clipboard
| Challenge: | a benchmark is designed to evaluate the capability of Large Multimodal Models (LMMs) in converting complex, structured digital graphics into executable code. |
| Approach: | They propose a benchmark to evaluate the capability of Large Multimodal Models to convert digital graphics into executable code. |
| Outcome: | The proposed benchmark exposes the performance gap among leading LMMs . the benchmark features 1130 meticulously curated samples . |
Copied to clipboard
| Challenge: | Mixture of experts (MoE) is a popular technique to improve capacity of Large Language Models (LLMs) but memory-constrained devices are a major concern in edge AI training and serving. |
| Approach: | They propose a framework for efficient serving of MoE-based large language models with tunable memory budgets. |
| Outcome: | Experiments show that SwapMoE can reduce memory consumption while maintaining reasonable accuracy. |
Copied to clipboard
| Challenge: | Existing evaluations of large language models fail to reflect fine-grained capabilities . existing benchmarks are manually curated or domain-generic, limiting scalability and alignment with real use cases. |
| Approach: | They propose a framework that allows custom construction of benchmarks from large-scale scientific data to evaluate application-specific scientific capabilities in LLMs. |
| Outcome: | The proposed framework reveals fine-grained differences in scientific capabilities that standard benchmarks overlook . it allows custom construction of benchmarks from large-scale scientific data to evaluate application-specific capabilities in LLMs. |
Copied to clipboard
| Challenge: | Existing approaches to discourse parsing focus on studying the semantic and syntactic aspects of EDU pairs, but they do not address long span dependencies. |
| Approach: | They propose a new transition-based discourse parser that takes discourse cohesion into account by using memory networks. |
| Outcome: | The proposed method outperforms traditional features and improves performance on the RST discourse treebank. |
Copied to clipboard
| Challenge: | Existing studies have demonstrated that pre-trained LLMs are limited in certain domains, such as programming, mathematics, biomedical, or finance. |
| Approach: | They propose a new post-pretraining method with an expansion of Transformer blocks to tune the expanded blocks using only new corpus, efficiently and effectively improving the model’s knowledge while mitigating forgetting. |
| Outcome: | The proposed model outperforms existing models in programming and math and its instruction-following counterpart LLaMA Pro-8.3B in general tasks, programming, and mathematics. |
Copied to clipboard
| Challenge: | Existing studies on large language models focus on literal-level translation quality, such as adequacy and fluency. |
| Approach: | They propose a Culture-Aware Novel-Driven Parallel Dataset for Machine Translation and a multi-dimensional evaluation framework for assessing cultural translation quality. |
| Outcome: | The proposed model improves evaluation reliability in LLM-as-a-judge scenarios under culture-aware constraints. |
Copied to clipboard
| Challenge: | Existing large language models (LLMs) ignore this diversity by reasoning in a single dominant language. |
| Approach: | They propose a family of reasoning models that can adaptively reason in an advantageous language on a per-instance basis. |
| Outcome: | The proposed model can reason in a single dominant language on a per-instance basis. |
Copied to clipboard
| Challenge: | Existing approaches to enhance agent capabilities for Large Language Models treat all tokens equally . however, reasoning tokens versus boilerplate tokens differ in importance and learning complexity . recent research has focused on enhancing agent capabilities in large language models . |
| Approach: | They propose a Shuffle-Aware Discriminator (SHAD) for adaptive token discrimination . they propose SHAD method which adaptively emphasizes reasoning tokens during fine-tuning . |
| Outcome: | The proposed method improves performance over standard fine-tuning methods. |
Copied to clipboard
| Challenge: | Structured data has been central to corporate data strategies for decades . however, with the advancement of large language models (LLMs), there has been a significant shift towards the effective utilization of unstructured data. |
| Approach: | They propose an automatic evaluation data generation method to assess LLMs’ reasoning capabilities on structure-rich text. |
| Outcome: | The proposed method supports 8 structured languages and 29 tasks, generating data with adjustable complexity through controllable nesting and structural width. |
Copied to clipboard
| Challenge: | Existing approaches to generate training data with pre-trained language models have been found effective in various scenarios. |
| Approach: | They propose an unsupervised zero-shot learning method that generates a dataset from scratch and trains a tiny task model under supervision of the synthesized dataset. |
| Outcome: | The proposed method is annotated-free and efficient, but can provide useful insights from the perspective of data-free model-agnostic knowledge distillation and unreferenced text generation evaluation. |
Copied to clipboard
| Challenge: | Existing self-play approaches to developing general reasoning in language models rely on terminal game outcomes. |
| Approach: | They propose a game-based reasoning transfer model that addresses two barriers to reasoning transfer. |
| Outcome: | The proposed model improves mathematical reasoning, general reasoning, and code generation benchmarks. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown remarkable capabilities in environmental perception, reasoning-based decision-making, and simulating complex human behaviors, particularly in interactive role-playing contexts. |
| Approach: | They propose a framework to assess LLMs' proficiency in portraying advanced human behaviors through murder mystery games using eight intricately crafted scripts. |
| Outcome: | The framework evaluates LLMs' performance in portraying advanced human behaviors through murder mystery games. |
Copied to clipboard
| Challenge: | Existing fine-tuning approaches that focus on English-centric training corpora often introduce implicit cross-lingual alignment, overlooking the potential for more profound, latent-level cross-linguistic interactions. |
| Approach: | They propose a multilingual fine-tuning paradigm that explicitly establishes a cross-lingual connection mechanism at the latent level. |
| Outcome: | The proposed model outperforms vanilla SFT and offers a strong latent-level alternative to data-level augmentation methods. |
Copied to clipboard
| Challenge: | Existing methods to learn speech representations for end-to-end speech-totext translation (ST) neglect the representation discrepancy across modalities. |
| Approach: | They propose a method to calibrate the representation discrepancy between modalities by mixing up the representation sequences of different modality inputs. |
| Outcome: | The proposed method alleviates the cross-modal representation discrepancy and improves on a strong baseline on eight translation directions. |
Copied to clipboard
| Challenge: | Due to the vast amounts of data and computational resources required for model development, protecting the model’s parameters and training data has become an urgent and crucial concern. |
| Approach: | They define "reverse engineering" techniques as attacks on large language models and provide an in-depth analysis of them. |
| Outcome: | The proposed attacks are described as “reverse engineering” techniques on LMs and provide an introduction to existing protective strategies. |
Copied to clipboard
| Challenge: | Existing studies have focused on instance-level unlearning, specifically removing predefined instances containing sensitive content. |
| Approach: | They propose a task to erase entity-related knowledge from the target model completely by analyzing the forget set and its size. |
| Outcome: | The proposed task systematically evaluates popular unlearning algorithms and reveals that the knowledge coverage of the forget set and its size play pivotal roles. |
Copied to clipboard
| Challenge: | Document-level relation extraction is a challenging task as it requires reasoning across multiple sentences. |
| Approach: | They propose to use a recommend-revise scheme to reduce the workload of annotators by providing them with candidate relation instances from distant supervision to supplement and remove relational facts. |
| Outcome: | The proposed dataset is the first large-scale and human-annotated dataset for relation extraction. |
Copied to clipboard
| Challenge: | Existing approaches to improve social intelligence of AI systems employ retrospective attributions and lack theoretical grounding. |
| Approach: | They propose a framework that uses Shapley values to ensure fair credit distribution with axiomatic guarantees of efficiency, symmetry, and marginality. |
| Outcome: | The proposed framework matches or exceeds proprietary models including GPT-4o and Claude-3.5-Sonnet. |
Copied to clipboard
| Challenge: | In-context Learning (ICL) is a new paradigm for large language model evaluation. |
| Approach: | They propose an open-source toolkit for ICL and LLM evaluation. |
| Outcome: | The proposed framework is highly flexible and flexible and can be easily combined with other tools to suit users' needs. |
Copied to clipboard
| Challenge: | Existing frameworks for retrieval-augmented large language models (LLMs) are lacking in LFQA faithfulness testing. |
| Approach: | They propose a framework to teach retrieval-augmented large language models to explicitly discriminate between faithful and unfaithful generations. |
| Outcome: | The proposed framework outperforms GPT-4o in LFQA scenarios and outperformed existing benchmarks. |
Copied to clipboard
| Challenge: | Existing methods for open attribute value extraction for emerging entities are noisy or incomplete, even missing. |
| Approach: | They propose a knowledge-guided reinforcement learning framework for open attribute value extraction for emerging entities. |
| Outcome: | The proposed framework outperforms baselines by 16.5 - 27.8%. |
Copied to clipboard
| Challenge: | Recent work on dataset-generation-based zero-shot learning has shown promising results by training a task-specific model with a dataset synthesized from large pre-trained language models (PLMs). |
| Approach: | They propose a progressive zero-shot dataset generation framework which leverages feedback from the task-specific model to guide the generation of new training data via in-context examples. |
| Outcome: | The proposed framework achieves on-par or superior performance with only 1% synthetic dataset size, when compared to baseline methods without in-context feedback. |
Copied to clipboard
| Challenge: | Existing relevance ranking methods focus on text modality, incapable of fully exploiting cross-modal cues present in video. |
| Approach: | They propose a QUery-Aware pre-training model with multi-modality that integrates video tag information as alignment targets and enhances ranking optimization method based on ordinal regression. |
| Outcome: | The proposed model significantly improves video search performance. |