Papers by Ye Feng

40 papers
HATA: Trainable and Hardware-Efficient Hash-Aware Top-k Attention for Scalable Large Model Inference (2025.findings-acl)

Copied to clipboard

Challenge: Existing top-k attention methods struggle to strike a balance between efficiency and accuracy.
Approach: They propose a top-k attention approach that integrates low-overhead techniques into the Top-k Attention process to achieve 7.2 speedup compared to vanilla full attention.
Outcome: The proposed approach achieves 7.2 speedup compared to current top-k attention methods while maintaining model accuracy.
CRITICTOOL: Evaluating Self-Critique Capabilities of Large Language Models in Tool-Calling Error Scenarios (2025.emnlp-main)

Copied to clipboard

Challenge: a number of tools are used to perform complex tasks, but the tool utilization process can cause errors.
Approach: They propose a critique evaluation benchmark for tool learning that analyzes function-calling errors on tool evaluation benchmarks.
Outcome: The proposed critique evaluation benchmark holds diverse tool-use errors with varying complexities, which better reflects real-world scenarios.
Dynamic Schema Graph Fusion Network for Multi-Domain Dialogue State Tracking (2022.acl-long)

Copied to clipboard

Challenge: Existing approaches to model the relations between domains and slots fail to address these issues and can be generalized to unseen domains.
Approach: They propose a Dynamic Schema Graph Fusion Network which generates a dynamic schema graph to explicitly fuse prior slot-domain membership relations and dialogue-aware dynamic slot relations.
Outcome: The proposed model outperforms existing methods on benchmark datasets showing that it can extract users' goals or intentions as dialogue states and keep them updated over the whole dialogue.
GlobeSumm: A Challenging Benchmark Towards Unifying Multi-lingual, Cross-lingual and Multi-document News Summarization (2024.emnlp-main)

Copied to clipboard

Challenge: Current studies focus on single-language or single-document tasks for news summarization . lack of a benchmark inhibits researchers from adequately studying this invaluable problem.
Approach: They propose a novel task that unifies Multi-lingual, Cross-lingual and Multi-document Summarization into one task.
Outcome: The proposed task encapsulates the real-world requirements all-in-one and is validated by extensive analysis.
Zero-shot Text Classification via Reinforced Self-training (2020.acl-main)

Copied to clipboard

Challenge: Existing methods to learn from unlabeled data are difficult for zero-shot text classification tasks.
Approach: They propose a self-training based method to efficiently leverage unlabeled data.
Outcome: The proposed method significantly outperforms existing methods in zero-shot text classification tasks on benchmarks and a real-world e-commerce dataset.
MTVQA: Benchmarking Multilingual Text-Centric Visual Question Answering (2025.findings-acl)

Copied to clipboard

Challenge: Text-Centric Visual Question Answering (TEC-VQA) is a text-centric visual task understanding tool.
Approach: They introduce a benchmark that features human expert annotations across 9 languages . they prioritize the text in question-answer pairs while disregarding visual text in images .
Outcome: The proposed benchmarks prioritize the text in question-answer pairs while disregarding visual text in images.
Unlocking Multilingual Reasoning Capability of LLMs and LVLMs through Representation Engineering (2026.acl-long)

Copied to clipboard

Challenge: Existing approaches to enhance multilingual reasoning capabilities rely on costly multilingual training or employ prompting with external translation tools.
Approach: They propose a training-free inference-time method to enhance multilingual reasoning capabilities via Representation Engineering without additional training data or tools.
Outcome: The proposed method outperforms existing methods on four reasoning benchmarks in English and Thai and Swahili.
A Frustratingly Easy Plug-and-Play Detection-and-Reasoning Module for Chinese Spelling Check (2023.findings-emnlp)

Copied to clipboard

Challenge: Recent years, Chinese Spelling Check (CSC) has been greatly improved by designing task-specific pre-training methods or introducing auxiliary tasks.
Approach: They propose to decompose Chinese Spelling Check into detection, reasoning, and searching subtasks and to train a module that is compatible with existing CSC models.
Outcome: The proposed module can be trained for one model and benefit other models.
Three Sentences Are All You Need: Local Path Enhanced Document Relation Extraction (2021.acl-short)

Copied to clipboard

Challenge: Document-level relation extraction (RE) is more challenging than sentence RE as it often requires reasoning over multiple sentences.
Approach: They propose a method to heuristically select evidence sentences for document-level relation extraction.
Outcome: The proposed method can be easily combined with BiLSTM to achieve good performance on benchmark datasets even better than fancy graph neural network based methods.
Alleviating Hallucinations from Knowledge Misalignment in Large Language Models via Selective Abstention Learning (2025.acl-long)

Copied to clipboard

Challenge: Large language models (LLMs) suffer from severe hallucination issues due to the knowledge misalignment between the pre-training stage and the supervised fine-tuning stage.
Approach: They propose a training objective with an abstention mechanism that selectively rejects tokens that misalign with the desired knowledge distribution via a special [REJ] token.
Outcome: The proposed model selectively rejects tokens that misalign with the desired knowledge distribution via a special [REJ] token.
CLAIM: Mitigating Multilingual Object Hallucination in Large Vision-Language Models with Cross-Lingual Attention Intervention (2025.acl-long)

Copied to clipboard

Challenge: Large Vision-Language Models (LVLMs) have impressive multimodal abilities but remain prone to multilingual object hallucination.
Approach: They propose a cross-lingual attention intervention method to mitigate multilingual object hallucination in LVLMs by aligning attention patterns.
Outcome: The proposed method improves 13.56% (up to 30%) on the POPE and 21.75% on the hallucination subsets across languages.
HCSpec: Two-Tier Horizontal Cascade Speculative Decoding for High-Efficiency Large Language Model Inference (2026.acl-long)

Copied to clipboard

Challenge: Existing approaches to decode large language models adopt a homogeneous architecture . autoregressive decoding is a bottleneck because tokens must be generated sequentially .
Approach: They propose a framework that organizes heterogeneous position-specialized draft modules into a horizontal cascade.
Outcome: The proposed framework outperforms the current state-of-the-art (EAGLE3) and achieves 3.72x acceleration over vanilla decoding.
A Bounding Box is Worth One Token - Interleaving Layout and Text in a Large Language Model for Document Understanding (2025.findings-acl)

Copied to clipboard

Challenge: Existing methods for integrating spatial layouts with text have limitations . existing methods produce overly long text sequences or lack autoregressive traits of LLMs .
Approach: They introduce Interleaving Layout and Text in a Large Language Model (LayTextLLM) they use OCR-derived text and spatial layouts to integrate with LLMs for document understanding .
Outcome: The proposed model shows an increase in performance in KIE and VQA tasks.
ASSIST: Towards Label Noise-Robust Dialogue State Tracking (2022.findings-acl)

Copied to clipboard

Challenge: Existing versions of MultiWOZ 2.0 have been published, but there are still lots of noisy labels in the training set.
Approach: They propose a framework to train dialogue state tracking models from noisy labels instead of improving annotation quality further by using auxiliary models.
Outcome: The proposed framework improves the goal accuracy of DST models by 28.16% on MultiWOZ 2.0 and 8.41% on MultiWoz 2.4, compared to using only the vanilla noisy labels.
DetectBench: Can Large Language Model Detect and Piece Together Implicit Evidence? (2024.findings-emnlp)

Copied to clipboard

Challenge: Existing LLMs' abilities to detect evidence in long contexts are far inferior to humans.
Approach: They propose a benchmark to assess LLMs' abilities in evidence and multi-step commonsense reasoning within a long context.
Outcome: The proposed method improves the performance of LLMs in evidence detection and commonsense reasoning.
Fundamental Capabilities of Large Language Models and their Applications in Domain Scenarios: A Survey (2024.acl-long)

Copied to clipboard

Challenge: Existing benchmark-based evaluations cannot accurately reflect the performance of real-world applications.
Approach: They propose a reliable strategy for domains to choose more robust LLMs for real-world applications.
Outcome: The proposed strategy addresses the challenges faced by domains to choose more robust LLMs for real-world applications.
Nature-Inspired Population-Based Evolution of Large Language Models (2026.acl-long)

Copied to clipboard

Challenge: a new framework for population-based evolution of large language models is emerging . a population-driven evolution of LLMs is a key component of evolution, authors say .
Approach: They propose a framework that allows for population-based evolution of large language models . they start with a population of parent LLMs and allow this population to evolve .
Outcome: The proposed framework outperforms existing methods on 12 datasets.
Omni-I2C: A Holistic Benchmark for High-Fidelity Image-to-Code Generation (2026.acl-long)

Copied to clipboard

Challenge: a benchmark is designed to evaluate the capability of Large Multimodal Models (LMMs) in converting complex, structured digital graphics into executable code.
Approach: They propose a benchmark to evaluate the capability of Large Multimodal Models to convert digital graphics into executable code.
Outcome: The proposed benchmark exposes the performance gap among leading LMMs . the benchmark features 1130 meticulously curated samples .
SwapMoE: Serving Off-the-shelf MoE-based Large Language Models with Tunable Memory Budget (2024.acl-long)

Copied to clipboard

Challenge: Mixture of experts (MoE) is a popular technique to improve capacity of Large Language Models (LLMs) but memory-constrained devices are a major concern in edge AI training and serving.
Approach: They propose a framework for efficient serving of MoE-based large language models with tunable memory budgets.
Outcome: Experiments show that SwapMoE can reduce memory consumption while maintaining reasonable accuracy.
SciCustom: A Framework for Custom Evaluation of Scientific Capabilities in Large Language Models (2026.acl-long)

Copied to clipboard

Challenge: Existing evaluations of large language models fail to reflect fine-grained capabilities . existing benchmarks are manually curated or domain-generic, limiting scalability and alignment with real use cases.
Approach: They propose a framework that allows custom construction of benchmarks from large-scale scientific data to evaluate application-specific scientific capabilities in LLMs.
Outcome: The proposed framework reveals fine-grained differences in scientific capabilities that standard benchmarks overlook . it allows custom construction of benchmarks from large-scale scientific data to evaluate application-specific capabilities in LLMs.
Modeling discourse cohesion for discourse parsing via memory network (P18-2)

Copied to clipboard

Challenge: Existing approaches to discourse parsing focus on studying the semantic and syntactic aspects of EDU pairs, but they do not address long span dependencies.
Approach: They propose a new transition-based discourse parser that takes discourse cohesion into account by using memory networks.
Outcome: The proposed method outperforms traditional features and improves performance on the RST discourse treebank.
LLaMA Pro: Progressive LLaMA with Block Expansion (2024.acl-long)

Copied to clipboard

Challenge: Existing studies have demonstrated that pre-trained LLMs are limited in certain domains, such as programming, mathematics, biomedical, or finance.
Approach: They propose a new post-pretraining method with an expansion of Transformer blocks to tune the expanded blocks using only new corpus, efficiently and effectively improving the model’s knowledge while mitigating forgetting.
Outcome: The proposed model outperforms existing models in programming and math and its instruction-following counterpart LLaMA Pro-8.3B in general tasks, programming, and mathematics.
Culture-Aware Machine Translation in Large Language Models: Benchmarking and Investigation (2026.acl-long)

Copied to clipboard

Challenge: Existing studies on large language models focus on literal-level translation quality, such as adequacy and fluency.
Approach: They propose a Culture-Aware Novel-Driven Parallel Dataset for Machine Translation and a multi-dimensional evaluation framework for assessing cultural translation quality.
Outcome: The proposed model improves evaluation reliability in LLM-as-a-judge scenarios under culture-aware constraints.
x1: Learning to Think Adaptively Across Languages and Cultures (2026.findings-acl)

Copied to clipboard

Challenge: Existing large language models (LLMs) ignore this diversity by reasoning in a single dominant language.
Approach: They propose a family of reasoning models that can adaptively reason in an advantageous language on a per-instance basis.
Outcome: The proposed model can reason in a single dominant language on a per-instance basis.
Disentangling Reasoning Tokens and Boilerplate Tokens For Language Model Fine-tuning (2025.findings-acl)

Copied to clipboard

Challenge: Existing approaches to enhance agent capabilities for Large Language Models treat all tokens equally . however, reasoning tokens versus boilerplate tokens differ in importance and learning complexity . recent research has focused on enhancing agent capabilities in large language models .
Approach: They propose a Shuffle-Aware Discriminator (SHAD) for adaptive token discrimination . they propose SHAD method which adaptively emphasizes reasoning tokens during fine-tuning .
Outcome: The proposed method improves performance over standard fine-tuning methods.
StrucText-Eval: Evaluating Large Language Model’s Reasoning Ability in Structure-Rich Text (2025.acl-long)

Copied to clipboard

Challenge: Structured data has been central to corporate data strategies for decades . however, with the advancement of large language models (LLMs), there has been a significant shift towards the effective utilization of unstructured data.
Approach: They propose an automatic evaluation data generation method to assess LLMs’ reasoning capabilities on structure-rich text.
Outcome: The proposed method supports 8 structured languages and 29 tasks, generating data with adjustable complexity through controllable nesting and structural width.
ZeroGen: Efficient Zero-shot Learning via Dataset Generation (2022.emnlp-main)

Copied to clipboard

Challenge: Existing approaches to generate training data with pre-trained language models have been found effective in various scenarios.
Approach: They propose an unsupervised zero-shot learning method that generates a dataset from scratch and trains a tiny task model under supervision of the synthesized dataset.
Outcome: The proposed method is annotated-free and efficient, but can provide useful insights from the perspective of data-free model-agnostic knowledge distillation and unreferenced text generation evaluation.
Stratagem: Learning Transferable Reasoning via Trajectory-Modulated Game Self-Play (2026.acl-long)

Copied to clipboard

Challenge: Existing self-play approaches to developing general reasoning in language models rely on terminal game outcomes.
Approach: They propose a game-based reasoning transfer model that addresses two barriers to reasoning transfer.
Outcome: The proposed model improves mathematical reasoning, general reasoning, and code generation benchmarks.
MIRAGE: Exploring How Large Language Models Perform in Complex Social Interactive Environments (2025.acl-short)

Copied to clipboard

Challenge: Large Language Models (LLMs) have shown remarkable capabilities in environmental perception, reasoning-based decision-making, and simulating complex human behaviors, particularly in interactive role-playing contexts.
Approach: They propose a framework to assess LLMs' proficiency in portraying advanced human behaviors through murder mystery games using eight intricately crafted scripts.
Outcome: The framework evaluates LLMs' performance in portraying advanced human behaviors through murder mystery games.
CC-Tuning: A Cross-Lingual Connection Mechanism for Improving Joint Multilingual Supervised Fine-Tuning (2025.acl-long)

Copied to clipboard

Challenge: Existing fine-tuning approaches that focus on English-centric training corpora often introduce implicit cross-lingual alignment, overlooking the potential for more profound, latent-level cross-linguistic interactions.
Approach: They propose a multilingual fine-tuning paradigm that explicitly establishes a cross-lingual connection mechanism at the latent level.
Outcome: The proposed model outperforms vanilla SFT and offers a strong latent-level alternative to data-level augmentation methods.
STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translation (2022.acl-long)

Copied to clipboard

Challenge: Existing methods to learn speech representations for end-to-end speech-totext translation (ST) neglect the representation discrepancy across modalities.
Approach: They propose a method to calibrate the representation discrepancy between modalities by mixing up the representation sequences of different modality inputs.
Outcome: The proposed method alleviates the cross-modal representation discrepancy and improves on a strong baseline on eight translation directions.
Towards Reverse Engineering of Language Models: A Survey (2025.findings-emnlp)

Copied to clipboard

Challenge: Due to the vast amounts of data and computational resources required for model development, protecting the model’s parameters and training data has become an urgent and crucial concern.
Approach: They define "reverse engineering" techniques as attacks on large language models and provide an in-depth analysis of them.
Outcome: The proposed attacks are described as “reverse engineering” techniques on LMs and provide an introduction to existing protective strategies.
Unveiling Entity-Level Unlearning for Large Language Models: A Comprehensive Analysis (2025.coling-main)

Copied to clipboard

Challenge: Existing studies have focused on instance-level unlearning, specifically removing predefined instances containing sensitive content.
Approach: They propose a task to erase entity-related knowledge from the target model completely by analyzing the forget set and its size.
Outcome: The proposed task systematically evaluates popular unlearning algorithms and reveals that the knowledge coverage of the forget set and its size play pivotal roles.
Does Recommend-Revise Produce Reliable Annotations? An Analysis on Missing Instances in DocRED (2022.acl-long)

Copied to clipboard

Challenge: Document-level relation extraction is a challenging task as it requires reasoning across multiple sentences.
Approach: They propose to use a recommend-revise scheme to reduce the workload of annotators by providing them with candidate relation instances from distant supervision to supplement and remove relational facts.
Outcome: The proposed dataset is the first large-scale and human-annotated dataset for relation extraction.
SAVOIR: Learning Social Savoir-Faire via Shapley-based Reward Attribution (2026.findings-acl)

Copied to clipboard

Challenge: Existing approaches to improve social intelligence of AI systems employ retrospective attributions and lack theoretical grounding.
Approach: They propose a framework that uses Shapley values to ensure fair credit distribution with axiomatic guarantees of efficiency, symmetry, and marginality.
Outcome: The proposed framework matches or exceeds proprietary models including GPT-4o and Claude-3.5-Sonnet.
OpenICL: An Open-Source Framework for In-context Learning (2023.acl-demo)

Copied to clipboard

Challenge: In-context Learning (ICL) is a new paradigm for large language model evaluation.
Approach: They propose an open-source toolkit for ICL and LLM evaluation.
Outcome: The proposed framework is highly flexible and flexible and can be easily combined with other tools to suit users' needs.
Improving Contextual Faithfulness of Large Language Models via Retrieval Heads-Induced Optimization (2025.acl-long)

Copied to clipboard

Challenge: Existing frameworks for retrieval-augmented large language models (LLMs) are lacking in LFQA faithfulness testing.
Approach: They propose a framework to teach retrieval-augmented large language models to explicitly discriminate between faithful and unfaithful generations.
Outcome: The proposed framework outperforms GPT-4o in LFQA scenarios and outperformed existing benchmarks.
Knowledge-guided Open Attribute Value Extraction with Reinforcement Learning (2020.emnlp-main)

Copied to clipboard

Challenge: Existing methods for open attribute value extraction for emerging entities are noisy or incomplete, even missing.
Approach: They propose a knowledge-guided reinforcement learning framework for open attribute value extraction for emerging entities.
Outcome: The proposed framework outperforms baselines by 16.5 - 27.8%.
ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback (2022.findings-emnlp)

Copied to clipboard

Challenge: Recent work on dataset-generation-based zero-shot learning has shown promising results by training a task-specific model with a dataset synthesized from large pre-trained language models (PLMs).
Approach: They propose a progressive zero-shot dataset generation framework which leverages feedback from the task-specific model to guide the generation of new training data via in-context examples.
Outcome: The proposed framework achieves on-par or superior performance with only 1% synthetic dataset size, when compared to baseline methods without in-context feedback.
Query-aware Multi-modal based Ranking Relevance in Video Search (2023.emnlp-industry)

Copied to clipboard

Challenge: Existing relevance ranking methods focus on text modality, incapable of fully exploiting cross-modal cues present in video.
Approach: They propose a QUery-Aware pre-training model with multi-modality that integrates video tag information as alignment targets and enhances ranking optimization method based on ordinal regression.
Outcome: The proposed model significantly improves video search performance.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations