Papers by Feng Yao

49 papers
Enhancing Speech-to-Speech Dialogue Modeling with End-to-End Retrieval-Augmented Generation (2025.findings-emnlp)

Copied to clipboard

Challenge: End-to-end speech-to speech (S2S) dialogue systems face key challenges in incorporating external knowledge into their models.
Approach: They propose a framework that directly retrieves relevant textual knowledge from speech queries.
Outcome: The proposed framework improves the performance of end-to-end speech-tospeech dialogue systems while achieving higher retrieval efficiency.
SGA-MCTS: Decoupling Planning from Execution via Training-Free Atomic Experience Retrieval (2026.findings-acl)

Copied to clipboard

Challenge: a new framework casts LLM planning as non-parametric retrieval, but high latency of inference-time search and supervised fine-tuning are limitations.
Approach: They propose a framework that casts LLM planning as non-parametric retrieval . they leverage Monte Carlo Tree Search to explore the solution space .
Outcome: Empirical results show that SGA-MCTS can match the performance of SOTA systems without task-specific fine-tuning.
Rethinking Diverse Human Preference Learning through Principal Component Analysis (2025.findings-acl)

Copied to clipboard

Challenge: Decomposed Reward Models extract diverse human preferences from binary comparisons without fine-grained annotations.
Approach: They propose a decomposed reward model that extracts diverse human preferences from binary comparisons without fine-grained annotations.
Outcome: The proposed approach extracts diverse human preferences from binary comparisons without fine-grained annotations.
Agentic Memory: Learning Unified Long-Term and Short-Term Memory Management for Large Language Model Agents (2026.acl-long)

Copied to clipboard

Challenge: Existing methods handle long-term memory (LTM) and short-term (STM) as separate components, relying on heuristics or auxiliary controllers, which limits adaptability and end-to-end optimization.
Approach: They propose a framework that integrates LTM and STM management directly into the agent's policy and propose 'agentic memory' to train such unified behaviors.
Outcome: The proposed framework outperforms strong memory-augmented baselines on five long-horizon benchmarks and achieves higher-quality long-term memory and more efficient context usage.
LLM-Powered Benchmark Factory: Reliable, Generic, and Efficient (2026.acl-long)

Copied to clipboard

Challenge: Using generic and efficient benchmark generators, human annotators are limited by inefficiency . current benchmark generator methods rely on seed signals, leading to long cycles and high costs .
Approach: They propose a framework to evaluate LLMs as generic benchmark generators and integrate them as BenchMaker.
Outcome: The proposed framework achieves comparable performance to human-annotated benchmarks on most metrics.
Natural Answer Generation with Heterogeneous Memory (N18-1)

Copied to clipboard

Challenge: Recent work on memory augmented encoder-decoder frameworks has shown promising progress for natural language generation tasks.
Approach: They propose a memory-augmented encoder-decoder framework that takes care of memory contents from different sources to explicitly avoid repetition.
Outcome: The proposed approach can produce readable and meaningful answer sentences while maintaining high coverage for given answer information.
OmniEvent: A Comprehensive, Fair, and Easy-to-Use Toolkit for Event Understanding (2023.emnlp-demo)

Copied to clipboard

Challenge: Event understanding is fundamental for humans to understand the world.
Approach: They propose an event understanding toolkit called OmniEvent that is comprehensive and fair . it supports mainstream modeling paradigms and the processing of 15 widely-used datasets .
Outcome: The toolkit supports mainstream modeling paradigms and the processing of 15 widely-used English and Chinese datasets.
ShifCon: Enhancing Non-Dominant Language Capabilities with a Shift-based Multilingual Contrastive Framework (2025.acl-long)

Copied to clipboard

Challenge: Experiments show that ShifCon significantly enhances the performance of non-dominant languages due to the imbalance in training data across languages.
Approach: They propose a Shift-based multilingual Contrastive framework that aligns the internal forward process of other languages toward that of the dominant one.
Outcome: The proposed framework significantly improves performance of non-dominant languages, particularly for low-resource ones.
Make Every Penny Count: Difficulty-Adaptive Self-Consistency for Cost-Efficient Reasoning (2025.findings-naacl)

Copied to clipboard

Challenge: Existing decoding strategies for chain-of-thought reasoning do not exploit prior information about question difficulty.
Approach: They propose a decoding strategy called self-consistency to improve reasoning performance by adjusting the number of samples based on the posterior distribution of a set of pre-samples.
Outcome: The proposed method outperforms baseline methods on arithmetic, commonsense and symbolic reasoning tasks while achieving comparable performance.
Adaptive Backtracking for Privacy Protection in Large Language Models (2026.findings-acl)

Copied to clipboard

Challenge: Existing privacy protection methods are prone to privacy leakage, but they are not effective in ensuring the privacy of users.
Approach: They propose to capture latent leakage tendency of large language models during generation process and to construct a new benchmark for personal information.
Outcome: The proposed method improves privacy by up to 14% over strong baselines against adversarial attacks, avoiding the degradation of response utility.
Focused Large Language Models are Stable Many-Shot Learners (2024.emnlp-main)

Copied to clipboard

Challenge: In-Context Learning (ICL) enables large language models to achieve rapid task adaptation by learning from demonstrations.
Approach: They propose a training-free method that disperses model attention from the query . they propose 'focus' search strategy that uses model perplexity to ensure sufficient attention .
Outcome: The proposed method achieves an average performance improvement of 5.2% over vanilla ICL and scales well with many-shot demonstrations.
InsBank: Evolving Instruction Subset for Ongoing Alignment (2025.findings-emnlp)

Copied to clipboard

Challenge: Recent studies emphasize that quality and diversity of instruction data are more crucial than quantity, highlighting the need to select diverse, high-quality subsets to reduce training costs.
Approach: They propose to use a continuously updated repository to integrate the latest valuable instruction data with a progressive evolution framework to evolve InsBank over time.
Outcome: The proposed framework outperforms baselines in InsBank evolution and extracts budget-specific subsets.
Integrate the Essence and Eliminate the Dross: Fine-Grained Self-Consistency for Free-Form Language Generation (2024.acl-long)

Copied to clipboard

Challenge: Existing methods to improve output quality without aggregating input tokens are limited by the complexity of aggregation of responses.
Approach: They propose to extract and integrate segment-level commonalities from candidate samples to enhance performance of LLMs in open-ended and reasoning tasks.
Outcome: The proposed method improves performance on reasoning, code generation and mathematical reasoning tasks without requiring additional models and overlooking the knowledge present among the candidates.
Cognitive-Level Adaptive Generation via Capability-Aware Retrieval and Style Adaptation (2025.findings-emnlp)

Copied to clipboard

Challenge: Large Language Models struggle to adapt content to users with differing cognitive capacities, leading to cognitive misalignment.
Approach: They propose a cognitive-level alignment framework that aligns both knowledge complexity and presentation style with user cognition.
Outcome: The proposed framework aligns knowledge complexity and presentation style with user cognition.
Data Contamination Can Cross Language Barriers (2024.emnlp-main)

Copied to clipboard

Challenge: Existing methods to detect contamination of public benchmarks are too superficial to reflect deeper forms of contamination.
Approach: They propose generalization-based approaches to unmask a cross-lingual form of contamination that inflates LLMs’ performance while evading current detection methods.
Outcome: The proposed model outperforms existing detection methods while avoiding contamination of public benchmarks in the pre-training data.
D2PCM:A Multi-Turn Dialogue Dataset with Personalized Contextual Memory (2026.findings-acl)

Copied to clipboard

Challenge: Conventional interactive algorithms have predominantly treated memory as a contextual element, neglecting the nuanced cognitive processes involved in individualized memory encoding and retrieval.
Approach: They propose a multi-turn dialogue dataset with Personalized Contextual Memory to facilitate advanced research on personalized memory processing.
Outcome: The proposed datasets provide a comprehensive benchmark to facilitate advanced research on personalized memory processing.
MiCRo: Mixture Modeling and Context-aware Routing for Personalized Preference Learning (2025.emnlp-main)

Copied to clipboard

Challenge: Existing reward models assume a global reward function, limiting personalization and pluralistic alignment.
Approach: They propose a framework that leverages binary preference datasets to enhance personalized preference learning.
Outcome: The proposed framework captures diverse human preferences without fine-grained annotations and significantly improves personalized preference learning on downstream tasks.
Beyond One-Size-Fits-All: Tailored Benchmarks for Efficient Evaluation (2025.acl-long)

Copied to clipboard

Challenge: Existing efficient methods estimate performance of models on large benchmarks, but these methods rely on the assumption that target models have high prediction consistency with source models.
Approach: They propose a method that conducts customized evaluation tailored to each target model.
Outcome: The proposed method reduces the MAE of estimates by 31.4% on benchmarks across 300 models.
Agentic Verification for Ambiguous Query Disambiguation (2026.findings-acl)

Copied to clipboard

Challenge: Prior Diversify-then-Verify pipelines generate interpretations and then retrieve evidence . ambiguous queries require RAG to disambiguate into interpretations that can be answered from corpus .
Approach: They propose a novel approach that unifies diversification with verification by integrating retriever relevance and generator answerability feedback early.
Outcome: The proposed approach improves grounding-aware F1 by 23% over baselines across multiple LLMs.
Operation-guided Neural Networks for High Fidelity Data-To-Text Generation (D18-1)

Copied to clipboard

Challenge: Recent neural models for data-to-text generation generate descriptions that are not consistent with structured data.
Approach: They propose a framework for data-to-text generation that uses symbolic operations to generate texts from structured data.
Outcome: The proposed framework improves the fidelity of the generated texts to the input structured data.
DeepPlanner: Scaling Planning Capability for Deep Research Agents via Advantage Shaping (2026.findings-acl)

Copied to clipboard

Challenge: Existing approaches to planning involve implicit planning or introduce explicit planners without systematically optimizing the planning stage.
Approach: They propose an end-to-end RL framework that enhances the planning capabilities of deep research agents.
Outcome: Experiments show that DeepPlanner improves planning quality and achieves state-of-the-art results under a lower training budget.
Evaluating the Smooth Control of Attribute Intensity in Text Generation with LLMs (2024.findings-acl)

Copied to clipboard

Challenge: Controllable text generation is increasingly tailored to individual preferences.
Approach: They propose to evaluate the attribute intensity of text generated by large language models on five different attributes for error, variation of the generated sentence's intensities and relevance to the generation questions.
Outcome: The proposed methods are based on Elo rating system and GPT4 and are able to be trained without training.
AutoSchemaKG: Autonomous Knowledge Graph Construction through Dynamic Schema Induction from Web-Scale Corpora (2026.acl-long)

Copied to clipboard

Challenge: Existing knowledge graph construction frameworks require predefined schemas, limiting their scalability and domain coverage.
Approach: They propose a framework for fully autonomous knowledge graph construction that eliminates the need for predefined schemas.
Outcome: The proposed framework outperforms state-of-the-art models on multi-hop QA tasks and enhances LLM factuality.
G-Cap: A Game Character Caption Generator (2026.acl-long)

Copied to clipboard

Challenge: Existing studies on Large Vision-Language Models (LVLMs) primarily focus on real-world scenarios, leaving surreal, highly stylized, and semantically hybrid virtual-world situations significantly underexplored.
Approach: They propose to use a manually annotated benchmark to evaluate LVLMs' ability to perceive and describe game character from the virtual-world.
Outcome: The proposed task evaluates LVLMs’ ability to perceive and describe game character from the virtual-world.
Improving Factual Consistency of News Summarization by Contrastive Preference Optimization (2024.findings-emnlp)

Copied to clipboard

Challenge: Recent advances in news summarization have created problems with “hallucinations” that are factually inconsistent with the source text.
Approach: They propose to disentangle LLMs’ propensities to generate faithful and fake content by adopting a probing-based specific training method to improve their capacity of distinguishing two types of propensity.
Outcome: The proposed method disentangles LLMs’ propensities to generate faithful and fake content and improves their ability to distinguish between two types of propensity.
The Devil is in the Details: On the Pitfalls of Event Extraction Evaluation (2023.findings-acl)

Copied to clipboard

Challenge: Event extraction (EE) is a fundamental information extraction task aimed at extracting events from plain texts.
Approach: They propose to specify data preprocessing, standardize outputs, and provide pipeline evaluation results to avoid these pitfalls.
Outcome: The results show that the evaluations are reliable and lack pipeline evaluations.
CogLM: Tracking Cognitive Development of Large Language Models (2025.naacl-long)

Copied to clipboard

Challenge: Large Language Models (LLMs) have recently shown remarkable abilities across a wide variety of tasks, but few studies have explored the reasons behind the evolutionary relationship among various abilities.
Approach: They construct a benchmark CogLM based on Piaget's Theory of Cognitive Development (PTC) which measures the cognitive levels of Large Language Models (LLMs) using 1,220 questions spanning 10 cognitive abilities crafted by more than 20 human experts.
Outcome: The proposed framework provides a comprehensive testbed for the cognitive levels of LLMs.
Rose-SQL: Role-State Evolution Guided Structured Reasoning for Multi-Turn Text-to-SQL (2026.acl-long)

Copied to clipboard

Challenge: Existing approaches to multi-turn Text-to-SQL tasks rely on unstable APIs or expensive fine-tuning.
Approach: They propose a training-free framework that leverages small-scale LRMs through in-context learning to enable accurate context-dependent parsing.
Outcome: The proposed framework outperforms in-context learning baselines at the 4B scale and surpasses state-of-the-art models at the 8B and 14B scales.
LEVEN: A Large-Scale Chinese Legal Event Detection Dataset (2022.findings-acl)

Copied to clipboard

Challenge: Existing legal event detection datasets only cover incomprehensive event types and have limited annotated data.
Approach: They present a large-scale Chinese legal event detection dataset . they use legal events as side information to promote downstream applications .
Outcome: The proposed method improves 2.2 points precision in low-resource judgment prediction and 1.5 points precision for unsupervised case retrieval.
Beyond Scaling: Predicting Patent Approval with Domain-specific Fine-grained Claim Dependency Graph (2024.acl-long)

Copied to clipboard

Challenge: Scaling up language models has demonstrated predictable improvement and unprecedented abilities in many language tasks.
Approach: They propose a fine-grained cLAim depeNdency graph that captures the dependencies within the patent data and extends the embedding-based state-of-the-art (SOTA) they then explore prompt-based methods to harness proprietary LLMs' potential, but find the best results close to random guessing, underlining the ineffectiveness of model scaling-up.
Outcome: The proposed graph methods outperform the standard model scaling methods in the patent approval prediction task and show that they are cost-effective.
Speculative Decoding for Multi-Sample Inference (2025.findings-emnlp)

Copied to clipboard

Challenge: Speculative decoding method exploits consensus of parallel reasoning paths to synthesize high-quality draft tokens without auxiliary models or external databases.
Approach: They propose a speculative decoding method that exploits the consensus of parallel reasoning paths to synthesize high-quality draft tokens without auxiliary models or external databases.
Outcome: The proposed method exploits the intrinsic consensus of parallel reasoning paths to synthesize high-quality draft tokens without auxiliary models or databases.
Chain-of-Procedure: Hierarchical Visual-Language Reasoning for Procedural QA (2026.findings-acl)

Copied to clipboard

Challenge: Recent advances in vision-language models (VLMs) have achieved impressive results on standard image-text tasks, yet their capability in visual procedure question answering (VP-QA) remains largely unexplored.
Approach: They propose a multimodal benchmark specifically designed for visual procedural reasoning that synergizes cross-modal procedure retrieval, context-aware step decomposition, and the next step prediction.
Outcome: The proposed framework significantly outperforms baselines on visual procedure question answering (VP-QA) Experiments on six VLMs show that it performs better than baselines.
A Simple Recipe towards Reducing Hallucination in Neural Surface Realisation (P19-1)

Copied to clipboard

Challenge: Recent neural language generation systems often hallucinate contents when trained on loosely corresponding pairs of the input structure and text.
Approach: They propose to integrate a language understanding module for data refinement with self-training iterations to induce strong equivalence between the input data and the paired text.
Outcome: Experiments on the E2E challenge dataset show that the proposed framework reduces relative unaligned noise by 50% compared with the current state-of-the-art ensemble generator.
Cuckoo: An IE Free Rider Hatched by Massive Nutrition in LLM’s Nest (2025.acl-long)

Copied to clipboard

Challenge: Massive high-quality data, both pre-training raw texts and post-training annotations, have been carefully prepared to incubate advanced large language models (LLMs).
Approach: They propose to reframe next-token prediction into extraction for tokens already present in the context of LLMs by reframing next-tongue prediction into IE models.
Outcome: The proposed model learns 102.6M extractive data converted from pre-training and post-training data with better performance than existing pre-trained IE models.
From Discrimination to Generation: Low-Resource Intent Detection with Language Model Instruction Tuning (2024.findings-acl)

Copied to clipboard

Challenge: Existing studies fine-tune discriminative models on specific defined intent classes, preventing them from being directly adopted to new intent domains.
Approach: They propose to use a pre-trained generative intent model to detect new intents from different domains with no parameter updates.
Outcome: The proposed model outperforms baselines that need further fine-tuning or domain-specific samples.
Thinking Traps in Long Chain-of-Thought: A Measurable Study and Trap-Aware Adaptive Restart (2026.findings-acl)

Copied to clipboard

Challenge: Experiments show that extended generation does not guarantee correctness . a recurring pattern in Long-CoT failures is a problem for large reasoning models .
Approach: They propose a test-time control framework that truncates the trajectory before the trap segment and adaptively restarts decoding.
Outcome: Experiments show that TAAR improves reasoning performance without fine-tuning model parameters.
Cascaded Mutual Modulation for Visual Reasoning (D18-1)

Copied to clipboard

Challenge: Visual reasoning is a multi-step and compositional problem that requires intensive text-vision interactions.
Approach: They propose a visual reasoning model that uses a feature-wise linear modulation technique to enable textual/visual pipelines to mutually control each other.
Outcome: The proposed model outperforms existing models on visual reasoning benchmarks CLEVR and NLVR . it can generate a textual answer to a visual question answering problem with images .
Beyond Static Profiles: Capturing the Fluidity of User Preferences in Diverse Scenarios (2026.findings-acl)

Copied to clipboard

Challenge: Existing approaches to personalize Large Language Models often default to homogeneous behaviors . preferences can shift, and conflict, depending on context, authors argue .
Approach: They propose a hierarchical taxonomy to differentiate between stable and situational preferences . they use a dataset of 10k meticulously curated preferences to test their taxonomies .
Outcome: The proposed model differentiates between stable and situational preferences based on curated user preferences . it provides a practical testbed for advancing dynamic, context-aware personalization in conversational agents.
From Sub-Ability Diagnosis to Human-Aligned Generation: Bridging the Gap for Text Length Control via MarkerGen (2025.acl-long)

Copied to clipboard

Challenge: Existing methods to control text length are lacking in LCTG, posing a major limitation for practical applications.
Approach: They propose a plug-and-play approach that decomposes LCTG sub-abilities with human patterns as reference and performs detailed error analysis.
Outcome: The proposed method significantly improves LCTG across various settings, exhibiting outstanding effectiveness and generalizability.
MAPLE: Enhancing Review Generation with Multi-Aspect Prompt LEarning in Explainable Recommendation (2025.acl-long)

Copied to clipboard

Challenge: Existing models that generate generic aspects do not provide personalized informative recommendations.
Approach: They propose a model that integrates aspect category as another input dimension to facilitate memorizing fine-grained aspect terms.
Outcome: The proposed model outperforms baseline model on restaurant review datasets in the restaurant domain.
Revisiting Self-Consistency from Dynamic Distributional Alignment Perspective on Answer Aggregation (2025.findings-acl)

Copied to clipboard

Challenge: Existing studies on self-consistency show that it improves reasoning abilities by aggregating diverse stochastic samples.
Approach: They propose a confidence-driven mechanism that dynamically calibrates temperature to align with high probability modes.
Outcome: The proposed method outperforms fixed-diversity baselines on reasoning tasks and improves both average and best-case performance.
Clear Up Confusion: Iterative Differential Generation for Fine-grained Intent Detection with Contrastive Feedback (2025.coling-main)

Copied to clipboard

Challenge: Recent studies on fine-grained intent detection have focused on collecting large-scale and high-quality samples via crowdsourcing resulting in data scarcity.
Approach: They propose an iterative differential generation framework with contrastive feedback to generate high-quality pseudo samples and accurately capture the crucial nuances in target class distribution.
Outcome: The proposed framework generates high-quality pseudo samples and captures crucial nuances in target class distribution.
xLAM: A Family of Large Action Models to Empower AI Agent Systems (2025.naacl-long)

Copied to clipboard

Challenge: Autonomous agents powered by large language models (LLMs) have attracted significant research interest, but there are few standards for developing specialized models for agent tasks.
Approach: They propose a series of large action models with dense and mixture-of-expert architectures that unifies, augments, and synthesizes diverse datasets to enhance agent generalizability and performance.
Outcome: The proposed models outperform GPT-4, Claude-3, and many other models in terms of tool use and outperformed GPT-based models on multiple agent ability benchmarks.
AlphaQT-Bench: Diagnosing the Gap between Financial Code Generation and Quantitative Reasoning in LLMs (2026.findings-acl)

Copied to clipboard

Challenge: Existing benchmarks rely on outcome-driven metrics such as profitability and look-ahead bias.
Approach: They propose a diagnostic benchmark for instruction-grounded financial code generation under strict semantic and temporal constraints.
Outcome: The proposed benchmarks show that the models fail under causal, structural, or functional constraints.
Towards Reverse Engineering of Language Models: A Survey (2025.findings-emnlp)

Copied to clipboard

Challenge: Due to the vast amounts of data and computational resources required for model development, protecting the model’s parameters and training data has become an urgent and crucial concern.
Approach: They define "reverse engineering" techniques as attacks on large language models and provide an in-depth analysis of them.
Outcome: The proposed attacks are described as “reverse engineering” techniques on LMs and provide an introduction to existing protective strategies.
Dynamic Guided and Domain Applicable Safeguards for Enhanced Security in Large Language Models (2025.findings-naacl)

Copied to clipboard

Challenge: Existing defense methods struggle with two key issues: inadequate defense capabilities and over-defensiveness.
Approach: They propose a multi-agents-based framework that leverages accurate external information to provide an unbiased summary of user intentions and safety response guidance.
Outcome: Experiments on popular jailbreak attacks and benign datasets show that the proposed framework can enhance LLM's robustness against jailbreaks without compromising its general functionality.
Poor-Supervised Evaluation for SuperLLM via Mutual Consistency (2024.findings-acl)

Copied to clipboard

Challenge: evaluating superLLMs is especially difficult because of their intelligence-intensive nature.
Approach: They propose an evaluation benchmark with accurate labels for SuperLLMs whose capabilities surpass those of humans . they first prove that consistency between model under evaluation and reference model can equalize the true capabilities of the model to be evaluated .
Outcome: The proposed evaluation benchmarks can assess the true capabilities of the model to be evaluated without accurate labels.
Thinking Alignment of Scenario-Oriented User Simulation (2026.acl-long)

Copied to clipboard

Challenge: Existing user simulators based on prompting to role-play or SFT focus on imitating textual utterances without considering multi-faceted cognitive processes that underlie human decision-making during interactions.
Approach: They construct a user-simulator dataset that augments 51k human–LLM conversations by reconstructing the user’s inner reasoning during and at the end of each dialogue.
Outcome: The proposed user simulators augment 51k human–LLM conversations by reconstructing the user’s inner reasoning both during and at the end of each dialogue.
CGBridge: Bridging Code Graphs and Large Language Models for Better Structure-Aware Code Understanding (2026.findings-acl)

Copied to clipboard

Challenge: Existing structure-aware approaches treat structure as serialized text prompts or auxiliary training objectives, failing to provide explicit guidance during inference.
Approach: They propose a plug-and-play method that enhances Large Language Models with Code Graph information through an external, trainable Bridge module.
Outcome: The proposed method decouples structural reasoning from textual generation without updating the backbone.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations