Papers by Yao Zhu

36 papers
Layer-wise Importance Matters: Less Memory for Better Performance in Parameter-efficient Fine-tuning of Large Language Models (2024.findings-emnlp)

Copied to clipboard

Challenge: Parameter-Efficient Fine-Tuning (PEFT) methods have gained popularity for adapting pre-trained Large Language Models (LLMs) to downstream tasks.
Approach: They propose a method to optimize the importance of full layers with layer-wise importance scoring by leveraging the estimated importance scores.
Outcome: The proposed method is compatible with PEFT methods that operate on a per-layer basis and achieves better performance.
LIFTED: Multimodal Clinical Trial Outcome Prediction via Large Language Models and Mixture-of-Experts (2025.findings-emnlp)

Copied to clipboard

Challenge: Clinical trials are costly and pivotal processes that require substantial expenses . a new approach to integrate multimodal data for clinical outcome prediction is needed .
Approach: a proposed framework transforms modality-specific data into natural language descriptions . a sparse Mixture-of-Experts mechanism then identifies shared patterns across modalities .
Outcome: a proposed framework outperforms baseline methods in predicting clinical trial outcomes . it transforms modality-specific data into natural language descriptions, encoded via unified encoders .
OmniEvent: A Comprehensive, Fair, and Easy-to-Use Toolkit for Event Understanding (2023.emnlp-demo)

Copied to clipboard

Challenge: Event understanding is fundamental for humans to understand the world.
Approach: They propose an event understanding toolkit called OmniEvent that is comprehensive and fair . it supports mainstream modeling paradigms and the processing of 15 widely-used datasets .
Outcome: The toolkit supports mainstream modeling paradigms and the processing of 15 widely-used English and Chinese datasets.
ZipVoice-Dialog: Non-Autoregressive Spoken Dialogue Generation with Flow Matching (2026.findings-acl)

Copied to clipboard

Challenge: Existing autoregressive models for dialogue generation suffer from high latency and stability issues.
Approach: They propose a non-autoregressive (NAR) zero-shot spoken dialogue generation model based on flow-matching.
Outcome: The proposed model outperforms existing models in speech generation due to poor speech intelligibility and turn-taking precision.
ActionStudio: A Lightweight Framework for Data and Training of Large Action Models (2025.emnlp-main)

Copied to clipboard

Challenge: Existing infrastructure for efficient agentic data processing and model training remains underdeveloped.
Approach: They propose a lightweight and extensible data and training framework for large action models . they propose to unify diverse agent trajectories using Unified Format 2.0 .
Outcome: The proposed framework shows 9 higher throughput than existing frameworks and performs well across public and realistic agent benchmarks.
PersonaBench: Evaluating AI Models on Understanding Personal Information through Accessing (Synthetic) Private User Data (2025.findings-acl)

Copied to clipboard

Challenge: Existing research lacks direct access to such data, making benchmarking difficult due to privacy concerns.
Approach: They propose a synthetic data pipeline that generates realistic user profiles and private documents and a benchmark to evaluate models' ability to understand personal information.
Outcome: The proposed pipeline generates realistic user profiles and private documents, enabling PersonaBench, a benchmark for evaluating models’ ability to understand personal information.
StructMem: Structured Memory for Long-Horizon Behavior in LLMs (2026.acl-short)

Copied to clipboard

Challenge: Existing memory systems lack structure and efficiency in capturing relationships between events.
Approach: They propose a structure-enriched hierarchical memory framework that preserves event-level bindings and induces cross-event connections.
Outcome: The proposed framework preserves event-level bindings and induces cross-event connections while reducing token usage, API calls, and runtime compared to prior memory systems.
Navigating the Shortcut Maze: A Comprehensive Analysis of Shortcut Learning in Text Classification by Language Models (2024.findings-emnlp)

Copied to clipboard

Challenge: Language models (LMs) often rely on spurious correlations rather than causally relevant features to improve accuracy and generalizability.
Approach: They propose a benchmark that categorizes shortcuts into occurrence, style, and concept . they aim to explore the nuanced ways shortcuts influence the performance of LMs .
Outcome: The proposed benchmark categorizes shortcuts into occurrence, style, and concept . it systematically investigates models’ resilience and susceptibilities to sophisticated shortcuts .
i-Code V2: An Autoregressive Generation Framework over Vision, Language, and Speech Data (2024.findings-naacl)

Copied to clipboard

Challenge: i-Code V2 is one of the first models capable of generating natural language from any combination of Vision, Language, and Speech data.
Approach: They propose to create a model that can generate natural language from any combination of Vision, Language, and Speech data.
Outcome: i-Code V2 matches or outperforms state-of-the-art single- and dual-modality baselines on 7 multimodal tasks.
MathSight: A Benchmark Exploring Have Vision-Language Models Really Seen in University-Level Mathematical Reasoning? (2026.acl-long)

Copied to clipboard

Challenge: Existing benchmarks rarely isolate how much visual information contributes to reasoning . a growing collection of benchmarks has catalyzed rapid progress in multimodal reasoning - but how much it contributes remains unclear .
Approach: They propose a university-level multimodal mathematical reasoning benchmark to quantify the effect of visual input.
Outcome: The proposed benchmark disentangles and quantifies the effect of visual input on multimodal reasoning models.
DIDS: Domain Impact-aware Data Sampling for Large Language Model Training (2025.emnlp-main)

Copied to clipboard

Challenge: Existing approaches for optimizing domain-level sampling strategies struggle with maintaining intra-domain consistency and accurately measuring domain impact.
Approach: They propose to use a Fisher-Information Matrix-guided metric to measure domain impact to ensure intra-domain consistency and accuracy.
Outcome: The proposed model achieves 3.4% higher average performance while maintaining comparable training efficiency.
Representation Learning with Ordered Relation Paths for Knowledge Graph Completion (D19-1)

Copied to clipboard

Challenge: Existing knowledge graphs are incomplete and lack the order of relations in paths.
Approach: They propose a method which takes relation paths into account but ignores order of relations in paths which is important for reasoning.
Outcome: The proposed method performs better than state-of-the-art methods on two benchmark datasets.
Think-Search-Patch: A Retrieval-Augmented Reasoning Framework for Repository-Level Code Repair (2025.emnlp-industry)

Copied to clipboard

Challenge: Large language models suffer from multiple-file coding scenarios with strong inter-file dependencies . experimental results show that large language models exhibit inadequate performance in multi-file scenarios .
Approach: They propose a retrieval-augmented reasoning framework for repository-level code repair . they use a dataset to generate standardized patches based on the key snippets .
Outcome: The proposed framework improves retrieval accuracy and repair success on SWE-bench Lite . it surpasses models with larger size in managing extensive code contexts and fixing bugs spanning across multiple files.
GradOT: Training-free Gradient-preserving Offsite-tuning for Large Language Models (2025.acl-long)

Copied to clipboard

Challenge: Existing methods for offsite-tuning of large language models require high computational costs and lack theoretical analysis.
Approach: They propose an offsite-tuning approach that selectively applies compression techniques such as rank compression and channel pruning to preserve the gradients of fine-tuned adapters while ensuring privacy.
Outcome: The proposed method surpasses existing OT methods in privacy protection and model performance.
FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation (D18-1)

Copied to clipboard

Challenge: Empirical results show that even the most competitive few-shot learning models struggle on this task, especially as compared with humans.
Approach: They propose a Few-Shot Relation Classification Dataset consisting of 70, 000 sentences on 100 relations derived from Wikipedia and annotated by crowdworkers.
Outcome: The proposed methods perform well on the most competitive few-shot learning models, especially as compared with humans.
Composition-based Heterogeneous Graph Multi-channel Attention Network for Multi-aspect Multi-sentiment Classification (2022.coling-1)

Copied to clipboard

Challenge: Existing methods for Aspect-based sentiment analysis (ABSA) focus on aspect terms with the same sentiment polarity . current methods focus on sentences with only one aspect term or multiple aspect terms .
Approach: They propose a novel method to model inter-aspect relationships and aspect-context relationships simultaneously using a heterogeneous graph.
Outcome: The proposed method can predict sentiments towards the given aspect term in a sentence . it can provide more detailed predictions compared with sentence-level sentiment analysis.
Enhancing LLM-based Hatred and Toxicity Detection with Meta-Toxic Knowledge Graph (2025.findings-acl)

Copied to clipboard

Challenge: Existing methods to address toxicity issues with large language models are inadequate . lack of domain-specific knowledge leads to false negatives and excessive sensitivity to toxic speech limits freedom of speech.
Approach: They propose a method that leverages graph search on a meta-toxic knowledge graph to enhance hatred and toxicity detection.
Outcome: The proposed method lowers false positive rate and improves toxicity detection performance in out-of-domain scenarios.
AutoPRM: Automating Procedural Supervision for Multi-Step Reasoning via Controllable Question Decomposition (2024.naacl-long)

Copied to clipboard

Challenge: Recent advances in large language models (LLMs) have shown promise in multi-step reasoning tasks, yet relying on extensive manual labeling to provide procedural feedback remains a significant impediment.
Approach: They propose a self-supervised framework that decomposes complex problems into manageable subquestions with a controllable granularity switch and sequentially applies reinforcement learning to iteratively improve the subquest solver.
Outcome: The proposed framework improves performance on mathematical and commonsense reasoning tasks over SOTA.
RULE: Reliable Multimodal RAG for Factuality in Medical Vision Language Models (2024.emnlp-main)

Copied to clipboard

Challenge: Existing medical large vision language models often generate inaccurate and irrelevant answers that do not align with established medical facts.
Approach: They propose a strategy for controlling factuality risk through calibrated selection of the number of retrieved contexts and a preference dataset to fine-tune the model.
Outcome: The proposed model achieves an average improvement of 20.8% on three medical VQA datasets.
Fraud-R1 : A Multi-Round Benchmark for Assessing the Robustness of LLM Against Augmented Fraud and Phishing Inducements (2025.findings-acl)

Copied to clipboard

Challenge: Existing fraud detection benchmarks focus on single-turn classification tasks, failing to capture dynamic nature of real-world fraud attempts.
Approach: They propose a bilingual benchmark to assess LLMs' ability to resist fraud and phishing attacks across five key fraud categories: Fraudulent Services, Impersonation, Phishing Scams, Fake Job Postings, and Online Relationships.
Outcome: The proposed model improves in role-play settings and in e-commerce and recommendation systems.
GEMv2: Multilingual NLG Benchmarking in a Single Line of Code (2022.emnlp-demos)

Copied to clipboard

Challenge: Evaluations in machine learning rarely use the latest metrics, datasets, or human evaluation in favor of remaining compatible with prior work.
Approach: They propose to use the Generation, Evaluation, and Metrics Benchmark to integrate new evaluation methods into existing evaluations.
Outcome: The proposed evaluation infrastructure bridges the gap between the advantages of leaderboards and in-depth and evolving evaluations by allowing model developers to benefit from each other's work.
Chart2Code53: A Large-Scale Diverse and Complex Dataset for Enhancing Chart-to-Code Generation (2025.emnlp-main)

Copied to clipboard

Challenge: Existing Chart2code-related training datasets suffer from limited scale, limited type coverage, and inadequate complexity.
Approach: They propose to synthesize chart2code-related training datasets using web plotting code and chart images to address these challenges.
Outcome: The proposed dataset exhibits the greatest diversity and higher complexity compared to other open-source Chart2code related datasets.
What’s Not Said Still Hurts: A Description-Based Evaluation Framework for Measuring Social Bias in LLMs (2025.findings-emnlp)

Copied to clipboard

Challenge: Existing benchmarks evaluate bias by term-based mode, but they fail to capture hidden biases in nuanced settings.
Approach: They propose a dataset to assess bias at the semantic level that bias concepts are hidden within naturalistic, subtly framed contexts in real-world scenarios.
Outcome: The proposed dataset shows that models reduce bias in response at term level, but reinforce bias in nuanced settings.
MedVerse: Efficient and Reliable Medical Reasoning via DAG-Structured Parallel Execution (2026.acl-long)

Copied to clipboard

Challenge: Recent advances in large reasoning models have broadened the capabilities of medical artificial intelligence.
Approach: They propose a reasoning framework for complex medical inference that reformulates medical reasoning as a parallelizable directed acyclic graph process based on Petri Net theory.
Outcome: The proposed reasoning framework improves strong general-purpose LLMs by up to 8.9%.
TCSinger 2: Customizable Multilingual Zero-shot Singing Voice Synthesis (2025.findings-acl)

Copied to clipboard

Challenge: Existing zero-shot singing voice synthesis models depend on phoneme and note boundary annotations, limiting their robustness and producing poor transitions between phonemes and notes.
Approach: They propose a multi-task multilingual zero-shot SVS model with style transfer and style control based on various prompts.
Outcome: Experimental results show that TCSinger 2 outperforms baseline models in subjective and objective metrics across multiple related tasks.
Unleashing the Potential of Large Language Models through Spectral Modulation (2024.findings-emnlp)

Copied to clipboard

Challenge: Large Language Models (LLMs) have demonstrated impressive capabilities across various domains, garnering significant attention from both academia and industry.
Approach: They propose to conduct spectral modulation in the parameter space of LLMs to integrate with various models in a plug-and-play manner.
Outcome: The proposed approach improves performance by 10.12% with spectral modulation.
Digest the Knowledge: Large Language Models empowered Message Passing for Knowledge Graph Question Answering (2025.acl-long)

Copied to clipboard

Challenge: Existing methods to augment large language models (LLMs) with external knowledge are unorganized and unorganized.
Approach: They propose a method that learns a concise facts graph and encodes it into multi-level lists of texts to augment LLMs.
Outcome: The proposed method improves on all 5 knowledge graph question answering datasets and offers human-level semantic explainability.
Rethinking the Video Sampling and Reasoning Strategies for Temporal Sentence Grounding (2022.findings-emnlp)

Copied to clipboard

Challenge: Existing methods for temporal sentence grounding ignore two crucial issues . 1) Boundary-bias: the video downsampling process may lose these two frames . 2) Reasoning-biases: such incorrect new boundary frames lead to the reasoning bias .
Approach: They propose a siamese sampling mechanism to generate additional contextual frames . they use a reasoning strategy to learn the inter-relationship among these frames a .
Outcome: Extensive experiments demonstrate the effectiveness of a new siamese sampling network on three challenging datasets.
OmniDiagram: Advancing Unified Diagram Code Generation via Visual Interrogation Reward (2026.findings-acl)

Copied to clipboard

Challenge: Existing studies on programmable diagram generation focus on a narrow set of tasks and languages.
Approach: They propose a unified framework that integrates diverse diagram code languages and task definitions.
Outcome: The proposed framework can bridge complex visual information with executable code across diverse tasks and languages.
Language models can learn implicit multi-hop reasoning, but only if they have lots of training data (2025.emnlp-main)

Copied to clipboard

Challenge: Existing studies explore the ability of language models to solve multi-hop reasoning tasks without chain of thought.
Approach: They propose to use GPT2-style language models to train k-hop reasoning models . they show that the required training data grows exponentially in k .
Outcome: The proposed models can learn implicit reasoning without chain-of-thoughts, the authors show . their training data grows exponentially in k, and the required number of transformer layers grows linearly in the model.
OAgents: An Empirical Study of Building Effective Agents (2025.findings-emnlp)

Copied to clipboard

Challenge: a recent study shows that agent research practices are far from standard, rigorous . lack of a standard evaluation protocol makes previous works not reproducible, authors say .
Approach: They conduct an empirical study on the GAIA benchmark to investigate agent design choices . they find that lack of a standard evaluation protocol makes previous works not reproducible .
Outcome: The proposed framework achieves state-of-the-art performance among open-source projects.
Making RALM Robust to Irrelevant Contexts via Layer Knowledge Guided Attention (2025.findings-acl)

Copied to clipboard

Challenge: Large language models (LLMs) face factual hallucination and knowledge obsolescence when tackling knowledge-intensive tasks.
Approach: They propose a layer-knowledge guided attention method which harnesses the layer-wise knowledge of large language models to optimize per-layer attention on useful passages.
Outcome: The proposed method outperforms existing methods on RALM benchmarks.
xLAM: A Family of Large Action Models to Empower AI Agent Systems (2025.naacl-long)

Copied to clipboard

Challenge: Autonomous agents powered by large language models (LLMs) have attracted significant research interest, but there are few standards for developing specialized models for agent tasks.
Approach: They propose a series of large action models with dense and mixture-of-expert architectures that unifies, augments, and synthesizes diverse datasets to enhance agent generalizability and performance.
Outcome: The proposed models outperform GPT-4, Claude-3, and many other models in terms of tool use and outperformed GPT-based models on multiple agent ability benchmarks.
i-Code Studio: A Configurable and Composable Framework for Integrative AI (2024.emnlp-demo)

Copied to clipboard

Challenge: Existing frameworks for Integrative AI lack flexibility and composability to handle multimodal tasks.
Approach: They propose a configurable framework for Integrative AI that orchestrates multiple pre-trained models to conduct complex multimodal tasks.
Outcome: The proposed framework achieves impressive results on zero-shot multimodal tasks . it can communicate and personalize for users, and it can be used in a multimodal agent .
ToolHop: A Query-Driven Benchmark for Evaluating Large Language Models in Multi-Hop Tool Use (2025.acl-long)

Copied to clipboard

Challenge: Effective evaluation of multi-hop tool use is critical for analyzing the understanding, reasoning, and function-calling capabilities of large language models.
Approach: They propose a dataset that provides rigorous evaluation of multi-hop tool use.
Outcome: The proposed model achieves 49.04% accuracy across five model families.
Invisible Prompts, Visible Threats: Malicious Font Injection in External Resources for Large Language Models (2025.findings-emnlp)

Copied to clipboard

Challenge: Large Language Models (LLMs) are increasingly equipped with capabilities of real-time web search and integrated with protocols like the Model Context Protocol (MCP).
Approach: They investigate the vulnerability of Large Language Models to hidden adversarial prompts . they evaluate two critical attack scenarios: malicious content relay and sensitive data leakage .
Outcome: The proposed extension could introduce new security vulnerabilities.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations