Papers by Qu Yang

29 papers
OmniCharacter: Towards Immersive Role-Playing Agents with Seamless Speech-Language Personality Interaction (2025.acl-long)

Copied to clipboard

Challenge: Existing methods focus on replicating dialogues in textual form, neglecting the role’s voice traits as a crucial effect in interaction, which tends to be more immersive experiences in realistic scenarios.
Approach: They propose a first seamless speech-language personality interaction model to achieve immersive RPAs with low latency.
Outcome: The proposed model exhibits role-specific personality traits and vocal traits throughout the interaction, enabling a mixture of speech and language responses.
AgentCourt: Simulating Court with Adversarial Evolvable Lawyer Agents (2025.findings-acl)

Copied to clipboard

Challenge: Existing legal language models struggle with dynamic courtroom interactions, resulting in overfitting to standardized legal tasks.
Approach: They propose a new adversarial evolutionary approach for agents that performs dynamic knowledge learning and evolution through structured adversarials in a simulated courtroom program.
Outcome: The proposed approach outperforms existing LLM-based models in three critical dimensions: cognitive agility, professional knowledge, and logical rigor.
Learn Like Humans: Use Meta-cognitive Reflection for Efficient Self-Improvement (2026.acl-long)

Copied to clipboard

Challenge: Existing self-improving frameworks rely on inefficient, multi-turn recursive loops that incur high computational costs.
Approach: They propose a framework that achieves efficient self-evolution within a single recurrence cycle.
Outcome: The proposed framework outperforms state-of-the-art self-evolving systems while significantly reducing computational overhead.
An Empirical Study of LLM-as-a-Judge for LLM Evaluation: Fine-tuned Judge Model is not a General Substitute for GPT-4 (2025.findings-acl)

Copied to clipboard

Challenge: Recent studies have fine-tuned judge models based on open-source LLMs to evaluate the quality of other LLM.
Approach: They propose to use open-source LLMs to evaluate Large Language Models (LLMs) their empirical results show that the models underperform GPT-4 in several dimensions .
Outcome: The proposed models outperform GPT-4 on several dimensions including generalizability, fairness and adaptability.
Towards Probing Speech-Specific Risks in Large Multimodal Models: A Taxonomy, Benchmark, and Insights (2024.emnlp-main)

Copied to clipboard

Challenge: Large Multimodal Models have demonstrated a strong capability to understand multimodal information and interact with human users.
Approach: They propose a speech-specific risk taxonomy to assess LMMs' ability to detect high-risk interactions in multimodal settings.
Outcome: The proposed model is based on a speech-specific risk taxonomy covering 8 risk categories . it shows that the models are ineffective in detecting paralinguistic-specific risks in speech .
LLM-Enhanced Self-Evolving Reinforcement Learning for Multi-Step E-Commerce Payment Fraud Risk Detection (2025.acl-industry)

Copied to clipboard

Challenge: e-commerce payment fraud detection is a new area for reinforcement learning (RL) and Large Language Models (LLMs).
Approach: They propose to integrate reinforcement learning (RL) with Large Language Models (LLMs) by framing transaction risk as a multi-step Markov Decision Process (MDP), RL optimizes risk detection across multiple payment stages.
Outcome: The proposed approach improves fraud detection accuracy and demonstrates zero-shot capability.
Evaluating Memory Capability in Continuous Lifelog Scenario (2026.findings-acl)

Copied to clipboard

Challenge: Existing benchmarks focus on online one-on-one chatting or human-AI interactions, neglecting real-world scenarios.
Approach: They propose a framework to curate a lifelog benchmark that combines two subsets of audio data to address temporal leakage in offline settings.
Outcome: The proposed framework outperforms existing benchmarks on live chats and AI interactions.
Social Welfare Function Leaderboard: On the Emergence of LLM Agents as the Welfare Dictator (2026.findings-acl)

Copied to clipboard

Challenge: Large language models (LLMs) are increasingly entrusted with high-stakes decisions that affect human welfare.
Approach: They evaluate 20 state-of-the-art Large language models (LLMs) and 20 LLM dictators to create a social welfare function benchmark.
Outcome: The proposed model creates dilemma between maximizing collective efficiency and ensuring distributive fairness.
Beyond Quantity: Trajectory Diversity Scaling for Code Agents (2026.findings-acl)

Copied to clipboard

Challenge: Code large language models (LLMs) are becoming tool-interactive agents . quantity-centric scaling exhibits an early bottleneck that underutilizes trajectory data . et al.: a new approach to scale trajectory diversity improves tool-use generalization .
Approach: They propose a Trajectory Diversity Scaling-based data synthesis framework for code agents that scales performance through diversity rather than raw volume.
Outcome: Experiments on general tool-use benchmarks and code agent tasks show that TDScaling improves tool-user generalization and inherent coding proficiency.
FedPETuning: When Federated Learning Meets the Parameter-Efficient Tuning Methods of Pre-trained Language Models (2023.findings-acl)

Copied to clipboard

Challenge: Existing research on federated learning (FL) for pre-trained language models (PLMs) with increasing concerns about data privacy, enterprises or institutions are not allowed to collect data from end devices or local clients to a centralized server for fine-tuning PLMs.
Approach: They investigate the parameter-efficient tuning of pre-trained language models (PLMs) and develop a federated benchmark for four representative PETuning methods .
Outcome: The proposed method can defend against privacy attacks and maintain acceptable performance with reducing heavy resource consumption.
Too Good to be Bad: On the Failure of LLMs to Role-Play Villains (2026.findings-acl)

Copied to clipboard

Challenge: Large Language Models (LLMs) are increasingly tasked with creative generation, but their ability to portray non-prosocial, antagonistic personas remains largely unexamined.
Approach: They propose a moral alignment benchmark to test the safety of large language models . they find that models struggle with traits directly antithetical to safety principles .
Outcome: The proposed model fails to accurately portray morally ambiguous or villainous characters . the model fails most with traits directly antithetical to safety principles .
Adversarial Category Alignment Network for Cross-domain Sentiment Classification (N19-1)

Copied to clipboard

Challenge: Existing methods for cross-domain sentiment classification focus on aligning marginal distribution without taking category-specific decision boundaries into consideration.
Approach: They propose an adversarial category alignment network to enhance category consistency . experimental results show the proposed method can achieve state-of-the-art performance .
Outcome: The proposed method achieves state-of-the-art performance and produces more discriminative features on benchmark datasets.
Hierarchical-Task-Aware Multi-modal Mixture of Incremental LoRA Experts for Embodied Continual Learning (2025.acl-long)

Copied to clipboard

Challenge: Existing continual learning setups for embodied intelligence focus on executing low-level actions, neglecting the ability to learn high-level planning and multi-level knowledge.
Approach: They propose a Hierarchical Embodied Continual Learning Setups (HEC) that divides the agent’s continual learning process into two layers: high-level instructions and low-level actions.
Outcome: The proposed method reduces the forgetting of old tasks compared to other methods, while orthogonally training the remaining parts.
Can MLLMs Understand the Deep Implication Behind Chinese Images? (2025.acl-long)

Copied to clipboard

Challenge: MLLMs perform poorly on traditional culture images, indicating limitations in understanding high-level semantics and lacking a deep knowledge base of Chinese traditional culture.
Approach: They propose to use Chinese images to assess MLLMs' higher-order perception and understanding of Chinese visual content.
Outcome: The proposed model incorporates images that represent Chinese traditional culture, such as famous Chinese traditional paintings, to ensure the authenticity of the Chinese context.
Aspect and Sentiment Aware Abstractive Review Summarization (C18-1)

Copied to clipboard

Challenge: Abstractive summarization is a task that generates short and concise summaries of user generated reviews.
Approach: They propose an interactive attention mechanism to learn the representations of context and aspect words within reviews, acted as an encoder.
Outcome: The proposed model achieves impressive results compared to other strong competitors on a real-life dataset.
Large Dual Encoders Are Generalizable Retrievers (2022.emnlp-main)

Copied to clipboard

Challenge: Experimental results show that dual encoders outperform sparse and dense retrievers on the BEIR dataset significantly.
Approach: They challenge belief that bottleneck layer is too limited for out-of-domain generalization . they scale up the model while keeping bottleneck as a single dot-product with a fixed size .
Outcome: The proposed model outperforms sparse and dense retrievers on the BEIR dataset significantly.
COIG-P: A High-Quality and Large-Scale Chinese Preference Dataset for Alignment with Human Values (2026.findings-eacl)

Copied to clipboard

Challenge: Existing Chinese preference datasets suffer from limited scale, restricted domain coverage, and insufficiently rigorous data validation.
Approach: They propose an LLM-based data annotation pipeline with no human intervention to annotate Chinese preference datasets.
Outcome: The proposed pipeline outperforms existing Chinese preference datasets on AlignBench and Chinese Reward Benchmark.
FCGCL: Fine- and Coarse-Granularity Contrastive Learning for Speech Translation (2022.findings-emnlp)

Copied to clipboard

Challenge: Existing methods to perform implicit knowledge transfer from machine translation to ST model are difficult because of the task complexity and data scarcity.
Approach: They recommend a method which conducts explicit knowledge transfer from MT to ST model by fine and coarse granularity contrastive learning.
Outcome: The proposed method improves the performance of the end-to-end speech translation model on all 8 languages.
Reshaping Representation Space to Balance the Safety and Over-rejection in Large Audio Language Models (2025.emnlp-main)

Copied to clipboard

Challenge: Large Audio Language Models (LALMs) have demonstrated unprecedented capabilities in natural language understanding and generation, revolutionizing human-machine dialogue.
Approach: They propose an unsupervised safety-fine-tuning strategy that reshapes LALMs representation space to enhance existing LALM safety-alignment while balancing the risk of over-rejection.
Outcome: The proposed approach improves LALMs safety under three input conditions while increasing over-rejection rate by only 0.88% on average.
Treble Counterfactual VLMs: A Causal Approach to Hallucination (2025.findings-emnlp)

Copied to clipboard

Challenge: Existing studies link hallucination to data or representation biases, but their causal origins remain unclear.
Approach: They propose a causal framework to analyze and mitigate hallucination in vision-language models by using counterfactual analysis to estimate the Natural Direct Effect (NDE) of each modality and their interaction.
Outcome: The proposed framework significantly reduces hallucination while preserving task performance while retaining reliability.
MetaMixSpeech: Meta Task Augmentation for Low-Resource Speech Recognition (2025.findings-emnlp)

Copied to clipboard

Challenge: Meta-learning has proven to be a powerful paradigm for improving speech recognition performance . however, multilingual meta learning also faces challenges such as task overfitting and learner overfit .
Approach: a new method is proposed to augment meta-training tasks with "more data" the method incorporates both support and query augmentations .
Outcome: The proposed method achieves a 6.35% improvement in the word error rate on FLEURS and Common Voice datasets.
Meta-Adapter for Self-Supervised Speech Models: A Solution to Low-Resource Speech Recognition Challenges (2024.lrec-main)

Copied to clipboard

Challenge: Existing self-supervised learning models can learn latent representations from large amounts of unlabeled data, but they are expensive to fine-tune.
Approach: They develop a meta-adapter to obtain meta-initialized parameters for self-supervised models . meta-Adapters show better generalization and extensibility than traditional pretraining methods .
Outcome: Experiments on common voice and FLEURS datasets show Meta-Adapter performs better on low-resource languages . authors show it can be used on 12 low-source languages, but it requires huge computational resources .
GaLa: Hypergraph-Guided Visual Language Models for Procedural Planning (2026.findings-acl)

Copied to clipboard

Challenge: Existing methods for procedural planning over-rely on visual inputs and lack structured semantic information.
Approach: They propose a vision–language framework for multimodal procedural planning that exploits implicit spatial relations and deep semantics encoded in object attributes.
Outcome: The proposed framework outperforms existing methods in terms of execution success rate, LCS, and planning correctness.
QuantumQA: Enhancing Scientific Reasoning via Physics-Consistent Dataset and Verification-Aware Reinforcement Learning (2026.acl-long)

Copied to clipboard

Challenge: Large language models lack reliability in scientific domains that require strict adherence to physical constraints.
Approach: They propose a large-scale dataset constructed via a task-adaptive strategy and a hybrid verification protocol that combines deterministic solvers with semantic auditing to guarantee scientific rigor.
Outcome: The proposed model outperforms baselines and general-purpose preference models and is competitive with proprietary models.
Self-Evaluation of Large Language Model based on Glass-box Features (2024.findings-emnlp)

Copied to clipboard

Challenge: Existing evaluation methods rely on external evaluators, focusing on training and prompting strategies, but model-aware glass-box features are overlooked.
Approach: They propose to use model-aware glass-box features to evaluate an LLM's output.
Outcome: The proposed model-aware features are reliable quality indicators for self-evaluation on public benchmarks.
Audio Is the Achilles’ Heel: Red Teaming Audio Large Multimodal Models (2025.naacl-long)

Copied to clipboard

Challenge: Large Language Models (LMMs) have demonstrated ability to interact with humans through text . however, safety of audio LMMs remains under-explored .
Approach: They red team the safety of five audio LMMs under three settings . they find that audio Lmms suffer an average attack success rate of 69.14% on harmful questions .
Outcome: a new study shows that audio LMMs suffer an average success rate on harmful questions . the authors also show that the models exhibit safety vulnerabilities when distracted .
LIME: Less Is More for MLLM Evaluation (2025.findings-acl)

Copied to clipboard

Challenge: Existing MLLM benchmarks and unified evaluation frameworks cannot accurately and efficiently reflect the ability of MLMLs.
Approach: They propose a semi-automated benchmark curated using a pipeline that filters out uninformative samples and eliminates answer leakage by focusing on tasks that require image-based understanding.
Outcome: The proposed benchmark reduces the number of samples by 76% and evaluation time by 77% while it can more effectively distinguish different models’ abilities.
Learning From the Source Document: Unsupervised Abstractive Summarization (2022.findings-emnlp)

Copied to clipboard

Challenge: Existing methods for abstractive summarization are under supervised training, but obtaining high-quality and large-scale datasets for supervised learning is laboriously difficult.
Approach: They propose an unsupervised method that leverages contrastive learning to generate summaries by rewriting and paraphrasing the source documents to generate good summary.
Outcome: The proposed method outperforms baseline methods on extensive experiments on source documents and fake documents.
BatonVoice: An Operationalist Framework for Enhancing Controllable Speech Synthesis with Linguistic Intelligence from LLMs (2026.acl-long)

Copied to clipboard

Challenge: Existing approaches often fail to leverage the linguistic intelligence of Large Language Models (LLMs) Existing models lack the ability to follow text instructions for controllable Text-to-Speech (TTS).
Approach: They propose a framework where an LLM acts as a conductor, understanding user instructions and generating a textual plan - explicit vocal features.
Outcome: The proposed model outperforms open- and closed-source models in speech synthesis and achieves zero-shot cross-lingual generalization.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations