Papers by Jiawei Yang

43 papers
RoleCDE: Benchmarking and Mitigating Role–Alignment Trade-offs in Role-Playing Agents (2026.findings-acl)

Copied to clipboard

Challenge: Existing benchmarks for role-playing agents only evaluate surface-level fidelity and provide limited insight into decision making under role–alignment value conflicts.
Approach: They propose a benchmark to evaluate RPAs under role–alignment value conflicts . they use 8k diverse role profiles and 240k dilemma instances to evaluate role-aware decision making .
Outcome: The proposed benchmark covers 8k diverse role profiles and scenarios and nearly 240k dilemma instances across three difficulty levels and eight role categories.
HATA: Trainable and Hardware-Efficient Hash-Aware Top-k Attention for Scalable Large Model Inference (2025.findings-acl)

Copied to clipboard

Challenge: Existing top-k attention methods struggle to strike a balance between efficiency and accuracy.
Approach: They propose a top-k attention approach that integrates low-overhead techniques into the Top-k Attention process to achieve 7.2 speedup compared to vanilla full attention.
Outcome: The proposed approach achieves 7.2 speedup compared to current top-k attention methods while maintaining model accuracy.
Parallax: Visualizing and Understanding the Semantics of Embedding Spaces via Algebraic Formulae (P19-3)

Copied to clipboard

Challenge: Embeddings are a fundamental component of many modern machine learning and natural language processing models.
Approach: They propose a tool for visualizing embedding spaces using parametric projections . they demonstrate the power of Parallax and propose % task-oriented approach .
Outcome: The proposed tool is based on two-dimensional projections without interpretable semantics . it enhances interpretability and allows for more fine-grained analysis .
Towards a Unified Multi-Dimensional Evaluator for Text Generation (2022.emnlp-main)

Copied to clipboard

Challenge: Existing evaluation frameworks for natural language generation are dominated by similarity-based metrics.
Approach: They propose a multi-dimensional evaluator for natural language generation that integrates multiple dimensions into one evaluer.
Outcome: The proposed evaluator improves on three typical NLG tasks and improves with external knowledge.
Refining Source Representations with Relation Networks for Neural Machine Translation (C18-1)

Copied to clipboard

Challenge: Existing neural machine translation frameworks that forget distant information and disregard relationship between source and target words are not effective.
Approach: They propose to use relation networks to learn better representations of the source . they propose to associate source words with each other to help retain their relationships .
Outcome: Experiments show that the proposed approach outperforms the encoder-decoder framework on several datasets.
CMR Scaling Law: Predicting Critical Mixture Ratios for Continual Pre-training of Language Models (2024.emnlp-main)

Copied to clipboard

Challenge: Large Language Models (LLMs) excel in diverse tasks but often underperform in specialized fields due to limited domain-specific or proprietary corpus.
Approach: They propose a power-law relationship between loss, mixture ratio, and training tokens scale and formalize the trade-off between general and domain-specific capabilities.
Outcome: The proposed model achieves the desired domain transfer while maintaining general ability and highest utilization of available resources.
CB-Whisper: Contextual Biasing Whisper Using Open-Vocabulary Keyword-Spotting (2024.lrec-main)

Copied to clipboard

Challenge: End-to-end automatic speech recognition systems struggle to recognize rare name entities such as personal names, organizations and terminologies that are not frequently encountered in the training data.
Approach: They propose a convolutional neural network-based ASR system that performs open-vocabulary keyword-spotting before the decoder to match the features between the entities and the utterances.
Outcome: The proposed system significantly improves mixed-error-rate (MER) and entity recall compared to the original Whisper model on three internal datasets and two publicly available datasets.
Navigating the Infinite Dynamic Web Space: Effective In-Context Exploration via Cognitive Multi-Agent Collaboration (2026.eacl-long)

Copied to clipboard

Challenge: Existing methods for dynamic web navigation rely on greedy strategies or value estimation, struggle to achieve effective backtracking and are heavily dependent on proprietary models.
Approach: They propose a cognitive multi-agent collaboration framework that enhances cyberspace exploration capability through In-Context Exploration.
Outcome: The proposed framework surpasses the proprietary model Claude-3.5 Sonnet on the WebArena benchmark.
DynaMiTE: Discovering Explosive Topic Evolutions with User Guidance (2023.findings-acl)

Copied to clipboard

Challenge: Existing Dynamic topic models are either fully supervised, requiring expensive human annotations, or fully unsupervised, producing topic evolutions that often do not cater to a user’s needs.
Approach: They propose to use a framework that ensembles semantic similarity, category indicative, and time indicative scores to produce informative topic evolutions.
Outcome: The proposed framework can be used to discover topic evolutions from temporal corpora that align with user-provided category names and uniquely capture topics at each time step.
The Shifted and The Overlooked: A Task-oriented Investigation of User-GPT Interactions (2023.emnlp-main)

Copied to clipboard

Challenge: Recent advances in large language models (LLMs) have produced models that exhibit remarkable performance across a variety of NLP tasks.
Approach: They analyze a large-scale collection of user-GPT conversations to identify a significant gap between academic research in NLP and the needs of real-world NLP applications.
Outcome: The proposed model outperforms existing models in a large-scale collection of user-GPT conversations and identifies a significant gap between the tasks that users frequently request from LLMs and the tasks commonly studied in academic research.
VPL: Visual Proxy Learning Framework for Zero-Shot Medical Image Diagnosis (2024.findings-emnlp)

Copied to clipboard

Challenge: Insufficient medical text precision and the modal disparity between text and vision spaces pose challenges for vision-language models like CLIP.
Approach: They propose a visual proxy learning framework that combines a text refinement module and a stable Sinkhorn algorithm to enhance the diagnostic performance.
Outcome: The proposed model outperforms the state-of-the-art CLIP inference by 1.69% to 15.31% on five datasets covering various diseases.
Think Better, Not Longer: Token-Level Marginal Utility for Efficient Reasoning in Large Reasoning Models (2026.acl-long)

Copied to clipboard

Challenge: Large reasoning models (LRMs) generate explicit Chain-of-Thought rationales, but often suffer from "overthinking".
Approach: They propose a unified training framework to synthesize concise reasoning chains by identifying tokens that reduce the model’s likelihood of the correct answer.
Outcome: Experiments on deepSeek-R1-Distill-Qwen backbones show that MUTO yields better efficiency-accuracy Pareto frontier.
Topic-Oriented Open Relation Extraction with A Priori Seed Generation (2024.emnlp-main)

Copied to clipboard

Challenge: Existing methods for open relation extraction give sub-optimal results on specific topics.
Approach: They propose a method that leverages the built-in knowledge of large language models to maintain a dynamic seed relation dictionary for the topic.
Outcome: The proposed approach empowers better topic-oriented control over the generated relations and improves ORE performance along the five dimensions, especially on specialized and narrow topics.
CLaSp: In-Context Layer Skip for Self-Speculative Decoding (2025.acl-long)

Copied to clipboard

Challenge: Existing methods for drafting Large Language Models require additional modules to be trained, which can be challenging to implement and ensure compatibility across various LLMs.
Approach: They propose an in-context layer-skipping strategy for self-speculative decoding that uses a plug-and-play mechanism to skip intermediate layers of the verify model to construct a compressed draft model.
Outcome: The proposed method achieves a speedup of 1.3 1.7 on LLaMA3 series models without altering the original distribution of the generated text.
Incomplete Utterance Rewriting by A Two-Phase Locate-and-Fill Regime (2023.findings-acl)

Copied to clipboard

Challenge: Existing models with incomplete utterances have too large search space, resulting in poor quality of rewriting results.
Approach: They propose a 2-phase rewriting framework which predicts empty slots in the utterance that need to be completed and generates the part to be filled into each position.
Outcome: The proposed framework achieves state-of-the-art results on several public rewriting datasets.
SAIS: Supervising and Augmenting Intermediate Steps for Document-Level Relation Extraction (2022.naacl-main)

Copied to clipboard

Challenge: Existing methods for relation extraction only implicitly learn to model relevant contexts and entity types while being trained for RE.
Approach: They propose to explicitly teach the model to capture relevant contexts and entity types by supervising and augmenting intermediate steps (SAIS) for RE.
Outcome: The proposed method outperforms the runner-up method on three benchmarks by 5.04% . textual contexts and entity types are the major information sources that lead to the success of previous approaches.
Grounding Agent Memory in Contextual Intent (2026.findings-acl)

Copied to clipboard

Challenge: Large language models are deployed in long-horizon tasks that require agents to track interleaved goals, resolve references to prior information, and coordinate actions over extended trajectories.
Approach: They propose an agentic memory system that indexes each trajectory step with a structured retrieval cue, contextual intent, and retrieves history by matching the current step’s intent.
Outcome: The proposed system outperforms the strongest benchmark by 35.6%, with the largest gains as trajectory length increases.
Compositional Data Augmentation for Abstractive Conversation Summarization (2023.acl-long)

Copied to clipboard

Challenge: Abstractive conversation summarization systems rely on large-scale annotated summaries, but collecting and annotating these conversations can be time-consuming and labor-intensive.
Approach: They propose a method for generating diverse and high-quality pairs of conversations and summaries by extracting conversation structures and organizing meaningful conversation snippets.
Outcome: The proposed method outperforms baseline methods on SAMSum and DialogSum datasets and achieves a 10% increase in ROUGE scores with limited data.
Fundamental Capabilities of Large Language Models and their Applications in Domain Scenarios: A Survey (2024.acl-long)

Copied to clipboard

Challenge: Existing benchmark-based evaluations cannot accurately reflect the performance of real-world applications.
Approach: They propose a reliable strategy for domains to choose more robust LLMs for real-world applications.
Outcome: The proposed strategy addresses the challenges faced by domains to choose more robust LLMs for real-world applications.
Recall and Learn: A Memory-augmented Solver for Math Word Problems (2021.findings-emnlp)

Copied to clipboard

Challenge: Existing methods for solving math word problems are based on template-based generation which results in limited generalization capability.
Approach: They propose a human-like analogical learning method for the math word problem . it uses modules of memory, representation, analogy, and reasoning to make a new exercise .
Outcome: The proposed method outperforms state-of-the-art models on two well-known datasets.
Diffusion Based Counterfactual Augmentation for Dual Sentiment Classification (2024.lrec-main)

Copied to clipboard

Challenge: Existing methods to augment sentiment models have failed to mitigate spurious association problem inherent in the original data.
Approach: They propose a framework for enhancing sentiment models using an antonymous paradigm and contrastive learning to generate high-quality samples.
Outcome: The proposed framework achieves state-of-the-art performance on four benchmark datasets.
DPN-LE: Dual Personality Neuron Localization and Editing for Large Language Models (2026.findings-acl)

Copied to clipboard

Challenge: Current methods for editing personality traits in large language models can change personalities but reduce performance.
Approach: They propose a novel paradigm for personality editing that locates and edits LLM neurons and enables competitive personality control at inference time.
Outcome: Experiments on LLaMA-3-8B-Instruct and Qwen2.5-7B-instruct show that the proposed approach can improve performance and improve performance.
Scientific Paper Retrieval with LLM-Guided Semantic-Based Ranking (2025.findings-emnlp)

Copied to clipboard

Challenge: Recent studies also use large language models (LLMs) for query understanding, but these methods lack grounding in corpus-specific knowledge and may generate unreliable or unfaithful content.
Approach: They propose a paper retrieval framework that combines large language models (LLMs) with a concept-based semantic index to capture scientific concepts.
Outcome: The proposed framework improves the performance of various base retrievers, surpasses strong existing LLM-based baselines, and remains highly efficient.
AutoBreach: Universal and Adaptive Jailbreaking with Efficient Wordplay-Guided Optimization via Multi-LLMs (2025.findings-naacl)

Copied to clipboard

Challenge: Existing jailbreak research exhibits limitations in universality, validity, and efficiency . Existing methods for jailbreaking LLMs have limited validity and effectiveness .
Approach: They propose a black-box approach that uses wordplay-guided mapping rule sampling to create universal adversarial prompts.
Outcome: The proposed method efficiently identifies security vulnerabilities across various LLMs, achieving an average success rate of over 80% with fewer than 10 queries.
A Game-Theoretica Negotiation Framework for Cross-Cultural Consensus (2026.acl-long)

Copied to clipboard

Challenge: Large language models exhibit pronounced WEIRD cultural bias, marginalizing diverse viewpoints and posing challenges for reconciling diverse populations with varying cultural backgrounds and value systems.
Approach: They propose a framework for cross-cultural fairness using a Nash Equilibrium . they propose equilibriums that iteratively propose and refine natural-language guidelines .
Outcome: The proposed framework generates higher-quality and more balanced consensus . it finetunes diverse LLM architectures with negotiation data, reducing cultural distances by 95.53%.
RAG4ITOps: A Supervised Fine-Tunable and Comprehensive RAG Framework for IT Operations and Maintenance (2024.emnlp-industry)

Copied to clipboard

Challenge: Large Language Models (LLMs) have improved the open-domain QA’s performance, but how to efficiently handle enterprise-exclusive corpora and build domain-specific QA systems are still not studied for industrial applications.
Approach: They propose a general and comprehensive framework based on Retrieval Augmented Generation (RAG) and facilitate the whole business process of establishing QA systems for IT operations and maintenance.
Outcome: The proposed framework achieves superior results on two kinds of QA tasks.
From Behavioral Performance to Internal Competence: Interpreting Vision-Language Models with VLM-Lens (2025.emnlp-demos)

Copied to clipboard

Challenge: Existing vision-language models are based on exactmatch based accuracy and its derivations to evaluate performance.
Approach: They propose a toolkit that supports systematic benchmarking, analysis, and interpretation of vision-language models by extracting intermediate outputs from any layer during the forward pass of open-source VLMs.
Outcome: The proposed toolkit supports 16 state-of-the-art base VLMs and their over 30 variants and is extensible to accommodate new models without changing the core logic.
Transforming Visual Scene Graphs to Image Captions (2023.acl-long)

Copied to clipboard

Challenge: Existing approaches to generate captions using image captioning are based on multi-head attention (MHA)
Approach: They propose to transform scene graphs into more descriptive captions by using multi-head attention to build a Graph Neural Network (GNN) . they construct a Mixture-of-Expert (MOE)-based decoder where each expert is built on MHA for discriminating the graph embeddings to generate different kinds of words.
Outcome: The proposed framework can generate captions from multiple visual features and objects . it is based on a mixture-of-expert (MOE)-based decoder based upon MHA .
Red Teaming Large Reasoning Models (2026.acl-long)

Copied to clipboard

Challenge: Large Reasoning Models (LRMs) have emerged as a powerful advancement in multi-step reasoning tasks, but they introduce safety and reliability risks, such as CoT-hijacking and prompt-induced inefficiencies.
Approach: They propose a unified benchmark to assess the trustworthiness of Large Reasoning Models.
Outcome: The proposed benchmark evaluates truthfulness, safety and efficiency on 26 models.
From Language to Driving: A Dual-Loop SLM-Enhanced Framework for Multi-Planner Scheduling via a Domain-Specific Language (2026.acl-long)

Copied to clipboard

Challenge: Recent large language model-based AD research offers new avenues to address this challenge.
Approach: They propose a small language model (SLM) for high-level semantic reasoning and schedule generation, while an inner loop performs low-level, high-frequency schedule execution and vehicle control.
Outcome: The proposed framework improves instruction completion rates while maintaining high safety and compliance relative to multiple baselines.
PSST: A Benchmark for Evaluation-driven Text Public-Speaking Style Transfer (2024.findings-emnlp)

Copied to clipboard

Challenge: Existing methods to transfer text style focus on sentence-level data, limiting performance . current LLMs struggle to generate public speaking texts that align with human preferences .
Approach: They propose a task to transform official texts into public-speaking styles by analyzing real-world data.
Outcome: The proposed task aims to transform public speaking texts into public-speaking styles . the proposed framework analyzes characteristics and identifies problems of stylized texts .
CMIG: Conceptual Metaphor Theory-Inspired Framework for Metaphorical Image Generation (2026.findings-acl)

Copied to clipboard

Challenge: Existing text-to-image systems often produce visually plausible but semantically literal outputs.
Approach: They propose a structured prompting framework inspired by Conceptual Metaphor Theory . they propose to identify source–target mappings, filter projectable source attributes and select a visual realization strategy in a reproducible reasoning workflow.
Outcome: The proposed framework improves semantic alignment and controllability on metaphor prompts.
PibE-MPP: A Play-it-by-Ear Masking Performance Plug-in for LLMs (2026.findings-acl)

Copied to clipboard

Challenge: Random masking is a widely adopted classic baseline in large language models (LLMs).
Approach: They propose a play-it-by-ear masking performance plug-in which enables LLMs to adaptively select masking target combinations for each task.
Outcome: The proposed performance plug-in retains the advantages and mitigates the drawbacks of random masking in large language models.
CoLLiE: Collaborative Training of Large Language Models in an Efficient Way (2023.emnlp-demo)

Copied to clipboard

Challenge: Large language models (LLMs) are increasingly pivotal in a wide range of tasks . however, the resources required for training these models necessitate efficient solutions .
Approach: They propose a library that facilitates collaborative training of large language models . they use 3D parallelism, parameter-efficient fine-tuning methods and optimizers .
Outcome: The proposed library has proven superior training efficiency in comparison with prevalent solutions in pre-training and fine-tuning scenarios.
Generative Data Augmentation with Contrastive Learning for Zero-Shot Stance Detection (2022.emnlp-main)

Copied to clipboard

Challenge: Existing methods for zero-shot stance detection are labor-intensive to train for each new target.
Approach: They propose a generative data augmentation approach to generate training samples containing unseen and seen targets and map them into the same embedding space with contrastive learning.
Outcome: The proposed model achieves state-of-the-art on most topics in the task of zero-shot stance detection.
PSP: Pre-trained Soft Prompts for Few-Shot Abstractive Summarization (2022.coling-1)

Copied to clipboard

Challenge: Experimental results show that our method outperforms full-model tuning in few-shot abstractive summarization tasks.
Approach: They propose a soft prompts architecture with prompt pre-training and prompt fine-tuning paradigm to support few-shot abstractive summarization.
Outcome: The proposed model outperforms Prompt Tuning and Profix-Tuning on CNN/DailyMail and XSum datasets and outperfies Profix Tuning by a large margin.
Incorporating Latent Meanings of Morphological Compositions to Enhance Word Embeddings (P18-1)

Copied to clipboard

Challenge: Existing word embedding methods learn semantic information at word level while neglecting meaningful inner structures of words like morphemes.
Approach: They propose to use latent meanings of morphological compositions of words to train word embeddings.
Outcome: The proposed models outperform baseline models on word similarity, syntactic analogy and text classification tasks.
Unsupervised Multi-Granularity Summarization (2022.findings-emnlp)

Copied to clipboard

Challenge: Experimental results confirm the substantial superiority of GranuSum on multi-granularity summarization over strong baselines.
Approach: They propose to rank events by their salience and annotate a benchmark for GranuSum that contains multiple summaries at different granularities for each document cluster.
Outcome: The proposed framework is capable of producing multi-granular summaries in unsupervised manner over strong baselines.
Calibrating Pseudo-Labeling with Class Distribution for Semi-supervised Text Classification (2025.emnlp-main)

Copied to clipboard

Challenge: Existing studies develop effective pseudo-labeling methods, but they struggle with unlabeled data that have imbalanced classes mismatched with the labeled data.
Approach: They propose to use pseudo-labeling to train text classification models with few labeled data and massive unlabeled data.
Outcome: Empirical results show that the proposed model outperforms state-of-the-art methods on 3 common benchmarks.
Unveiling and Addressing Pseudo Forgetting in Large Language Models (2025.findings-acl)

Copied to clipboard

Challenge: Existing efforts to mitigate catastrophic forgetting in continual learning have not been studied.
Approach: They propose a rationale-guided replay framework that allows models to leverage their capabilities and provide partial external correct rationales to the original instructions.
Outcome: The proposed framework mitigates pseudo forgetting while maintaining model plasticity.
LIME: Less Is More for MLLM Evaluation (2025.findings-acl)

Copied to clipboard

Challenge: Existing MLLM benchmarks and unified evaluation frameworks cannot accurately and efficiently reflect the ability of MLMLs.
Approach: They propose a semi-automated benchmark curated using a pipeline that filters out uninformative samples and eliminates answer leakage by focusing on tasks that require image-based understanding.
Outcome: The proposed benchmark reduces the number of samples by 76% and evaluation time by 77% while it can more effectively distinguish different models’ abilities.
SampleMix: A Sample-wise Pre-training Data Mixing Strategy by Coordinating Data Quality and Diversity (2025.findings-emnlp)

Copied to clipboard

Challenge: Existing methods for pretraining data mixing for large language models neglect significant inter-domain overlaps and commonalities, failing to control the global diversity of the constructed training dataset.
Approach: They propose a sample-wise data mixture approach that performs global cross-domain sampling by systematically evaluating the quality and diversity of each sample.
Outcome: The proposed method exceeds existing domain-based methods in multiple downstream tasks and perplexity assessments.
SafeRAG: Benchmarking Security in Retrieval-Augmented Generation of Large Language Model (2025.acl-long)

Copied to clipboard

Challenge: Existing approaches to integrating external knowledge into large language models (LLMs) however, the incorporation of external knowledge increases the vulnerability of LLMs .
Approach: They propose a benchmark to evaluate the RAG security using a dataset . they classify attack tasks into silver noise, inter-context conflict, soft ad, and white Denial-of-Service .
Outcome: The proposed benchmark evaluates the security of RAG against 14 representative RAG components.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations