Papers by Yue Jiang

45 papers
UniCodec: Unified Audio Codec with Single Domain-Adaptive Codebook (2025.acl-long)

Copied to clipboard

Challenge: Existing neural audio codecs are not capable of handling multi-domain audio data . et al., 2023) integrate speech modality with text-based large language models .
Approach: They propose a unified audio codec with a single codebook to support multi-domain audio data . they propose combining a mix-of-experts strategy and a partitioned domain-adaptive codebook method .
Outcome: The proposed codec outperforms existing codecs on acoustic and semantic representation capabilities.
TC–RAG: Turing–Complete RAG’s Case study on Medical LLM Systems (2025.acl-long)

Copied to clipboard

Challenge: Existing approaches to RAG neglect system state variables, resulting in poor performance and erroneous knowledge accumulation.
Approach: They propose a framework that incorporates a Turing Complete System to manage state variables and manage retrieval halting.
Outcome: The proposed framework improves on seven real-world healthcare datasets and shows that it is more accurate than existing methods.
Small Models Struggle to Learn from Strong Reasoners (2025.findings-acl)

Copied to clipboard

Challenge: a small learning gap exists between large and small language models . long CoT data and large model responses are not beneficial for small models - a problem that may be due to the small student model's ability to handle distribution shifts.
Approach: They propose a mix distillation strategy that balances reasoning complexity by combining long and short CoT examples or reasoning from both larger and smaller models.
Outcome: The proposed strategy outperforms training on large and small models on short CoT and small model CoT.
RoseRAG: Robust Retrieval-augmented Generation with Small-scale LLMs via Margin-aware Preference Optimization (2025.findings-acl)

Copied to clipboard

Challenge: Large language models (LLMs) have impressive performance but face high computational costs and latency, limiting their deployment in resource-constrained settings.
Approach: They propose a robust RAG framework for large language models via Margin-aware Preference Optimization to enhance the accuracy and reliability of SLMs.
Outcome: The proposed framework surpasses state-of-the-art benchmarks on three open-domain question answering tasks.
3DS: Medical Domain Adaptation of LLMs via Decomposed Difficulty-based Data Selection (2025.emnlp-main)

Copied to clipboard

Challenge: Effective domain adaptation typically involves supervised fine-tuning on carefully selected instruction-tuned data.
Approach: They propose a model-centric data selection framework that aligns data selection with the model’s knowledge distribution to improve model performance.
Outcome: The proposed framework outperforms existing methods by up to 2.97% accuracy in the healthcare domain.
Muffin or Chihuahua? Challenging Multimodal Large Language Models with Multipanel VQA (2024.acl-long)

Copied to clipboard

Challenge: Multipanel images are a common form of visual representations, and humans can achieve approximately 99% accuracy on these questions.
Approach: They propose a benchmark that tests multipanel visual reasoning models with 6,600 triplets of questions, answers, and multipanel images.
Outcome: The proposed benchmark features 6,600 triplets of questions, answers, and multipanel images that challenge state-of-the-art Multimodal Large Language Models (MLLMs) human users can attain approximately 99% accuracy on these questions, compared with previous benchmarks.
InquireMobile: Teaching VLM-based Mobile Agent to Request Human Assistance via Reinforcement Fine-Tuning (2026.acl-long)

Copied to clipboard

Challenge: Recent advances in Vision-Language Models (VLMs) have enabled mobile agents to perceive and interact with real-world mobile environments based on human instructions.
Approach: They propose a vision-language model that actively seeks human confirmation at critical decision points and a model inspired by reinforcement learning.
Outcome: The proposed model achieves an improvement of 46.8% in inquiry success rate and the best overall success rate among existing baselines on InquireBench.
NaSGEC: a Multi-Domain Chinese Grammatical Error Correction Dataset from Native Speaker Texts (2023.findings-acl)

Copied to clipboard

Challenge: Recent studies on Chinese grammatical error correction focus on learning essays.
Approach: They propose a Chinese grammatical error correction dataset that annotates multiple references for 12,500 sentences from three native domains.
Outcome: The proposed dataset can be used to facilitate research on Chinese grammatical error correction (CGEC) for native speaker texts from multiple domains.
LLM-Based Human-Agent Collaboration and Interaction Systems: A Survey (2026.findings-acl)

Copied to clipboard

Challenge: Recent advances in large language models (LLMs) have sparked growing interest in building fully autonomous agents.
Approach: They propose to integrate human-provided information, feedback, or control into the agent system to enhance system performance, reliability, and safety.
Outcome: The proposed systems improve system performance, reliability, and safety by integrating human-provided information, feedback, or control into the agent system.
HyKGE: A Hypothesis Knowledge Graph Enhanced RAG Framework for Accurate and Reliable Medical LLMs Responses (2025.acl-long)

Copied to clipboard

Challenge: Recent approaches suffer from insufficient and repetitive knowledge retrieval, tedious and time-consuming query parsing, and monotonous knowledge utilization.
Approach: They propose a retrieval-augmented generation framework which leverages LLMs’ powerful reasoning capacity to compensate for the incompleteness of user queries.
Outcome: The proposed framework improves the accuracy and reliability of Large Language Models (LLMs) by combining the rich knowledge of LLMs with Hypothesis Outputs.
A Unified Supervised and Unsupervised Dialogue Topic Segmentation Framework Based on Utterance Pair Modeling (2025.naacl-long)

Copied to clipboard

Challenge: Unsupervised methods for dialogue topic segmentation are difficult to surpass due to short sentences, serious references and non-standard language.
Approach: They propose a method to divide a dialogue into different topic paragraphs to better understand its structure and content.
Outcome: The proposed method achieves the best results on multiple benchmark datasets across different scenarios.
Enhancing Cross-lingual Prompting with Dual Prompt Augmentation (2023.findings-acl)

Copied to clipboard

Challenge: a recent study shows that prompting is superior for multilingual/cross-lingual problems . despite its effectiveness on English tasks, its potential for cross-lingual problem is under-explored .
Approach: They propose a framework for prompting that can be used to augment cross-lingual prompts.
Outcome: The proposed framework achieves 46.54% with only 16 English training examples per class, significantly better than fine-tuning.
HiddenDetect: Detecting Jailbreak Attacks against Multimodal Large Language Models via Monitoring Hidden States (2025.acl-long)

Copied to clipboard

Challenge: Existing studies focus on posthoc alignment techniques, but the underlying safety mechanisms within LVLMs remain unexplored.
Approach: They propose a tuning-free framework that leverages internal activations to enhance safety.
Outcome: The proposed framework outperforms state-of-the-art methods in detecting jailbreak attacks against large vision-language models.
Data-Efficient Selection via Grammatical Complexity in Continual Pre-training of Domain-Specific LLMs (2025.emnlp-main)

Copied to clipboard

Challenge: Existing data selection strategies for continual pre-training of large language models often rely on scarce labeled data or computationally expensive LLMs.
Approach: They propose an annotation-independent data selection framework for CPT that evaluates grammatical complexity using lexical diversity and syntactic complexity.
Outcome: The proposed framework outperforms baselines on a financial dataset and surpasses full-data training by 1.7% using only 20% of the data.
Aerial Vision-and-Dialog Navigation (2023.findings-acl)

Copied to clipboard

Challenge: Aerial visionand-dialling navigation (AVDN) is a new approach to autonomous drones that can converse with humans and follow natural language commands to complete tasks.
Approach: They propose to use Aerial Visionand-Dialog Navigation (AVDN) to navigate a drone via natural language conversation by collecting a dataset of over 3k recorded navigation trajectories with asynchronous human-human dialogs between commanders and followers.
Outcome: The proposed system can converse with humans and follow natural language commands to fly to the expected destination.
Convert Language Model into a Value-based Strategic Planner (2025.acl-industry)

Copied to clipboard

Challenge: Emotional support conversation (ESC) aims to alleviate the emotional distress of individuals through effective conversations.
Approach: They propose a framework that bootstraps the planning during ESC and determines the optimal strategy based on long-term returns.
Outcome: The proposed framework outperforms baseline models on ESC datasets and can be used to guide the LLM to response.
LLMEval-Fair: A Large-Scale Longitudinal Study on Robust and Fair Evaluation of Large Language Models (2026.acl-long)

Copied to clipboard

Challenge: Existing evaluation of Large Language Models on static benchmarks is vulnerable to data contamination and leaderboard overfitting.
Approach: LLMEval-Fair framework provides a framework for dynamic evaluation of Large Language Models . evaluators use a proprietary bank of 220k graduate-level questions to analyze model data .
Outcome: LLMEval-Fair provides robust and credible evaluation framework for Large Language Models . it provides a strong empirical validation for the dynamic evaluation paradigm .
ModRWKV: Transformer Multimodality in Linear Time (2025.emnlp-main)

Copied to clipboard

Challenge: Currently, multimodal studies are based on large language models with quadratic-complexity Transformer architectures.
Approach: They propose a decoupled multimodal framework built upon the RWKV7 architecture as its LLM backbone and a lightweight architecture to achieve multi-source information fusion.
Outcome: The proposed framework achieves multi-source information fusion through dynamically adaptable heterogeneous modality encoders.
Opinion Tree Parsing for Aspect-based Sentiment Analysis (2023.findings-acl)

Copied to clipboard

Challenge: Existing generative models for aspect-based sentiment analysis lack structure well-formedness guarantees and built-in elements alignments.
Approach: They propose an opinion tree parsing model which parses all sentiment elements from an opinion-tree.
Outcome: The proposed model is much faster than previous models and can explore correlations among sentiment elements.
From Curated Data to Scalable Models: Continual Pre-training of Dense and MoE Large Language Models for Tibetan (2026.acl-long)

Copied to clipboard

Challenge: Large language models have achieved remarkable success across a wide range of tasks, yet their performance remains heavily biased toward high-resource languages.
Approach: They propose a pipeline for advancing Tibetan language modeling through multilingual continual pre-training with Tibetan, Chinese, and English.
Outcome: The proposed model outperforms open-source and Tibetan-focused models on diverse tasks.
BLADE: Benchmarking Language Model Agents for Data-Driven Science (2024.findings-emnlp)

Copied to clipboard

Challenge: Language model-based agents can be used to conduct and support data-driven science, but evaluating them on open-ended tasks is challenging due to multiple valid approaches, partially correct steps, and different ways to express the same decisions.
Approach: They propose a benchmark to automatically evaluate agents’ multifaceted approaches to open-ended research questions.
Outcome: BLADE evaluates agents’ multifaceted approaches to open-ended research questions using data from 12 datasets and research questions drawn from existing scientific literature.
Sentient Agent as a Judge: Evaluating Higher-Order Social Cognition in Large Language Models (2026.findings-acl)

Copied to clipboard

Challenge: Large language models (LLMs) have evolved from statistical sequence predictors to sophisticated autonomous agents capable of reasoning, planning, and sustaining multi-turn conversa-tions.
Approach: They propose a system that instantiates a "Sentient Agent" that simulates human-like emotional changes and inner thoughts to provide a more realistic evaluation of the model in multi-turn conversations.
Outcome: The proposed framework measures the agent's higher-order social cognition in multi-turn conversations.
Dream to Chat: Model-based Reinforcement Learning on Dialogues with User Belief Modeling (2025.findings-emnlp)

Copied to clipboard

Challenge: a framework for constructing dialogue world models for natural language tasks is currently lacking.
Approach: They propose a framework that can be used to train a dialogue world model.
Outcome: The proposed framework can predict future utterances and user beliefs . it can achieve state-of-the-art performance on emotion classification and sentiment identification .
From Remembering to Metacognition: Do Existing Benchmarks Accurately Evaluate LLMs? (2025.findings-emnlp)

Copied to clipboard

Challenge: Existing benchmark datasets focus on low-level cognitive tasks while providing limited coverage of higher-level reasoning skills.
Approach: They analyze the cognitive depth of popular LLM benchmarks using Bloom’s Taxonomy to evaluate both the cognitive and knowledge dimensions.
Outcome: The results show that incorporating higher-level cognitive instructions into the current instruction fine-tuning process improves model performance.
Structure-aware Domain Knowledge Injection for Large Language Models (2025.acl-long)

Copied to clipboard

Challenge: Structure-aware Continual Pre-Training (SCPT) and Structure-Aware Supervised Fine-Tuning (SSFT) are two-stage strategies for knowledge injection and alignment that reduces the training corpus needs to 5% while achieving 100% of traditional knowledge injection performance.
Approach: They propose a method to efficiently transform foundation Large Language Models into domain specialists by using two-stage strategies: Structure-aware Continual Pre-Training and Structure-Aware Supervised Fine-Tuning.
Outcome: The proposed method significantly reduces the training corpus needs to a mere 5% while achieving 100% of traditional knowledge injection performance.
Chain-of-Specificity: Enhancing Task-Specific Constraint Adherence in Large Language Models (2025.coling-main)

Copied to clipboard

Challenge: Existing approaches to enhancing large language models fail to emphasize specific constraints and unlock the underlying knowledge.
Approach: They propose a method that emphasizes specific constraints and unlocks knowledge within LLMs by iteratively emphasising on specific constraints.
Outcome: The proposed method outperforms existing methods in enhancing generated content, especially in terms of specificity.
A Survey on MLLM-based Visually Rich Document Understanding: Methods, Challenges, and Emerging Trends (2026.findings-acl)

Copied to clipboard

Challenge: Visually Rich Document Understanding (VRDU) frameworks are a key area of research . early approaches to VRDU relied on manually crafted rules and domain-specific heuristics . conventional deep learning approaches do not integrate the diverse modalities in documents .
Approach: They review recent advances in MLLM-based Visually Rich Document Understanding (VRDU) their findings highlight emerging trends and promising research directions .
Outcome: The proposed frameworks are scalable, reliable, and adaptable, the authors argue . their findings highlight emerging trends and promising research directions .
UrbanLLM: Autonomous Urban Activity Planning and Management with Large Language Models (2024.findings-emnlp)

Copied to clipboard

Challenge: UrbanLLM is a fine-tuned large language model designed to tackle diverse urban problems.
Approach: They propose a fine-tuned large language model to tackle diverse urban problems . UrbanLLM decomposes urban-related queries into manageable sub-tasks .
Outcome: The proposed model outperforms existing models in urban planning and management tasks.
Generalized Supervised Attention for Text Generation (2021.findings-acl)

Copied to clipboard

Challenge: Existing supervised attention methods that use human knowledge to learn better alignments are costly or infeasible.
Approach: They propose a generalized supervised attention method based on quasi alignments that are easier to obtain than ideal alignments.
Outcome: The proposed framework improves generation performance and is robust against errors in attention supervision.
Fine-Tuning Pre-trained Language Model with Weak Supervision: A Contrastive-Regularized Self-Training Approach (2021.naacl-main)

Copied to clipboard

Challenge: Fine-tuned pre-trained language models (LMs) have enormous success in many natural language processing tasks, but they still require excessive labeled data in the fine-tuning stage.
Approach: They propose a framework to enable fine-tuning pre-trained language models with weak supervision without any labeled data.
Outcome: The proposed framework outperforms the strongest baseline and achieves competitive performance with fully-supervised fine-tuning methods.
Read Anywhere Pointed: Layout-aware GUI Screen Reading with Tree-of-Lens Grounding (2024.emnlp-main)

Copied to clipboard

Challenge: Existing models for GUI understanding ignore a key GUI-referring task: screen reading based on user-indicated points.
Approach: They propose a Tree-of-Lens agent that constructs a Hierarchical Layout Tree based on user input points and a GUI screenshot.
Outcome: The proposed agent can interpret the Screen Point-and-Read task on mobile, web, and operating systems.
RiskLab: A Controlled Toolkit for Probing Emergent Risks in LLM-Based Multi-Agent Systems (2026.acl-demo)

Copied to clipboard

Challenge: Recent advances in large language model (LLM) agents have accelerated deployment of multi-agent systems for complex tasks.
Approach: They propose an open-source toolkit for instantiating, probing, and measuring emergent risks in LLM-based multi-agent systems under controlled conditions.
Outcome: The proposed toolkit is based on a structured topology–environment–protocol–agent–task quintuple enabling reproducible studies of how communication structure, coordination mechanisms, and incentives shape system-level risks.
Cross-Lingual Pitfalls: Automatic Probing Cross-Lingual Weakness of Multilingual Large Language Models (2025.acl-long)

Copied to clipboard

Challenge: Large language models have achieved remarkable success in Natural Language Processing, yet their cross-lingual consistency remains a significant challenge.
Approach: They propose a method to identify cross-lingual weaknesses in Large Language Models . they construct bilingual question pairs that expose performance discrepancies between English and target languages .
Outcome: The proposed method uncovers over 50% accuracy drops in target languages across models.
EmoAgent: Assessing and Safeguarding Human-AI Interaction for Mental Health Safety (2025.emnlp-main)

Copied to clipboard

Challenge: EmoAgent evaluates and mitigates mental health hazards in human-AI interactions, especially for vulnerable human users with psychological disorders.
Approach: EmoAgent is a multi-agent AI framework designed to evaluate and mitigate mental health hazards in human-AI interactions.
Outcome: EmoAgent evaluates and mitigates mental health hazards in human-AI interactions.
Temporal Sampling for Forgotten Reasoning in LLMs (2026.acl-long)

Copied to clipboard

Challenge: a new metric measures the percentage of questions that were answered incorrectly during fine-tuning .
Approach: They propose a decoding strategy that draws outputs from multiple checkpoints along the training trajectory.
Outcome: The proposed method improves reasoning performance and consistency across benchmarks.
VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluation (2024.acl-long)

Copied to clipboard

Challenge: Existing metrics for conditional image generation are opaque and lack explainability . evaluators of these metrics have limited ability to evaluate image synthesis tasks .
Approach: They propose a Visual Instruction-guided Explainable metric for evaluating conditional image models.
Outcome: The proposed model achieves a high Spearman correlation with human evaluations, but is weaker than GPT-4o and GPT-v in evaluating synthetic images.
Exploring Reasoning Reward Model for Agents (2026.findings-acl)

Copied to clipboard

Challenge: Existing methods for agentic reinforcement learning rely on sparse outcome-based reward for training, leading to suboptimal results.
Approach: They propose an agent-based reward model that produces structured feedback for agentic trajectories, including an explicit reasoning trace and a focused critique.
Outcome: The proposed model produces structured feedback for agentic trajectories including an explicit reasoning trace, a focused critique, and an overall score that evaluates process performance.
Multimodal Inconsistency Reasoning (MMIR): A New Benchmark for Multimodal Reasoning Models (2025.findings-acl)

Copied to clipboard

Challenge: Existing Multimodal Large Language Models (MLLMs) are predominantly trained on consistent visual-textual inputs, leaving open the question of whether they can handle semantic mismatches in layout-rich content.
Approach: They propose to use multimodal inconsistency reasoning to assess MLLMs' ability to reason about semantic mismatches in webpages, presentation slides, and posters.
Outcome: The proposed model outperforms open-source models in detecting inconsistencies in webpages, presentation slides, and posters while remaining vulnerable to inconsistent errors.
Self-Training with Differentiable Teacher (2022.findings-naacl)

Copied to clipboard

Challenge: Existing methods for self-training are interpreted as teacher-student frameworks, where the teacher generates pseudo-labels and the student makes predictions.
Approach: They propose a differentiable self-training method that treats teacher-student as a Stackelberg game where a leader is always in a more advantageous position than a follower.
Outcome: The proposed model outperforms existing methods on semi- and weakly-supervised learning tasks on semi and weak supervised tasks.
H-LegalKI: A Hierarchical Legal Knowledge Integration Framework for Legal Community Question Answering (2024.findings-emnlp)

Copied to clipboard

Challenge: Legal question answering (LQA) aims to bridge the gap between limited availability of legal professionals and the extensive volume of legal issues.
Approach: They propose a legal knowledge retriever and a hierarchical legal knowledge integration framework to address multiple user-specific circumstances.
Outcome: The proposed framework outperforms baselines on the legal community question-answering dataset.
DFAMS: Dynamic-flow guided Federated Alignment based Multi-prototype Search (2026.acl-long)

Copied to clipboard

Challenge: Existing methods for ambiguous queries struggle to retrieve high-quality documents . DFAMS outperforms advanced FR methods by 14.37% in knowledge classification accuracy .
Approach: They propose a framework that leverages dynamic information flow to identify latent query intents and construct semantically aligned knowledge partitions for accurate retrieval across heterogeneous sources.
Outcome: The proposed framework outperforms existing methods in classification accuracy and retrieval recall tests.
SC2: Towards Enhancing Content Preservation and Style Consistency in Long Text Style Transfer (2024.acl-long)

Copied to clipboard

Challenge: Existing methods for short TST are difficult to implement and can cause content degradation.
Approach: They propose a method to vary the style polarity of text while preserving semantic content.
Outcome: The proposed method improves over baselines and is highly efficient.
ProMed: Shapley Information Gain Guided Reinforcement Learning for Proactive Medical LLMs (2026.acl-long)

Copied to clipboard

Challenge: Existing medical Large Language Models (LLMs) follow a reactive paradigm, risking diagnostic errors by answering before seeking sufficient details.
Approach: They propose a reinforcement learning framework that transitions LLMs toward a proactive paradigm, enabling them to ask clinically valuable questions before decision-making.
Outcome: Experiments on partial-information medical benchmarks show that ProMed outperforms state-of-the-art methods by 6.29% on average and delivers a 54.45% gain over the reactive paradigm.
BatonVoice: An Operationalist Framework for Enhancing Controllable Speech Synthesis with Linguistic Intelligence from LLMs (2026.acl-long)

Copied to clipboard

Challenge: Existing approaches often fail to leverage the linguistic intelligence of Large Language Models (LLMs) Existing models lack the ability to follow text instructions for controllable Text-to-Speech (TTS).
Approach: They propose a framework where an LLM acts as a conductor, understanding user instructions and generating a textual plan - explicit vocal features.
Outcome: The proposed model outperforms open- and closed-source models in speech synthesis and achieves zero-shot cross-lingual generalization.
Mobile-R1: Towards Interactive Capability for VLM-Based Mobile Agent via Systematic Training (2026.acl-long)

Copied to clipboard

Challenge: Existing approaches to training agents for visual-language models trap them in local optima, hindering exploration and error correction with the environment.
Approach: They propose a hierarchical training recipe that bridges atomic action execution and strategic task completion.
Outcome: The proposed training recipe bridges atomic action execution and strategic task completion.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations