Papers by Ju Zhang
TeleMelody: Lyric-to-Melody Generation with a Template-Based Two-Stage Method (2022.emnlp-main)
Copied to clipboard
Zeqian Ju, Peiling Lu, Xu Tan, Rui Wang, Chen Zhang, Songruoyao Wu, Kejun Zhang, Xiang-Yang Li, Tao Qin, Tie-Yan Liu
| Challenge: | a new lyric-to-melody generation system bridges the gap between lyrics and melodies . previous generation systems lack paired data and lack of control on generated melodie. |
| Approach: | They develop a lyric-to-melody generation system with music template to bridge the gap between lyrics and melodies. |
| Outcome: | The proposed system bridges the gap between lyrics and melodies by using music template. |
HEAL: Hybrid Enhancement with LLM-based Agents for Text-attributed Hypergraph Self-supervised Representation Learning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to enhance text-attributed hypergraph self-supervised learning are limited by label scarcity. |
| Approach: | They propose a data-centric approach that leverages large language models to enhance hypergraph self-supervised learning by integrating hyperedges into a self-representation framework. |
| Outcome: | The proposed approach generates informative nodes and hyperedges through multi-round interaction with LLM-based agents. |
LLMEmbed: Rethinking Lightweight LLM’s Genuine Function in Text Classification (2024.acl-long)
Copied to clipboard
| Challenge: | Recent attempts to improve text classification performance are based on heuristic Chain-of-Thought (CoT) LLMEmbed is a simple and effective transfer learning strategy that can be used to improve the performance of large language models. |
| Approach: | They propose a simple transfer learning strategy to improve text classification using heuristic Chain-of-Thought. |
| Outcome: | The proposed method achieves strong performance on publicly available datasets while using low training overhead. |
EVA: Evolving Semantic Adversaries for Red-Teaming GUI Agents Against Environmental Injection Attacks (2026.findings-acl)
Copied to clipboard
Yijie Lu, Manman Zhao, Tianjie Ju, Zihe Yan, Xinbei Ma, Yuan Guo, Daizong Ding, Gongshen Liu, Zhuosheng Zhang
| Challenge: | Existing methods for red-teaming face a trade-off between requiring target-specific knowledge and incurring prohibitive computational costs. |
| Approach: | They propose a framework that evolves payloads exclusively on the semantic dimension via a discovery-deployment pipeline. |
| Outcome: | Experiments show that EVA outperforms baselines in terms of attack success rate while evolving benign seeds into successful attacks within 1.18 to 1.71 iterations. |
When Words Smile: Generating Diverse Emotional Facial Expressions from Text (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing systems that generate only coarse facial expressions ignore the rich and dynamic nature of face-to-face communication. |
| Approach: | They propose an end-to-end text-to expression model that explicitly focuses on emotional dynamics. |
| Outcome: | The proposed model outperforms baselines on 15,000 text–3D expression pairs on a large-scale dataset. |
Joint Multi-modal Aspect-Sentiment Analysis with Auxiliary Cross-modal Relation Detection (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on aspect-level sentiment analysis focus on extracting aspect terms and sentiment polarities separately. |
| Approach: | They propose a multi-modal joint learning approach with auxiliary cross-modal relation detection for multi-dimensional aspect-level sentiment analysis. |
| Outcome: | The proposed approach can obtain all aspect-level sentiment polarities dependent on the jointly extracted specific aspects. |
CMQA: A Dataset of Conditional Question Answering with Multiple-Span Answers (2022.coling-1)
Copied to clipboard
| Challenge: | Existing QA datasets only contain unconditional and parallel answers . conditional question answering with hierarchical multi-span answers is challenging for the community to solve . |
| Approach: | They propose a conditional question answering task with hierarchical multi-span answers . they propose CMQA, which contains conditional and hierarchic samples . |
| Outcome: | The proposed task can be used to build more reliable and sophisticated QA systems. |
A Survey on Efficient Large Language Model Training: From Data-centric Perspectives (2025.acl-long)
Copied to clipboard
Junyu Luo, Bohan Wu, Xiao Luo, Zhiping Xiao, Yiqiao Jin, Rong-Cheng Tu, Nan Yin, Yifan Wang, Jingyang Yuan, Wei Ju, Ming Zhang
| Challenge: | achieving data-efficient post-training of Large Language Models is a key research question. |
| Approach: | They propose a taxonomy of data-efficient LLM post-training methods from a data-centric perspective. |
| Outcome: | The proposed methods cover data selection, data quality enhancement, synthetic data generation, data distillation and compression, and self-evolving data ecosystems. |
ConCISE: Confidence-guided Compression in Step-by-step Efficient Reasoning (2025.emnlp-main)
Copied to clipboard
Ziqing Qiao, Yongheng Deng, Jiali Zeng, Dong Wang, Lai Wei, Guanbo Wang, Fandong Meng, Jie Zhou, Ju Ren, Yaoxue Zhang
| Challenge: | Existing methods for fine-tuning-based compression suffer from verbose outputs, increasing computational overhead. |
| Approach: | They propose a framework to generate concise reasoning chains using Confidence Injection and Early Stopping. |
| Outcome: | The proposed framework reduces the length of the model by up to 50% while maintaining high task accuracy. |
Investigating Multi-Hop Factual Shortcuts in Knowledge Editing of Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Recent work has demonstrated the power of large language models in recalling knowledge and reasoning. |
| Approach: | They propose to erase shortcut neurons to mitigate the associated risks . 20% of the failures are attributed to shortcuts, they find . |
| Outcome: | The proposed approach reduces failures in multi-hop knowledge editing caused by shortcuts by 20% . |
MMAG: Multimodal Learning for Mucus Anomaly Grading in Nasal Endoscopy via Semantic Attribute Prompting (2025.emnlp-main)
Copied to clipboard
| Challenge: | Accurate grading of rhinitis severity relies heavily on the characterization of key secretions, notably clear nasal discharge (CND) and purulent nasal secretion (PUS). |
| Approach: | They propose a framework that integrates structured prompts with rank-aware vision-language modeling for joint detection and grading. |
| Outcome: | The proposed model improves AUC and F1 scores on CND and PUS datasets by 6.31% and 4.79%. |
OS-Kairos: Adaptive Interaction for MLLM-Powered GUI Agents (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing efforts to build GUI agents focused on the autonomous mode have failed to address the problem of over-execution. |
| Approach: | They propose an adaptive GUI agent that predicts confidence levels at each interaction step and elicits adaptive interaction. |
| Outcome: | The proposed GUI agent outperforms existing models on a complex dataset and on established benchmarks. |
Embracing Large Language Models in Traffic Flow Forecasting (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to predict future traffic flows capture spatio-temporal dependencies, but they fail to adapt to test-time environmental changes. |
| Approach: | They propose to use large language models to help traffic flow forecasting by capturing spatio-temporal dependencies and using a large language model to select the most likely result. |
| Outcome: | The proposed method is based on large language models (LLMs) and an LLM-based selector. |
Stacked Acoustic-and-Textual Encoding: Integrating the Pre-trained Models into Speech Translation Encoders (2021.acl-long)
Copied to clipboard
| Challenge: | End-to-end Speech Translation (E2E ST) encoders lack global context representation, whereas MT encoder lacks it. |
| Approach: | They propose a Stacked Acoustic-and-Textual Encoding method for speech translation . they propose an adaptor module to alleviate representation inconsistency . |
| Outcome: | The proposed method achieves state-of-the-art BLEU scores of 18.3 and 25.2 on two ST tasks. |
PED: Route-Decoupled Diagnostics for Persona Consistency in Spoken Agents (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluations of persona-emotion decoupling frameworks do not isolate which component caused the failure . current evaluations do little to isolate the cause of the failure, making fixes slow and ad hoc . |
| Approach: | They propose a diagnostic evaluation framework that decomposes persona expression into two observable routes. |
| Outcome: | The proposed framework decomposes persona expression into two observable routes . it can be used to perform route-comparable, reference-based analyses of separability, drift, failures and coupling . |
MedDialog: Large-scale Medical Dialogue Datasets (2020.emnlp-main)
Copied to clipboard
Guangtao Zeng, Wenmian Yang, Zeqian Ju, Yue Yang, Sicheng Wang, Ruisi Zhang, Meng Zhou, Jiaqi Zeng, Xiangyu Dong, Ruoyu Zhang, Hongchao Fang, Penghui Zhu, Shu Chen, Pengtao Xie
| Challenge: | telemedicine is a medical practice that provides patient care remotely using video conferencing tools. |
| Approach: | They build large-scale medical dialogue datasets to facilitate research . they pretrain several models on the Chinese MedDialog dataset and compare their performance . |
| Outcome: | The proposed datasets show that models trained on MedDialog can generate doctor-like medical dialogues. |
Learning to Improve Persona Consistency in Multi-party Dialogue Generation via Text Knowledge Enhancement (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods suffer from incomprehensive persona tags that have unique and obscure meanings to describe human’s personality. |
| Approach: | They propose a graph convolution network model with addressee selecting mechanism that integrates personas, dialogue utterances, and external text knowledge in a unified graph. |
| Outcome: | The proposed model outperforms baselines by large margins and improves persona consistency in the generated responses. |
Hidden Ghost Hand: Unveiling Backdoor Vulnerabilities in MLLM-Powered Mobile GUI Agents (2025.findings-emnlp)
Copied to clipboard
Pengzhou Cheng, Haowen Hu, Zheng Wu, Zongru Wu, Tianjie Ju, Daizong Ding, Zhuosheng Zhang, Gongshen Liu
| Challenge: | MLLM-powered GUI agents expose multiple interaction-level triggers, causing backdoor attacks . backdoor injection maximizes feature difference across sample classes, improving flexibility . |
| Approach: | They propose a framework for red-teaming backdoor attacks using MLLMs . they construct composite triggers by combining goal and interaction levels . |
| Outcome: | The proposed framework is effective and stealthy for red-teaming backdoor attacks. |
How Far are We from Robust Long Abstractive Summarization? (2022.emnlp-main)
Copied to clipboard
| Challenge: | Abstractive summarization has made tremendous progress in recent years . however, even under a short document setting, abstractive models often generate summaries that are repetitive, ungrammatical, and factually inconsistent with the source. |
| Approach: | They perform fine-grained human annotations to evaluate long document abstractive summarization systems and develop factual consistency metrics. |
| Outcome: | The proposed model can generate more relevant summaries but not factual ones. |
UER: An Open-Source Toolkit for Pre-training Models (D19-3)
Copied to clipboard
Zhe Zhao, Hui Chen, Jinbin Zhang, Xin Zhao, Tao Liu, Wei Lu, Xi Chen, Haotang Deng, Qi Ju, Xiaoyong Du
| Challenge: | Existing work on pre-training models have shown that it is important to use a framework to deploy various pre- training models efficiently. |
| Approach: | They propose an assemble-on-demand pre-training toolkit that assembles pre-trained models on demand and encapsulates them with rich modules. |
| Outcome: | The proposed framework can reproduce state-of-the-art models or develop models that remain unexplored. |
Multifaceted Evaluation of Audio-Visual Capability for MLLMs: Effectiveness, Efficiency, Generalizability and Robustness (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Multi-modal large language models have been used for processing and understanding information from diverse modalities. |
| Approach: | They propose to evaluate the audio-visual capabilities of multi-modal large language models . they focus on effectiveness, efficiency, generalizability, and robustness . |
| Outcome: | The proposed models exhibit strong zero-shot and few-shot generalization abilities . their success relies heavily on the vision modality, which impairs performance when visual input is corrupted or missing. |
Grape: Knowledge Graph Enhanced Passage Reader for Open-domain Question Answering (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Open-domain question answering (QA) models employ a retriever-reader pipeline . however, state-of-the-art readers fail to capture complex relationships between entities . |
| Approach: | They propose a knowledge graph enhanced passage reader that captures entities in questions and retrieved passages. |
| Outcome: | The proposed knowledge graph enhanced passage reader improves on open-domain QA benchmarks by up to 2.2 exact match scores. |
MIND: Towards Immersive Psychological Healing with Multi-Agent Inner Dialogue (2025.findings-emnlp)
Copied to clipboard
Yujia Chen, Changsong Li, Yiming Wang, Tianjie Ju, Qingqing Xiao, Nan Zhang, Zifan Kong, Peng Wang, Binyu Yan
| Challenge: | Mental health issues are worsening in today’s competitive society, such as depression and anxiety. |
| Approach: | They propose a multi-agent inner dialogue paradigm that provides more immersive psychological healing environments. |
| Outcome: | The proposed paradigm provides more immersive psychological healing environments. |
On the Generation of Medical Dialogs for COVID-19 (2021.acl-short)
Copied to clipboard
Meng Zhou, Zechen Li, Bowen Tan, Guangtao Zeng, Wenmian Yang, Xuehai He, Zeqian Ju, Subrato Chakravorty, Shu Chen, Xingyi Yang, Yichen Zhang, Qingyang Wu, Zhou Yu, Kun Xu, Eric Xing, Pengtao Xie
| Challenge: | under the pandemic of COVID-19, people experiencing COVI D19-related symptoms have a pressing need to consult doctors. |
| Approach: | They develop a medical dialog system that can provide COVID19-related consultations . they use two dialog datasets containing conversations between doctors and patients . |
| Outcome: | The proposed system can provide COVID19-related consultations, but is too small compared with general-domain dialog datasets. |
On the Robustness of Editing Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have exhibited impressive success and significant potential. |
| Approach: | They propose to modify the knowledge memory with minimum computational cost while preserving the performance on the retained knowledge. |
| Outcome: | The proposed methods avoid retraining to update the model parameters and have demonstrated promising performance and efficiency. |
E-ViC: Reasoning Beyond Text via Embodied Visual Chain for Spatial Intelligence (2026.acl-long)
Copied to clipboard
Junbo Qi, Yi Zhang, Hanchu Ni, Che Liu, Zhimin Yao, Ruilin Yang, Xiancong Ren, Liangjian Wen, Wei Ge, Yuya Ieiri, Osamu Yoshie, Yong Dai, Xiaozhu Ju
| Challenge: | Existing Vision-Language Models (VLMs) lack spatial reasoning, despite text-based CoTs . e-ViC reframes spatial intelligence as a verifiable, tool-using capability, argues a new study. |
| Approach: | They propose a framework that moves reasoning beyond text into the visual domain . they ground reasoning in pixel-level interactions to enable human-like "look-and-confirm" strategies . |
| Outcome: | The proposed framework outperforms existing Vision-Language Models with an average gain of 10.1%. |
Exploring Contrast Consistency of Open-Domain Question Answering Systems on Minimally Edited Questions (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing studies on the ability of a model to make consistently correct predictions in the presence of perturbations have not been conducted in open-domain question answering (OpenQA). |
| Approach: | They propose a query-side contrastive loss to improve the dense passage retriever (DPR) to improve DPR training. |
| Outcome: | The proposed approach improves the density of the dense passage retriever (DPR) training set without sacrificing accuracy on standard test sets. |
A Hierarchical Explanation Generation Method Based on Feature Interaction Detection (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work on hierarchical attributions tends to limit text groups to a continuous text span, which is difficult for humans to read. |
| Approach: | They propose a method which captures feature interactions and converts non-hierarchical explanations into hierarchical versions. |
| Outcome: | The proposed method can convert ubiquitous non-hierarchical explanations into their corresponding hierarchical versions. |
MemRec: Collaborative Memory-Augmented Agentic Recommender System (2026.acl-long)
Copied to clipboard
Weixin Chen, Yuhan Zhao, Jingyuan Huang, Zihe Ye, Mingxuan Ju, Tong Zhao, Neil Shah, Li Chen, Yongfeng Zhang
| Challenge: | Existing recommender systems rely on semantic user and item memories to make predictions, but these memories are kept in isolation. |
| Approach: | They propose a framework that architecturally decouples memory management from reasoning to decouple memory management and reasoning from the user and item memories. |
| Outcome: | The proposed framework decouples memory management from reasoning and achieves state-of-the-art performance on four benchmarks. |
Multi-modal Multi-label Emotion Detection with Modality and Label Dependence (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on multi-label emotion detection focus on one modality . current studies focus on label dependence, but there is no consensus on the model . |
| Approach: | They propose a multi-modal sequence-to-set approach to model label dependence and modality dependence in a multiple-modal scenario. |
| Outcome: | The proposed approach is able to model the label dependence and the modality dependence in a multi-modal scenario. |
Enhancing Multiple-choice Machine Reading Comprehension by Punishing Illogical Interpretations (2021.emnlp-main)
Copied to clipboard
| Challenge: | Multiple-choice MRC is one of the most studied tasks in MRC due to the convenience of evaluation and the flexibility of answer format. |
| Approach: | They propose to use multiple-choice MRC to explain a trained model and reveal how it arrives at the prediction by punishing illogical attributions. |
| Outcome: | The proposed method improves model performance without external information and model structure change without any external information. |
Benchmarking Multi-National Value Alignment for Large Language Models (2025.findings-acl)
Copied to clipboard
Chengyi Ju, Weijie Shi, Chengzhong Liu, Jiaming Ji, Jipeng Zhang, Ruiyuan Zhang, Jiajie Xu, Yaodong Yang, Sirui Han, Yike Guo
| Challenge: | Existing studies on large language models focus on ethical reviews, failing to capture the diversity of national values. |
| Approach: | They propose a national value extraction pipeline to efficiently construct value assessment datasets and a model-based model with instruction tagging to process raw data sources. |
| Outcome: | The proposed benchmark evaluates the alignment of LLMs with the values of five major nations: China, the United States, the UK, France, and Germany. |
Semi-supervised Fine-tuning for Large Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing LLMs require labeled data, which can be costly in real-world applications. |
| Approach: | They propose a framework that can fully exploit labeled and unlabeled data for LLM fine-tuning . they conducted experiments using GPT-4o-mini and Llama-3.1 on seven general or domain-specific datasets . |
| Outcome: | The proposed framework can fully exploit labeled and unlabeled data for LLM alignment from a propagate-and-select manner. |
MMEvalPro: Calibrating Multimodal Benchmarks Towards Trustworthy and Efficient Evaluation (2025.naacl-long)
Copied to clipboard
Jinsheng Huang, Liang Chen, Taian Guo, Fu Zeng, Yusheng Zhao, Bohan Wu, Ye Yuan, Haozhe Zhao, Zhihui Guo, Yichi Zhang, Jingyang Yuan, Wei Ju, Luchen Liu, Tianyu Liu, Baobao Chang, Ming Zhang
| Challenge: | Large Multimodal Models (LMMs) exhibit impressive cross-modal understanding and reasoning abilities, but many benchmarks suffer from systematic biases. |
| Approach: | They propose a benchmark to avoid Type-I errors by creating one perception question and one knowledge anchor question through a meticulous annotation process. |
| Outcome: | The proposed benchmark avoids Type-I errors while maintaining reliability of MCQ evaluations. |
Logic Traps in Evaluating Attribution Scores (2022.acl-long)
Copied to clipboard
| Challenge: | Modern deep learning models are notoriously opaque, which has motivated the development of methods for interpreting how deep models predict. |
| Approach: | They propose to review existing methods for evaluating attribution scores and summarize the logic traps in these methods. |
| Outcome: | The proposed methods show that they do not contain logic traps and that they are not reliable. |
Prompt Combines Paraphrase: Teaching Pre-trained Models to Understand Rare Biomedical Words (2022.coling-1)
Copied to clipboard
Haochun Wang, Chi Liu, Nuwa Xi, Sendong Zhao, Meizhi Ju, Shiwei Zhang, Ziheng Zhang, Yefeng Zheng, Bing Qin, Ting Liu
| Challenge: | Pre-trained models perform poorly with limited data and rare biomedical words. |
| Approach: | They propose to use prompt to fine-tune pre-trained models for biomedical domain tuning with a simple approach. |
| Outcome: | The proposed method achieves up to 6% improvement in biomedical natural language inference task without any extra parameters or training steps using few-shot vanilla prompt settings. |
Mitigating Training Imbalance in LLM Fine-Tuning via Selective Parameter Merging (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies suggest that the order of training samples can affect model performance, but this is not the case. |
| Approach: | They propose to merge supervised fine-tuning models with different data orders to mitigate this imbalance by parameter merging. |
| Outcome: | The proposed method outperforms the weighted-average method on five datasets. |