Papers by Jiang Du
S2S-Arena: Evaluating Paralinguistic Instruction Following in Speech-to-Speech Models (2026.acl-long)
Copied to clipboard
Feng Jiang, Zhiyu Lin, Yiyang Liu, Liumeng Xue, Fan Bu, Yuhao Du, Xiangying Chen, Benyou Wang, Haizhou Li
| Challenge: | Existing benchmarks rely heavily on text-based evaluation and largely ignore paralinguistic cues such as prosody, emotion, and speaker traits. |
| Approach: | They propose a speech-native benchmark for evaluating instruction-following S2S models with explicit assessment of both semantic understanding and paralinguistic expression. |
| Outcome: | The proposed system enables more natural, robust, and human-aligned speech agents. |
Unveiling Project-Specific Bias in Neural Code Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) based neural code models struggle to generalize effectively to real-world inter-project out-of-distribution data. |
| Approach: | They propose a Cond-Idf measurement to measure the relatedness of a token with a label and its project-specificness. |
| Outcome: | The proposed framework improves both inter-project OOD generalization and adversarial robustness while not sacrificing accuracy on intra-project IID data. |
Dynamic Parallel Tree Search for Efficient LLM Reasoning (2025.acl-long)
Copied to clipboard
Yifu Ding, Wentao Jiang, Shunyu Liu, Yongcheng Jing, Jinyang Guo, Yingjie Wang, Jing Zhang, Zengmao Wang, Ziwei Liu, Bo Du, Xianglong Liu, Dacheng Tao
| Challenge: | Recent methods focus on search accuracy while overlooking computational efficiency. |
| Approach: | They propose a parallelism framework that dynamically optimizes reasoning path in inference. |
| Outcome: | The proposed framework improves efficiency by 2-4 on average while maintaining or even surpassing existing reasoning algorithms in accuracy. |
MIKE: A New Benchmark for Fine-grained Multimodal Entity Knowledge Editing (2024.findings-acl)
Copied to clipboard
Jiaqi Li, Miaozeng Du, Chuanyi Zhang, Yongrui Chen, Nan Hu, Guilin Qi, Haiyun Jiang, Siyuan Cheng, Bozhong Tian
| Challenge: | Current benchmarks focus on coarse-grained knowledge, leaving the intricacies of fine-grounded knowledge unexplored. |
| Approach: | They propose a benchmark and dataset specifically designed for FG multimodal entity knowledge editing. |
| Outcome: | The proposed benchmark underscoring the complexity of FG knowledge editing in MLLMs. |
MINED: Probing and Updating with Multimodal Time-Sensitive Knowledge for Large Multimodal Models (2026.findings-acl)
Copied to clipboard
Kailin Jiang, Ning Jiang, Yuntao Du, Yuchen Ren, Yuchen Li, Yifan Gao, Jinhe Bi, Yunpu Ma, Bin Li, Lei Liu, Qing Li
| Challenge: | Existing benchmarks for Large Multimodal Models (LMMs) are constrained by static representations, inadequately evaluating their ability to understand time-sensitive knowledge. |
| Approach: | They propose a benchmark containing 2,104 time-sensitive knowledge samples spanning six knowledge types to evaluate temporal awareness along 6 key dimensions and 11 challenging tasks. |
| Outcome: | The proposed benchmark measures temporal awareness along 6 key dimensions and 11 tasks, while most open-source LMMs still lack time understanding ability. |
Data-Efficient Selection via Grammatical Complexity in Continual Pre-training of Domain-Specific LLMs (2025.emnlp-main)
Copied to clipboard
Yizhou Ying, Geng Zhang, Cui Danxin, Chengyu Du, Guanglei Yue, Sihang Jiang, Jiaqing Liang, Yifei Fu, Hailin Hu, Yanghua Xiao
| Challenge: | Existing data selection strategies for continual pre-training of large language models often rely on scarce labeled data or computationally expensive LLMs. |
| Approach: | They propose an annotation-independent data selection framework for CPT that evaluates grammatical complexity using lexical diversity and syntactic complexity. |
| Outcome: | The proposed framework outperforms baselines on a financial dataset and surpasses full-data training by 1.7% using only 20% of the data. |
SeCuRepair: Semantics-Aligned, Curriculum-Driven, and Reasoning-Enhanced Vulnerability Repair Framework (2026.acl-long)
Copied to clipboard
Chengran Yang, Ting Zhang, Jinfeng Jiang, Xin Zhou, Haoye Tian, Mingzhe Du, Jieke Shi, Junkai Chen, Yikun Li, Eng Lieh Ouh, Lwin Khin Shar, David Lo
| Challenge: | Existing methods for automating vulnerability repair suffer from syntactic overfitting . nvd published 49,230 Common Vulnerabilities and Exposures (CVE) records in 2025 alone . |
| Approach: | They propose a semantic-aware reward framework that optimizes for code semantic equivalence rather than lexical mimicry. |
| Outcome: | The proposed framework outperforms state-of-the-art frameworks on repository-level splits . it incorporates expert-aligned reasoning mechanism that grounds patch generation in structured diagnosis. |
From Specific-MLLMs to Omni-MLLMs: A Survey on MLLMs Aligned with Multi-modalities (2025.findings-acl)
Copied to clipboard
Shixin Jiang, Jiafeng Liang, Jiyuan Wang, Xuan Dong, Heng Chang, Weijiang Yu, Jinhua Du, Ming Liu, Bing Qin
| Challenge: | MLLMs are able to integrate multiple modalities into a single model to tackle complex tasks in real-world scenarios. |
| Approach: | They propose a comprehensive survey of Omni-MLLMs to address the challenges and opportunities of multimodal modeling. |
| Outcome: | The proposed model can integrate multiple modalities into a single model and provide novel perspectives. |
Neural Parameter Search for Slimmer Fine-Tuned Models and Better Transfer (2025.acl-long)
Copied to clipboard
Guodong Du, Zitao Fang, Jing Li, Junlin Li, Runhua Jiang, Shuyang Yu, Yifei Guo, Yangneng Chen, Sim Kuan Goh, Ho-Kin Tang, Daojing He, Honghai Liu, Min Zhang
| Challenge: | Foundational models and their checkpoints have advanced deep learning, boosting performance across applications. |
| Approach: | They propose a method for pruning fine-tuned models by calculating differences between them and original model. |
| Outcome: | The proposed method can improve performance across vision, NLP, and multi-modal benchmarks. |
Eliciting Knowledge from Experts: Automatic Transcript Parsing for Cognitive Task Analysis (P19-1)
Copied to clipboard
| Challenge: | Cognitive task analysis (CTA) is a type of analysis used to elicit and represent the knowledge and thought processes of domain experts. |
| Approach: | They propose a weakly-supervised framework for automated CTA transcript parsing . they partition the parser process into a sequence labeling task and a text span-pair relation extraction task with distant supervision from human-curated protocol files. |
| Outcome: | The proposed framework reduces human labor and scales the task to a small scale. |
VarMAE: Pre-training of Variational Masked Autoencoder for Domain-adaptive Language Understanding (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models have been widely applied to standard benchmarks due to the limited resources available in a domain. |
| Approach: | They propose a Transformer-based language model called VarMAE for domain-adaptive language understanding that encodes the context of a token into a smooth latent distribution. |
| Outcome: | Experiments on science- and finance-domain NLU tasks show that the proposed model can be efficiently adapted to new domains with limited resources. |
SciEvent: Benchmarking Multi-domain Scientific Event Extraction (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on scientific information extraction relies on entity-relation extraction in narrow domains . current models struggle in domains such as sociology and humanities . |
| Approach: | They propose a multi-domain benchmark for scientific abstract annotations using a unified event extraction schema. |
| Outcome: | The proposed benchmark includes 500 abstracts across five research domains with manual annotations of event segments, triggers, and fine-grained arguments. |
Revisiting the Markov Property for Machine Translation (2024.findings-eacl)
Copied to clipboard
| Challenge: | Statistical machine translation (SMT) has employed Markov models, but autoregressive models are less effective. |
| Approach: | They propose to use a Markov Autoregressive Transformer to model neural machine translation using four WMT benchmarks. |
| Outcome: | The proposed model performs better than autoregressive models on four WMT benchmarks. |
Weakly Supervised Contrastive Learning for Chest X-Ray Report Generation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Radiology report generation aims at generating descriptive text from radiology images automatically. |
| Approach: | They propose a weakly supervised contrastive loss method that generates descriptive text from radiology images automatically. |
| Outcome: | The proposed method outperforms previous work on correctness and text generation metrics for two public benchmarks. |
COIG-P: A High-Quality and Large-Scale Chinese Preference Dataset for Alignment with Human Values (2026.findings-eacl)
Copied to clipboard
Siwei Wu, JinCheng Ren, Xeron Du, Shuyue Guo, Xingwei Qu, Yiming Liang, Jie Liu, Yunwen Li, Tyler Loakman, Tianyu Zheng, Boyu Feng, Huaqing Yuan, Zili Wang, Jiaheng Liu, Wenhao Huang, Chenglin Cai, Haoran Que, Jian Yang, Yuelin Bai, Zekun Moore Wang, Zhouliang Yu, Qunshu Lin, Ding Pan, Yuchen Eleanor Jiang, Tiannan Wang, Wangchunshu Zhou, Shenzhi Wang, Xingyuan Bu, Minghao Liu, Guoyin Wang, Ge Zhang, Chenghua Lin
| Challenge: | Existing Chinese preference datasets suffer from limited scale, restricted domain coverage, and insufficiently rigorous data validation. |
| Approach: | They propose an LLM-based data annotation pipeline with no human intervention to annotate Chinese preference datasets. |
| Outcome: | The proposed pipeline outperforms existing Chinese preference datasets on AlignBench and Chinese Reward Benchmark. |
ngram-OAXE: Phrase-Based Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translation (2022.coling-1)
Copied to clipboard
| Challenge: | Recent studies have incorporated approaches to improving the standard cross-entropy loss to ameliorate the effect of multimodality. |
| Approach: | They propose a new training oaxe loss which removes the penalty of word order errors in the standard cross-entropy loss. |
| Outcome: | Extensive experiments on NAT benchmarks show that the proposed approach improves translation quality and improves model performance. |
Optimizing Cross-Client Domain Coverage for Federated Instruction Tuning of Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Federated domain-specific instruction tuning (FedDIT) for large language models (LLMs) aims to enhance performance in specialized domains using distributed private and limited data. |
| Approach: | They introduce an algorithm that explicitly maximizes cross-client domain coverage through diversity-oriented client center selection and retrieval-based augmentation. |
| Outcome: | The proposed algorithm achieves performance gains of 29.19% and domain coverage improvements of 4.82%-21.36% over 11 baselines. |
Shallow-to-Deep Training for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that deep training is 1:4 faster than training from scratch. |
| Approach: | They propose a shallow-to-deep training method that learns deep models by stacking shallow models. |
| Outcome: | The proposed method is 1:4 faster than training from scratch and achieves BLEU scores of 30:33 and 43:29 on two translation tasks. |
Enhancing Multimodal Named Entity Recognition through Adaptive Mixup Image Augmentation (2025.coling-main)
Copied to clipboard
| Challenge: | Current named entity recognition methods struggle with text-image mismatch problem due to a lack of visual context. |
| Approach: | They propose an adaptive mixup image augmentation method that generates augmented images based on matching score between text and image . |
| Outcome: | The proposed method can be integrated into existing models and demonstrate consistent performance improvements. |
Feedback-Driven Tool-Use Improvements in Large Language Models via Automated Build Environments (2026.findings-acl)
Copied to clipboard
Junjie Ye, Changhao Jiang, Zhengyin Du, Yufei Xu, Xuesong Yao, Zhiheng Xi, Xiaoran Fan, Qi Zhang, Tao Gui, Xuanjing Huang, Jiecao Chen
| Challenge: | Currently, there are no efficient reinforcement learning (RL) frameworks specifically designed for tool use. |
| Approach: | They propose an automated environment construction pipeline that incorporates scenario decomposition, document generation, function integration, complexity scaling, and localized deployment to enable high-quality training environments without external tools. |
| Outcome: | The proposed framework significantly improves the models’ tool-use performance without degrading their general capabilities. |
Knowledge Fusion By Evolving Weights of Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Experimental results on mainstream language models show that Evolver outperforms previous state-of-the-art models by large margins due to the high training costs of large language models. |
| Approach: | They propose a method to integrate multiple models from diverse training scenarios into a unified model. |
| Outcome: | The proposed method outperforms state-of-the-art models on mainstream language models by large margins. |
QG-SMS: Enhancing Test Item Analysis via Student Modeling and Simulation (2025.acl-long)
Copied to clipboard
| Challenge: | Question Generation (QG) tasks are often evaluated using reference-based metrics such as ROUGE and BLEU. |
| Approach: | They propose a QG evaluation framework that leverages Large Language Model for Student Modeling and Simulation to perform test item analysis. |
| Outcome: | The proposed framework improves the QG task and human-simulated student profiles. |