Papers by JingBo Zhu

Leveraging Unit Language Guidance to Advance Speech Modeling in Textless Speech-to-Speech Translation (2025.findings-acl)

Challenge:	Recent training-based TTS methods, such as continued reinforcement learning, have surged in popularity, while training-free TTS approaches are gradually fading from prominence.
Approach:	They propose a fine-grained sequential scaling method guided by process verification that integrates training-free TTS methods with other classical parallel scaling methods at the step level.
Outcome:	Experiments on five instruction-tuned large language models (LLMs) show that training-free TTS methods can extend reasoning performance boundaries.

Copied to clipboard

Yuhao Zhang, Xiangnan Ma, Kaiqi Kou, Peizhuo Liu, Weiqiao Shan, Benyou Wang, Tong Xiao, Yuxin Huang, Zhengtao Yu, JingBo Zhu

Enhancing Speech Large Language Models with Prompt-Aware Mixture of Audio Encoders (2025.emnlp-main)

Challenge:	Existing textless speech-to-speech translation models have two main challenges: 1) learning cross-modal features and 2) learning alignment of difference languages in long sequences.
Approach:	They propose a unit language to overcome two main modeling challenges . they propose task prompt modeling to utilize the unit language in guiding the modeling process.
Outcome:	The proposed language improves over a strong baseline and achieves comparable performance to models trained with text.

Copied to clipboard

Weiqiao Shan, Yuang Li, Yuhao Zhang, Yingfeng Luo, Chen Xu, Xiaofeng Zhao, Long Meng, Yunfei Lu, Min Zhang, Hao Yang, Tong Xiao, JingBo Zhu

EIT: Enhanced Interactive Transformer (2024.acl-long)

Challenge:	Existing work on integrating audio encoders with large language models (LLMs) has focused on semantic understanding tasks, but different tasks may require distinct features that emphasize either semantic or acoustic aspects.
Approach:	They propose to use a prompt-aware mixture to enhance the Speech LLM that uses multiple audio encoders to extract different features based on the prompt.
Outcome:	The proposed approach outperforms all single-encoder Speech LLMs on ASR, speaker number verification, and AC tasks.

Copied to clipboard

Tong Zheng, Bei Li, Huiwen Bao, Tong Xiao, JingBo Zhu

Beyond Decoder-only: Large Language Models Can be Good Encoders for Machine Translation (2025.findings-acl)

Challenge:	Existing multi-view learning models prioritize complementarity while ignoring consensus . EMHA allows for efficient modeling of global dependencies among tokens in parallel .
Approach:	They propose an enhanced multi-head self-attention (EMHA) that prioritizes complementarity while ignoring consensus.
Outcome:	The proposed method favors consensus among heads by introducing two models . it is superior on a wide range of language tasks with a modest increase in model size .

Copied to clipboard

Yingfeng Luo, Tong Zheng, Yongyu Mu, Bei Li, Qinghong Zhang, Yongqi Gao, Ziqiang Xu, Peinan Feng, Xiaoqian Liu, Tong Xiao, JingBo Zhu

Revisiting Interpolation Augmentation for Speech-to-Text Generation (2024.findings-acl)

Challenge:	Recent advances in machine translation have focused on a single pre-trained decoder . encoder-decoder architectures have received relatively little attention in NMT .
Approach:	They propose a method that leverages LLMs as MT encoders and pairs them with lightweight decoders to develop universal translation models.
Outcome:	The proposed method matches or surpasses baselines in terms of translation quality but achieves 75% reduction in memory footprint of the KV cache.

Copied to clipboard

Chen Xu, Jie Wang, Xiaoqian Liu, Qian Dong, Chunliang Zhang, Tong Xiao, JingBo Zhu, Dapeng Man, Wu Yang

Text Generation

Challenge:	Existing approaches to speech-to-text generation tasks are limited by the lack of extensive labeled datasets.
Approach:	They propose to use interpolation augmentation to construct virtual training samples by transforming inputs and labels to enhance generalization in other domains.
Outcome:	The proposed approach significantly improves performance across diverse tasks, architectures, and data scales, offering a promising avenue for more robust S2T systems in resource-constrained settings.

Revealing the Parallel Multilingual Learning within Large Language Models (2024.emnlp-main)

Copied to clipboard

Yongyu Mu, Peinan Feng, Zhiquan Cao, Yuzhang Wu, Bei Li, Chenglong Wang, Tong Xiao, Kai Song, Tongran Liu, Chunliang Zhang, JingBo Zhu

Hybrid Alignment Training for Large Language Models (2024.findings-acl)

Challenge:	Large language models (LLMs) can handle multilingual and cross-lingual text within a single input; however, previous studies focusing on using English as the pivot language to enhance language understanding and reasoning focus on using multiple languages.
Approach:	They propose to use parallel multilingual input to enhance the model's comprehension of the input and to examine how multilingual processing affects prediction.
Outcome:	The proposed model can handle multilingual and cross-lingual text within a single input, but previous studies focused on using English as the pivot language to enhance language understanding and reasoning.

Copied to clipboard

Chenglong Wang, Hang Zhou, Kaiyan Chang, Bei Li, Yongyu Mu, Tong Xiao, Tongran Liu, JingBo Zhu

SERM: Self-Evolving Relevance Model with Agent-Driven Learning from Massive Query Streams (2026.findings-acl)

Challenge:	Existing approaches to align large language models with instructions and preferences are conflicting . et al., 2023b) show that hybrid alignment training can outperform baselines .
Approach:	They propose a hybrid alignment training approach based on alternating alignment and modified elastic weight consolidation methods to achieve better collaboration between different alignment tasks.
Outcome:	The proposed approach outperforms baseline alignment training methods on summarization and dialogue tasks.

Copied to clipboard

Chenglong Wang, Canjia Li, Xingzhao Zhu, Yifu Huo, Huiyu Wang, Weixiong Lin, Yun Yang, Qiaozhi He, Tian Hua Zhou, null Changxiaojia, JingBo Zhu, Tong Xiao

Challenge:	Existing approaches to generate relevance judgments are limited due to dynamic nature of query distributions.
Approach:	They propose a self-evolving relevance model approach to generalize queries to practical search scenarios . they use a multi-agent sample miner and a relevance annotator to generate reliable labels .
Outcome:	The proposed approach can achieve significant performance gains on a large-scale industrial platform, validated by offline multilingual evaluations and online testing.

Incorporating Probing Signals into Multimodal Machine Translation via Visual Question-Answering Pairs (2023.findings-emnlp)

Copied to clipboard

Yuxin Zuo, Bei Li, Chuanhao Lv, Tong Zheng, Tong Xiao, JingBo Zhu

BIASEDTALES-ML: A Multilingual Dataset for Analyzing Narrative Attribute Distributions in LLM-Generated Stories (2026.findings-acl)

Challenge:	Existing studies show that multimodal machine translation systems exhibit decreased sensitivity to visual information when text inputs are complete.
Approach:	They propose to generate parallel VQA style pairs from source text to foster more robust cross-modal interaction.
Outcome:	The proposed approach generates parallel VQA style pairs from the source text, fostering more robust cross-modal interaction.

Copied to clipboard

Yuxuan Ouyang, Yingfeng Luo, JingBo Zhu, Tong Xiao

Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-aligned Quality Estimation (2024.emnlp-main)

Challenge:	Existing studies on the use of Large Language Models (LLMs) focus primarily on English, leaving the cross-lingual generalization of aligned behavior underexplored.
Approach:	They propose a structured generator-extractor pipeline and a multi-dimensional distributional analysis framework to examine how narrative attributes vary across languages, models, and social conditions.
Outcome:	The proposed model reveals substantial cross-lingual variability in narrative generation patterns, indicating that distributions observed in English do not always exhibit similar characteristics in other languages, particularly in lower-resource settings.

Copied to clipboard

Yuan Ge, Yilun Liu, Chi Hu, Weibin Meng, Shimin Tao, Xiaofeng Zhao, Mahong Xia, Zhang Li, Boxing Chen, Hao Yang, Bei Li, Tong Xiao, JingBo Zhu

Bypassing Neural Evaluations for Fast Audio Editing via Adaptive Trajectory Extrapolation (2026.findings-acl)

Challenge:	Existing methods for instruction data selection have limitations such as relying on fragile external APIs, being affected by biases in GPT models, or reducing the diversity of the selected instruction dataset.
Approach:	They propose an industrial-friendly, expert-aligned and diversity-preserved instruction data selection method: Clustering and Ranking (CaR).
Outcome:	The proposed method outperforms Alpaca's existing methods by 32.1% in GPT-4 evaluations.

Copied to clipboard

Xiaoqian Liu, Zhengkun Ge, Jianjin Wang, Haoran Zhang, Yuan Ge, Kaiyan Chang, Chen Xu, Tong Xiao, Zhengtao Yu, Linfeng Zhang, JingBo Zhu

On the Emotion Understanding of Synthesized Speech (2026.acl-long)

Challenge:	Recent advances in audio diffusion models have significantly improved text-to-audio editing via inversion techniques, but these models typically rely on dense, fixed-step sampling trajectories to maintain structural integrity.
Approach:	They propose a model-agnostic Adaptive Trajectory Extrapolation framework that accelerates inversion-based editing process by dynamically evaluating only the most critical generative phases.
Outcome:	The proposed framework achieves a 3.9 speedup with negligible loss in fidelity.

Copied to clipboard

Yuan Ge, Haishu Zhao, AoKai Hao, Junxiang Zhang, Bei Li, Xiaoqian Liu, Chenglong Wang, Jianjin Wang, Bingsen Zhou, Bingyu Liu, JingBo Zhu, Zhengtao Yu, Tong Xiao

Exploiting Target Language Data for Neural Machine Translation Beyond Back Translation (2024.findings-acl)

Challenge:	Existing models for emotion understanding do not capture fundamental features of synthesized speech.
Approach:	They evaluate emotion recognition models on synthesized speech using SER models and generative models.
Outcome:	The proposed model can't generalize to synthesized speech because of speech token prediction . generative models tend to infer emotion from textual semantics while ignoring paralinguistic cues.

Copied to clipboard

Abudurexiti Reheman, Yingfeng Luo, Junhao Ruan, Chunliang Zhang, Anxiang Ma, Tong Xiao, JingBo Zhu

Text Generation

Challenge:	Neural Machine Translation (NMT) encounters challenges when translating in new domains and low-resource languages.
Approach:	They propose a variant of k-nearest neighbor machine translation that utilizes target language data by constructing a pseudo datastore.
Outcome:	The proposed method exhibits strong domain adaptation capability in both high-resource and low-resourced machine translation.

ODE Transformer: An Ordinary Differential Equation-Inspired Model for Sequence Generation (2022.acl-long)

Copied to clipboard

Bei Li, Quan Du, Tao Zhou, Yi Jing, Shuhan Zhou, Xin Zeng, Tong Xiao, JingBo Zhu, Xuebo Liu, Min Zhang

HEAL: A Hypothesis-Based Preference-Aware Analysis Framework (2025.findings-emnlp)

Challenge:	Residual networks are an Euler discretization of solutions to Ordinary Differential Equations (ODE).
Approach:	They propose a residual block of layers in Transformer that can be described as a higher-order solution to ODE.
Outcome:	The proposed architecture can gain large improvements over strong baselines at a slight cost in inference efficiency.

Copied to clipboard

Yifu Huo, Chenglong Wang, Qiren Zhu, Shunjie Xing, Tong Xiao, Chunliang Zhang, Tongran Liu, JingBo Zhu

Challenge:	Preference optimization methods like DPO are often evaluated on a single response, overlooking other outputs.
Approach:	They propose a Hypothesis-based PrEference-aware AnaLysis Framework that formulates preference alignment as a re-ranking process within hypothesis spaces.
Outcome:	The proposed evaluation paradigm re-ranks preference alignment as a reranking process within hypothesis spaces.

Position IDs Matter: An Enhanced Position Layout for Efficient Context Compression in Large Language Models (2025.findings-emnlp)

Copied to clipboard

Runsong Zhao, Xin Liu, Xinyu Liu, Pengcheng Huang, Chunyang Xiao, Tong Xiao, JingBo Zhu

Semantic Text Processing

Challenge:	Existing methods to compress context information ignore holistic contextual dependencies.
Approach:	They propose a method that adjusts position encodings to minimize the distance between context tokens and special tokens.
Outcome:	Enhanced Position Layout (EPL) improves compression of context information in large language models.

MTR-Suite: A Framework for Evaluating and Synthesizing Conversational Retrieval Benchmarks (2026.acl-long)

Copied to clipboard

Junhao Ruan, Abudukeyumu Abudula, Bei Li, Yongjing Yin, Xinyu Liu, Kechen Jiao, Xin Chen, Jingang Wang, Xunliang Cai, Tong Xiao, JingBo Zhu

Natural Language Interfaces

Challenge:	Existing conversational retrieval benchmarks suffer from costly, sparse human annotation or rigid, unnatural automated heuristics.
Approach:	They propose a framework for auditing, synthesizing, and benchmarking conversational retrieval.
Outcome:	The proposed framework is based on three LLM-based auditors and a multi-agent system . it mimics production-style challenges (hard topic switching, verbosity) and offers superior discriminative power.

IIET: Efficient Numerical Transformer via Implicit Iterative Euler Method (2025.emnlp-main)

Copied to clipboard

Xinyu Liu, Bei Li, Jiahao Liu, Junhao Ruan, Kechen Jiao, Hongyin Tang, Jingang Wang, Tong Xiao, JingBo Zhu

Green & Sustainable NLP

Challenge:	High-order numerical methods enhance performance in tasks like NLP but introduce a performance-efficiency trade-off due to increased computational overhead.
Approach:	They propose an iterative implicit Euler Transformer which simplifies high-order numerical methods by iterating implicit Eule.
Outcome:	The proposed method improves accuracy and reduces inference overhead by 55% while retaining 99.4% of the original task accuracy.

Language-Specific Layer Matters: Efficient Multilingual Enhancement for Large Vision-Language Models (2025.findings-emnlp)

Copied to clipboard

Yuchun Fan, Yilin Wang, Yongyu Mu, Lei Huang, Bei Li, Xiaocheng Feng, Tong Xiao, JingBo Zhu

Forgetting Curve: A Reliable Method for Evaluating Memorization Capability for Long-Context Models (2024.emnlp-main)

Challenge:	Large vision-language models exhibit an imbalance in multilingual capabilities .
Approach:	They propose a training recipe that achieves efficient multilingual enhancement for LVLMs by Precise Language Specific layers fine-tuning.
Outcome:	The proposed training recipe achieves efficient multilingual enhancement for LVLMs by fine-tuning language specific layers.

Copied to clipboard

Xinyu Liu, Runsong Zhao, Pengcheng Huang, Chunyang Xiao, Bei Li, Jingang Wang, Tong Xiao, JingBo Zhu

PartialFormer: Modeling Part Instead of Whole for Machine Translation (2024.findings-acl)

Challenge:	Existing methods, tasks and benchmarks to measure model’s effective memory length are limited.
Approach:	They propose a method called forgetting curve to measure the memorization capability of long-context models.
Outcome:	The proposed method is robust to the tested corpus and experimental settings, and can be applied to any model size.

Copied to clipboard

Tong Zheng, Bei Li, Huiwen Bao, Jiale Wang, Weiqiao Shan, Tong Xiao, JingBo Zhu

Text Generation

Challenge:	Existing feed-forward neural networks have significant computational and parametric overhead.
Approach:	They propose a parameter-efficient Transformer architecture that utilizes multiple smaller FFNs to reduce parameters and computation while maintaining essential hidden dimensions.
Outcome:	The proposed architecture reduces computational and parameter overhead while maintaining essential hidden dimensions.

CoMeT: Collaborative Memory Transformer for Efficient Long Context Modeling (2026.acl-long)

Copied to clipboard

Runsong Zhao, Shilei Liu, Jiwei Tang, Langming Liu, Haibin Chen, Weidong Zhang, Yujin Yuan, Tong Xiao, JingBo Zhu, Wenbo Su, Bo Zheng

SPS: Steering Probability Squeezing for Better Exploration in Reinforcement Learning for Large Language Models (2026.findings-acl)

Challenge:	a novel architecture that enables LLMs to handle arbitrarily long sequences with constant memory usage and linear time complexity is a major barrier to long-context processing.
Approach:	They propose a novel architecture that enables LLMs to handle arbitrarily long sequences with constant memory usage and linear time complexity.
Outcome:	The proposed architecture can handle arbitrarily long sequences with constant memory usage and linear time complexity.

Copied to clipboard

Yifu Huo, Chenglong Wang, Ziming Zhu, Shunjie Xing, Peinan Feng, Tongran Liu, Qiaozhi He, Tian Hua Zhou, null Changxiaojia, JingBo Zhu, Zhengtao Yu, Tong Xiao

On Vision Features in Multimodal Machine Translation (2022.acl-long)

Challenge:	Reinforcement learning (RL) training typically improves single-sample success rates but limited exploration of diverse reasoning trajectories.
Approach:	They propose a training paradigm that interleaves conventional RL with inverse reinforcement learning (IRL) they propose 'Steering Probability Squeezing' to enhance exploration without external supervision .
Outcome:	The proposed training paradigm improves Pass@k and improves exploration of diverse reasoning trajectories without external supervision.

Copied to clipboard

Bei Li, Chuanhao Lv, Zefan Zhou, Tao Zhou, Tong Xiao, Anxiang Ma, JingBo Zhu

Enhancing Neural Machine Translation Through Target Language Data: A kNN-LM Approach for Domain Adaptation (2025.acl-long)

Challenge:	Recent work on multimodal machine translation (MMT) has focused on the way of incorporating vision features into translation but little attention is given to the quality of vision models.
Approach:	They develop a selective attention model to study the patch-level contribution of an image in multimodal machine translation.
Outcome:	The proposed model is able to learn translation from the visual modality on probing tasks and is compared with existing models.

Copied to clipboard

Abudurexiti Reheman, Hongyu Liu, Junhao Ruan, Abudukeyumu Abudula, Yingfeng Luo, Tong Xiao, JingBo Zhu

NiuTrans.LMT: Toward Inclusive and Scalable Multilingual Machine Translation with LLMs (2026.acl-long)

Challenge:	Neural machine translation (NMT) has made significant progress in recent years, yet often suffers from translating in new domains, which is called domain adaptation.
Approach:	They propose a method that leverages semantically similar target language sentences in the kNN framework and generates a probability distribution over these sentences during decoding.
Outcome:	The proposed method generates a probability distribution over similar target language sentences and then interpolates with the model’s distribution.

Copied to clipboard

Yingfeng Luo, Ziqiang Xu, Yuxuan Ouyang, MuRun Yang, DingYang Lin, Kaiyan Chang, Tong Zheng, Bei Li, Peinan Feng, Quan Du, Tong Xiao, JingBo Zhu

Teaching Language Models to Self-Improve by Learning from Language Feedback (2024.findings-acl)

Challenge:	Large language models have significantly advanced Multilingual Machine Translation (MMT) yet scaling to many languages while maintaining robust performance across directions remains challenging.
Approach:	They propose a strategy to reduce the number of translations in one direction . they propose auxiliary parallel sentences to promote cross-lingual transfer .
Outcome:	The proposed model performs on par with or better than substantially larger baselines.

Copied to clipboard

Chi Hu, Yimin Hu, Hang Cao, Tong Xiao, JingBo Zhu