Papers by Jiaming Zhou
ProUIE: A Macro-to-Micro Progressive Learning Method for LLM-based Universal Information Extraction (2026.findings-acl)
Copied to clipboard
Wenda Liu, Song Zhigang, Shuai Nie, Guangyao Liu, Lisung Chen, Binyu Yang, Yaran Chen, Peng Zhou, Hongzhen Wang, Yuchen Liu, Wenyue Hu, Jiaming Xu, Runyu Shi, Ying Huang
| Challenge: | ProUIE improves universal information extraction (UIE) without external information . many LLM-based methods rely on extra schema cues, external resources or complex alignment and verification pipelines . |
| Approach: | They propose a Macro-to-Micro progressive learning approach that improves UIE without external information. |
| Outcome: | ProUIE outperforms instruction-tuned baselines on average for NER and RE while using a smaller backbone. |
RealTalk-CN: A Realistic Chinese Speech Task-Oriented Dialogue Benchmark with Cross-Modal Analysis (2026.acl-long)
Copied to clipboard
| Challenge: | Recent advances in speech large language models have enabled end-to-end spoken interactions, but their robustness in real-world applications remains unclear. |
| Approach: | They propose a multi-turn, multi-domain speech–text TOD dataset for Chinese users . it contains 5.4k dialogues with annotations for dialogue states, disfluency types, speaker characteristics . |
| Outcome: | The proposed model can be used to evaluate speech large language models in real-world scenarios . the proposed model is based on 5.4k real human-to-human dialogues with annotations . |
Omni-RewardBench: Toward a Comprehensive Evaluation of Generative Reward Models Across Modalities (2026.acl-long)
Copied to clipboard
Chi-Min Chan, Yujin Zhou, Pengcheng Wen, Boqin Yin, Jiaming Ji, Juntao Dai, Wei Xue, Sirui Han, Yike Guo
| Challenge: | Existing evaluation benchmarks for ORMs are largely text-centric or limited to bimodal tasks . a new study examines the effectiveness of Omni-RewardBench for ORms across modalities . |
| Approach: | They propose a hybrid automatic-annotation and human-verification pipeline to construct high-quality evaluation data. |
| Outcome: | The proposed model is the first benchmark for comprehensive evaluation of ORMs across modalities. |
RWKV: Reinventing RNNs for the Transformer Era (2023.findings-emnlp)
Copied to clipboard
Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Leon Derczynski, Xingjian Du, Matteo Grella, Kranthi Gv, Xuzheng He, Haowen Hou, Przemyslaw Kazienko, Jan Kocon, Jiaming Kong, Bartłomiej Koptyra, Hayden Lau, Jiaju Lin, Krishna Sri Ipsit Mantri, Ferdinand Mom, Atsushi Saito, Guangyu Song, Xiangru Tang, Johan Wind, Stanisław Woźniak, Zhenyuan Zhang, Qinghua Zhou, Jian Zhu, Rui-Jie Zhu
| Challenge: | recurrent neural networks struggle to match the performance of Transformers due to limitations in parallelization and scalability. |
| Approach: | They propose a model architecture that combines the efficient parallelizable training of transformers with the efficient inference of RNNs. |
| Outcome: | The proposed model performs on par with similarly sized RNNs, suggesting future work can leverage this architecture to create more efficient models. |
I2E: From Image Pixels to Actionable Interactive Environments for Text-Guided Image Editing (2026.acl-long)
Copied to clipboard
Jinghan Yu, Junhao Xiao, Chenyu Zhu, Jiaming Li, Jia Li, HanMing Deng, Xirui Wang, Guoli Jia, Jianjun Li, Xiang Bai, Bowen Zhou, Zhiyuan Ma
| Challenge: | Existing text-guided image editing methods rely on end-to-end pixel-level inpainting paradigm . existing models lack such intermediate representations and Reasoning-then-action process . |
| Approach: | They propose a "Decompose-then-Action" paradigm that revisits image editing as an actionable interaction process within a structured environment. |
| Outcome: | The proposed paradigm outperforms existing methods in compositional editing tasks. |
ChildMandarin: A Comprehensive Mandarin Speech Dataset for Young Children Aged 3-5 (2025.acl-long)
Copied to clipboard
Jiaming Zhou, Shiyao Wang, Shiwan Zhao, Jiabei He, Haoqin Sun, Hui Wang, Cheng Liu, Aobo Kong, Yujie Guo, Xi Yang, Yequan Wang, Yonghua Lin, Yong Qin
| Challenge: | Automatic speech recognition systems have advanced significantly with models like Whisper, Conformer, and self-supervised frameworks such as Wav2vec 2.0. |
| Approach: | They propose to use Mandarin speech datasets to analyze pronunciation and tone of children aged 3 to 5 and evaluate their models on speaker verification (SV) They find that the datasets are more robust than those used by adult speech recognition systems and are open-source and available for all academic purposes. |
| Outcome: | The proposed dataset includes 41.25 hours of speech with carefully crafted manual transcriptions, collected from 397 speakers across various provinces in China, with balanced gender representation. |
DIFFA-2: A Practical Diffusion Large Language Model for General Audio Understanding (2026.findings-acl)
Copied to clipboard
| Challenge: | Autoregressive (AR) large audio language models are expensive in data and computation . prior work shows diffusion-based LALMs can improve audio understanding under matched settings . |
| Approach: | They propose a diffusion-based LALM that upgrades the speech encoder and employs dual semantic and acoustic adapters. |
| Outcome: | a new model improves over existing autoregressive large language models and is competitive to strong AR models . the proposed model can make use of limited training data and improve inference efficiency . a recent study shows that diffusion-based models can improve audio understanding . |
AudioPrivacy: Parallel Audio Dataset for Speaker Profiling with Diverse Audio Types and Rich Attributes (2026.findings-acl)
Copied to clipboard
| Challenge: | Speech signals convey abundant speaker-related metadata, yet current privacy research focuses on identity-centric voiceprint protection, leaving sensitive Speaker Attribute Privacy (SAP) underexplored. |
| Approach: | They propose a large-scale Chinese dataset to evaluate speaker-related privacy leakage . the dataset includes 227.3 hours of audio from 1,000 speakers . |
| Outcome: | The proposed model systematically evaluates speaker-related privacy leakage in everyday scenarios. |
ChildTalk: A Multi-Dialect Chinese Child Speech Corpus with Full-Length Child–Caregiver Conversations for Speech Recognition (2026.findings-acl)
Copied to clipboard
| Challenge: | Automatic speech recognition (ASR) for children remains challenging due to developmental variability and the scarcity of high-quality corpora. |
| Approach: | They propose a large-scale Chinese child speech corpus that contains 112.5 hours of speech from 498 children and 500 caregivers. |
| Outcome: | The proposed model improves in-domain and cross-domain performance on children's speech. |
EmotionTalk: An Interactive Chinese Multimodal Emotion Dataset With Rich Annotations (2026.findings-acl)
Copied to clipboard
Haoqin Sun, Jinghua Zhao, Xuechen Wang, Shiwan Zhao, Jiaming Zhou, Hui Wang, Xi Yang, Yequan Wang, Yonghua Lin
| Challenge: | Existing datasets face issues such as low quality, limited scale, and incomplete modalities, hindering model performance. |
| Approach: | They propose to use Chinese multimodal datasets to capture authentic emotional interplay from 19 professional actors. |
| Outcome: | The EmotionTalk dataset spans 23.6 hours of dyadic conversations across diverse scenarios. |
STORYTELLER: An Enhanced Plot-Planning Framework for Coherent and Cohesive Story Generation (2025.findings-acl)
Copied to clipboard
Jiaming Li, Yukun Chen, Ziqiang Liu, Minghuan Tan, Lei Zhang, Yunshui Li, Run Luo, Longze Chen, Jing Luo, Ahmadreza Argha, Hamid Alinejad-Rokny, Wei Zhou, Min Yang
| Challenge: | Existing methods for storytelling lack coherence and consistency, compromising the overall storytelling experience. |
| Approach: | They propose a novel approach that improves the coherence and consistency of automatically generated stories by managing plot nodes and enabling dynamic interactions between different parts of the story. |
| Outcome: | The proposed approach outperforms existing methods in 84.33% of the trials. |
PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preference (2025.acl-long)
Copied to clipboard
Jiaming Ji, Donghai Hong, Borong Zhang, Boyuan Chen, Josef Dai, Boren Zheng, Tianyi Alex Qiu, Jiayi Zhou, Kaile Wang, Boxun Li, Sirui Han, Yike Guo, Yaodong Yang
| Challenge: | Using large-scale annotation data, large language models can generate noise, errors and biases, leading to unexpected behaviours. |
| Approach: | They propose a dataset to promote safety alignment in large language models . they separate helpfulness and harmlessness annotations for question-answering pairs . |
| Outcome: | The proposed dataset provides 44.6k prompts and 265k question-answer pairs with safety meta-labels for 19 harm categories and three severity levels, with answers generated by Llama-family models. |
Language Models Resist Alignment: Evidence From Data Compression (2025.acl-long)
Copied to clipboard
Jiaming Ji, Kaile Wang, Tianyi Alex Qiu, Boyuan Chen, Jiayi Zhou, Changye Li, Hantao Lou, Josef Dai, Yunhuai Liu, Yaodong Yang
| Challenge: | Large language models (LLMs) may exhibit undesirable behaviors due to the inevitable biases and harmful content present in training. |
| Approach: | They propose to investigate the elasticity of large language models by examining their performance. |
| Outcome: | The proposed model performance declines rapidly before reverting to the pre-training distribution, the authors show . the proposed model weight and code are available at pku-lm-res ist-alignment.github.io. |
Making RALM Robust to Irrelevant Contexts via Layer Knowledge Guided Attention (2025.findings-acl)
Copied to clipboard
Weijie Shi, Hao Chen, Jiaming Li, Yao Zhao, Yazhong Zhang, Qijin Chen, Jipeng Zhang, Ruiyuan Zhang, Jia Zhu, Jiajie Xu, Xiaofang Zhou
| Challenge: | Large language models (LLMs) face factual hallucination and knowledge obsolescence when tackling knowledge-intensive tasks. |
| Approach: | They propose a layer-knowledge guided attention method which harnesses the layer-wise knowledge of large language models to optimize per-layer attention on useful passages. |
| Outcome: | The proposed method outperforms existing methods on RALM benchmarks. |
Reward Generalization in RLHF: A Topological Perspective (2025.findings-acl)
Copied to clipboard
Tianyi Alex Qiu, Fanzhi Zeng, Jiaming Ji, Dong Yan, Kaile Wang, Jiayi Zhou, Yang Han, Josef Dai, Xuehai Pan, Yaodong Yang
| Challenge: | Existing alignment methods share a common topology of information flow, but their alternatives have not been thoroughly explored. |
| Approach: | They propose a theory of reward generalization in reinforcement learning from human feedback . they propose induced Bayesian networks to model the impact of dataset topologies on reward generalisation . |
| Outcome: | The proposed method achieves an average win rate of 65% on three NLP tasks. |
Evaluating Robustness of Large Audio Language Models to Audio Injection: An Empirical Study (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Audio-Language Models (LALMs) are increasingly being deployed in real-world applications, yet their robustness against malicious audio injection remains underexplored. |
| Approach: | They quantitatively assess their vulnerabilities and resilience using metrics: the Defense Success Rate, Context Robustness Score, and Judgment Robustic Index. |
| Outcome: | The proposed models demonstrate significant performance disparities across four attack scenarios. |
SpeechLLM-as-Judges: Towards General and Interpretable Speech Quality Evaluation (2026.acl-long)
Copied to clipboard
Hui Wang, Jinghua Zhao, Yifan Yang, Shujie Liu, Junyang Chen, Yanzhe Zhang, Shiwan Zhao, Jinyu Li, Jiaming Zhou, Haoqin Sun, Yan Lu, Yong Qin
| Challenge: | Existing methods for evaluating the perceptual quality of synthetic speech are limited due to the complexity of perceptual quality factors and the diversity of speech generation tasks. |
| Approach: | They propose a new paradigm for enabling large language models to conduct structured speech quality evaluation using a large-scale dataset. |
| Outcome: | The proposed model performs well across tasks and languages. |