Papers by Junyu Wang
Unity in Diversity: Collaborative Pre-training Across Multimodal Medical Sources (2024.acl-long)
Copied to clipboard
Xiaochen Wang, Junyu Luo, Jiaqi Wang, Yuan Zhong, Xiaokun Zhang, Yaqing Wang, Parminder Bhatia, Cao Xiao, Fenglong Ma
| Challenge: | Current pre-training techniques rely on a limited scope of medical data, limiting the range of downstream tasks. |
| Approach: | They propose a pre-training strategy that unifies patient data within individual sources and captures explicit and implicit correlations between patients across different sources. |
| Outcome: | The proposed strategy bridges the gap between multimodal medical sources by aggregating patient data within individual sources and capturing explicit and implicit correlations between patients across sources. |
CoRelation: Boosting Automatic ICD Coding through Contextualized Code Relation Learning (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for boosting ICD coding performance lack a model for complex code relations . current methods overlook the importance of context in clinical notes . |
| Approach: | They propose a contextualized and flexible framework to enhance learning of ICD code relations . they use clinical notes to model all possible code relations using a dependent learning paradigm . |
| Outcome: | The proposed approach improves on six public ICD coding datasets compared to state-of-the-art models. |
A Survey on Efficient Large Language Model Training: From Data-centric Perspectives (2025.acl-long)
Copied to clipboard
Junyu Luo, Bohan Wu, Xiao Luo, Zhiping Xiao, Yiqiao Jin, Rong-Cheng Tu, Nan Yin, Yifan Wang, Jingyang Yuan, Wei Ju, Ming Zhang
| Challenge: | achieving data-efficient post-training of Large Language Models is a key research question. |
| Approach: | They propose a taxonomy of data-efficient LLM post-training methods from a data-centric perspective. |
| Outcome: | The proposed methods cover data selection, data quality enhancement, synthetic data generation, data distillation and compression, and self-evolving data ecosystems. |
PclGPT: A Large Language Model for Patronizing and Condescending Language Detection (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Patronizing and condescending language is an essential branch of toxic language . pre-trained language models perform poorly in detecting PCL due to its implicit toxicity traits . |
| Approach: | They propose a novel LLM benchmark for patronizing and condescending language . they use a dataset to analyze the toxicity of patronizing condescending languages . |
| Outcome: | The proposed model can detect patronizing and condescending language (PCL) the model can be used to analyze the toxicity of the language and to improve the detection. |
Is LLM an Overconfident Judge? Unveiling the Capabilities of LLMs in Detecting Offensive Language with Annotation Disagreement (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing studies focus on evaluating large language models' ability to handle disagreement cases. |
| Approach: | They evaluate the performance of large language models in detecting offensive language at varying levels of agreement. |
| Outcome: | The proposed model improves detection accuracy and model alignment with human judgment by using disagreement samples in training. |
MELA: Multilingual Evaluation of Linguistic Acceptability (2024.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks on linguistic acceptability have been used to evaluate language models' ability to distinguish between acceptable and unacceptable sentences. |
| Approach: | They present the largest benchmark to date on linguistic acceptability: MELA . they establish LLM baselines on this benchmark and investigate cross-lingual transfer in acceptability judgements with XLM-R. |
| Outcome: | The proposed model outperforms open-source models on cross-lingual transfer in acceptability judgements. |
AlphaOne: Reasoning Models Thinking Slow and Fast at Test Time (2025.emnlp-main)
Copied to clipboard
Junyu Zhang, Runpei Dong, Han Wang, Xuying Ning, Haoran Geng, Peihao Li, Xialin He, Yutong Bai, Jitendra Malik, Saurabh Gupta, Huan Zhang
| Challenge: | Existing monotonic scaling methods for large reasoning models are not reliable. |
| Approach: | They propose a universal framework for modulating reasoning progress in large reasoning models at test time. |
| Outcome: | The proposed framework unifies and generalizes existing monotonic scaling methods and enables flexible and dense slow-to-fast reasoning modulation. |
Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention (2025.acl-long)
Copied to clipboard
Jingyang Yuan, Huazuo Gao, Damai Dai, Junyu Luo, Liang Zhao, Zhengyan Zhang, Zhenda Xie, Yuxing Wei, Lean Wang, Zhiping Xiao, Yuqing Wang, Chong Ruan, Ming Zhang, Wenfeng Liang, Wangding Zeng
| Challenge: | Long-context modeling is crucial for next-generation language models, but high computational cost of standard attention mechanisms poses significant computational challenges. |
| Approach: | They propose a natively trained Sparse Attention mechanism that integrates algorithms with hardware-aligned optimizations to achieve efficient long-context modeling. |
| Outcome: | The proposed model maintains or exceeds Full Attention models across general benchmarks, long-context tasks, and instruction-based reasoning. |
Evaluating the Expressive Appropriateness of Speech in Rich Contexts (2026.acl-long)
Copied to clipboard
Tianrui Wang, Ziyang Ma, Yizhou Peng, Haoyu Wang, Zhikang Niu, Zikang Huang, Yihao Wu, Yi-Wen Chao, Yu Jiang, Yuheng Lu, Guanrou Yang, Xuanchen Li, Hexin Liu, Chunyu Qiang, Cheng Gong, Yifan Yang, Tianchi Liu, Junyu Wang, Nana Hou, Meng Ge, Fuming You, Yang Wei, Zhongqian Sun, Hu Haifeng, Xiaobao Wang, Eng Siong Chng, Xie Chen, Longbiao Wang, Jianwu Dang
| Challenge: | Existing methods for evaluating expressive speech focus on word accuracy, naturalness, signal quality, or emotional intensity at the utterance level. |
| Approach: | They propose a framework for Evaluating Expressive Appropriateness in speech that assesses whether a speech sample aligns with the underlying communicative intent implied by its discourse-level narrative context. |
| Outcome: | The proposed framework outperforms existing speech evaluation and analysis systems on a human-annotated test set. |
UPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluation (2023.emnlp-main)
Copied to clipboard
Daixuan Cheng, Shaohan Huang, Junyu Bi, Yuefeng Zhan, Jianfeng Liu, Yujing Wang, Hao Sun, Furu Wei, Weiwei Deng, Qi Zhang
| Challenge: | Large Language Models (LLMs) have impressive capabilities but need for task-specific prompt engineering can hinder their generalization. |
| Approach: | They propose a lightweight and versatile retriever that automatically retrieves prompts for a given zero-shot task input. |
| Outcome: | The proposed model is universally applicable across tasks and models . it mitigates hallucination problem in chatGPT, and it improves even the strongest LLMs. |
Hierarchical Pretraining on Multimodal Electronic Health Records (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing pretraining models on EHR data are too specific, limiting their transferability. |
| Approach: | They propose a general, unified pretraining framework for hierarchically multimodal EHR data that can be used to train models on a large dataset before fine-tuning it on 'upstream' tasks. |
| Outcome: | The proposed model performs on eight downstream tasks spanning three levels and compares with baselines on 18 different tasks. |