Papers by Bo Xue
Syntax-guided Localized Self-attention by Constituency Syntactic Distance (2022.findings-emnlp)
Copied to clipboard
Shengyuan Hou, Jushi Kai, Haotian Xue, Bingyu Zhu, Bo Yuan, Longtao Huang, Xinbing Wang, Zhouhan Lin
| Challenge: | Recent studies have shown that Transformers is implicitly learning syntactic information from data, albeit is highly dependent on the quality and scale of the training data. |
| Approach: | They propose a syntax-guided localized self-attention model that allows directly incorporating grammar structures from an external constituency parser. |
| Outcome: | The proposed model improves translation performance on a variety of datasets, from small to large datasets and with different source languages. |
From Adversarial Arms Race to Model-centric Evaluation: Motivating a Unified Automatic Robustness Evaluation Framework (2023.findings-acl)
Copied to clipboard
Yangyi Chen, Hongcheng Gao, Ganqu Cui, Lifan Yuan, Dehan Kong, Hanlu Wu, Ning Shi, Bo Yuan, Longtao Huang, Hui Xue, Zhiyuan Liu, Maosong Sun, Heng Ji
| Challenge: | Existing models of robustness evaluation are incomprehensive, impractical, and invalid . |
| Approach: | They propose a framework for automatic robustness evaluation that shifts towards model-centric evaluation to further exploit the advantages of adversarial attacks. |
| Outcome: | The proposed framework is based on a model-centric evaluation protocol and a robustness evaluation protocol. |
DREAM: Improving Video-Text Retrieval Through Relevance-Based Augmentation Using Large Foundation Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent advances in video-text retrieval models have limited training data annotations. |
| Approach: | They propose a Video-Text Retrieval Paradigm with Relevance-based Augmentation which enhances video and text data using large foundation models to learn more generalized features. |
| Outcome: | The proposed method improves video-text retrieval performance over existing methods. |
Reinforcement Learning on Pre-Training Data (2026.acl-long)
Copied to clipboard
Siheng Li, Kejiao Li, Zenan Xu, Guanhua Huang, Kun Li, Haoyuan Wu, null Wujiajia, Zihao Zheng, Chenchen Zhang, Kun Shi, Xue Gong, Qi Yi, Ruibin Xiong, Tingqiang Xu, Yuhao Jiang, Jianfeng Yan, Yuyuan Zeng, Guanghui Xu, Jinbao Xue, Zhijiang xu, Zheng Fang, Shuai LI, Qibin Liu, Xiaoxue Li, Zhuoyu Li, Yangyu Tao, Fei Gao, Cheng Jiang, Bochao Wang, Kai Liu, Jianchen Zhu, Wai Lam, Bo Zhou, Di Wang
| Challenge: | Recent progress in large language models is driven by scaling of training compute through pre-training with nexttoken prediction (NTP) or post-training (RL) Pre-training using NTP enables models to acquire extensive knowledge and skills from general data, but it suffers from data inefficiency and catastrophic forgetting in continual learning settings. |
| Approach: | They propose to scale training compute through pre-training with next-token prediction (NTP) or post-training by scaling reinforcement learning (RL) to improve learning from general data. |
| Outcome: | Experiments on multiple benchmarks and models show that the proposed approach improves continual pre-training and provides a strong foundation for post-training on Qwen3-8B-Base. |
Low-probability Tokens Sustain Exploration in Reinforcement Learning with Verifiable Reward (2026.findings-acl)
Copied to clipboard
Guanhua Huang, Tingqiang Xu, Mingze Wang, Qi Yi, Xue Gong, Siheng Li, Ruibin Xiong, Kejiao Li, Yuhao Jiang, Bo Zhou
| Challenge: | Recent studies show that RLVR training is slow and results plateau as policy entropy collapses . low-probability regularization (Lp-Reg) reduces the number of low-quality exploratory tokens induced by RL training . |
| Approach: | They propose a method to reduce RLVR over-penalization by eliminating low-probability exploratory tokens . they propose 'Low-provability Regularization' to reduce the gradual elimination of low-quality exploratory entropy tokens. |
| Outcome: | The proposed method eliminates low-probability exploratory tokens and prevents suppression of potentially valuable low-property candidates. |
SparkRA: A Retrieval-Augmented Knowledge Service System Based on Spark Large Language Model (2024.emnlp-demo)
Copied to clipboard
Dayong Wu, Jiaqi Li, Baoxin Wang, Honghong Zhao, Siyuan Xue, Yanjie Yang, Zhijun Chang, Rui Zhang, Li Qian, Bo Wang, Shijin Wang, Zhixiong Zhang, Guoping Hu
| Challenge: | Large language models (LLMs) have shown remarkable achievements across various language tasks. |
| Approach: | They propose a scientific literature LLM and a knowledge service system based on it . they collect scientific literature and then pre-train it using autoregressive training . |
| Outcome: | The proposed system provides literature investigation, paper reading, and academic writing functions. |
Exploring and Verbalizing Academic Ideas by Concept Co-occurrence (2023.acl-long)
Copied to clipboard
| Challenge: | a new framework for academic idea inspiration is being developed for academic research assistants . number of academic publications is increasing exponentially, making it difficult for an independent researcher to understand these papers thoroughly. |
| Approach: | They propose a framework based on concept co-occurrence for academic idea inspiration . they construct evolving concept graphs according to the co-existence relationship of concepts from 20 disciplines or topics . |
| Outcome: | The proposed system can be used to explore connections between academic concepts and verbalize the new ideas. |
RoChBert: Towards Robust BERT Fine-tuning for Chinese (2022.findings-emnlp)
Copied to clipboard
Zihan Zhang, Jinfeng Li, Ning Shi, Bo Yuan, Xiangyu Liu, Rong Zhang, Hui Xue, Donghong Sun, Chao Zhang
| Challenge: | Pre-trained language models (e.g., BERT) have been proved vulnerable to adversarial texts. |
| Approach: | They propose to fuse Chinese phonetic and glyph features into pre-trained models by using a more comprehensive adversarial graph. |
| Outcome: | The proposed framework outperforms existing methods in significant ways on a wide range of tasks while remaining accurate on benign texts. |
RTCFake: Speech Deepfake Detection in Real-Time Communication (2026.findings-acl)
Copied to clipboard
Jun Xue, Zhuolin Yi, Yihuan Huang, Yanzhen Ren, Yujie Chen, Cunhang Fan, Zicheng Su, Yongcheng Zhang, Bo Cai
| Challenge: | Existing detection studies focus on offline simulations and struggle to cope with complex distortions introduced during RTC transmission. |
| Approach: | They propose a large-scale speech deepfake dataset tailored for RTC scenarios . the dataset is constructed by transmitting speech through multiple social media and conferencing platforms . |
| Outcome: | The proposed dataset is constructed by transmitting speech through multiple mainstream social media and conferencing platforms, enabling precise pairing between offline and online speech. |
Quantifying Compositionality of Classic and State-of-the-Art Embeddings (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Static word embeddings make strong claims about compositionality, but the SOTA generative models go too far in the other direction. |
| Approach: | a new study evaluates the compositionality of word embeddings by canonical correlation analysis . strong compositional signals are observed in later training stages across data modalities . |
| Outcome: | a new evaluation of compositional models shows that they exploit access meanings when justified . strong compositional signals are observed in later training stages and in deeper layers of the transformer-based model before a decline at the top layer. |
CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databases (D19-1)
Copied to clipboard
Tao Yu, Rui Zhang, Heyang Er, Suyi Li, Eric Xue, Bo Pang, Xi Victoria Lin, Yi Chern Tan, Tianze Shi, Zihan Li, Youxuan Jiang, Michihiro Yasunaga, Sungrok Shim, Tao Chen, Alexander Fabbri, Zifan Li, Luyao Chen, Yuwen Zhang, Shreya Dixit, Vincent Zhang, Caiming Xiong, Richard Socher, Walter Lasecki, Dragomir Radev
| Challenge: | CoSQL is a corpus for building cross-domain, general-purpose database querying dialogue systems. |
| Approach: | They present a corpus for building cross-domain, general-purpose database querying dialogue systems . they use a Wizard-of-Oz collection of 3k turns plus 10k+ annotated SQL queries . |
| Outcome: | The proposed corpus is based on a Wizard-of-Oz dataset of 3k dialogues querying 200 complex DBs spanning 138 domains. |
How Good Are LLMs at Out-of-Distribution Detection? (2024.lrec-main)
Copied to clipboard
| Challenge: | Out-of-distribution (OOD) detection is crucial for ensuring AI safety . large language models (LLMs) are becoming more prevalent due to their scale, pre-training objectives, and paradigms used for inference. |
| Approach: | They propose to use large language models to investigate out-of-distribution (OOD) detection in machine learning. |
| Outcome: | The proposed method outperforms other OOD detectors in zero-grad and fine-tuning scenarios. |
Dynamic Evil Score-Guided Decoding: An Efficient Decoding Framework For Red-Team Model (2025.findings-acl)
Copied to clipboard
Cong Gao, Bo Zhang, Linkang Yang, Minghao Hu, Zhunchen Luo, Xiaoying Bai, Guotong Geng, Jun Zhang, Yunhua Xue
| Challenge: | Existing red-teaming methods require expensive fine-tuning, especially for large LLMs. |
| Approach: | They propose a red-teaming method that uses an ‘evil score’ to evaluate the potential of tokens to contribute to harmful outputs during decoding. |
| Outcome: | The proposed method achieves an ASR of 92.83% on the Llama-3.2-3B-Instruct model, compared to 83.48% with adversarial fine-tuning while using less computational resources. |
Balance Act: Mitigating Hubness in Cross-Modal Retrieval with Query and Gallery Banks (2023.emnlp-main)
Copied to clipboard
| Challenge: | a small number of gallery data points are frequently retrieved, resulting in a decline in retrieval performance. |
| Approach: | They propose a framework that leverages both gallery and query data to address hubness . they propose dual inverted softmax and dual dynamic inverted hardmax methods to normalize similarity . |
| Outcome: | The proposed framework reduces the occurrence of hubs during inference while improving similarity between non-hubs and queries. |
Rhombus: Incentivizing Coordination in Parallel Thinking through Reinforcement Learning (2026.findings-acl)
Copied to clipboard
Ziyuan Nan, Qi Yi, Di Huang, Yutong Wu, Guanhua Huang, Xue Gong, Kejiao Li, Yuhao Jiang, Chenchen Zhang, Zenan Xu, Xing Hu, Bo Zhou
| Challenge: | Parallel thinking is a promising avenue for scaling test-time compute in Large Language Models . however, coordinating the exploration and aggregation stages remains challenging . |
| Approach: | They propose a parallel thinking framework that explicitly incentivizes coordination between components via end-to-end reinforcement learning. |
| Outcome: | The proposed framework improves accuracy by 6.0% over long chain-of-thought baselines while reducing wall-clock latency by 39.4% under matched token budgets. |