Papers by Bo Xiong
ARM: Alignment with Residual Energy-Based Model (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) acquire a wide range of abilities and abilities, but their behavior does not align with human preferences. |
| Approach: | They propose to minimize a forward Kullback–Leibler divergence from a target policy to a parameteric policy instead of a reverse KL as in RLHF methods. |
| Outcome: | The proposed method can learn an aligned policy by minimizing a forward Kullback–Leibler divergence from a target policy to a parameteric policy instead of a reverse KL as in RLHF methods. |
SiLP: Enhancing Non-Dominant Language Capabilities with a Selective Bidirectional Language Projection Framework (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods to improve performance of large language models rely on additional training objectives or language-specific parameters. |
| Approach: | They propose a bidirectional language projection framework that enables efficient multilingual alignment and language shift using the intrinsic parameters. |
| Outcome: | The proposed framework improves performance of non-dominant languages and improves internal representations. |
SCRIPT: Self-Critic PreTraining of Transformers (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for representation learning of text are masked language modeling (MLM) a language model is trained to learn universal contextual embeddings, which are fine-tuned on a down-stream task. |
| Approach: | They propose a self-critic pretraining transformer for representation learning of text . they demonstrate improved sample-efficiency and improved performance over strong baselines . |
| Outcome: | The proposed model improves sample-efficiency and performance over strong baselines. |
UHGEval: Benchmarking the Hallucination of Chinese Large Language Models via Unconstrained Generation (2024.acl-long)
Copied to clipboard
Xun Liang, Shichao Song, Simin Niu, Zhiyu Li, Feiyu Xiong, Bo Tang, Yezhaohui Wang, Dawei He, Cheng Peng, Zhonghao Wang, Haiying Deng
| Challenge: | Large language models (LLMs) produce hallucinated text, compromising their practical utility in professional contexts. |
| Approach: | They have developed an unconstrained hallucination generation evaluation benchmark that contains hallucines generated by large language models with minimal restrictions. |
| Outcome: | The proposed benchmarks are based on a Chinese-language dataset that is lacking in the field. |
FastMem: Fast Memorization of Prompt Improves Context Awareness of Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models struggle with context awareness, leading to inaccuracies in tasks requiring faithful adherence to provided information. |
| Approach: | They propose a method to enhance LLMs' context awareness by updating only the last Feed-Forward Network module to maximize the likelihood of the prompt before inference . |
| Outcome: | The proposed method improves the accuracy of Llama 3-8B-Inst on the NQ-SWAP dataset from 59.1% to 71.6% and reduces the output structure failure rate of Qwen 1.5-4B-Chat from 34.9% to 25.5%. |
Temporal Fact Reasoning over Hyper-Relational Knowledge Graphs (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models of temporal fact reasoning do not explicitly specify temporal information for each fact. |
| Approach: | They propose a new type of data structure called hyper-relational TKG to study temporal fact reasoning over HKGs. |
| Outcome: | The proposed model is based on two new benchmark HTKG datasets . it provides additional key-value pairs (i.e., qualifiers) for each KG fact . |
zrLLM: Zero-Shot Relational Learning on Temporal Knowledge Graphs with Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods to forecast links on temporal knowledge graphs are embedding-based . but they face a strong challenge in modeling the unseen zero-shot relations . |
| Approach: | They propose to embed knowledge graphs (TKGF) entities and relations based on observed contexts into embedding-based methods to model unseen zero-shot relations. |
| Outcome: | The proposed methods show strong performance on traditional TKG forecasting benchmarks, but they face a strong challenge in modeling unseen zero-shot relations that have no prior graph context. |
SParC: Cross-Domain Semantic Parsing in Context (P19-1)
Copied to clipboard
Tao Yu, Rui Zhang, Michihiro Yasunaga, Yi Chern Tan, Xi Victoria Lin, Suyi Li, Heyang Er, Irene Li, Bo Pang, Tao Chen, Emily Ji, Shreya Dixit, David Proctor, Sungrok Shim, Jonathan Kraft, Vincent Zhang, Caiming Xiong, Richard Socher, Dragomir Radev
| Challenge: | Xu et al., 2017): a dataset for cross-domain semantic parsing in context with 4,298 question sequences. |
| Approach: | They present a dataset for cross-domainSemanticParsing inContext that consists of 4,298 coherent question sequences. |
| Outcome: | The proposed dataset demonstrates that it has greater semantic diversity and can be generalized to unseen domains due to its cross-domain nature and the unseened databases at test time. |
MaskTab: Scalable Masked Tabular Pretraining with Scaling Laws and Distillation for Industrial Classification (2026.findings-acl)
Copied to clipboard
| Challenge: | Tabular data is high-dimensional, riddled with missing entries, and rarely labeled at scale. |
| Approach: | They propose a unified pre-training framework for industrial-scale tabular data . MaskTab encodes missing values via dedicated learnable tokens . |
| Outcome: | The proposed framework outperforms XGBoost and MaskTab-L on industrial-scale . it achieves +5.04% AUC and +8.28% KS over prior art under rigorous scaling . |
ShopSimulator: Evaluating and Exploring RL-Driven LLM Agent for Shopping Assistants (2026.acl-long)
Copied to clipboard
Pei Wang, Yanan Wu, Xiaoshuai Song, Weixun Wang, Gengru Chen, Zhongwen Li, Kezhong Yan, Qi Liu, Ken Deng, Shuaibing Zhao, Shaopan Xiong, Xuepeng Liu, Xuefeng Chen, Wanxi Deng, Wenbo Su, Bo Zheng
| Challenge: | Existing studies on large language model-based agents focus on evaluation benchmarks without training support. |
| Approach: | They propose a large-scale Chinese shopping simulation environment that uses large language models to train agents. |
| Outcome: | The proposed model performs poorly in a large-scale and challenging shopping environment in China. |
MegaPairs: Massive Data Synthesis for Universal Multimodal Retrieval (2025.acl-long)
Copied to clipboard
Junjie Zhou, Yongping Xiong, Zheng Liu, Ze Liu, Shitao Xiao, Yueze Wang, Bo Zhao, Chen Jason Zhang, Defu Lian
| Challenge: | despite the growing demand for multimodal retrieval, there is a lack of training data. |
| Approach: | They propose a data synthesis method that leverages vision language models and open-domain images to generate high-quality data. |
| Outcome: | The proposed method outperforms baseline models on 70 more datasets and can scale up. |
VISTA: Visualized Text Embedding For Universal Multi-Modal Retrieval (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for multimodal retrieval are mostly text-oriented, which lack the capability to process visual information. |
| Approach: | They propose a multi-modal multi-text embedding model VISTA which extends a powerful text encoder with the image understanding capability by introducing visual token embedds. |
| Outcome: | The proposed model achieves superior performance across a variety of multi-modal retrieval tasks in zero-shot and supervised settings. |
Knowledge Graph Embeddings using Neural Ito Process: From Multiple Walks to Stochastic Trajectories (2023.findings-acl)
Copied to clipboard
Mojtaba Nayyeri, Bo Xiong, Majid Mohammadi, Mst. Mahfuja Akter, Mirza Mohtashim Alam, Jens Lehmann, Steffen Staab
| Challenge: | Existing knowledge graph embeddings have problems expressing knowledge graphs because they model a specific relation r from a head h to tails by transitioning deterministically to exactly one other point in the embeddable space. |
| Approach: | They propose a framework that models relations between nodes by relation-specific, stochastic transitions. |
| Outcome: | The proposed framework is expressive and generic subsuming state-of-the-art models operating on low-dimensional manifolds. |
Reinforcement Learning on Pre-Training Data (2026.acl-long)
Copied to clipboard
Siheng Li, Kejiao Li, Zenan Xu, Guanhua Huang, Kun Li, Haoyuan Wu, null Wujiajia, Zihao Zheng, Chenchen Zhang, Kun Shi, Xue Gong, Qi Yi, Ruibin Xiong, Tingqiang Xu, Yuhao Jiang, Jianfeng Yan, Yuyuan Zeng, Guanghui Xu, Jinbao Xue, Zhijiang xu, Zheng Fang, Shuai LI, Qibin Liu, Xiaoxue Li, Zhuoyu Li, Yangyu Tao, Fei Gao, Cheng Jiang, Bochao Wang, Kai Liu, Jianchen Zhu, Wai Lam, Bo Zhou, Di Wang
| Challenge: | Recent progress in large language models is driven by scaling of training compute through pre-training with nexttoken prediction (NTP) or post-training (RL) Pre-training using NTP enables models to acquire extensive knowledge and skills from general data, but it suffers from data inefficiency and catastrophic forgetting in continual learning settings. |
| Approach: | They propose to scale training compute through pre-training with next-token prediction (NTP) or post-training by scaling reinforcement learning (RL) to improve learning from general data. |
| Outcome: | Experiments on multiple benchmarks and models show that the proposed approach improves continual pre-training and provides a strong foundation for post-training on Qwen3-8B-Base. |
Conformalized Answer Set Prediction for Knowledge Graph Embedding (2025.naacl-long)
Copied to clipboard
| Challenge: | Knowledge graph embeddings (KGE) map entities and predicates into numerical vectors, providing non-classical reasoning capabilities based on similarities and analogies between entities and relations. |
| Approach: | They propose to use knowledge graph embeddings to provide non-classical reasoning capabilities by exploiting similarities and analogies between entities and relations. |
| Outcome: | The proposed model can generate answer sets with probabilistic guarantees on four benchmark datasets and is scaled well with respect to the difficulty of the query. |
Controlled Text Generation for Large Language Model with Dynamic Attribute Graphs (2024.findings-acl)
Copied to clipboard
| Challenge: | Controlled Text Generation (CTG) aims to produce texts that exhibit specific desired attributes. |
| Approach: | They propose a pluggable CTG framework for Large Language Models to control text . they use attribute scorers to evaluate attributes of sentences and construct dynamic attribute graphs . |
| Outcome: | The proposed framework achieves a peak improvement of 19.29% over baseline methods in two tasks. |
LANDeRMT: Dectecting and Routing Language-Aware Neurons for Selectively Finetuning LLMs to Machine Translation (2024.acl-long)
Copied to clipboard
| Challenge: | Existing studies have shown promising results in multilingual translation with limited bilingual supervision. |
| Approach: | They propose a Language-Aware Neuron Detecting and Routing framework that fine tunes LLMs to Machine Translation with diverse translation training data. |
| Outcome: | The proposed framework selectively finetunes LLMs to MT tasks with diverse translation training data. |
Low-probability Tokens Sustain Exploration in Reinforcement Learning with Verifiable Reward (2026.findings-acl)
Copied to clipboard
Guanhua Huang, Tingqiang Xu, Mingze Wang, Qi Yi, Xue Gong, Siheng Li, Ruibin Xiong, Kejiao Li, Yuhao Jiang, Bo Zhou
| Challenge: | Recent studies show that RLVR training is slow and results plateau as policy entropy collapses . low-probability regularization (Lp-Reg) reduces the number of low-quality exploratory tokens induced by RL training . |
| Approach: | They propose a method to reduce RLVR over-penalization by eliminating low-probability exploratory tokens . they propose 'Low-provability Regularization' to reduce the gradual elimination of low-quality exploratory entropy tokens. |
| Outcome: | The proposed method eliminates low-probability exploratory tokens and prevents suppression of potentially valuable low-property candidates. |
SharPT: Shared Latent Space Prompt Tuning (2023.findings-eacl)
Copied to clipboard
| Challenge: | Prompt tuning is an efficient method for adapting large language models, but it is difficult and expensive to identify the source task that provides optimal prompts. |
| Approach: | They propose to learn a shared latent space which captures a set of basis skills from a mixture of source tasks and then transfer them to target tasks. |
| Outcome: | The proposed method outperforms previous methods on NLI, sentence completion, QA, conference resolution, word sense disambiguation and on various model scales. |
Adversarial Preference Learning for Robust LLM Alignment (2025.findings-acl)
Copied to clipboard
Yuanfu Wang, Pengyu Wang, Chenyang Xi, Bo Tang, Junyi Zhu, Wenqiang Wei, Chen Chen, Chao Yang, Jingfeng Zhang, Chaochao Lu, Yijun Niu, Keming Mao, Zhiyu Li, Feiyu Xiong, Jie Hu, Mingchuan Yang
| Challenge: | Modern language models rely on Reinforcement Learning from Human Feedback (RLHF) to encourage safe behaviors, but they remain vulnerable to adversarial attacks due to three key limitations: (1) the inefficiency and high cost of human annotation; (2) the vast diversity of potential adversarials; and (3) the risk of feedback bias and reward hacking. |
| Approach: | They propose an iterative adversarial training method that incorporates three key innovations to address these challenges. |
| Outcome: | Experiments on Mistral-7B-Instruct-v0.3 show that the proposed method significantly enhances robustness and reduces harmful outputs from 5.88% to 0.43%. |
LLM-Based Multi-Hop Question Answering with Knowledge Graph Integration in Evolving Environments (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for knowledge editing in Large Language Models face difficulties with multi-hop questions that require accurate fact identification and sequential logical reasoning. |
| Approach: | They propose a method that merges explicit knowledge representations of Knowledge Graphs with the linguistic flexibility of Large Language Models to convert free-form language into structured queries and fact triples. |
| Outcome: | The proposed method significantly surpasses state-of-the-art knowledge editing methods in the multi-hop question answering benchmark, MQuAKE. |
SEMMA: A Semantic Aware Knowledge Graph Foundation Model (2025.emnlp-main)
Copied to clipboard
Arvindh Arun, Sumit Kumar, Mojtaba Nayyeri, Bo Xiong, Ponnurangam Kumaraguru, Antonio Vergari, Steffen Staab
| Challenge: | Existing Knowledge Graph Foundation Models (KGFMs) rely on graph structure, overlooking the rich semantic signals encoded in textual attributes. |
| Approach: | They propose a dual-module KGFM that integrates transferable textual semantics alongside structure to generate relation identifiers. |
| Outcome: | The proposed model outperforms ULTRA and ULtra in fully inductive link prediction in more challenging generalization settings. |
Predictive Multiplicity of Knowledge Graph Embeddings in Link Prediction (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Knowledge graph embeddings (KGE) models are often used to predict missing links for knowledge graphs (KGs) however, multiple KG embedds can give conflicting predictions for unseen queries. |
| Approach: | They define predictive multiplicity in link prediction and introduce evaluation metrics to measure it using commonly used benchmark datasets. |
| Outcome: | The proposed methods significantly mitigat conflicts by 66% to 78% in link prediction. |
MoC: Mixtures of Text Chunking Learners for Retrieval-Augmented Generation System (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for text chunking are limited by text chunks and lack of domain-specific knowledge. |
| Approach: | They propose a dual-metric evaluation method to quantify text chunking quality . they aim to generate a structured list of chunking regular expressions . |
| Outcome: | The proposed method enables direct quantification of chunking quality . it substantiates the need to integrate LLMs into chunking process . |
Text2Mem: A Unified Memory Operation Language for Memory Operating System (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing memory frameworks lack a formal, executable specification for memory control. |
| Approach: | They propose a unified memory operation language that standardizes translation of natural-language instructions into reliable execution. |
| Outcome: | The proposed language standardizes translation of natural-language instructions into reliable execution. |
Normalized Contrastive Learning for Text-Video Retrieval (2022.emnlp-main)
Copied to clipboard
| Challenge: | Cross-modal contrastive learning suffers from incorrect normalization of the sum retrieval probabilities of each text or video instance. |
| Approach: | They propose a normalized contrastive learning algorithm that normalizes the sum retrieval probabilities of each instance so that every text and video instance is fairly represented. |
| Outcome: | Empirical results show that NCL brings significant gains in text-video retrieval on different model architectures without any architecture engineering. |
TGEA: An Error-Annotated Dataset and Benchmark Tasks for TextGeneration from Pretrained Language Models (2021.acl-long)
Copied to clipboard
| Challenge: | Using pretrained language models, we propose an error-annotated dataset for text generation . we use carefully selected prompt words to guide GPT-2 to generate candidate sentences . |
| Approach: | They propose an error-annotated dataset with multiple benchmark tasks for text generation from pretrained language models. |
| Outcome: | The proposed dataset covers 24 types of errors according to common sense and linguistics. |
NewsBench: A Systematic Evaluation Framework for Assessing Editorial Capabilities of Large Language Models in Chinese Journalism (2024.acl-long)
Copied to clipboard
Miao Li, Ming-Bin Chen, Bo Tang, ShengbinHou ShengbinHou, Pengyu Wang, Haiying Deng, Zhiyu Li, Feiyu Xiong, Keming Mao, Cheng Peng, Yi Luo
| Challenge: | a novel evaluation framework assesses the capabilities of Large Language Models (LLMs) for editorial capabilities in Chinese journalism. |
| Approach: | They propose to use a benchmark dataset to assess the capabilities of Large Language Models (LLMs) for editorial capabilities in Chinese journalism. |
| Outcome: | The proposed evaluation framework is based on a dataset of 1,267 test samples in 24 news domains. |
GuessArena: Guess Who I Am? A Self-Adaptive Framework for Evaluating LLMs in Domain-Specific Knowledge and Reasoning (2025.acl-long)
Copied to clipboard
| Challenge: | Existing evaluation methods for large language models rely on static benchmarks and standardized evaluation protocols. |
| Approach: | They propose an adaptive evaluation framework that integrates dynamic domain knowledge modeling with progressive reasoning assessment to improve evaluation fidelity. |
| Outcome: | Empirical results show that the framework distinguishes LLMs in terms of domain knowledge coverage and reasoning chain completeness. |
Long Document Summarization with Top-down and Bottom-up Inference (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent models infer latent representations of words or tokens with a transformer encoder, which is bottom-up and thus does not capture long-distance context well. |
| Approach: | They propose a method to infer latent representations of words or tokens in documents . they assume a hierarchical structure of a document where top-level captures long range dependency . |
| Outcome: | The proposed model can summarize an entire book and achieve competitive performance on a wide range of document summarization benchmarks. |
CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databases (D19-1)
Copied to clipboard
Tao Yu, Rui Zhang, Heyang Er, Suyi Li, Eric Xue, Bo Pang, Xi Victoria Lin, Yi Chern Tan, Tianze Shi, Zihan Li, Youxuan Jiang, Michihiro Yasunaga, Sungrok Shim, Tao Chen, Alexander Fabbri, Zifan Li, Luyao Chen, Yuwen Zhang, Shreya Dixit, Vincent Zhang, Caiming Xiong, Richard Socher, Walter Lasecki, Dragomir Radev
| Challenge: | CoSQL is a corpus for building cross-domain, general-purpose database querying dialogue systems. |
| Approach: | They present a corpus for building cross-domain, general-purpose database querying dialogue systems . they use a Wizard-of-Oz collection of 3k turns plus 10k+ annotated SQL queries . |
| Outcome: | The proposed corpus is based on a Wizard-of-Oz dataset of 3k dialogues querying 200 complex DBs spanning 138 domains. |
A Systematic Examination of Preference Learning through the Lens of Instruction-Following (2025.naacl-long)
Copied to clipboard
Joongwon Kim, Anirudh Goyal, Aston Zhang, Bo Xiong, Rui Hou, Melanie Kambadur, Dhruv Mahajan, Hannaneh Hajishirzi, Liang Tan
| Challenge: | a recent study has found that preference learning is a key tool for enhancing LLM training and alignment. |
| Approach: | They use a synthetic data generation pipeline to generate 48,000 unique instruction-following prompts with 23 verifiable constraints to obtain preference pairs. |
| Outcome: | The proposed pipeline generates 48,000 unique instruction-following prompts with 23 verifiable constraints that enable fine-grained and automated quality assessments of model responses. |
Predicate-Conditional Conformalized Answer Sets for Knowledge Graph Embeddings (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods provide probabilistic guarantees over a reference set of queries and answers, but they fail to identify when the answers to a query are uncertain. |
| Approach: | They propose a method that approximates predicate-conditional coverage guarantees while maintaining compact prediction sets. |
| Outcome: | The proposed method provides predicate-conditional coverage guarantees while maintaining compact prediction sets. |
GraphNarrator: Generating Textual Explanations for Graph Neural Networks (2025.acl-long)
Copied to clipboard
| Challenge: | Graph representation learning has garnered significant attention due to its broad applications in various domains, such as recommendation systems and social network analysis. |
| Approach: | They propose to use a generative language model to map input-output pairs to explanations reflecting the model’s decision-making process to generate a model that generates pseudo-labels that capture the model's decisions from saliency-based explanations. |
| Outcome: | Extensive experiments show that GraphNarrator produces human-preferred explanations that are faithful, concise, and human-like. |
Shrinking Embeddings for Hyper-Relational Knowledge Graphs (2023.acl-long)
Copied to clipboard
| Challenge: | Existing studies have focused on binary relational KGs where each fact is represented by a triple. |
| Approach: | They propose a geometric hyper-relational KG embedding method that explicitly models qualifier monotonicity, qualifier implication, and qualifier mutual exclusion. |
| Outcome: | The proposed method outperforms existing methods on three benchmarks of hyper-relational KGs. |
CARE-STaR: Constraint-aware Self-taught Reasoner (2025.findings-acl)
Copied to clipboard
Zhiliang Li, Bo Tang, Yijun Niu, Beihong Jin, Qiwen Shi, Yuchen Feng, Zhiyu Li, Jie Hu, Mingchuan Yang, Feiyu Xiong
| Challenge: | Recent research on instruction following has demonstrated that LLMs can handle complex instructions. |
| Approach: | They propose to assign constraints to different levels of constraints in instructions . they use chain-of-thought and self-taught reasoner methods to identify constraints . |
| Outcome: | The proposed method outperforms supervised fine-tuning (SFT) on three instruction-following benchmarks. |
Inside Out: Evolving User-Centric Core Memory Trees for Long-Term Personalized Dialogue Systems (2026.acl-long)
Copied to clipboard
| Challenge: | Existing personalized dialogue systems struggle to reconcile unbounded interactions with finite context constraints. |
| Approach: | They propose a framework that utilizes a globally maintained PersonaTree as the carrier of long-term user profiling. |
| Outcome: | The proposed framework outperforms existing systems in suppressing contextual noise and persona inconsistency. |
SafeRAG: Benchmarking Security in Retrieval-Augmented Generation of Large Language Model (2025.acl-long)
Copied to clipboard
Xun Liang, Simin Niu, Zhiyu Li, Sensen Zhang, Hanyu Wang, Feiyu Xiong, Zhaoxin Fan, Bo Tang, Jihao Zhao, Jiawei Yang, Shichao Song, Mengwei Wang
| Challenge: | Existing approaches to integrating external knowledge into large language models (LLMs) however, the incorporation of external knowledge increases the vulnerability of LLMs . |
| Approach: | They propose a benchmark to evaluate the RAG security using a dataset . they classify attack tasks into silver noise, inter-context conflict, soft ad, and white Denial-of-Service . |
| Outcome: | The proposed benchmark evaluates the security of RAG against 14 representative RAG components. |
MLAS-LoRA: Language-Aware Parameters Detection and LoRA-Based Knowledge Transfer for Multilingual Machine Translation (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated strong performance even with limited parallel data. |
| Approach: | They propose a multiple language-aware LoRA knowledge transfer framework that selectively adapts LLMs to MT by transferring knowledge from a large teacher to a small student model. |
| Outcome: | The proposed framework outperforms baseline models on multilingual language pairs by +1.7 BLEU on average. |