Papers by Xiaohan Xu
V-ALPHASOCIAL: Benchmark and Self-Reflective Chain-of-Thought Generation for Visual Social Commonsense Reasoning (2025.findings-acl)
Copied to clipboard
Zongyu Lin, Zhikun Xu, Xiaohan Song, Yixin Wan, Xingcheng Yao, Tsung-Han Lin, Selina Song, Pranav Subbaraman, Ben Zhou, Kai-Wei Chang, Yizhou Sun
| Challenge: | Social commonsense reasoning is a multimodal task that requires both textual and visual cues. |
| Approach: | They propose a method that integrates visual cues into social commonsense reasoning tasks. |
| Outcome: | The proposed method improves social commonsense reasoning on a multimodal foundation model. |
AlignBench: Benchmarking Chinese Alignment of Large Language Models (2024.acl-long)
Copied to clipboard
Xiao Liu, Xuanyu Lei, Shengyuan Wang, Yue Huang, Andrew Feng, Bosi Wen, Jiale Cheng, Pei Ke, Yifan Xu, Weng Lam Tam, Xiaohan Zhang, Lichao Sun, Xiaotao Gu, Hongning Wang, Jing Zhang, Minlie Huang, Yuxiao Dong, Jie Tang
| Challenge: | Effective evaluation of alignment for emerging Chinese LLMs is still significantly lacking, calling for real-scenario grounded, open-ended, challenging and automatic evaluations tailored for alignment. |
| Approach: | They propose a multi-dimensional benchmark for evaluating LLMs’ alignment in Chinese with 8 main categories, 683 real-scenario rooted queries and corresponding human verified references. |
| Outcome: | The benchmark uses a human-in-the-loop data curation pipeline, 683 real-scenario rooted queries and human verified references. |
Communication Efficient Federated Learning for Multilingual Neural Machine Translation with Adapter (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing frameworks for federated multilingual neural machine translation (Fed-MNMT) are limited in language resources. |
| Approach: | They propose a framework that keeps PLMs frozen and only transfers lightweight adapter modules between clients. |
| Outcome: | The proposed framework reduces communication cost by over 98% while achieving similar or even better performance compared to baselines. |
ChatGLM-Math: Improving Math Problem-Solving in Large Language Models with a Self-Critique Pipeline (2024.findings-emnlp)
Copied to clipboard
Yifan Xu, Xiao Liu, Xinghan Liu, Zhenyu Hou, Yueyan Li, Xiaohan Zhang, Zihan Wang, Aohan Zeng, Zhengxiao Du, Zhao Wenyi, Jie Tang, Yuxiao Dong
| Challenge: | Large language models (LLMs) have shown excellent mastering of human language but struggle in real-world applications that require mathematical problem-solving. |
| Approach: | They propose a pipeline to train a general Math-Critique model from the LLM itself to provide feedback signals and employ rejective fine-tuning and direct preference optimization over the Llm's own generations for data collection. |
| Outcome: | The proposed pipeline outperforms existing LLMs that could be two times larger. |
Holistic Sentence Embeddings for Better Out-of-Distribution Detection (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained language models have shown superior performance for textual OOD detection, but they estimate sample distance scores in the last-layer CLS embedding space. |
| Approach: | They propose to use token averaging and layer combination to boost OOD detection by deriving more holistic sentence embeddings. |
| Outcome: | The proposed method surpasses the state-of-the-art on a comprehensive suite of benchmarks by a 9.33% FAR95 margin. |
Leveraging Large Language Models for NLG Evaluation: Advances and Challenges (2024.emnlp-main)
Copied to clipboard
| Challenge: | introducing Large Language Models (LLMs) has opened new avenues for assessing generated content quality, e.g., coherence, creativity, and context relevance. |
| Approach: | They propose a taxonomy for organizing existing LLM-based evaluation metrics and a structured framework to understand and compare them. |
| Outcome: | The proposed taxonomy offers a framework to understand and compare LLM-based evaluation methods. |
Re-Reading Improves Reasoning in Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Unlike thought-eliciting prompting methods, RE2 shifts the focus to the input by processing questions twice, thereby enhancing the understanding process. |
| Approach: | They introduce a simple, yet general and effective prompting method, RE2, which rereads the question as input. |
| Outcome: | The proposed method demonstrates strong generality and compatibility with most thought-eliciting prompting methods, including CoT. |
EasyEdit: An Easy-to-use Knowledge Editing Framework for Large Language Models (2024.acl-demos)
Copied to clipboard
Peng Wang, Ningyu Zhang, Bozhong Tian, Zekun Xi, Yunzhi Yao, Ziwen Xu, Mengru Wang, Shengyu Mao, Xiaohan Wang, Siyuan Cheng, Kangwei Liu, Yuansheng Ni, Guozhou Zheng, Huajun Chen
| Challenge: | Large Language Models (LLMs) suffer from knowledge cutoff or fallacy issues, which means they are unaware of unseen events or generate text with incorrect facts owing to outdated/noisy data. |
| Approach: | They propose an easy-to-use knowledge editing framework for Large Language Models that allows users to easily edit updated knowledge and adjust undesired behavior while minimizing the impact on unrelated inputs. |
| Outcome: | The proposed framework surpasses traditional fine-tuning in terms of reliability and generalization. |
Expose Backdoors on the Way: A Feature-Based Efficient Defense against Textual Backdoor Attacks (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing online backdoor defense methods for NLP models focus on anomalies at input or output level, causing fragility to adaptive attacks and high computational cost. |
| Approach: | They propose a feature-based online defense method to detect poisoned samples . they use a distance-based anomaly score to distinguish poisones from clean samples based on feature-level regularization . |
| Outcome: | The proposed method outperforms existing methods in sentiment analysis and offense detection tasks. |
AlignMMBench: Evaluating Chinese Multimodal Alignment in Large Vision-Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks focus on basic abilities using nonverbal methods, such as yes-no and multiple-choice questions. |
| Approach: | They propose a benchmark that provides more nuanced evaluations of alignment capabilities for large Vision-Language Models (VLMs) they use a rule-calibrated evaluator that exceeds GPT-4's evaluation ability and a “alignment score” to assess the robustness and stability of models across diverse prompts. |
| Outcome: | The proposed benchmark covers 13 tasks across three categories and includes both single-turn and multi-turn dialogue scenarios. |
Fine-Tuning Deteriorates General Textual Out-of-Distribution Detection by Distorting Task-Agnostic Features (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for detecting out-of-distribution inputs are underexplored . detecting semantic and non-semantic shifts is difficult for pre-tuned pre-trainers . |
| Approach: | They propose a general OOD score that integrates confidence scores from task-agnostic and task-specific representations to improve detecting semantic and non-semantic shifts. |
| Outcome: | The proposed method improves on two cross-task benchmarks with semantic and non-semantic shifts. |
A Cause-Effect Look at Alleviating Hallucination of Knowledge-grounded Dialogue Generation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing dialogue systems have demonstrated impressive performance conducting fluent and natural-sounding conversations, but they are plagued by the Knowledge Hallucination problem. |
| Approach: | They propose a method that exploits the dialogue-knowledge interaction to reduce hallucination by using external knowledge resources to generate more informative responses. |
| Outcome: | The proposed method reduces hallucination without disrupting other dialogue performance while keeping adaptive to different generation models. |