Papers by Yu Qiao

P-FOLIO: Evaluating and Improving Logical Reasoning with Abundant Human-Written Reasoning Chains (2024.findings-emnlp)

Challenge:	Mental health problems are a challenge to our modern society, and their prevalence is predicted to increase worldwide.
Approach:	They propose a large-scale, carefully constructed dataset for MHC detection built on high-precision patterns and the approach proposed for English.
Outcome:	The proposed model leverages engineered (psycho-)linguistic features as well as BERT-German to facilitate further research and conduct extensive experiments.

Copied to clipboard

Simeng Han, Aaron Yu, Rui Shen, Zhenting Qi, Martin Riddell, Wenfei Zhou, Yujie Qiao, Yilun Zhao, Semih Yavuz, Ye Liu, Shafiq Joty, Yingbo Zhou, Caiming Xiong, Dragomir Radev, Rex Ying, Arman Cohan

Information Extraction & Text Mining

Challenge:	Existing methods on understanding the capabilities of LLMs in logical reasoning rely on binary entailment classification or synthetically derived rationales.
Approach:	They propose to annotate a human-annotated dataset consisting of diverse and complex reasoning chains for a set of realistic logical reasoning stories also written by humans.
Outcome:	The proposed model outperforms existing methods on understanding the capabilities of LLMs in logical reasoning by 10% or more.

Fake Alignment: Are LLMs Really Aligned Well? (2024.naacl-long)

Copied to clipboard

Yixu Wang, Yan Teng, Kexin Huang, Chengqi Lyu, Songyang Zhang, Wenwei Zhang, Xingjun Ma, Yu-Gang Jiang, Yu Qiao, Yingchun Wang

Inference-Time Language Model Alignment via Integrated Value Guidance (2024.findings-emnlp)

Challenge:	Existing studies on large language models have shown that they are poorly aligned in practice.
Approach:	They propose a framework to evaluate safety in large language models . they propose two new metrics to quantify fake alignment and obtain corrected performance estimation.
Outcome:	The proposed framework and two metrics show that some models with purported safety are poorly aligned in practice.

Copied to clipboard

Zhixuan Liu, Zhanhui Zhou, Yuanfu Wang, Chao Yang, Yu Qiao

Towards Tracing Trustworthiness Dynamics: Revisiting Pre-training Period of Large Language Models (2024.findings-acl)

Challenge:	Large language models are fine-tuned to align with human preferences, but tuning large models is computationally intensive and complex.
Approach:	They propose a method that uses implicit and explicit value functions to guide language model decoding at token and chunk-level respectively.
Outcome:	The proposed method outperforms traditional methods and circumvents the complexities of fine-tuning.

Copied to clipboard

Chen Qian, Jie Zhang, Wei Yao, Dongrui Liu, Zhenfei Yin, Yu Qiao, Yong Liu, Jing Shao

LLaMAX: Scaling Linguistic Horizons of LLM by Enhancing Translation Capabilities Beyond 100 Languages (2024.findings-emnlp)

Challenge:	Existing studies focus on pre-trained LLMs to better understand and improve their trustworthiness.
Approach:	They apply linear probing to LLMs to explore five key dimensions of trustworthiness: reliability, privacy, toxicity, fairness, and robustness.
Outcome:	The proposed model can distinguish concepts in each trustworthiness dimension, suggesting that it can be trained in early pre-training.

Copied to clipboard

Yinquan Lu, Wenhao Zhu, Lei Li, Yu Qiao, Fei Yuan

Multilinguality

Challenge:	Large Language Models (LLMs) exhibit exceptional translation capabilities in high-resource language tasks, yet their effectiveness in low-resourced languages is suboptimal.
Approach:	They conduct extensive multilingual continual pre-training on the LLaMA series models and develop LLiMAX for translation support across more than 100 languages.
Outcome:	The proposed model achieves higher translation performance than existing open-source models and performs on-par with specialized translation model on the Flores-101 benchmark.

Understanding the Dynamics of Second Language Writing through Keystroke Logging and Complexity Contours (2020.lrec-1)

Copied to clipboard

Elma Kerz, Fabio Pruneri, Daniel Wiechmann, Yu Qiao, Marcus Ströbel

Challenge:	a large corpus of data on keystroke logging is available for research on literacy (reading and writing) this resource is a reflection of the urgent need to obtain ecologically valid data .
Approach:	They propose to use Etherpad's keystroke logging data to analyze complex texts . they propose to relate behavioral data to indices of syntactic and lexical complexity .
Outcome:	The proposed method aims to improve alignment between keystroke-logging measures and cognitive processes and L2 writing performance measures.

Lego-MT: Learning Detachable Models for Massively Multilingual Machine Translation (2023.findings-acl)

Copied to clipboard

Fei Yuan, Yinquan Lu, Wenhao Zhu, Lingpeng Kong, Lei Li, Yu Qiao, Jingjing Xu

Multilinguality

Challenge:	Existing monolithic models for multilingual neural machine translation encounter parameter interference and inefficient inference for large models.
Approach:	They propose a detachable multi-way model that assigns each language to an individual branch . they use data from OPUS to build a translation benchmark covering 433 languages .
Outcome:	The proposed model outperforms existing models in OPUS and is faster than existing models.

DeepKE: A Deep Learning Based Knowledge Extraction Toolkit for Knowledge Base Population (2022.emnlp-demos)

Copied to clipboard

Ningyu Zhang, Xin Xu, Liankuan Tao, Haiyang Yu, Hongbin Ye, Shuofei Qiao, Xin Xie, Xiang Chen, Zhoubo Li, Lei Li

Information Extraction & Text Mining

Challenge:	Existing knowledge extraction tools are not complete due to emerging entities and relations in real-world applications.
Approach:	They propose an open-source knowledge extraction toolkit DeepKE that supports low-resource, document-level and multimodal scenarios in the knowledge base population.
Outcome:	The proposed toolkit supports low-resource, document-level and multimodal scenarios in the knowledge base population.

Attacks, Defenses and Evaluations for LLM Conversation Safety: A Survey (2024.naacl-long)

Copied to clipboard

Zhichen Dong, Zhanhui Zhou, Chao Yang, Jing Shao, Yu Qiao

Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct Preference Optimization (2024.findings-acl)

Challenge:	Large Language Models (LLMs) are now commonplace in conversation applications, but their misuse for generating harmful responses has raised serious societal concerns.
Approach:	They provide a comprehensive overview of recent studies covering attacks, defenses, and evaluations of Large Language Models (LLMs) .
Outcome:	The proposed review summarizes three aspects of LLM conversation safety: attacks, defenses, and evaluations.

Copied to clipboard

Zhanhui Zhou, Jie Liu, Jing Shao, Xiangyu Yue, Chao Yang, Wanli Ouyang, Yu Qiao

Natural-Language Policies to Executable Decisions: An Interpretable Large Language Model Framework (2026.acl-industry)

Challenge:	Recent approaches to language model alignment assume homogeneous human preferences, but actual human preferences vary widely and are hard to satisfy with a single language model.
Approach:	They propose an RL-free extension of Direct Preference Optimization (DPO) that folds language modeling directly into reward modeling and trains language models as collective reward models that combine all objectives with specific weights.
Outcome:	The proposed method matches or outperforms existing methods in safety alignment and long-form question answering.

Copied to clipboard

Ziqiang Zhang, Jing Ma, Zilong Wang, Jiayuan Chen, Yi Qiao, Yu He, Wei Zhang, Dai Cheng, Xiaoyu Shen

OS-Symphony: A Holistic Framework for Robust and Generalist Computer-Using Agents (2026.acl-long)

Challenge:	a production-grade pricing system for tourism is challenging due to unstructured nature of travel orders and ever-evolving pricing policies.
Approach:	They propose a production-grade pricing system with a strict decision boundary . they propose to combine structured extraction and bounded policy/path selection with interpretable condition trees .
Outcome:	The proposed system processed 3,960 orders in six months and reduced the order management team from 15-20 to 3 . the system reduced the per-order handling time from 10 minutes to 2 minutes.

Copied to clipboard

Bowen Yang, Kaiming Jin, Zhenyu Wu, Zhaoyang Liu, Qiushi Sun, Zehao Li, JingJing Xie, Zhoumianze Liu, Fangzhi Xu, Kanzhi Cheng, Yian Wang, Qingyun Li, Yu Qiao, Zun Wang, Zichen Ding

Robustness in NLP

Challenge:	Vision-Language Models (VLMs) lack visual-aware tutorial retrieval and historical visual context curation and pruning.
Approach:	They propose a framework that integrates an orchestrator and a Reflection-Memory Agent for robust automation.
Outcome:	Experimental results show that OS-Symphony delivers substantial performance gains across model scales.

OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis (2025.acl-long)

Copied to clipboard

Qiushi Sun, Kanzhi Cheng, Zichen Ding, Chuanyang Jin, Yian Wang, Fangzhi Xu, Zhenyu Wu, Chengyou Jia, Liheng Chen, Zhoumianze Liu, Ben Kao, Guohao Li, Junxian He, Yu Qiao, Zhiyong Wu

RolePlot: A Systematic Framework for Evaluating and Enhancing the Plot-Progression Capabilities of Role-Playing Agents (2025.acl-long)

Challenge:	Graphical User Interface (GUI) agents powered by Vision-Language Models (VLMs) have demonstrated human-like computer control capability.
Approach:	They propose a GUI data synthesis pipeline that reverse engineers GUI trajectory construction process by executing pre-defined tasks.
Outcome:	The proposed GUI data synthesis pipeline overcomes the bottlenecks of previous methods that rely on pre-defined tasks and limited data diversity.

Copied to clipboard

Pinyi Zhang, Siyu An, Lingfeng Qiao, Yifei Yu, Jingyang Chen, Jie Wang, Di Yin, Xing Sun, Kai Zhang

Dialogue Systems & Conversational Agents

Challenge:	Existing research has focused on role-playing agents’ ability to portray specified characters, but their ability to advance the plot requires substantial improvements to deliver more engaging interaction.
Approach:	They propose a role-playing framework to evaluate and enhance the plot-progression capabilities of role-players.
Outcome:	The proposed framework improves RPAs’ ability to time plot developments and yields a significant increase in conversation turns and sustained higher arousal levels.

Uni-MMMU: A Massive Multi-discipline Multimodal Unified Benchmark (2026.acl-long)

Copied to clipboard

Kai Zou, Ziqi Huang, Yuhao Dong, Shulin Tian, Dian Zheng, Hongbo Liu, Jingwen He, Bin Liu, Yu Qiao, Ziwei Liu

Multimodality

Challenge:	Existing evaluations treat visual understanding and generation in isolation or overlook tasks that inherently couple them.
Approach:	They propose a benchmark that examines the bidirectional synergy between generation and understanding across eight reasoning-centric domains.
Outcome:	The proposed model systematically unfolds the bidirectional synergy between generation and understanding across eight reasoning-centric domains.

Symbol-LLM: Towards Foundational Symbol-centric Interface For Large Language Models (2024.acl-long)

Copied to clipboard

Fangzhi Xu, Zhiyong Wu, Qiushi Sun, Siyu Ren, Fei Yuan, Shuai Yuan, Qika Lin, Yu Qiao, Jun Liu

LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shifts (2025.acl-long)

Challenge:	Large Language Models (LLMs) have limitations when it comes to comprehending and expressing world knowledge that extends beyond the boundaries of natural language.
Approach:	They propose a model that integrates symbolic data into LLM training without loss of generality ability.
Outcome:	The proposed model performs better on symbol- and NL-centric tasks.

Copied to clipboard

Qibing Ren, Hao Li, Dongrui Liu, Zhanxu Xie, Xiaoya Lu, Yu Qiao, Lei Sha, Junchi Yan, Lizhuang Ma, Jing Shao

Sequential-NIAH: A Needle-In-A-Haystack Benchmark for Extracting Sequential Needles from Long Contexts (2025.emnlp-main)

Challenge:	Current safety training focuses on teaching models to reject harmful queries, but recent research shows that adversarial attacks or jailbreak methods bypass these safety mechanisms.
Approach:	They propose to use a new attack method to craft multi-turn toxic prompts that gradually lead LLMs to reveal unsafe content.
Outcome:	The proposed method outperforms existing methods in diversity, effectiveness, and efficiency across aligned LLMs.

Copied to clipboard

Yifei Yu, Qian-Wen Zhang, Lingfeng Qiao, Di Yin, Fang Li, Jie Wang, Chen Zeng Xi, Suncong Zheng, Xiaolong Liang, Xing Sun

Information Extraction & Text Mining

Challenge:	Recent models have extended Corresponding Author. context lengths to millions of tokens while maintaining reasoning and comprehension capabilities.
Approach:	They propose a benchmark to evaluate the ability of large language models to extract sequential information items from long contexts.
Outcome:	The proposed model achieves maximum accuracy of 63.50% on six well-known LLMs.

SPADE: A Big Five-Mturk Dataset of Argumentative Speech Enriched with Socio-Demographics for Personality Detection (2022.lrec-1)

Copied to clipboard

Elma Kerz, Yu Qiao, Sourabh Zanwar, Daniel Wiechmann

Speech & Audio in NLP

Challenge:	Recent efforts to create such datasets from social media do not include continuous and contextualized language use.
Approach:	They propose to use argumentative speech to generate a dataset with continuous arguments labeled with the Big Five personality traits and enriched with socio-demographic data.
Outcome:	The proposed model leverages 436 (psycho)linguistic features extracted from transcribed speech and speaker-level metainformation with transformers to investigate which types of features contribute to the prediction of individual personality traits.

Evaluation Agent: Efficient and Promptable Evaluation Framework for Visual Generative Models (2025.acl-long)

Copied to clipboard

Fan Zhang, Shulin Tian, Ziqi Huang, Yu Qiao, Ziwei Liu

What to Fuse and How to Fuse: Exploring Emotion and Personality Fusion Strategies for Explainable Mental Disorder Detection (2023.findings-acl)

Challenge:	Existing evaluation methods rely on rigid pipelines that overlook user needs and provide numerical results without clear explanations.
Approach:	They propose an evaluation framework that employs human-like strategies for efficient, dynamic, multi-round evaluations using only a few samples per round.
Outcome:	The evaluation agent framework reduces evaluation time to 10% of traditional methods while delivering comparable results.

Copied to clipboard

Sourabh Zanwar, Xiaofei Li, Daniel Wiechmann, Yu Qiao, Elma Kerz

Interactive Evolution: A Neural-Symbolic Self-Training Framework For Large Language Models (2025.acl-long)

Challenge:	Mental health disorders (MHD) are one of the greatest challenges facing our healthcare systems and modern societies in general.
Approach:	They integrate and extend the research by conducting extensive experiments with three types of deep learning-based fusion strategies: feature-level fusion, model fusion and task fusion.
Outcome:	The proposed techniques show that they can be used to improve mental health detection from textual data.

Copied to clipboard

Fangzhi Xu, Qiushi Sun, Kanzhi Cheng, Jun Liu, Yu Qiao, Zhiyong Wu

SEER: Facilitating Structured Reasoning and Explanation via Reinforcement Learning (2024.acl-long)

Challenge:	Existing methods to fine-tune Large Language Models without human annotations are lacking in the field of natural language training.
Approach:	They propose an environment-guided neural-symbolic self-training framework to overcome two main challenges: the scarcity of symbolic data and the limited proficiency of LLMs in processing symbolic language.
Outcome:	The proposed framework overcomes two main challenges: the scarcity of symbolic data, and the limited proficiency of LLMs in processing symbolic language.

Copied to clipboard

Guoxin Chen, Kexin Tang, Chao Yang, Fuying Ye, Yu Qiao, Yiming Qian

MoPS: Modular Story Premise Synthesis for Open-Ended Automatic Story Generation (2024.acl-long)

Challenge:	Existing methods focus on single-step reasoning, ignoring logical dependencies between steps.
Approach:	They propose a method that maximizes a structure-based return to facilitate structured reasoning and explanation.
Outcome:	The proposed method outperforms state-of-the-art methods on EntailmentBank and STREET benchmarks.

Copied to clipboard

Yan Ma, Yu Qiao, Pengfei Liu

Text Generation

Challenge:	Existing sources of story premises are limited by a lack of diversity, uneven quality, and high costs that make them difficult to scale.
Approach:	They propose a method which breaks down story premises into modules like background and persona for automated design and generation.
Outcome:	The proposed framework excels in diversity, fascination, completeness, and originality compared to those induced from large language models and captured from public datasets.

GRASS: Gradient-based Adaptive Layer-wise Importance Sampling for Memory-efficient Large Language Model Fine-tuning (2026.findings-acl)

Copied to clipboard

Kaiyuan Tian, Linbo Qiao, Yu Tang, Gongqingjian Jiang, Baihui Liu, Yifu Gao, Xialin Su, Dongsheng Li

ChartAssistant: A Universal Chart Multimodal Language Model via Chart-to-Table Pre-training and Multitask Instruction Tuning (2024.findings-acl)

Challenge:	Low-rank adaptation methods for large language models limit expressiveness and performance . layer-wise fine-tuning methods overlook variations in layer importance across tasks and training stages, resulting in suboptimal performance on downstream tasks.
Approach:	They propose a gradient-based adaptive layer-wise importance sampling framework that updates only a subset of parameters to reduce memory usage.
Outcome:	The proposed framework outperforms state-of-the-art methods in accuracy and memory usage.

Copied to clipboard

Fanqing Meng, Wenqi Shao, Quanfeng Lu, Peng Gao, Kaipeng Zhang, Yu Qiao, Ping Luo

Multimodality

Challenge:	Charts are an effective tool for understanding data patterns, but their combination of graphical elements and textual components poses challenges for general-purpose multimodal models.
Approach:	They propose a chart-based vision-language model for universal chart comprehension and reasoning that leverages a dataset of chart-related tasks.
Outcome:	The proposed model outperforms the state-of-the-art charts with zero-shot setting on various chart tasks.

MedCite: Can Language Models Generate Verifiable Text for Medicine? (2025.findings-acl)

Copied to clipboard

Xiao Wang, Mengjue Tan, Qiao Jin, Guangzhi Xiong, Yu Hu, Aidong Zhang, Zhiyong Lu, Minjia Zhang

SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models (2024.findings-acl)

Challenge:	Existing LLM-based medical question answering systems lack citation generation and evaluation capabilities, raising concerns about their adoption in practice.
Approach:	They propose a framework that facilitates the design and evaluation of LLM citations for medical tasks and a retrieval-citation method that generates high-quality citation.
Outcome:	The proposed method achieves superior citation precision and recall improvements compared to strong baseline methods and correlates well with annotation results from professional experts.

Copied to clipboard

Lijun Li, Bowen Dong, Ruohui Wang, Xuhao Hu, Wangmeng Zuo, Dahua Lin, Yu Qiao, Jing Shao

Robustness in NLP

Challenge:	SALAD-Bench is a safety benchmark specifically designed for LLMs . it provides a robust source for evaluating both attack and defense algorithms .
Approach:	They propose a hierarchical safety benchmark specifically designed for LLMs . it uses a taxonomy of questions spanning three levels and a robust taxonomies based on a QA pair .
Outcome:	The proposed safety benchmark shows that LLMs are resilient against emerging threats and the effectiveness of contemporary defense methods.

OpenICL: An Open-Source Framework for In-context Learning (2023.acl-demo)

Copied to clipboard

Zhenyu Wu, Yaoxiang Wang, Jiacheng Ye, Zhiyong Wu, Jiangtao Feng, Jingjing Xu, Yu Qiao

Measuring the Impact of (Psycho-)Linguistic and Readability Features and Their Spill Over Effects on the Prediction of Eye Movement Patterns (2022.acl-long)

Challenge:	In-context Learning (ICL) is a new paradigm for large language model evaluation.
Approach:	They propose an open-source toolkit for ICL and LLM evaluation.
Outcome:	The proposed framework is highly flexible and flexible and can be easily combined with other tools to suit users' needs.

Copied to clipboard

Daniel Wiechmann, Yu Qiao, Elma Kerz, Justus Mattern