Papers by Tianyang Xu
Towards Enhanced Immersion and Agency for LLM-based Interactive Drama (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies have focused on the role of immersion and agency in interactive drama. |
| Approach: | They propose a playwriting-guided generation method that helps LLMs craft dramatic stories with substantially improved structures and narrative quality. |
| Outcome: | The proposed method improves storytelling quality and immersion and agency, while allowing agents to refine their reactions to align with the player’s intentions. |
Variational Autoencoder with Disentanglement Priors for Low-Resource Task-Specific Natural Language Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for task-specific natural language generation do not contain any labeled examples. |
| Approach: | They propose a variational autoencoder with disentanglement priors for task-specific natural language generation with none or a handful of task-related labeled examples. |
| Outcome: | The proposed model outperforms baseline models in terms of data augmentation and text style transfer in the few-shot setting. |
Sensitivity-LoRA : Low-Load Sensitivity-Based Fine-Tuning for Large Language Models (2025.findings-emnlp)
Copied to clipboard
Hao Zhang, Bo Huang, Zhenjia Li, Xi Xiao, Hui Yi Leong, Zumeng Zhang, Xinwei Long, Tianyang Wang, Hao Xu
| Challenge: | Low-Rank Adaptation (LoRA) is a promising approach to adapting LLMs to specialized tasks . existing rank allocation techniques remain computationally inefficient and unstable . |
| Approach: | They propose a low-rank adapted model that approximates model weight updates using low-ranked decomposition. |
| Outcome: | The proposed method is limited by its uniform rank allocation to each incremental matrix . it leverages the second-order derivatives of the loss function to capture weight sensitivity . |
Understanding the Language Model to Solve the Symbolic Multi-Step Reasoning Problem from the Perspective of Buffer Mechanism (2025.findings-emnlp)
Copied to clipboard
Zhiwei Wang, Yunji Wang, Zhongwang Zhang, Zhangchen Zhou, Hui Jin, Tianyang Hu, Jiacheng Sun, Zhenguo Li, Yaoyu Zhang, Zhi-Qin John Xu
| Challenge: | Large language models struggle with complex reasoning tasks, such as mathematical problem-solving. |
| Approach: | They constructed a symbolic multi-step reasoning task to investigate the information propagation mechanisms in Transformer models when solving the task through direct answering and Chain-of-Thought (CoT) reasoning. |
| Outcome: | The proposed algorithm improves on 7 multi-step reasoning datasets, while introducing only 132 trainable parameters. |
The Harmonic Structure of Information Contours (2025.acl-long)
Copied to clipboard
Eleftheria Tsipidi, Samuel Kiegeland, Franz Nowak, Tianyang Xu, Ethan Wilcox, Alex Warstadt, Ryan Cotterell, Mario Giulianelli
| Challenge: | Language typically does not maintain a uniform information rate, but it fluctuates around a global average . a new study suggests periodicity may be a factor in information rate oscillations . |
| Approach: | They propose a hypothesis that language does not maintain a uniform information rate . they apply harmonic regression and introduce a new extension to detect periodicity . |
| Outcome: | The proposed method reveals that language oscillates at periodic intervals across frequencies . it also offers a framework for uncovering structural pressures at various levels of linguistic granularity. |
HyperAdaLoRA: Accelerating LoRA Rank Allocation During Training via Hypernetworks without Sacrificing Performance (2026.findings-acl)
Copied to clipboard
Hao Zhang, Zhenjia Li, Yifan Gao, Xi Xiao, Heng Zhang, Shuyang Zhang, null Xiaoxincc, Bo Huang, Yuhang Wu, Tianyang Wang, Hao Xu
| Challenge: | Low-Rank Adaptation (LoRA) assumes a uniform rank r for each incremental matrix, not accounting for the varying significance of weight matrices across modules and layers. |
| Approach: | They propose a framework that allows for faster convergence of low-rank adaptive models . they use a hypernetwork to prune the outputs of the hypernetworks to generate parameters . |
| Outcome: | The proposed framework accelerates convergence of AdaLoRA by leveraging a hypernetwork. |
SHIELD: Evaluation and Defense Strategies for Copyright Compliance in LLM Text Generation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have transformed machine learning but have raised significant legal concerns due to their potential to produce text that infringes on copyrights. |
| Approach: | They propose a lightweight, real-time defense mechanism to prevent the generation of copyrighted text by evaluating methods and testing attack strategies. |
| Outcome: | The proposed defense significantly reduces the volume of copyrighted text generated by LLMs by effectively refusing malicious requests. |
SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationales (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to elicit confidence from large language models are limited to binary or inaccurate group-level confidence estimates. |
| Approach: | They propose a training framework that teaches LLMs to express more fine-grained confidence estimates. |
| Outcome: | The proposed training framework reduces the confidence calibration error and maintains the performance of the model. |
Can Language Models Learn Typologically Implausible Languages? (2026.tacl-1)
Copied to clipboard
| Challenge: | Language models provide a naturalistic framework for studying artificial language learning . authors: typological universals and tendencies are thought to be caused by a learning bias . |
| Approach: | They propose to train LMs on highly naturalistic counterfactual versions of English and Japanese . they show that LM learn subtly implausible languages more slowly . |
| Outcome: | The proposed language models learn subtly implausible languages more slowly compared to human models . the findings suggest that LMs exhibit typologically aligned learning preferences . |
Not All Directions Matter: Towards Structured and Task-Aware Low-Rank Model Adaptation (2026.acl-long)
Copied to clipboard
Xi Xiao, Chenrui Ma, Yunbei Zhang, Chen Liu, Zhuxuanzi Wang, Yanshu Li, Lin Zhao, Guosheng Hu, Tianyang Wang, Hao Xu
| Challenge: | Low-Rank Adaptation (LoRA) is a key parameter-efficient fine-tuning method . however, its effectiveness is hampered by semantic drift and structural incoherence . |
| Approach: | They propose a low-rank Adaptation framework that tackles semantic drift and structural incoherence by pruning task-irrelevant directions. |
| Outcome: | Experiments on large language models, vision models, and vision models show that the proposed framework outperforms LoRA and advanced dynamic rank allocation and sparsity-based methods. |
Probing for Reading Times (2026.acl-long)
Copied to clipboard
Eleftheria Tsipidi, Samuel Kiegeland, Francesco Ignazio Re, Tianyang Xu, Mario Giulianelli, Karolina Stanczak, Ryan Cotterell
| Challenge: | a large body of work on probing has demonstrated that language model representations encode a wealth of linguistic information, but it remains unclear whether they also capture cognitive signals about human processing. |
| Approach: | They use regularized linear regression to compare language model representations against scalar predictors. |
| Outcome: | The representations from early layers outperform surprisal in predicting early-pass measures such as first fixation and gaze duration. |
DISK: Domain-constrained Instance Sketch for Math Word Problem Generation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods for generating MWP text from equations are inflexible and require pre-defined templates. |
| Approach: | They propose a neural model which generates MWPs from equations by constructing a Quantity Cell Graph from the retrieved MWp instance and reasoning over it. |
| Outcome: | The proposed model performs impressively on educational MWP set and on human evaluation metrics. |
Verification-Aware Planning for Multi-Agent Systems (2026.eacl-long)
Copied to clipboard
| Challenge: | Large language model (LLM) agents are increasingly deployed to tackle complex tasks . multi-agent collaboration introduces new challenges in planning, coordination, and verification . |
| Approach: | They propose a framework for multi-agent collaboration with verification-aware planning . the framework decomposes tasks, models subtask dependencies, and encodes planner-defined passing criteria as subtask verification functions (VFs) |
| Outcome: | The proposed framework outperforms baselines on diverse datasets while improving system robustness and interpretability. |
Getting More Juice Out of Your Data: Hard Pair Refinement Enhances Visual-Language Models Without Extra Data (2025.naacl-long)
Copied to clipboard
Haonan Wang, Minbin Huang, Runhui Huang, Lanqing Hong, Hang Xu, Tianyang Hu, Xiaodan Liang, Zhenguo Li, Hong Cheng, Kenji Kawaguchi
| Challenge: | Contrastive Language-Image Pre-training (CLIP) is a standard for cross-modal image-text representation learning. |
| Approach: | They propose a framework that enhances pre-trained CLIP models by exploiting challenging text-image pairs within existing datasets. |
| Outcome: | The proposed framework improves CLIP models by exploiting text-image pairs in training. |
Mixture-of-Domain-Adapters: Decoupling and Injecting Domain Knowledge to Pre-trained Language Models’ Memories (2023.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models demonstrate excellent abilities to understand texts in the generic domain while struggling in a specific domain. |
| Approach: | They propose to decouple the feed-forward networks of the Transformer architecture into two parts to maintain old-domain knowledge and a mixture-of-adapters gate to inject domain-specific knowledge in parallel. |
| Outcome: | The proposed method achieves superior performance on in-domain, out-of-domain and knowledge-intensive tasks. |
Open-Theatre: An Open-Source Toolkit for LLM-based Interactive Drama (2025.emnlp-demos)
Copied to clipboard
| Challenge: | Existing tools for creating, modifying, and experimenting with interactive dramas are limited. |
| Approach: | They propose an open-source toolkit for creating configurable LLM-based interactive drama. |
| Outcome: | The proposed toolkit enhances narrative coherence and realistic behavior in interactions with agents. |
Cross-Modal Taxonomic Generalization in (Vision-) Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies have shown that language models learn from surface form to learn from more grounded evidence. |
| Approach: | They propose to use a vision-language model to learn hypernyms from images . they find that the model can recover this knowledge and generalize even when there is no hypernomia in the image. |
| Outcome: | The proposed model can recover this knowledge and generalize even when the model receives no evidence of hypernyms during training. |