Papers by Wei Xiong

40 papers
Enhance Robustness of Language Models against Variation Attack through Graph Integration (2024.lrec-main)

Copied to clipboard

Challenge: Pre-trained language models (PLMs) are used in many NLP applications but their vulnerability to adversarial attacks can lead to false or misleading information being distributed.
Approach: They propose a method to incorporate a Chinese character variation graph into pre-trained language models to increase their robustness against character variation attacks in Chinese content.
Outcome: The proposed method outperforms existing language models in combating adversarial attacks in Chinese content.
LightSeq: A High Performance Inference Library for Transformers (2021.naacl-industry)

Copied to clipboard

Challenge: Existing inference frameworks for natural language processing are not the best choice for online service of sequence processing problems.
Approach: They propose a highly efficient inference library for Transformer models that includes GPU optimization techniques to streamline computation and reduce memory footprint.
Outcome: The proposed library achieves 14x speedup compared with TensorFlow and 1.4x speed up compared to a concurrent CUDA implementation.
SA-DETR:Span Aware Detection Transformer for Moment Retrieval (2025.coling-main)

Copied to clipboard

Challenge: Moment Retrieval aims to locate video segments related to text.
Approach: They propose a method that leverages the importance of instance related span anchors . they initialize span anchor using instance related fuse token and supervise them with GT labels .
Outcome: The proposed method achieves competitive results on QVHighlights, Charades-STA and TACoS.
LoRAMoE: Alleviating World Knowledge Forgetting in Large Language Models via MoE-Style Plugin (2024.acl-long)

Copied to clipboard

Challenge: Experimental results show that, as the instruction data increases, LoRAMoE can significantly improve the ability to process downstream tasks, while maintaining the world knowledge stored in the LLM.
Approach: They propose a framework that introduces several low-rank adapters and integrates them by using a router network to freeze the backbone model and force a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks.
Outcome: The proposed framework freezes the backbone model and forces a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks, to alleviate world knowledge forgetting.
From Lists to Emojis: How Format Bias Affects Model Alignment (2025.acl-long)

Copied to clipboard

Challenge: Format biases in reinforcement learning from human feedback are underexplored . despite its effectiveness, RLHF faces challenges, including policy and regulatory constraints .
Approach: They extend the study of preference biases beyond verbosity bias to a wider range of format biase . they show that with a small amount of biased data, they can inject significant bias into the reward model .
Outcome: The proposed approach can be easily exploited by large language models to achieve higher rankings on popular benchmarks like AlpacaEval and LMSYS Chatbot Arena.
AgentBank: Towards Generalized LLM Agents via Fine-Tuning on 50000+ Interaction Trajectories (2024.findings-emnlp)

Copied to clipboard

Challenge: Existing studies focus on specialized agents designed for particular tasks.
Approach: They propose to scale annotated interaction trajectories and fine-tune LLMs on AgentBank to get a series of agent models, Samoyed.
Outcome: The proposed model can scale to get generalized agent capabilities.
Temporal Scaling Law for Large Language Models (2025.emnlp-main)

Copied to clipboard

Challenge: Existing studies have found that the test loss of LLMs scales as power-laws with model size, computational budget, and dataset size.
Approach: They propose a concept of Temporal Scaling Law to study test loss of LLMs . they break down test loss into fine-grained token positions and develop a dynamic hyperbolic-law .
Outcome: The proposed model predicts the test loss of LLMs as the training steps scale up.
Federated LoRA Fine-Tuning with Pipelined Error-Mitigated Aggregation and Matrix-Wise Freezing (2026.findings-acl)

Copied to clipboard

Challenge: Existing methods for fine-tuning large language models often suffer from biased model aggregation and are hindered by significant communication and computation burden.
Approach: They propose a Federated low-rank adaptation system for large language models that leverages pipelined error-mitigated model aggregation and adaptive matrix-wise parameter freezing to mitigate aggregations.
Outcome: The proposed system improves time-to-target by 2.17-8.48 on real-world datasets.
Mitigating the Alignment Tax of RLHF (2024.emnlp-main)

Copied to clipboard

Challenge: Large Language Models (LLMs) acquire a wide range of abilities during pre-training, but aligning LLMs under Reinforcement Learning with Human Feedback (RLHF) can lead to forgetting pretrained abilities, which is also known as the alignment tax.
Approach: They propose to use a model averaging technique to find the most powerful alignment-forging Pareto front among RLHF algorithms.
Outcome: The proposed method achieves the strongest alignment-forging Pareto front among competing methods.
DialogGen: Multi-modal Interactive Dialogue System with Multi-turn Text-Image Generation (2025.findings-naacl)

Copied to clipboard

Challenge: Text-to-image (T2I) generation models have advanced in recent years, but effective interaction with these models is challenging for average users due to the need for specialized prompt engineering knowledge and the inability to perform multi-turn image generation.
Approach: They propose to use off-the-shelf MLLMs and T2I models to build a multi-modal interactive dialogue system (MIDS) that can generate correct output modalities and coherence of output images.
Outcome: The proposed pipeline can generate correct output modalities and coherent multi-modal outputs compared with other state-of-the-art models.
Multi-Granularity Semantic Revision for Large Language Model Distillation (2026.acl-long)

Copied to clipboard

Challenge: Existing methods for generating large language models rely on student-generated outputs, which introduce generation errors and misguide the distillation process.
Approach: They propose a multi-granularity semantic revision method for LLM distillation that corrects errors using teacher-generated tokens and re-generates the sequence to minimize errors.
Outcome: The proposed method reduces errors and misguides distillation on student models and improves consistency between teacher and student outputs.
Multi-Agent Collaboration via Cross-Team Orchestration (2025.findings-acl)

Copied to clipboard

Challenge: Large Language Models (LLMs) have significantly impacted various domains, especially through organized LLM-driven autonomous agents.
Approach: They propose a framework that enables orchestrated teams to jointly propose various task-oriented solutions and interact with their insights in a self-independence while cross-team collaboration environment for superior solutions generation.
Outcome: Experiments show that the framework can generate better software quality compared to state-of-the-art frameworks.
A Comprehensive Evaluation of Quantization Strategies for Large Language Models (2024.findings-acl)

Copied to clipboard

Challenge: Quantization studies have focused on instruction-tuned LLMs, leaving their performance on other benchmarks unclear.
Approach: They propose a framework to evaluate quantized large language models using four dimensions . they propose to reduce the bits needed for model weights or activations with minimal performance loss .
Outcome: The proposed framework can retain comparable performance to non-quantized LLMs on most benchmarks.
Anti-Length Shift: Dynamic Outlier Truncation for Training Efficient Reasoning Models (2026.acl-long)

Copied to clipboard

Challenge: Existing efficient reasoning methods rely on explicit length penalties for excessive verbosity on simple queries.
Approach: They propose a training-time intervention that selectively suppresses redundant tokens . they find length shift occurs when models generate unnecessary reasoning on trivial inputs - a phenomenon that is often unexplored .
Outcome: The proposed method reduces inference token usage by 78% while increasing accuracy compared to the initial policy and surpasses state-of-the-art efficient reasoning methods.
AnchorSeg: Language Grounded Query Banks for Reasoning Segmentation (2026.acl-long)

Copied to clipboard

Challenge: Existing models rely on a single segmentation token whose hidden state implicitly encodes both semantic reasoning and spatial localization . Existing methods rely only on SEG>, which encodes semantic reasoning, limiting the model's ability to explicitly disentangle what to segment from where to segment.
Approach: They propose a method which reformulates reasoning segmentation as a structured conditional generation process over image tokens conditioned on language grounded query banks.
Outcome: The proposed model bridges token-level predictions and pixel-level supervision by decoupling spatial grounding from semantic reasoning through structured language grounded query banks.
ChatSOP: An SOP-Guided MCTS Planning Framework for Controllable LLM Dialogue Agents (2025.acl-long)

Copied to clipboard

Challenge: Existing models that use Large Language Models (LLMs) show superior performance in various tasks, but lack of controllability leads to unfocused conversations or task failure.
Approach: They propose a standard operating procedure (SOP) framework to regulate dialogue flow by integrating Chain of Thought reasoning and supervised fine-tuning for SOP prediction.
Outcome: The proposed method achieves a 27.95% improvement in action accuracy compared to baseline models based on GPT-3.5 and also shows notable gains for open-source models.
LMFlow: An Extensible Toolkit for Finetuning and Inference of Large Foundation Models (2024.naacl-demo)

Copied to clipboard

Challenge: Foundation models have demonstrated a great ability to achieve general human-level intelligence far beyond traditional approaches.
Approach: They propose a toolkit to simplify the finetuning of general foundation models.
Outcome: The proposed toolkit simplifies the domain- and task-aware finetuning of general foundation models with limited computing resources.
Revisiting Entropy in Reinforcement Learning for Large Reasoning Models (2026.findings-acl)

Copied to clipboard

Challenge: Reinforcement learning with verifiable rewards (RLVR) has emerged as a paradigm for enhancing the reasoning capabilities of large language models.
Approach: They propose a positive-advantage reweighting approach that regulates model entropy by adjusting the loss weights assigned to tokens with positive advantages during RLVR training.
Outcome: The proposed approach regulates model entropy by adjusting loss weights assigned to tokens with positive advantages during RLVR training while maintaining competitive performance.
WorkTeam: Constructing Workflows from Natural Language with Multi-Agents (2025.naacl-industry)

Copied to clipboard

Challenge: Existing workflow construction methods require specialized knowledge and task-switching skills.
Approach: They propose a multi-agent workflow framework that incorporates a supervisor, orchestrator, and filler agent.
Outcome: The proposed framework significantly increases the success rate of workflow construction . the proposed framework is based on a dataset of 3,695 real-world business samples .
Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards (2024.acl-long)

Copied to clipboard

Challenge: Reinforcement Learning from Human Feedback (RLHF) relies on scalar rewards to capture user preferences.
Approach: They propose a framework that integrates multi-objective reward modeling to represent diverse preference profiles.
Outcome: The proposed method improves performance across reward objectives and targets.
Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Experts (2024.findings-emnlp)

Copied to clipboard

Challenge: Reinforcement learning from human feedback (RLHF) is the primary method for aligning large language models with human preferences.
Approach: They propose to train an Absolute-Rating Multi-Objective Reward Model with multi-dimensional absolute-rating data.
Outcome: The proposed model outperforms the LLM-as-a-judge method on RewardBench . it achieves state-of-the-art performance on the benchmark .
Not All Voices Are Rewarded Equally: Probing and Repairing Reward Models across Human Diversity (2025.findings-emnlp)

Copied to clipboard

Challenge: Using real-world datasets, we conduct the most comprehensive study to date, auditing various state-of-the-art reward models across nine sensitive attributes, including age, gender, ethnicity, etc.
Approach: They propose a method to mitigate group disparities in reward modeling by using real-world data.
Outcome: The proposed method is based on a population-based dataset with nine demographic attributes, including gender, ethnicity, age, gender, and ethnicity.
TokenSelect: Efficient Long-Context Inference and Length Extrapolation for LLMs via Dynamic Token-Level KV Cache Selection (2025.emnlp-main)

Copied to clipboard

Challenge: Rapid advances in Large Language Models have spurred demand for processing extended context sequences . however, performance degradation due to sequence lengths out-of-distribution and excessively long inference times are limiting LLMs in long-context scenarios.
Approach: They propose a training-free method for efficient and accurate long-context inference . they selectively involves a few critical KV cache tokens in attention calculation .
Outcome: The proposed method speeds up attention computation and accelerates inference time while reducing selection overhead.
Adversarial Preference Learning for Robust LLM Alignment (2025.findings-acl)

Copied to clipboard

Challenge: Modern language models rely on Reinforcement Learning from Human Feedback (RLHF) to encourage safe behaviors, but they remain vulnerable to adversarial attacks due to three key limitations: (1) the inefficiency and high cost of human annotation; (2) the vast diversity of potential adversarials; and (3) the risk of feedback bias and reward hacking.
Approach: They propose an iterative adversarial training method that incorporates three key innovations to address these challenges.
Outcome: Experiments on Mistral-7B-Instruct-v0.3 show that the proposed method significantly enhances robustness and reduces harmful outputs from 5.88% to 0.43%.
Long Document Ranking with Query-Directed Sparse Transformer (2020.findings-emnlp)

Copied to clipboard

Challenge: Existing approaches to document ranking require long documents to be broken to fit in pretrained models.
Approach: They propose a Query-Directed Sparse attention model that induces IR-axiomatic structures in transformer self-attention.
Outcome: The proposed model enforces the principle properties desired in ranking while also enjoying efficiency from sparsity.
Fast Quiet-STaR: Thinking Without Thought Tokens (2025.findings-emnlp)

Copied to clipboard

Challenge: Large Language Models have achieved impressive performance across a range of tasks, but further gains require more than scaling up model sizes or training data.
Approach: They propose a method that gradually reduces the number of thought tokens . this method allows models to internalize more abstract reasoning processes .
Outcome: The proposed framework preserves the benefits of token-level reasoning while reducing computational cost.
Watch Every Step! LLM Agent Learning via Iterative Step-level Process Refinement (2024.emnlp-main)

Copied to clipboard

Challenge: Recent approaches to enhance agent performance focus on outcome rewards, which may lead to errors or suboptimal actions due to the absence of process supervision signals.
Approach: They propose a step-level framework that provides detailed step-by-step guidance to enhance agent training by using Monte Carlo methods.
Outcome: The proposed framework outperforms strong baselines on three tasks and shows that it is effective in augmenting efficiency and its applicability to diverse models.
MMNMT: Modularizing Multilingual Neural Machine Translation with Flexibly Assembled MoE and Dense Blocks (2023.emnlp-main)

Copied to clipboard

Challenge: Mixture-of-Experts (MoE) based sparse architectures are prone to overfitting on low-resource language translation.
Approach: They propose a modularized MNMT framework that flexibly assembles dense and MoE-based sparse modules to achieve the best of both worlds.
Outcome: The proposed framework outperforms existing models on low-resource language translation and zero-shot translation on benchmark datasets.
Breaking the Stage Barrier: A Novel Single-Stage Approach to Long Context Extension for Large Language Models (2025.coling-main)

Copied to clipboard

Challenge: Recent studies show that Large language models struggle with handling long token sequences due to limited training context size.
Approach: They propose a single-stage continual pretraining method to equip LLMs with long context modeling capabilities.
Outcome: The proposed method outperforms existing methods on 4 language modeling benchmarks.
SoftDedup: an Efficient Data Reweighting Method for Speeding Up Language Model Pre-training (2024.acl-long)

Copied to clipboard

Challenge: Current methods focus on detecting and removing duplicates, which risks the loss of valuable information and neglects the varying degrees of duplication.
Approach: They propose a method that maintains dataset integrity while selectively reducing the sampling weight of data with high commonness.
Outcome: The proposed method significantly improves training efficiency on deduplicated datasets and improves downstream accuracy by 1.77%.
DSMoE: Matrix-Partitioned Experts with Dynamic Routing for Computation-Efficient Dense LLMs (2025.emnlp-main)

Copied to clipboard

Challenge: Existing sparsification methods like pruning can lose model knowledge through parameter removal.
Approach: They propose a novel approach that achieves sparsification by partitioning pre-trained FFN layers into computational blocks.
Outcome: The proposed approach achieves superior performance across language modeling and downstream tasks under equivalent computational constraints.
PersonaAgent: Bridging Memory and Action for Personalized LLM Agents (2026.findings-acl)

Copied to clipboard

Challenge: Existing Large Language Model (LLM) enabled agents lack flexibility to respond to users’ varying needs and preferences.
Approach: They propose a test-time user-preference alignment strategy that optimizes the persona prompt, ensuring real-time preference alignment through textual loss feedback between simulated and ground-truth responses.
Outcome: The proposed framework outperforms baseline methods in real-time and in real applications.
End-to-End Open-Domain Question Answering with BERTserini (N19-4)

Copied to clipboard

Challenge: a new open-domain question answering system integrates best practices from IR with a BERT-based reader to identify answers from a large corpus of Wikipedia articles.
Approach: They propose an end-to-end question answering system that integrates BERT with an IR reader.
Outcome: The proposed system improves on a standard benchmark test collection.
QAEncoder: Towards Aligned Representation Learning in Question Answering Systems (2025.acl-long)

Copied to clipboard

Challenge: Modern QA systems entail retrieval-augmented generation (RAG) for accurate and trustworthy responses, but the inherent gap between user queries and relevant documents hinders precise matching.
Approach: They propose a retrieval-augmented generation (RAG)-based approach to bridge this gap by attaching document fingerprints to the embedding to estimate the expectation of potential queries.
Outcome: Experiments across diverse datasets, languages, and embedding models confirm the proposed solution is simple-yet-effective with zero additional index storage, retrieval latency, training costs, or catastrophic forgetting and hallucination issues.
T2R-BENCH: A Benchmark for Real World Table-to-Report Task (2025.emnlp-main)

Copied to clipboard

Challenge: Existing table benchmarks lack the capacity to adequately assess the practical application of table reasoning in industrial applications.
Approach: They propose a bilingual table-to-report task and a table-based benchmark to assess the quality of table reasoning.
Outcome: The proposed task is based on a bilingual benchmark with 457 industrial tables and evaluation criteria to measure the quality of report generation.
OS Agents: A Survey on MLLM-based Agents for Computer, Phone and Browser Use (2025.acl-long)

Copied to clipboard

Challenge: a new generation of (M)LLMs is enabling the creation of superintelligent AI assistants . OS Agents can complete tasks autonomously and have the potential to significantly enhance the lives of billions of users worldwide.
Approach: They propose to build OS Agents that operate within operating systems' GUIs and GUIs . they examine evaluation metrics and benchmarks to identify promising directions .
Outcome: The proposed agents are based on operating systems (OS) and operating systems frameworks.
Detecting Customer Complaint Escalation with Recurrent Neural Networks and Manually-Engineered Features (N19-2)

Copied to clipboard

Challenge: e-commerce companies often have the option of escalating complaints by filing grievances with a government authority . this is detrimental to an ecommerce company, but this problem is challenging to solve by integrating recurrent neural networks with manually-engineered features.
Approach: They propose a model that integrates recurrent neural networks with manually-engineered features to identify cases where the customer expresses such an intent.
Outcome: The proposed model outperforms baseline models and provides better recall and triage for specialized agents.
polyBART: A Chemical Linguist for Polymer Property Prediction and Generative Design (2025.findings-emnlp)

Copied to clipboard

Challenge: a new language model for polymer design is needed to accurately predict polymers' properties.
Approach: They propose a language model-driven polymer discovery capability that enables rapid and accurate exploration of the polymer design space.
Outcome: The proposed polyBART is the first language model capable of bidirectional translation between polymer structures and properties.
TagRouter: Learning Route to LLMs through Tags for Open-Domain Text Generation Tasks (2025.findings-acl)

Copied to clipboard

Challenge: Existing models with limited performance and limited training can be difficult to use in large-scale applications.
Approach: They propose a training-free model routing method that optimizes synergy among multiple LLMs for open-domain text generation tasks.
Outcome: The proposed method outperforms 13 baseline models and reduces costs by 17.20%.
OntoGuard: Enforcing Action Admissibility for LLM Agents in Complex Interactive Environments (2026.findings-acl)

Copied to clipboard

Challenge: Existing approaches to large language models (LLMs) are limited by their ability to enforce environmental and behavioral admissibility.
Approach: They propose an ontological framework to guard LLM agents by enforcing environmental and behavioral admissibility.
Outcome: Experiments on ScienceWorld and VirtualHome show that OntoGuard can enforce environmental and behavioral admissibility while preventing invalid actions.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations