Papers by Zhiheng Huang

AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments (2026.acl-long)

Challenge:	Existing processes that reward for each step are one-directional and lack a mechanism to model the distance to the final target.
Approach:	They propose a process supervision model that evaluates the correctness of previous steps and the probability of future success.
Outcome:	The proposed model outperforms existing supervision models like ORM and PRM on reasoning tasks and improves solution re-design.

Copied to clipboard

Zhiheng Xi, Dingwen Yang, Jiaqi Liu, Jixuan Huang, Honglin Guo, Baodai Huang, Tinggang Chen, Qi Zhang, Zhonghang Lu, Chenyu Liu, Jiajun Sun, Jiazheng Zhang, Dingwei Zhu, Xin Guo, Junzhe Wang, Zhihao Zhang, Yuming Yang, Junjie Ye, Minghe Gao, Dongrui Liu, Jiaming Ji, Guohao Li, Tao Gui, Qi Zhang, Xuanjing Huang

LoRAMoE: Alleviating World Knowledge Forgetting in Large Language Models via MoE-Style Plugin (2024.acl-long)

Challenge:	Existing benchmarks evaluate agents in simplified, idealized settings, relying on pre-packaged tool interfaces, overlooking critical steps, and assume inputs are clean and fully specified.
Approach:	They propose a framework that evaluates language agents in simplified, idealized settings . they show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 .
Outcome:	Experiments on 15 proprietary and open-source models show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 .

Copied to clipboard

Shihan Dou, Enyu Zhou, Yan Liu, Songyang Gao, Wei Shen, Limao Xiong, Yuhao Zhou, Xiao Wang, Zhiheng Xi, Xiaoran Fan, Shiliang Pu, Jiang Zhu, Rui Zheng, Tao Gui, Qi Zhang, Xuanjing Huang

Improving Discriminative Capability of Reward Models in RLHF Using Contrastive Learning (2024.emnlp-main)

Challenge:	Experimental results show that, as the instruction data increases, LoRAMoE can significantly improve the ability to process downstream tasks, while maintaining the world knowledge stored in the LLM.
Approach:	They propose a framework that introduces several low-rank adapters and integrates them by using a router network to freeze the backbone model and force a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks.
Outcome:	The proposed framework freezes the backbone model and forces a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks, to alleviate world knowledge forgetting.

Copied to clipboard

Lu Chen, Rui Zheng, Binghai Wang, Senjie Jin, Caishuang Huang, Junjie Ye, Zhihao Zhang, Yuhao Zhou, Zhiheng Xi, Tao Gui, Qi Zhang, Xuanjing Huang

Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models (2026.findings-acl)

Challenge:	Current methods rely on ranking losses to teach reward model to assess preferences, but they are susceptible to noise and ambiguous data, often failing to deeply understand human intentions.
Approach:	They propose a method that incorporates contrastive learning into the reward modeling process to enhance generalization and stabilize the reinforcement learning training process.
Outcome:	The proposed method enhances generalization of the reward model, stabilizes the reinforcement learning training process, and improves the final alignment with human preferences.

Copied to clipboard

Hengyuan Zhang, Zhihao Zhang, Ercong Nie, Mingyang Wang, Zunhai Su, Yiwei Wang, Qianli Wang, Shuzhou Yuan, Xufeng Duan, Qibo Xue, Zeping Yu, Chenming Shang, Xiao Liang, Jing Xiong, Hui Shen, Chaofan Tao, Zhengwu Liu, Senjie Jin, Zhiheng Xi, Dongdong Zhang, Sophia Ananiadou, Tao Gui, Ruobing Xie, Hayden Kwok-Hay So, Hinrich Schuetze, Xuanjing Huang, Qi Zhang, Ngai Wong

Explainability & Interpretability in NLP

Challenge:	Existing literature on mechanistic interpretation (MI) treats it as an observational science, leaving practical applications underexplored.
Approach:	They propose a survey structured around the pipeline to identify and improve MI models.
Outcome:	The proposed framework enables tangible improvements in Alignment, Capability, and Efficiency.

Hybrid Hierarchical Retrieval for Open-Domain Question Answering (2023.findings-acl)

Copied to clipboard

Manoj Ghuhan Arivazhagan, Lan Liu, Peng Qi, Xinchi Chen, William Yang Wang, Zhiheng Huang

Information Retrieval

Challenge:	Recent work shows that dense hierarchical retrieval (DHR) can outperform dense passage retrieval.
Approach:	They propose a framework that applies sparse, dense and a combination of them to document and passage retrieval.
Outcome:	The proposed framework can outperform dense hierarchical retrieval (DHR) and sparse retrievers (BM25) on open-domain question answering (ODQA) datasets with an average improvement of 4.69% on recall@100 over DHR.

CityEQA: A Hierarchical LLM Agent on Embodied Question Answering Benchmark in City Space (2025.emnlp-main)

Copied to clipboard

Yong Zhao, Kai Xu, Zhengqiu Zhu, Yue Hu, Zhiheng Zheng, Yingfeng Chen, Yatai Ji, Chen Gao, Yong Li, Jincai Huang

Natural Language Interfaces

Challenge:	Embodied Question Answering (EQA) tasks are primarily focused on indoor environments, leaving the complexities of urban settings unexplored.
Approach:	They propose a task where an embodied agent answers open-vocabulary questions in dynamic city spaces.
Outcome:	The proposed agent achieves 60.7% of human-level answering accuracy compared to baselines . the proposed agent outperforms existing agents in open-ended city spaces .

AgentV-RL: Scaling Reward Modeling with Agentic Verifier (2026.findings-acl)

Copied to clipboard

Jiazheng Zhang, Ziche Fu, Zhiheng Xi, Wenqing Jing, Mingxu Chai, Wei He, Guoqiang Zhang, Chenghao Fan, Chenxin An, Wenxiang Chen, Zhicheng Liu, Haojie Pan, Dingwei Zhu, Tao Gui, Qi Zhang, Xuanjing Huang

PFDial: A Structured Dialogue Instruction Fine-tuning Method Based on UML Flowcharts (2025.findings-acl)

Challenge:	Existing approaches to improve LLM reasoning are limited in complex domains and lack external grounding makes verifiers unreliable on computation-intensive tasks.
Approach:	They propose a framework that transforms reward modeling into a multi-turn, tool-augmented deliberative process.
Outcome:	The proposed framework surpasses state-of-the-art ORMs by 25.2% under parallel and sequential TTS.

Copied to clipboard

Ming Zhang, Yuhui Wang, Yujiong Shen, Tingyi Yang, Changhao Jiang, Yilong Wu, Shihan Dou, Qinhao Chen, Zhiheng Xi, Zhihao Zhang, Yi Dong, Zhen Wang, Zhihui Fei, Mingyang Wan, Tao Liang, Guojun Ma, Qi Zhang, Tao Gui, Xuanjing Huang

Dialogue Systems & Conversational Agents

Challenge:	Large Language Models (LLMs) have shown remarkable progress in dialogue and reasoning, but they struggle to solve strictly constrained dialogue tasks.
Approach:	They construct a dataset that contains 12,705 high-quality Chinese dialogue instructions from 440 flowcharts containing 5,055 process nodes.
Outcome:	The proposed model outperforms GPT-4o models on backward transitions and outperformed GPT-42 models on the same dataset.

Counteracting the Matthew Effect in Self-Improvement of LVLMs through Head-Tail Re-balancing (2026.acl-long)

Copied to clipboard

Xin Guo, Zhiheng Xi, Yiwen Ding, Yitao Zhai, Xiaowei Shi, Xunliang Cai, Tao Gui, Qi Zhang, Xuanjing Huang

Mitigating Tail Narrowing in LLM Self-Improvement via Socratic-Guided Sampling (2025.naacl-long)

Challenge:	Large vision language models have impressive reasoning capabilities across complex multimodal tasks.
Approach:	They propose to use distribution-reshaping and trajectory-rebalancing to improve visual reasoning capabilities.
Outcome:	Experiments on Qwen2-VL-7B-Instruct and InternVL2.5-4B models show that their methods outperform baselines by 3.86 points.

Copied to clipboard

Yiwen Ding, Zhiheng Xi, Wei He, Lizhuoyuan Lizhuoyuan, Yitao Zhai, Shi Xiaowei, Xunliang Cai, Tao Gui, Qi Zhang, Xuanjing Huang

Self-Polish: Enhance Reasoning in Large Language Models via Problem Refinement (2023.findings-emnlp)

Challenge:	Large language models (LLMs) generate solutions themselves and iteratively train on filtered, high-quality rationales, but performance reaches a ceiling after a few iterations.
Approach:	They propose a strategy to improve the efficiency of sampling heavy-tailed data by using Socratic-style guidance signals to help LLMs reasoning with complex queries.
Outcome:	The proposed approach is effective on difficult queries and on held-out tasks, while requiring human supervision.

Copied to clipboard

Zhiheng Xi, Senjie Jin, Yuhao Zhou, Rui Zheng, Songyang Gao, Jia Liu, Tao Gui, Qi Zhang, Xuanjing Huang

AgentGym: Evaluating and Training Large Language Model-based Agents across Diverse Environments (2025.acl-long)

Challenge:	Existing prompting methods have been used to enhance multistep reasoning capabilities of large language models, but they have overlooked the potential of formulating higher-quality problems.
Approach:	They propose a method that starts from the problem side and refines problems to be more comprehensible and solvable for models.
Outcome:	The proposed method achieves notable and consistent effectiveness on five reasoning benchmarks across different models.

Copied to clipboard

Zhiheng Xi, Yiwen Ding, Wenxiang Chen, Boyang Hong, Honglin Guo, Junzhe Wang, Xin Guo, Dingwen Yang, Chenyang Liao, Wei He, Songyang Gao, Lu Chen, Rui Zheng, Yicheng Zou, Tao Gui, Qi Zhang, Xipeng Qiu, Xuanjing Huang, Zuxuan Wu, Yu-Gang Jiang

Multi-Programming Language Sandbox for LLMs (2025.acl-demo)

Challenge:	Large language models (LLMs) are promising foundations to build generally-capable agents . however, the community lacks a unified interactive framework that covers diverse environments for comprehensive evaluation of agents.
Approach:	They propose a framework that features 7 real-world scenarios, 14 environments, and 89 tasks for unified, real-time, and concurrent agent interaction.
Outcome:	The proposed framework features 7 real-world scenarios, 14 environments, and 89 tasks for unified, real-time, and concurrent agent interaction.

Copied to clipboard

Shihan Dou, Jiazheng Zhang, Jianxiang Zang, Yunbo Tao, Weikang Zhou, Haoxiang Jia, Shichun Liu, Yuming Yang, Shenxi Wu, Zhiheng Xi, Muling Wu, Rui Zheng, Changze Lv, Limao Xiong, Shaoqing Zhang, Lin Zhang, Wenyu Zhan, Rongxiang Weng, Jingang Wang, Xunliang Cai, Yueming Wu, Ming Wen, Yixin Cao, Tao Gui, Xipeng Qiu, Qi Zhang, Xuanjing Huang

Contrastive Document Representation Learning with Graph Attention Networks (2021.findings-emnlp)

Challenge:	MPLSandbox is an out-of-the-box multi-programming language sandbox designed to provide unified and comprehensive feedback from compiler and analysis tools for Large Language Models (LLMs).
Approach:	They propose a multi-programming language sandbox that provides unified feedback from compilers and analysis tools for Large Language Models.
Outcome:	The proposed multi-language sandbox can provide comprehensive feedback from compilers and analysis tools for large language models (LLMs).

Copied to clipboard

Peng Xu, Xinchi Chen, Xiaofei Ma, Zhiheng Huang, Bing Xiang

Entailment Tree Explanations via Iterative Retrieval-Generation Reasoner (2022.findings-naacl)

Challenge:	Existing methods for document representation learning are significantly affected by the scarcity of document-level data.
Approach:	They propose to use a graph attention network on top of the available pretrained Transformers model to learn document embeddings.
Outcome:	Empirically, the proposed approach is effective in document classification and document retrieval tasks.

Copied to clipboard

Danilo Neves Ribeiro, Shen Wang, Xiaofei Ma, Rui Dong, Xiaokai Wei, Henghui Zhu, Xinchi Chen, Peng Xu, Zhiheng Huang, Andrew Arnold, Dan Roth

Question Answering

Challenge:	Large language models have achieved high performance on various natural language benchmarks, but the explainability of their output remains elusive.
Approach:	They propose an architecture called iterative retrieval-generation reasoner that generates an entailment tree that explains a given hypothesis by using premises from C.
Outcome:	The proposed model outperforms existing benchmarks on premise retrieval and entailment tree generation with around 300% gain in overall correctness.

LLMEval-Fair: A Large-Scale Longitudinal Study on Robust and Fair Evaluation of Large Language Models (2026.acl-long)

Copied to clipboard

Ming Zhang, Yujiong Shen, Jingyi Deng, Yuhui Wang, Huayu Sha, Kexin Tan, Qiyuan Peng, Yue Zhang, Junzhe Wang, Shichun Liu, Yueyuan Huang, Jingqi Tong, Changhao Jiang, Yilong Wu, Zhihao Zhang, Mingqi Wu, Mingxu Chai, Zhiheng Xi, Shihan Dou, Tao Gui, Qi Zhang, Xuanjing Huang

RobustQA: Benchmarking the Robustness of Domain Adaptation for Open-Domain Question Answering (2023.findings-acl)

Challenge:	Existing evaluation of Large Language Models on static benchmarks is vulnerable to data contamination and leaderboard overfitting.
Approach:	LLMEval-Fair framework provides a framework for dynamic evaluation of Large Language Models . evaluators use a proprietary bank of 220k graduate-level questions to analyze model data .
Outcome:	LLMEval-Fair provides robust and credible evaluation framework for Large Language Models . it provides a strong empirical validation for the dynamic evaluation paradigm .

Copied to clipboard

Rujun Han, Peng Qi, Yuhao Zhang, Lan Liu, Juliette Burger, William Yang Wang, Zhiheng Huang, Bing Xiang, Dan Roth

Subspace Defense: Discarding Adversarial Perturbations by Learning a Subspace for Clean Signals (2024.lrec-main)

Challenge:	Existing ODQA datasets consist mainly of Wikipedia corpus, and are insufficient to study models’ generalizability across diverse domains.
Approach:	They propose a benchmark to evaluate ODQA's domain robustness using Wikipedia corpus . they annotate QA pairs in retrieval datasets with rigorous quality control .
Outcome:	The proposed benchmark improves model performance on annotated QA pairs in retrieval datasets with rigorous quality control.

Copied to clipboard

Rui Zheng, Yuhao Zhou, Zhiheng Xi, Tao Gui, Qi Zhang, Xuanjing Huang

Connectivity Patterns are Task Embeddings (2023.findings-acl)

Challenge:	Existing models that extract discrete inputs into fixed-length representations are vulnerable to adversarial attacks that place perturbations on clean inputs to fool DNNs.
Approach:	They propose to inspect the subspaces of sample features through spectral analysis to better understand adversarial attacks.
Outcome:	The proposed strategy enables the model to inherently suppress adversaries, which boosts model robustness and motivates new directions of effective adversarial defense.

Copied to clipboard

Zhiheng Xi, Rui Zheng, Yuansen Zhang, Xuanjing Huang, Zhongyu Wei, Minlong Peng, Mingming Sun, Qi Zhang, Tao Gui

Reward Modeling Requires Automatic Adjustment Based on Data Quality (2024.findings-emnlp)

Challenge:	Existing methods for predicting inter-task transferability are sparse and task-specific.
Approach:	They propose a method that uses connectivity patterns of neurons as a unique identifier associated with a task.
Outcome:	The proposed method outperforms baselines in predicting inter-task transferability across data regimes and transfer settings while keeping high efficiency in computation and storage.

Copied to clipboard

Binghai Wang, Rui Zheng, Lu Chen, Zhiheng Xi, Wei Shen, Yuhao Zhou, Dong Yan, Tao Gui, Qi Zhang, Xuanjing Huang

Challenge:	Reinforcement Learning from Human Feedback (RLHF) is a method for aligning language models with human values.
Approach:	They propose a method that automatically adjusts reward modeling based on data quality . they use preference data to train a reward model that is more aligned with human values .
Outcome:	The proposed method stabilizes reward model training and significantly improves alignment performance on human preference datasets.

Characterizing the Impacts of Instances on Robustness (2023.findings-acl)

Copied to clipboard

Rui Zheng, Zhiheng Xi, Qin Liu, Wenbin Lai, Tao Gui, Qi Zhang, Xuanjing Huang, Jin Ma, Ying Shan, Weifeng Ge

Improving Cross-task Generalization of Unified Table-to-text Models with Compositional Task Configurations (2023.findings-acl)

Challenge:	Existing defense approaches focus on developing new model structures or training algorithms, but they do little to tap the potential of training instances.
Approach:	They propose a method that can distinguish between robust and non-robust instances according to the model’s sensitivity to perturbations on individual instances during training.
Outcome:	The proposed method can distinguish between robust and non-robust instances according to the model’s sensitivity to perturbations on individual instances during training.

Copied to clipboard

Jifan Chen, Yuhao Zhang, Lan Liu, Rui Dong, Xinchi Chen, Patrick Ng, William Yang Wang, Zhiheng Huang

ORTicket: Let One Robust BERT Ticket Transfer across Different Tasks (2024.lrec-main)

Challenge:	Existing methods for multitask learning typically use a dataset name as input prefix, which limits the effectiveness of multitask training.
Approach:	They propose compositional task configurations, a set of prompts prepended to the encoder to improve cross-task generalization of unified models.
Outcome:	The proposed model outperforms the UnifiedSKG baseline by noticeable margins in both in-domain and zero-shot settings.

Copied to clipboard

Yuhao Zhou, Wenxiang Chen, Rui Zheng, Zhiheng Xi, Tao Gui, Qi Zhang, Xuanjing Huang

Toward Optimal LLM Alignments Using Two-Player Games (2025.findings-emnlp)

Challenge:	Pretrained language models are susceptible to subtle perturbations and require multiple adversarial training during fine-tuning to improve their robustness.
Approach:	They propose a novel adversarial defense method ORTicket that fine-tunes a model for downstream tasks.
Outcome:	The proposed method achieves comparable robustness to other defense methods while maintaining the efficiency of fine-tuning.

Copied to clipboard

Rui Zheng, Hongyi Guo, Zhihan Liu, Xiaoying Zhang, Yuanshun Yao, Xiaojun Xu, Zhaoran Wang, Zhiheng Xi, Tao Gui, Qi Zhang, Xuanjing Huang, Yang Liu, Hang Li

Which Reasoning Trajectories Teach Students to Reason Better? A Simple Metric of Informative Alignment (2026.acl-long)

Challenge:	Alignment of large language models (LLM) is a process that ensures the model’s responses to user prompts align with human intentions and social values.
Approach:	They propose an alignment method based on a two-agent game consisting of an adversarial agent and a defensive agent.
Outcome:	The proposed method improves on a two-agent game with an adversarial agent and a defensive agent.

Copied to clipboard

Yuming Yang, Mingyoung Lai, Wanxu Zhao, Xiaoran Fan, Zhiheng Xi, Mingqi Wu, Chiyue Huang, Jun Zhao, Haijun Lv, Jian Tong, Yunhua Zhou, Yicheng Zou, Qipeng Guo, Tao Gui, Qi Zhang, Xuanjing Huang

Efficient Adversarial Training with Robust Early-Bird Tickets (2022.emnlp-main)

Challenge:	Existing methods assess suitability primarily through student likelihood, favoring trajectories that align closely with the student model’s current behavior but overlooking more informative ones.
Approach:	They propose a Rank–Surprisal Ratio metric that captures both alignment and informativeness to assess the suitability of a reasoning trajectory.
Outcome:	The proposed metric captures both alignment and informativeness to assess the suitability of a reasoning trajectory.

Copied to clipboard

Zhiheng Xi, Rui Zheng, Tao Gui, Qi Zhang, Xuanjing Huang

MM-Doc-R1: Training Agents for Long Document Visual Question Answering through Multi-turn Reinforcement Learning (2026.findings-acl)

Challenge:	Existing methods to improve the robustness of pre-trained language models are expensive because of the need to generate adversarial examples via gradient descent.
Approach:	They propose an adversarial optimization method that searches for robust tickets with structured sparsity in the early stage and fine-tunes tickets in the remaining time.
Outcome:	The proposed method achieves up to 7 13 training speedups while maintaining comparable or even better robustness compared to the most competitive state-of-the-art methods.

Copied to clipboard

Jiahang Lin, Kai Hu, Binghai Wang, Yuhao Zhou, Zhiheng Xi, Honglin Guo, Shichun Liu, Junzhe Wang, Shihan Dou, Enyu Zhou, Hang Yan, Zhenhua Han, Tao Gui, Qi Zhang, Xuanjing Huang

Visual Data in NLP

Challenge:	Existing work on long document visual question answering is based on Retrieval-Augmented Generation (RAG) where textual or visual content is encoded into embeddings and relevance is determined by similarity scores with respect to the original query.
Approach:	They propose a framework that employs an agentic, vision-aware workflow to address long document visual question answering through iterative information discovery and synthesis.
Outcome:	The proposed framework outperforms existing RL systems by 10.4% on the MMLongbench-Doc benchmark and demonstrates superior training performance over GRPO.

LLMEval-Med: A Real-world Clinical Benchmark for Medical LLMs with Physician Validation (2025.findings-emnlp)

Copied to clipboard

Ming Zhang, Yujiong Shen, Zelin Li, Huayu Sha, Binze Hu, Yuhui Wang, Chenhao Huang, Shichun Liu, Jingqi Tong, Changhao Jiang, Mingxu Chai, Zhiheng Xi, Shihan Dou, Tao Gui, Qi Zhang, Xuanjing Huang

TL-Training: A Task-Feature-Based Framework for Training Large Language Models in Tool Use (2025.findings-emnlp)

Challenge:	Current medical benchmarks have limitations in question design, data sources and evaluation methods.
Approach:	They propose a new benchmark covering five core medical areas . it includes 2,996 questions created from real-world electronic health records .
Outcome:	The proposed model covers five core medical areas and includes 2,996 questions created from real-world electronic health records and expert-designed clinical scenarios.

Copied to clipboard

Junjie Ye, Yilong Wu, Sixian Li, Yuming Yang, Zhiheng Xi, Tao Gui, Qi Zhang, Xuanjing Huang, Peng Wang, Zhongchao Shi, Jianping Fan, Zhengyin Du

Language Agnostic Multilingual Information Retrieval with Contrastive Learning (2023.findings-acl)

Challenge:	a new approach to training large language models (LLMs) overlooks task-specific characteristics in tool use, leading to performance bottlenecks.
Approach:	They propose a task-feature-based framework that mitigates the effects of suboptimal training data . they use a dataset to train large-scale LLMs and a reward mechanism tailored to error categories .
Outcome:	The proposed framework matches or surpasses open- and closed-source LLMs in tool-use performance using only 1,217 training data points.

Copied to clipboard

Xiyang Hu, Xinchi Chen, Peng Qi, Deguang Kong, Kunlun Liu, William Yang Wang, Zhiheng Huang

Inverse-Q*: Token Level Reinforcement Learning for Aligning Large Language Models Without Preference Data (2024.findings-emnlp)

Challenge:	Annotated training data is costly to obtain in many languages .
Approach:	They propose a semantic contrastive loss to align parallel sentences that share the same semantics in different languages and a language contrastive gain to leverage parallel sentence pairs to remove language-specific information from non-parallel corpora.
Outcome:	The proposed model improves retrieval performance while requiring less computational effort.

Copied to clipboard

Han Xia, Songyang Gao, Qiming Ge, Zhiheng Xi, Qi Zhang, Xuanjing Huang

Parrot: A Training Pipeline Enhances Both Program CoT and Natural Language CoT for Reasoning (2025.emnlp-main)

Challenge:	Reinforcement Learning from Human Feedback (RLHF) relies on complex methodologies like Proximal Policy Optimization (PPO) that require extensive hyper-parameter tuning and pose challenges in sample efficiency and stability.
Approach:	They propose an innovative framework that leverages direct preference optimization techniques but extends them by estimating the conditionally optimal policy directly from the model’s responses.
Outcome:	The proposed framework matches and exceeds the effectiveness of Proximal Policy Optimization (PPO) in terms of convergence speed and alignment of model responses with human preferences.

Copied to clipboard

Senjie Jin, Lu Chen, Zhiheng Xi, Yuhui Wang, Sirui Song, Yuhao Zhou, Xinbo Zhang, Peng Sun, Hong Lu, Tao Gui, Qi Zhang, Xuanjing Huang

RoCoIns: Enhancing Robustness of Large Language Models through Code-Style Instructions (2024.lrec-main)

Challenge:	Existing work focuses on enabling models to generate natural language chain-of-thought rationales or leverage executable and verifiable code, such as Python.
Approach:	They propose a novel training pipeline that integrates sequential P-CoT and N-Co T generation and a subtask hybrid training strategy to facilitate natural language transferability.
Outcome:	The proposed training pipeline improves both N-CoT and P-Co T performance over the RL baseline.

Copied to clipboard

Yuansen Zhang, Xiao Wang, Zhiheng Xi, Han Xia, Tao Gui, Qi Zhang, Xuanjing Huang

Beyond [CLS] through Ranking by Generation (2020.emnlp-main)

Challenge:	Large Language Models (LLMs) have shown remarkable capabilities in following human instructions and solving NLU tasks.
Approach:	They propose to use code style instructions to replace typically natural language instructions to provide more precise instructions and strengthen the robustness of LLMs.
Outcome:	The proposed method outperforms natural language models on eight robustness datasets and achieves an improvement of 5.68% in test set accuracy and a reduction of 5.66 points in Attack Success Rate (ASR).

Copied to clipboard

Cicero Nogueira dos Santos, Xiaofei Ma, Ramesh Nallapati, Zhiheng Huang, Bing Xiang

Information Retrieval

Challenge:	Recent work on generative ranking models for Information Retrieval has focused on discriminative methods that learn a similarity function to compare questions and candidates answers.
Approach:	They propose to use a language model to train a ranking function that model the semantic similarity of documents and queries instead of discriminative ranking functions.
Outcome:	The proposed approaches are as effective as state-of-the-art discriminative models for the answer selection task and show unlikelihood losses are reduced for IR.

Self-Demos: Eliciting Out-of-Demonstration Generalizability in Large Language Models (2024.findings-naacl)

Copied to clipboard

Wei He, Shichun Liu, Jun Zhao, Yiwen Ding, Yi Lu, Zhiheng Xi, Tao Gui, Qi Zhang, Xuanjing Huang

StepCoder: Improving Code Generation with Reinforcement Learning from Compiler Feedback (2024.acl-long)

Challenge:	Existing methods that rely on limited demos and out-of-demonstration (OOD) queries fail when faced with out- of-demotion queries.
Approach:	They propose a query-aware prompting method that elicits the inherent generalizability of large language models by query-based demo generation.
Outcome:	The proposed method outperforms state-of-the-art methods in the OOD setting and two public math benchmarks.

Copied to clipboard

Shihan Dou, Yan Liu, Haoxiang Jia, Enyu Zhou, Limao Xiong, Junjie Shan, Caishuang Huang, Xiao Wang, Xiaoran Fan, Zhiheng Xi, Yuhao Zhou, Tao Ji, Rui Zheng, Qi Zhang, Tao Gui, Xuanjing Huang

Programming Languages in NLP

Challenge:	Existing work integrates reinforcement learning with compiler feedback to enhance code generation quality but the long code generated by LLMs makes RL exploration ineffective.
Approach:	They propose a framework that integrates reinforcement learning and compiler feedback to enhance code generation quality.
Outcome:	The proposed framework outperforms state-of-the-art approaches in corresponding benchmarks and integrates reinforcement learning with compiler feedback to improve code generation quality.

Dancing in Chains: Reconciling Instruction Following and Faithfulness in Language Models (2024.emnlp-main)

Copied to clipboard

Zhengxuan Wu, Yuhao Zhang, Peng Qi, Yumo Xu, Rujun Han, Yian Zhang, Jifan Chen, Bonan Min, Zhiheng Huang

RealBehavior: A Framework for Faithfully Characterizing Foundation Models’ Human-like Behavior Mechanisms (2023.findings-emnlp)

Challenge:	Modern language models fail to follow human instructions while being faithful . a trade-off exists between instruction following and faithfulness when training LMs .
Approach:	They propose a method that relies on Reject-sampling by Self-instruct with Continued Fine-tuning to train LMs to follow human instructions while being faithful.
Outcome:	The proposed method outperforms vanilla MTL with high-quality data, but with significantly smaller data.

Copied to clipboard

Enyu Zhou, Rui Zheng, Zhiheng Xi, Songyang Gao, Xiaoran Fan, Zichu Fei, Jingting Ye, Tao Gui, Qi Zhang, Xuanjing Huang

Challenge:	Existing studies on human-like behaviors in foundation models do not verify their faithfulness . a simple application of psychological tools cannot faithfully characterize all human-type behaviors .
Approach:	They propose a framework to characterize humanoid behaviors in foundation models . they argue that a simple application of psychological tools cannot faithfully characterize all human-like behaviors .
Outcome:	The proposed framework assesses the faithfulness of results based on reproducibility, internal consistency, and generalizability.

Tokenization Consistency Matters for Generative Models on Extractive NLP Tasks (2023.findings-emnlp)

Copied to clipboard

Kaiser Sun, Peng Qi, Yuhao Zhang, Lan Liu, William Wang, Zhiheng Huang

Question Answering

Challenge:	Pretrained sequence-to-sequence (seq2sequ) models have been widely used to solve extractive tasks, where parts of the input are extracted to form the desired output.
Approach:	They propose a simple fix to tokenization inconsistency that damages extractive nature of generative models by causing performance drop and hallucination.
Outcome:	The proposed model performs better in both in-domain and out-of-domain datasets with a notable average of +1.7 F1 gain when a BART model is trained on SQuAD and evaluated on 8 QA datasets.

VRPO: Rethinking Value Modeling for Robust RL under Noisy Supervision in LLM Post-Training (2026.acl-long)

Copied to clipboard

Dingwei Zhu, Shihan Dou, Zhiheng Xi, Senjie Jin, Guoqiang Zhang, Jiazheng Zhang, Junjie Ye, Mingxu Chai, Enyu Zhou, Ming Zhang, Yuhui Wang, Caishuang Huang, Chenhao Huang, Yunke Zhang, Yuran Wang, Tao Gui, Qi Zhang, Xipeng Qiu, Xuanjing Huang

Feedback-Driven Tool-Use Improvements in Large Language Models via Automated Build Environments (2026.findings-acl)

Challenge:	Reinforcement Learning (RL) in real-world environments often suffers from ambiguous or incomplete supervision.
Approach:	They propose a framework that enhances value modeling for robust RL in LLM post-training by integrating auxiliary losses guided by entropy and perplexity from a frozen language model and variational information bottleneck.
Outcome:	The proposed framework outperforms baselines on multi-turn dialogue, math reasoning, and science QA with rule-based and model-based rewards.

Copied to clipboard

Junjie Ye, Changhao Jiang, Zhengyin Du, Yufei Xu, Xuesong Yao, Zhiheng Xi, Xiaoran Fan, Qi Zhang, Tao Gui, Xuanjing Huang, Jiecao Chen

From Scores to Preferences: Redefining Evaluation Paradigm for Speech Quality Reward Modeling (2026.findings-acl)

Challenge:	Currently, there are no efficient reinforcement learning (RL) frameworks specifically designed for tool use.
Approach:	They propose an automated environment construction pipeline that incorporates scenario decomposition, document generation, function integration, complexity scaling, and localized deployment to enable high-quality training environments without external tools.
Outcome:	The proposed framework significantly improves the models’ tool-use performance without degrading their general capabilities.

Copied to clipboard

Yifei Cao, Changhao Jiang, Jiabao Zhuang, Jiajun Sun, Ming Zhang, Zhiheng Xi, Hui Li, Shihan Dou, Yuran Wang, Yunke Zhang, Tao Ji, Tao Gui, Qi Zhang, Xuanjing Huang

ToolHop: A Query-Driven Benchmark for Evaluating Large Language Models in Multi-Hop Tool Use (2025.acl-long)

Challenge:	Experimental results show that the MOS-aware GRM significantly improves fine-grained speech quality discrimination.
Approach:	They propose a MOS-aware reward model that incorporates MOS gap into reward function during reinforcement learning.
Outcome:	The proposed model significantly improves fine-grained speech quality discrimination.

Copied to clipboard

Junjie Ye, Zhengyin Du, Xuesong Yao, Weijian Lin, Yufei Xu, Zehui Chen, Zaiyuan Wang, Sining Zhu, Zhiheng Xi, Siyu Yuan, Tao Gui, Qi Zhang, Xuanjing Huang, Jiecao Chen

Mitigating Object Hallucinations in MLLMs via Multi-Frequency Perturbations (2025.findings-emnlp)

Challenge:	Effective evaluation of multi-hop tool use is critical for analyzing the understanding, reasoning, and function-calling capabilities of large language models.
Approach:	They propose a dataset that provides rigorous evaluation of multi-hop tool use.
Outcome:	The proposed model achieves 49.04% accuracy across five model families.

Copied to clipboard

Shuo Li, Jiajun Sun, Guodong Zheng, Xiaoran Fan, Yujiong Shen, Yi Lu, Zhiheng Xi, Yuming Yang, Wenming Tan, Tao Ji, Tao Gui, Qi Zhang, Xuanjing Huang

Distill Visual Chart Reasoning Ability from LLMs to MLLMs (2025.findings-emnlp)

Challenge:	Multimodal large language models have demonstrated remarkable performance in visual-language tasks, but their authenticity is often compromised by object hallucinations.
Approach:	They propose a multi-frequency perturbation method that leverages both low-frequency and high-frequency features of images to perturb visual feature representations and explicitly suppress redundant frequency-domain features during inference.
Outcome:	The proposed method significantly mitigates object hallucinations across various model architectures.

Copied to clipboard

Wei He, Zhiheng Xi, Wanxu Zhao, Xiaoran Fan, Yiwen Ding, Zifei Shan, Tao Gui, Qi Zhang, Xuanjing Huang

Improve Transformer Models with Better Relative Position Embeddings (2020.findings-emnlp)

Challenge:	a new method for generating chart annotations is proposed to improve visual reasoning in multimodal large language models.
Approach:	They propose a code-as-intermediary translation method for distilling visual reasoning abilities from LLMs to MLLMs.
Outcome:	The proposed method is cost-effective, efficient and scalable.

Copied to clipboard

Zhiheng Huang, Davis Liang, Peng Xu, Bing Xiang

TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing (2021.acl-demo)

Challenge:	Existing methods for generating position embeddings are not fully utilized in NLP tasks.
Approach:	They propose to generalize the absolute position embedding to a generalized relative position embedded method . they also propose to use the relative embeddable method to improve the accuracy of large models .
Outcome:	The proposed method improves accuracy on the SQuAD1.1 dataset compared to previous methods . it can be easily adopted as a drop-in replacement for improving accuracy of large models .

Copied to clipboard

Xiao Wang, Qin Liu, Tao Gui, Qi Zhang, Yicheng Zou, Xin Zhou, Jiacheng Ye, Yongxin Zhang, Rui Zheng, Zexiong Pang, Qinzhuo Wu, Zhengyan Li, Chong Zhang, Ruotian Ma, Zichu Fei, Ruijian Cai, Jun Zhao, Xingwu Hu, Zhiheng Yan, Yiding Tan, Yuan Hu, Qiyuan Bian, Zhihua Liu, Shan Qin, Bolin Zhu, Xiaoyu Xing, Jinlan Fu, Yue Zhang, Minlong Peng, Xiaoqing Zheng, Yaqian Zhou, Zhongyu Wei, Xipeng Qiu, Xuanjing Huang

LoRACoE: Improving Large Language Model via Composition-based LoRA Expert (2025.emnlp-main)

Challenge:	Existing approaches to textual robustness evaluation focus on slightly modifying the input data, which maintains the original meaning and results in a different prediction.
Approach:	They propose a multilingual robustness evaluation toolkit for NLP that integrates universal text transformations, task-specific transformations and adversarial attack.
Outcome:	The toolkit includes universal text transformation, task-specific transformation, adversarial attack, subpopulation, and their combinations to provide comprehensive robustness analyses.

Copied to clipboard

Guanyu Li, Zhiheng Xi, Zhihao Zhang, Boyang Hong, Tao Gui, Qi Zhang, Xuanjing Huang

Beyond Scaling: Measuring and Predicting the Upper Bound of Knowledge Retention in Language Model Pre-Training (2026.acl-long)

Challenge:	Recent studies show that the Mixture of Experts architecture improves performance of large language models.
Approach:	They propose a method to build static experts using LoRA parameters . they propose to use rank-level parameters to build experts based on rank-based parameters based in LoRA module.
Outcome:	The proposed method improves task performance across a broader range of tasks.

Copied to clipboard

Changhao Jiang, Ming Zhang, Yifei Cao, Junjie Ye, Xiaoran Fan, Shihan Dou, Zhiheng Xi, Jiajun Sun, Yi Dong, Yujiong Shen, Jingqi Tong, Baoyu Fan, Tao Gui, Qi Zhang, Xuanjing Huang