Papers by Bei Liu
Adaptive Spatial and Temporal Redundancy Optimization for Efficient Reasoning in Large Language Models (2026.acl-long)
Copied to clipboard
Tianle Chen, Pengyu Cheng, Qiyuan Zhu, Jiacheng Wang, Bei Liu, Hao Gu, Ruijie Shen, Xiaofeng Hou, Sirui Han, Jiacheng Liu
| Challenge: | Existing research to improve CoT efficiency falls into three categories, each with distinct limitations. |
| Approach: | They propose a training-free framework that addresses both dimensions of CoT reasoning by applying a progressive precision reduction strategy coupled with an entropy-based confidence mechanism for adaptive termination. |
| Outcome: | Empirical results show that the proposed framework achieves 11.3 efficiency gain without compromising accuracy. |
Analytical FFN-to-MoE Restructuring via Activation Pattern Analysis (2026.acl-long)
Copied to clipboard
Zehua Pei, Hui-Ling Zhen, Lancheng Zou, Xianzhi Yu, Wulong Liu, Sinno Jialin Pan, Mingxuan Yuan, Bei Yu
| Challenge: | Large language models (LLMs) are fast but require expensive pre-training . a new approach to scale large language models into MoEs reduces inference costs . |
| Approach: | They propose an analytical post-training framework that rapidly restructures FFNs into sparse MoE architectures using only a small calibration dataset. |
| Outcome: | The proposed framework outperforms existing methods on a small calibration dataset. |
BTC-LLM: Efficient Sub-1-Bit LLM Quantization via Learnable Transformation and Binary Codebook (2026.acl-long)
Copied to clipboard
Hao Gu, Lujun Li, Hao Wang, Lei Wang, Zheyu Wang, Bei Liu, Jiacheng Liu, Qiyuan Zhu, Sirui Han, Yike Guo
| Challenge: | Recent sparsity-aware binarization approaches can achieve sub-1-bit compression, but they face performance degradation, mask-management overhead, and limited hardware compatibility. |
| Approach: | They propose a binary quantization framework that leverages binary pattern clustering and weight transformation to overcome performance degradation and mask-management overhead. |
| Outcome: | The proposed framework achieves state-of-the-art compression (1.11–0.7 bits) it maintains high performance with only a 3.1% accuracy drop in zero-shot benchmarks while delivering a 1.6 speedup over FP16. |
TEaR: Improving LLM-based Machine Translation with Systematic Self-Refinement (2025.findings-naacl)
Copied to clipboard
Zhaopeng Feng, Yan Zhang, Hao Li, Bei Wu, Jiayu Liao, Wenqiang Liu, Jun Lang, Yang Feng, Jian Wu, Zuozhu Liu
| Challenge: | Large Language Models (LLMs) have achieved impressive results in Machine Translation (MT). human evaluations reveal that LLM-generated translations still contain various errors. |
| Approach: | They propose a LLM-based self-refinement framework that feeds error information back into LLMs to facilitate self-finement, leading to enhanced translation quality. |
| Outcome: | The proposed framework outperforms internal refinement and feedback methods while ensuring a robust translation quality baseline. |
“What Do You Mean by That?” A Parser-Independent Interactive Approach for Enhancing Text-to-SQL (2020.emnlp-main)
Copied to clipboard
| Challenge: | In Natural Language Interfaces to Databases systems, text-to-SQL parsers allow users to query databases by using natural language questions. |
| Approach: | They propose a parser-independent interactive approach that interacts with users using multi-choice questions and can easily work with arbitrary parsers. |
| Outcome: | The proposed approach improves performance with limited interaction turns by using simulation and human evaluation on two cross-domain datasets with five state-of-the-art parsers. |
Reasoning Like Program Executors (2022.emnlp-main)
Copied to clipboard
Xinyu Pi, Qian Liu, Bei Chen, Morteza Ziyadi, Zeqi Lin, Qiang Fu, Yan Gao, Jian-Guang Lou, Weizhu Chen
| Challenge: | Existing language models are inadequate in reasoning, according to studies . a new reasoning pre-training paradigm is based on pretraining language models with programs . |
| Approach: | They propose a reasoning pre-training paradigm that empowers language models to harvest reasoning knowledge possessed by program executors. |
| Outcome: | The proposed reasoning pre-training paradigm can boost models' reasoning skills . it can be instantiated by different kinds of program executors and run on a single database . |
Beyond Decoder-only: Large Language Models Can be Good Encoders for Machine Translation (2025.findings-acl)
Copied to clipboard
Yingfeng Luo, Tong Zheng, Yongyu Mu, Bei Li, Qinghong Zhang, Yongqi Gao, Ziqiang Xu, Peinan Feng, Xiaoqian Liu, Tong Xiao, JingBo Zhu
| Challenge: | Recent advances in machine translation have focused on a single pre-trained decoder . encoder-decoder architectures have received relatively little attention in NMT . |
| Approach: | They propose a method that leverages LLMs as MT encoders and pairs them with lightweight decoders to develop universal translation models. |
| Outcome: | The proposed method matches or surpasses baselines in terms of translation quality but achieves 75% reduction in memory footprint of the KV cache. |
Does Multi-Encoder Help? A Case Study on Context-Aware Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | In encoder-decoder neural models, multiple encoders are used to represent contextual information in addition to the individual sentence. |
| Approach: | They propose to use multiple context encoders to encode the individual sentences in document-level neural machine translation (NMT) They propose a noisy dropout setup and a single-encoder approach to encode context sentences. |
| Outcome: | The proposed approach encodes the context and the current sentence without contexts. |
Revealing the Parallel Multilingual Learning within Large Language Models (2024.emnlp-main)
Copied to clipboard
Yongyu Mu, Peinan Feng, Zhiquan Cao, Yuzhang Wu, Bei Li, Chenglong Wang, Tong Xiao, Kai Song, Tongran Liu, Chunliang Zhang, JingBo Zhu
| Challenge: | Large language models (LLMs) can handle multilingual and cross-lingual text within a single input; however, previous studies focusing on using English as the pivot language to enhance language understanding and reasoning focus on using multiple languages. |
| Approach: | They propose to use parallel multilingual input to enhance the model's comprehension of the input and to examine how multilingual processing affects prediction. |
| Outcome: | The proposed model can handle multilingual and cross-lingual text within a single input, but previous studies focused on using English as the pivot language to enhance language understanding and reasoning. |
TCPO: Thought-Centric Preference Optimization for Effective Embodied Decision-making (2025.emnlp-main)
Copied to clipboard
Kechen Jiao, Zhirui Fang, Jiahao Liu, Bei Li, Qifan Wang, Xinyu Liu, Junhao Ruan, Zhongjian Qiao, Yifan Zhu, Yaxin Xu, Jingang Wang, Xiu Li
| Challenge: | Existing post-SFT methods for embodied AI are constrained by sparse rewards and action-only optimization, resulting in low sample efficiency, poor consistency, and model degradation. |
| Approach: | They propose to integrate Thought-Centric Preference Optimization (TCPO) into embodied decision-making by transforming sparse reward signals into richer step sample pairs. |
| Outcome: | The proposed approach achieves an average success rate of 26.67% in the ALFWorld environment, and a 6% improvement over RL4VLM. |
Hybrid Alignment Training for Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to align large language models with instructions and preferences are conflicting . et al., 2023b) show that hybrid alignment training can outperform baselines . |
| Approach: | They propose a hybrid alignment training approach based on alternating alignment and modified elastic weight consolidation methods to achieve better collaboration between different alignment tasks. |
| Outcome: | The proposed approach outperforms baseline alignment training methods on summarization and dialogue tasks. |
Debiasing Event Understanding for Visual Commonsense Tasks (2022.findings-acl)
Copied to clipboard
| Challenge: | a recent study shows that object-based event understanding is purely likelihood-based, leading to incorrect event prediction. |
| Approach: | They propose to mitigate object-based event understanding by optimizing aggregation with association-based prediction. |
| Outcome: | The proposed approach improves visual commonsense reasoning tasks by combining do-calculus with association-based prediction. |
LEMON: Language-Based Environment Manipulation via Execution-Guided Pre-training (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to language-based environment manipulation are difficult to generalize across environments. |
| Approach: | They propose a general framework for language-based environment manipulation tasks that can deal with various environments using the same generative language model. |
| Outcome: | The proposed framework achieves new state-of-the-art results on four of the tasks and the execution-guided pre-training strategy brings remarkable improvements on all experimental tasks. |
ProBench: Judging Multimodal Foundation Models on Open-ended Multi-domain Expert Tasks (2025.findings-acl)
Copied to clipboard
| Challenge: | Solving expert-level multimodal tasks requires strong user query understanding, domain-specific knowledge, and advanced reasoning abilities. |
| Approach: | They propose a benchmark of open-ended user queries encapsulating professional expertise and advanced reasoning. |
| Outcome: | The proposed benchmark is publicly accessible at TBC. |
One Pair Suffices: Unlocking Universal Zero-Shot Translation via Cross-Architecture Alignment (2026.acl-long)
Copied to clipboard
| Challenge: | Current paradigms for empowering Large Language Models with multilingual capabilities rely heavily on massive instruction tuning. |
| Approach: | They propose a hybrid cross-alignment approach that fuses a frozen NLLB encoder with a Qwen decoder via a closed-loop dual-adapter architecture. |
| Outcome: | The proposed model outperforms towerPlus-9B and Aya-101 on language-agnostic projection protocols. |
Bit-by-Bit: Progressive QAT Strategy with Outlier Channel Splitting for Stable Low-Bit LLMs (2026.acl-long)
Copied to clipboard
Binxing Xu, Hao Gu, Lujun Li, Hao Wang, Bei Liu, Jiacheng Liu, Qiyuan Zhu, Xintong Yang, Chao Li, Sirui Han, Yike Guo
| Challenge: | Existing approaches to training LLMs at ultra-low precisions suffer from convergence instability and substantial training costs. |
| Approach: | They propose a progressive QAT framework with outlier channel splitting to address these issues . they use nested structure of integer quantization grids to enable a "train once, deploy any precision" paradigm . |
| Outcome: | The proposed framework outperforms baselines on both Llama2/3 and W2A16, with an 11 speedup over BF16. |
ReMamba: Equip Mamba with Effective Long-Sequence Modeling (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Mamba models demonstrate superior inference efficiency and competitive performance on short-context tasks, but their capacity to comprehend long contexts is limited compared to transformer-based models. |
| Approach: | They propose a model which incorporates selective compression and adaptation techniques within a two-stage re-forward process, incurring minimal additional inference costs overhead. |
| Outcome: | The proposed model improves on the LongBench and L-Eval benchmarks by 3.2 and 1.6 points and attains performance almost on par with same-size transformer models. |
Mem-Gallery: Benchmarking Multimodal Long-Term Conversational Memory for MLLM Agents (2026.acl-long)
Copied to clipboard
Yuanchen Bei, Tianxin Wei, Xuying Ning, Yanjun Zhao, Zhining Liu, Xiao Lin, Yada Zhu, Hendrik Hamann, Jingrui He, Hanghang Tong
| Challenge: | Existing benchmarks evaluate multi-session memory in text-only conversations or assess multimodal understanding within localized contexts. |
| Approach: | They propose a benchmark for evaluating multimodal long-term conversational memory in MLLM agents. |
| Outcome: | The proposed framework assesses key memory capabilities along three functional dimensions: memory extraction and test-time adaptation, memory reasoning, and memory knowledge management. |
Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-aligned Quality Estimation (2024.emnlp-main)
Copied to clipboard
Yuan Ge, Yilun Liu, Chi Hu, Weibin Meng, Shimin Tao, Xiaofeng Zhao, Mahong Xia, Zhang Li, Boxing Chen, Hao Yang, Bei Li, Tong Xiao, JingBo Zhu
| Challenge: | Existing methods for instruction data selection have limitations such as relying on fragile external APIs, being affected by biases in GPT models, or reducing the diversity of the selected instruction dataset. |
| Approach: | They propose an industrial-friendly, expert-aligned and diversity-preserved instruction data selection method: Clustering and Ranking (CaR). |
| Outcome: | The proposed method outperforms Alpaca's existing methods by 32.1% in GPT-4 evaluations. |
On the Emotion Understanding of Synthesized Speech (2026.acl-long)
Copied to clipboard
Yuan Ge, Haishu Zhao, AoKai Hao, Junxiang Zhang, Bei Li, Xiaoqian Liu, Chenglong Wang, Jianjin Wang, Bingsen Zhou, Bingyu Liu, JingBo Zhu, Zhengtao Yu, Tong Xiao
| Challenge: | Existing models for emotion understanding do not capture fundamental features of synthesized speech. |
| Approach: | They evaluate emotion recognition models on synthesized speech using SER models and generative models. |
| Outcome: | The proposed model can't generalize to synthesized speech because of speech token prediction . generative models tend to infer emotion from textual semantics while ignoring paralinguistic cues. |
ODE Transformer: An Ordinary Differential Equation-Inspired Model for Sequence Generation (2022.acl-long)
Copied to clipboard
Bei Li, Quan Du, Tao Zhou, Yi Jing, Shuhan Zhou, Xin Zeng, Tong Xiao, JingBo Zhu, Xuebo Liu, Min Zhang
| Challenge: | Residual networks are an Euler discretization of solutions to Ordinary Differential Equations (ODE). |
| Approach: | They propose a residual block of layers in Transformer that can be described as a higher-order solution to ODE. |
| Outcome: | The proposed architecture can gain large improvements over strong baselines at a slight cost in inference efficiency. |
MTR-Suite: A Framework for Evaluating and Synthesizing Conversational Retrieval Benchmarks (2026.acl-long)
Copied to clipboard
Junhao Ruan, Abudukeyumu Abudula, Bei Li, Yongjing Yin, Xinyu Liu, Kechen Jiao, Xin Chen, Jingang Wang, Xunliang Cai, Tong Xiao, JingBo Zhu
| Challenge: | Existing conversational retrieval benchmarks suffer from costly, sparse human annotation or rigid, unnatural automated heuristics. |
| Approach: | They propose a framework for auditing, synthesizing, and benchmarking conversational retrieval. |
| Outcome: | The proposed framework is based on three LLM-based auditors and a multi-agent system . it mimics production-style challenges (hard topic switching, verbosity) and offers superior discriminative power. |
IIET: Efficient Numerical Transformer via Implicit Iterative Euler Method (2025.emnlp-main)
Copied to clipboard
Xinyu Liu, Bei Li, Jiahao Liu, Junhao Ruan, Kechen Jiao, Hongyin Tang, Jingang Wang, Tong Xiao, JingBo Zhu
| Challenge: | High-order numerical methods enhance performance in tasks like NLP but introduce a performance-efficiency trade-off due to increased computational overhead. |
| Approach: | They propose an iterative implicit Euler Transformer which simplifies high-order numerical methods by iterating implicit Eule. |
| Outcome: | The proposed method improves accuracy and reduces inference overhead by 55% while retaining 99.4% of the original task accuracy. |
Learning Algebraic Recombination for Compositional Generalization (2021.findings-acl)
Copied to clipboard
Chenyao Liu, Shengnan An, Zeqi Lin, Qian Liu, Bei Chen, Jian-Guang Lou, Lijie Wen, Nanning Zheng, Dongmei Zhang
| Challenge: | Neural sequence models exhibit limited compositional generalization ability in semantic parsing tasks. |
| Approach: | They propose an end-to-end neural model to learn algebraic recombination for compositional generalization. |
| Outcome: | The proposed model is based on two realistic and comprehensive compositional generalization benchmarks. |
BAPO: Boundary-Aware Policy Optimization for Reliable Agentic Search (2026.findings-acl)
Copied to clipboard
Shiyu Liu, Yongjing Yin, Jianhao Yan, Yunbo Tang, Qinggang Zhang, Bei Li, Xin Chen, Jingang Wang, Xunliang Cai, Jinsong Su
| Challenge: | Existing RL-based agentic search models fail to recognize reasoning boundaries and rarely admit "I DON'T KNOW" lack of reliability leads to plausible but unreliable answers, introducing significant risks . |
| Approach: | They propose a framework to cultivate reliable boundary awareness without compromising accuracy. |
| Outcome: | Experiments show that the proposed framework improves the reliability of agentic search models. |
Forgetting Curve: A Reliable Method for Evaluating Memorization Capability for Long-Context Models (2024.emnlp-main)
Copied to clipboard
Xinyu Liu, Runsong Zhao, Pengcheng Huang, Chunyang Xiao, Bei Li, Jingang Wang, Tong Xiao, JingBo Zhu
| Challenge: | Existing methods, tasks and benchmarks to measure model’s effective memory length are limited. |
| Approach: | They propose a method called forgetting curve to measure the memorization capability of long-context models. |
| Outcome: | The proposed method is robust to the tested corpus and experimental settings, and can be applied to any model size. |
You Impress Me: Dialogue Generation via Mutual Persona Perception (2020.acl-main)
Copied to clipboard
| Challenge: | Existing chit-chat systems tend to generate uninformative responses and lack coherent personality traits due to the diversity of speakers. |
| Approach: | They propose a transmitter-receiver framework which explicitly models understanding between interlocutors. |
| Outcome: | The proposed framework improves on a large public dataset, Persona-Chat, with a significant boost over the state-of-the-art frameworks. |
A Split-and-Recombine Approach for Follow-up Query Analysis (D19-1)
Copied to clipboard
| Challenge: | Context-dependent semantic parsing has proved to be an important but challenging task. |
| Approach: | They propose to perform follow-up query analysis to restate context-dependent queries with contextual information. |
| Outcome: | The proposed approach outperforms the state-of-the-art by nearly 8% on the FollowUp dataset . the extensibility of STAR on the SQA dataset is also promising . |
QaRL: Rollout-Aligned Quantization-Aware RL for Fast and Stable Training under Training–Inference Mismatch (2026.findings-acl)
Copied to clipboard
Hao Gu, Hao Wang, Jiacheng Liu, Lujun Li, Qiyuan Zhu, Bei Liu, Binxing Xu, Lei Wang, Xintong Yang, Sida Lin, Sirui Han, Yike Guo
| Challenge: | Recent work has shown that reinforcement learning with simple rule-based reward functions (RLVR) can induce emergent reasoning behaviors and yield gains in challenging domains such as math problem solving. |
| Approach: | They propose a rollout-alignment-quantization-aware RL which aligns training-side forward with the quantized rollout to minimize mismatch. |
| Outcome: | The proposed approach outperforms quantized-rollout training by +5.5 on Qwen3-30B-A3B MoE for math problems while maintaining low-bit throughput. |
3AM: An Ambiguity-Aware Multi-Modal Machine Translation Dataset (2024.lrec-main)
Copied to clipboard
Xinyu Ma, Xuebo Liu, Derek F. Wong, Jun Rao, Bei Li, Liang Ding, Lidia S. Chao, Dacheng Tao, Min Zhang
| Challenge: | Existing studies have shown that visual information in existing MMT datasets is insufficient, causing models to disregard it and overestimate their capabilities. |
| Approach: | They propose to use 3AM to create an ambiguity-aware multimodal machine translation dataset. |
| Outcome: | The proposed dataset includes more ambiguity and a greater variety of captions and images than other MMT datasets. |
Incomplete Utterance Rewriting as Semantic Segmentation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies focus on the task of incomplete utterance rewriting as a machine translation task. |
| Approach: | They propose a semantic segmentation task which incorporates edit operations into the problem and predicts a word-level edit matrix. |
| Outcome: | The proposed approach outperforms existing baselines on several datasets and is four times faster than the standard approach in inference. |
Partially-Aligned Data-to-Text Generation with Distant Supervision (2020.emnlp-main)
Copied to clipboard
| Challenge: | Using partially-aligned data is an alternative way of solving the dataset scarcity problem. |
| Approach: | They propose a task to generate human-readable text for describing some given structured data enabling more interpretability. |
| Outcome: | The proposed framework outperforms baseline models and validates the feasibility of using partially-aligned data. |
Shallow-to-Deep Training for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that deep training is 1:4 faster than training from scratch. |
| Approach: | They propose a shallow-to-deep training method that learns deep models by stacking shallow models. |
| Outcome: | The proposed method is 1:4 faster than training from scratch and achieves BLEU scores of 30:33 and 43:29 on two translation tasks. |
Leveraging Adjective-Noun Phrasing Knowledge for Comparison Relation Prediction in Text-to-SQL (D19-1)
Copied to clipboard
| Challenge: | Existing models for text-to-SQL do not explicitly introduce common knowledge to address comparison relations. |
| Approach: | They propose to leverage adjective-noun phrasing knowledge mined from the web to predict comparison relations in text-to-SQL. |
| Outcome: | The proposed approach improves on the original and re-split Spider datasets on comparison relation prediction. |
RouteLMT: Learned Sample Routing for Hybrid LLM Translation Deployment (2026.acl-industry)
Copied to clipboard
Yingfeng Luo, Hongyu Liu, DingYang Lin, Kaiyan Chang, Chenglong Wang, Bei Li, Quan Du, Tong Xiao, JingBo Zhu
| Challenge: | Existing routing strategies rely on heuristics, external predictors, or absolute quality estimation to capture whether the large model provides a worthwhile improvement over the small one. |
| Approach: | They propose a budget allocation problem for routing large model to large model . they propose heuristics, external predictors, or absolute quality estimation to determine the optimal signal for budgeted decisions. |
| Outcome: | The proposed model outperforms heuristics, quality/difficulty estimation baselines and achieves a superior quality–budget Pareto frontier. |
CONSTRUCTURE: Benchmarking CONcept STRUCTUre REasoning for Multimodal Large Language Models (2024.findings-emnlp)
Copied to clipboard
Zhiwei Zha, Xiangru Zhu, Yuanyi Xu, Chenghua Huang, Jingping Liu, Zhixu Li, Xuwu Wang, Yanghua Xiao, Bei Yang, Xiaoxiao Xu
| Challenge: | Existing multimodal large language models lack the ability to perceive the visual world with a deep concept structure cognition. |
| Approach: | They propose a concept-level benchmark to assess MLLMs’ hierarchical concept understanding and reasoning abilities. |
| Outcome: | The proposed model outperforms state-of-the-art models in concept structure reasoning evaluation. |
RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation (2023.emnlp-main)
Copied to clipboard
Fengji Zhang, Bei Chen, Yue Zhang, Jacky Keung, Jin Liu, Daoguang Zan, Yi Mao, Jian-Guang Lou, Weizhu Chen
| Challenge: | RepoCoder is a repository-level code completion framework that utilizes the useful information scattered in files. |
| Approach: | They propose a repository-level code completion framework called RepoCoder . it integrates a similarity-based retriever and a pre-trained code language model . they propose 'repoBench' benchmark to validate the framework's effectiveness . |
| Outcome: | The proposed framework outperforms the vanilla retrieval-augmented code completion approach in the real-world. |
CodeM: Less Data Yields More Versatility via Ability Matrix (2024.findings-acl)
Copied to clipboard
Daoguang Zan, Ailun Yu, Wei Liu, Bo Shen, Shaoxin Lin, Yongshun Gong, Yafen Yao, Yan Liu, Bei Guan, Weihua Luo, Yongji Wang, Qianxiang Wang, Lizhen Cui
| Challenge: | Recent efforts to train code large language models have been booming recently . however, this will incur significant costs in constructing data and training model considering the countless downstream scenarios. |
| Approach: | They propose a data construction strategy which decouples code LLMs’ abilities into two dimensions and constructs a lightweight training corpus that only covers a subset of target scenarios. |
| Outcome: | The proposed model can train a multilingual multitasking model using less data and training data. |