Papers by Lijun Wu
REST: Stress Testing Large Reasoning Models by Asking Multiple Problems at Once (2026.acl-long)
Copied to clipboard
| Challenge: | Recent Large Reasoning Models (LRMs) lack a narrow evaluation paradigm . a single-question evaluation setup suffers from two major limitations . |
| Approach: | They propose a stress-testing framework that exposes LRMs to multiple problems simultaneously. |
| Outcome: | The proposed framework outperforms existing models on reasoning benchmarks and state-of-the-art models. |
Tracing the Roots: A Multi-Agent Framework for Uncovering Data Lineage in Post-Training LLMs (2026.acl-long)
Copied to clipboard
Yu Li, Xiaoran Shang, Qizhi Pei, Yun Zhu, Xin Gao, Honglin Lin, Zhanping Zhong, Zhuoshi Pan, Zheng Liu, Xiaoyang Wang, Conghui He, Dahua Lin, Feng Zhao, Lijun Wu
| Challenge: | High-quality post-training data is the primary engine driving LLM capabilities . datasets are often treated as isolated artifacts, overlooking their true developmental context . |
| Approach: | They propose a framework to reconstruct the evolutionary graph of dataset development using data lineage. |
| Outcome: | The proposed framework characterizes domain-specific structural patterns in Math-oriented datasets and general-domain corpora. |
OpenHuEval: Evaluating Large Language Model on Hungarian Specifics (2025.findings-acl)
Copied to clipboard
Haote Yang, Xingjian Wei, Jiang Wu, Noémi Ligeti-Nagy, Jiaxing Sun, Yinfan Wang, Győző Zijian Yang, Junyuan Gao, Jingchao Wang, Bowen Jiang, Shasha Wang, Nanjun Yu, Zihao Zhang, Shixin Hong, Hongwei Liu, Wei Li, Songyang Zhang, Dahua Lin, Lijun Wu, Gábor Prószéky, Conghui He
| Challenge: | Recent advances in Large Language Models (LLMs) represent significant strides toward artificial general intelligence (AGI). |
| Approach: | They introduce OpenHuEval, the first benchmark for LLMs focusing on the Hungarian language and specifics. |
| Outcome: | The framework reveals intrinsic patterns and mechanisms of LLMs in non-English languages, with Hungarian serving as an example. |
Improving Temporal Generalization of Pre-trained Language Models with Lexical Semantic Change (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to improve neural language models perform poorly on emerging data. |
| Approach: | They propose a lexical-level masking strategy to post-train a neural language model using static data from past years. |
| Outcome: | The proposed method outperforms existing methods on two pre-trained language models, two classification tasks, and four benchmark datasets. |
ChartVerse: Scaling Chart Reasoning via Reliable Programmatic Synthesis from Scratch (2026.acl-long)
Copied to clipboard
Zheng Liu, Honglin Lin, Xiaoyang Wang, Xin Gao, Yu Li, Mengzhang Cai, Yun Zhu, Zhanping Zhong, Qizhi Pei, Zhuoshi Pan, Xiaoran Shang, Conghui He, Bin Cui, Wentao Zhang, Lijun Wu
| Challenge: | Existing open-source vision language models lack high-quality training data for chart reasoning . current models are simplistic and repetitive, while associated QA pairs are prone to hallucinations . |
| Approach: | They propose a framework to synthesize complex charts and reliable reasoning data from scratch. |
| Outcome: | Experimental results show that ChartVerse-8B surpasses existing models in QA and difficulty . lack of high-quality training data hampers development of open-source models . |
CipherBank: Exploring the Boundary of LLM Reasoning Capabilities through Cryptography Challenge (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated remarkable capabilities, but their capabilities in cryptographic decryption tasks remain underexplored. |
| Approach: | They propose a benchmark to evaluate the reasoning capabilities of large language models in cryptographic decryption tasks. |
| Outcome: | The proposed benchmark examines the reasoning capabilities of large language models in cryptographic decryption tasks. |
Dynamic and Efficient Inference for Text Generation via BERT Family (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods to improve performance of pre-trained language models are limited due to large-scale parameters and the universal autoregressive decoding paradigm. |
| Approach: | They propose a novel fine-tuning method which can make a single pre-trained model support Dynamic and Efficient infERence and achieve an adaptive trade-off between model performance and latency. |
| Outcome: | The proposed method achieves higher BLEU scores than the strong autoregressive Transformer model on translation tasks with 3 12 times speedup and faster inference speed compared with the BART model on four GLGE benchmark tasks. |
LEMMA: Learning from Errors for MatheMatical Advancement in LLMs (2025.findings-acl)
Copied to clipboard
Zhuoshi Pan, Yu Li, Honglin Lin, Qizhi Pei, Zinan Tang, Wei Wu, Chenlin Ming, H. Vicky Zhao, Conghui He, Lijun Wu
| Challenge: | Existing approaches focus on improving the quality of correct training data, neglecting the value contained in error data, thereby hindering the model’s reflective ability. |
| Approach: | They propose to improve LLM's reasoning ability by learning from error data and a grounded mistake augmentation method to collect representative errors. |
| Outcome: | The proposed model achieves significant performance improvements over other strong models with less than 90k data. |
GRAIT: Gradient-Driven Refusal-Aware Instruction Tuning for Effective Hallucination Mitigation (2025.findings-naacl)
Copied to clipboard
Runchuan Zhu, Xinke Jiang, Jiang Wu, Zhipeng Ma, Jiahe Song, Fengshuo Bai, Dahua Lin, Lijun Wu, Conghui He
| Challenge: | Experimental evaluations on open-ended and multiple-choice questions demonstrate GRAIT significantly outperforms existing RAIT methods in the overall performance. |
| Approach: | They propose a framework to reduce the risk of over-refusal and reduce hallucinations by rejecting unknown questions to minimize hallucinism and ensuring correct answers are not rejected. |
| Outcome: | The proposed framework outperforms existing methods on open-ended and multiple-choice questions. |
MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing (2026.acl-industry)
Copied to clipboard
Junbo Niu, Zheng Liu, Zhuangcheng Gu, Bin Wang, Linke Ouyang, Zhiyuan Zhao, Tao Chu, Tianyao He, Fan Wu, Qintong Zhang, Zhenjiang Jin, Guang Liang, Rui Zhang, Wenzheng Zhang, Yuan Qu, Zhifei Ren, Yuefeng Sun, Zirui Tang, Boyu Niu, Yuanhong Zheng, Dongsheng Ma, Ziyang Miao, Hejun Dong, Siyi Qian, Junyuan Zhang, Fangdong Wang, Jingzhou Chen, Xiaomeng Zhao, Liqun Wei, Wei Li, Shasha Wang, RuiLiang Xu, Yuanyuan Cao, Lu Chen, Qianqian Wu, Huaiyu Gu, Lindong Lu, Dechen Lin, null Shenguanlin, Xuanhe Zhou, Linfeng Zhang, Yuhang Zang, Xiaoyi Dong, Jiaqi Wang, Bo Zhang, Lei Bai, Pei Chu, Weijia Li, Jiang Wu, Lijun Wu, Zhenxiang Li, Guangyu Wang, Zhongying Tu, Chao Xu, Kai Chen, Bowen Zhou, Dahua Lin, Wentao Zhang, Conghui He
| Challenge: | Document images are characterized by higher resolutions, denser content, and more complex structural layouts. |
| Approach: | They propose a 1.2B-parameter document parsing vision-language model that decouples layout analysis from local content recognition. |
| Outcome: | The proposed model surpasses general-purpose and domain-specific models on multiple benchmarks while maintaining significantly lower computational overhead. |
Efficient Sequence Learning with Group Recurrent Networks (N18-1)
Copied to clipboard
| Challenge: | Recurrent neural networks have achieved state-of-the-art results in many artificial intelligence tasks, such as language modeling, neural machine translation and speech recognition. |
| Approach: | They propose an efficient architecture to improve the efficiency of such RNN model training by adopting the group strategy for recurrent layers while exploiting the representation rearrangement strategy between layers as well as time steps. |
| Outcome: | The proposed architecture achieves comparable or better accuracy compared with baselines, with a much smaller number of parameters and at a lower computational cost. |
Data Pollination: An Emergent Ecological Process Driving AI Population Evolution (2026.acl-long)
Copied to clipboard
| Challenge: | evidence from deployed systems suggests that language models interact through a shared data ecosystem. |
| Approach: | They propose to use data pollination to investigate stability dynamics under synthetic data training to investigate model collapse. |
| Outcome: | The proposed model can mitigate model collapse observed in recursive training, and improve performance across benchmarks. |
Unleashing the Reasoning Potential of LLMs by Critique Fine-Tuning on One Problem (2025.emnlp-main)
Copied to clipboard
| Challenge: | Critique Fine-Tuning (CFT) is a promising paradigm for unlocking the reasoning capabilities of large language models. |
| Approach: | They propose a method that leverages critique data generated from a single math problem to improve reasoning accuracy. |
| Outcome: | The proposed method surpasses one-shot RLVR while requiring 15 to 20 times less compute. |
Depth Growing for Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Neural machine translation models with tens and even more than a hundred blocks have shown effectiveness in image recognition. |
| Approach: | They propose a two-stage approach with three specially designed components to construct deeper NMT models. |
| Outcome: | The proposed approach improves on WMT14 EnglishGerman and EnglishFrench translation tasks. |
BioT5: Enriching Cross-modal Integration in Biology with Chemical Knowledge and Natural Language Associations (2023.emnlp-main)
Copied to clipboard
| Challenge: | et al., 2022) argue that the current models for drug discovery lack the ability to integrate molecules, proteins, and natural language. |
| Approach: | They propose a framework that integrates biological knowledge with chemical knowledge and natural language associations. |
| Outcome: | The proposed framework shows superior performance across a wide range of tasks. |
Efficient Pretraining Data Selection for Language Models via Multi-Actor Collaboration (2025.acl-long)
Copied to clipboard
Tianyi Bai, Ling Yang, Zhen Hao Wong, Fupeng Sun, Xinlin Zhuang, Jiahui Peng, Chi Zhang, Lijun Wu, Qiu Jiantao, Wentao Zhang, Binhang Yuan, Conghui He
| Challenge: | Efficient data selection is crucial to accelerate the pretraining of language models . limited research has addressed the inherent conflicts between data selection methods . |
| Approach: | They propose a multi-actor collaborative data selection mechanism that prioritizes data based on its specific criterion and updates prioritization rules using the current state of the model. |
| Outcome: | The proposed model accelerates convergence in LM pretraining and achieves an average relative performance gain of 10.5% across multiple language model benchmarks. |
MolXPT: Wrapping Molecules with Text for Generative Pre-training (2023.acl-short)
Copied to clipboard
| Challenge: | Experimental results show that Generative pre-trained Transformers (GPT) have great success in natural language processing. |
| Approach: | They propose a unified language model of text and molecules pre-trained on SMILES wrapped by text. |
| Outcome: | The proposed model outperforms strong baselines of molecular property prediction on MoleculeNet and performs comparably to the best model in text-molecule translation while using less than half of its parameters. |
A Study of Reinforcement Learning for Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Recent studies have shown that reinforcement learning (RL) is an effective approach for improving the performance of neural machine translation systems. |
| Approach: | They propose to leverage reinforcement learning to boost the performance of NMT systems trained with monolingual data. |
| Outcome: | The proposed method achieves competitive results on translation tasks in English-German, Chinese-English and English-English systems. |
Efficient Reasoning for LLMs through Speculative Chain-of-Thought (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for efficient reasoning focus on reducing the number of model parameters or shortening the chain-of-thought length. |
| Approach: | They propose a speculative chain-of-thought (SCoT) method to reduce reasoning latency by accelerating average reasoning speed through large and small model collaboration. |
| Outcome: | The proposed method reduces reasoning latency by 48%66% and 21%49% on GSM8K, MATH, GaoKao, CollegeMath and Olympiad datasets. |
The Data Frontier for Large Language Models: Selection, Synthesis, and Tools (2026.acl-tutorials)
Copied to clipboard
| Challenge: | acquiring and curating high-quality training data remains a significant bottleneck . acquiring such high-quality data is a key challenge for researchers and practitioners . |
| Approach: | This tutorial provides a comprehensive and practical guide to the state-of-the-art in data research directions for LLMs. |
| Outcome: | The tutorial covers methods for curating the most valuable information from vast, noisy datasets and the synthetic data revolution. |
CReSE: Benchmark Data and Automatic Evaluation Framework for Recommending Eligibility Criteria from Clinical Trial Information (2024.findings-eacl)
Copied to clipboard
| Challenge: | Eligibility criteria (EC) are defined as a set of conditions an individual must meet to participate in a clinical trial. |
| Approach: | They propose to recommend EC based on clinical trial information, including trial titles, and introduce an automatic evaluation framework to assess clinical validity of the EC recommendation model. |
| Outcome: | The proposed model outperforms existing language models pre-trained on the biomedical domain in EC clustering. |
R3: End-to-End Reasoning-based Planning for Multi-step Retrosynthesis via Reinforcement Learning (2026.acl-long)
Copied to clipboard
YiFei Wang, Qizhi Pei, Jiangtao Feng, Yuntian Shi, Yi Duan, Lihao Wang, Lei Bai, Lijun Wu, Wei-Ying Ma, Hao Zhou
| Challenge: | Experimental results show that R3 is a superior alternative to traditional search algorithms for multistep retrosynthesis planning. |
| Approach: | They propose a framework that reformulates multistep retrosynthetic planning as a generative reasoning task. |
| Outcome: | The proposed framework achieves state-of-the-art Top-1 accuracy of 43.7% on retrobench . it leverages Large Language Models to reformulate multistep retrosynthesis as a generative reasoning task. |
How to leverage the multimodal EHR data for better medical prediction? (2021.emnlp-main)
Copied to clipboard
| Challenge: | Using deep learning to improve healthcare is challenging due to the complexity of EHR data. |
| Approach: | They propose a method to integrate clinical notes from EHR and combine them with different data to improve prediction performance. |
| Outcome: | The proposed model outperforms the state-of-the-art method without clinical notes on two prediction tasks. |
A Strategic Coordination Framework of Small LMs Matches Large LMs in Data Synthesis (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models suffer from high computational costs and environmental inefficiency . smaller LMs are more accessible and sustainable, but their individual capabilities often fall short . a collaborative framework for small LM combines specialized roles to iterative refinement and quality control . |
| Approach: | They propose a framework that aggregates specialized roles across small LMs to iterative refinement and quality control typically achieved by a single large LM. |
| Outcome: | The proposed framework aggregates specialized roles across small LMs to iterative refinement and quality control typically achieved by large LM. |
Middo: Model-Informed Dynamic Data Optimization for Enhanced LLM Fine-Tuning via Closed-Loop Learning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to improve data quality face limitations in static dataset curation that fail to adapt to evolving model capabilities. |
| Approach: | They propose a self-evolving framework that uses model-aware data selection and context-preserving data refinement to improve LLM performance. |
| Outcome: | The proposed framework improves the quality of seed data and boosts LLM’s performance with improving accuracy by 7.15% on average while maintaining the original dataset scale. |
Machine Translation With Weakly Paired Documents (D19-1)
Copied to clipboard
| Challenge: | Recent studies explore the possibility of unsupervised machine translation with monolingual data only. |
| Approach: | They propose a method to mine bilingual sentences from weakly paired documents . they use word distribution-level alignments to constrain word distributions of two weakly-paired documents. |
| Outcome: | The proposed method outperforms previous results on six translation tasks using weakly paired bilingual documents and a large number of bilingual sentences. |
Beyond Error Propagation in Neural Machine Translation: Characteristics of Language Also Matter (D18-1)
Copied to clipboard
| Challenge: | Neural machine translation suffers from exposure bias and error propagation problem. |
| Approach: | They conduct a series of analyses to deeply understand the accuracy drop problem . they find that the left part of the translated sentence is often better than its right part . |
| Outcome: | The results show that the left part of the translated sentence is often better than its right part in left-to-right decoding models. |
Retrieved Sequence Augmentation for Protein Representation Learning (2024.emnlp-main)
Copied to clipboard
Chang Ma, Haiteng Zhao, Lin Zheng, Jiayi Xin, Qintong Li, Lijun Wu, Zhihong Deng, Yang Lu, Qi Liu, Sheng Wang, Lingpeng Kong
| Challenge: | Using multiple sequence alignments (MSA) to extract evolutionary knowledge is limited. |
| Approach: | They propose to use multiple sequence alignments to augment protein representations . they propose to employ Retrieved Sequence Augmentation to enhance protein representation learning . |
| Outcome: | The proposed method surpasses MSA Transformer by 5% in structural and property prediction tasks while being 373 times faster. |
JANUS: Joint Autoregressive and Non-autoregressive Training with Auxiliary Loss for Sequence Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to train autoregressive and non-autoregressive models only consider relevance of model parameters, ignoring correlations between the two manners. |
| Approach: | They propose a joint autoregressive and non-autoregressive training method using aUxiliary losS to enhance the model performance in both AR and NAR manners simultaneously. |
| Outcome: | The proposed method improves the model performance in both AR and NAR manners and reduces the inference latency. |
MathFusion: Enhancing Mathematical Problem-solving of LLM through Instruction Fusion (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown impressive progress in mathematical problem-solving . current approaches to enhance mathematical reasoning focus on instance-level modifications . |
| Approach: | They propose a framework that enhances mathematical reasoning through cross-problem instruction synthesis. |
| Outcome: | The proposed framework boosts mathematical reasoning by 18.0 points while maintaining high data efficiency. |
Exploiting Monolingual Data at Scale for Neural Machine Translation (D19-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) is a well-known and expensive task. |
| Approach: | They propose a method to use target-side monolingual data for neural machine translation and propose 'synthetic bitext' they propose generating synthetic bitext by translating monolingual into the other domain using models pretrained on genuine bitext. |
| Outcome: | The proposed approach achieves state-of-the-art results on WMT16, WMT17, WTM18 EnglishGerman translations and WTM19 GermanFrench translations. |
UniDrop: A Simple yet Effective Technique to Improve Transformer without Extra Cost (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to improve the performance of natural language processing models are over-parameterized and overfitted. |
| Approach: | They propose an approach to integrate dropout techniques into the training of Transformer models. |
| Outcome: | The proposed approach can achieve 1.5 BLEU improvement on IWSLT14 translation tasks and better accuracy for the classification even using strong pre-trained RoBERTa as backbone. |
MetaLadder: Ascending Mathematical Solution Quality via Analogical-Problem Reasoning Transfer (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Current paradigms generate CoT and answers directly for a given problem, diverging from human problem-solving strategies to some extent. |
| Approach: | They propose a framework that explicitly prompts LLMs to recall and reflect on meta-problems alongside their CoT solutions before addressing the target problem. |
| Outcome: | The proposed framework outperforms standard CoT-based methods on mathematical benchmarks and significantly improves their reasoning accuracy. |
Soft Contextual Data Augmentation for Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Existing methods for enhancing training data are limited in natural language tasks due to text characteristics. |
| Approach: | They propose a data augmentation method that softly augments a randomly chosen word in a sentence by its contextual mixture of multiple related words. |
| Outcome: | The proposed method outperforms baseline methods on small and large scale machine translation datasets. |
BioT5+: Towards Generalized Biological Understanding with IUPAC Integration and Multi-task Tuning (2024.findings-acl)
Copied to clipboard
Qizhi Pei, Lijun Wu, Kaiyuan Gao, Xiaozhuan Liang, Yin Fang, Jinhua Zhu, Shufang Xie, Tao Qin, Rui Yan
| Challenge: | BioT5+ is an extension of the BioT5, but lacked a nuanced understanding of molecular structures. |
| Approach: | They propose a new bio-entity modeling framework, BioT5+, which integrates IUPAC names and molecule data. |
| Outcome: | The proposed model bridges the gap between molecular representations and textual descriptions and improves the grounded reasoning of bio-text and bio-sequences. |