Papers by Weijie Li
Legal Mathematical Reasoning with LLMs: Procedural Alignment through Two-Stage Reinforcement Learning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing legal mathematical reasoning models lack structured numerical reasoning . existing models perform poorly on LexNum, while LexPam improves both mathematical accuracy and legal coherence. |
| Approach: | They propose a legal mathematical reasoning benchmark LexNum and LexPam to address this problem . LexPam is a two-stage reinforcement learning framework for efficient legal reasoning training. |
| Outcome: | The proposed framework improves mathematical accuracy and legal coherence . it also improves legal cohesion and generalizes effectively across tasks and domains. |
Topology-of-Question-Decomposition: Enhancing Large Language Models with Information Retrieval for Knowledge-Intensive Tasks (2025.coling-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are constrained to chaining immediate reasoning steps and relying solely on parametric knowledge. |
| Approach: | They propose a framework that activates retrieval only when necessary to improve answer accuracy. |
| Outcome: | Experiments show that the proposed framework improves performance in knowledge-intensive tasks. |
Towards Advanced Mathematical Reasoning for LLMs via First-Order Logic Theorem Proving (2025.emnlp-main)
Copied to clipboard
Chuxue Cao, Mengze Li, Juntao Dai, Jinluan Yang, Zijian Zhao, Shengyu Zhang, Weijie Shi, Chengzhong Liu, Sirui Han, Yike Guo
| Challenge: | Large language models (LLMs) have shown promising first-order logic (FOL) reasoning capabilities with applications in various areas, but their effectiveness in complex mathematical reasoning involving multi-step FOL deductions remains under-explored. |
| Approach: | They propose a self-adaptive solution that enhances the Diversity and REAsonability of LLMs’ generation strategies by introducing an Axiom-Driven Strategy Diversification mechanism and a Sub-Proposition Error Feedback to help LLM reflect on and correct their proofs. |
| Outcome: | The proposed model improves diversity and REAsonability of LLMs’ generation strategies by introducing an Axiom-Driven Strategy Diversification mechanism and a Sub-Proposition Error Feedback to help LLM reflect on and correct proofs. |
Beyond High-Entropy Exploration: Correctness-Aware Low-Entropy Segment-Based Advantage Shaping for Reasoning LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent work studies RLVR through token entropy, arguing that high-entropies drive exploration and should receive stronger updates. |
| Approach: | They propose a correctness-aware reinforcement framework that performs fine-grained advantage modulation over low-entropy segments. |
| Outcome: | The proposed framework improves accuracy over strong RL baselines across three backbones and six math benchmarks while maintaining high-entropy exploration. |
Enhancing Tool Learning in Large Language Models with Hierarchical Error Checklists (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have advanced natural language processing, but their effectiveness is often hampered by parameter mis-filling during tool calling. |
| Approach: | They propose a hierarchical tool error checklist framework to diagnose and mitigate tool-calling errors without relying on extensive real-world interactions. |
| Outcome: | The proposed framework improves parameter-filling accuracy and tool-calling success rates compared to baseline methods. |
ReTRE: Benchmarking LLM Transfer Robustness with Structure-Preserving Variants (2026.acl-long)
Copied to clipboard
| Challenge: | Learning transfer theory emphasizes that applying acquired knowledge to novel manifestations is a key signal of deep understanding |
| Approach: | They propose a benchmark that probes transfer robustness along two rewrite levels: Near Transfer and Far Transfer. |
| Outcome: | The proposed benchmark demonstrates that large language models are robust when faced with novel manifestations of the same problem. |
STAR: SQL Guided Pre-Training for Context-dependent Text-to-SQL Parsing (2022.findings-emnlp)
Copied to clipboard
Zefeng Cai, Xiangyu Li, Binyuan Hui, Min Yang, Bowen Li, Binhua Li, Zheng Cao, Weijie Li, Fei Huang, Luo Si, Yongbin Li
| Challenge: | Extensive experiments show that STAR outperforms previous pre-training methods and ranks first on the leaderboard . text-to-SQL parsing aims to translate natural language (NL) questions into executable SQL queries . |
| Approach: | They propose a SQL guided pre-training framework STAR for context-dependent text-to-SQL parsing . they propose two objectives that explore context-dependence of NL utterances and SQL queries . |
| Outcome: | The proposed framework outperforms existing methods on two downstream benchmarks and ranks first on the leaderboard. |
TencentPretrain: A Scalable and Flexible Toolkit for Pre-training Models of Different Modalities (2023.acl-demo)
Copied to clipboard
Zhe Zhao, Yudong Li, Cheng Hou, Jing Zhao, Rong Tian, Weijie Liu, Yiren Chen, Ningyuan Sun, Haoyan Liu, Weiquan Mao, Han Guo, Weigang Gou, Taiqiang Wu, Tao Zhu, Wenhang Shi, Chen Chen, Shan Huang, Sihong Chen, Liqun Liu, Feifei Li, Xiaoshuai Chen, Xingwu Sun, Zhanhui Kang, Xiaoyong Du, Linlin Shen, Kimmo Yan
| Challenge: | Several pre-training models of different modalities are showing a rising trend of homogeneity in their model structures. |
| Approach: | They propose a toolkit that supports pre-training models of different modalities. |
| Outcome: | The proposed toolkit can match the performance of the original implementations on text, vision, and audio benchmarks. |
MedQPA-Gen: Medical Question Proposing and Answering for Report Generation (2026.findings-acl)
Copied to clipboard
Weijie Liang, Xiyue Zhu, Ruike Zhu, Chenhao Li, Cheng Tang, Zhiyu Liu, Zhihua Gong, Shirui Luo, Yudu Li, Volodymyr Kindratenko
| Challenge: | Existing training paradigms fail to explicitly target factual accuracy, resulting in inaccuracies and serious patient safety risks. |
| Approach: | They propose an automatic and generalizable report evaluation technique that uses question proposing and answering to enable controllable, structured reasoning grounded in medical domain knowledge and the factual correctness of the report. |
| Outcome: | The proposed method can improve human preference scores and perform better on downstream tasks. |
Multi-stage Distillation Framework for Cross-Lingual Semantic Similarity Matching (2022.findings-naacl)
Copied to clipboard
Kunbo Ding, Weijie Liu, Yuejian Fang, Zhe Zhao, Qi Ju, Xuefeng Yang, Rong Tian, Zhu Tao, Haoyan Liu, Han Guo, Xingyu Bai, Weiquan Mao, Yudong Li, Weigang Guo, Taiqiang Wu, Ningyuan Sun
| Challenge: | Existing studies have shown that cross-lingual knowledge distillation can improve the performance of pre-trained models for cross-linguistic similarity matching tasks. |
| Approach: | They propose a multi-stage distillation framework for constructing a small-size but high-performance cross-lingual model using contrastive learning, bottleneck, and parameter recurrent strategies. |
| Outcome: | The proposed model can compress the size of XLM-R and MiniLM by more than 50% while the performance is only reduced by about 1%. |
CSL: A Large-scale Chinese Scientific Literature Dataset (2022.coling-1)
Copied to clipboard
| Challenge: | Existing datasets centered around the English language restrict development of Chinese scientific NLP. |
| Approach: | They present a large-scale Chinese scientific literature dataset based on Chinese papers . they use semi-structured data as a natural annotation for many supervised NLP tasks . |
| Outcome: | The proposed dataset can serve as a Chinese corpus and perform many supervised tasks. |
Parameter-efficient Continual Learning Framework in Industrial Real-time Text Classification System (2022.naacl-industry)
Copied to clipboard
Tao Zhu, Zhe Zhao, Weijie Liu, Jiachi Liu, Yiren Chen, Weiquan Mao, Haoyan Liu, Kunbo Ding, Yudong Li, Xuefeng Yang
| Challenge: | Existing continual learning methods use data replay, parameter isolation and regularization to mitigate catastrophic forgetting. |
| Approach: | They propose a parameter-efficient continual learning framework that updates parameters offline and then trains using an online regularization method. |
| Outcome: | The proposed framework reduces catastrophic forgetting and saves the model with the changed parameters instead of all parameters. |
Giving Control Back to Models: Enabling Offensive Language Detection Models to Autonomously Identify and Mitigate Biases (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models often rely on specific words to predict offensive content, compromising model fairness and potentially exacerbates biases against vulnerable and minority groups. |
| Approach: | They propose a bias self-awareness and data self-iteration framework to help models identify and mitigate biases by integrating multiple natural language processing techniques. |
| Outcome: | The proposed framework reduces false positive rate of models in in-distribution and out-of-difference tests, enhances model accuracy and fairness, and shows promising performance improvements on larger datasets. |
LegalReasoner: Step-wised Verification-Correction for Legal Judgment Reasoning (2025.acl-long)
Copied to clipboard
Weijie Shi, Han Zhu, Jiaming Ji, Mengze Li, Jipeng Zhang, Ruiyuan Zhang, Jia Zhu, Jiajie Xu, Sirui Han, Yike Guo
| Challenge: | Existing legal judgment prediction methods struggle with logical errors when conducting complex legal reasoning. |
| Approach: | They propose a method which enhances LJP reliability through step-wise verification and correction of the reasoning process. |
| Outcome: | The proposed model significantly improves concordance with court decisions from 72.37 to 80.27 on LLAMA-3.1-70B. |
Co-Eval: Augmenting LLM-based Evaluation with Machine Metrics (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing LLMs suffer from biases and misalignment due to limited functional understanding and knowledge gaps. |
| Approach: | They introduce a framework that leverages a criteria planner model and optimized machine metrics to enhance the scalability and fairness of LLM-based evaluation. |
| Outcome: | The proposed framework reduces biases and improves alignment with human preferences, with gains of up to 0.324 in Spearman correlation. |
SpARK: An Embarrassingly Simple Sparse Watermarking in LLMs with Enhanced Text Quality (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for detecting and monitoring generated text face a trade-off between the quality of the generated text and the effectiveness of the watermarking process. |
| Approach: | They propose a new type of LLM watermark, Sparse WatermARK, which uses watermarks to a small subset of generated tokens distributed across the text. |
| Outcome: | The proposed method outperforms existing methods in detectability and quality while maintaining generated text quality. |
Making RALM Robust to Irrelevant Contexts via Layer Knowledge Guided Attention (2025.findings-acl)
Copied to clipboard
Weijie Shi, Hao Chen, Jiaming Li, Yao Zhao, Yazhong Zhang, Qijin Chen, Jipeng Zhang, Ruiyuan Zhang, Jia Zhu, Jiajie Xu, Xiaofang Zhou
| Challenge: | Large language models (LLMs) face factual hallucination and knowledge obsolescence when tackling knowledge-intensive tasks. |
| Approach: | They propose a layer-knowledge guided attention method which harnesses the layer-wise knowledge of large language models to optimize per-layer attention on useful passages. |
| Outcome: | The proposed method outperforms existing methods on RALM benchmarks. |