Papers by Zhen Lei
OpenForecast: A Large-Scale Open-Ended Event Forecasting Dataset (2025.coling-main)
Copied to clipboard
| Challenge: | Existing closed-ended event forecasting methods are constrained by a limited answer space. |
| Approach: | They introduce OpenForecast, a large-scale open-ended dataset with three open-ending event forecasting tasks and an automatic LLM-based method for complex events. |
| Outcome: | The proposed method can be used to evaluate the ability of complex event forecasting of large language models. |
Factorized Learning Assisted with Large Language Model for Gloss-free Sign Language Translation (2024.lrec-main)
Copied to clipboard
| Challenge: | Previous Sign Language Translation methods have relied on gloss annotations to improve performance, but labeling high-quality glosses is labor-intensive and inefficient. |
| Approach: | They propose to integrate Large Language Model (LLM) into SLT by factorizing learning into two stages to improve the learning curve. |
| Outcome: | The proposed approach improves on three SLT datasets conducted under the gloss-free setting. |
DCP: Dual-Cue Pruning for Efficient Large Vision-Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing pruning methods for large vision language models use visual tokens to prune . existing methods fail to balance efficiency and semantic alignment due to large number of visual token. |
| Approach: | They propose a cross-modal pruning framework that considers textual semantics and visual self-attention to combine them to achieve efficient inference acceleration. |
| Outcome: | The proposed pruning framework can retain only 25% of the visual tokens, with a minimal performance degradation of only 0.063% on LLaVA-1.5-13B. |
DVMap: Fine-Grained Pluralistic Value Alignment via High-Consensus Demographic-Value Mapping (2026.acl-long)
Copied to clipboard
| Challenge: | Current Large Language Models (LLMs) rely on coarse-grained national labels for pluralistic value alignment. |
| Approach: | They propose a framework for fine-grained pluralistic value alignment using demographic constraints. |
| Outcome: | The proposed framework can identify groups with predictable, high-consensus value preference . it achieves 48.6% accuracy, surpassing open-source LLM DeepSeek-v3.2 . |
From Curated Data to Scalable Models: Continual Pre-training of Dense and MoE Large Language Models for Tibetan (2026.acl-long)
Copied to clipboard
Lei Yang, Leiyu Pan, Bojian Xiong, Renren Jin, Shaowei Zhang, Yue Chen, Ling Shi, Jiang Zhou, Junru Wu, Zhen Wang, Jianxiang Peng, Juesi Xiao, Tianyu Dong, Zhuowen Han, Zhuo Chen, Yuqi Ren, Deyi Xiong
| Challenge: | Large language models have achieved remarkable success across a wide range of tasks, yet their performance remains heavily biased toward high-resource languages. |
| Approach: | They propose a pipeline for advancing Tibetan language modeling through multilingual continual pre-training with Tibetan, Chinese, and English. |
| Outcome: | The proposed model outperforms open-source and Tibetan-focused models on diverse tasks. |
Learning from Diverse Reasoning Paths with Routing and Collaboration (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies suggest that the reasoning abilities of large language models (LLMs) grows with model size and pre-training data. |
| Approach: | They propose to combine quality filtering, conditional routing, and cooperative peer teaching to transfer knowledge from powerful teacher models to compact and transparent students. |
| Outcome: | Experiments show that QR-Distill is superior to traditional methods. |
LLMRouterBench: A Massive Benchmark and Unified Framework for LLM Routing (2026.findings-acl)
Copied to clipboard
Hao Li, Yiqun Zhang, Zhaoyan Guo, Chenxu Wang, Shengji Tang, Qiaosheng Zhang, Yang Chen, Biqing Qi, Peng Ye, Lei Bai, Zhen Wang, Shuyue Hu
| Challenge: | Large language model (LLM) routing assigns each query to the best suitable model from an ensemble. |
| Approach: | They introduce a large-scale benchmark and unified framework for LLM routing . they find that many routing methods exhibit similar performance under unified evaluation . |
| Outcome: | The proposed benchmark provides comprehensive metrics for both performance-oriented and performance-cost trade-off routing. |
CausalGaze: Unveiling Hallucinations via Counterfactual Graph Intervention in Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing classification-based methods capture noise and spurious correlations while overlooking the underlying causal mechanisms. |
| Approach: | They propose a hallucination detection framework based on structural causal models that captures static and passive signals from internal states and employs counterfactual interventions to disentangle causal reasoning paths from incidental noise. |
| Outcome: | Experiments on four datasets and three widely used LLMs show that the proposed framework improves AUROC and interpretability. |
OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems (2024.acl-long)
Copied to clipboard
Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, Maosong Sun
| Challenge: | Large Language Models (LLMs) and Large Multimodal Models have exceeded general human capabilities in various tasks. |
| Approach: | They present an Olympiad-level bilingual multimodal scientific benchmark featuring 8,476 problems from Olympiad level mathematics and physics competitions. |
| Outcome: | The best performing model, GPT-4V, attains an average score of 17.97% on OlympiadBench, with a mere 10.74% in physics, highlighting the benchmark rigor and the intricacy of physical reasoning. |
Beyond Inherent Cognition Biases in LLM-Based Event Forecasting: A Multi-Cognition Agentic Framework (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models exhibit human-like cognitive biases in event forecasting . a human-curated dataset reveals significant cognitive bias in LLMs . |
| Approach: | They propose a human-curated dataset to explore LLMs' cognitive biases . they leverage LLM participants to act as multi-cognition event participants . |
| Outcome: | The proposed framework alleviates cognitive biases in LLMs and offers diverse perspectives. |
LLaSE-G1: Incentivizing Generalization Capability for LLaMA-based Speech Enhancement (2025.acl-long)
Copied to clipboard
Boyi Kang, Xinfa Zhu, Zihan Zhang, Zhen Ye, Mingshuai Liu, Ziqian Wang, Yike Zhu, Guobin Ma, Jun Chen, Longshuai Xiao, Chao Weng, Wei Xue, Lei Xie
| Challenge: | Recent advances in language models have demonstrated strong capabilities in semantic understanding and contextual modeling. |
| Approach: | They propose a LLaMA-based language model that incentivizes generalization capabilities for speech enhancement. |
| Outcome: | The proposed language model outperforms prior task-specific discriminative and generative models in acoustic enhancement tasks. |
EasyInstruct: An Easy-to-use Instruction Processing Framework for Large Language Models (2024.acl-demos)
Copied to clipboard
Yixin Ou, Ningyu Zhang, Honghao Gui, Ziwen Xu, Shuofei Qiao, Runnan Fang, Lei Li, Zhen Bi, Guozhou Zheng, Huajun Chen
| Challenge: | Large Language Models (LLMs) have improved performance across tasks and domains . instruction tuning is a crucial technique to enhance the capabilities of LLMs - but there is no standard open-source instruction processing framework available for the community . |
| Approach: | They propose an open-source instruction tuning framework for Large Language Models that modularizes instruction generation, selection, prompting and their combination and interaction. |
| Outcome: | The proposed framework is open-source and available on Github. |
UNO Arena for Evaluating Sequential Decision-Making Capability of Large Language Models (2024.emnlp-main)
Copied to clipboard
Zhanyue Qin, Haochuan Wang, Deyuan Liu, Ziyang Song, Cunhang Fan, Zhao Lv, Jinlin Wu, Zhen Lei, Zhiying Tu, Dianhui Chu, Xiaoyan Yu, Dianbo Sui
| Challenge: | Existing LLMs demonstrate powerful capabilities between tasks, but can they make sequential decisions? |
| Approach: | They propose to evaluate sequential decision-making capability of large language models (LLMs) using novel metrics based Monte Carlo methods. |
| Outcome: | The proposed benchmark improves sequential decision-making performance compared to the vanilla LLM player. |
Benchmarking the Fine-Grained Discriminability in Image-Text Retrieval via Controlled Contrastive Differences (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing cross-modal image-text retrieval models often retrieve samples with inconsistent details. |
| Approach: | They propose two fine-grained image-text retrieval benchmarks that incorporate extensive contrastive samples with one controlled contrastive difference from its anchor. |
| Outcome: | Extensive experiments show that contrastive samples can significantly degrade retrieval performance. |
CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark (2022.acl-long)
Copied to clipboard
Ningyu Zhang, Mosha Chen, Zhen Bi, Xiaozhuan Liang, Lei Li, Xin Shang, Kangping Yin, Chuanqi Tan, Jian Xu, Fei Huang, Luo Si, Yuan Ni, Guotong Xie, Zhifang Sui, Baobao Chang, Hui Zong, Zheng Yuan, Linfeng Li, Jun Yan, Hongying Zan, Kunli Zhang, Buzhou Tang, Qingcai Chen
| Challenge: | a new benchmark for biomedical language understanding is being developed in Chinese . most benchmarks are limited to English, which makes it difficult to replicate success in other languages. |
| Approach: | They propose to use Chinese biomedical language understanding evaluation benchmarks to evaluate Chinese models. |
| Outcome: | The proposed benchmarks show that the current models perform worse than the human ceiling. |
ASCM: An Answer Space Clustered Prompting Method without Answer Engineering (2022.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models have shown a great impact on NLP tasks. |
| Approach: | They propose an answer space clustered prompting model and a synonym initialization method that automatically categorizes all answer tokens in a semantic-clustered embedding space. |
| Outcome: | Experiments show that the proposed method outperforms existing state-of-the-art methods in few-shot settings. |