Papers by Kai Zhou

36 papers
Capability Salience Vector: Fine-grained Alignment of Loss and Capabilities for Downstream Task Scaling Law (2025.acl-long)

Copied to clipboard

Challenge: Large language models have demonstrated impressive performance across a wide range of tasks, but this achievement comes with the trade-off of significant computational demands.
Approach: They propose a scaling law that decomposes the overall validation loss and assigns different importance weights to tokens to assess a specific meta-capability.
Outcome: The proposed model can predict the loss trending of models across different levels of computation without a gap between validation loss and model's downstream capabilities.
Annotations Are Not All You Need: A Cross-modal Knowledge Transfer Network for Unsupervised Temporal Sentence Grounding (2023.findings-emnlp)

Copied to clipboard

Challenge: Existing work on temporal sentence grounding rely on expensive video-query paired annotations . despite this, there are no ground-truth annotations in the current work .
Approach: They propose to use paired video-query and segment boundary annotations to generate temporal sentence grounding without training.
Outcome: The proposed model outperforms existing unsupervised methods and beats supervised ones on two challenging datasets.
ReviewRL: Towards Automated Scientific Review with RL (2025.emnlp-main)

Copied to clipboard

Challenge: Existing automated review systems struggle with factual accuracy, rating consistency, and analytical depth.
Approach: They propose a framework for generating comprehensive and factually grounded scientific paper reviews using supervised fine-tuning and reinforcement learning.
Outcome: The proposed framework outperforms existing methods on ICLR 2025 papers.
RubricBench: Aligning Model-Generated Rubrics with Human Standards (2026.acl-long)

Copied to clipboard

Challenge: Existing benchmarks lack discriminative complexity and ground-truth rubric annotations required for rigorous evaluation.
Approach: They propose a curated benchmark with 1,147 pairwise comparisons to assess the reliability of rubric-based evaluation.
Outcome: The proposed benchmarks show that they support diverse domains, exhibit discriminative ability, provide high-quality annotations, and include human-authored rubrics.
Multimodal Large Language Models for Text-rich Image Understanding: A Comprehensive Review (2025.findings-acl)

Copied to clipboard

Challenge: Recent advances in vision-language models have unified perception and understanding tasks within Visual Question Answering paradigms.
Approach: They propose to outline timeline, architecture, and pipeline of nearly all TIU MLLMs and review their performance on mainstream benchmarks.
Outcome: The proposed models perform well on mainstream benchmarks and are compared with other models.
QuZO: Quantized Zeroth-Order Fine-Tuning for Large Language Models (2025.emnlp-main)

Copied to clipboard

Challenge: Large Language Models (LLMs) are quantized to lower precision to reduce memory cost and latency in inference.
Approach: They propose a quantized zeroth-order framework for fine-tuning Large Language Models (LLMs) using low-precision forward passes.
Outcome: The proposed method achieves comparable results to first-order methods in FP8 and superior accuracy in INT8 and INT4 training.
MusKGC: A Flexible Multi-source Knowledge Enhancement Framework for Open-World Knowledge Graph Completion (2025.emnlp-main)

Copied to clipboard

Challenge: Open-world knowledge graph completion (KGC) aims to infer novel facts by enriching existing graphs with external knowledge sources while maintaining semantic consistency under the open-world assumption (OWA).
Approach: They propose a multi-source knowledge enhancement framework based on an open-world assumption (OWA) that integrates external knowledge sources and a new evaluation strategy to validate new facts.
Outcome: The proposed model achieves SOTA performance across benchmarks and the evaluation strategy effectively assesses new facts under OWA.
BadWindtunnel: Defending Backdoor in High-noise Simulated Training with Confidence Variance (2025.findings-acl)

Copied to clipboard

Challenge: Current backdoor attack defenders in NLP typically involve data reduction or model pruning, risking losing crucial information.
Approach: They propose a backdoor defender that allows precise control over training conditions to model backdoor learning behavior without affecting the final model.
Outcome: The proposed model reduces the backdoor learning behavior without affecting the final model.
Firewall Routing: Blocking Leads to Better Hybrid Inference for LLMs (2025.emnlp-main)

Copied to clipboard

Challenge: Large language models have significantly enhanced performance across various NLP tasks . high computational costs and latency associated with deploying such models pose bottlenecks .
Approach: They propose a dynamic hybrid inference framework that efficiently selects between a strong and a weak LLM based on the complexity of the query.
Outcome: The proposed method outperforms existing routing strategies by up to 5.29% in APGR . large models often introduce higher latency, making them less suitable for real-time or resource-constrained applications.
Instance-Guided Prompt Learning for Few-Shot Text Matching (2022.findings-emnlp)

Copied to clipboard

Challenge: Few-shot text matching is a more practical technique to determine whether two texts are semantically identical.
Approach: They propose a pluggable prompt learning method for few-shot text matching . they use the semantics of instances to regulate the effects of the gate on the prompt tokens .
Outcome: The proposed method outperforms baselines on MRPC and QQP.
CLUE: A Chinese Language Understanding Evaluation Benchmark (2020.coling-main)

Copied to clipboard

Challenge: Existing language evaluation benchmarks for English are limited to English . lack of such benchmarks makes it difficult to replicate success in other languages .
Approach: They introduce a large-scale Chinese language understanding evaluation benchmark . the benchmark uses a set of current state-of-the-art pre-trained Chinese models .
Outcome: The first large-scale Chinese Language Understanding Evaluation (CLUE) benchmark is released . the benchmark evaluates models across a wide range of tasks on original Chinese text . existing language evaluation benchmarks are mostly limited to English .
Reinforcement Learning on Pre-Training Data (2026.acl-long)

Copied to clipboard

Challenge: Recent progress in large language models is driven by scaling of training compute through pre-training with nexttoken prediction (NTP) or post-training (RL) Pre-training using NTP enables models to acquire extensive knowledge and skills from general data, but it suffers from data inefficiency and catastrophic forgetting in continual learning settings.
Approach: They propose to scale training compute through pre-training with next-token prediction (NTP) or post-training by scaling reinforcement learning (RL) to improve learning from general data.
Outcome: Experiments on multiple benchmarks and models show that the proposed approach improves continual pre-training and provides a strong foundation for post-training on Qwen3-8B-Base.
PsychEval: A Multi-Session and Multi-Therapy Benchmark for High-Realism AI Psychological Counselor (2026.findings-acl)

Copied to clipboard

Challenge: Existing models focus on a single therapy, but complex cases require flexible strategies among various therapies.
Approach: They propose a multi-session, multi-therapy, and highly realistic benchmark . it is designed to address three key challenges: 1) can we train a highly realistic AI counselor? 2) How to systematically evaluate an AI counselor?"
Outcome: The proposed benchmark is annotated with extensive professional skills and includes over 677 meta-skills and 4577 atomic skills.
LADR: Locality-Aware Dynamic Rescue for Efficient Text-to-Image Generation with Diffusion Large Language Models (2026.acl-long)

Copied to clipboard

Challenge: Existing methods for inference are expensive and lack spatial redundancy . Discrete Diffusion Language Models are a promising paradigm for multimodal generation .
Approach: They propose a locality-aware dynamic rescue method that exploits spatial Markov property of images.
Outcome: The proposed method achieves an approximate 4 speedup over baselines on four text-to-image generation benchmarks.
X-LeBench: A Benchmark for Extremely Long Egocentric Video Understanding (2025.findings-emnlp)

Copied to clipboard

Challenge: Existing benchmark datasets focus on short to moderately long videos, leaving a substantial gap in evaluating extensive, ultra-long egocentric video recordings.
Approach: X-LeBench is a benchmark dataset designed to evaluate long egocentric video recordings . it uses a life-logging pipeline to produce realistic, coherent daily plans .
Outcome: X-LeBench is a new benchmark dataset designed to evaluate long-form egocentric video understanding . the approach produces realistic, coherent daily plans aligned with real-world video data .
CIF-Bench: A Chinese Instruction-Following Benchmark for Evaluating the Generalizability of Large Language Models (2024.findings-acl)

Copied to clipboard

Challenge: a recent study shows that large language models have limited generalization in low-resource languages like Chinese.
Approach: They propose to evaluate the zero-shot generalizability of large language models to the Chinese language . they release only half of the dataset publicly, with the remainder kept private .
Outcome: The Chinese Instruction-Following Benchmark evaluates the generalizability of LLMs to the Chinese language.
Intuitive Fine-Tuning: Towards Simplifying Alignment into a Single Process (2025.acl-long)

Copied to clipboard

Challenge: Supervised Fine-Tuning (SFT) and Preference Optimization (PO) are key processes for aligning Language Models with human preferences post pre-training.
Approach: They propose to combine Supervised Fine-Tuning and Preference Optimization (PO) with two sub-processes defined at token level within the Markov Decision Process (MDP)
Outcome: The proposed process performs comparably or even superiorly to SFT and some typical PO methods across several tasks, particularly those requires generation, reasoning, and fact-following abilities.
CAMIEval: Enhancing NLG Evaluation through Multidimensional Comparative Instruction-Following Analysis (2025.naacl-long)

Copied to clipboard

Challenge: Evaluating the quality of texts generated by language models has always been a challenging task in natural language processing (NLP).
Approach: They propose a multidimensional comparative evaluation method based on instruction-following that combines relevance, factuality, and adherence with a concrete Chain-of-Thoughts process to enhance the accuracy of evaluations.
Outcome: The proposed method outperforms existing methods in correlation with human evaluations on two NLG evaluation benchmarks.
SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation (2026.findings-acl)

Copied to clipboard

Challenge: Autoregressive (AR) language models are a dominant paradigm in the field of parallelism and non-causal modeling.
Approach: They propose a blockwise discrete diffusion model that preserves AR-compatible serving while enabling parallel intra-block generation.
Outcome: The proposed model achieves theoretical speedups over 5 and wall-clock speedup of 2.3 on H200 GPUs in latency-critical regimes.
Structure-aware Domain Knowledge Injection for Large Language Models (2025.acl-long)

Copied to clipboard

Challenge: Structure-aware Continual Pre-Training (SCPT) and Structure-Aware Supervised Fine-Tuning (SSFT) are two-stage strategies for knowledge injection and alignment that reduces the training corpus needs to 5% while achieving 100% of traditional knowledge injection performance.
Approach: They propose a method to efficiently transform foundation Large Language Models into domain specialists by using two-stage strategies: Structure-aware Continual Pre-Training and Structure-Aware Supervised Fine-Tuning.
Outcome: The proposed method significantly reduces the training corpus needs to a mere 5% while achieving 100% of traditional knowledge injection performance.
MathBench: Evaluating the Theory and Application Proficiency of LLMs with a Hierarchical Mathematics Benchmark (2024.findings-acl)

Copied to clipboard

Challenge: Recent advances in large language models have showcased significant improvements in mathematics, but traditional benchmarks like GSM8k offer a unidimensional perspective.
Approach: MathBench is a benchmark that rigorously assesses the mathematical capabilities of large language models.
Outcome: MathBench spans a wide range of mathematical disciplines, offering a detailed evaluation of both theoretical understanding and practical problem-solving skills.
Training-Free Test-Time Contrastive Learning for Large Language Models (2026.findings-acl)

Copied to clipboard

Challenge: Existing training-free alternatives to training-based models are static or depend on external guidance.
Approach: They propose a training-free adaptation framework that enables a frozen LLM to improve online by distilling supervision from its own inference experiences.
Outcome: The proposed framework outperforms existing test-time adaptation methods under online evaluation.
Unsupervised Context Rewriting for Open Domain Conversation (D19-1)

Copied to clipboard

Challenge: Existing approaches to model conversation context have drawbacks, such as lack of coreferences and long dependency.
Approach: They propose a context rewriting method which explicitly rewrites the last utterance by considering context history.
Outcome: The proposed method outperforms baselines in terms of rewriting quality, multi-turn response generation, and end-to-end retrieval-based chatbots.
FiNE: Filtering and Improving Noisy Data Elaborately with Large Language Models (2025.naacl-long)

Copied to clipboard

Challenge: Currently, there are two mainstream methods for improving data integrity: data filtering and data augmentation.
Approach: They propose a method to improve data integrity by combining data filtering and data augmentation with LLMs.
Outcome: The proposed method surpasses the open-source chat version on HalluQA by 8.45 on the open source version.
MPPO: Multi Pair-wise Preference Optimization for LLMs with Arbitrary Negative Samples (2025.coling-main)

Copied to clipboard

Challenge: Existing preference optimization methods such as DPO and KTO are inherently derived from PPO, requiring a reference model that adds GPU memory resources and relies heavily on abundant preference data.
Approach: They propose an algorithm that leverages the average likelihood of model responses to fit the reward function and maximizes the utilization of preference data.
Outcome: The proposed algorithm outperforms DPO, ORPO, and SimPO on MT-Bench and Arena-Hard.
LawBench: Benchmarking Legal Knowledge of Large Language Models (2024.emnlp-main)

Copied to clipboard

Challenge: LegalBench evaluated 20 LLMs in 162 legal tasks in 20 countries and jurisdictions.
Approach: They present a comprehensive evaluation of 21 popular Large Language Models and the first comparative analysis of the empirical results.
Outcome: The proposed benchmarks are based on the Bloom’s cognitive taxonomy and are compared to 21 popular LLMs.
ProReason: Multi-Modal Proactive Reasoning with Decoupled Eyesight and Wisdom (2025.emnlp-main)

Copied to clipboard

Challenge: Large vision-language models often prioritize language knowledge over image information on visual reasoning tasks, incurring performance degradation.
Approach: They propose a visual reasoning framework that decouples vision-reasoning capabilities and multi-run proactive perception.
Outcome: The proposed framework outperforms existing models on benchmarks for open-source and closed-source models with 13.2% performance gain.
MM-Doc-R1: Training Agents for Long Document Visual Question Answering through Multi-turn Reinforcement Learning (2026.findings-acl)

Copied to clipboard

Challenge: Existing work on long document visual question answering is based on Retrieval-Augmented Generation (RAG) where textual or visual content is encoded into embeddings and relevance is determined by similarity scores with respect to the original query.
Approach: They propose a framework that employs an agentic, vision-aware workflow to address long document visual question answering through iterative information discovery and synthesis.
Outcome: The proposed framework outperforms existing RL systems by 10.4% on the MMLongbench-Doc benchmark and demonstrates superior training performance over GRPO.
MPBench: A Comprehensive Multimodal Reasoning Benchmark for Process Errors Identification (2025.findings-acl)

Copied to clipboard

Challenge: Existing benchmarks of large language models focus on error detection, neglecting other scenarios like reasoning search.
Approach: et al. propose a multi-task, multimodal benchmark to assess effectiveness of PRMs . step correctness, answers aggregation and reasoning process search are evaluated . ethical principles of MPBench are based on a set of evaluation paradigms based in a text-based benchmark .
Outcome: a new benchmark assesses the effectiveness of large language models (LLMs) in multiple scenarios . it uses three evaluation paradigms to assess the effectiveness and compares them with existing models . a the proposed model improves reasoning accuracy by providing stepwise feedback for multi-step reasoning results .
LAMCL: A Length-aware Momentum Contrastive Learning Framework for Multiscale Machine-Revised Text Detection (2026.acl-long)

Copied to clipboard

Challenge: Recent detection methods struggle to capture fine-grained semantic differences, especially for short texts.
Approach: They propose a framework for machine-revised text detection that integrates two modules to enhance discriminative semantic features.
Outcome: The proposed method outperforms existing detectors in identifying machine-revised text across diverse practical scenarios, tasks, and LLMs.
RCBSF: A Multi-Agent Framework for Automated Contract Revision via Stackelberg Game (2026.findings-acl)

Copied to clipboard

Challenge: despite the adoption of Large Language Models (LLMs), contract revision remains impeded because generic models treat strict legal constraints as mere suggestions.
Approach: They propose a risk-constrained bilevel Stackelberg framework that models high-stakes revision as a strategic interaction rather than an open-ended conversation.
Outcome: The proposed framework achieves state-of-the-art performance with an average RRR of 84.21% and enhanced token efficiency.
Self-Correcting RAG: Enhancing Faithfulness via MMKP Context Selection and NLI-Guided MCTS (2026.findings-acl)

Copied to clipboard

Challenge: Existing approaches to retrieval-augmented generation still face problems with low context utilization and frequent hallucinations.
Approach: They propose a framework that reformulates retrieval and generation as constrained optimization and path planning.
Outcome: The proposed framework significantly improves reasoning accuracy on complex queries while reducing hallucinations.
Is LLM a Reliable Reviewer? A Comprehensive Evaluation of LLM on Automatic Paper Reviewing Tasks (2024.lrec-main)

Copied to clipboard

Challenge: Existing datasets and methods targeting review-related tasks have not thoroughly inspected model's review ability.
Approach: They propose to evaluate GPT-3.5 and GPT-4 on two types of tasks under different settings: the score prediction task and the review generation task.
Outcome: The proposed model can give passable decisions (> 60% accuracy) on single options, but it always makes mistakes.
Scaling Behavior for Large Language Models regarding Numeral Systems: An Example using Pythia (2024.findings-emnlp)

Copied to clipboard

Challenge: Large Language Models (LLMs) are struggling with performing numeric operations accurately.
Approach: They propose to use different numeral systems to scale different numerates in transformer-based large language models.
Outcome: The proposed model is more data-efficient than base 10 and base 10 3 . the model is also more efficient on addition and multiplication .
PruneVid: Visual Token Pruning for Efficient Video Large Language Models (2025.findings-acl)

Copied to clipboard

Challenge: Existing approaches to video token pruning face significant computational challenges due to the redundancy inherent in video data.
Approach: They propose a training-free visual token pruning method that reduces the redundancy inherent in video data and leverages LLMs’ inherent ability to selectively prune visual tokens irrelevant to specific queries.
Outcome: The proposed method can prune over 80% of tokens while maintaining competitive performance when combined with different video LLMs.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations