Papers by Zhiyuan Zhao

Seq1F1B: Efficient Sequence-Level Pipeline Parallelism for Large Language Model Training (2025.naacl-long)

Challenge:	Prompt-learning is a new paradigm in natural language processing, adapting pre-trained language models to cloze-style prediction, autoregressive modeling, or sequence to sequence generation.
Approach:	They propose a framework for prompt-learning that integrates pre-trained language models with a unified framework.
Outcome:	The proposed framework is easy to use and flexible enough to integrate with other frameworks.

Copied to clipboard

Sun Ao, Weilin Zhao, Xu Han, Cheng Yang, Xinrong Zhang, Zhiyuan Liu, Chuan Shi, Maosong Sun

MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing (2026.acl-industry)

Challenge:	Current PP methods face severe bottlenecks, including pipeline bubbles and memory footprint.
Approach:	They propose a sequence-level one-forward-one-backward (1F1B) PP method for training LLMs on long sequences with high throughput and memory efficiency.
Outcome:	The proposed method achieves 1.14X training throughput with half memory footprint compared to baseline methods . it trains an LLM with 30B parameters on sequences up to 64k tokens using 64X NVIDIA A100 GPUs .

Copied to clipboard

Junbo Niu, Zheng Liu, Zhuangcheng Gu, Bin Wang, Linke Ouyang, Zhiyuan Zhao, Tao Chu, Tianyao He, Fan Wu, Qintong Zhang, Zhenjiang Jin, Guang Liang, Rui Zhang, Wenzheng Zhang, Yuan Qu, Zhifei Ren, Yuefeng Sun, Zirui Tang, Boyu Niu, Yuanhong Zheng, Dongsheng Ma, Ziyang Miao, Hejun Dong, Siyi Qian, Junyuan Zhang, Fangdong Wang, Jingzhou Chen, Xiaomeng Zhao, Liqun Wei, Wei Li, Shasha Wang, RuiLiang Xu, Yuanyuan Cao, Lu Chen, Qianqian Wu, Huaiyu Gu, Lindong Lu, Dechen Lin, null Shenguanlin, Xuanhe Zhou, Linfeng Zhang, Yuhang Zang, Xiaoyi Dong, Jiaqi Wang, Bo Zhang, Lei Bai, Pei Chu, Weijia Li, Jiang Wu, Lijun Wu, Zhenxiang Li, Guangyu Wang, Zhongying Tu, Chao Xu, Kai Chen, Bowen Zhou, Dahua Lin, Wentao Zhang, Conghui He

Green & Sustainable NLP

Challenge:	Document images are characterized by higher resolutions, denser content, and more complex structural layouts.
Approach:	They propose a 1.2B-parameter document parsing vision-language model that decouples layout analysis from local content recognition.
Outcome:	The proposed model surpasses general-purpose and domain-specific models on multiple benchmarks while maintaining significantly lower computational overhead.

OpenDelta: A Plug-and-play Library for Parameter-efficient Adaptation of Pre-trained Models (2023.acl-demo)

Copied to clipboard

Shengding Hu, Ning Ding, Weilin Zhao, Xingtai Lv, Zhen Zhang, Zhiyuan Liu, Maosong Sun

Efficiency-Effectiveness Reranking FLOPs for LLM-based Rerankers (2025.emnlp-industry)

Challenge:	Existing implementations that modify the code of the backbone PTMs and hard-code specific delta tuning methods for each PTM have limited the practicality and flexibility of delta tuning.
Approach:	They propose an open-source library that provides a plug-and-play implementation of delta tuning methods for pre-trained models.
Outcome:	The proposed methods eliminate the need to modify the backbone PTMs’ code, making OpenDelta compatible with different, even novel PTM.

Copied to clipboard

Zhiyuan Peng, Ting-Ruen Wei, Tingyu Song, Yilun Zhao

ChartCoder: Advancing Multimodal Large Language Model for Chart-to-Code Generation (2025.acl-long)

Challenge:	Existing studies evaluate the efficiency of LLM-based rerankers using proxy metrics such as latency and the number of forward passes.
Approach:	They propose to use a large language model to evaluate the efficiency of LLM-based rerankers . they propose to measure ranking quality and query processing efficiency using an interpretable FLOPs estimator .
Outcome:	The proposed metrics evaluate LLM-based rerankers with different architectures without running any experiments.

Copied to clipboard

Xuanle Zhao, Xianzhen Luo, Qi Shi, Chi Chen, Shuo Wang, Zhiyuan Liu, Maosong Sun

HyKGE: A Hypothesis Knowledge Graph Enhanced RAG Framework for Accurate and Reliable Medical LLMs Responses (2025.acl-long)

Challenge:	Existing open-source MLLMs fail to fully capture dense information embedded in charts . current models still face significant challenges in understanding and analyzing visual tasks such as captioning and question answering.
Approach:	They propose a chart-to-code MLLM which leverages Code LLMs as the language backbone to enhance the executability of the generated code.
Outcome:	The proposed model surpasses existing open-source models on chart-to-code benchmarks with only 7B parameters and provides lossless representations that contain all critical details.

Copied to clipboard

Xinke Jiang, Ruizhe Zhang, Yongxin Xu, Rihong Qiu, Yue Fang, Zhiyuan Wang, Jinyi Tang, Hongxin Ding, Xu Chu, Junfeng Zhao, Yasha Wang

Knowledge Representation

Challenge:	Recent approaches suffer from insufficient and repetitive knowledge retrieval, tedious and time-consuming query parsing, and monotonous knowledge utilization.
Approach:	They propose a retrieval-augmented generation framework which leverages LLMs’ powerful reasoning capacity to compensate for the incompleteness of user queries.
Outcome:	The proposed framework improves the accuracy and reliability of Large Language Models (LLMs) by combining the rich knowledge of LLMs with Hypothesis Outputs.

APB: Accelerating Distributed Long-Context Inference by Passing Compressed Context Blocks across GPUs (2025.acl-long)

Copied to clipboard

Yuxiang Huang, Mingye Li, Xu Han, Chaojun Xiao, Weilin Zhao, Sun Ao, Hao Zhou, Jie Zhou, Zhiyuan Liu, Maosong Sun

LLMs Caught in the Crossfire: Malware Requests and Jailbreak Challenges (2025.acl-long)

Challenge:	Long-context inference is crucial for advancing large language models, but its prefill speed remains a bottleneck.
Approach:	They propose an efficient long-context inference framework that leverages multi-host approximate attention to enhance prefill speed.
Outcome:	The proposed framework achieves speedups of 9.2, 4.2, and 1.6 without any degradation in performance.

Copied to clipboard

Haoyang Li, Huan Gao, Zhiyuan Zhao, Zhiyu Lin, Junyu Gao, Xuelong Li

RepoGenesis: Benchmarking End-to-End Microservice Generation from Readme to Repository (2026.acl-long)

Challenge:	Large Language Models (LLMs) have a high vulnerability to jailbreak attacks that leverage crafted prompts to generate malicious outputs.
Approach:	They propose to use large language models to test their security against jailbreak attacks that leverage crafted prompts to generate malicious outputs.
Outcome:	The proposed model is based on 320 manually crafted malicious code generation requirements, covering 11 jailbreak methods and 29 code functionality categories.

Copied to clipboard

Zhiyuan Peng, Xin Yin, Pu Zhao, Fangkai Yang, Lu Wang, Ran Jia, Xu Chen, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang

Challenge:	Existing benchmarks focus on isolated function/class-level generation, neglecting complete microservice repository generation.
Approach:	They propose a multilingual benchmark for repository-level end-to-end web microservice generation that reflects real-world development workflows.
Outcome:	The benchmark compared 106 repositories across 18 domains and 11 frameworks and 1,258 API endpoints and 2,335 test cases.

ITAKE: Interactive Unstructured Text Annotation and Knowledge Extraction System with LLMs and ModelOps (2024.acl-demos)

Copied to clipboard

Jiahe Song, Hongxin Ding, Zhiyuan Wang, Yongxin Xu, Yasha Wang, Junfeng Zhao

Information Extraction & Text Mining

Challenge:	Unstructured text data contains a large amount of valuable knowledge, but there are many tools that do not meet the needs of actual business.
Approach:	They propose an unstructured text annotation and knowledge extraction system that integrates Large Language Models and ModelOps to improve model supervision and performance.
Outcome:	The proposed system integrates large language models and ModelOps to improve performance in low-resource contexts.

LSTPrompt: Large Language Models as Zero-Shot Time Series Forecasters by Long-Short-Term Prompting (2024.findings-acl)

Copied to clipboard

Haoxin Liu, Zhiyuan Zhao, Jindong Wang, Harshavardhan Kamarthi, B. Aditya Prakash

Fusing Highly Specialized Language Models for Comprehensive Expertise (2025.acl-long)

Challenge:	Existing prompting methods oversimplify time-series forecasting (TSF) time-Series data are ubiquitous across various domains, including public health, finance and energy.
Approach:	They propose a method for prompting off-the-shelf Large Language Models (LLMs) they decompose TSF into short-term and long-term forecasting sub-tasks, tailoring prompts to each .
Outcome:	The proposed approach decomposes TSF into short-term and long-term forecasting sub-tasks, tailoring prompts to each.

Copied to clipboard

Ning Ding, Yulin Chen, Ganqu Cui, Xingtai Lv, Weilin Zhao, Kaiyan Zhang, Ruobing Xie, Bowen Zhou, Zhiyuan Liu, Maosong Sun

Ouroboros: Generating Longer Drafts Phrase by Phrase for Faster Speculative Decoding (2024.emnlp-main)

Challenge:	Existing models that focus on language, programming code, and mathematical symbols are not able to achieve mastery of all three domains simultaneously.
Approach:	They propose to fuse highly-specialized models that are already sufficiently trained on different domains to achieve a highly-specific model.
Outcome:	The proposed model could achieve mastery of the three crucial domains simultaneously.

Copied to clipboard

Weilin Zhao, Yuxiang Huang, Xu Han, Wang Xu, Chaojun Xiao, Xinrong Zhang, Yewei Fang, Kaihuo Zhang, Zhiyuan Liu, Maosong Sun

Challenge:	Speculative decoding is a widely used method that accelerates the generation process of large language models (LLMs) drafting efficiency has become a bottleneck in the final speedup of speculative drafting, therefore generating longer drafts at less cost can lead to better speedup.
Approach:	They propose a method that uses existing model to drafting and target LLM to verify draft in a low-cost parallel manner.
Outcome:	The proposed method can achieve speedups of up to 2.4 over speculative decoding and 3.9 over vanilla decoding without fine-tuning draft and target models.

DeepNote: Note-Centric Deep Retrieval-Augmented Generation (2025.findings-emnlp)

Copied to clipboard

Ruobing Wang, Qingfei Zhao, Yukun Yan, Daren Zha, Yuxuan Chen, Shi Yu, Zhenghao Liu, Yixuan Wang, Shuo Wang, Xu Han, Zhiyuan Liu, Maosong Sun

Information Retrieval

Challenge:	. - (EN)
Approach:	. - (EN)
Outcome:	. - (EN)

Comprehensive Benchmarking of Long-Form Speech Generation in Diverse Scenarios (2026.findings-acl)

Copied to clipboard

Changhao Pan, Rui Yang, Han Wang, Zhuan Zhou, Xuming He, Wenxiang Guo, Ziyue Jiang, Ruiqi Li, Yu Zhang, Chenyuhao Wen, Ke Lei, Xiang Yin, Jingyu Lu, Zhiyuan Zhu, Zhou Zhao

BMInf: An Efficient Toolkit for Big Model Inference and Tuning (2022.acl-demo)

Challenge:	Existing evaluation benchmarks for long-form speech are limited to limited domains, creating a significant gap with the diverse downstream applications.
Approach:	They propose a benchmark that decomposes "long-form speech quality" into specific, disentangled dimensions.
Outcome:	The proposed benchmark decomposes “long-form speech quality” into specific, disentangled dimensions.

Copied to clipboard

Xu Han, Guoyang Zeng, Weilin Zhao, Zhiyuan Liu, Zhengyan Zhang, Jie Zhou, Jun Zhang, Jia Chao, Maosong Sun

Green & Sustainable NLP

Challenge:	Recent years, pre-trained language models (PLMs) have achieved promising results on various NLP tasks.
Approach:	They propose an open-source toolkit for big model inference and tuning which can support big model tuning at extremely low computation cost.
Outcome:	The proposed toolkit can support big model inference and tuning at extremely low computation cost.

APB-V: Accelerating Long-Video Understanding via Sequence-Parallelism-aware Approximate Attention (2026.acl-long)

Copied to clipboard

Yuxiang Huang, Mingye Li, Xu Han, Chaojun Xiao, Weilin Zhao, Ao Sun, Ziqi Yuan, Hao Zhou, Fandong Meng, Zhiyuan Liu

FR-Spec: Accelerating Large-Vocabulary Language Models via Frequency-Ranked Speculative Sampling (2025.acl-long)

Challenge:	Existing methods for long-video inference use compression or sparse attention . existing methods restrict LMMs from handling longer, more complex videos .
Approach:	They propose a sequence-parallel framework with optimized attention that accelerates long-video inference across multiple GPUs.
Outcome:	The proposed framework delivers speedups of 12.72x, 1.70x, and 1.18x over FlashAttn, ZigZagRing, and APB without significant performance loss.

Copied to clipboard

Weilin Zhao, Tengyu Pan, Xu Han, Yudi Zhang, Sun Ao, Yuxiang Huang, Kaihuo Zhang, Weilun Zhao, Yuxuan Li, Jie Zhou, Hao Zhou, Jianyong Wang, Maosong Sun, Zhiyuan Liu

AutoReproduce: Automatic AI Experiment Reproduction with Paper Lineage (2026.acl-long)

Challenge:	Speculative sampling is an efficient way to accelerate the auto-regressive generation process of large language models.
Approach:	They propose a frequency-ranked speculative sampling framework that optimizes draft candidate selection through vocabulary space compression.
Outcome:	Experiments show that FR-Spec reduces LM Head computation overhead by 75% while ensuring the equivalence of the final output distribution.

Copied to clipboard

Xuanle Zhao, Zilin Sang, Yuxuan Li, Qi Shi, Weilun Zhao, Shuo Wang, Duzhen Zhang, Xu Han, Zhiyuan Liu, Maosong Sun

Challenge:	Efficient reproduction of research papers requires deep domain expertise.
Approach:	They propose a framework that systematically mines implicit knowledge from the cited literature to reproduce experimental code in a complete, end-to-end manner.
Outcome:	The proposed framework surpasses baselines across all metrics and reproduces experimental code in a complete, end-to-end manner.

WebUIBench: A Comprehensive Benchmark for Evaluating Multimodal Large Language Models in WebUI-to-Code (2025.findings-acl)

Copied to clipboard

Zhiyu Lin, Zhengda Zhou, Zhiyuan Zhao, Tianrui Wan, Yilun Ma, Junyu Gao, Xuelong Li

A Query-Response Framework for Whole-Page Complex-Layout Document Image Translation with Relevant Regional Concentration (2025.findings-acl)

Challenge:	Existing benchmarks for large language models focus on webpage generation outcomes.
Approach:	They propose a multi-view evaluation framework to evaluate MLLMs in four key areas: WebUI Perception, HTML Programming, WebUI-HTML Understanding, and WebUI to code.
Outcome:	The proposed framework evaluates MLLMs in four key areas: WebUI Perception, HTML Programming, WebUI-HTML Understanding, and WebUI to code.

Copied to clipboard

Zhiyang Zhang, Yaping Zhang, Yupu Liang, Zhiyuan Chen, Lu Xiang, Yang Zhao, Yu Zhou, Chengqing Zong

Machine Translation

Challenge:	Existing methods for document image translation rely on the vanilla encoder-decoder paradigm . a novel dynamic aggregation mechanism is designed to enhance the text semantics in query features toward translation.
Approach:	They propose a Query-Response DIT framework that reformulates the DIT task into a parallel response/translation process of multiple queries.
Outcome:	The proposed framework improves translation quality on four translation directions on three benchmarks.

LoRE: Enhancing Search Relevance with Progressive Chain-of-Thought and Preference Alignment (2026.findings-acl)

Copied to clipboard

Chenji Lu, Zhuo Chen, Hui Zhao, Zhiyuan Zeng, Gang Zhao, Junjie Ren, null Lihaoran, Songyan Liu, Pengjie Wang, Chuan Yu, Jian Xu, Bo Zheng

Reasoning

Challenge:	E-commerce search relevance is a critical component of retrieval systems.
Approach:	They propose a large-generative model for search relevance that trains reasoning knowledge, multi-modal understanding and rule awareness into three core competencies.
Outcome:	The proposed model outperforms GPT-5 in Macro-F1 and achieves 27% online gain.

BMCook: A Task-agnostic Compression Toolkit for Big Models (2022.emnlp-demos)

Copied to clipboard

Zhengyan Zhang, Baitao Gong, Yingfa Chen, Xu Han, Guoyang Zeng, Weilin Zhao, Yanxu Chen, Zhiyuan Liu, Maosong Sun

ReaGeo: Reasoning-Enhanced End-to-End Geocoding with LLMs (2026.findings-acl)

Challenge:	Existing efforts to compress medium-sized models for specific tasks have limited results.
Approach:	They propose a task-agnostic compression toolkit for big models that implements quantization, pruning, distillation and MoEfication methods.
Outcome:	The proposed tool improves performance on a model with 3 billion parameters by 12x . it also outperforms the original model on three typical NLP benchmarks.

Copied to clipboard

Jian Cui, Zhiyuan Ren, Desheng Weng, Yongqi Zhao, Gong Wenbin, Yu Lei, Zhenning Dong

Reasoning

Challenge:	Existing methods rely on text retrieval and geographic knowledge bases to generate coordinates, and they are prone to error propagation and dependency on structured knowledge bases.
Approach:	They propose to use large language models to convert geographic coordinates into geohash sequences and introduce a Chain-of-Thought mechanism to enhance the model’s reasoning over spatial relationships.
Outcome:	The proposed framework can handle explicit address queries in single-point predictions and effectively resolve vague relative location queries.

GUICourse: From General Vision Language Model to Versatile GUI Agent (2025.acl-long)

Copied to clipboard

Wentong Chen, Junbo Cui, Jinyi Hu, Yujia Qin, Junjie Fang, Yue Zhao, Chongyi Wang, Jun Liu, Guirong Chen, Yupeng Huo, Yuan Yao, Yankai Lin, Zhiyuan Liu, Maosong Sun

Visual Data in NLP

Challenge:	Graphical User Interfaces (GUIs) are a pivotal medium for human-computer interaction.
Approach:	They propose a series of datasets for training visual-based GUI agents using general VLMs.
Outcome:	The proposed GUICourse datasets show that even a small-sized GUI agent performs better on GUI tasks.

Integrating Representation Subspace Mapping with Unimodal Auxiliary Loss for Attention-based Multimodal Emotion Recognition (2024.lrec-main)

Copied to clipboard

Xulong Du, Xingnan Zhang, Dandan Wang, Yingying Xu, Zhiyuan Wu, Shiqing Zhang, Xiaoming Zhao, Jun Yu, Liangliang Lou

Sentiment Analysis

Challenge:	Existing methods to identify emotions rely on a large modality gap in their representations .
Approach:	They propose a representation subspace mapping module that maps each modality into two distinct subspaces and a cross-modality attention module that leverages auxiliary loss to remove the noise unrelated to emotion classification.
Outcome:	The proposed approach achieves superior performance to state-of-the-art MER methods on the IEMOCAP and MSP-Improv datasets.

DecorateLM: Data Engineering through Corpus Rating, Tagging, and Editing with Language Models (2024.emnlp-main)

Copied to clipboard

Ranchi Zhao, Zhen Thai, Yifan Zhang, Shengding Hu, Jie Zhou, Yunqi Ba, Jie Cai, Zhiyuan Liu, Maosong Sun

SWE-Mutation: Can LLMs Generate Reliable Test Suites in Software Engineering? (2026.findings-acl)

Challenge:	Large Language Models (LLMs) are pre-trained on vast datasets composed of billions of tokens harvested from diverse text sources.
Approach:	They propose a data engineering method to refine the pretraining corpus through data rating, tagging and editing.
Outcome:	The proposed method improves the quality of the pretraining corpus by enhancing 100 billion tokens of the training corpus.

Copied to clipboard

Yuxuan Sun, Yuze Zhao, Yufeng Wang, Yao Du, Zhiyuan Ma, Jinbo Wang, Mengdi Zhang, Kai Zhang, Zhenya Huang

A Language Model based Evaluator for Sentence Compression (P18-2)

Challenge:	Evaluating software engineering capabilities is a core component of large language models (LLMs).
Approach:	They propose a benchmark to evaluate LLM-generated test suites that introduces mutated solutions that attempt to "fool" them.
Outcome:	The proposed test suites are based on 2,636 mutated variants derived from 800 original instances and include a multilingual subset spanning nine programming languages.

Copied to clipboard

Yang Zhao, Zhiyuan Luo, Akiko Aizawa

Versatile Framework for Song Generation with Prompt-based Control (2025.findings-emnlp)

Challenge:	Existing methods to produce readable sentence compression are based on machine learning or syntactic tree-based approaches.
Approach:	They propose a language-model-based evaluator for deletion-based sentence compression . they propose deleting operations on source sentences to obtain best target compression based on the proposed model .
Outcome:	The proposed model generates more readable compression comparable to strong baselines.

Copied to clipboard

Yu Zhang, Wenxiang Guo, Changhao Pan, Zhiyuan Zhu, Ruiqi Li, Jingyu Lu, Rongjie Huang, Ruiyuan Zhang, Zhiqing Hong, Ziyue Jiang, Zhou Zhao

TCSinger 2: Customizable Multilingual Zero-shot Singing Voice Synthesis (2025.findings-acl)

Challenge:	Existing methods for song generation fail to generate vocals with prompt-based control and proper alignment.
Approach:	VersBand is a multi-task song generation framework for synthesizing high-quality songs with prompt-based control.
Outcome:	Experimental results show that VersBand performs better than baseline models across multiple song generation tasks.

Copied to clipboard

Yu Zhang, Wenxiang Guo, Changhao Pan, Dongyu Yao, Zhiyuan Zhu, Ziyue Jiang, Yuhan Wang, Tao Jin, Zhou Zhao

MoC: Mixtures of Text Chunking Learners for Retrieval-Augmented Generation System (2025.acl-long)

Challenge:	Existing zero-shot singing voice synthesis models depend on phoneme and note boundary annotations, limiting their robustness and producing poor transitions between phonemes and notes.
Approach:	They propose a multi-task multilingual zero-shot SVS model with style transfer and style control based on various prompts.
Outcome:	Experimental results show that TCSinger 2 outperforms baseline models in subjective and objective metrics across multiple related tasks.

Copied to clipboard

Jihao Zhao, Zhiyuan Ji, Zhaoxin Fan, Hanyu Wang, Simin Niu, Bo Tang, Feiyu Xiong, Zhiyu Li

Chunking

Challenge:	Existing methods for text chunking are limited by text chunks and lack of domain-specific knowledge.
Approach:	They propose a dual-metric evaluation method to quantify text chunking quality . they aim to generate a structured list of chunking regular expressions .
Outcome:	The proposed method enables direct quantification of chunking quality . it substantiates the need to integrate LLMs into chunking process .

Beyond the Turn-Based Game: Enabling Real-Time Conversations with Duplex Models (2024.emnlp-main)

Copied to clipboard

Xinrong Zhang, Yingfa Chen, Shengding Hu, Xu Han, Zihang Xu, Yuanwei Xu, Weilin Zhao, Maosong Sun, Zhiyuan Liu

Dialogue Systems & Conversational Agents

Challenge:	Large language models (LLMs) are increasingly permeating daily lives and require real-time interactions that mirror human conversations.
Approach:	They propose to use time-division-multiplexing to process queries and responses pseudo-simultaneously.
Outcome:	The proposed model can listen to users while generating output and adjust to provide instant feedback.

UltraEval: A Lightweight Platform for Flexible and Comprehensive Evaluation for LLMs (2024.acl-demos)

Copied to clipboard

Chaoqun He, Renjie Luo, Shengding Hu, Ranchi Zhao, Jie Zhou, Hanghao Wu, Jiajie Zhang, Xu Han, Zhiyuan Liu, Maosong Sun

LongRecipe: Recipe for Efficient Long Context Generalization in Large Language Models (2025.acl-long)

Challenge:	Existing evaluation platforms are complex and poorly modularized, hindering seamless incorporation into researcher’s workflows.
Approach:	They propose a lightweight evaluation framework characterized by lightweight, comprehensiveness, modularity, and efficiency that integrates models, data, and metrics into a unified evaluation workflow.
Outcome:	The proposed evaluation framework is lightweight, comprehensive, modular, and efficient.

Copied to clipboard

Zhiyuan Hu, Yuliang Liu, Jinman Zhao, Suyuchen Wang, WangYan WangYan, Wei Shen, Qing Gu, Anh Tuan Luu, See-Kiong Ng, Zhiwei Jiang, Bryan Hooi

Fully Hyperbolic Neural Networks (2022.acl-long)

Challenge:	Large language models face significant challenges in handling long-context tasks because of their limited effective context window size during pretraining, which restricts their ability to generalize over extended sequences.
Approach:	They propose a training strategy for extending the context window of LLMs including impactful token analysis, position index transformation, and training optimization strategies.
Outcome:	Experiments on three types of LLMs show that LongRecipe can utilize long sequences while requiring only 30% of the target context window size.

Copied to clipboard

Weize Chen, Xu Han, Yankai Lin, Hexu Zhao, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

Challenge:	Existing hyperbolic neural networks encode features in the hyperbolical space yet formalize most of their operations in the tangent space.
Approach:	They propose a fully hyperbolic framework to build hyperbolical networks based on the Lorentz model by adapting Lorentzer transformations to formalize essential operations of neural networks.
Outcome:	The proposed framework has better performance on four NLP tasks compared with existing hyperbolic models .

Revisit Overconfidence for OOD Detection: Reassigned Contrastive Learning with Adaptive Class-dependent Threshold (2022.naacl-main)

Copied to clipboard

Yanan Wu, Keqing He, Yuanmeng Yan, QiXiang Gao, Zhiyuan Zeng, Fujia Zheng, Lulu Zhao, Huixing Jiang, Wei Wu, Weiran Xu

Progressive Self-Training with Discriminator for Aspect Term Extraction (2021.emnlp-main)

Challenge:	Existing methods for OOD detection are based on labeled in-domain data . detecting out-of-domain (OOD) or unknown intents is challenging .
Approach:	They propose a novel reassigned contrastive learning method to discriminate IND intents for over-confident OOD and an adaptive class-dependent local threshold mechanism to separate similar IND and OOD intents.
Outcome:	The proposed method is effective for both aspects of overconfidence issues.

Copied to clipboard

Qianlong Wang, Zhiyuan Wen, Qin Zhao, Min Yang, Ruifeng Xu

Information Extraction & Text Mining

Challenge:	Existing approaches to extract aspect terms from review sentences are limited due to lack of annotated data.
Approach:	They propose to refine conventional self-training to progressive self-teaching to reduce noise . they use a discriminator to filter the noisy pseudo-labels.
Outcome:	The proposed model outperforms baseline models and achieves state-of-the-art performance on four SemEval datasets.

MALoRA: Mixture of Asymmetric Low-Rank Adaptation for Enhanced Multi-Task Learning (2025.findings-naacl)

Copied to clipboard

Xujia Wang, Haiyan Zhao, Shuo Wang, Hanqing Wang, Zhiyuan Liu

Improving MLLM’s Document Image Machine Translation via Synchronously Self-reviewing Its OCR Proficiency (2025.findings-acl)

Challenge:	Large Language Models (LLMs) can be fine-tuned to new tasks, but in multi-task scenarios, training imbalance and seesaw effect often arise.
Approach:	They propose a flexible fine-tuning framework that leverages asymmetric optimization among LoRA experts to reduce training imbalance and improve performance.
Outcome:	The proposed framework outperforms baseline methods in inter- and intra-task learning scenarios.

Copied to clipboard

Yupu Liang, Yaping Zhang, Zhiyang Zhang, Zhiyuan Chen, Yang Zhao, Lu Xiang, Chengqing Zong, Yu Zhou

Multilinguality

Challenge:	Multimodal Large Language Models (MLLMs) have shown strong performance in document image tasks, especially Optical Character Recognition (OCR). However, they struggle with Document Image Machine Translation (DIMT), which requires handling both cross-modal and cross-lingual challenges.
Approach:	They propose a novel fine-tuning paradigm that allows the model to generate OCR text before producing translation text, which allows it to leverage its strong monolingual OCR ability while learning to translate text across languages.
Outcome:	The proposed model can leverage its strong monolingual OCR ability while learning to translate text across languages.

Large Language Models Are Still Misled by Simple Bias Ensembles (2026.findings-acl)

Copied to clipboard

Zhouhao Sun, Zhiyuan Kan, Xiao Ding, Li Du, Bibo Cai, Yang Zhao, Bing Qin, Ting Liu

Rewarding the Rare: Uniqueness-Aware RL for Creative Problem Solving in LLMs (2026.findings-acl)

Challenge:	Existing benchmarks for large language models are constrained to datasets where each sample is manually injected with only one type of bias.
Approach:	They propose a multi-bias benchmark where each sample contains multiple types of biases.
Outcome:	The proposed benchmark shows that existing LLMs and debiasing methods perform poorly on this benchmark, highlighting the challenge of eliminating compounded biases.

Copied to clipboard

Zhiyuan Hu, Yucheng Wang, Yufei He, Jiaying Wu, Yilun Zhao, See-Kiong Ng, Cynthia Breazeal, Anh Tuan Luu, Hae Won Park, Bryan Hooi

Information Extraction & Text Mining

Challenge:	Reinforcement learning (RL) is a paradigm for post-training large language models, but it suffers from exploration collapse . a new study finds that RL fails to reward correct solutions that exhibit rare high-level strategies .
Approach:	They propose a method that rewards correct solutions that exhibit rare high-level strategies by clustering rollouts according to their high- level solution strategies.
Outcome:	The proposed approach improves pass@k across large sampling budgets and increases area under the pass@K curve (AUC@K) without sacrificing pass@1.

Plug-and-Play Document Modules for Pre-trained Models (2023.acl-long)

Copied to clipboard

Chaojun Xiao, Zhengyan Zhang, Xu Han, Chi-Min Chan, Yankai Lin, Zhiyuan Liu, Xiangyang Li, Zhonghua Li, Zhao Cao, Maosong Sun

Unleashing Low-Bit Inference on Ascend NPUs: A Comprehensive Evaluation of HiFloat Formats (2026.acl-industry)

Challenge:	Large-scale pre-trained models have been widely adopted for document-oriented NLP tasks, such as question answering.
Approach:	They propose to decouple document encoding from downstream tasks by introducing a document plugin into the backbone of a PTM.
Outcome:	The proposed model can encode documents once and for all across different scenarios.

Copied to clipboard

Pengxiang Zhao, Hui-Ling Zhen, Xing Li, Han Bao, Weizhe Lin, Zhiyuan Yang, Yu Zi Wei, Xin Wang, Mingxuan Yuan, Xianzhi Yu, Zhenhua Dong

Green & Sustainable NLP

Challenge:	Low-bit floating-point formats like MXFP and NVFP4 offer new opportunities for precision and efficiency.
Approach:	They evaluate HiFloat (HiF8 and HiF4), a family of floating-point formats tailored for Ascend NPUs.
Outcome:	The proposed formats excel with high-variance data and are compatible with state-of-the-art quantization frameworks.

Learning Diverse Responses with Prefix-Conditioned Supervised Fine-Tuning (2026.acl-long)

Copied to clipboard

Zhiyuan Fan, Guanqiao Chen, Yanyi Huang, Mingkuan Zhao, Dadi Guo, Yi R. Fung