Papers by Bowen Zhou

MARS2: Scaling Multi-Agent Tree Search via Reinforcement Learning for Code Generation (2026.acl-long)

Challenge:	Existing methods for fine-tuning pre-trained large language models in a parameter-efficient manner are gaining traction within the research community.
Approach:	They propose a method of low-rank adaptation that enables dynamic adjustments to the intrinsic rank during the adaptation process.
Outcome:	The proposed approach outperforms the current method with a fixed and unalterable intrinsic rank and a low-rank adaptation process.

Copied to clipboard

Pengfei Li, Shijie Wang, Fangyuan Li, Yikun Fu, Kaifeng Liu, Kaiyan Zhang, Dazhi Zhang, Yuqiang Li, Biqing Qi, Bowen Zhou

Programming Languages in NLP

Challenge:	Existing approaches to reinforcement learning are decoupled from structured search due to limited trajectory diversity.
Approach:	They propose a unified RL framework that integrates multiple agents within a shared tree-structured search environment.
Outcome:	Experiments show that MARS2 improves performance across diverse model combinations and training settings.

Multi-hop Reading Comprehension across Multiple Documents by Reasoning over Heterogeneous Graphs (P19-1)

Copied to clipboard

Ming Tu, Guangtao Wang, Jing Huang, Yun Tang, Xiaodong He, Bowen Zhou

ReviewRL: Towards Automated Scientific Review with RL (2025.emnlp-main)

Challenge:	Existing models to tackle multi-hop reading comprehension (RC) are focusing on a single document or paragraph, but they lack the ability to do reasoning across multiple documents.
Approach:	They propose a heterogeneous document-entity graph with different types of nodes and edges to solve multi-hop RC problem.
Outcome:	The proposed model can do reasoning over the proposed graph with nodes representation initialized with co-attention and self-attention based context encoders.

Copied to clipboard

Sihang Zeng, Kai Tian, Kaiyan Zhang, Yuru Wang, Junqi Gao, Runze Liu, Sa Yang, Jingxuan Li, Xinwei Long, Jiaheng Ma, Biqing Qi, Bowen Zhou

Challenge:	Existing automated review systems struggle with factual accuracy, rating consistency, and analytical depth.
Approach:	They propose a framework for generating comprehensive and factually grounded scientific paper reviews using supervised fine-tuning and reinforcement learning.
Outcome:	The proposed framework outperforms existing methods on ICLR 2025 papers.

SGG: Learning to Select, Guide, and Generate for Keyphrase Generation (2021.naacl-main)

Copied to clipboard

Jing Zhao, Junwei Bao, Yifan Wang, Youzheng Wu, Xiaodong He, Bowen Zhou

Text Generation

Challenge:	Existing keyphrase generation approaches synchronously generate present and absent keyphrases without explicitly distinguishing these two categories.
Approach:	They propose to deal with present and absent keyphrases separately with different mechanisms by using a hierarchical neural network with a pointing-based selector and a selection-guided generator.
Outcome:	The proposed model outperforms baselines on four keyphrase generation tasks and shows extensibility in natural language generation tasks.

I2E: From Image Pixels to Actionable Interactive Environments for Text-Guided Image Editing (2026.acl-long)

Copied to clipboard

Jinghan Yu, Junhao Xiao, Chenyu Zhu, Jiaming Li, Jia Li, HanMing Deng, Xirui Wang, Guoli Jia, Jianjun Li, Xiang Bai, Bowen Zhou, Zhiyuan Ma

CoGenesis: A Framework Collaborating Large and Small Language Models for Secure Context-Aware Instruction Following (2024.acl-long)

Challenge:	Existing text-guided image editing methods rely on end-to-end pixel-level inpainting paradigm . existing models lack such intermediate representations and Reasoning-then-action process .
Approach:	They propose a "Decompose-then-Action" paradigm that revisits image editing as an actionable interaction process within a structured environment.
Outcome:	The proposed paradigm outperforms existing methods in compositional editing tasks.

Copied to clipboard

Kaiyan Zhang, Jianyu Wang, Ermo Hua, Biqing Qi, Ning Ding, Bowen Zhou

MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing (2026.acl-industry)

Challenge:	Large-scale language models (LLMs) are increasingly exposed to private data and are becoming more and more prevalent.
Approach:	They propose a collaborative generation framework that integrates large and small language models to address privacy concerns logically.
Outcome:	The proposed framework combines large and small models to address privacy concerns logically.

Copied to clipboard

Junbo Niu, Zheng Liu, Zhuangcheng Gu, Bin Wang, Linke Ouyang, Zhiyuan Zhao, Tao Chu, Tianyao He, Fan Wu, Qintong Zhang, Zhenjiang Jin, Guang Liang, Rui Zhang, Wenzheng Zhang, Yuan Qu, Zhifei Ren, Yuefeng Sun, Zirui Tang, Boyu Niu, Yuanhong Zheng, Dongsheng Ma, Ziyang Miao, Hejun Dong, Siyi Qian, Junyuan Zhang, Fangdong Wang, Jingzhou Chen, Xiaomeng Zhao, Liqun Wei, Wei Li, Shasha Wang, RuiLiang Xu, Yuanyuan Cao, Lu Chen, Qianqian Wu, Huaiyu Gu, Lindong Lu, Dechen Lin, null Shenguanlin, Xuanhe Zhou, Linfeng Zhang, Yuhang Zang, Xiaoyi Dong, Jiaqi Wang, Bo Zhang, Lei Bai, Pei Chu, Weijia Li, Jiang Wu, Lijun Wu, Zhenxiang Li, Guangyu Wang, Zhongying Tu, Chao Xu, Kai Chen, Bowen Zhou, Dahua Lin, Wentao Zhang, Conghui He

Green & Sustainable NLP

Challenge:	Document images are characterized by higher resolutions, denser content, and more complex structural layouts.
Approach:	They propose a 1.2B-parameter document parsing vision-language model that decouples layout analysis from local content recognition.
Outcome:	The proposed model surpasses general-purpose and domain-specific models on multiple benchmarks while maintaining significantly lower computational overhead.

Maximum Score Routing For Mixture-of-Experts (2025.findings-acl)

Copied to clipboard

Bowen Dong, Yilong Fan, Yutao Sun, Zhenyu Li, Tengyu Pan, Zhou Xun, Jianyong Wang

Green & Sustainable NLP

Challenge:	Traditional mixture-of-experts (MoE) networks impose an expert capacity constraint to ensure GPU-friendly computation.
Approach:	They propose a routing paradigm that dynamically allocates input tokens to top-k experts through differentiable sparse transformations, enabling scalable model capacity while preserving computational efficiency.
Outcome:	The proposed model achieves lower training losses and higher evaluation scores at equivalent FLOPs compared to constrained and unconstrained baselines.

Controllable Preference Optimization: Toward Controllable Multi-Objective Alignment (2024.emnlp-main)

Copied to clipboard

Yiju Guo, Ganqu Cui, Lifan Yuan, Ning Ding, Zexu Sun, Bowen Sun, Huimin Chen, Ruobing Xie, Jie Zhou, Yankai Lin, Zhiyuan Liu, Maosong Sun

Challenge:	Existing algorithms for achieving optimal alignment are mostly unidirectional . a recent study suggests that large language models can be ground with evident preferences .
Approach:	They propose to ground large language models with evident preferences . they propose to use controllable preference optimization to specify different objectives .
Outcome:	The proposed models can provide responses that match various preferences among the ”3H” desiderata.

PaD: Program-aided Distillation Can Teach Small Models Reasoning Better than Chain-of-thought Fine-tuning (2024.naacl-long)

Copied to clipboard

Xuekai Zhu, Biqing Qi, Kaiyan Zhang, Xinwei Long, Zhouhan Lin, Bowen Zhou

ProcessBench: Identifying Process Errors in Mathematical Reasoning (2025.acl-long)

Challenge:	Large language models excel in various tasks, but their huge size and inaccessibility of parameters present challenges for practical deployment.
Approach:	They propose to use CoT data to distill task-specific ability from large language models to smaller models . they use reasoning programs to suppress errors in distilled data and improve distillation quality .
Outcome:	The proposed model outperforms LLMs on arithmetic reasoning, symbolic reasoning, and general ability.

Copied to clipboard

Chujie Zheng, Zhenru Zhang, Beichen Zhang, Runji Lin, Keming Lu, Bowen Yu, Dayiheng Liu, Jingren Zhou, Junyang Lin

RMTBench: Benchmarking LLMs Through Multi-Turn User-Centric Role-Playing (2025.findings-emnlp)

Challenge:	Existing models fail to generalize to more challenging math problems, authors say . existing benchmarks related to assessing language models' reasoning process are limited .
Approach:	They propose a tool to measure language models' ability to identify erroneous steps in reasoning . they use two types of models: process reward models and critic models .
Outcome:	The proposed model outperforms existing models in evaluating language models' reasoning process . the best open-source model has demonstrated the critique capability competitive with the proprietary model .

Copied to clipboard

Hao Xiang, Tianyi Tang, Yang Su, Bowen Yu, An Yang, Fei Huang, Yichang Zhang, Yaojie Lu, Hongyu Lin, Xianpei Han, Jingren Zhou, Junyang Lin, Le Sun

Dialogue Systems & Conversational Agents

Challenge:	Existing benchmarks focus on character-centric approach and fail to reflect real-world applications.
Approach:	RMTBench is a user-centric bilingual role-playing benchmark featuring 80 diverse characters and over 8,000 dialogue rounds.
Outcome:	RMTBench features 80 diverse characters and over 8,000 dialogue rounds.

LongDPO: Unlock Better Long-form Generation Abilities for LLMs via Critique-augmented Stepwise Information (2025.findings-acl)

Copied to clipboard

Bowen Ping, Jiali Zeng, Fandong Meng, Shuo Wang, Jie Zhou, Shanghang Zhang

Challenge:	Recent advances in large language models have improved their capacity to handle long text inputs, but current models still exhibit unsatisfactory performance in long-form generation.
Approach:	They propose a method to enhance long-form text generation through step-level supervision by leveraging Monte Carlo Tree Search to collect stepwise preference pairs and employ a global memory pool to maintain factual accuracy.
Outcome:	The proposed method improves performance on long-form generation benchmarks while maintaining lossless performance on several general benchmarks.

Improving Conversational Recommendation Systems’ Quality with Context-Aware Item Meta-Information (2022.findings-naacl)

Copied to clipboard

Bowen Yang, Cong Han, Yu Li, Lei Zuo, Zhou Yu

K-PLUG: Knowledge-injected Pre-trained Language Model for Natural Language Understanding and Generation in E-Commerce (2021.findings-emnlp)

Challenge:	Existing approaches to integrate the recommendation function and dialog generation function smoothly are lacking.
Approach:	They propose to integrate dialog context for recommendation and dialog generation better using a pre-trained language model and an item metadata encoder to integrate the recommendation and dialogue generation.
Outcome:	The proposed architecture improves the integration of recommendation and dialog generation functions.

Copied to clipboard

Song Xu, Haoran Li, Peng Yuan, Yujia Wang, Youzheng Wu, Xiaodong He, Ying Liu, Bowen Zhou

Fusing Highly Specialized Language Models for Comprehensive Expertise (2025.acl-long)

Challenge:	Existing pre-trained language models are not explicitly aware of domain-specific knowledge, which is essential for downstream tasks in many domains, such as tasks in e-commerce scenarios.
Approach:	They propose a knowledge-injected pre-trained language model that can be transferred to both natural language understanding and generation tasks.
Outcome:	The proposed model significantly outperforms baselines across the board in e-commerce scenarios.

Copied to clipboard

Ning Ding, Yulin Chen, Ganqu Cui, Xingtai Lv, Weilin Zhao, Kaiyan Zhang, Ruobing Xie, Bowen Zhou, Zhiyuan Liu, Maosong Sun

Many Heads Are Better Than One: Improved Scientific Idea Generation by A LLM-Based Multi-Agent System (2025.acl-long)

Challenge:	Existing models that focus on language, programming code, and mathematical symbols are not able to achieve mastery of all three domains simultaneously.
Approach:	They propose to fuse highly-specialized models that are already sufficiently trained on different domains to achieve a highly-specific model.
Outcome:	The proposed model could achieve mastery of the three crucial domains simultaneously.

Copied to clipboard

Haoyang Su, Renqi Chen, Shixiang Tang, Zhenfei Yin, Xinzhe Zheng, Jinzhe Li, Biqing Qi, Qi Wu, Hui Li, Wanli Ouyang, Philip Torr, Bowen Zhou, Nanqing Dong

Trust in Internal or External Knowledge? Generative Multi-Modal Entity Linking with Knowledge Retriever (2024.findings-acl)

Challenge:	Recent AI methods have shown promise in tasks such as hypothesis generation and experimental design, but they fail to replicate the collaborative nature of real-world scientific practices.
Approach:	They propose a virtual scientific system that mimics the collaborative nature of scientific research by organizing a team of agents to generate, evaluate, and refine research ideas.
Outcome:	The proposed system outperforms the state-of-the-art method in producing new scientific ideas and offers valuable insights to guide future research.

Copied to clipboard

Xinwei Long, Jiali Zeng, Fandong Meng, Jie Zhou, Bowen Zhou

On Large Language Models’ Hallucination with Regard to Known Facts (2024.naacl-long)

Challenge:	Existing generative approaches struggle with the knowledge gap between visual entity information and the intrinsic parametric knowledge of LLMs.
Approach:	They propose a knowledge retrieval method that leverages external sources to enhance visual entity information and a prioritization scheme that handles noisy retrieval results.
Outcome:	The proposed method shows improvements of 3.0% to 6.5% across all evaluation metrics compared to baselines.

Copied to clipboard

Che Jiang, Biqing Qi, Xiangyu Hong, Dayuan Fu, Yang Cheng, Fandong Meng, Mo Yu, Bowen Zhou, Jie Zhou

Learning to Decouple Relations: Few-Shot Relation Classification with Entity-Guided Attention and Confusion-Aware Training (2020.coling-main)

Challenge:	Large language models are successful in answering factoid questions but are also prone to hallucination.
Approach:	They propose self-reporting to the model when faced with such limitations.
Outcome:	The proposed classifier can detect hallucinations with an 88% success rate and can be used to answer factoid questions with correct answer knowledge.

Copied to clipboard

Yingyao Wang, Junwei Bao, Guangyi Liu, Youzheng Wu, Xiaodong He, Bowen Zhou, Tiejun Zhao

Information Extraction & Text Mining

Challenge:	Existing few-shot relation classifiers struggle to distinguish them with few annotated instances due to high co-occurrence of some relations .
Approach:	They propose a few-shot relation classification model with two mechanisms to decouple easily-confused relations.
Outcome:	The proposed model achieves comparable and even better results to strong baselines in terms of accuracy.

Intuitive Fine-Tuning: Towards Simplifying Alignment into a Single Process (2025.acl-long)

Copied to clipboard

Ermo Hua, Biqing Qi, Kaiyan Zhang, Kai Tian, Xingtai Lv, Ning Ding, Bowen Zhou

Graph Ensemble Learning over Multiple Dependency Trees for Aspect-level Sentiment Classification (2021.naacl-main)

Challenge:	Supervised Fine-Tuning (SFT) and Preference Optimization (PO) are key processes for aligning Language Models with human preferences post pre-training.
Approach:	They propose to combine Supervised Fine-Tuning and Preference Optimization (PO) with two sub-processes defined at token level within the Markov Decision Process (MDP)
Outcome:	The proposed process performs comparably or even superiorly to SFT and some typical PO methods across several tasks, particularly those requires generation, reasoning, and fact-following abilities.

Copied to clipboard

Xiaochen Hou, Peng Qi, Guangtao Wang, Rex Ying, Jing Huang, Xiaodong He, Bowen Zhou

Sentiment Analysis

Challenge:	Recent work on aspect-level sentiment classification has shown that syntactic information is effective in capturing long-range syntaktic relations that are obscure from the surface form.
Approach:	They propose a graph ensemble technique that integrates syntactic structures with GNNs to better leverage syntaktic information in the face of parsing errors.
Outcome:	The proposed model outperforms models with single dependency tree and beats other models without adding model parameters.

Fundamental Capabilities of Large Language Models and their Applications in Domain Scenarios: A Survey (2024.acl-long)

Copied to clipboard

Jiawei Li, Yizhe Yang, Yu Bai, Xiaofeng Zhou, Yinghao Li, Huashan Sun, Yuhang Liu, Xingpeng Si, Yuhao Ye, Yixiao Wu, 林一冠林一冠, Bin Xu, Ren Bowen, Chong Feng, Yang Gao, Heyan Huang

CMD: a framework for Context-aware Model self-Detoxification (2024.emnlp-main)

Challenge:	Existing benchmark-based evaluations cannot accurately reflect the performance of real-world applications.
Approach:	They propose a reliable strategy for domains to choose more robust LLMs for real-world applications.
Outcome:	The proposed strategy addresses the challenges faced by domains to choose more robust LLMs for real-world applications.

Copied to clipboard

Zecheng Tang, Keyan Zhou, Juntao Li, Yuyang Ding, Pinzheng Wang, Yan Bowen, Renjie Hua, Min Zhang

SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation (2026.findings-acl)

Challenge:	Existing methods of text detoxification fail to achieve a decent balance between effectiveness and generation quality.
Approach:	They propose a text detoxification framework that pays attention to both context and detoxification process.
Outcome:	Experiments on various LLMs show that the proposed framework can yield the best performance compared to baselines.

Copied to clipboard

Shuang Cheng, Yihan Bian, Dawei Liu, Yuhua Jiang, Yihao Liu, Linfeng Zhang, Qian Yao, Zhongbo Tian, Wenhai Wang, Qipeng Guo, Kai Chen, Biqing Qi, Bowen Zhou

Self-Attention Guided Copy Mechanism for Abstractive Summarization (2020.acl-main)

Challenge:	Autoregressive (AR) language models are a dominant paradigm in the field of parallelism and non-causal modeling.
Approach:	They propose a blockwise discrete diffusion model that preserves AR-compatible serving while enabling parallel intra-block generation.
Outcome:	The proposed model achieves theoretical speedups over 5 and wall-clock speedup of 2.3 on H200 GPUs in latency-critical regimes.

Copied to clipboard

Song Xu, Haoran Li, Peng Yuan, Youzheng Wu, Xiaodong He, Bowen Zhou

Scalable Efficient Training of Large Language Models with Low-dimensional Projected Attention (2024.emnlp-main)

Challenge:	Abstractive summarization models have been widely used to extract words from source into summary, but how to ensure that important words in source are copied remains a challenge.
Approach:	They propose a Transformer-based model to enhance copy mechanism by identifying the importance of each source word based on the degree centrality.
Outcome:	The proposed model outperforms baseline methods on CNN/Daily Mail and Gigaword datasets.

Copied to clipboard

Xingtai Lv, Ning Ding, Kaiyan Zhang, Ermo Hua, Ganqu Cui, Bowen Zhou

P-MMEval: A Parallel Multilingual Multitask Benchmark for Consistent Evaluation of LLMs (2025.emnlp-main)

Challenge:	Existing studies have found that low-rank pre-training often compromises effectiveness.
Approach:	They propose to apply low-dimensional module only to the attention layer to improve both effectiveness and efficiency.
Outcome:	The proposed model saves 12.4% time while improving test perplexity and on downstream tasks compared with vanilla Transformer.

Copied to clipboard

Yidan Zhang, Yu Wan, Boyi Deng, Baosong Yang, Hao-Ran Wei, Fei Huang, Bowen Yu, Dayiheng Liu, Junyang Lin, Fei Huang, Jingren Zhou

Multilinguality

Challenge:	Recent advances in large language models showcase varied multilingual capabilities across tasks . previous assessments focused on fundamental natural language processing (NLP) or isolated capability-specific tasks.
Approach:	They propose a multilingual multitask benchmark to assess multilingual capabilities . they use a large-scale benchmark covering fundamental and capability-specialized datasets .
Outcome:	The proposed benchmark compares models and tasks across languages and tasks and examines knowledge transfer from English to other languages.

The Lessons of Developing Process Reward Models in Mathematical Reasoning (2025.findings-acl)

Copied to clipboard

Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jingren Zhou, Junyang Lin

V-GameGym: Visual Game Generation for Code Large Language Models (2026.findings-acl)

Challenge:	a recent study shows that process reward models can make mistakes, leading to wrong conclusions.
Approach:	They propose a consensus filtering mechanism that integrates MC estimation with LLM-as-a-judge to improve model performance and data efficiency.
Outcome:	The proposed model outperforms existing open-source alternatives and provides practical guidelines for future research.

Copied to clipboard

Wei Zhang, Jian Yang, Renshuai Tao, Linzheng Chai, Shuyue Guo, Jiajun Wu, Xiaoming Chen, Ganqu Cui, Ning Ding, Xander Xu, HU Wei, Bowen Zhou

LongWeave: A Long-Form Generation Benchmark Bridging Real-World Relevance and Verifiability (2025.findings-emnlp)

Challenge:	Existing code-related benchmarks focus on single modality rather than visual game development.
Approach:	They propose a multimodal benchmark for evaluating code large language models in visual game generation that integrates a clustering-based curation methodology and a pipeline for visual code synthesis.
Outcome:	The proposed framework assesses code generation and visual game generation using a sandbox environment.

Copied to clipboard

Zikai Xiao, Fei Huang, Jianhong Tu, Jianhui Wei, Wen Ma, Yuxuan Zhou, Jian Wu, Bowen Yu, Zuozhu Liu, Junyang Lin

Challenge:	Existing benchmarks for long-form generation assess real-world queries with hard-to-verify metrics or use synthetic setups that overlook real-life intricacies.
Approach:	They propose a new approach that balances verifiable and real-world assessment with Target-Anchored Evaluation.
Outcome:	The proposed model balances real-world and verifiable assessment with Target-Anchored Evaluation (TAE) it generates queries, textual materials, and anchors based on verifier targets within real-life scenarios .

LongInsightBench: A Comprehensive Benchmark for Evaluating Omni-Modal Models on Human-Centric Long-Video Understanding. (2026.findings-acl)

Copied to clipboard

ZhaoYang Han, Qihan Lin, Hao Liang, Bowen Chen, Zhou Liu, Wentao Zhang

Living in the Moment: Can Large Language Models Grasp Co-Temporal Reasoning? (2024.acl-long)

Challenge:	LongInsightBench is the first benchmark designed to assess models’ ability to understand long videos, with a focus on human language, viewpoints, actions, and other contextual elements.
Approach:	They propose a benchmark to assess models’ ability to understand long videos with a focus on human language, viewpoints, actions, and other contextual elements.
Outcome:	The proposed model excels in three key areas: a) long-duration, human-centric videos; b) diversifying and challenging task scenarios; c) quality assurance pipeline; and d) reliability.

Copied to clipboard

Zhaochen Su, Juntao Li, Jun Zhang, Tong Zhu, Xiaoye Qu, Pan Zhou, Yan Bowen, Yu Cheng, Min Zhang

SDAR-VL: Stable and Efficient Block-wise Diffusion for Vision-Language Understanding (2026.acl-long)

Challenge:	Current temporal reasoning datasets are limited to questions about single or isolated events, falling short in mirroring the realistic temporal characteristics involving concurrent nature and intricate temporal interconnections.
Approach:	They propose a co-temporal Question Answering benchmark that contains four co-time scenarios with 4,748 samples for evaluating the co-timing abilities of large language models.
Outcome:	The proposed benchmarks show that current LLMs struggle on CoTempQA tasks even when enhanced with Chain of Thought methodologies.

Copied to clipboard

Shuang Cheng, Yuhua Jiang, Zineng Zhou, Dawei Liu, Tao Wang, Linfeng Zhang, Biqing Qi, Bowen Zhou

Self-Steering Optimization: Autonomous Preference Optimization for Large Language Models (2025.findings-acl)

Challenge:	Existing block-wise discrete diffusion models lack robust autoregressive (AR) decoders.
Approach:	They propose a block-wise discrete diffusion framework for large-scale vision-language understanding with a progressive beta noise curriculum.
Outcome:	The proposed framework improves training efficiency, convergence stability, and task performance over conventional block diffusion.

Copied to clipboard

Hao Xiang, Bowen Yu, Hongyu Lin, Keming Lu, Yaojie Lu, Xianpei Han, Ben He, Le Sun, Jingren Zhou, Junyang Lin

On the token distance modeling ability of higher RoPE attention dimension (2024.findings-emnlp)

Challenge:	Prior research focused on developing data generation methods, while insufficient attention has been paid to quality control mechanisms and often produces inaccurate and unhelpful data.
Approach:	They propose an algorithm that automatically generates high-quality preference data, eliminating manual annotation requirements.
Outcome:	The proposed algorithm outperforms baselines in human preference alignment and reward optimization.

Copied to clipboard

Xiangyu Hong, Che Jiang, Biqing Qi, Fandong Meng, Mo Yu, Bowen Zhou, Jie Zhou

Representation Learning

Challenge:	Existing work on extending the context length of language models based on Rotary position embedding (RoPE) has shown promising results in capturing longer-range contextual information.
Approach:	They propose to use a hidden dimension of an attention head to investigate its contribution to capturing long-distance dependencies.
Outcome:	The proposed model can capture long-distance dependencies by extending the attention of a particular dimension of an attention head.

MSI-Agent: Incorporating Multi-Scale Insight into Embodied Agents for Superior Planning and Decision-Making (2024.emnlp-main)

Copied to clipboard

Dayuan Fu, Biqing Qi, Yihuai Gao, Che Jiang, Guanting Dong, Bowen Zhou

Challenge:	Insight is a form of long-term memory for an agent but lack of general insight can undermine its effectiveness.
Approach:	They propose an embodied agent that summarises and utilizes insight effectively across different scales and generates task-specific and high-level insight, stores it in a database, and then uses relevant insight from it.
Outcome:	The proposed agent outperforms a similar agent when planning by GPT3.5 and is more robust when faced with domain-shifting scenarios.

On the Faithfulness for E-commerce Product Summarization (2020.coling-main)

Copied to clipboard

Peng Yuan, Haoran Li, Song Xu, Youzheng Wu, Xiaodong He, Bowen Zhou

Text Generation

Challenge:	e-commerce product summarization requires consistency between product attributes and summary . inconsistent product summaries can mislead users and decrease public credibility .
Approach:	They propose a model to generate e-commerce product summaries with product attributes . they encode product attribute table and constrain attribute words to be presented only through copying .
Outcome:	The proposed model significantly improves the faithfulness of e-commerce product summarization tasks.

Predicting Rewards Alongside Tokens: Non-disruptive Parameter Insertion for Efficient Inference Intervention in Large Language Model (2024.emnlp-main)

Copied to clipboard

Chenhan Yuan, Fei Huang, Ru Peng, Keming Lu, Bowen Yu, Chang Zhou, Jingren Zhou

Large Language Models are Superpositions of All Characters: Attaining Arbitrary Role-play via Self-Alignment (2024.acl-long)

Challenge:	Existing approaches to fine tune LLMs produce unsafe responses and unreliable reasoning, but this solution introduces substantial time and space overhead due to the separate models required.
Approach:	They propose to insert extra parameters into transformer architecture to predict calibration signals along with original LLM output.
Outcome:	The proposed model reduces time and space costs while enabling seamless online deployment.

Copied to clipboard

Keming Lu, Bowen Yu, Chang Zhou, Jingren Zhou

Orthogonal Relation Transforms with Graph Context Modeling for Knowledge Graph Embedding (2020.acl-main)

Challenge:	Existing work cheaply emulates LLMs, allowing users to create profiles for their preferred characters.
Approach:	They propose a self-alignment method that encourages an instruction-following LLM to simulate role-play dialogues as a variant of reading comprehension.
Outcome:	The proposed model outperforms open-source role-play benchmarks and the roleplay subset of MT-Bench in multiple parameters.

Copied to clipboard

Yun Tang, Jing Huang, Guangtao Wang, Xiaodong He, Bowen Zhou

Language Models can Evaluate Themselves via Probability Discrepancy (2024.findings-acl)

Challenge:	Existing knowledge graph embeddings have improved the knowledge graph link prediction task, but complex relations such as N-to-1, 1-to-N and N- to-N remain challenging to predict.
Approach:	They propose to extend RotatE from 2D complex domain to high dimensional space with orthogonal transforms to model relations.
Outcome:	The proposed method improves on N-to-1, 1-to-N and N- to-N cases while maintaining the capability for modeling symmetric/anti-symmetric, inverse and compositional relations.

Copied to clipboard

Tingyu Xia, Bowen Yu, Yuan Wu, Yi Chang, Chang Zhou

On the Generation of Medical Dialogs for COVID-19 (2021.acl-short)

Challenge:	Existing evaluation frameworks focus on superficial text differences and fail to align with human judgment.
Approach:	They propose a new method to evaluate the performance of Large Language Models (LLMs) by calculating probability discrepancies between original response generation and revised versions of LLMs.
Outcome:	The proposed method eliminates the need for training an additional evaluation model or relying on external proprietary models such as GPT-4 as a judger.

Copied to clipboard

Meng Zhou, Zechen Li, Bowen Tan, Guangtao Zeng, Wenmian Yang, Xuehai He, Zeqian Ju, Subrato Chakravorty, Shu Chen, Xingyi Yang, Yichen Zhang, Qingyang Wu, Zhou Yu, Kun Xu, Eric Xing, Pengtao Xie

Dialogue Systems & Conversational Agents

Challenge:	under the pandemic of COVID-19, people experiencing COVI D19-related symptoms have a pressing need to consult doctors.
Approach:	They develop a medical dialog system that can provide COVID19-related consultations . they use two dialog datasets containing conversations between doctors and patients .
Outcome:	The proposed system can provide COVID19-related consultations, but is too small compared with general-domain dialog datasets.

Adaptive Test-Time Compute Allocation with Evolving In-Context Demonstrations (2026.findings-acl)

Copied to clipboard

Bowen Zuo, Dongruo Zhou, Yinglun Zhu

Challenge:	Existing approaches to scaling test-time compute rely on static compute allocation or sample from fixed generation distributions.
Approach:	They propose a test-time compute allocation framework that jointly adapts where computation is spent and how generation is performed.
Outcome:	The proposed approach outperforms baselines while consuming less inference-time compute.

RoR: Read-over-Read for Long Document Machine Reading Comprehension (2021.findings-emnlp)

Copied to clipboard

Jing Zhao, Junwei Bao, Yifan Wang, Yongwei Zhou, Youzheng Wu, Xiaodong He, Bowen Zhou

Machine Reading Comprehension

Challenge:	Existing models for machine reading comprehension are limited to individual chunks due to encoding length constraint.
Approach:	They propose a read-over-read method that expands the reading field from chunk to document by predicting regional answers for each chunk.
Outcome:	Extensive experiments on QuAC and TriviaQA show that the proposed model performs well for long document reading.

Multimodal Joint Attribute Prediction and Value Extraction for E-commerce Product (2020.emnlp-main)

Copied to clipboard

Tiangang Zhu, Yue Wang, Haoran Li, Youzheng Wu, Xiaodong He, Bowen Zhou

Learn to Copy from the Copying History: Correlational Copy Network for Abstractive Summarization (2021.emnlp-main)

Challenge:	In the real world, product attribute values are incomplete and vary over time, which hinders practical applications.
Approach:	They propose a multimodal method to jointly predict product attributes and extract values from product images using multimodal product information.
Outcome:	The proposed method can predict product attributes and extract values from product images with the help of product images.

Copied to clipboard

Haoran Li, Song Xu, Peng Yuan, Yujia Wang, Youzheng Wu, Xiaodong He, Bowen Zhou

Text Generation

Challenge:	Existing methods for abstractive summarization use encoder-decoder attention, but this leads to incomplete copying.
Approach:	They propose a copying scheme that takes advantage of prior copying distributions and explicitly encourages the model to copy the input word that is relevant to the previously copied one.
Outcome:	The proposed scheme achieves state-of-the-art on summarization benchmarks . it takes advantage of prior copying distributions and explicitly encourages copying .

SMR: State Memory Replay for Long Sequence Modeling (2024.findings-acl)

Copied to clipboard

Biqing Qi, Junqi Gao, Kaiyan Zhang, Dong Li, Jianxing Liu, Ligang Wu, Bowen Zhou

Challenge:	Existing state space models (SSMs) address non-uniform sampling, but their recursive structures impede efficient SSM computation via convolution.
Approach:	They propose a plug-and-play mechanism to solve the Non-Stable State problem by adjusting input sequences with early memories.
Outcome:	The proposed method overcomes the non-uniform sample processing problem . it can achieve Sampling Step Adaptation (SSA) by adjusting input sequences with early memories.

Enhancing Chat Language Models by Scaling High-quality Instructional Conversations (2023.emnlp-main)

Copied to clipboard

Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Shengding Hu, Zhiyuan Liu, Maosong Sun, Bowen Zhou

LLMBox: A Comprehensive Library for Large Language Models (2024.acl-demos)

Challenge:	a recent study validates the effectiveness of chat language models by fine-tuning instruction data.
Approach:	They propose to use a large-scale dataset of instructional conversations to fine-tune a conversational model on instruction data.
Outcome:	The proposed model outperforms open-source models in key metrics including scale, average length, diversity, coherence, etc.

Copied to clipboard

Tianyi Tang, Hu Yiwen, Bingqian Li, Wenyang Luo, ZiJing Qin, Haoxiang Sun, Jiapeng Wang, Shiyi Xu, Xiaoxue Cheng, Geyang Guo, Han Peng, Bowen Zheng, Yiru Tang, Yingqian Min, Yushuo Chen, Jie Chen, Ranchi Zhao, Luran Ding, Yuhao Wang, Zican Dong, Xia Chunxuan, Junyi Li, Kun Zhou, Xin Zhao, Ji-Rong Wen

CRaSh: Clustering, Removing, and Sharing Enhance Fine-tuning without Full Large Language Model (2023.emnlp-main)

Challenge:	a library to facilitate the development, use, and evaluation of large language models (LLMs) is presented.
Approach:	They propose a unified library to facilitate the development, use and evaluation of large language models (LLMs).
Outcome:	The proposed library is based on extensive experiments in a variety of evaluation settings.

Copied to clipboard

Kaiyan Zhang, Ning Ding, Biqing Qi, Xuekai Zhu, Xinwei Long, Bowen Zhou

Nirvana: A Specialized Generalist Model With Task-Aware Memory Mechanism (2026.acl-long)

Challenge:	Instruction tuning is an effective way of aligning large language models with private instruction data.
Approach:	They propose a training-free strategy to derive improved emulators from LLMs by using Offsite-Tuning (OFT) they propose CRaSh, which transfers transformer blocks between centralized LLM and downstream emulators .
Outcome:	The proposed technique boosts performance of large language models with billions of parameters.

Copied to clipboard

Yuhua Jiang, Shuang Cheng, Yihao Liu, Ermo Hua, Che Jiang, Weigao Sun, Yu Cheng, Feifei Gao, Biqing Qi, Bowen Zhou