Papers by Chao Chen

Think Wider, Detect Sharper: Reinforced Reference Coverage for Document-Level Self-Contradiction Detection (2025.emnlp-main)

Challenge:	Empathetic response generation attempts to empower dialogue systems to perceive speakers’ emotions and generate empathetic responses accordingly.
Approach:	They propose to combine trait and state emotions for Empathetic Response Model to enable dialogue systems to perceive speakers' emotions and generate empathetic responses accordingly.
Outcome:	The proposed model outperforms state-of-the-art models and generates more empathetic responses.

Copied to clipboard

Yuhao Chen, Yuanjie Lyu, Shuochen Liu, Chao Zhang, Junhui Lv, Tong Xu

Hephaestus: Improving Fundamental Agent Capabilities of Large Language Models through Continual Pre-Training (2025.naacl-long)

Challenge:	Recent approaches to document-level contradiction detection (DSCD) only gain marginal improvement and often introduce inconsistencies across repeated responses.
Approach:	They propose a method that combines supervised fine-tuning and reinforcement learning to enhance document-level contradiction detection (DSCD) they propose to use a task-specific reward function to expand the model’s reasoning scope, boosting both accuracy and consistency.
Outcome:	The proposed method significantly boosts Llama 3.1-8B-Instruct’s accuracy from 38.5% to 51.1%, and consistency from 59.6% to76.2%.

Copied to clipboard

Yuchen Zhuang, Jingfeng Yang, Haoming Jiang, Xin Liu, Kewei Cheng, Sanket Lokegaonkar, Yifan Gao, Qing Ping, Tianyi Liu, Binxuan Huang, Zheng Li, Zhengyang Wang, Pei Chen, Ruijie Wang, Rongzhi Zhang, Nasser Zalmout, Priyanka Nigam, Bing Yin, Chao Zhang

Attention-Enhancing Backdoor Attacks Against BERT-based Models (2023.findings-emnlp)

Challenge:	Existing LLMs often rely on complex prompting or extensive fine-tuning to introduce new capabilities while preserving strong generalizability.
Approach:	They propose a large-scale pre-training corpus to enhance LLM agents' capabilities . they use 103B agent-specific data encompassing 76,537 APIs .
Outcome:	The proposed training corpus outperforms open-source LLMs and commercial LLM agents on three agent benchmarks.

Copied to clipboard

Weimin Lyu, Songzhu Zheng, Lu Pang, Haibin Ling, Chao Chen

Robustness in NLP

Challenge:	Existing textual backdoor attacks focus on generating stealthy triggers or modifying model weights.
Approach:	They propose a Trojan Attention Loss (TAL) which enhances the Trojan behavior by directly manipulating attention patterns.
Outcome:	The proposed method improves the effectiveness of the backdoor attacks on different backbone models and tasks.

MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing (2026.acl-industry)

Copied to clipboard

Junbo Niu, Zheng Liu, Zhuangcheng Gu, Bin Wang, Linke Ouyang, Zhiyuan Zhao, Tao Chu, Tianyao He, Fan Wu, Qintong Zhang, Zhenjiang Jin, Guang Liang, Rui Zhang, Wenzheng Zhang, Yuan Qu, Zhifei Ren, Yuefeng Sun, Zirui Tang, Boyu Niu, Yuanhong Zheng, Dongsheng Ma, Ziyang Miao, Hejun Dong, Siyi Qian, Junyuan Zhang, Fangdong Wang, Jingzhou Chen, Xiaomeng Zhao, Liqun Wei, Wei Li, Shasha Wang, RuiLiang Xu, Yuanyuan Cao, Lu Chen, Qianqian Wu, Huaiyu Gu, Lindong Lu, Dechen Lin, null Shenguanlin, Xuanhe Zhou, Linfeng Zhang, Yuhang Zang, Xiaoyi Dong, Jiaqi Wang, Bo Zhang, Lei Bai, Pei Chu, Weijia Li, Jiang Wu, Lijun Wu, Zhenxiang Li, Guangyu Wang, Zhongying Tu, Chao Xu, Kai Chen, Bowen Zhou, Dahua Lin, Wentao Zhang, Conghui He

An Effective Pronunciation Assessment Approach Leveraging Hierarchical Transformers and Pre-training Strategies (2024.acl-long)

Challenge:	Document images are characterized by higher resolutions, denser content, and more complex structural layouts.
Approach:	They propose a 1.2B-parameter document parsing vision-language model that decouples layout analysis from local content recognition.
Outcome:	The proposed model surpasses general-purpose and domain-specific models on multiple benchmarks while maintaining significantly lower computational overhead.

Copied to clipboard

Bi-Cheng Yan, Jiun-Ting Li, Yi-Cheng Wang, Hsin Wei Wang, Tien-Hong Lo, Yung-Chang Hsu, Wei-Cheng Chao, Berlin Chen

Step Guided Reasoning: Improving Mathematical Reasoning using Guidance Generation and Step Reasoning (2025.emnlp-main)

Challenge:	Existing attempts to quantify a second language learner’s pronunciation proficiency in a target language often sideline the hierarchy of linguistic units and relatedness among the pronunciation aspects.
Approach:	They propose a hierarchical automatic pronunciation assessment method that models the intrinsic structures of an utterance while considering the relatedness among the pronunciation aspects.
Outcome:	The proposed method can be used to quantify a second language learner’s pronunciation proficiency in a target language by providing fine-grained feedback with multiple pronunciation aspect scores at various linguistic levels.

Copied to clipboard

Lang Cao, Yingtian Zou, Chao Peng, Renhong Chen, Wu Ning, Yitong Li

Attention Mechanism with Energy-Friendly Operations (2022.findings-acl)

Challenge:	Existing approaches to improve mathematical reasoning require extensive datasets for training or depend on few-shot methods that compromise computational accuracy.
Approach:	They propose a training-free adaptation framework that efficiently equips general-purpose pre-trained language models with enhanced mathematical reasoning capabilities.
Outcome:	The proposed framework outperforms Qwen2.5-72B-Math-Instruct on MMLU-STEM with a score of 90.9%, compared to 87.3%.

Copied to clipboard

Yu Wan, Baosong Yang, Dayiheng Liu, Rong Xiao, Derek Wong, Haibo Zhang, Boxing Chen, Lidia Chao

Challenge:	Empirical results show that attention mechanism can be improved from the energy consumption aspects.
Approach:	They propose to replace multiplications with either selective operations or additions to reduce energy consumption.
Outcome:	The proposed model achieves competitable accuracy while saving 99% and 66% energy during alignment calculation and the whole attention procedure.

WorkForceAgent-R1: Incentivizing Reasoning Capability in LLM-based Web Agents via Reinforcement Learning (2026.findings-eacl)

Copied to clipboard

Yuchen Zhuang, Di Jin, Jiaao Chen, Wenqi Shi, Hanrui Wang, Chao Zhang

Robustness in NLP

Challenge:	Existing web agents relying on supervised fine-tuning struggle with generalization and robustness due to insufficient reasoning capabilities when handling the inherently dynamic nature of web interactions.
Approach:	They propose a large language model-empowered web agent that trains using a rule-based reinforcement learning framework to enhance single-step reasoning and planning for business-oriented web navigation tasks.
Outcome:	The proposed agent outperforms baseline LLM-based agents on the WorkArena benchmark by 10.26–16.59%.

Med-SRAF: A Multi-Agent Framework for Medical Reasoning via Semantic Routing and Agentic Fusion (2026.findings-acl)

Copied to clipboard

Xiao Li, Zhuo Chen, Jun Xia, Hongxin Xiang, Chao Wang, Wenjie Du, Yang Wang

Hierarchical Attention Generates Better Proofs (2025.acl-long)

Challenge:	Existing RAG methods suffer from a two-part problem: semantic drift and concatenation fallacy . et al.: rapid development of Large Language Models has led to a paradigm shift in artificial intelligence .
Approach:	They propose a multi-agent retrieval augmentation framework guided by medical domain knowledge to address these challenges.
Outcome:	The proposed framework outperforms existing general RAG baselines on five widely used medical benchmarks.

Copied to clipboard

Jianlong Chen, Chao Li, Yang Yuan, Andrew C Yao

ArchiDocGen: Multi-Agent Framework for Expository Document Generation in the Architectural Industry (2025.acl-industry)

Challenge:	Large language models (LLMs) have shown promise in formal theorem proving, but their token-level processing often fails to capture the inherent hierarchical nature of mathematical proofs.
Approach:	They propose a regularization method that aligns LLMs’ attention mechanisms with mathematical reasoning structures and establishes a five-level hierarchy from foundational elements to high-level concepts.
Outcome:	The proposed method improves proof success rates by 2.05% on miniF2F and 1.69% on ProofNet while reducing proof complexity by 23.81% and 16.50% respectively.

Copied to clipboard

Junjie Jiang, Haodong Wu, Yongqi Zhang, Songyue Guo, Bingcen Liu, Caleb Chen Cao, Ruizhe Shao, Chao Guan, Peng Xu, Lei Chen

Text Generation

Challenge:	drafting method statements is labor-intensive and time-consuming . traditional methods involve using static templates filled in manually by engineers .
Approach:	They propose a framework that automates method statement generation by using multi-agent collaboration.
Outcome:	The proposed framework achieves 4.38 ContentScore, excelling in specialization, completeness, organization, and clarity.

From Scenes to Elements: Multi-Granularity Evidence Retrieval for Verifiable Multimodal RAG (2026.findings-acl)

Copied to clipboard

Guanhua Chen, Chuyue Huang, Yutong Yao, Shudong Liu, Xueqing Song, Lidia S. Chao, Derek F. Wong

An Emotional Comfort Framework for Improving User Satisfaction in E-Commerce Customer Service Chatbots (2021.naacl-industry)

Challenge:	Existing multimodal Retrieval-Augmented Generation (RAG) systems retrieve evidence at coarse granularities, making failures unverifiable.
Approach:	They propose a multimodal benchmark that features real-world landmarks with annotations across multiple viewpoints and a framework that treats visual elements as first-class retrieval units through three stages: element-level detection and classification, multi-granularity cross-modal alignment for evidence retrieval, and attribution-constrained generation.
Outcome:	The proposed framework achieves up to 29.2% improvement over six strong baselines for this task.

Copied to clipboard

Shuangyong Song, Chao Wang, Haiqing Chen, Huan Chen

Natural Language Interfaces

Challenge:	E-commerce has grown rapidly over the last several years, and chatbots for intelligent customer service are simultaneously drawing attention.
Approach:	They propose a framework to obtain proper answer to customers’ emotional questions using emotion classification model and text matching.
Outcome:	The proposed framework is very promising on real online systems.

Streamlining the Collaborative Chain of Models into A Single Forward Pass in Generation-Based Tasks (2025.findings-acl)

Copied to clipboard

Yuanjie Lyu, Chao Zhang, Yuhao Chen, Yong Chen, Tong Xu

CERES: Pretraining of Graph-Conditioned Transformer for Semi-Structured Session Data (2022.naacl-main)

Challenge:	Recent advances in "Chain of Models" approach increase resource demands as each model must be deployed separately.
Approach:	They propose a prompt-tuning method that enables models to share hidden states . they modify input and attention masks during training to eliminate redundant forward passes .
Outcome:	Empirical results show that FTHSS matches the performance of traditional model chains while improving inference efficiency.

Copied to clipboard

Rui Feng, Chen Luo, Qingyu Yin, Bing Yin, Tuo Zhao, Chao Zhang

Multi-Stage LLM Fine-Tuning with a Continual Learning Setting (2025.findings-naacl)

Challenge:	Despite advances in self-supervised learning, there is a lack of models that can effectively capture both intra- and intra-item semantics for semi-structured session data.
Approach:	They propose a graph-based transformer model for semi-structured session data that captures both intra- and intra-item semantics.
Outcome:	The proposed model outperforms baselines in three session search and entity linking tasks by up to 9%.

Copied to clipboard

Changhao Guan, Chao Huang, Hongliang Li, You Li, Ning Cheng, Zihe Liu, Yufeng Chen, Jinan Xu, Jian Liu

Traffic-R1: Reinforced LLMs Bring Human-Like Reasoning to Traffic Signal Control Systems (2026.acl-long)

Challenge:	Large language models (LLMs) have made significant progress in knowledge-intensive applications, but they may face a multi-stage continuous learning scenario.
Approach:	They propose a multi-stage continuous learning paradigm that includes a preference-based learning bias to identify potential knowledge conflicts and a self-distillation-based data augmentation strategy to expand and enrich the training corpus.
Outcome:	The proposed learning paradigm achieves a significant improvement in accuracy after 7 stages of fine-tuning compared to previous methods while preserving general knowledge.

Copied to clipboard

Xingchen Zou, Yuhao Yang, Zheng Chen, Xixuan Hao, Yiqi Chen, Chao Huang, Yuxuan Liang

One Pair Suffices: Unlocking Universal Zero-Shot Translation via Cross-Architecture Alignment (2026.acl-long)

Challenge:	Rapid urbanization and surging vehicle ownership intensify congestion . rapid urbanization drives crash rates, slow emergency response, and burden transit-poor communities .
Approach:	They introduce a 3B-parameter foundation model with human-like reasoning for Traffic signal control (TSC) they use reinforcement learning and network communication to convert LLM into a traffic-control model that operates like a human traffic agent.
Outcome:	The proposed model outperforms baselines and training-intensive RL controllers on a simulated traffic environment and reduces queue lengths by more than 5%.

Copied to clipboard

Hao Zong, Cong Hu Yuan, Chao Bei, Wentao Chen, Huan Liu, Kaiyu Huang, Degen Huang

Do LLMs Behave as Claimed? Investigating How LLMs Follow Their Own Claims using Counterfactual Questions (2025.emnlp-main)

Challenge:	Current paradigms for empowering Large Language Models with multilingual capabilities rely heavily on massive instruction tuning.
Approach:	They propose a hybrid cross-alignment approach that fuses a frozen NLLB encoder with a Qwen decoder via a closed-loop dual-adapter architecture.
Outcome:	The proposed model outperforms towerPlus-9B and Aya-101 on language-agnostic projection protocols.

Copied to clipboard

Haochen Shi, Shaobo Li, Guoqing Chao, Xiaoliang Shi, Wentao Chen, Zhenzhou Ji

Enhancing Neural Topic Model with Multi-Level Supervisions from Seed Words (2023.findings-acl)

Challenge:	Existing evaluation frameworks rely on curated datasets that, once public, may be accessed by newer LLMs.
Approach:	They propose a framework that generates counterfactual questions and answers from existing evaluation datasets and uses them to evaluate LLMs.
Outcome:	The proposed evaluation framework reduces the risk of data leakage by allowing the LLMs to respond to counterfactual questions and verify their claims.

Copied to clipboard

Yang Lin, Xin Gao, Xu Chu, Yasha Wang, Junfeng Zhao, Chao Chen

Information Extraction & Text Mining

Challenge:	Existing topic seed words are difficult to incorporate into topic models due to the semantic diversity of natural language.
Approach:	They propose a neural topic model enhanced with supervisions from seed words on word and document levels.
Outcome:	The proposed model outperforms the state-of-the-art seeded topic models in terms of topic quality and classification accuracy.

An Effective Automated Speaking Assessment Approach to Mitigating Data Scarcity and Imbalanced Distribution (2024.findings-naacl)

Copied to clipboard

Tien-Hong Lo, Fu-An Chao, Tzu-i Wu, Yao-Ting Sung, Berlin Chen

ReSel: N-ary Relation Extraction from Scientific Text and Tables by Learning to Retrieve and Select (2022.emnlp-main)

Challenge:	Automated speaking assessment (ASA) typically involves automatic speech recognition (ASR) and hand-crafted feature extraction from the transcript of a learner’s speech.
Approach:	They propose to use metric-based classification and loss re-weighting to model the impact of different SSL-based embedding features on the CEFR score.
Outcome:	The proposed model outperforms baselines on the ICNALE benchmark dataset, achieving a significant improvement of more than 10% in CEFR prediction accuracy.

Copied to clipboard

Yuchen Zhuang, Yinghao Li, Junyang Zhang, Yue Yu, Yingjun Mou, Xiang Chen, Le Song, Chao Zhang

Information Extraction & Text Mining

Challenge:	Our proposed method extracts N-ary relation tuples from scientific articles.
Approach:	They propose a method that decomposes the task into two stages . they propose modal query and modal entity selection . their results show that ReSel outperforms state-of-the-art baselines significantly .
Outcome:	The proposed method outperforms state-of-the-art baselines on three scientific information extraction datasets.

Idea23D: Collaborative LMM Agents Enable 3D Model Generation from Interleaved Multimodal Inputs (2025.coling-main)

Copied to clipboard

Junhao Chen, Xiang Li, Xiaojun Ye, Chao Li, Zhaoxin Fan, Hao Zhao

A Two-Stage Prediction-Aware Contrastive Learning Framework for Multi-Intent NLU (2024.lrec-main)

Challenge:	Existing 3D AIGC methods don’t fully unleash human creativity.
Approach:	They propose a framework that generates 3D content from multimodal inputs . they propose 198 multimodal text inputs for 3D generation tasks .
Outcome:	The proposed framework generates 3D content from multimodal inputs without human intervention.

Copied to clipboard

Guanhua Chen, Yutong Yao, Derek F. Wong, Lidia S. Chao

Reasoning in the Dark: Interleaved Vision-Text Reasoning in Latent Space (2026.findings-acl)

Challenge:	Multi-intent natural language understanding (NLU) models lack the rich information between the shared intents, especially in low-data scenarios.
Approach:	They propose a two-stage framework for multi-intent natural language understanding to harness shared intent information by word-level pre-training and prediction-aware contrastive fine-tuning.
Outcome:	The proposed framework surpasses baselines on low-data and full-data scenarios.

Copied to clipboard

Chao Chen, Zhixin Ma, Yongqi Li, Yupeng Hu, Yinwei Wei, Wenjie Li, Liqiang Nie

GlimpRouter: Efficient Collaborative Inference by Glimpsing One Token of Thoughts (2026.findings-acl)

Challenge:	Existing multimodal reasoning methods depend on explicit reasoning steps that require labor-intensive vision-text annotations and inherently introduce significant inference latency.
Approach:	They propose a method that integrates visual and visual information into the reasoning process to improve the performance of multimodal LLMs.
Outcome:	The proposed method achieves an average performance increase of 5.45% while achieving a speed increase of over 5 times compared to existing methods.

Copied to clipboard

Wenhao Zeng, Xuteng Zhang, Yuling Shi, Chao Hu, Yuting Chen, Beijun Shen, Xiaodong Gu

Palette of Language Models: A Solver for Controlled Text Generation (2025.naacl-long)

Challenge:	Existing routing strategies rely on local token probabilities or post-hoc verification, introducing significant inference overhead.
Approach:	They propose a step-wise collaboration framework that generates only the first token of each reasoning step and routes it to a larger model only when initial token entropy exceeds a threshold.
Outcome:	The proposed approach reduces inference latency while preserving accuracy.

Copied to clipboard

Zhe Yang, Yi Huang, Yaqin Chen, XiaotingWu XiaotingWu, Junlan Feng, Chao Deng

End-to-End Optimization of LLM-Driven Multi-Agent Search Systems via Heterogeneous-Group-Based Reinforcement Learning (2026.acl-long)

Challenge:	Recent advances in large language models have revolutionized text generation with their remarkable capabilities.
Approach:	They propose to combine a single-attribute model with a discriminative model to achieve a combination strategy that incorporates positive correlation and attribute enhancement.
Outcome:	The proposed method is adapted for single-attribute control scenario and achieves surpassing results.

Copied to clipboard

Guanzhong Chen, Shaoxiong Yang, Chao Li, Wei Liu, Jian Luan, Zenglin Xu

M3AV: A Multimodal, Multigenre, and Multipurpose Audio-Visual Academic Lecture Dataset (2024.acl-long)

Challenge:	Existing multi-agent reinforcement learning methods depend on large critic networks to evaluate joint actions, leading to instability and high memory costs.
Approach:	They propose a method to optimize large language models for agent-specific roles . they propose combining agent-based frameworks with retrieval-augmented generation .
Outcome:	Experiments show that multi-agent group policy optimization outperforms baselines in task performance and computational efficiency.

Copied to clipboard

Zhe Chen, Heyang Liu, Wenyi Yu, Guangzhi Sun, Hongcheng Liu, Ji Wu, Chao Zhang, Yu Wang, Yanfeng Wang

UrbanLLM: Autonomous Urban Activity Planning and Management with Large Language Models (2024.findings-emnlp)

Challenge:	Publishing open-source academic video recordings is an emerging approach to sharing knowledge online.
Approach:	They propose a multimodal, multigenre, and multipurpose audio-visual academic lecture dataset with human annotations for multimodal content recognition and understanding tasks.
Outcome:	The proposed dataset can be used for multiple audio-visual recognition and understanding tasks.

Copied to clipboard

Yue Jiang, Qin Chao, Yile Chen, Xiucheng Li, Shuai Liu, Gao Cong

Towards Efficient and Multifaceted Computer-assisted Pronunciation Training Leveraging Hierarchical Selective State Space Model and Decoupled Cross-entropy Loss (2025.naacl-long)

Challenge:	UrbanLLM is a fine-tuned large language model designed to tackle diverse urban problems.
Approach:	They propose a fine-tuned large language model to tackle diverse urban problems . UrbanLLM decomposes urban-related queries into manageable sub-tasks .
Outcome:	The proposed model outperforms existing models in urban planning and management tasks.

Copied to clipboard

Fu-An Chao, Berlin Chen

Self-Paced Learning for Neural Machine Translation (2020.emnlp-main)

Challenge:	APA and MDD are two of the main tasks of computer-assisted pronunciation training (CAPT) systems.
Approach:	They propose a computer-assisted pronunciation training approach that integrates APA and MDD tasks in parallel.
Outcome:	The proposed approach improves on APA and MDD tasks, and achieves an F1 score of 63.85%.

Copied to clipboard

Yu Wan, Baosong Yang, Derek F. Wong, Yikai Zhou, Lidia S. Chao, Haibo Zhang, Boxing Chen

Machine Translation

Challenge:	Existing studies have shown that the training of neural machine translation (NMT) rely on the quality of artificial schedule drawn up with the handcrafted features, e.g. sentence length or word rarity.
Approach:	They propose to train NMT model using a self-paced learning approach that allows it to quantify the learning confidence over training examples and flexibly govern its learning via regulating the loss in each iteration step.
Outcome:	The proposed model outperforms baseline models and those trained with human-designed curricula on translation quality and convergence speed.

Self-Generated Critiques Boost Reward Modeling for Language Models (2025.naacl-long)

Copied to clipboard

Yue Yu, Zhengxing Chen, Aston Zhang, Liang Tan, Chenguang Zhu, Richard Yuanzhe Pang, Yundi Qian, Xuewei Wang, Suchin Gururangan, Chao Zhang, Melanie Kambadur, Dhruv Mahajan, Rui Hou

GenDis: Generative-Discriminative Dual-View Co-Training for Generalized Category Discovery (2026.acl-long)

Challenge:	Existing reward models produce scalar scores and struggle to incorporate critiques in a natural language format.
Approach:	They propose a framework that predicts critiques and rewards using self-generated critiques without extra supervision.
Outcome:	The proposed framework improves reward modeling accuracy by 3.7%-7.3% compared to standard reward models and LLM judges.

Copied to clipboard

Xi Chen, Chuan Qin, Jinpeng Li, Shasha Hu, Chao Wang, Hengshu Zhu, Hui Xiong

Challenge:	Existing methods rely on one-hot discriminative supervision, leading to overfitting on seen classes and poor generalization to unseen ones.
Approach:	They propose a Generative–Discriminative Dual-View Co-Training framework that unifies discriminative classification and semantic label generation within an LLM.
Outcome:	The proposed framework outperforms existing methods on five benchmarks on the generalized category discovery (GCD) task.

EDSD: Entropy-Driven Design for Faster Speculative Decoding (2026.acl-long)

Copied to clipboard

Longkai Cheng, Ximing Wang, Jiangcai Zhu, Kailai Shao, Chao Chen, Haixiang Hu

TokenSelect: Efficient Long-Context Inference and Length Extrapolation for LLMs via Dynamic Token-Level KV Cache Selection (2025.emnlp-main)

Challenge:	Existing methods for speculative decoding incur substantial training overhead to mitigate information misalignment between autoregressive draft model training and decoding.
Approach:	They propose an Entropy-Driven Speculative Decoding framework that uses entropy as a unified, interpretable signal for both draft model training and architectural design.
Outcome:	Experiments on seven large language models show that EDSD improves training efficiency by 24.8% and increases acceptance length by 4.0% compared to state-of-the-art methods.

Copied to clipboard

Wei Wu, Zhuoshi Pan, Kun Fu, Chao Wang, Liyi Chen, Yunchu Bai, Tianfu Wang, Zheng Wang, Hui Xiong

Adversarial Preference Learning for Robust LLM Alignment (2025.findings-acl)

Challenge:	Rapid advances in Large Language Models have spurred demand for processing extended context sequences . however, performance degradation due to sequence lengths out-of-distribution and excessively long inference times are limiting LLMs in long-context scenarios.
Approach:	They propose a training-free method for efficient and accurate long-context inference . they selectively involves a few critical KV cache tokens in attention calculation .
Outcome:	The proposed method speeds up attention computation and accelerates inference time while reducing selection overhead.

Copied to clipboard

Yuanfu Wang, Pengyu Wang, Chenyang Xi, Bo Tang, Junyi Zhu, Wenqiang Wei, Chen Chen, Chao Yang, Jingfeng Zhang, Chaochao Lu, Yijun Niu, Keming Mao, Zhiyu Li, Feiyu Xiong, Jie Hu, Mingchuan Yang

Task-Agnostic Detector for Insertion-Based Backdoor Attacks (2024.findings-naacl)

Challenge:	Modern language models rely on Reinforcement Learning from Human Feedback (RLHF) to encourage safe behaviors, but they remain vulnerable to adversarial attacks due to three key limitations: (1) the inefficiency and high cost of human annotation; (2) the vast diversity of potential adversarials; and (3) the risk of feedback bias and reward hacking.
Approach:	They propose an iterative adversarial training method that incorporates three key innovations to address these challenges.
Outcome:	Experiments on Mistral-7B-Instruct-v0.3 show that the proposed method significantly enhances robustness and reduces harmful outputs from 5.88% to 0.43%.

Copied to clipboard

Weimin Lyu, Xiao Lin, Songzhu Zheng, Lu Pang, Haibin Ling, Susmit Jha, Chao Chen

Robustness in NLP

Challenge:	Existing methods for textual backdoor detection are task-specific and less effective beyond sentence classification.
Approach:	They propose a task-agnostic method for backdoor detection that leverages final layer logits and an efficient pooling technique.
Outcome:	TABDet can jointly learn from diverse task-specific models, demonstrating superior detection efficacy over traditional methods.

Learning-by-Narrating: Narrative Pre-Training for Zero-Shot Dialogue Comprehension (2022.acl-short)

Copied to clipboard

Chao Zhao, Wenlin Yao, Dian Yu, Kaiqiang Song, Dong Yu, Jianshu Chen

MMAD:Multi-modal Movie Audio Description (2024.lrec-main)

Challenge:	Existing models for dialogue comprehension are not available for the pre-training of such a model.
Approach:	They propose a narrative-guided pre-training strategy that learns by narrating key information from a dialogue input.
Outcome:	The proposed model performs better on four dialogue-based tasks and is comparable to existing models.

Copied to clipboard

Xiaojun Ye, Junhao Chen, Xiang Li, Haidong Xin, Chao Li, Sheng Zhou, Jiajun Bu

UniTE: Unified Translation Evaluation (2022.acl-long)

Challenge:	Current methods of creating accessible movies rely on manual work, resulting in high costs and limited scalability.
Approach:	They propose a multi-modal movie audio description pipeline that generates narrations of information that is not accessible through unimodal hearing in movies.
Outcome:	The proposed pipeline surpasses existing baselines in performance on widely used datasets.

Copied to clipboard

Yu Wan, Dayiheng Liu, Baosong Yang, Haibo Zhang, Boxing Chen, Derek Wong, Lidia Chao

Multilinguality

Challenge:	Recent methods for evaluation of translation quality are focused on one task, ignoring commonalities .
Approach:	They propose a unified framework engaged with abilities to handle all three evaluation tasks.
Outcome:	The proposed framework can universally surpass state-of-the-art or winner methods across tasks.

Retrieve-Plan-Generation: An Iterative Planning and Answering Framework for Knowledge-Intensive LLM Generation (2024.emnlp-main)

Copied to clipboard

Yuanjie Lyu, Zihan Niu, Zheyong Xie, Chao Zhang, Tong Xu, Yang Wang, Enhong Chen

SGIC: A Self-Guided Iterative Calibration Framework for RAG (2025.acl-long)

Challenge:	Large language models (LLMs) often produce factual errors due to limited internal knowledge.
Approach:	They propose a retrieval-augmented generation framework that generates plan tokens to guide subsequent generation.
Outcome:	The proposed framework improves the accuracy of large language models with external knowledge sources.

Copied to clipboard

Guanhua Chen, Yutong Yao, Lidia S. Chao, Xuebo Liu, Derek F. Wong

Prompting Large Language Models to Tackle the Full Software Development Lifecycle: A Case Study (2025.coling-main)

Challenge:	Existing studies on retrieval-augmented generation (RAG) focus on extracting relevant documents or refinement of specialized instructions.
Approach:	They propose a framework that provides LLMs with specific cues to improve their calibration efficacy . they propose an iterative self-calibration training set that harnesses uncertainty scores .
Outcome:	The proposed framework significantly improves performance on both closed-source and open-source LLMs.

Copied to clipboard

Bowen Li, Wenhan Wu, Ziwei Tang, Lin Shi, John Yang, Jinyang Li, Shunyu Yao, Chen Qian, Binyuan Hui, Qicheng Zhang, Zhiyin Yu, He Du, Ping Yang, Dahua Lin, Chao Peng, Kai Chen

Red Teaming Large Reasoning Models (2026.acl-long)

Challenge:	Existing benchmarks focused on simplified or isolated aspects of coding, ignoring the full spectrum of programming challenges.
Approach:	They propose a case study that examines the performance of large language models across the entire software development lifecycle with four programming languages, multiple domains, and carefully designed and verified metrics for each task.
Outcome:	The proposed model performs across the entire software development lifecycle, including design, environment setup, implementation, acceptance testing, and unit testing.

Copied to clipboard

Jiawei Chen, Yang Yang, Chao Yu, Yu Tian, Zhi Cao, Xue Yang, Linghao Li, Hang Su, Zhaoxia Yin

Boosting Data Utilization for Multilingual Dense Retrieval (2025.emnlp-main)

Challenge:	Large Reasoning Models (LRMs) have emerged as a powerful advancement in multi-step reasoning tasks, but they introduce safety and reliability risks, such as CoT-hijacking and prompt-induced inefficiencies.
Approach:	They propose a unified benchmark to assess the trustworthiness of Large Reasoning Models.
Outcome:	The proposed benchmark evaluates truthfulness, safety and efficiency on 26 models.

Copied to clipboard

Chao Huang, Fengran Mo, Yufeng Chen, Changhao Guan, Zhenrui Yue, Xinyu Wang, Jinan Xu, Kaiyu Huang

ShredBench: Evaluating the Semantic Reasoning Capabilities of Multimodal LLMs in Document Reconstruction (2026.findings-acl)

Challenge:	Existing studies focus on fine-tuning multilingual dense retrieval models, but data scarcity for low-resource languages makes it difficult to align representations in a shared vector space.
Approach:	They propose to obtain high-quality hard negative samples and effective mini-batch data to boost data utilization for multilingual dense retrieval by obtaining high-quality negative samples.
Outcome:	The proposed method outperforms existing baselines on a multilingual retrieval benchmark, MIRACL, with 16 languages.

Copied to clipboard

Zichun Guo, Yuling Shi, Wenhao Zeng, Chao Hu, Haotian Lin, Terry Yue Zhuo, Jiawei Chen, Xiaodong Gu, Wenping Ma

Beyond Overlap Metrics: Rewarding Reasoning and Preferences for Faithful Multi-Role Dialogue Summarization (2026.findings-acl)

Challenge:	Empirical evaluations on state-of-the-art MLLMs reveal a significant performance gap . ML models lack the fine-grained cross-modal reasoning required to bridge visual discontinuities.
Approach:	They propose a benchmark that renders fragmented documents directly from Markdown to facilitate evaluation of VRDU tasks.
Outcome:	The proposed benchmark renders fragmented documents directly from Markdown.

Copied to clipboard

Xiaoyong Mei, Tingting Zuo, Da Chen, Guangyu Hu, Xiangyu Wen, Chao Duan, Mingyan Zhang, Fudan Zheng

Toward Consistent World Models with Multi-Token Prediction and Latent Semantic Enhancement (2026.acl-long)

Challenge:	Existing methods for multi-role dialogue summarization favor surface-level imitation of references rather than genuine gains in faithfulness or alignment with human preferences.
Approach:	They propose a framework that couples explicit cognitive-style reasoning with reward-based optimization for multi-role dialogue summarization.
Outcome:	The proposed framework matches strong baselines on ROUGE and BERTScore, while in-depth analysis on SAMSum shows clear gains in factual faithfulness and model-based preference alignment.

Copied to clipboard

Qimin Zhong, Hao Liao, Haiming Qin, Mingyang Zhou, Rui Mao, Wei Chen, Naipeng Chao

SEER: Facilitating Structured Reasoning and Explanation via Reinforcement Learning (2024.acl-long)

Challenge:	Existing methods to learn internal world models rely on one-step supervision . however, standard MTP suffers from structural hallucinations .
Approach:	They propose a method which anchors predictions to ground-truth hidden state trajectories.
Outcome:	The proposed method bridges the gap between discrete tokens and continuous state representations, reducing structural hallucinations, and improving robustness to perturbations.

Copied to clipboard

Guoxin Chen, Kexin Tang, Chao Yang, Fuying Ye, Yu Qiao, Yiming Qian

LLaSE-G1: Incentivizing Generalization Capability for LLaMA-based Speech Enhancement (2025.acl-long)

Challenge:	Existing methods focus on single-step reasoning, ignoring logical dependencies between steps.
Approach:	They propose a method that maximizes a structure-based return to facilitate structured reasoning and explanation.
Outcome:	The proposed method outperforms state-of-the-art methods on EntailmentBank and STREET benchmarks.

Copied to clipboard

Boyi Kang, Xinfa Zhu, Zihan Zhang, Zhen Ye, Mingshuai Liu, Ziqian Wang, Yike Zhu, Guobin Ma, Jun Chen, Longshuai Xiao, Chao Weng, Wei Xue, Lei Xie

On the Representation Geometry of LoRA Model Merging (2026.findings-acl)

Challenge:	Recent advances in language models have demonstrated strong capabilities in semantic understanding and contextual modeling.
Approach:	They propose a LLaMA-based language model that incentivizes generalization capabilities for speech enhancement.
Outcome:	The proposed language model outperforms prior task-specific discriminative and generative models in acoustic enhancement tasks.

Copied to clipboard

Chenyang Lu, Jiaru Li, Jinman Zhao, Xinran Chen, Yining Wang, Renyi Cai, Yuchen Li, Chao He

Representation Learning

Challenge:	Existing methods for low-rank Adaptation (LoRA) fine-tuning focus on globally shared structure . combining SVD with CUR improves performance of LoRA model merging .
Approach:	They propose a training-free method that combines SVD and CUR decomposition to improve LoRA merging performance.
Outcome:	The proposed procedure improves on vision and language benchmarks.

MURAL: Multimodal, Multitask Representations Across Languages (2021.findings-emnlp)

Copied to clipboard

Aashi Jain, Mandy Guo, Krishna Srinivasan, Ting Chen, Sneha Kudugunta, Chao Jia, Yinfei Yang, Jason Baldridge

Chain-of-Procedure: Hierarchical Visual-Language Reasoning for Procedural QA (2026.findings-acl)

Challenge:	Image-caption pairs and translation pairs provide the means to learn deep representations of and connections between languages.
Approach:	They propose a dual encoder that integrates image-text matching and translation pairs to solve two tasks by learning from billions of pairs.
Outcome:	The proposed encoder outperforms ALIGN's cross-modal retrieval performance on well-resourced languages and significantly improves on under-resource languages.

Copied to clipboard

Guanhua Chen, Yutong Yao, Shenghe Sun, Ci-jun Gao, Shudong Liu, Lidia S. Chao, Feng Wan, Derek F. Wong

LightReasoner: Can Small Language Models Teach Large Language Models Reasoning? (2026.acl-long)

Challenge:	Recent advances in vision-language models (VLMs) have achieved impressive results on standard image-text tasks, yet their capability in visual procedure question answering (VP-QA) remains largely unexplored.
Approach:	They propose a multimodal benchmark specifically designed for visual procedural reasoning that synergizes cross-modal procedure retrieval, context-aware step decomposition, and the next step prediction.
Outcome:	The proposed framework significantly outperforms baselines on visual procedure question answering (VP-QA) Experiments on six VLMs show that it performs better than baselines.

Copied to clipboard

Jingyuan Wang, Yankai Chen, Zhonghang Li, Chao Huang

Evaluating the Expressive Appropriateness of Speech in Rich Contexts (2026.acl-long)

Challenge:	Large language models (LLMs) have demonstrated remarkable progress in reasoning, but are resource-intensive and require large curated datasets.
Approach:	They propose a framework that leverages the behavioral divergence between a stronger expert model and a weaker amateur model.
Outcome:	The proposed framework improves accuracy by up to 28.1% while reducing time consumption by 90% and tuning token usage by 99%.

Copied to clipboard

Tianrui Wang, Ziyang Ma, Yizhou Peng, Haoyu Wang, Zhikang Niu, Zikang Huang, Yihao Wu, Yi-Wen Chao, Yu Jiang, Yuheng Lu, Guanrou Yang, Xuanchen Li, Hexin Liu, Chunyu Qiang, Cheng Gong, Yifan Yang, Tianchi Liu, Junyu Wang, Nana Hou, Meng Ge, Fuming You, Yang Wei, Zhongqian Sun, Hu Haifeng, Xiaobao Wang, Eng Siong Chng, Xie Chen, Longbiao Wang, Jianwu Dang

Speech & Audio in NLP

Challenge:	Existing methods for evaluating expressive speech focus on word accuracy, naturalness, signal quality, or emotional intensity at the utterance level.
Approach:	They propose a framework for Evaluating Expressive Appropriateness in speech that assesses whether a speech sample aligns with the underlying communicative intent implied by its discourse-level narrative context.
Outcome:	The proposed framework outperforms existing speech evaluation and analysis systems on a human-annotated test set.

Lost in the Context: Insufficient and Distracted Attention to Contexts in Preference Modeling (2025.acl-long)

Copied to clipboard

Shihan Dou, Jiayi Chen, Chenhao Huang, Feng Chen, Wei Chengzhi, Huiyuan Zheng, Shichun Liu, Yan Liu, Chenxiao Liu, Chao Xin, Lin Yan, Zongzhang Zhang, Tao Gui, Qi Zhang, Xuanjing Huang

Challenge:	Existing reward models concatenate contexts and responses, but they often ignore crucial segments of the context that are important for evaluating the response quality.
Approach:	They propose a reward model that evaluates the response quality based on a given context and assigns a rewards reward.
Outcome:	The proposed framework significantly improves preference modeling by increasing attention to relevant information within the context and achieves better generalizability.

LLM as a metric critic for low resource relation identification (2024.findings-emnlp)

Copied to clipboard

Zhe Yang, Yi Huang, Yaqin Chen, Xiaoting Wu, Junlan Feng, Chao Deng

SPEAK: Spiking Neurons as an Entropy-Aware Tokenizer for Large Language Models (2026.acl-long)

Challenge:	Existing studies show that small language models (SLMs) overfit in low resource situations . however, the gap between pre-training and fine-tuning leads to performance decay .
Approach:	They propose to combine large language models and LLM for relation identification by co-evolution . they propose to use a masked language model prompt to generate a relation identification task .
Outcome:	The proposed model can handle low resource relation identification tasks with minimal overfitting . the proposed model provides essential background knowledge to assist training process .

Copied to clipboard

Ming Chen, Wenyao Li, Chao Liang, Shi Gu, Peng Lin, De Ma, Huajin Tang, Qian Zheng, Gang Pan

Towards Interpretable Clinical Diagnosis with Bayesian Network Ensembles Stacked on Entity-Aware CNNs (2020.acl-main)

Challenge:	Existing tokenizers fail to explicitly leverage historical tokenization results . large language models (LLMs) have demonstrated remarkable effectiveness across NLP tasks .
Approach:	They propose a tokenizer that integrates spiking neurons to explicitly leverage historical tokenization results.
Outcome:	The proposed tokenizer leverages historical tokenization results, but does not selectively leverage history based on contextual relevance.

Copied to clipboard

Jun Chen, Xiaoya Dai, Quan Yuan, Chao Lu, Haifeng Huang

Explainability & Interpretability in NLP

Challenge:	a novel framework for text-based diagnosis of diseases requires appropriate balance between accuracy and interpretability.
Approach:	They propose a framework that stacks Bayesian Network Ensembles on top of CNN to build an accurate yet interpretable diagnosis system.
Outcome:	The proposed framework outperforms the previous automatic diagnosis methods in accuracy performance and the diagnosis explanation of the framework is reasonable.

Self-Training with Differentiable Teacher (2022.findings-naacl)

Copied to clipboard

Simiao Zuo, Yue Yu, Chen Liang, Haoming Jiang, Siawpeng Er, Chao Zhang, Tuo Zhao, Hongyuan Zha

An Iterative Associative Memory Model for Empathetic Response Generation (2024.acl-long)

Challenge:	Existing methods for self-training are interpreted as teacher-student frameworks, where the teacher generates pseudo-labels and the student makes predictions.
Approach:	They propose a differentiable self-training method that treats teacher-student as a Stackelberg game where a leader is always in a more advantageous position than a follower.
Outcome:	The proposed model outperforms existing methods on semi- and weakly-supervised learning tasks on semi and weak supervised tasks.

Copied to clipboard

Zhou Yang, Zhaochun Ren, Wang Yufeng, Haizhou Sun, Chao Chen, Xiaofei Zhu, Xiangwen Liao

Crafting Customisable Characters with LLMs: A Persona-Driven Role-Playing Agent Framework (2025.findings-emnlp)

Challenge:	Existing methods for empathetic response generation ignore the associated words between dialogue utterances.
Approach:	They propose an iterative associative memory model to capture associated words between dialogue utterances and situations, dialogue history, and a memory module for storing associated words.
Outcome:	The proposed model captures key words between dialogue utterances and situations, dialogue history, and a memory module, thereby accurately and nuancedly comprehending the utterables.

Copied to clipboard

Bohao Yang, Dong Liu, Chenghao Xiao, Kun Zhao, Chen Tang, Chao Li, Lin Yuan, Yang Guang, Chenghua Lin

Challenge:	Large Language Models (LLMs) are capable of generating human-like text, but the potential for freely customisable characters remains underexplored.
Approach:	They propose a framework which employs Large Language Models to create freely customisable characters through personalised characteristic feature injection.
Outcome:	The proposed framework provides valuable insights for developing more accurate and customisable human simulacra.

A Study of the Attention Abnormality in Trojaned BERTs (2022.naacl-main)

Copied to clipboard

Weimin Lyu, Songzhu Zheng, Tengfei Ma, Chao Chen

D4: a Chinese Dialogue Dataset for Depression-Diagnosis-Oriented Chat (2022.emnlp-main)

Challenge:	In computer vision, the trigger can be a fixed pattern overlaid on the images or videos.
Approach:	They propose an attention-based Trojan detector to distinguish Trojaned models from clean ones by observing the attention focus drifting behavior of Trojanes.
Outcome:	The proposed detector is based on transformer’s attention and can distinguish Trojan models from clean ones.

Copied to clipboard

Binwei Yao, Chao Shi, Likai Zou, Lingfeng Dai, Mengyue Wu, Lu Chen, Zhen Wang, Kai Yu

Understanding LLMs’ Cross-Lingual Context Retrieval: How Good It Is And Where It Comes From (2025.emnlp-main)

Challenge:	Existing human-machine dialogue systems are not able to provide diagnostic information for depression diagnosis due to stigma associated with mental illness.
Approach:	They propose to construct a Chinese Dialogue Dataset for depression-diagnosis-oriented chat based on clinical depression diagnostic criteria.
Outcome:	The proposed system can be used to diagnose depression using a Chinese Dialogue Dataset.

Copied to clipboard

Changjiang Gao, Hankun Lin, Xin Huang, Xue Han, Junlan Feng, Chao Deng, Jiajun Chen, Shujian Huang

Cross-Lingual Transfer

Challenge:	Cross-lingual context retrieval is a fundamental aspect of cross-lingual alignment, but the performance and mechanism of it for large language models (LLMs) remains unclear.
Approach:	They evaluate cross-lingual context retrieval of over 40 large language models . they use cross-linguistic machine reading comprehension as a representative scenario .
Outcome:	The results show that open LLMs show strong cross-lingual context retrieval ability . the results also show that their oracle performances improve after training .

Target-Guided Structured Attention Network for Target-Dependent Sentiment Analysis (2020.tacl-1)

Copied to clipboard

Ji Zhang, Chengyao Chen, Pengfei Liu, Chao He, Cane Wing-Ki Leung

Sentiment Analysis

Challenge:	TDSA aims to classify the sentiment of a text towards a given target.
Approach:	They propose a novel Target-Guided Structured Attention Network (TG-SAN) which captures target-related contexts for TDSA in a fine-to-coarse manner.
Outcome:	The proposed network outperforms the state-of-the-art in terms of accuracy and Marco-F1 on three benchmarks with three major findings.

Migician: Revealing the Magic of Free-Form Multi-Image Grounding in Multimodal Large Language Models (2025.findings-acl)

Copied to clipboard

You Li, Heyu Huang, Chi Chen, Kaiyu Huang, Chao Huang, Zonghao Guo, Zhiyuan Liu, Jinan Xu, Yuhua Li, Ruixuan Li, Maosong Sun

Teaching LLM to be Persuasive: Reward-Enhanced Policy Optimization for Alignment from Heterogeneous Rewards (2026.acl-industry)

Challenge:	Existing MLLMs still struggle to achieve precise grounding in multi-image scenarios.
Approach:	They propose a Chain-of-Thought framework that integrates single-image grounding with multi-image comprehension to address this challenge.
Outcome:	The proposed model outperforms existing models in multi-image grounding tasks by 24.94% and surpasses larger 70B models.

Copied to clipboard

Xia Zeng, Yihan Chen, Luhui Liu, Chao Luo, Ye Chen, null Zhuangzhuoran

Self-Adjust Softmax (2025.emnlp-main)

Challenge:	a large language model (LLM) is used as a business development agent for persuasive price negotiation in online travel agencies.
Approach:	They propose a reward-enhancing policy optimization method that integrates three complementary reward sources-a preference-trained reward model and an LLM-as-a-judge.
Outcome:	The proposed method improves average dialogue rating to 4.63 (+0.33 over GRPO) and raises share of conversations with at least one excellent response to 66.67% (+23.34 pp over grepo).

Copied to clipboard

Chuanyang Zheng, Yihang Gao, Guoxuan Chen, Han Shi, Jing Xiong, Xiaozhe Ren, Chao Huang, Zhenguo Li, Yu Li

QualiSpeech: A Speech Quality Assessment Dataset with Natural Language Reasoning and Descriptions (2025.acl-long)

Challenge:	Usually, tokens with larger attention scores are important for the final prediction.
Approach:	They propose to modify softmax(z) to z softmax and its normalized variant to improve the Transformer attention mechanism by making minor adjustments to the softmax function.
Outcome:	The proposed model provides enhanced gradient properties compared to the vanilla softmax function.

Copied to clipboard

Siyin Wang, Wenyi Yu, Xianzhao Chen, Xiaohai Tian, Jun Zhang, Lu Lu, Yu Tsao, Junichi Yamagishi, Yuxuan Wang, Chao Zhang

Frustratingly Easy Label Projection for Cross-lingual Transfer (2023.findings-acl)

Challenge:	Existing datasets lack comprehensive annotations for speech quality assessment . existing methods lack detailed annotations, resulting in inaccurate evaluations.
Approach:	They propose a low-level speech quality assessment dataset incorporating natural language descriptions and a Benchmark to evaluate low- level speech understanding capabilities of auditory large language models.
Outcome:	The proposed model can be used to evaluate the low-level speech understanding capabilities of auditory large language models.

Copied to clipboard

Yang Chen, Chao Jiang, Alan Ritter, Wei Xu

Multilinguality

Challenge:	Existing approaches to improve cross-lingual transfer performance are based on word alignment, but no empirical studies have evaluated their effectiveness or limitations.
Approach:	They propose a mark-then-translate method that integrates translation and projection by inserting special markers around the labeled spans in the original sentence.
Outcome:	The proposed method outperforms word alignment-based methods in 57 languages and three tasks.

LongTableBench: Benchmarking Long-Context Table Reasoning across Real-World Formats and Domains (2025.findings-emnlp)

Copied to clipboard

Liyao Li, Jiaming Tian, Hao Chen, Wentao Ye, Chao Ye, Haobo Wang, Ningtao Wang, Xing Fu, Gang Chen, Junbo Zhao

LLM×MapReduce: Simplified Long-Sequence Processing using Large Language Models (2025.acl-long)

Challenge:	Evaluating 52 LLMs reveals that only the strongest models maintain robust performance under increasing context lengths and format diversity.
Approach:	They propose a benchmark for evaluating long-context reasoning over semi-structured tables across diverse formats, tasks, and domains.
Outcome:	The proposed model outperforms compression-based approaches on tasks requiring semantic integration.

Copied to clipboard

Zihan Zhou, Chong Li, Xinyi Chen, Shuo Wang, Yu Chao, Zhili Li, Haoyu Wang, Qi Shi, Zhixing Tan, Xu Han, Xiaodong Shi, Zhiyuan Liu, Maosong Sun

Modelling Variability in Human Annotator Simulation (2024.findings-acl)

Challenge:	Existing studies have focused on extending the context length of large language models (LLMs) due to their quadratic computational complexity and a lack of high-quality long training examples, most LLMs are trained with a limited window size.
Approach:	They propose a training-free framework that enables large language models to effectively process long texts using a divide-and-conquer strategy for comprehensive document understanding.
Outcome:	The proposed framework outperforms open-source and commercial long-context LLMs and is compatible with several models.

Copied to clipboard

Wen Wu, Wenlin Chen, Chao Zhang, Phil Woodland

How Far Does BERT Look At: Distance-based Clustering and Analysis of BERT’s Attention (2020.coling-main)

Challenge:	Human annotator simulation (HAS) is a cost-effective alternative to human evaluation tasks.
Approach:	They propose a framework to model human annotation variability via meta-learning . conditional softmax flow model leverages diverse human annotations via meta learning . results demonstrate that method can predict aggregated behaviours of human annotators .
Outcome:	The proposed method achieves state-of-the-art performance on two real-world human evaluation tasks: emotion recognition and toxic speech detection.

Copied to clipboard

Yue Guan, Jingwen Leng, Chao Li, Quan Chen, Minyi Guo

StablePT : Towards Stable Prompting for Few-shot Learning via Input Separation (2024.findings-emnlp)

Challenge:	Recent work on multi-head attention mechanism shows heuristics and clues in analyzing various aspects of the mechanism.
Approach:	They propose to cluster attention heatmaps into significantly different patterns through unsupervised clustering on top of a set of proposed features.
Outcome:	The proposed features can explain and calibrate different attention heads in Transformer models.

Copied to clipboard

Xiaoming Liu, Chen Liu, Zhaohan Zhang, Chengzhengxu Li, Longtian Wang, Yu Lan, Chao Shen

scRAG: Hybrid Retrieval-Augmented Generation for LLM-based Cross-Tissue Single-Cell Annotation (2025.findings-acl)

Challenge:	Existing studies on prompt tuning have shown that language models can be effective few-shot learners with prompting.
Approach:	They propose to treat the hard prompt and soft prompt as separate inputs to mitigate noise brought by prompt initialization.
Outcome:	Experimental results show that the proposed method outperforms state-of-the-art methods by 6.97% in accuracy and reduces the standard deviation by 1.92 on average.

Copied to clipboard

Zhiyin Yu, Chao Zheng, Chong Chen, Xian-Sheng Hua, Xiao Luo

Aria-UI: Visual Grounding for GUI Instructions (2025.findings-acl)

Challenge:	Large language models (LLMs) have demonstrated impressive potential in a wide range of fields, including biology, genomics and healthcare.
Approach:	They propose a framework that integrates advanced LLM-based RAG techniques into cross-tissue single-cell annotation.
Outcome:	The proposed framework outperforms baseline models, generalist models, domain-specific methods, and trained classifiers on a cross-tissue dataset.

Copied to clipboard

Yuhao Yang, Yue Wang, Dongxu Li, Ziyang Luo, Bei Chen, Chao Huang, Junnan Li