Papers by Qi Cao

Muse: Towards Reproducible Long-Form Song Generation with Fine-Grained Style Control (2026.findings-acl)

Challenge:	Recent large language models (LLMs) have demonstrated remarkable generalization abilities in mathematics and reasoning tasks.
Approach:	They pre-trained decoder-based language models from scratch using ten programming languages and three natural language datasets.
Outcome:	The proposed models outperform natural languages on logical reasoning tasks.

Copied to clipboard

Changhao Jiang, Jiahao Chen, Zhenghao Xiang, Zhixiong Yang, Hanchen Wang, Jiabao Zhuang, Xinmeng Che, Jiajun Sun, Hui Li, Yifei Cao, Shihan Dou, Ming Zhang, Junjie Ye, Tao Ji, Tao Gui, Qi Zhang, Xuanjing Huang

Speech & Audio in NLP

Challenge:	Recent commercial systems such as Suno demonstrate strong capabilities in long-form song generation, but academic research remains non-reproducible due to the lack of publicly available training data.
Approach:	They propose a system for long-form song generation with fine-grained style conditioning that includes a licensed synthetic dataset and a song generation model, Muse.
Outcome:	The proposed system achieves competitive performance on phoneme error rate, text–music style similarity, and audio aesthetic quality while enabling controllable segment-level generation across different musical structures.

Large Language Models are not Fair Evaluators (2024.acl-long)

Copied to clipboard

Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Lingpeng Kong, Qi Liu, Tianyu Liu, Zhifang Sui

Argue with Me Tersely: Towards Sentence-Level Counter-Argument Generation (2023.emnlp-main)

Challenge:	Existing evaluation frameworks that use large language models as referees are insufficient for accurately assessing their alignment with human intent.
Approach:	They propose a calibration framework to address positional bias in large language models as evaluators by manually annotating the “win/tie/lose” outcomes of responses from ChatGPT and Vicuna-13B in the Vicun A Benchmark’s question prompt.
Outcome:	The proposed framework alleviates evaluation bias, resulting in closer alignment with human judgments.

Copied to clipboard

Jiayu Lin, Rong Ye, Meng Han, Qi Zhang, Ruofei Lai, Xinyu Zhang, Zhao Cao, Xuanjing Huang, Zhongyu Wei

Text Generation

Challenge:	Existing work describes paragraph-level counter-argument generation task as paragraph-based . however, sentence-level generation can be quite different due to its unique constraints and brevity-focused challenges.
Approach:	They propose a benchmark framework for sentence-level counter-argument generation . they use an annotated debate forum dataset to generate high-quality counter-argments .
Outcome:	The proposed framework and evaluator are competitive in counter-argument generation tasks.

MAIN: Mutual Alignment Is Necessary for instruction tuning (2025.emnlp-main)

Copied to clipboard

Fanyi Yang, Jianfeng Liu, Xin Zhang, Haoyu Liu, Xixin Cao, Yuefeng Zhan, Hao Sun, Weiwei Deng, Feng Sun, Qi Zhang

Bit-Flip Error Resilience in LLMs: A Comprehensive Analysis and Defense Framework (2025.emnlp-main)

Challenge:	Instruction tuning has enabled large language models to achieve remarkable performance, yet its success heavily depends on the availability of high-quality instruction-response pairs.
Approach:	They propose a mutual alignment framework which enforces coherence between instructions and responses through mutual constraints.
Outcome:	The proposed framework generalizes well across model architectures and sizes, achieving state-of-the-art performance on LLaMA, Mistral, and Qwen models across diverse benchmarks.

Copied to clipboard

Yuhang Chen, Zhen Tan, Ajay Kumar Jaiswal, Huaizhi Qu, Xinyu Zhao, Qi Lin, Yu Cheng, Andrew Kwong, Zhichao Cao, Tianlong Chen

Two Heads Are Better Than One: Improving Fake News Video Detection by Correlating with Neighbors (2023.findings-acl)

Challenge:	Bit-flip errors (BFEs) are hardware faults where individual bits in memory or processing units are unintentionally flipped.
Approach:	They propose a novel defense strategy to mitigate bit-flip errors (BFEs) they propose bfe protection and a self-correction mechanism to minimize performance degradation .
Outcome:	The proposed defense strategy minimizes performance degradation while significantly improving robustness against BFEs.

Copied to clipboard

Peng Qi, Yuyang Zhao, Yufeng Shen, Wei Ji, Juan Cao, Tat-Seng Chua

The Mirage of Model Editing: Revisiting Evaluation in the Wild (2025.acl-long)

Challenge:	Existing frameworks for detecting fake news videos are limited . a new approach is proposed to integrate neighborhood information of new videos .
Approach:	They propose a framework for automatically detecting fake news videos . it integrates neighborhood relationship of new videos belonging to same event .
Outcome:	The proposed framework improves performance of existing detectors and graph aggregation and debunking rectification modules.

Copied to clipboard

Wanli Yang, Fei Sun, Jiajun Tan, Xinyu Ma, Qi Cao, Dawei Yin, Huawei Shen, Xueqi Cheng

Question Answering

Challenge:	despite near-perfect results, effectiveness of model editing in real-world applications remains unclear.
Approach:	They propose QAEdit and WILD to better reflect real-world use of model editing . they propose a benchmark aligned with widely used question answering datasets and a task-agnostic evaluation framework .
Outcome:	The proposed QAEdit benchmark and WILD evaluation framework show that current models perform worse than previously reported.

UniICL: An Efficient ICL Framework Unifying Compression, Selection, and Generation (2025.acl-long)

Copied to clipboard

Jun Gao, Qi Lv, Zili Wang, Tianxiang Wu, Ziqiang Cao, Wenjie Li

Responsible & Trustworthy NLP

Challenge:	Existing methods to improve reasoning abilities of Large Language Models (LLMs) have limitations due to excessive growth in context length, causing large hardware burden.
Approach:	They propose a novel Unified ICL framework that unifies demonstration compression, demonstration selection, and final response generation.
Outcome:	The proposed framework unifies demonstration compression, demonstration selection, and final response generation.

Multi-Programming Language Sandbox for LLMs (2025.acl-demo)

Copied to clipboard

Shihan Dou, Jiazheng Zhang, Jianxiang Zang, Yunbo Tao, Weikang Zhou, Haoxiang Jia, Shichun Liu, Yuming Yang, Shenxi Wu, Zhiheng Xi, Muling Wu, Rui Zheng, Changze Lv, Limao Xiong, Shaoqing Zhang, Lin Zhang, Wenyu Zhan, Rongxiang Weng, Jingang Wang, Xunliang Cai, Yueming Wu, Ming Wen, Yixin Cao, Tao Gui, Xipeng Qiu, Qi Zhang, Xuanjing Huang

Probabilistic Tree-of-thought Reasoning for Answering Knowledge-intensive Complex Questions (2023.findings-emnlp)

Challenge:	MPLSandbox is an out-of-the-box multi-programming language sandbox designed to provide unified and comprehensive feedback from compiler and analysis tools for Large Language Models (LLMs).
Approach:	They propose a multi-programming language sandbox that provides unified feedback from compilers and analysis tools for Large Language Models.
Outcome:	The proposed multi-language sandbox can provide comprehensive feedback from compilers and analysis tools for large language models (LLMs).

Copied to clipboard

Shulin Cao, Jiajie Zhang, Jiaxin Shi, Xin Lv, Zijun Yao, Qi Tian, Lei Hou, Juanzi Li

Reasoning

Challenge:	Large language models (LLMs) are capable of answering knowledge-intensive complex questions with chain-of-thought reasoning.
Approach:	They propose a method to solve complex questions with a tree-of-thought approach using parametric knowledge and retrieved external knowledge to augment CoT reasoning.
Outcome:	The proposed approach outperforms SOTA methods on three Complex QA datasets under the open-domain setting.

CoLA: A Choice Leakage Attack Framework to Expose Privacy Risks in Subset Training (2026.acl-long)

Copied to clipboard

Qi Li, Cheng-Long Wang, Yinzhi Cao, Di Wang

Responsible & Trustworthy NLP

Challenge:	Existing threat models underestimate subset-training privacy risks because of the scale of modern datasets.
Approach:	They propose a unified framework for analyzing privacy leakage in subset selection based on side-channel metadata from the subset process or via the outputs of the target model.
Outcome:	The proposed framework analyzes privacy leakage in subset selection based on two different scenarios .

GAPO: Robust Advantage Estimation for Real-World Code LLMs (2026.findings-acl)

Copied to clipboard

Jianqing Zhang, Zhezheng Hao, Wei Xia, Hande Dong, Hong Wang, Chenxing Wei, Yuyan Zhou, Yubin Qi, Qiang Lin, Jian Cao

Robustness in NLP

Challenge:	Reinforcement learning (RL) is widely used for post-training large language models (LLMs) in code editing, but in real-world code editing scenarios, reward distributions are often skewed with unpredictable noise, leading to distorted advantage computation and increased rollout outliers.
Approach:	They propose a group-relative method that finds an interval with the highest SNR and uses the median of that interval as an adaptive Q to replace the group mean in advantage calculation.
Outcome:	The proposed method improves on nine instruction-tuned LLMs while remaining plug-and-play and efficient.

Attribution-Based Analysis and Optimization of Modular Agentic Workflows (2026.findings-acl)

Copied to clipboard

Yingxuan Yang, Bo Huang, Siyuan Qi, Chao Feng, Haoyi Hu, Yuxuan Zhu, Jinbo Hu, Haoran Zhao, Ziyi He, Xiao Liu, ZongYu Wang, Muning Wen, Lin Qiu, Xuezhi Cao, Xunliang Cai, Yong Yu, Weinan Zhang

Challenge:	Large Language Models (LLMs) have driven the rise of agentic workflows . yet, how can we attribute performance gains to individual upgrades and their interactions?
Approach:	They propose a game-theoretic framework that models component upgrades as players and evaluates component coalitions to compute Shapley values.
Outcome:	The proposed framework provides interaction-aware attribution and recommendation for model allocation under a fixed workflow structure.

Evaluating LLMs’ Assessment of Mixed-Context Hallucination Through the Lens of Summarization (2025.findings-acl)

Copied to clipboard

Siya Qi, Rui Cao, Yulan He, Zheng Yuan

Text Generation

Challenge:	Large language models generate coherent text and follow instructions across diverse tasks, but a critical challenge in scaling LLM applications is hallucination, where the generated content lacks factual grounding or deviates from the intended discourse context.
Approach:	They use summarization as a representative task to evaluate LLMs' capability in detecting mixed-context hallucinations, specifically distinguishing between factual and non-factual hallucinos.
Outcome:	The proposed model distinguishes between factual and non-factual hallucinations, and their performance bottlenecks.

SeaKR: Self-aware Knowledge Retrieval for Adaptive Retrieval Augmented Generation (2025.acl-long)

Copied to clipboard

Zijun Yao, Weijian Qi, Liangming Pan, Shulin Cao, Linmei Hu, Liu Weichuan, Lei Hou, Juanzi Li

Information Retrieval

Challenge:	Adaptive Retrieval-Augmented Generation (RAG) is an effective strategy to alleviate hallucination of large language models (LLMs).
Approach:	They propose a novel adaptive RAG model that extracts self-aware uncertainty of large language models from their internal states and invokes retrieval accordingly.
Outcome:	The proposed model outperforms existing adaptive RAG methods on complex and simple Question Answering datasets.

Toward Secure Tuning: Mitigating Security Risks from Instruction Fine-Tuning (2026.acl-long)

Copied to clipboard

Yanrui Du, Fenglei Fan, Sendong Zhao, Jiawei Cao, Ming Ma, Danyang Zhao, Shuren Qi, Ting Liu, Bing Qin

When to Trust LLMs: Aligning Confidence with Response Quality (2024.findings-acl)

Challenge:	Instruction Fine-Tuning (IFT) has emerged as a critical technique for customizing Large Language Models (LLMs) however, recent studies have revealed that IFT can compromise the built-in security mechanisms of LLMs, posing significant security risks.
Approach:	They propose a method that shifts learning burden onto security-robust parameters and propose 'warm-up' phase that preferentially trains Mods_Rob to learn low-level features with minimal security risk.
Outcome:	The proposed method reduces security risks without sacrificing performance gains across knowledge-intensive datasets.

Copied to clipboard

Shuchang Tao, Liuyi Yao, Hanxing Ding, Yuexiang Xie, Qi Cao, Fei Sun, Jinyang Gao, Huawei Shen, Bolin Ding

Challenge:	Existing methods express reliability by confidence level, but lack objective guidance . Existing approaches express reliability but lack guidance on when to trust LLMs .
Approach:	They propose a reward-based approach to align confidence with quality to ensure reliability . they propose 'conqORD' to help model to verbalize greater confidence for higher quality responses .
Outcome:	Experiments show that CONQORD significantly improves confidence and response accuracy . the proposed approach can be used to determine reliability of large language models .

Anchored Cyclic Generation: A Novel Paradigm for Long-Sequence Symbolic Music Generation (2026.findings-acl)

Copied to clipboard

Boyu Cao, Lekai Qian, Dehan Li, Haoyu Gu, Mingda Xu, Qi Liu

PKAD: Pretrained Knowledge is All You Need to Detect and Mitigate Textual Backdoor Attacks (2024.findings-emnlp)

Challenge:	Autoregressive sequence modeling has been successful in many domains, but maintaining long-term coherence and structural integrity remains a challenge.
Approach:	They propose an ACG paradigm that relies on anchor features from previously generated musical content to guide subsequent generation during the autoregressive process.
Outcome:	The proposed framework outperforms existing methods in symbolic music generation tasks.

Copied to clipboard

Yu Chen, Qi Cao, Kaike Zhang, Xuchao Liu, Huawei Shen

Robustness in NLP

Challenge:	Current defense methods can be classified into inference-time and training-time ones based on their execution phase.
Approach:	They propose a two-stage poison detection strategy using pre-trained language models to detect poisoned samples before model training.
Outcome:	The proposed method achieves better performance than current methods more quickly and with fewer training costs.

Answer When Needed, Forget When Not: Language Models Pretend to Forget via In-Context Knowledge Unlearning (2025.findings-acl)

Copied to clipboard

Shota Takashiro, Takeshi Kojima, Andrew Gambardella, Qi Cao, Yusuke Iwasawa, Yutaka Matsuo

Hi-ArG: Exploring the Integration of Hierarchical Argumentation Graphs in Language Pretraining (2023.emnlp-main)

Challenge:	Large language models (LLMs) are increasingly required to selectively unlearn specific information.
Approach:	They propose a method which fine-tunes pre-trained LLMs to enable prompt unlearning of target knowledge within the context while preserving unrelated information.
Outcome:	The proposed method achieves up to 95% forget accuracy while retaining 80% of unrelated knowledge, significantly outperforming baselines in both in-domain and out-of-domain scenarios.

Copied to clipboard

Jingcong Liang, Rong Ye, Meng Han, Qi Zhang, Ruofei Lai, Xinyu Zhang, Zhao Cao, Xuanjing Huang, Zhongyu Wei

ChemAmp: Amplified Chemistry Tools via Composable Agents (2026.findings-acl)

Challenge:	Recent studies have discussed its capability to assist language models for various applications.
Approach:	They propose a structure to organize arguments using the Hierarchical Argumentation Graph (Hi-ArG) and propose two approaches to exploit Hi-AarG, including a text-graph multi-modal model GreaseArR and a framework augmented with graph information.
Outcome:	The proposed structure supersedes existing language models on two argumentation tasks while incorporating graph information during further training improves vanilla language models.

Copied to clipboard

Zhucong Li, Powei Chang, Jin Xiao, Zhijian Zhou, Qianyu He, Jiaqing Liang, Fenglei Cao, Xu Yinghui, Yuan Qi

Challenge:	LLM-based agents are powerful tools for automating complex scientific workflows, especially in chemistry, but their single-task performance is limited by tool constraints.
Approach:	They propose a framework that optimizes the collective capabilities of specialized tools by dynamic coordination within individual tasks.
Outcome:	The proposed framework outperforms chemistry-specialized models, generalist LLMs, and agent systems with tool orchestration.

Reasoning over Hierarchical Question Decomposition Tree for Explainable Question Answering (2023.acl-long)

Copied to clipboard

Jiajie Zhang, Shulin Cao, Tingjian Zhang, Xin Lv, Juanzi Li, Lei Hou, Jiaxin Shi, Qi Tian

Question Answering

Challenge:	Existing XQA methods focus on reasoning on a single knowledge source, e.g., structured knowledge bases, unstructured corpora, etc. Existing work in XQA focuses on integrating information from heterogeneous knowledge sources.
Approach:	They propose to leverage question decomposing for heterogeneous knowledge integration by breaking down a complex question into simpler ones and selecting the appropriate knowledge source for each sub-question.
Outcome:	The proposed framework outperforms SOTA methods on complex QA datasets.

Improving the Language Understanding Capabilities of Large Language Models Using Reinforcement Learning (2025.findings-emnlp)

Copied to clipboard

Sai Ashish Somayajula, Bokai Hu, Qi Cao, Xin Pan, Pengtao Xie

Too Consistent to Detect: A Study of Self-Consistent Errors in LLMs (2025.emnlp-main)

Challenge:	Instruction-fine-tuned large language models (LLMs) under 14B parameters underperform on NLU tasks . we explore a framework to improve the NLU capabilities of LLMs .
Approach:	They propose to use Proximal Policy Optimization to improve NLU capabilities . they frame NLU as a reinforcement learning environment and optimize for reward signals .
Outcome:	The proposed framework outperforms supervised fine-tuning on GLUE and superGLUE tasks.

Copied to clipboard

Hexiang Tan, Fei Sun, Sha Liu, Du Su, Qi Cao, Xin Chen, Jingang Wang, Xunliang Cai, Yuanzhuo Wang, Huawei Shen, Xueqi Cheng

Challenge:	Existing detection methods fail to account for self-consistent error . study identifies self-consistency errors and evaluates them .
Approach:	They propose a method that fuses hidden state evidence from an external verifier LLM to detect self-consistent errors.
Outcome:	The proposed method significantly enhances performance on self-consistent errors across three LLM families.

Blinded by Generated Contexts: How Language Models Merge Generated and Retrieved Contexts When Knowledge Conflicts? (2024.acl-long)

Copied to clipboard

Hexiang Tan, Fei Sun, Wanli Yang, Yuanzhuo Wang, Qi Cao, Xueqi Cheng

Clustered Self-Assessment: A Simple yet Effective Method for Uncertainty Quantification in Large Language Models (2026.findings-acl)

Challenge:	Recent advances in augmenting Large Language Models (LLMs) with auxiliary information have significantly revolutionized their efficacy in knowledge-intensive tasks.
Approach:	They propose a systematic framework to identify whether LLMs’ responses are attributed to either generated or retrieved contexts.
Outcome:	The proposed framework identifies whether LLMs’ responses are attributed to either generated or retrieved contexts.

Copied to clipboard

Qi Cao, Takeshi Kojima, Andrew Gambardella, Helinyi Peng, Yutaka Matsuo, Yusuke Iwasawa

Text Anomaly Detection with Simplified Isolation Kernel (2025.findings-emnlp)

Challenge:	Existing methods for uncertainty quantification in large language models rely on indirect signals, such as entropy across sampled generations, which can be difficult to interpret and do not fully leverage the model’s ability to assess its own uncertainty.
Approach:	They propose a method that groups sampled generations into semantically distinct clusters and uses the probability assigned by the LLM to each option as a confidence estimate.
Outcome:	The proposed method outperforms baseline methods and achieves competitive performance with as few as two additional samples.

Copied to clipboard

Yang Cao, Sikun Yang, Yujiu Yang, Lianyong Qi, Ming Liu

GraphQ IR: Unifying the Semantic Parsing of Graph Query Languages with One Intermediate Representation (2022.emnlp-main)

Challenge:	High-dimensional dense embeddings extracted by large language models pose memory requirements and high computation time.
Approach:	They propose a method that maps high-dimensional dense embeddings to lower-dimensional sparse representations while preserving crucial anomaly characteristics.
Outcome:	The proposed method achieves better detection performance than 11 SOTA anomaly detection algorithms while maintaining computational efficiency and low memory cost.

Copied to clipboard

Lunyiu Nie, Shulin Cao, Jiaxin Shi, Jiuding Sun, Qi Tian, Lei Hou, Juanzi Li, Jidong Zhai

XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation (2020.emnlp-main)

Challenge:	Existing approaches to neural semantic parsing are limited by the semantic gap between natural and formal languages.
Approach:	They propose a unified intermediate representation for graph query languages, named GraphQ IR, which has a natural-language-like expression that bridges the semantic gap and formally defined syntax that maintains the graph structure.
Outcome:	The proposed representation can convert user queries into graphQ IR, which can later be losslessly compiled into various downstream graph query languages.

Copied to clipboard

Yaobo Liang, Nan Duan, Yeyun Gong, Ning Wu, Fenfei Guo, Weizhen Qi, Ming Gong, Linjun Shou, Daxin Jiang, Guihong Cao, Xiaodong Fan, Ruofei Zhang, Rahul Agrawal, Edward Cui, Sining Wei, Taroon Bharti, Ying Qiao, Jiun-Hung Chen, Winnie Wu, Shuguang Liu, Fan Yang, Daniel Campos, Rangan Majumder, Ming Zhou

Multilinguality

Challenge:	XGLUE provides a benchmark dataset to train large-scale cross-lingual pre-trained models . XCLUE provides 11 diversified tasks that cover both understanding and generation scenarios .
Approach:	They introduce a new benchmark dataset to train large-scale cross-lingual pre-trained models using multilingual and bilingual corpora.
Outcome:	The proposed dataset is labeled in English and includes only natural language understanding tasks.

SimRPD: Optimizing Recruitment Proactive Dialogue Agents through Simulator-Based Data Evaluation and Selection (2026.acl-industry)

Copied to clipboard

Zhiyong Cao, Dunqiang Liu, Qi Dai, Haojun Xu, Huai Yuen Khor, Hao Wang, Huan He, Yafei Liu, Ke Ma, Ruqian Shi, Sicheng Zhou, Sijia Yao

Dialogue Systems & Conversational Agents

Challenge:	High-quality data in training proactive dialogue agents is scarce, despite fine-tuning and reinforcement learning . a recent study has shown that the effectiveness of supervised fine-touring is limited by the lack of high-quality, domain-specific training data.
Approach:	They propose a framework for training recruitment proactive dialogue agents using a high-fidelity user simulator and a multi-dimensional evaluation framework based on Chain-of-Intention.
Outcome:	The proposed framework outperforms existing simulator-based data selection strategies in a real-world recruitment scenario.

Semantic Token Clustering for Efficient Uncertainty Quantification in Large Language Models (2026.eacl-short)

Copied to clipboard

Qi Cao, Andrew Gambardella, Takeshi Kojima, Yutaka Matsuo, Yusuke Iwasawa

Low-Entropy Watermark Detection via Bayes’ Rule Derived Detector (2025.findings-acl)

Challenge:	Large language models have limited truthfulness and tendency toward overconfidence constrain reliability in factual tasks.
Approach:	They propose an efficient method that leverages semantic information encoded in LLMs to quantify uncertainty.
Outcome:	The proposed method achieves comparable performance to baselines while significantly reducing computational overhead.

Copied to clipboard

Beining Huang, Du Su, Fei Sun, Qi Cao, Huawei Shen, Xueqi Cheng

Numerical Reasoning

Challenge:	Existing methods for text watermarking ignore strong evidences embedded in low-entropy tokens, causing statistical measures to falsely indicate the absence of a watermark.
Approach:	They propose a Bayes' Rule derived watermark Detector which exploits watermark information from every token by leveraging the posterior probability of watermark’s presence.
Outcome:	The proposed method achieves 50% and 70% relative improvements over baselines in code generation and math problem-solving tasks.

Towards Better Entity Linking with Multi-View Enhanced Distillation (2023.acl-long)

Copied to clipboard

Yi Liu, Yuan Tian, Jianxun Lian, Xinlong Wang, Yanan Cao, Fang Fang, Wen Zhang, Haizhen Huang, Weiwei Deng, Qi Zhang

From Scores to Preferences: Redefining Evaluation Paradigm for Speech Quality Reward Modeling (2026.findings-acl)

Challenge:	Entity linking is a fundamental task in Natural Language Processing (NLP), connecting mentions within unstructured contexts to their corresponding entities in a Knowledge Base (KB).
Approach:	They propose a dual-encoder framework that can efficiently match mentions to two-encoding frameworks by a global-view.
Outcome:	The proposed framework achieves state-of-the-art on several entity linking benchmarks.

Copied to clipboard

Yifei Cao, Changhao Jiang, Jiabao Zhuang, Jiajun Sun, Ming Zhang, Zhiheng Xi, Hui Li, Shihan Dou, Yuran Wang, Yunke Zhang, Tao Ji, Tao Gui, Qi Zhang, Xuanjing Huang

Unnatural Error Correction: GPT-4 Can Almost Perfectly Handle Unnatural Scrambled Text (2023.emnlp-main)

Challenge:	Experimental results show that the MOS-aware GRM significantly improves fine-grained speech quality discrimination.
Approach:	They propose a MOS-aware reward model that incorporates MOS gap into reward function during reinforcement learning.
Outcome:	The proposed model significantly improves fine-grained speech quality discrimination.

Copied to clipboard

Qi Cao, Takeshi Kojima, Yutaka Matsuo, Yusuke Iwasawa

EvoWiki: Evaluating LLMs on Evolving Knowledge (2025.acl-long)

Challenge:	Large language models exhibit remarkable proficiency in many tasks, but much about their inner workings remains unclear.
Approach:	They propose a test suite to measure the resilience of Large Language Models (LLMs) when subjected to extensive character-level permutations.
Outcome:	The proposed test suite measures the resilience of large language models to scrambled inputs.

Copied to clipboard

Wei Tang, Yixin Cao, Yang Deng, Jiahao Ying, Bo Wang, Yizhe Yang, Yuyue Zhao, Qi Zhang, Xuanjing Huang, Yu-Gang Jiang, Yong Liao

Challenge:	Existing knowledge evolution benchmarks are static and fail to capture the evolving nature of LLMs and knowledge.
Approach:	They propose an evolving dataset that categorizes information into stable, evolved, and uncharted states.
Outcome:	The proposed dataset is auto-updatable and enables evaluation of continuously changing knowledge and newly released LLMs.

Beyond Scaling: Measuring and Predicting the Upper Bound of Knowledge Retention in Language Model Pre-Training (2026.acl-long)

Copied to clipboard

Changhao Jiang, Ming Zhang, Yifei Cao, Junjie Ye, Xiaoran Fan, Shihan Dou, Zhiheng Xi, Jiajun Sun, Yi Dong, Yujiong Shen, Jingqi Tong, Baoyu Fan, Tao Gui, Qi Zhang, Xuanjing Huang