Papers by Qi Luo

A Goal Without a Plan Is Just a Wish: Efficient and Effective Global Planner Training for Long-Horizon Agent Tasks (2026.acl-long)

Challenge:	Existing methods to optimise pretraining performance have not addressed the complexities of domain-adaptive continual pretraining.
Approach:	They propose a framework that dynamically assesses learning velocity and adjusts data proportions accordingly, favouring slower learning domains while de-emphasising faster learning ones.
Outcome:	The proposed framework achieves performance gains in math and code reasoning tasks and command-line generation benchmarks.

Copied to clipboard

Shuzheng Si, Haozhe Zhao, Kangyang Luo, Gang Chen, Fanchao Qi, Minjia Zhang, Baobao Chang, Maosong Sun

Green & Sustainable NLP

Challenge:	Recent advances in large language models (LLMs) have leapt from static chatbots to versatile agents that tackle complex tasks such as science experiments.
Approach:	They propose a plan-and-execute framework and propose 'EAGLET' to enhance the executor agent's planning abilities without human effort.
Outcome:	The proposed method outperforms existing methods on three long-horizon tasks and reduces training costs by 8 compared to baselines.

Aegis:An Advanced LLM-Based Multi-Agent for Intelligent Functional Safety Engineering (2024.emnlp-industry)

Copied to clipboard

Lu Shi, Bin Qi, Jiarui Luo, Yang Zhang, Zhanzhao Liang, Zhaowei Gao, Wenke Deng, Lin Sun

Information Retrieval

Challenge:	Aegis is an advanced LLM-based multi-agent for intelligent functional safety engineering that can perform all phases of a vehicle's lifecycle, including design, development, production, operation, and decommissioning.
Approach:	They introduce Aegis: An Advanced LLM-Based Multi-Agent for Intelligent Functional Safety Engineering.
Outcome:	The proposed solution can perform Hazard Analysis and Risk Assessment (HARA), document Functional Safety Requirements (FSR), and plan test cases for Automatic Emergency Braking (AEB) systems.

FaithLens: Detecting and Explaining Faithfulness Hallucination (2026.findings-acl)

Copied to clipboard

Shuzheng Si, Qingyi Wang, Haozhe Zhao, Yuzhuo Bai, Guanqiao Chen, Kangyang Luo, Gang Chen, Fanchao Qi, Minjia Zhang, Baobao Chang, Maosong Sun

Responsible & Trustworthy NLP

Challenge:	Recent progress in large language models (LLMs) has revolutionized text generation.
Approach:	They propose a faithfulness hallucination detection model that can provide binary predictions and corresponding explanations to improve trustworthiness.
Outcome:	The proposed model outperforms advanced models on 12 diverse tasks.

LongTutor: Benchmarking Large Language Models for Long-term Personalized Tutoring (2026.acl-long)

Copied to clipboard

Ning Li, Zheng Zhang, Zhenya Huang, Rui Li, Yi Zhan, Yinbo Luo, Qi Liu, Enhong Chen

An Auto-Encoder Matching Model for Learning Utterance-Level Semantic Dependency in Dialogue Generation (D18-1)

Challenge:	Existing evaluations focus on isolated, short-term interactions, overlooking the inherently long-term nature of learning.
Approach:	They propose a benchmark for long-term personalized tutoring based on an annotated learning log . they propose an automated generator–verifier pipeline to enable benchmark expansion .
Outcome:	The proposed benchmarks evaluate LLMs across three progressive tasks: evidence acquisition, knowledge state diagnosis, and adaptive teaching action.

Copied to clipboard

Liangchen Luo, Jingjing Xu, Junyang Lin, Qi Zeng, Xu Sun

Dialogue Systems & Conversational Agents

Challenge:	Experimental results show that our model can generate semantically coherent responses compared to baseline models.
Approach:	They propose an Auto-Encoder Matching model to learn utterance-level semantic dependency . their model contains two auto-encoders and one mapping module .
Outcome:	Experimental results show that the proposed model can generate high coherence and fluency compared to baseline models.

ChartCoder: Advancing Multimodal Large Language Model for Chart-to-Code Generation (2025.acl-long)

Copied to clipboard

Xuanle Zhao, Xianzhen Luo, Qi Shi, Chi Chen, Shuo Wang, Zhiyuan Liu, Maosong Sun

Document Segmentation Matters for Retrieval-Augmented Generation (2025.findings-acl)

Challenge:	Existing open-source MLLMs fail to fully capture dense information embedded in charts . current models still face significant challenges in understanding and analyzing visual tasks such as captioning and question answering.
Approach:	They propose a chart-to-code MLLM which leverages Code LLMs as the language backbone to enhance the executability of the generated code.
Outcome:	The proposed model surpasses existing open-source models on chart-to-code benchmarks with only 7B parameters and provides lossless representations that contain all critical details.

Copied to clipboard

Zhitong Wang, Cheng Gao, Chaojun Xiao, Yufei Huang, Shuzheng Si, Kangyang Luo, Yuzhuo Bai, Wenhao Li, Tangjian Duan, Chuancheng Lv, Guoshan Lu, Gang Chen, Fanchao Qi, Maosong Sun

Syntactic Text Processing

Challenge:	Existing rule-based chunking methods lead to suboptimal splits, where overly large chunks introduce irrelevant information and small chunks lack semantic coherence.
Approach:	They propose a method that leverages document summaries as pseudo-instructions to guide chunking by computing semantic similarity between sentences and the summary.
Outcome:	Experiments on multiple open-domain question-answering benchmarks show that PIC significantly improves retrieval accuracy (Hits@k) and end-to-end QA performance (Exact Match) without any additional training.

Aurora-M: Open Source Continual Pre-training for Multilingual Language and Code (2025.coling-industry)

Copied to clipboard

Taishi Nakamura, Mayank Mishra, Simone Tedeschi, Yekun Chai, Jason T. Stillerman, Felix Friedrich, Prateek Yadav, Tanmay Laud, Vu Minh Chien, Terry Yue Zhuo, Diganta Misra, Ben Bogin, Xuan-Son Vu, Marzena Karpinska, Arnav Varma Dantuluri, Wojciech Kusa, Tommaso Furlanello, Rio Yokota, Niklas Muennighoff, Suhas Pai, Tosin Adewumi, Veronika Laippala, Xiaozhe Yao, Adalberto Barbosa Junior, Aleksandr Drozd, Jordan Clive, Kshitij Gupta, Liangyu Chen, Qi Sun, Ken Tsui, Nour Moustafa-Fahmy, Nicolo Monti, Tai Dang, Ziyang Luo, Tien-Tung Bui, Roberto Navigli, Virendra Mehta, Matthew Blumberg, Victor May, Hiep Nguyen, Sampo Pyysalo

Parallelism and Generation Order in Masked Diffusion Language Models: Limits Today, Potential Tomorrow (2026.findings-acl)

Challenge:	Pretrained language models are integral part of AI applications, but their high computational cost limits accessibility.
Approach:	They evaluate Aurora-M, a 15B parameter multilingual open-source model trained on English, Finnish, Hindi, Japanese, Vietnamese, and code.
Outcome:	The proposed model outperforms existing models on English, Finnish, Hindi, Japanese, Vietnamese, and code.

Copied to clipboard

Yangyang Zhong, Yanmei Gu, Zhengqing Zang, Xiaomeng Li, Yuqi Ding, Xibei Jia, Yuting Shen, Zhenzhong Lan, Liwang Zhu, Weiping Liu, Junlin Zhou, Haisheng Liu, Zhong Xin Yu, Pengxin Luo, Donglian Qi, Yunfeng Yan, Junbo Zhao

Experience is the Teacher: Reusing Atomic Thoughts from LLMs to Improve Medical Dialogue (2026.findings-acl)

Challenge:	Autoregressive (AR) language models dominate modern natural language processing due to strong likelihood-based training objectives and reliable left-to-right decoding.
Approach:	They characterize MDLM behavior along two dimensions: parallelism strength and generation order . authors propose a Generate-then-Edit paradigm that mitigates dependency loss .
Outcome:	The proposed model improves on tasks that require "backward information" the Generate-then-Edit paradigm improves parallel decoding efficiency while reducing dependency loss.

Copied to clipboard

Guangya Yu, Hui Luo, Qi Ye, Ruihui Hou, Weiyan Zhang, Mingxi Shang, Xuanwu Li, ChunMing Wang, Tong Ruan

Dialogue Systems & Conversational Agents

Challenge:	Recent large reasoning models (LLMs) lack dynamic and diverse thinking capabilities . reusing atomic thoughts provides a practical pathway toward dynamic reasoning .
Approach:	They propose a framework that extracts atomic thoughts from teacher models and reuses them to guide reasoning and generate responses.
Outcome:	The proposed framework extracts atomic thoughts from teacher models and reuses them to guide reasoning and generate responses.

Stepwise Perplexity-Guided Refinement for Efficient Chain-of-Thought Reasoning in Large Language Models (2025.findings-acl)

Copied to clipboard

Yingqian Cui, Pengfei He, Jingying Zeng, Hui Liu, Xianfeng Tang, Zhenwei Dai, Yan Han, Chen Luo, Jing Huang, Zhen Li, Suhang Wang, Yue Xing, Jiliang Tang, Qi He

MARS-Bench: A Multi-turn Athletic Real-world Scenario Benchmark for Dialogue Evaluation (2025.findings-emnlp)

Challenge:	Chain-of-Thought (CoT) reasoning has improved the performance of large language models (LLMs) however, the detailed reasoning process in CoT often incurs long generation times and high computational costs due to the inclusion of unnecessary steps.
Approach:	They propose a method to identify critical reasoning steps using perplexity as a measure of their importance.
Outcome:	The proposed method achieves a better balance between reasoning accuracy and efficiency of CoT.

Copied to clipboard

Chenghao Yang, Yinbo Luo, Zhoufutu Wen, Qi Chu, Tao Gong, Longxiang Liu, Kaiyuan Zhang, Jianpeng Jiao, Ge Zhang, Wenhao Huang, Nenghai Yu

Dialogue Systems & Conversational Agents

Challenge:	Large Language Models (LLMs) have been widely adopted in real-world dialogue applications, but their robustness is criticized all along.
Approach:	They propose to use play-by-play text commentary to build a multi-turn athletic real-world scenario dialogue benchmark to evaluate three critical aspects of multi-turned conversations: ultra multi- turn, interactive multi-twist, and cross-turn tasks.
Outcome:	The proposed benchmarks outperform open-source LLMs on three critical aspects of multi-turn conversations: ultra multi-turned, interactive multi- turn, and cross-turn tasks.

LLM4Decompile: Decompiling Binary Code with Large Language Models (2024.emnlp-main)

Copied to clipboard

Hanzhuo Tan, Qi Luo, Jing Li, Yuqun Zhang

XAL: EXplainable Active Learning Makes Classifiers Better Low-resource Learners (2024.naacl-long)

Challenge:	Decompilation aims to convert binary code to high-level source code, but traditional tools like Ghidra often produce results that are difficult to read and execute.
Approach:	They propose an open-source LLM series trained to decompile binary code . they optimize the LLM training process and introduce the Llm4Decompile-End models .
Outcome:	The proposed models outperform GPT-4o and Ghidra on the HumanEval and ExeBench benchmarks by over 100% in terms of re-executability rate.

Copied to clipboard

Yun Luo, Zhen Yang, Fandong Meng, Yingjie Li, Fang Guo, Qinglin Qi, Jie Zhou, Yue Zhang

Responsible & Trustworthy NLP

Challenge:	Existing methods for active learning rely on model uncertainty or disagreement to pick unlabeled data, leading to over-confidence in superficial patterns and lack of exploration.
Approach:	They propose to use a bi-directional encoder and a uni-directional decoder to generate and score an explanation for low-resource text classification.
Outcome:	The proposed model improves on 9 strong baselines on six datasets and can generate explanations for its predictions.

AutoSchemaKG: Autonomous Knowledge Graph Construction through Dynamic Schema Induction from Web-Scale Corpora (2026.acl-long)

Copied to clipboard

Jiaxin Bai, Wei Fan, Qi Hu, Qing Zong, Chunyang Li, Hong Ting Tsang, Hongyu Luo, Yauwai Yim, Haoyu Huang, Xiao Zhou, Feng Qin, Tianshi Zheng, Xi Peng, Xin Yao, Huiwen Yang, Leijie Wu, JI Yi, Gong Zhang, Renhai Chen, Yangqiu Song

SciCustom: A Framework for Custom Evaluation of Scientific Capabilities in Large Language Models (2026.acl-long)

Challenge:	Existing knowledge graph construction frameworks require predefined schemas, limiting their scalability and domain coverage.
Approach:	They propose a framework for fully autonomous knowledge graph construction that eliminates the need for predefined schemas.
Outcome:	The proposed framework outperforms state-of-the-art models on multi-hop QA tasks and enhances LLM factuality.

Copied to clipboard

Yiyang Gu, Junwei Yang, Junyu Luo, Ye Yuan, Bin Feng, Yingce Xia, Shufang Xie, Kaili Liu, Bohan Wu, Qi Shi, Haoran Li, Beier Xiao, Zhiping Xiao, Xiao Luo, Weizhi Zhang, Philip S. Yu, Zequn Liu, Ming Zhang

SimRAG: Self-Improving Retrieval-Augmented Generation for Adapting Large Language Models to Specialized Domains (2025.naacl-long)

Challenge:	Existing evaluations of large language models fail to reflect fine-grained capabilities . existing benchmarks are manually curated or domain-generic, limiting scalability and alignment with real use cases.
Approach:	They propose a framework that allows custom construction of benchmarks from large-scale scientific data to evaluate application-specific scientific capabilities in LLMs.
Outcome:	The proposed framework reveals fine-grained differences in scientific capabilities that standard benchmarks overlook . it allows custom construction of benchmarks from large-scale scientific data to evaluate application-specific capabilities in LLMs.

Copied to clipboard

Ran Xu, Hui Liu, Sreyashi Nag, Zhenwei Dai, Yaochen Xie, Xianfeng Tang, Chen Luo, Yang Li, Joyce C. Ho, Carl Yang, Qi He

Natural Language Interfaces

Challenge:	Retrieval-augmented generation (RAG) enhances the question answering abilities of large language models (LLMs) however, adapting general-purpose RAG systems to specialized fields poses unique challenges due to distribution shifts and limited access to domain-specific data.
Approach:	They propose a method that equips large language models with joint capabilities of question answering and question generation for domain adaptation.
Outcome:	Experiments on 11 datasets across three different domains verify the efficacy of SimRAG over baselines by 1.2%–8.6%.

ChartEdit: How Far Are MLLMs From Automating Chart Analysis? Evaluating MLLMs’ Capability via Chart Editing (2025.findings-acl)

Copied to clipboard

Xuanle Zhao, Xuexin Liu, Yang Haoyue, Xianzhen Luo, Fanhu Zeng, Jianling Li, Qi Shi, Chi Chen

OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems (2024.acl-long)

Challenge:	Existing evaluations of multimodal large language models rely on limited case studies . however, they lack the ability to generate accurate edits according to the instructions .
Approach:	They propose a benchmark for chart editing that includes 1,405 edit instructions applied to 233 real-world charts.
Outcome:	The proposed benchmark includes 1,405 diverse editing instructions applied to 233 real-world charts.

Copied to clipboard

Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, Maosong Sun

GA-SAM: Gradient-Strength based Adaptive Sharpness-Aware Minimization for Improved Generalization (2022.emnlp-main)

Challenge:	Large Language Models (LLMs) and Large Multimodal Models have exceeded general human capabilities in various tasks.
Approach:	They present an Olympiad-level bilingual multimodal scientific benchmark featuring 8,476 problems from Olympiad level mathematics and physics competitions.
Outcome:	The best performing model, GPT-4V, attains an average score of 17.97% on OlympiadBench, with a mere 10.74% in physics, highlighting the benchmark rigor and the intricacy of physical reasoning.

Copied to clipboard

Zhiyuan Zhang, Ruixuan Luo, Qi Su, Xu Sun

TP-Detector: Detecting Turning Points in the Engineering Process of Large-scale Projects (2023.emnlp-demo)

Challenge:	Recent studies show flat minima tend to imply better generalization abilities . however, it has some difficulty implying SAM to some natural language tasks .
Approach:	They propose a flatness-aware minimization algorithm that can be applied to natural language tasks . they propose to use parameter corruptions to explain why flat minima generalize better .
Outcome:	The proposed algorithm can generalize better for flat minima that are robust against corruptions or perturbations.

Copied to clipboard

Qi Wu, WenHan Chao, Xian Zhou, Zhunchen Luo

Challenge:	Extensive experiments demonstrate the effectiveness of our proposed method on a constructed dataset compared to baseline methods.
Approach:	They propose a novel task of detecting turning points in the engineering process of large-scale projects by treating news streams as a window with multiple instances.
Outcome:	The proposed mode is able to detect transitions in news streams with multiple instances.

Enhancing LLM-based Search Agents via Contribution Weighted Group Relative Policy Optimization (2026.acl-long)

Copied to clipboard

Junzhe Wang, Zhiheng Xi, Yajie Yang, Hao Luo, Shihan Dou, Tao Gui, Qi Zhang

Information Retrieval

Challenge:	Existing approaches to training large language models suffer from unstable value estimation, whereas outcome supervision struggles with credit assignment due to sparse, trajectory-level rewards.
Approach:	They propose a framework that integrates process supervision into group relative policy optimization.
Outcome:	The proposed framework outperforms standard GRPO on knowledge-intensive benchmarks by 5.0% and 6.3% on Qwen3-1.7B.

R3-RAG: Learning Step-by-Step Reasoning and Retrieval for LLMs via Reinforcement Learning (2025.findings-emnlp)

Copied to clipboard

Yuan Li, Qi Luo, Xiaonan Li, Bufan Li, Qinyuan Cheng, Bo Wang, Yining Zheng, Yuxin Wang, Zhangyue Yin, Xipeng Qiu

Aligning Large Language Models to Follow Instructions and Hallucinate Less via Effective Data Filtering (2025.acl-long)

Challenge:	RAG systems that integrate external knowledge with Large Language Models often become bottlenecks due to their limited parameters compared to LLMs and their inability to perform step-by-step reasoning.
Approach:	They propose a model that integrates external knowledge with Large Language Models to enhance factual correctness and mitigate hallucination.
Outcome:	The proposed model outperforms baselines and can transfer well to different retrievers.

Copied to clipboard

Shuzheng Si, Haozhe Zhao, Gang Chen, Cheng Gao, Yuzhuo Bai, Zhitong Wang, Kaikai An, Kangyang Luo, Chen Qian, Fanchao Qi, Baobao Chang, Maosong Sun

CENTAUR: Bridging the Impossible Trinity of Privacy, Efficiency, and Performance in Privacy-Preserving Transformer Inference (2025.acl-long)

Challenge:	Existing studies show that training LLMs on data containing unfamiliar knowledge during instruction tuning can encourage hallucinations.
Approach:	They propose a framework that measures how familiar the LLM is with instruction data and introduce an expert-aligned reward model to ensure the quality of selected samples.
Outcome:	The proposed framework reduces hallucinations while maintaining a competitive ability to follow instructions.

Copied to clipboard

Jinglong Luo, Guanzhong Chen, Yehong Zhang, Shiyu Liu, Hui Wang, Yue Yu, Xun Zhou, Yuan Qi, Zenglin Xu

Responsible & Trustworthy NLP

Challenge:	Existing privacy-preserving Transformer Inference frameworks suffer from high computational overhead and performance losses.
Approach:	They propose a framework that integrates random permutations and SMPC to address the "impossible trinity" CENTAUR resists diverse data reconstruction attacks and boosts inference speed by 5.030.4 times .
Outcome:	CENTAUR achieves an unprecedented balance between privacy, efficiency, and performance.

EcomScriptBench: A Multi-task Benchmark for E-commerce Script Planning via Step-wise Intention-Driven Product Association (2025.acl-long)

Copied to clipboard

Weiqi Wang, Limeng Cui, Xin Liu, Sreyashi Nag, Wenju Xu, Chen Luo, Sheikh Muhammad Sarwar, Yang Li, Hansu Gu, Hui Liu, Changlong Yu, Jiaxin Bai, Yifan Gao, Haiyang Zhang, Qi He, Shuiwang Ji, Yangqiu Song

Grammar-Based Code Representation: Is It a Worthy Pursuit for LLMs? (2025.findings-acl)

Challenge:	Goal-oriented script planning is used by humans to plan for typical activities . however, this capability remains underexplored due to several challenges .
Approach:	They propose a framework that enables product-enriched scripts by associating products with each step based on the semantic similarity between the actions and their purchase intentions.
Outcome:	The proposed framework can generate product-enriched scripts from 2.4 million scripts . human annotations are conducted to provide gold labels for a sampled subset .

Copied to clipboard

Qingyuan Liang, Zhao Zhang, Zeyu Sun, Zheng Lin, Qi Luo, Xiao Yueyi, Yizhou Chen, Yuqun Zhang, Haotian Zhang, Lu Zhang, Chenbin Chenbin, Yingfei Xiong

Cognitive Alpha Mining via LLM-Driven Code-Based Evolution (2026.acl-long)

Challenge:	Existing research demonstrates the effectiveness of grammar-based code representations in small-scale models, showing their ability to reduce syntax errors and enhance performance.
Approach:	They develop a series of billion-scale grammar-based code representations that incorporate grammar rules into the code generation process.
Outcome:	Experiments on HumanEval and MBPP show that grammar-based representations reduce syntax errors and improve performance even in billion-scale models.

Copied to clipboard

Fengyuan Liu, Yi Huang, Sichun Luo, Yuqi Wang, Yazheng Yang, Xinye Li, Zefa Hu, Junlan Feng, Qi Liu

Reasoning

Challenge:	Existing approaches to finding effective predictive signals from financial data are limited by their complexity and low signal-to-noise ratio.
Approach:	They propose a framework that combines code-level alpha representation with LLM-driven reasoning and evolutionary search.
Outcome:	The proposed framework combines code-level alpha representation with LLM-driven reasoning and evolutionary search.

OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models (2025.acl-long)

Copied to clipboard

Siming Huang, Tianhao Cheng, Jason Klein Liu, Weidi Xu, Jiaran Hao, Liuyihan Song, Yang Xu, Jian Yang, Jiaheng Liu, Chenchen Zhang, Linzheng Chai, Ruifeng Yuan, Xianzhen Luo, Qiufeng Wang, YuanTao Fan, Qingfu Zhu, Zhaoxiang Zhang, Yang Gao, Jie Fu, Qian Liu, Houyi Li, Ge Zhang, Yuan Qi, Xu Yinghui, Wei Chu, Zili Wang

Is Cognition Consistent with Perception? Assessing and Mitigating Multimodal Knowledge Conflicts in Document Understanding (2025.emnlp-main)

Challenge:	Code LLMs lack reproducible data pipelines and training protocols for reproducible advancements in code intelligence.
Approach:	They propose a top-tier code LLM that releases model weights and inference code . reproducible data pipelines, rigorous experimental ablation results and training protocols are included .
Outcome:	The proposed model achieves comparable performance to leading models and serves as an "open cookbook" reproducible training data, rigorous experimental ablation results, and detailed training protocols are also included in the model.

Copied to clipboard

Zirui Shao, Feiyu Gao, Zhaoqing Zhu, Chuwei Luo, Hangdi Xing, Zhi Yu, Qi Zheng, Ming Yan, Jiajun Bu

FaithBench: A Diverse Hallucination Benchmark for Summarization by Modern LLMs (2025.naacl-short)

Challenge:	Multimodal large language models (MLLMs) have shown impressive capabilities in document understanding due to different types of annotation noise in training.
Approach:	They propose a method to reduce C&P knowledge conflicts across all tested MLLMs . they propose to use annotation noise to train models to understand document content .
Outcome:	The proposed method reduces C&P knowledge conflicts across all tested MLLMs and enhances their performance in both cognitive and perceptual tasks.

Copied to clipboard

Forrest Sheng Bao, Miaoran Li, Renyi Qu, Ge Luo, Erana Wan, Yujia Tang, Weisi Fan, Manveer Singh Tamber, Suleman Kazi, Vivek Sourabh, Mike Qi, Ruixuan Tu, Chenyu Xu, Matthew Gonzales, Ofer Mendelevitch, Amin Ahmad

Text Generation

Challenge:	Existing evaluations of hallucinations in large language models suffer from a lack of diversity and recency in the LLM and LLM families considered.
Approach:	They propose a summarization hallucination benchmark that challenges models to disagree on hallucines . they use models to generate answers or summaries from textual input .
Outcome:	The proposed model combines the best of 10 modern LLMs with ground truth annotations.

GATEAU: Selecting Influential Samples for Long Context Alignment (2025.emnlp-main)

Copied to clipboard

Shuzheng Si, Haozhe Zhao, Gang Chen, Yunshui Li, Kangyang Luo, Chuancheng Lv, Kaikai An, Fanchao Qi, Baobao Chang, Maosong Sun

Challenge:	Existing studies have attempted to scale up the available data volume by synthesizing long instruction-following samples, but a lack of a well-defined strategy for ensuring data quality may introduce low-quality samples and restrict the model’s performance.
Approach:	They propose a framework to identify influential samples enriched with long-range dependency relations that can be used to align large language models to handle instructions with extremely long contexts.
Outcome:	The proposed framework identifies samples with long-range dependency relations and shows that the model trained on these samples exhibits better instruction-following and long-context understanding capabilities.

PerSphere: A Comprehensive Framework for Multi-Faceted Perspective Retrieval and Summarization (2025.acl-long)

Copied to clipboard

Yun Luo, Yingjie Li, Xiangkun Hu, Qinglin Qi, Fang Guo, Qipeng Guo, Zheng Zhang, Yue Zhang

Information Extraction & Text Mining

Challenge:	Experimental results show that the main challenge lies in long context and perspective extraction.
Approach:	They propose a benchmark to facilitate multi-faceted perspective retrieval and summarization . they propose measurable metrics to evaluate the comprehensiveness of the retrieval pipeline .
Outcome:	The proposed system breaks free from information silos by combining two opposing claims . it can be used to extract multiple perspectives and improve performance on the platform .

AlphaQT-Bench: Diagnosing the Gap between Financial Code Generation and Quantitative Reasoning in LLMs (2026.findings-acl)

Copied to clipboard

Sichun Luo, Yi Huang, Shichang Meng, Fengyuan Liu, Mukai Li, Qinghua Yao, Zefa Hu, Junlan Feng, Qi Liu

Programming Languages in NLP

Challenge:	Existing benchmarks rely on outcome-driven metrics such as profitability and look-ahead bias.
Approach:	They propose a diagnostic benchmark for instruction-grounded financial code generation under strict semantic and temporal constraints.
Outcome:	The proposed benchmarks show that the models fail under causal, structural, or functional constraints.

Reasoning with Graphs: Structuring Implicit Knowledge to Enhance LLMs Reasoning (2025.findings-acl)

Copied to clipboard

Haoyu Han, Yaochen Xie, Hui Liu, Xianfeng Tang, Sreyashi Nag, William Headden, Yang Li, Chen Luo, Shuiwang Ji, Qi He, Jiliang Tang