Papers by Qipeng Guo

Identifying Semantic Induction Heads to Understand In-Context Learning (2024.findings-acl)

Challenge:	Large language models have demonstrated impressive performance across a wide range of tasks, but this achievement comes with the trade-off of significant computational demands.
Approach:	They propose a scaling law that decomposes the overall validation loss and assigns different importance weights to tokens to assess a specific meta-capability.
Outcome:	The proposed model can predict the loss trending of models across different levels of computation without a gap between validation loss and model's downstream capabilities.

Copied to clipboard

Jie Ren, Qipeng Guo, Hang Yan, Dongrui Liu, Quanshi Zhang, Xipeng Qiu, Dahua Lin

DORE: Document Ordered Relation Extraction based on Generative Framework (2022.findings-emnlp)

Challenge:	Large language models (LLMs) have demonstrated remarkable performance, but lack of transparency in their inference logic raises concerns about their trustworthiness.
Approach:	They conduct a detailed analysis of the operations of attention heads to understand their in-context learning of LLMs.
Outcome:	The proposed analysis of attention heads reveals that they increase the output logits of object tokens and recall objects . the proposed model is a novel approach to understand the in-context learning of large language models.

Copied to clipboard

Qipeng Guo, Yuqing Yang, Hang Yan, Xipeng Qiu, Zheng Zhang

An AMR-based Link Prediction Approach for Document-level Event Argument Extraction (2023.acl-long)

Challenge:	Existing generative methods do not fit document-level relation extraction tasks where there are multiple entities and relational facts.
Approach:	They propose to generate a symbolic and ordered sequence from relation matrix which is easier to learn and introduce several negative sampling strategies to improve the performance with balanced signals.
Outcome:	The proposed method can improve the performance of the generative DocRE models on four datasets.

Copied to clipboard

Yuqing Yang, Qipeng Guo, Xiangkun Hu, Yue Zhang, Xipeng Qiu, Zheng Zhang

Representation Learning

Challenge:	Recent work has introduced Abstract Meaning Representation (AMR) for Document-level Event Argument Extraction (Doc-level EAE) however, in these works AMR is used only implicitly, for instance, as additional features or training signals.
Approach:	They propose a novel AMR-based graph structure which uses graph neural networks to find event arguments from unstructured text.
Outcome:	The proposed graph structure outperforms the state-of-the-art models by 3.63pt and 2.33pt F1 and reduces inference time by 56%.

Synergetic Event Understanding: A Collaborative Approach to Cross-Document Event Coreference Resolution with Large Language Models (2024.acl-long)

Copied to clipboard

Qingkai Min, Qipeng Guo, Xiangkun Hu, Songfang Huang, Zheng Zhang, Yue Zhang

SciPedia: Unlocking the Value of Scientific Data for Pre-training (2026.acl-long)

Challenge:	Existing approaches to cross-document event coreference resolution are prone to learning simple co-occurrences due to the complexity of contexts.
Approach:	They propose a collaborative approach to cross-document event coreference resolution that leverages both a universally capable LLM and a task-specific SLM.
Outcome:	The proposed approach surpasses the performance of both large and small language models individually, underscoring its effectiveness in diverse scenarios.

Copied to clipboard

Yiwei Qin, Zhen Huang, Tiantian Mi, Weiye Si, Qipeng Guo, Siyuan Feng, Pengfei Liu

A Unified Generative Framework for Various NER Subtasks (2021.acl-long)

Challenge:	High-quality scientific data is critical for advancing LLMs, yet academic literature remains underutilized.
Approach:	They construct a large-scale raw scientific corpus but identify a critical Learnability Gap . they develop a multi-stage pipeline featuring content cleaning and pedagogical augmentation .
Outcome:	The proposed approach boosts average performance by +2.12 (3B) and +2.95 (7B) on in-domain tasks.

Copied to clipboard

Hang Yan, Tao Gui, Junqi Dai, Qipeng Guo, Zheng Zhang, Xipeng Qiu

StoryAnalogy: Deriving Story-level Analogies from Large Language Models to Unlock Analogical Understanding (2023.emnlp-main)

Challenge:	Named Entity Recognition (NER) is the task of identifying spans that represent entities in sentences.
Approach:	They propose to formulate NER subtasks as entity span sequence generation task . framework can be used to solve all three kinds of NER tasks without tagging schema .
Outcome:	The proposed framework achieves state-of-the-art (SoTA) or near SoTA performance on eight English NER datasets.

Copied to clipboard

Cheng Jiayang, Lin Qiu, Tsz Chan, Tianqing Fang, Weiqi Wang, Chunkit Chan, Dongyu Ru, Qipeng Guo, Hongming Zhang, Yangqiu Song, Yue Zhang, Zheng Zhang

Explicit Memory Learning with Expectation Maximization (2024.emnlp-main)

Challenge:	Analogy-making between narratives is crucial for human reasoning . despite its importance, there has been limited research on story analogies .
Approach:	They construct a large-scale story-level analogy corpus with 24K story pairs . they find that the tasks are incredibly difficult for large language models such as ChatGPT .
Outcome:	The proposed corpus contains 24K story pairs from diverse domains with human annotations on two similarities from the extended Structure-Mapping Theory.

Copied to clipboard

Zhangyue Yin, Qiushi Sun, Qipeng Guo, Zhiyuan Zeng, Qinyuan Cheng, Xipeng Qiu, Xuanjing Huang

Challenge:	Large Language Models lack reliable learning mechanisms for updating information across interactions.
Approach:	They propose a framework that enhances explicit memory updates via the Expectation-Maximization algorithm.
Outcome:	The proposed framework outperforms existing methods without memory or with static external memory on streaming inference tasks.

RLET: A Reinforcement Learning Based Approach for Explainable QA with Entailment Trees (2022.emnlp-main)

Copied to clipboard

Tengxiao Liu, Qipeng Guo, Xiangkun Hu, Yue Zhang, Xipeng Qiu, Zheng Zhang

Enhancing Uncertainty-Based Hallucination Detection with Stronger Focus (2023.emnlp-main)

Challenge:	Existing structured reasoning frameworks lack internal decision probability and cannot model the tree as a whole.
Approach:	They propose a Reinforcement Learning based Entailment Tree generation framework that is trained using the cumulative signals across the whole tree.
Outcome:	The proposed framework offers explicit deductions with entailment steps in a tree structure.

Copied to clipboard

Tianhang Zhang, Lin Qiu, Qipeng Guo, Cheng Deng, Yue Zhang, Zheng Zhang, Chenghu Zhou, Xinbing Wang, Luoyi Fu

Code Needs Comments: Enhancing Code LLMs with Comment Augmentation (2024.findings-acl)

Challenge:	Existing methods for detecting hallucinations in LLMs rely on external knowledge for reference retrieval or require sampling multiple responses for consistency verification.
Approach:	They propose a reference-free, uncertainty-based method for detecting hallucinations in Large Language Models that imitates human focus in factuality checking from three aspects: focus on the most informative keywords; focus on unreliable tokens in historical context; focus of token properties such as token type and token frequency.
Outcome:	The proposed method achieves state-of-the-art performance across all evaluation metrics and eliminates the need for additional information.

Copied to clipboard

Demin Song, Honglin Guo, Yunhua Zhou, Shuhao Xing, Yudong Wang, Zifan Song, Wenwei Zhang, Qipeng Guo, Hang Yan, Xipeng Qiu, Dahua Lin

Programming Languages in NLP

Challenge:	Large Language Models (LLMs) require a deep understanding of programming languages and their correlation with natural languages (NLs).
Approach:	They propose a data augmentation method that generates comments for existing code and a filtering strategy that filters out code data poorly correlated with natural language.
Outcome:	The proposed method outperforms the model trained on the augmented data and the model further trained on data without augmentation on two widely-used programming skill benchmarks.

Turn Waste into Worth: Rectifying Top-k Router of MoE (2024.emnlp-main)

Copied to clipboard

Zhiyuan Zeng, Qipeng Guo, Zhaoye Fei, Zhangyue Yin, Yunhua Zhou, Linyang Li, Tianxiang Sun, Hang Yan, Dahua Lin, Xipeng Qiu

Green & Sustainable NLP

Challenge:	Top-k router suffers from redundancy computation and memory costs due to unbalanced routing . some experts are overflow, where exceeding tokens are dropped, while others are empty, which are padded with zeros, negatively impacting model performance.
Approach:	They propose a top-k router that is unbalanced and uses a multi-gPU system to handle dropped tokens and padding.
Outcome:	The proposed model surpasses the top-1 router by 4.7% in terms of performance . the top-k router suffers from redundancy computation and memory costs .

Do Large Language Models Know What They Don’t Know? (2023.findings-acl)

Copied to clipboard

Zhangyue Yin, Qiushi Sun, Qipeng Guo, Jiawen Wu, Xipeng Qiu, Xuanjing Huang

UnitCoder: Scalable Code Synthesis from Pre-training Corpora (2025.emnlp-main)

Challenge:	Large language models (LLMs) have vast knowledge that allows them to excel in various NLP tasks.
Approach:	They propose an automated method to detect uncertainty in the responses of large language models and a dataset to measure their self-knowledge.
Outcome:	The proposed method detects uncertainty in the responses of large language models and provides a novel measure of their self-knowledge.

Copied to clipboard

Yichuan Ma, Yunfan Shao, Peiji Li, Demin Song, Qipeng Guo, Linyang Li, Xipeng Qiu, Kai Chen

Firewall Routing: Blocking Leads to Better Hybrid Inference for LLMs (2025.emnlp-main)

Challenge:	Large Language Models (LLMs) excel at code understanding and generation, yet code generation remains a challenge.
Approach:	They propose a model that supervises pre-training data quality through automatically generated unit tests while ensuring correctness via an iterative fix and refine flow.
Outcome:	The proposed model improves performance on a large dataset with high quality pre-training data.

Copied to clipboard

Runyu Peng, Yunhua Zhou, Kai Lv, Yang Gao, Qipeng Guo, Xipeng Qiu

Challenge:	Large language models have significantly enhanced performance across various NLP tasks . high computational costs and latency associated with deploying such models pose bottlenecks .
Approach:	They propose a dynamic hybrid inference framework that efficiently selects between a strong and a weak LLM based on the complexity of the query.
Outcome:	The proposed method outperforms existing routing strategies by up to 5.29% in APGR . large models often introduce higher latency, making them less suitable for real-time or resource-constrained applications.

LongWanjuan: Towards Systematic Measurement for Long Text Quality (2024.findings-emnlp)

Copied to clipboard

Xiaoran Liu, Kai Lv, Qipeng Guo, Hang Yan, Conghui He, Xipeng Qiu, Dahua Lin

How to Set the Learning Rate for Large-Scale Pre-training? (2026.findings-acl)

Challenge:	Existing efforts to improve data quality have focused on deduplication and the evaluation of data diversity and difficulty.
Approach:	They propose a set of metrics to evaluate the quality of long texts by evaluating three fundamental linguistic dimensions: coherence, cohesion, and complexity.
Outcome:	The proposed model improves on long-text tasks with over 160B tokens and categorizes long texts into holistic, aggregated, and chaotic types.

Copied to clipboard

Yunhua Zhou, Shuhao Xing, Junhao Huang, Xipeng Qiu, Qipeng Guo

Timely Machine: Awareness of Time Makes Test-Time Scaling Agentic (2026.acl-long)

Challenge:	Optimal configuration of the learning rate (LR) is a fundamental yet formidable challenge in large-scale pre-training.
Approach:	They propose a Fitting Paradigm and a Transfer Paradigme to investigate fit and transfer . they propose scalability and elucidate the reasons why module-wise parameter tuning underperforms .
Outcome:	The proposed model reduces the search complexity by reducing the search cost by lowering the search factor.

Copied to clipboard

Yichuan Ma, Linyang Li, Yongkang Chen, Peiji Li, Xiaozhe Li, Qipeng Guo, Dahua Lin, Kai Chen

Reasoning in Flux: Enhancing Large Language Models Reasoning through Uncertainty-aware Adaptive Guidance (2024.acl-long)

Challenge:	Large language models (LLMs) tackle complex reasoning tasks, but test-time scaling is becoming expensive.
Approach:	They propose to redefine test-time as wall-clock time, where models dynamically adjust strategies based on time budgets.
Outcome:	The proposed model improves time budget awareness and boosts performance across Timely-Eval.

Copied to clipboard

Zhangyue Yin, Qiushi Sun, Qipeng Guo, Zhiyuan Zeng, Xiaonan Li, Junqi Dai, Qinyuan Cheng, Xuanjing Huang, Xipeng Qiu

GenWiki: A Dataset of 1.3 Million Content-Sharing Text and Graphs for Unsupervised Graph-to-Text Generation (2020.coling-main)

Challenge:	Extensive experiments across various reasoning tasks demonstrate that UAG not only enhances the reasoning abilities of LLMs but consistently outperforms several strong baselines with minimal computational overhead.
Approach:	They propose an approach to guide LLMs onto an accurate and reliable trajectory by identifying and adjusting uncertainty signals within each step of the reasoning chain.
Outcome:	The proposed approach outperforms strong baselines and outperformed strong models with minimal computational overhead.

Copied to clipboard

Zhijing Jin, Qipeng Guo, Xipeng Qiu, Zheng Zhang

Structured Data in NLP

Challenge:	a large-scale, general-domain dataset is needed for knowledge graph-to-text generation . data collection is expensive and data-intensive, making it difficult to get good annotation .
Approach:	They propose to use a large-scale, general-domain dataset to generate unsupervised text from knowledge graphs.
Outcome:	The proposed dataset has 1.3M text and graph examples, and is a benchmark for future research . good annotation is expensive and difficult to get, and it's difficult to check quality .

Exploiting Abstract Meaning Representation for Open-Domain Question Answering (2023.findings-acl)

Copied to clipboard

Cunxiang Wang, Zhikun Xu, Qipeng Guo, Xiangkun Hu, Xuefeng Bai, Zheng Zhang, Yue Zhang

Full Parameter Fine-tuning for Large Language Models with Limited Resources (2024.acl-long)

Challenge:	Existing work attempts to address these challenges using Pretrained Language Models (PLMs) but the diversity of surface form expressions can hinder the model’s ability to capture accurate correlations, especially when the context is lengthy and complex.
Approach:	They propose a method known as Graph-as-Token (GST) to incorporate AMRs into PLMs to assist the model in understanding complex semantic information.
Outcome:	The proposed method outperforms existing methods and significantly improves performance on both Natural Questions and TriviaQA.

Copied to clipboard

Kai Lv, Yuqing Yang, Tengxiao Liu, Qipeng Guo, Xipeng Qiu

SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation (2026.findings-acl)

Challenge:	Large Language Models (LLMs) require massive GPU resources for training.
Approach:	They propose a parameter-efficient optimization that fuses the gradient computation and parameter update in one step to reduce memory usage.
Outcome:	The proposed method reduces memory usage to 10.8% compared to the standard approach.

Copied to clipboard

Shuang Cheng, Yihan Bian, Dawei Liu, Yuhua Jiang, Yihao Liu, Linfeng Zhang, Qian Yao, Zhongbo Tian, Wenhai Wang, Qipeng Guo, Kai Chen, Biqing Qi, Bowen Zhou

Rethinking Multiple-Choice Questions for RLVR: Unlocking Potential via Distractor Design (2026.findings-acl)

Challenge:	Autoregressive (AR) language models are a dominant paradigm in the field of parallelism and non-causal modeling.
Approach:	They propose a blockwise discrete diffusion model that preserves AR-compatible serving while enabling parallel intra-block generation.
Outcome:	The proposed model achieves theoretical speedups over 5 and wall-clock speedup of 2.3 on H200 GPUs in latency-critical regimes.

Copied to clipboard

Xu Guo, Qiming Ge, Jian Tong, Kedi Chen, Jin Zhang, Xiaogui Yang, Xuan Gao, Haijun Lv, Zhihui Lu, Yicheng Zou, Qipeng Guo

Exchange-of-Thought: Enhancing Large Language Model Capabilities through Cross-Model Communication (2023.emnlp-main)

Challenge:	Existing approaches to RLVR use multiple-choice questions as verifiable rewards . however, not all tasks provide reliable verification .
Approach:	They propose a framework that actively constructs high-quality distractors to block elimination shortcuts and promote deep reasoning.
Outcome:	The proposed method significantly improves reasoning capabilities of Large Language Models.

Copied to clipboard

Zhangyue Yin, Qiushi Sun, Cheng Chang, Qipeng Guo, Junqi Dai, Xuanjing Huang, Xipeng Qiu

GeometryZero: Advancing Geometry Solving via Group Contrastive Policy Optimization (2026.findings-acl)

Challenge:	Large Language Models (LLMs) have made significant strides in complex reasoning tasks, but their reasoning is often constrained by their intrinsic understanding, lacking external insights.
Approach:	They propose a framework that enables cross-model communication during problem-solving.
Outcome:	The proposed framework surpasses established baselines in complex reasoning tasks and is cost-effective.

Copied to clipboard

Yikun Wang, Yibin Wang, Dianyi Wang, Zimian Peng, Qipeng Guo, Dacheng Tao, Jiaqi Wang

Memorize Step by Step: Efficient Long-Context Prefilling with Incremental Memory and Decremental Chunk (2024.emnlp-main)

Challenge:	Existing methods for auxiliary construction training are expensive and underperform . Existing Corresponding Author training methods lack self-correction capabilities in reasoning chains.
Approach:	They propose a reinforcement learning framework that rewards auxiliary construction with geometric reasoning by grouping construction rewards with a Length Reward.
Outcome:	Experiments on Geometry3K and MathVista show that GeometryZero outperforms baselines on auxiliary constructions.

Copied to clipboard

Zhiyuan Zeng, Qipeng Guo, Xiaoran Liu, Zhangyue Yin, Wentao Shu, Mianqiu Huang, Bo Wang, Yunhua Zhou, Linlin Li, Qun Liu, Xipeng Qiu

Green & Sustainable NLP

Challenge:	Existing methods to optimize LLM for long sequences for long documents are slow and consume memory.
Approach:	They propose a method that starts with a small memory size and gradually increases it . they propose Decremental Chunk based on Incremental Memory (IMDC) which reduces chunk size while increasing memory size .
Outcome:	The proposed method is faster (1.45x) and reduces GPU memory consumption by 23.3% compared to fixed-size memory.

What are the Essential Factors in Crafting Effective Long Context Multi-Hop Instruction Datasets? Insights and Best Practices (2025.acl-long)

Copied to clipboard

Zhi Chen, Qiguang Chen, Libo Qin, Qipeng Guo, Haijun Lv, Yicheng Zou, Hang Yan, Kai Chen, Dahua Lin

Challenge:	Existing methods to generate long-context instruction-tuning data are limited by poor quality and fewer than 35% of samples are multi-hop .
Approach:	They propose a framework that integrates a quality verification agent, a single-hop question generation agent, and a multi-hop questions merger agent to enhance model performance.
Outcome:	The proposed framework significantly improves data quality with high-quality, multi-hop, and diverse data.

Knowledge-Centric Hallucination Detection (2024.emnlp-main)

Copied to clipboard

Xiangkun Hu, Dongyu Ru, Lin Qiu, Qipeng Guo, Tianhang Zhang, Yang Xu, Yun Luo, Pengfei Liu, Yue Zhang, Zheng Zhang

VisuoThink: Empowering LVLM Reasoning with Multimodal Tree Search (2025.acl-long)

Challenge:	Large Language Models (LLMs) have shown impressive capabilities but a tendency to hallucinate.
Approach:	They propose a framework that introduces claim-triplets to represent claims in LLM responses and evaluates them against a reference.
Outcome:	The proposed framework outperforms prior methods by 18.2 to 27.2 points on a benchmark spanning various NLP tasks and annotated 11k claim-triplets from 2.1k responses by seven LLMs.

Copied to clipboard

Yikun Wang, Siyin Wang, Qinyuan Cheng, Zhaoye Fei, Liang Ding, Qipeng Guo, Dacheng Tao, Xipeng Qiu

Multimodality

Challenge:	Existing approaches to large vision-language models fail to capture interleaved nature of human visual-verbal reasoning processes.
Approach:	They propose a framework that integrates visuospatial and linguistic domains to facilitate multimodal slow thinking by enabling progressive visual-textual reasoning.
Outcome:	Experiments show that VisuoThink significantly improves reasoning capabilities even without fine-tuning.

AdaLomo: Low-memory Optimization with Adaptive Learning Rate (2024.findings-acl)

Copied to clipboard

Kai Lv, Hang Yan, Qipeng Guo, Haijun Lv, Xipeng Qiu

ARISE: An Adaptive Resolution-Aware Metric for Test-Time Scaling Evaluation in Large Reasoning Models (2026.findings-acl)

Challenge:	Large language models require substantial memory for training, thereby setting a high hardware threshold.
Approach:	They propose a low-memory optimization technique that reduces memory footprint . they propose an adaptive learning rate for each parameter and a grouped update normalization to stabilize convergence .
Outcome:	The proposed low-memory optimization performs better than the prevailing algorithm for large language models, AdamW.

Copied to clipboard

Zhangyue Yin, Qiushi Sun, Zhiyuan Zeng, Zhiyuan Yu, Qipeng Guo, Xuanjing Huang, Xipeng Qiu

Which Reasoning Trajectories Teach Students to Reason Better? A Simple Metric of Informative Alignment (2026.acl-long)

Challenge:	Existing evaluation methods for test-time scaling are limited.
Approach:	They propose an adaptive resolution-aware scaling evaluation metric specifically designed to assess the test-time scaling effectiveness of large reasoning models.
Outcome:	The proposed metric provides a reliable and fine-grained measurement of test-time scaling capabilities, revealing significant variations in scaling efficiency across models.

Copied to clipboard

Yuming Yang, Mingyoung Lai, Wanxu Zhao, Xiaoran Fan, Zhiheng Xi, Mingqi Wu, Chiyue Huang, Jun Zhao, Haijun Lv, Jian Tong, Yunhua Zhou, Yicheng Zou, Qipeng Guo, Tao Gui, Qi Zhang, Xuanjing Huang

F-Eval: Asssessing Fundamental Abilities with Refined Evaluation Methods (2024.acl-long)

Challenge:	Existing methods assess suitability primarily through student likelihood, favoring trajectories that align closely with the student model’s current behavior but overlooking more informative ones.
Approach:	They propose a Rank–Surprisal Ratio metric that captures both alignment and informativeness to assess the suitability of a reasoning trajectory.
Outcome:	The proposed metric captures both alignment and informativeness to assess the suitability of a reasoning trajectory.

Copied to clipboard

Yu Sun, Keyuchen Keyuchen, Shujie Wang, Peiji Li, Qipeng Guo, Hang Yan, Xipeng Qiu, Xuanjing Huang, Dahua Lin

Challenge:	Large language models (LLMs) have been evaluated for their instruction-following capabilities but lack references to their fundamental abilities.
Approach:	They propose a bilingual evaluation benchmark to evaluate the fundamental abilities of large language models including expression, commonsense and logic.
Outcome:	The proposed evaluation methods show higher correlation coefficients and larger distinction than other evaluators.

BERT-ATTACK: Adversarial Attack Against BERT Using BERT (2020.emnlp-main)

Copied to clipboard

Linyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue, Xipeng Qiu

Robustness in NLP

Challenge:	Current approaches to generate adversarial samples for discrete data are heuristic replacement strategies that are difficult to implement in continuous data.
Approach:	They propose a method to generate adversarial samples using pre-trained masked language models using BERT.
Outcome:	The proposed method outperforms state-of-the-art methods in success rate and perturb percentage while remaining fluent and semantically preserved.

Dual Cache for Long Document Neural Coreference Resolution (2023.acl-long)

Copied to clipboard

Qipeng Guo, Xiangkun Hu, Yue Zhang, Xipeng Qiu, Zheng Zhang

Powering Verifiable Learning via Automated Evolutionary Data Synthesis (2026.acl-long)

Challenge:	Recent studies show the effectiveness of cache-based neural coreference resolution models on long documents.
Approach:	They propose a hybrid cache that integrates two eviction policies to capture global and local entities separately and improves F1 score of coreference by 0.7 5.7pt.
Outcome:	The proposed model outperforms existing models on four benchmarks while saving up to 83% of inference time.

Copied to clipboard

He Du, Bowen Li, Aijun Yang, Siyang He, Qipeng Guo, Kai Chen, Dacheng Tao

Dialogue Meaning Representation for Task-Oriented Dialogue Systems (2022.findings-emnlp)

Challenge:	Existing approaches to building generalizable verifiable data are task-specific and lack a principled, universal evaluator of verifikatability.
Approach:	They propose a task-agnostic, strategy-guided, executably-checkable data synthesis framework that synthesizes problems, diverse candidate solutions and verification artifacts from a single source.
Outcome:	The proposed framework synthesizes problems, candidates, and verification artifacts from human-annotated and strategy-induced checks and iteratively discovers strategies.

Copied to clipboard

Xiangkun Hu, Junqi Dai, Hang Yan, Yi Zhang, Qipeng Guo, Xipeng Qiu, Zheng Zhang

Natural Language Interfaces

Challenge:	Existing work on dialogue meaning representations is limited in scalability for complex expressions.
Approach:	They propose a pliable and easily extendable representation for task-oriented dialogue . they propose an inheritance hierarchy mechanism focusing on domain extensibility .
Outcome:	The proposed representation can be easily extended to a task-oriented dialogue dataset.

Benchmarking Hallucination in Large Language Models Based on Unanswerable Math Word Problem (2024.lrec-main)

Copied to clipboard

YuHong Sun, Zhangyue Yin, Qipeng Guo, Jiawen Wu, Xipeng Qiu, Hui Zhao

CoLAKE: Contextualized Language and Knowledge Embedding (2020.coling-main)

Challenge:	Large language models (LLMs) are highly effective in various natural language processing tasks, but can produce unreliable conjectures in ambiguous contexts, which is known as hallucination.
Approach:	They propose a method to evaluate LLM hallucination in Question Answering based on the unanswerable math word problem (UMWP) . they combine text similarity and mathematical expression detection to determine whether LLM considers the question unanswered.
Outcome:	The proposed method combines text similarity and mathematical expression detection to determine whether the LLM considers the question unanswerable.

Copied to clipboard

Tianxiang Sun, Yunfan Shao, Xipeng Qiu, Qipeng Guo, Yaru Hu, Xuanjing Huang, Zheng Zhang

Qipeng Guo, Xipeng Qiu, Pengfei Liu, Yunfan Shao, Xiangyang Xue, Zheng Zhang

Challenge:	Existing models for integrating factual knowledge into pre-trained language models are shallow, static, and separately pre-train entities.
Approach:	They propose a method which integrates knowledge contexts from large-scale knowledge bases into a unified data structure.
Outcome:	The proposed model outperforms existing models on knowledge-driven tasks and knowledge probing tasks.

Star-Transformer (N19-1)

Copied to clipboard

CoLLiE: Collaborative Training of Large Language Models in an Efficient Way (2023.emnlp-demo)

Challenge:	Existing models with fully-connected attention connections are heavy and require large training data.
Approach:	They propose a lightweight alternative to the Transformer by sparsifying the fully-connected structure with a star-shaped topology.
Outcome:	The proposed model achieves significant performance improvements on 22 datasets on four tasks.

Copied to clipboard

Kai Lv, Shuo Zhang, Tianle Gu, Shuhao Xing, Jiawei Hong, Keyu Chen, Xiaoran Liu, Yuqing Yang, Honglin Guo, Tengxiao Liu, Yu Sun, Qipeng Guo, Hang Yan, Xipeng Qiu

A Survey of Inductive Reasoning for Large Language Models (2026.acl-long)

Challenge:	Large language models (LLMs) are increasingly pivotal in a wide range of tasks . however, the resources required for training these models necessitate efficient solutions .
Approach:	They propose a library that facilitates collaborative training of large language models . they use 3D parallelism, parameter-efficient fine-tuning methods and optimizers .
Outcome:	The proposed library has proven superior training efficiency in comparison with prevalent solutions in pre-training and fine-tuning scenarios.

Copied to clipboard

Kedi Chen, Dezhao Ruan, Yuhao Dan, Yaoting Wang, Siyu Yan, Xuecheng Wu, Yinqi Zhang, Qin Chen, Jie Zhou, Liang He, Biqing Qi, Linyang Li, Qipeng Guo, Xiaoming Shi, Wei Zhang

Towards Economical Inference: Enabling DeepSeek’s Multi-Head Latent Attention in Any Transformer-based LLMs (2025.acl-long)

Challenge:	Inductive reasoning is an important task for large language models (LLMs).
Approach:	They propose a survey of inductive reasoning for large language models . they categorize methods into three main areas: post-training enhancement, test-time exploration, and data augmentation.
Outcome:	The proposed method improves inductive reasoning in large language models.

Copied to clipboard

Tao Ji, Bin Guo, Yuanbin Wu, Qipeng Guo, Shenlixing Shenlixing, Chenzhan Chenzhan, Xipeng Qiu, Qi Zhang, Tao Gui

Plan, Verify and Switch: Integrated Reasoning with Diverse X-of-Thoughts (2023.emnlp-main)

Challenge:	Multi-head Latent Attention (MLA) is an innovative architecture designed to ensure efficient and economical inference by significantly compressing the Key-Value (KV) cache into a latent vector.
Approach:	They propose a data-efficient fine-tuning method for transitioning from MHA to MLA using a latent vector cache.
Outcome:	The proposed architecture reduces the KV cache size of Llama2-7B by 92.19%, with only 1% drop in LongBench performance.

Copied to clipboard

Tengxiao Liu, Qipeng Guo, Yuqing Yang, Xiangkun Hu, Yue Zhang, Xipeng Qiu, Zheng Zhang

CritiQ: Mining Data Quality Criteria from Human Preferences (2025.acl-long)

Challenge:	Existing methods for large language models (LLMs) have been used to prompt different reasoning thoughts, such as Chain of Thought and Program of Though.
Approach:	They propose a framework that prompts large language models with diverse reasoning thoughts by iterating between different prompting methods.
Outcome:	The proposed framework is able to generate multiple reasoning thoughts in 10 popular math reasoning datasets and is orthogonal to recent work that makes improvements on single reasoning methods and can generalise to logical reasoning domain.

Copied to clipboard

Honglin Guo, Kai Lv, Qipeng Guo, Tianyi Liang, Zhiheng Xi, Demin Song, Qiuyinzhe Zhang, Yu Sun, Kai Chen, Xipeng Qiu, Tao Gui

Information Retrieval

Challenge:	Existing methods to train language models rely on manual design, perplexity, or careful prompt engineering.
Approach:	They propose a method that automatically mines criteria from human preferences for data quality with only 30 human-annotated pairs and performs efficient data selection.
Outcome:	The proposed method improves on human-annotated test sets and shows high accuracy on code, math, and logic domains.

Multi-Document Event Extraction Using Large and Small Language Models (2025.emnlp-main)

Copied to clipboard

Qingkai Min, Zitian Qu, Qipeng Guo, Xiangkun Hu, Zheng Zhang, Yue Zhang

PerSphere: A Comprehensive Framework for Multi-Faceted Perspective Retrieval and Summarization (2025.acl-long)

Challenge:	Existing approaches to multi-document event extraction have limited attention . despite its practical significance, this task has inherent challenges .
Approach:	They propose a collaborative framework that integrates large language models for multi-step reasoning and fine-tuned small language models to handle key subtasks.
Outcome:	The proposed framework outperforms existing methods and provides new insights into collaborative reasoning to tackle the complexities of multi-document event extraction.

Copied to clipboard

Yun Luo, Yingjie Li, Xiangkun Hu, Qinglin Qi, Fang Guo, Qipeng Guo, Zheng Zhang, Yue Zhang

CodeEvo: Interaction-Driven Synthesis of Code-centric Data through Hybrid and Iterative Feedback (2026.acl-long)

Challenge:	Experimental results show that the main challenge lies in long context and perspective extraction.
Approach:	They propose a benchmark to facilitate multi-faceted perspective retrieval and summarization . they propose measurable metrics to evaluate the comprehensiveness of the retrieval pipeline .
Outcome:	The proposed system breaks free from information silos by combining two opposing claims . it can be used to extract multiple perspectives and improve performance on the platform .

Copied to clipboard

Qiushi Sun, Jingyang Gong, Lei Li, Qipeng Guo, Fei Yuan

Challenge:	Existing methods for generating instruction-code pairs rely on rigid heuristics and are labor-intensive.
Approach:	They propose a dual-agent architecture that integrates a Coder and a Reviewer to orchestrate the generation trajectory.
Outcome:	The proposed architecture outperforms baselines on a large-scale dataset of instruction-code pairs with stepped difficulty levels.

Aggregation of Reasoning: A Hierarchical Framework for Enhancing Answer Selection in Large Language Models (2024.lrec-main)

Copied to clipboard

Zhangyue Yin, Qiushi Sun, Qipeng Guo, Zhiyuan Zeng, Xiaonan Li, Tianxiang Sun, Cheng Chang, Qinyuan Cheng, Ding Wang, Xiaofeng Mou, Xipeng Qiu, Xuanjing Huang