Papers by Qi Yi

43 papers
Disentangled Multi-span Evolutionary Network against Temporal Knowledge Graph Reasoning (2025.findings-acl)

Copied to clipboard

Challenge: Existing methods for temporal knowledge Graphs neglect internal structural interactions between subgraphs and ignore potential smooth features that do not lead to semantic changes.
Approach: They propose to use a disentangled multi-span evolutionary network to capture local neighbor features while perceiving historical neighbor semantic information.
Outcome: Extensive experiments show that the proposed model outperforms the state-of-the-art in TKG reasoning by 22.7%.
AT²PO: Agentic Turn-based Policy Optimization via Tree Search (2026.acl-long)

Copied to clipboard

Challenge: Recent advances in large language models (LLMs) have catalyzed the development of autonomous agents capable of executing complex, multi-turn tasks.
Approach: They propose a framework for agentic reinforcement learning that integrates turn-level tree search with tree search to address key challenges.
Outcome: The proposed framework addresses key challenges: limited exploration diversity, sparse credit assignment, and misaligned policy optimization.
A Skeleton-Based Model for Promoting Coherence Among Sentences in Narrative Story Generation (D18-1)

Copied to clipboard

Challenge: Existing models for narrative story generation lack semantic dependency among sentences.
Approach: They propose a skeleton-based model that generates the most critical phrases and expands them to a complete sentence.
Outcome: The proposed model can generate significantly more coherent stories according to human evaluation and automatic evaluation.
TextMixer: Mixing Multiple Inputs for Privacy-Preserving Inference (2023.findings-emnlp)

Copied to clipboard

Challenge: Pre-trained language models (PLMs) are often deployed as cloud services, enabling users to upload textual data and perform inference remotely.
Approach: They propose a privacy-preserving inference framework called MixPi which aims to obfuscate a user's private input by mixing it with multiple other inputs.
Outcome: The proposed framework surpasses existing privacy-preserving methods on token and sentence classification tasks.
Towards Building a Robust Toxicity Predictor (2023.acl-industry)

Copied to clipboard

Challenge: Recent studies have focused on robustness of toxicity language predictors, but this is problematic for real-world toxicity detection.
Approach: They propose a novel adversarial attack that exploits greedy search strategies to fool toxic text classifiers.
Outcome: The proposed attack can detect weaker toxicity language detectors even against unseen attacks.
From Superficial to Deep: Integrating External Knowledge for Follow-up Question Generation Using Knowledge Graph and LLM (2025.coling-main)

Copied to clipboard

Challenge: Existing methods for generating follow-up questions are limited to shallow contextual questions that are uninspiring and have a large gap to the human level.
Approach: They propose a three-stage external knowledge-enhanced follow-up question generation method which generates questions by identifying contextual topics, building a knowledge graph online, and finally combining these with a large language model to generate the final question.
Outcome: The proposed method generates questions by identifying contextual topics, building a knowledge graph (KG) online, and finally combining these with a large language model to generate the final question.
LongTutor: Benchmarking Large Language Models for Long-term Personalized Tutoring (2026.acl-long)

Copied to clipboard

Challenge: Existing evaluations focus on isolated, short-term interactions, overlooking the inherently long-term nature of learning.
Approach: They propose a benchmark for long-term personalized tutoring based on an annotated learning log . they propose an automated generator–verifier pipeline to enable benchmark expansion .
Outcome: The proposed benchmarks evaluate LLMs across three progressive tasks: evidence acquisition, knowledge state diagnosis, and adaptive teaching action.
Tiny-NewsRec: Effective and Efficient PLM-based News Recommendation (2022.emnlp-main)

Copied to clipboard

Challenge: Existing work fine tunes the PLM with the news recommendation task, which can cause a domain shift problem.
Approach: They propose a self-supervised method to adapt general PLM to news domain with a contrastive matching task between news titles and news bodies.
Outcome: The proposed method can improve both the effectiveness and efficiency of the large PLM-based news recommendation model while maintaining its performance.
PFDial: A Structured Dialogue Instruction Fine-tuning Method Based on UML Flowcharts (2025.findings-acl)

Copied to clipboard

Challenge: Large Language Models (LLMs) have shown remarkable progress in dialogue and reasoning, but they struggle to solve strictly constrained dialogue tasks.
Approach: They construct a dataset that contains 12,705 high-quality Chinese dialogue instructions from 440 flowcharts containing 5,055 process nodes.
Outcome: The proposed model outperforms GPT-4o models on backward transitions and outperformed GPT-42 models on the same dataset.
SRF: Enhancing Document-Level Relation Extraction with a Novel Secondary Reasoning Framework (2024.emnlp-main)

Copied to clipboard

Challenge: Existing methods for document-level relation extraction ignore bidirectional mention interaction when generating relational features for entity pairs.
Approach: They propose a document-level relation extraction model that incorporates bidirectional mention fusion and a simple yet effective evidence extraction module for relation prediction.
Outcome: The proposed model achieves SOTA performance and the proposed method is effective and general when integrated into existing models.
Reading Order Matters: Information Extraction from Visually-rich Documents by Token Path Prediction (2023.emnlp-main)

Copied to clipboard

Challenge: Recent advances in multimodal pre-trained models have significantly improved information extraction from visually-rich documents (VrDs).
Approach: They propose a method to predict token sequences within visually-rich documents by a simple prediction head.
Outcome: The proposed method can be used to predict token mentions as token sequences within documents.
PLAWBENCH: A Rubric-Based Benchmark for Evaluating LLMs in Real-World Legal Practice (2026.acl-long)

Copied to clipboard

Challenge: Existing benchmarks for large language models (LLMs) are coarse, single-dimensional metrics and do not explicitly assess fine-grained legal reasoning.
Approach: They propose a Practical Law Benchmark to evaluate large language models in real-world legal practice scenarios.
Outcome: The proposed model is based on 850 questions and 13 scenarios with expert-designed evaluation rubrics.
LongHeads: Multi-Head Attention is Secretly a Long Context Processor (2024.findings-emnlp)

Copied to clipboard

Challenge: Large language models struggle to process lengthy inputs due to limited length generalization and attention’s quadratic computational demands.
Approach: They propose a training-free framework that allows each head to attend to important context chunks instead of allowing each head a full sentence .
Outcome: The proposed framework unlocks multi-head attention's untapped potential by allowing each head to attend to important context chunks instead of the full sentence.
LONGAGENT: Achieving Question Answering for 128k-Token-Long Documents through Multi-Agent Collaboration (2024.emnlp-main)

Copied to clipboard

Challenge: Large language models (LLMs) have been successful in understanding language and processing text, but their cost prohibits their practical applications.
Approach: They propose a multi-agent collaboration method that breaks down lengthy documents into smaller, more manageable chunks and organizes the member agents to read their assigned chunks.
Outcome: The proposed method achieves 16.42% and 1.63% accuracy gains over existing models on single-hop and multi-hop QA settings.
Open Domain Question Answering with Conflicting Contexts (2025.findings-naacl)

Copied to clipboard

Challenge: Open domain question answering systems often rely on information retrieved from large collections of text to answer questions.
Approach: They evaluate and benchmark three powerful Large Language Models with a dataset . they find that 25% of unambiguous open domain questions can lead to conflicting contexts .
Outcome: The proposed model can't be used to answer questions with conflicting contexts . it can be fine tuned to provide richer information into the model's training .
Reinforcement Learning on Pre-Training Data (2026.acl-long)

Copied to clipboard

Challenge: Recent progress in large language models is driven by scaling of training compute through pre-training with nexttoken prediction (NTP) or post-training (RL) Pre-training using NTP enables models to acquire extensive knowledge and skills from general data, but it suffers from data inefficiency and catastrophic forgetting in continual learning settings.
Approach: They propose to scale training compute through pre-training with next-token prediction (NTP) or post-training by scaling reinforcement learning (RL) to improve learning from general data.
Outcome: Experiments on multiple benchmarks and models show that the proposed approach improves continual pre-training and provides a strong foundation for post-training on Qwen3-8B-Base.
A Global Past-Future Early Exit Method for Accelerating Inference of Pre-trained Language Models (2021.naacl-main)

Copied to clipboard

Challenge: Existing methods to accelerate inference speed of pre-trained language models are limited to local representations of exit layer . current models are associated with large memory requirement and high computational cost, which slow down inference and further encumber the application of PLMs.
Approach: They propose a method to exit early without passing through all inference layers . they take into consideration all the linguistic information embedded in the past layers a global perspective .
Outcome: The proposed method outperforms existing methods by a large margin . it uses linguistic information embedded in the past layers and future features . the proposed method is scalable and cost-effective .
Beyond the Tip of Efficiency: Uncovering the Submerged Threats of Jailbreak Attacks in Small Language Models (2025.findings-acl)

Copied to clipboard

Challenge: Small language models (SLMs) have become increasingly prominent in the deployment on edge devices due to their high efficiency and low computational cost.
Approach: They evaluate the security performance of 13 state-of-the-art small language models under various jailbreak attacks.
Outcome: The proposed methods demonstrate that SLMs are quite susceptible to jailbreak attacks and some are even vulnerable to harmful prompts.
Low-probability Tokens Sustain Exploration in Reinforcement Learning with Verifiable Reward (2026.findings-acl)

Copied to clipboard

Challenge: Recent studies show that RLVR training is slow and results plateau as policy entropy collapses . low-probability regularization (Lp-Reg) reduces the number of low-quality exploratory tokens induced by RL training .
Approach: They propose a method to reduce RLVR over-penalization by eliminating low-probability exploratory tokens . they propose 'Low-provability Regularization' to reduce the gradual elimination of low-quality exploratory entropy tokens.
Outcome: The proposed method eliminates low-probability exploratory tokens and prevents suppression of potentially valuable low-property candidates.
Unsupervised Graph-Text Mutual Conversion with a Unified Pretrained Language Model (2023.acl-long)

Copied to clipboard

Challenge: Existing unsupervised approaches for learning knowledge graphs require multiple modules and require entity information or relation type for training.
Approach: They propose a method that uses a unified pretrained language model to achieve fully unsupervised graph-text mutual conversion for the first time.
Outcome: The proposed method outperforms state-of-the-art methods for G2T and T2G tasks by fine-tuning only one pretrained model.
Eval-RAR: Evaluation-Driven Retrieval-Augmented Reasoning via Reinforcement Learning (2026.findings-acl)

Copied to clipboard

Challenge: Existing methods for retrieval-augmented generation fail to provide explicit supervision for internal reasoning process.
Approach: They propose an Evaluation-driven Retrieval-Augmented Reasoning framework that uses reinforcement learning and a fine-grained evaluation reward to optimize the process.
Outcome: Eval-RAR outperforms existing methods on QA benchmarks on seven single-hop and multi-hop tasks.
AutoSchemaKG: Autonomous Knowledge Graph Construction through Dynamic Schema Induction from Web-Scale Corpora (2026.acl-long)

Copied to clipboard

Challenge: Existing knowledge graph construction frameworks require predefined schemas, limiting their scalability and domain coverage.
Approach: They propose a framework for fully autonomous knowledge graph construction that eliminates the need for predefined schemas.
Outcome: The proposed framework outperforms state-of-the-art models on multi-hop QA tasks and enhances LLM factuality.
DAC: A Dynamic Attention-aware Approach for Task-Agnostic Prompt Compression (2025.acl-long)

Copied to clipboard

Challenge: Existing methods rely on information entropy as the metric to compress lexical units, but ignore attention-critical tokens and information . recent advent of In-Context Learning (ICL), Chain-of-Thought (CoT), and Retrieval Augmented Generation (RAG) technologies has significantly invigorated the landscape of applications based on Large Language Models (LLMs).
Approach: They propose a dynamic attention-aware approach to task-agnostic prompt compression . they integrate entropy and attention information to achieve fine-grained prompt compression.
Outcome: Experiments show that the proposed approach improves across tasks and LLMs.
QuoteR: A Benchmark of Quote Recommendation for Writing (2022.acl-long)

Copied to clipboard

Challenge: Existing methods to recommend quotes are evaluated on unpublished datasets .
Approach: They propose to build a dataset that is open and contains three parts including English, standard Chinese and classical Chinese.
Outcome: The proposed model outperforms existing methods on all three parts of QuoteR.
Pretrain-KGE: Learning Knowledge Representation from Pretrained Language Models (2020.findings-emnlp)

Copied to clipboard

Challenge: Existing knowledge graph embedding models suffer from limited knowledge representation due to sparse and noisy dataset annotations.
Approach: They propose to use pretrained language models to enhance knowledge representation by leveraging world knowledge from pretrained models.
Outcome: Extensive experiments show that the proposed framework can improve results over existing models.
Unveiling the Deficiencies of Pre-trained Text-and-Layout Models in Real-world Visually-rich Document Information Extraction (2026.findings-eacl)

Copied to clipboard

Challenge: PTLMs have shown remarkable success in multiple information extraction tasks . however, their performance in real-world scenarios falls short of expectations .
Approach: They propose to use an entity-centric dataset to evaluate PTLMs' performance . they find that inadequate annotations in benchmark datasets lead to spurious correlations .
Outcome: The proposed dataset disentangles the falsely-coupled segment and entity annotations that arises from the block-level annotation of FUNSD.
Modeling Layout Reading Order as Ordering Relations for Visually-rich Document Understanding (2024.emnlp-main)

Copied to clipboard

Challenge: Existing models of layout reading order do not convey the complete reading order information in the layout.
Approach: They propose to model layout reading order as ordering relations over layout elements . they propose a reading-order-relation-enhancing pipeline to improve model performance .
Outcome: The proposed model outperforms existing models on a visual-rich document dataset and on eight cross-domain VrD-IE/QA tasks without targeted optimization.
CoTKR: Chain-of-Thought Enhanced Knowledge Rewriting for Complex Knowledge Graph Question Answering (2024.emnlp-main)

Copied to clipboard

Challenge: Existing knowledge rewriting methods may include irrelevant information, omit crucial details, or fail to align with the question’s semantics.
Approach: They propose a new rewriting method CoTKR for generating reasoning traces and corresponding knowledge in an interleaved manner, thereby mitigating the limitations of single-step knowledge rewrite.
Outcome: The proposed method mitigates the limitations of single-step knowledge rewriting and bridges the preference gap between the knowledge reactor and the question answering (QA) model.
Fin-STAR: Structure-as-Semantics to Resolve Implicitness in Financial Retrieval (2026.findings-acl)

Copied to clipboard

Challenge: Existing Retrieval-Augmented Generation systems treat structure as a physical navigational skeleton rather than intrinsic semantic knowledge.
Approach: They propose a framework that redefining hierarchy as intrinsic semantics and uses snippets to enrich hierarchical lineage.
Outcome: The proposed framework outperforms state-of-the-art hierarchical and graph-based benchmarks on FinTierQA Gold.
E-ViC: Reasoning Beyond Text via Embodied Visual Chain for Spatial Intelligence (2026.acl-long)

Copied to clipboard

Challenge: Existing Vision-Language Models (VLMs) lack spatial reasoning, despite text-based CoTs . e-ViC reframes spatial intelligence as a verifiable, tool-using capability, argues a new study.
Approach: They propose a framework that moves reasoning beyond text into the visual domain . they ground reasoning in pixel-level interactions to enable human-like "look-and-confirm" strategies .
Outcome: The proposed framework outperforms existing Vision-Language Models with an average gain of 10.1%.
Cognitive Alpha Mining via LLM-Driven Code-Based Evolution (2026.acl-long)

Copied to clipboard

Challenge: Existing approaches to finding effective predictive signals from financial data are limited by their complexity and low signal-to-noise ratio.
Approach: They propose a framework that combines code-level alpha representation with LLM-driven reasoning and evolutionary search.
Outcome: The proposed framework combines code-level alpha representation with LLM-driven reasoning and evolutionary search.
Measuring Human Contribution in AI-Assisted Content Generation (2026.acl-long)

Copied to clipboard

Challenge: generative AI has created a new way to generate content with humans . varying degrees of human contribution in content generation poses significant challenges for the delineation of originality .
Approach: They propose a framework to measure human contribution in AI-assisted content generation by calculating mutual information between human input and AI-aided output relative to self-information of AI-assist output.
Outcome: The proposed measure discriminates between varying degrees of human contribution across multiple creative domains and is validated in real-world applications.
Self-Demos: Eliciting Out-of-Demonstration Generalizability in Large Language Models (2024.findings-naacl)

Copied to clipboard

Challenge: Existing methods that rely on limited demos and out-of-demonstration (OOD) queries fail when faced with out- of-demotion queries.
Approach: They propose a query-aware prompting method that elicits the inherent generalizability of large language models by query-based demo generation.
Outcome: The proposed method outperforms state-of-the-art methods in the OOD setting and two public math benchmarks.
Making Harmful Behaviors Unlearnable for Large Language Models (2024.findings-acl)

Copied to clipboard

Challenge: Large language models (LLMs) are often customized by fine-tuning for the requirements of different domains.
Approach: They propose a controllable training framework to make undesired behaviors unlearnable during the fine-tuning process.
Outcome: The proposed framework makes undesired behaviors unlearnable during the fine-tuning process while preserving the ability to learn other information.
Towards Better Entity Linking with Multi-View Enhanced Distillation (2023.acl-long)

Copied to clipboard

Challenge: Entity linking is a fundamental task in Natural Language Processing (NLP), connecting mentions within unstructured contexts to their corresponding entities in a Knowledge Base (KB).
Approach: They propose a dual-encoder framework that can efficiently match mentions to two-encoding frameworks by a global-view.
Outcome: The proposed framework achieves state-of-the-art on several entity linking benchmarks.
MicroEdit: Neuron-level Knowledge Disentanglement and Localization in Lifelong Model Editing (2025.emnlp-main)

Copied to clipboard

Challenge: Existing methods for enhancing large language models are designed for single or limited edits, lacking the capacity to support long-term, multi-round knowledge updates.
Approach: They propose a neuron-level editing method that performs minimal interventions within large language models (LLMs) by leveraging a sparse autoencoder, MicroEdit disentangles knowledge representations and activates only a minimal set of necessary neurons for precise parameter updates.
Outcome: Extensive experiments show that MicroEdit outperforms prior methods and robustly handles lifelong knowledge editing across QA and Hallucination settings on LLaM and Mistral.
Role Prompting Guided Domain Adaptation with General Capability Preserve for Large Language Models (2024.findings-naacl)

Copied to clipboard

Challenge: Large Language Models (LLMs) suffer catastrophic forgetting when tailored to specific domains . authors present a novel approach to manage multi-domain LLM adaptation .
Approach: They propose a strategy to manage multi-domain LLM adaptation using self-distillation and role integration.
Outcome: The proposed model alleviates catastrophic forgetting and inter-domain confusion while maintaining robust general capabilities.
AlphaQT-Bench: Diagnosing the Gap between Financial Code Generation and Quantitative Reasoning in LLMs (2026.findings-acl)

Copied to clipboard

Challenge: Existing benchmarks rely on outcome-driven metrics such as profitability and look-ahead bias.
Approach: They propose a diagnostic benchmark for instruction-grounded financial code generation under strict semantic and temporal constraints.
Outcome: The proposed benchmarks show that the models fail under causal, structural, or functional constraints.
Mitigating Object Hallucinations in MLLMs via Multi-Frequency Perturbations (2025.findings-emnlp)

Copied to clipboard

Challenge: Multimodal large language models have demonstrated remarkable performance in visual-language tasks, but their authenticity is often compromised by object hallucinations.
Approach: They propose a multi-frequency perturbation method that leverages both low-frequency and high-frequency features of images to perturb visual feature representations and explicitly suppress redundant frequency-domain features during inference.
Outcome: The proposed method significantly mitigates object hallucinations across various model architectures.
Beyond Scaling: Measuring and Predicting the Upper Bound of Knowledge Retention in Language Model Pre-Training (2026.acl-long)

Copied to clipboard

Challenge: Existing methods to predict performance of large language models are lacking . authors propose a size-dependent mutual information predictor for closed-book question answering accuracy .
Approach: They propose a size-dependent mutual information predictor that integrates knowledge frequency, knowledge specificity, and model size to forecast closed-book question answering accuracy.
Outcome: The proposed method outperforms baseline models and achieves R2 > 0.7 in predicting QA accuracy without additional training.
DialogVED: A Pre-trained Latent Variable Encoder-Decoder Model for Dialog Response Generation (2022.acl-long)

Copied to clipboard

Challenge: Existing pre-trained dialog models shed light on various downstream tasks in natural language processing (NLP).
Approach: They propose a dialog pre-training framework that introduces latent variables into the enhanced encoder-decoder pre-train framework to increase relevance and diversity of responses.
Outcome: The proposed model achieves state-of-the-art on personaChat, DailyDialog, and DSTC7-AVSD datasets.
TextObfuscator: Making Pre-trained Language Model a Privacy Protector via Obfuscating Word Representations (2023.findings-acl)

Copied to clipboard

Challenge: Existing inference services are plagued by privacy concerns, such as sharing sensitive data with service providers.
Approach: They propose a framework for protecting inference privacy by applying random perturbations to clustered representations.
Outcome: The proposed framework protects inference privacy by applying random perturbations to clustered representations.
Rhombus: Incentivizing Coordination in Parallel Thinking through Reinforcement Learning (2026.findings-acl)

Copied to clipboard

Challenge: Parallel thinking is a promising avenue for scaling test-time compute in Large Language Models . however, coordinating the exploration and aggregation stages remains challenging .
Approach: They propose a parallel thinking framework that explicitly incentivizes coordination between components via end-to-end reinforcement learning.
Outcome: The proposed framework improves accuracy by 6.0% over long chain-of-thought baselines while reducing wall-clock latency by 39.4% under matched token budgets.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations